nebchi commited on
Commit
cbcd3f7
ยท
verified ยท
1 Parent(s): d83e639

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -96,16 +96,20 @@ datasets:
96
 
97
  ---
98
 
99
- ## Running with the `pipeline` API
100
 
101
  You can initialize the model and processor for inference with `pipeline` as follows.
102
 
 
 
 
 
103
  ```python
104
  from openai import OpenAI
105
 
106
  client = OpenAI(
107
  base_url="model-endpoint",
108
- api_key="api-key" # vllm์€ API key๊ฐ€ ํ•„์š” ์—†์ง€๋งŒ ํ•„๋“œ๋Š” ํ•„์ˆ˜
109
  )
110
 
111
  SYSTEM_PROMPT = """๋‹น์‹ ์€ ์ŠคํŒธ ๋ฌธ์ž๋กœ ํŒ์ •ํ•œ ๊ทผ๊ฑฐ๋ฅผ ์ƒ์„ฑํ•˜๋Š” ๋Œ€ํ˜• ์–ธ์–ด ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค.
@@ -141,18 +145,14 @@ response = client.chat.completions.create(
141
  )
142
  print(response.choices[0].message.content)
143
 
144
- ๐Ÿง  Example Output
 
 
145
  - ํŒ์ • ๊ทผ๊ฑฐ: ๋‚ด์šฉ์˜ ๋ชฉ์ 
146
  - ์ŠคํŒธ ํŒ์ • ์ด์œ : ์œ„ ์ŠคํŒธ ๋ฌธ์ž๋Š” ๊ธˆ์œต ์ƒํ’ˆ๊ณผ ๋Œ€์ถœ ๊ด€๋ จ ๊ถŒ์œ  ๋‚ด์šฉ์„ ํฌํ•จํ•˜๊ณ  ์žˆ์œผ๋ฉฐ,
147
  โ€˜์ง€๊ธˆ ๋ฐ”๋กœโ€™, โ€˜์ฆ‰์‹œ ์‹ ์ฒญโ€™๊ณผ ๊ฐ™์€ ์‹ฌ๋ฆฌ์  ์••๋ฐ• ์–ด๊ตฌ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ˆ˜์‹ ์ž์˜ ํ–‰๋™์„ ์œ ๋„ํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
148
  ```
149
 
150
- ## Running with vLLM
151
-
152
- ```sh
153
- vllm serve Devocean-06/Spam_Filter-gemma
154
- ```
155
-
156
  ---
157
 
158
  ## Software
 
96
 
97
  ---
98
 
99
+ ## Running with the `vllm` API
100
 
101
  You can initialize the model and processor for inference with `pipeline` as follows.
102
 
103
+ ```sh
104
+ vllm serve Devocean-06/Spam_Filter-gemma
105
+ ```
106
+
107
  ```python
108
  from openai import OpenAI
109
 
110
  client = OpenAI(
111
  base_url="model-endpoint",
112
+ api_key="api-key"
113
  )
114
 
115
  SYSTEM_PROMPT = """๋‹น์‹ ์€ ์ŠคํŒธ ๋ฌธ์ž๋กœ ํŒ์ •ํ•œ ๊ทผ๊ฑฐ๋ฅผ ์ƒ์„ฑํ•˜๋Š” ๋Œ€ํ˜• ์–ธ์–ด ๋ชจ๋ธ์ž…๋‹ˆ๋‹ค.
 
145
  )
146
  print(response.choices[0].message.content)
147
 
148
+ ```
149
+ ## ๐Ÿง  Example Output
150
+ ```sh
151
  - ํŒ์ • ๊ทผ๊ฑฐ: ๋‚ด์šฉ์˜ ๋ชฉ์ 
152
  - ์ŠคํŒธ ํŒ์ • ์ด์œ : ์œ„ ์ŠคํŒธ ๋ฌธ์ž๋Š” ๊ธˆ์œต ์ƒํ’ˆ๊ณผ ๋Œ€์ถœ ๊ด€๋ จ ๊ถŒ์œ  ๋‚ด์šฉ์„ ํฌํ•จํ•˜๊ณ  ์žˆ์œผ๋ฉฐ,
153
  โ€˜์ง€๊ธˆ ๋ฐ”๋กœโ€™, โ€˜์ฆ‰์‹œ ์‹ ์ฒญโ€™๊ณผ ๊ฐ™์€ ์‹ฌ๋ฆฌ์  ์••๋ฐ• ์–ด๊ตฌ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ˆ˜์‹ ์ž์˜ ํ–‰๋™์„ ์œ ๋„ํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.
154
  ```
155
 
 
 
 
 
 
 
156
  ---
157
 
158
  ## Software