Yixin Song
commited on
Update README.md
Browse files
README.md
CHANGED
|
@@ -21,7 +21,7 @@ Limitation: Due to SmallThinker's current limitations in instruction following,
|
|
| 21 |
SmallThinker is designed for the following use cases:
|
| 22 |
|
| 23 |
1. **Edge Deployment:** Its small size makes it ideal for deployment on resource-constrained devices.
|
| 24 |
-
2. **Draft Model for QwQ-32B-Preview:** SmallThinker can serve as a fast and efficient draft model for the larger QwQ-32B-Preview model.
|
| 25 |
|
| 26 |
## Limitations & Disclaimer
|
| 27 |
|
|
|
|
| 21 |
SmallThinker is designed for the following use cases:
|
| 22 |
|
| 23 |
1. **Edge Deployment:** Its small size makes it ideal for deployment on resource-constrained devices.
|
| 24 |
+
2. **Draft Model for QwQ-32B-Preview:** SmallThinker can serve as a fast and efficient draft model for the larger QwQ-32B-Preview model. From my test, in llama.cpp we can get 70% speedup (from 40 tokens/s to 70 tokens/s).
|
| 25 |
|
| 26 |
## Limitations & Disclaimer
|
| 27 |
|