Instructions to use bigcode/starcoderplus-megatron with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use bigcode/starcoderplus-megatron with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="bigcode/starcoderplus-megatron")# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("bigcode/starcoderplus-megatron", dtype="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use bigcode/starcoderplus-megatron with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bigcode/starcoderplus-megatron" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigcode/starcoderplus-megatron", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/bigcode/starcoderplus-megatron
- SGLang
How to use bigcode/starcoderplus-megatron with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "bigcode/starcoderplus-megatron" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigcode/starcoderplus-megatron", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "bigcode/starcoderplus-megatron" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bigcode/starcoderplus-megatron", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use bigcode/starcoderplus-megatron with Docker Model Runner:
docker model run hf.co/bigcode/starcoderplus-megatron
add iter_400
Browse files- mp_rank_00_000/model_optim_rng.pt +3 -0
- mp_rank_00_001/model_optim_rng.pt +3 -0
- mp_rank_00_002/model_optim_rng.pt +3 -0
- mp_rank_00_003/model_optim_rng.pt +3 -0
- mp_rank_01_000/model_optim_rng.pt +3 -0
- mp_rank_01_001/model_optim_rng.pt +3 -0
- mp_rank_01_002/model_optim_rng.pt +3 -0
- mp_rank_01_003/model_optim_rng.pt +3 -0
- mp_rank_02_000/model_optim_rng.pt +3 -0
- mp_rank_02_001/model_optim_rng.pt +3 -0
- mp_rank_02_002/model_optim_rng.pt +3 -0
- mp_rank_02_003/model_optim_rng.pt +3 -0
- mp_rank_03_000/model_optim_rng.pt +3 -0
- mp_rank_03_001/model_optim_rng.pt +3 -0
- mp_rank_03_002/model_optim_rng.pt +3 -0
- mp_rank_03_003/model_optim_rng.pt +3 -0
mp_rank_00_000/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f48eee82c37463571eef33bc582bb0555b9c96bb7f264ca480f10e872c3fb688
|
| 3 |
+
size 15200402387
|
mp_rank_00_001/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7fa89d99e4c15a5cb96efbc7335f60d4129aa00ae7595619372dd2af13162831
|
| 3 |
+
size 13438791707
|
mp_rank_00_002/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:37db08f3a9f7dca2aff22baa621b41de093fb396cedf6dd849e2011ed3873acd
|
| 3 |
+
size 13438791771
|
mp_rank_00_003/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7111d36dd63548347aa8ec231a31a2a3e69b4f42f31c75be91c116e2a433d0f9
|
| 3 |
+
size 14495932379
|
mp_rank_01_000/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:87ad3be82528ee348ddfa41773883f9cce57a5026c0eebcee2239666a36c49b2
|
| 3 |
+
size 15200402387
|
mp_rank_01_001/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9b36509c9d082f0c86886b13e8ad43ab7a3e0fcab5d2a2beac70788200efbfae
|
| 3 |
+
size 13438791707
|
mp_rank_01_002/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5f4d6c9bbf5456abac0410ea097108c223d087aba4322f827086dfa0128923c1
|
| 3 |
+
size 13438791771
|
mp_rank_01_003/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:18a1a355a8c3a116a16066015c1b8786355185f28cdf5aab40ac7ae96a242df6
|
| 3 |
+
size 14495932379
|
mp_rank_02_000/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b5a7ef3f4063ef90654fbe3cece778b857e0e1ff1b22762acf15730c33ef7def
|
| 3 |
+
size 15200402387
|
mp_rank_02_001/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ab4adfdf3c0f79bda8a6f4a37a759f436ccab06263729a0b06018b478935cbb
|
| 3 |
+
size 13438791707
|
mp_rank_02_002/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7f47ff14e8da8d0e938c87b0ed828e343f6113b18fef5661d1aba8a7de5ee5ed
|
| 3 |
+
size 13438791771
|
mp_rank_02_003/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:0586896c729425a961f816bba29f80fd60f3a8ce2250d462c73783e16a19918f
|
| 3 |
+
size 14495932379
|
mp_rank_03_000/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3f5cd8b4fec4db6211e495286dc55416cabb2c470a920174bba3466ff558648c
|
| 3 |
+
size 15200402387
|
mp_rank_03_001/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9e397f3d6548446dc387084a70aa9cd148eb1a2ce8c5badf3a001edc70ab96d3
|
| 3 |
+
size 13438791707
|
mp_rank_03_002/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2fea66ee21c03d745546259ea8cb1baa4c8cbd24d4a8b1084e9cd87515cb7a7f
|
| 3 |
+
size 13438791771
|
mp_rank_03_003/model_optim_rng.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7058b1e6e836417700911e75f31d145150ab8d3486f2870abd1603fa4677abb0
|
| 3 |
+
size 14495932379
|