Instructions to use TIE-Pilot/qwen3-4b-grpo-dapo-ckpts with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TIE-Pilot/qwen3-4b-grpo-dapo-ckpts with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TIE-Pilot/qwen3-4b-grpo-dapo-ckpts")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("TIE-Pilot/qwen3-4b-grpo-dapo-ckpts", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TIE-Pilot/qwen3-4b-grpo-dapo-ckpts with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/TIE-Pilot/qwen3-4b-grpo-dapo-ckpts

SGLang

How to use TIE-Pilot/qwen3-4b-grpo-dapo-ckpts with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use TIE-Pilot/qwen3-4b-grpo-dapo-ckpts with Docker Model Runner:
```
docker model run hf.co/TIE-Pilot/qwen3-4b-grpo-dapo-ckpts
```

Qwen3-4B GRPO/DAPO-Math — dense checkpoint family (25 steps)

This repository is not a single model. It is the complete checkpoint family of one full-parameter GRPO run on Qwen/Qwen3-4B: the policy exported every 10 optimizer steps, from step 10 to step 250, as 25 self-contained Hugging Face model directories.

The point of publishing all 25 is the trajectory, not the endpoint. Work on speculative decoding, drafter drift, reward hacking, and RL training dynamics generally needs to compare a policy against itself at nearby points in training, and that is exactly what is normally thrown away when only the final checkpoint is released.

Layout

step_10/    step_20/    step_30/   ...   step_250/

Each step_N/ is a standard HF model directory (config.json, model-0000*-of-00002.safetensors, model.safetensors.index.json, tokenizer files) and loads directly:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts",
    subfolder="step_250",
    dtype="bfloat16",
)
tok = AutoTokenizer.from_pretrained(
    "TIE-Pilot/qwen3-4b-grpo-dapo-ckpts", subfolder="step_250"
)

Or fetch a single step without pulling the whole 188 GB:

hf download TIE-Pilot/qwen3-4b-grpo-dapo-ckpts --include "step_250/*" --local-dir ./ckpt

Training setup

Trained with verl, full-parameter FSDP, vLLM rollouts.


Base model	`Qwen/Qwen3-4B`
Algorithm	GRPO (`adv_estimator=grpo`)
Dataset	DAPO-Math
KL	`use_kl_in_reward=False`, `use_kl_loss=True`, `kl_loss_coef=0.001`, `low_var_kl`
Learning rate	1e-6
Train batch / mini-batch	64 / 32 (16 / 8 per GPU × 4 GPUs)
Rollouts per prompt	8, temperature 0.9
Max prompt / response	1024 / 4096 tokens
Epochs	8
Save frequency	every 10 steps
Precision	bfloat16 weights

Checkpoints were reconstructed from verl's sharded FSDP output with verl.model_merger merge --backend fsdp.

Caveats

No per-step evaluation is included. These are raw policy exports; reward curves and benchmark scores were not published alongside them.
Step 250 is the final checkpoint of the run, not a checkpoint selected for best validation performance.
This is a research byproduct from a speculative-decoding project. It is a standard GRPO run and is not intended to be a strong math model — its value is the density of the trajectory.

7B sibling family: TIE-Pilot/qwen25-7b-base-grpo-dapo-ckpts

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for TIE-Pilot/qwen3-4b-grpo-dapo-ckpts

Base model

Qwen/Qwen3-4B-Base

Finetuned

Qwen/Qwen3-4B

Finetuned

(1049)

this model

TIE-Pilot
/

qwen3-4b-grpo-dapo-ckpts

Qwen3-4B GRPO/DAPO-Math — dense checkpoint family (25 steps)

Layout

Training setup

Caveats

Related

Model tree for TIE-Pilot/qwen3-4b-grpo-dapo-ckpts