sergiopaniego/trackio-training-agents-4-bucket
7.23 MB
Sergio Paniego PRO
sergiopaniego
AI & ML interests
None yet
Recent Activity
posted an update about 3 hours ago
ICYMI, Async GRPO in TRL now supports LoRA and we wrote a looong blog testing it
> the adapter is a few megabytes, so the weight sync is a file instead of an NCCL transfer
> 3 HF Jobs: 1 trainer and 2 vLLM replicas
> the adapter travels through an HF Storage Bucket mounted in all 3 at the same path
> a proxy in front of the replicas routes each rollout to the one already holding its KV prefix
https://huggingface.co/blog/asyncgrpo-lora-hfjobs liked a Space about 6 hours ago
multimodalart/jev-decision-index updated a dataset about 11 hours ago
huggingface-projects/Deep-RL-Course-CertificationOrganizations
Posts 114
Post
26
ICYMI, Async GRPO in TRL now supports LoRA and we wrote a looong blog testing it
> the adapter is a few megabytes, so the weight sync is a file instead of an NCCL transfer
> 3 HF Jobs: 1 trainer and 2 vLLM replicas
> the adapter travels through an HF Storage Bucket mounted in all 3 at the same path
> a proxy in front of the replicas routes each rollout to the one already holding its KV prefix
https://huggingface.co/blog/asyncgrpo-lora-hfjobs
> the adapter is a few megabytes, so the weight sync is a file instead of an NCCL transfer
> 3 HF Jobs: 1 trainer and 2 vLLM replicas
> the adapter travels through an HF Storage Bucket mounted in all 3 at the same path
> a proxy in front of the replicas routes each rollout to the one already holding its KV prefix
https://huggingface.co/blog/asyncgrpo-lora-hfjobs
Articles 34
Article
10
One sandbox per rollout, or how labs run RL for agents in 2026
spaces 191
pinned
Running on Zero
Agents
Featured
123
VLM Object Understanding
🦀
Explore object detection, visual grounding, keypoint Detecti
pinned
Runtime error
Agents
11
Qwen2-VL-7B
🔥
Ask questions about charts in images
pinned
Running on Zero
Agents
SmolVLM-trl-dpo-rlaif-v
🔥
Generate text from an image and question
pinned
Runtime error
Agents
SmolVLM-trl-sft-ChartQA
🔥
Ask questions about charts in images
Running
Huggingface Static 25ced2
🎯
View and explore your data with an interactive dashboard
Running on CPU Upgrade
RL
Coding Environment Server
💻
Run Python code in an interactive coding environment
models 141
sergiopaniego/qwen3-1.7b-mbpp-grpo
Text Generation • 2B • Updated • 498
sergiopaniego/rl-envs-youtube-livestream-4-scripts
Updated
sergiopaniego/qwen3-1.7b-wordle-grpo
Text Generation • 2B • Updated • 149
sergiopaniego/Qwen3-4B-claude-code-local-grpo
Text Generation • 4B • Updated • 12
sergiopaniego/Qwen3-4B-claude-code-deepcoder-grpo
Text Generation • 4B • Updated • 11
sergiopaniego/pelican-svg-grpo-Qwen3-1.7B-judged
Text Generation • 2B • Updated • 11
sergiopaniego/pelican-svg-grpo-Qwen3-1.7B
Text Generation • 2B • Updated • 14
sergiopaniego/Qwen3-8B-opencode-deepcoder-grpo
Text Generation • 8B • Updated • 44 • 1
sergiopaniego/grpo-youtube-livestream-3-scripts
Reinforcement Learning • Updated • 2
sergiopaniego/Qwen3.5-4B-sdpo-math-hints
Updated • 1
datasets 16
sergiopaniego/opencode-rollout-trace
Updated • 345
sergiopaniego/pelican-svg-drawings
Viewer • Updated • 169 • 268 • 1
sergiopaniego/math-sdpo-hints-plain
Viewer • Updated • 600 • 25
sergiopaniego/math-sdpo-hints
Viewer • Updated • 600 • 34
sergiopaniego/gsm8k-sdpo-plain
Viewer • Updated • 700 • 14
sergiopaniego/gsm8k-sdpo-hints
Viewer • Updated • 700 • 34
sergiopaniego/pi-mono-chat
Viewer • Updated • 886 • 47
sergiopaniego/requests-pr-diff
Viewer • Updated • 1 • 165
sergiopaniego/trl-r2e-test
Viewer • Updated • 1 • 26
sergiopaniego/chain-sum-rollouts
Viewer • Updated • 50 • 13