Robotics
LeRobot
Safetensors
imitation-learning
x7s
smolvla
vision-language-action
offline-evaluation
# Launch finetuning on your dataset
python lerobot/scripts/train.py \
--policy.path=Stevenshuqing/LW-Bench-SmolVLA \
--dataset.repo_id=lerobot/svla_so101_pickplace \
--batch_size=64 \
--steps=20000 \
--output_dir=outputs/train/my_smolvla \
--job_name=my_smolvla_training \
--policy.device=cuda \
--wandb.enable=true# Run the policy using the record function
python -m lerobot.record \
--robot.type=so101_follower \
--robot.port=/dev/ttyACM0 \ # <- Use your port
--robot.id=my_blue_follower_arm \ # <- Use your robot id
--robot.cameras="{ front: {type: opencv, index_or_path: 8, width: 640, height: 480, fps: 30}}" \ # <- Use your cameras
--dataset.single_task="Grasp a lego block and put it in the bin." \ # <- Use the same task description you used in your dataset recording
--dataset.repo_id=HF_USER/dataset_name \ # <- This will be the dataset name on HF Hub
--dataset.episode_time_s=50 \
--dataset.num_episodes=10 \
--policy.path=Stevenshuqing/LW-Bench-SmolVLALW-Bench SmolVLA 5K for X7s
LeRobot SmolVLA policy fine-tuned from lerobot/smolvla_base on the audited X7s strict compositional split in LW-Compositional-Bench.
Training contract
- Dataset:
LightwheelAI/Lightwheel-Tasks-X7S - Train split: 91 clean tasks; tasks 7, 119, and 149 held out
- Inputs: three RGB cameras, 25D robot state, and canonical task language
- Output: native 21D X7s absolute action
- Hardware: 4 GPUs
- Seed: 1000
- Effective global batch: 64
- Checkpoint: 5,000 global optimizer updates
- Evaluation horizon: first 16 valid action steps
Offline 5K results
| Metric | Value |
|---|---|
| H16 MAE | 0.117617 |
| H16 RMSE | 0.227890 |
| Gripper balanced accuracy | 87.08% |
| Boundary-proxy MAE | 0.147732 |
| Interior MAE | 0.111848 |
| Boundary / interior MAE | 1.321x |
These results cover three strict held-out tasks and are descriptive. Offline action error does not replace closed-loop success evaluation.
Load with LeRobot
from lerobot.policies.smolvla.modeling_smolvla import SmolVLAPolicy
policy = SmolVLAPolicy.from_pretrained("Stevenshuqing/LW-Bench-SmolVLA")
policy.eval()
Use LeRobot 0.4.3 and the included preprocessor/postprocessor files. The repository contains the complete pretrained_model directory produced by LeRobot.
Provenance
- Base model: https://huggingface.co/lerobot/smolvla_base
- Code and protocol: https://github.com/stevenqing/LW-Compositional-Bench
- Full report: https://github.com/stevenqing/LW-Compositional-Bench/blob/main/artifacts/model-eval/x7s/rotation-v0/budget-5k/report.md
- Code revision:
2d0e26dd2fe2e91046f416bd65bc369f8568d7f0 model.safetensorsSHA256:f70959474aad109134f799e49dd313abf209380c8e1c808d8650e6d92287e242
Limitations
This checkpoint is specific to the X7s observation/action schema. The benchmark is offline because the available A100 GPUs cannot provide Isaac Sim RT camera rendering. Transition boundaries are action-space proxies based on gripper sign changes and robust continuous-action peaks, not human semantic annotations.
- Downloads last month
- 11
Model tree for Stevenshuqing/LW-Bench-SmolVLA
Base model
lerobot/smolvla_base
# See https://github.com/huggingface/lerobot?tab=readme-ov-file#installation for more details git clone https://github.com/huggingface/lerobot.git cd lerobot pip install -e .[smolvla]