XXXXyu/Cosmos3-Edge-Policy-DROID-Marlin-W4A16-AttnW8

Base model | Runtime | Benchmark details | Cosmos 3 report

A mixed W4/W8 Cosmos Lite deployment of NVIDIA's 4B Cosmos3 Edge DROID policy. On an RTX 4090 it uses 4.33 GB of post-load model allocation and serves a steady policy request in 571 ms p50 with guidance 3 / two denoise steps. It reaches 58% success over 50 paired RoboLab Banana rollouts.

At A Glance

Quant.	Denoise	Guidance	Peak VRAM	Request p50/p95	Banana SR
W4-AttnW8	2	3.0	8.74 GB	571/613 ms	58%

Three BananaInBowlTask episodes selected from the 50 paired rollout runs are shown above. This demo uses the three fastest successful episodes. Videos retain the original playback rate and full episode duration. Each final frame is held for three seconds; runs one and two fade to black before the next run, while the final run remains visible. All runs use the same guidance 3.0 / two-step sampler as the metrics. See the full quantization comparison and the five-model rollout view below.

Model Overview

This artifact is a weight-only derivative of nvidia/Cosmos3-Edge-Policy-DROID, an NVIDIA Cosmos3 Mixture-of-Transformers policy trained to generate DROID robot actions from language, RGB observations, and robot state. It retains attention projections at W8 and packs selected MLP projections to W4: 112 W4 and 224 W8 Linear modules. Activations and non-targeted weights remain BF16.

The self-contained bundle includes packed and residual weights, the native Edge processor and vision encoder, Wan VAE, portable runtime config, immutable source provenance, and per-file SHA256 hashes. The evaluated RoboLab contract returns one 32x8 action chunk, matching source checkpoint metadata.

Training And Evaluation Data

This derivative performs no additional policy training. Refer to the upstream model card for the base model's training description and data governance statements.

W4 packing uses activation statistics from 128 frames in 128 distinct successful episodes of the official nvidia/Cosmos3-DROID training split. These samples calibrate weight scales only; they do not update model parameters and are not specific to the Banana task. Closed-loop quality was evaluated on 50 paired RoboLab BananaInBowlTask rollouts.

Policy Interface

The validated RoboLab integration provides task text, robot state, and a 640x540 RGB composition made from three camera views. The runtime maps the image to the model's 736x544 inference bucket.

The released checkpoint and validated integration use a 32-step, 8-dimensional DROID action chunk. The chunk length is a property of this trained checkpoint and its deployment contract, not an immutable architectural limit of Cosmos 3. Changing it is outside the validated configuration.

This model is intended for robotics research, simulator evaluation, and controlled deployment development. Validate the observation/action contract, task success, and independent safety controls before using it on a real robot.

Benchmark Results

All rows use one RTX 4090, batch size one, guidance 3, 50 paired BananaInBowlTask episodes, and identical initial states. Latency is steady server policy-request latency and excludes RoboLab/IsaacSim and network roundtrip.

Quantization Comparison

This table fixes guidance to 3.0 and denoise steps to 2 for all variants.

Quantized model	Peak VRAM (GB)	Request p50 (ms)	Banana SR
W8A16	8.71	576	72%
W4A16	8.87	563	74%
W4A16-AttnW8	8.74	571	58%
W4A16-GenW8	8.79	570	80%

Two steps significantly outperformed four steps for BF16 (68% vs. 42%, exact McNemar p=0.0146) and improved several quantized variants. It is therefore the validated Edge comparison protocol, not only a latency optimization.

Sampling Comparison

Denoise	Request p50/p95	Success Rate	Wilson 95% CI
4	1,043/1,051 ms	50%	36.6-63.4%
2	571/613 ms	58%	44.2-70.6%

At two steps this strategy was not distinguishable from full W8 (9 paired wins / 16 losses, exact McNemar p=0.230) or from its own four-step sampler (14 wins / 10 losses, p=0.541).

See the Cosmos3 Edge benchmark for BF16 parity diagnostics, the full quantization matrix, and exact revisions.

Rollout Comparison

All five panels use guidance 3 / two denoise steps and three episodes selected with the same rule: the fastest three successes, except W4A16, which shows its fastest two successes and one failure. The displayed SR values come from all 50 paired rollouts, not only the selected videos.

Quick Start

git clone https://github.com/xxxxyu/cosmos-lite.git
cd cosmos-lite
CUDA_VISIBLE_DEVICES=0 examples/quantized_robot_policy/setup.sh

hf download "XXXXyu/Cosmos3-Edge-Policy-DROID-Marlin-W4A16-AttnW8" \
  --local-dir /data/cosmos3_edge_attnw8

BUNDLE_DIR=/data/cosmos3_edge_attnw8 STRATEGY=attention_w8 \
  examples/robolab_quant/pipeline.sh validate

BUNDLE_DIR=/data/cosmos3_edge_attnw8 POLICY_GPU=0 \
  GUIDANCE=3.0 NUM_STEPS=2 examples/robolab_quant/pipeline.sh serve

The model uses the Cosmos Lite bundle schema and vLLM Marlin WNA16 kernels. It is not a generic GPTQ/AWQ loader artifact.

Quantization And Bundle Details

Field	Value
Strategy	`attention_w8`
Packed linear modules	336
W4 / W8 modules	112 / 224
Bundle size	6.72 GB
Activation quantization	None; activations remain BF16
Weight kernel/layout	vLLM Marlin WNA16

Format Note

This release uses vLLM's Marlin WNA16 packed layout and kernel. It is not a backend-neutral GPTQ or AWQ checkpoint and should be loaded through Cosmos Lite's bundle loader. All four variants are weight-only W4A16/W8A16 or fixed mixed-precision combinations; activation quantization is not used.

The server binds to 127.0.0.1:8000 by default and has no built-in TLS or authentication. Do not expose it directly to an untrusted network.

Provenance

Base policy: nvidia/Cosmos3-Edge-Policy-DROID at 3ea407af3e156c0af3b4bb6edd85842cc9a58777
Calibration dataset: nvidia/Cosmos3-DROID at 5c11a20accb11497270a5247a7f1e66ad04c956c
Wan VAE: Wan-AI/Wan2.2-TI2V-5B at 921dbaf3f1674a56f47e83fb80a34bac8a8f203e
Runtime: Cosmos Lite

Limitations And Safety

RoboLab Banana results do not establish transfer to a new task, robot, camera layout, or action contract. Validate those changes with paired closed-loop rollouts. This policy is not real-robot safety certified; deployment requires independent E-stop, watchdog, workspace/joint/action limits, stale-command rejection, and operator supervision.