Stick9190/qwen3vl_8b_int8_convrot

Qwen3-VL 8B - INT8 ConvRot

INT8 ConvRot quantization of Qwen3-VL 8B for ComfyUI text-encoder and multimodal conditioning use. The 17.53 GB BF16 checkpoint is reduced to approximately 10.98 GB.

Conversion

• Tool: silveroxides/convert_to_quant
• Format: INT8 row-wise with embedded ConvRot metadata
• Language ConvRot group size: 256
• Vision ConvRot group size: 16
• Method: learned rounding (AdaRound) with low-memory streaming conversion
• Quantized: language-transformer blocks 3-34 and vision-transformer blocks 1-25 (324 matrices; ~6.55B parameters, or 74.8% of matrix parameters)

The token embedding, LM head, language blocks 0-2 and 35, vision boundary blocks, vision embeddings, and vision merger components remain BF16.

Layer and group selection

Qwen3-VL injects three DeepStack vision representations into language blocks 0-2, so those multimodal integration blocks remain BF16. Language block 35 is retained as the final block for general Qwen3-VL use.

Vision blocks 0 and 26 form the input and output boundaries of the 27-block vision tower. Keeping these boundary layers in BF16 is a conservative quality-preservation heuristic rather than a model-author-specified ConvRot requirement.

The language matrices are compatible with ConvRot group size 256. The vision tower uses input dimensions including 1152 and 4304, which are not divisible by 256 but are divisible by 16. A custom group size of 16 ensures that the selected vision weights receive ConvRot instead of silently falling back to plain row-wise INT8.

Command

ctq -i <input-model>.safetensors -o <output-model>.safetensors `
  --int8 --scaling_mode row `
  --convrot --convrot-group-size 256 `
  --custom-layers '^model\.visual\.blocks\.(?:[1-9]|1[0-9]|2[0-5])\.' `
  --custom-type int8 `
  --custom-scaling-mode row `
  --custom-convrot `
  --custom-convrot-group-size 16 `
  --comfy_quant --save-quant-metadata `
  --low-memory --device cuda `
  --exclude-layers '(^model\.embed_tokens\.weight$|^lm_head\.weight$|^model\.layers\.(0|1|2|35)\.|^model\.visual\.blocks\.(0|26)\.|^model\.visual\.(pos_embed|patch_embed|merger|deepstack_merger_list)(\.|$))' `
  --verbose NORMAL

Quantization is lossy, so outputs are not bit-identical to the original BF16 checkpoint.