MiMo-V2.5-DFlash → PrismaQuant MXFP4/BF16 (170 GB) — custom-code artifacts
WIP: custom-code adaptation preserved before final export. Target: a 170 GiB weight compressed-tensors checkpoint of XiaomiMiMo/MiMo-V2.5-DFlash, servable on stock vLLM 0.11+, DFlash/MTP speculative decoding intact.
Committed quant config (no KL measurement — operator override, time-boxed)
- Experts (256 routed, layers 1-47, ~302.8B params) → MXFP4 (W4).
- Sensitive set (attention qkv_proj+o_proj, dense layer-0 MLP) + MTP → BF16.
- router/gate + norms → FP32; embed + lm_head → BF16; in-model vision/audio towers → BF16 passthrough.
- audio_tokenizer/ DROPPED; dflash/ drafter → BF16, separate (not counted in 170).
- Measured footprint (safetensors headers): experts MXFP4 + all-BF16 sensitive ≈ 164.7 GiB (≤170 GiB).
Files
mimo_v2.py— PrismaQuant MiMoV2Profile (nested per-expert MoE, fused qkv, FP8-source, 3 dense MTP layers). Verified: detect_profile→MiMoV2Profile, 256 experts collapse per layer, dense-0 scored separately.mimo_v2.json— declarative model-structure spec.registry.patched.py— profile registration (before DeepseekV4 + DefaultProfile).calib/*.jsonl— domain calibration set (JSON/tool-call/tender-extraction), disjoint gate split.step1_keymatch.py— state-dict key-match diagnostic.
BLOCKER (documented for reproduction)
MiMo-V2.5-DFlash ships config-only (configuration_mimo_v2.py, no modeling_*.py; auto_map=AutoConfig only).
transformers v5.0 built-in MiMoV2FlashForCausalLM (model_type mimo_v2_flash) is STRUCTURALLY INCOMPATIBLE:
batched-3D experts (vs 72,192 separate per-expert tensors), split q/k/v (vs fused qkv_proj), no MTP.
=> A vendored modeling_mimo_v2.py (name-matching the checkpoint: nested 256-expert ModuleList w/ separate
gate/up/down_proj; fused qkv_proj + attention_sink_bias on 39 SWA layers; model.mtp.layers.{0,1,2}; dense layer-0)
is required for the transformers-based tooling path. MXFP4 being data-free also permits a direct tensor-level
requant that bypasses model-loading. dflash/dflash.py may seed the MTP/attention shim.