philipjohnbasile/hy3-demolition-mlx-reap25-v1-mtp

Hy3-Demolition-MLX reap25-v1-mtp

The MTP-equipped variant of reap25: its fused trunk with the Hy3 NextN (Multi-Token-Prediction) sidecar grafted back on (num_nextn_predict_layers=1, num_experts=144), for self-speculative decoding on MTPLX.

Status — honest

This artifact is MTP-equipped but not yet end-to-end runnable. mtplx inspect (release v2.0.1) recognizes it — "HY V3 MTP markers recognized, but MTPLX does not yet have a native MLX runtime backend for this family" — i.e. it is recognized-backend-pending. The native hy_v3 runtime backend is in flight as MTPLX PR #142, gated on hy_v3 reaching MTPLX's mlx-lm pin. Until that lands, use the AR daily-driver (reap25) instead.

How it was built

The MTP head consumes the trunk's final hidden state (hidden_size 4096, unchanged by pruning) and runs its own MoE on the global num_experts. So the base checkpoint's mtp.* sidecar grafts directly onto the fused AR trunk — no re-heal, no re-prune of the trunk.

• The MTP head's experts are sliced 192→144 (packing-safe whole-expert axis slice, scales-L2 pick) to match the pruned trunk's num_experts. The MTP head is a draft component MTPLX verifies every token against, so this affects draft acceptance (speed) only — never correctness.

Graft script + mtplx inspect receipts: https://github.com/PhilipJohnBasile/hy3-demolition-mlx (scripts/38_mtp_sidecar_graft.py, eval/receipts/mtplx_inspect_*.json).

Limitations

• Does not run on stock mlx_lm as MTP. The fast MTP path needs the MTPLX backend; mlx-lm's own per-token self-speculative loop is ~4.7× slower than AR (measured), which is why the MTPLX batched-verify backend is the target.
• Everything from the base reap25 card applies (quantized MoE, English/agent focus, no tool execution).
• End-to-end MTP behavior is unverified until the backend loads it; recognition is structural (mtplx inspect), not a live run.

Base recipe + receipts: https://github.com/PhilipJohnBasile/hy3-demolition-mlx

LM Studio / Ollama

Not yet. Same root blocker as MTPLX — hy_v3 must reach mainline mlx-lm (ml-explore/mlx-lm#1211). And note: even once loadable, LM Studio runs AR-only (no MTP speculative decoding) — the MTP heads only pay off on MTPLX. For LM Studio, use the AR sibling instead.

Run on a smaller Mac

The AR trunk of this model can run on 16–128 GB Macs via the SSD expert-streaming pager (src/hy3_streaming.py in the source repo, bit-identical, zero quality loss). The MTP head is small and stays resident; only trunk experts stream. See docs/64gb-feasibility.md for the measured 16/32/64 GB tiers.