bleysg/DeepSeek-V4-Flash-DSpark-drafter-GGUF

DeepSeek-V4-Flash DSpark drafter (Q2K-Q8 GGUF)

The DSpark block drafter for DeepSeek-V4-Flash, packaged for the ds4 C+CUDA inference engine — specifically the Entrpi/ds4 batched-serving fork (release v0.1.1+), where it drives lossless speculative decode at a 9-workload suite mean of 1.38x plain decode on a DGX Spark (GB10), peaking at 1.71x on structured content. Output is lossless by construction: the target model's verify forward is the sole token source.

What's in the file

The drafter's 3 full DeepSeek-V4 transformer layers (attention + 256-expert MoE), the target-fusing projection (main_proj/main_norm over target layers 40/41/42), the rank-256 Markov head, the confidence head, and the output head. embed_tokens/lm_head are shared from the target model at load time and are not duplicated here.

Quant policy: Q2_K routed experts (the ds4 v0.1.1 ship default — measured equal to Q4_K in acceptance and throughput, 4.2 GiB smaller), Q8_0 attention/projections/shared experts, F32 norms/gates, F16 Markov/confidence heads. ~6.5 GiB.

Extracted from the official checkpoint's drafter shards with gguf-tools/dspark_extract.py (FP8 e4m3/ue8m0 dequant identical to the engine's C implementation), routed experts requantized to Q2_K.

Usage

One-command serving on a DGX Spark via ds4-on-spark:

curl -sSL https://raw.githubusercontent.com/entrpi/ds4-on-spark/main/install.sh | bash -s -- --start

Or by hand, alongside the base + MTP GGUFs from antirez/deepseek-v4-gguf:

env DS4_CONT_MTP_MODE=2 DS4_CONT_DSPARK=1 \
    DS4_DSPARK_MODEL=DSpark-drafter-Q2K-Q8.gguf \
  ./ds4-server --cuda -m <base.gguf> --mtp <mtp.gguf> -c 32768 --port 8000

Speculation is content-dependent (52-91% draft acceptance across workloads). Since ds4 fork v0.1.1 that variance is bounded by default: a terminal yield-quench controller turns speculation off per request when acceptance can't pay the verify cost (worst case ~0.96x plain), and a kv-depth gate hands

64k-context requests to plain decode. Benchmarks, the two-corpus context frontier, and the break-even analysis: see the ds4-on-spark README.