vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF

Ternary Bonsai 27B with Embedded MTP — GGUF

One target, one embedded NextN predictor, one deployment artifact per representation.

qwen35 · 64 target blocks · 1 embedded NextN block · 866 tensors · Apache-2.0

---

Release summary

This repository publishes two self-contained GGUF representations of the same logical model graph:

1. TQ2_0 — primary, native ternary representation
2. Q2_K — compatibility-oriented standard representation

Each file contains both the 27B-class target and its embedded Qwen-style NextN/MTP predictor. There is no sidecar drafter, no second model identity and no --model-draft lifecycle.

The two files preserve the same logical tensor set, names, shapes and ordering. They do not claim bit-identical dequantized values, logits or generated text: the storage and dequantization contracts are different.

Artifacts

Priority	Representation	File	Exact size	SHA-256
1	Native TQ2_0	Ternary-Bonsai-27B-MTP-TQ2_0.gguf	8,785,215,776 bytes / 8.18 GiB	480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20
2	Standard Q2_K	Ternary-Bonsai-27B-MTP-Q2_K.gguf	10,491,466,016 bytes / 9.77 GiB	850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410

1. Native TQ2_0 — primary artifact

Choose this file when the target runtime supports the native TQ2_0 tensor type.

• smallest artifact in this repository;
• 480 tensors stored natively as TQ2_0;
• embedded target and MTP paths in one GGUF;
• preserves the native ternary storage contract;
• requires runtime support for qwen35, draft-mtp and TQ2_0.

2. Standard Q2_K — compatibility artifact

Choose this file when standard Q2_K tensor support is preferable to native TQ2_0 support.

• all 480 native ternary tensors are represented as ordinary Q2_K;
• the 17 tensors that were already Q2_K remain Q2_K;
• final inventory: 497 Q2_K tensors;
• no private GGUF tensor type and no custom tensor-type patch;
• larger than the native artifact because standard Q2_K carries its own block metadata;
• still requires runtime support for the qwen35 architecture and embedded draft-mtp execution.

The Q2_K file is a deterministic representation transform, not a generic calibration-based post-training quantization pass.

Which file should I use?

Requirement	Recommended file
Minimum repository artifact size	Ternary-Bonsai-27B-MTP-TQ2_0.gguf
Native ternary execution	Ternary-Bonsai-27B-MTP-TQ2_0.gguf
Runtime already supports TQ2_0	Ternary-Bonsai-27B-MTP-TQ2_0.gguf
Prefer standard Q2_K tensor storage	Ternary-Bonsai-27B-MTP-Q2_K.gguf
Runtime lacks TQ2_0 but supports Q2_K	Ternary-Bonsai-27B-MTP-Q2_K.gguf
Need embedded MTP without a sidecar drafter	either file
Need architecture-independent output equivalence between representations	neither; no such claim is made

Model graph

self-contained GGUF
│
├── qwen35 target
│   ├── target token embedding
│   ├── 64 transformer blocks
│   ├── final normalization
│   └── target output path
│
└── embedded MTP / NextN path
    ├── dedicated MTP token embedding
    ├── 1 Qwen-style NextN predictor block
    ├── MTP normalization
    └── MTP output path

The ordinary target path and the speculative path resolve from the same file and the same target identity.

Deployment property	External drafter	This repository
Target files	1	1
Drafter files	1	0
Runtime model identities	2	1
Independent target/drafter version skew	possible	structurally removed
Separate drafter checksum	required	not applicable
--model-draft / -md	required	not used
Embedded draft-mtp path	not intrinsic	yes

Tensor inventories

The files contain the same 866 logical tensors. The difference is the storage type of the 480 ternary tensors.

Native TQ2_0

TQ2_0  480
Q2_K    17
F32    359
Q4_K     1
Q5_K     6
Q6_K     3
-------------
total   866

Standard Q2_K

Q2_K   497
F32    359
Q4_K     1
Q5_K     6
Q6_K     3
-------------
total   866

Q2_K representation contract

The compatibility artifact converts each native ternary block into an ordinary 84-byte Q2_K block:

16 bytes  scale/min metadata
64 bytes  packed 2-bit symbols
 2 bytes  fp16 d
 2 bytes  fp16 dmin
---------
84 bytes  total

For the 480 converted tensors, the transform follows this contract:

packed Q2_K symbols = packed native ternary symbols

group metadata =
    0x00  when all 16 symbols in the logical group are the center symbol
    0xFF  otherwise

Q2_K d    = fp16(fp32(native_scale) / 15)
Q2_K dmin = fp16(fp32(native_scale) / 15)

Consequences:

• the packed discrete symbol identities are preserved by construction;
• the output uses the standard Q2_K block layout;
• the conversion is deterministic for a fixed source artifact;
• the conversion does not claim floating-point equivalence with native TQ2_0 dequantization;
• downstream logits and generated text may differ between representations.

This distinction is intentional: the Q2_K variant is a compatibility representation of the ternary code field, not an assertion that the two dequantizers are interchangeable.

Runtime contract

Use a llama.cpp revision that supports:

• the qwen35 architecture;
• the embedded draft-mtp path;
• TQ2_0 for the primary artifact, or standard Q2_K for the compatibility artifact.

The Q2_K variant removes the native tensor-type requirement. It does not remove the architecture or MTP-runtime requirements.

Runtime option names can change across llama.cpp revisions. Confirm the available flags with:

llama-cli --help | grep -E 'draft-mtp|spec-draft|model-draft'

Download

Primary TQ2_0

hf download vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF \
  Ternary-Bonsai-27B-MTP-TQ2_0.gguf \
  SHA256SUMS \
  --local-dir .

Compatibility Q2_K

hf download vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF \
  Ternary-Bonsai-27B-MTP-Q2_K.gguf \
  SHA256SUMS \
  --local-dir .

Verify either or both downloaded artifacts:

sha256sum -c SHA256SUMS --ignore-missing

Inference

Select one artifact:

MODEL="Ternary-Bonsai-27B-MTP-TQ2_0.gguf"
# or:
# MODEL="Ternary-Bonsai-27B-MTP-Q2_K.gguf"

Ordinary target decode

llama-cli \
  -m "$MODEL" \
  -p "Derive a cost model for speculative decoding." \
  -n 256

Embedded MTP

llama-cli \
  -m "$MODEL" \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  -p "Derive a cost model for speculative decoding." \
  -n 256

There is deliberately no -md / --model-draft argument.

OpenAI-compatible server

llama-server \
  -m "$MODEL" \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --host 127.0.0.1 \
  --port 8080

Speculative-decoding economics

Embedded MTP is beneficial only when the accepted speculative work amortizes predictor and verification cost:

(C_draft + C_verify) / E[committed draft tokens] < C_target-step

Acceptance rate is an intermediate statistic, not the optimization objective. The relevant endpoint is end-to-end committed-token throughput under a fixed workload and configuration.

A minimal controlled comparison is:

A. ordinary target decode
B. draft-mtp with n_max = 1
C. draft-mtp with n_max = 2

Hold constant:

• exact model file and checksum;
• prompt corpus and prompt order;
• context length;
• sampler chain and sampler parameters;
• random seed where supported;
• batch and micro-batch sizes;
• thread count and affinity;
• GPU-layer split and device placement;
• KV-cache types;
• warm-up policy;
• output token budget.

Report separately:

• prompt-evaluation throughput;
• committed generation throughput;
• attempted and accepted draft tokens;
• acceptance ratio;
• time to first token;
• peak host memory and device memory;
• full command line and runtime revision.

Do not compare the native and compatibility representations while attributing the entire difference to MTP: that changes two independent variables at once.

No hardware-independent speedup claim is made by this repository.

Portable artifact validation

The public claims below are properties of the files and metadata, not of a specific workstation.

Check	Native TQ2_0	Standard Q2_K
Exact byte length recorded	pass	pass
SHA-256 recorded	pass	pass
GGUF V3 parse	pass	pass
Architecture metadata: qwen35	pass	pass
Logical tensor count: 866	pass	pass
Target depth: 64 blocks	pass	pass
Embedded NextN depth: 1 block	pass	pass
Tensor-name, shape and order inventory	pass	pass
Expected type histogram	pass	pass
Converted native tensors	not applicable	480 / 480
Public metadata privacy scan	pass	pass
Hub object size and content hash verification	pass	verified at publication

The model card intentionally publishes no workstation-specific throughput, device name, driver name, local path, hostname or private build log.

Integrity

The checksum is the authoritative artifact identity; the filename is a human-readable label.

File	SHA-256
Ternary-Bonsai-27B-MTP-TQ2_0.gguf	480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20
Ternary-Bonsai-27B-MTP-Q2_K.gguf	850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410

Verify explicitly:

printf '%s  %s\n' \
  '480c695b0de5b78b720d7f500eb3e1650359d0ac8e18c29b231a14b2d9c9bf20' \
  'Ternary-Bonsai-27B-MTP-TQ2_0.gguf' \
  | sha256sum -c -

printf '%s  %s\n' \
  '850b4f4041c7d73e4442f23b1c70464f68fb76bf8717bc13ada1562680684410' \
  'Ternary-Bonsai-27B-MTP-Q2_K.gguf' \
  | sha256sum -c -

Construction and provenance

This is a model-engineering artifact. It is not a new pretraining run and not an instruction fine-tune.

The final graph combines:

• a pinned Ternary Bonsai 27B target trunk;
• a Qwen3.6-derived NextN/MTP donor package;
• a dedicated MTP embedding and output path;
• a stock-compatible qwen35 GGUF graph;
• two published storage representations of the same logical tensor graph.

Role	Immutable source	Revision	Source SHA-256
Ternary target trunk	prism-ml/Ternary-Bonsai-27B-gguf	3f8cc399dde45ac0475d023634974407af34907c	f659ca3dd7e28ada5d8b5f3637862d0d51ef433bde032ec4c8990ed27c91a385
MTP donor package	lym00/Qwen3.6-27B-MTP-ONLY-GGUF	03b35db648da71e23ecfb239f70661e069b16054	97697fc5278d4bfd0afaa733ba28c4338cb0396b98d5f5acb781e515f72b52c0

The donor package identifies Qwen3.6-derived MTP material but does not pin a separate underlying Qwen source revision. This card does not invent one.

The predictor was grafted from a compatible donor. It was not jointly trained with the final ternary target.

See NOTICE for attribution and LICENSE for terms.

Scope of claims

This repository claims:

• a self-contained GGUF graph with target and embedded MTP paths;
• exact artifact sizes and cryptographic identities;
• the documented tensor inventories and graph structure;
• complete conversion coverage for the 480 native ternary tensors in the Q2_K artifact;
• standard Q2_K storage for the compatibility representation.

This repository does not claim:

• a new pretrained or fine-tuned model;
• joint training of the final target and predictor;
• output equivalence between TQ2_0 and Q2_K;
• preservation of upstream benchmark scores after graph construction or quantization;
• a universal MTP speedup;
• compatibility with every historical or future runtime revision;
• multimodal capability or inclusion of a vision projector;
• endorsement by Prism ML, Qwen, Alibaba Cloud, lym00 or llama.cpp.

Known limitations

• Aggressive low-bit representations can affect accuracy, calibration and long-context behavior.
• The grafted predictor may be suboptimal relative to a predictor jointly trained against the final target.
• Speculative decoding can improve, match or reduce throughput depending on acceptance, backend, context, sampler and device placement.
• The Q2_K compatibility representation is larger than native TQ2_0.
• A successful parse and exact tensor inventory do not substitute for task-specific evaluation.
• Reproducible performance evaluation requires publishing the complete runtime and workload configuration.

License and attribution

Repository artifacts are distributed under Apache-2.0, subject to the included LICENSE and NOTICE files and the terms of the source artifacts.

This independent release is not endorsed by Prism ML, Qwen, Alibaba Cloud, lym00 or the llama.cpp project.

Citation

@misc{vinpix2026ternarybonsai27bembeddedmtp,
  title  = {Ternary Bonsai 27B with Embedded MTP: Native TQ2_0 and Standard Q2_K GGUF},
  author = {vinpix},
  year   = {2026},
  url    = {https://huggingface.co/vinpix/Ternary-Bonsai-27B-Stock-MTP-GGUF}
}

---

Primary representation: native TQ2_0
Compatibility representation: standard Q2_K
Deployment boundary: one GGUF, one checksum, no sidecar drafter