danielrmay/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-W4A16

NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B, INT4 (W4A16) weight-only

A 120B-class NVIDIA model, rebuilt to perform on two 3090 GPUs, not just fit.

Getting a large model to load on limited hardware is easy, and HF is full of quants that do exactly that but little more. Making one run well is harder - variables like model architecture, quantization format, runtime support, kernel availability, memory pressure and other environmental quirks don't just define if the resulting experience is useful, they define whether it's a vote of confidence in local inference at all. On consumer-multi-GPU rigs in the Ampere class, fitting a capable agentic model has often meant sacrificing context length, TTFT, or falling back to unoptimized CPU paths. Model families like NVIDIA Nemotron and Alibaba's Qwen have greatly improved the landscape, but choosing a model and quant that can fully exploit older consumer GPUs remains far harder than it should be. This mixed-precision quant targets the highest quality-preserving and fastest-serving path broadly supported for the most popular pre-owned multi-GPU configurations used by enthuasiasts today (RTX 3090s), and the quick starts below preserve the full context window while using fused kernels and CUDA graphs for acceleration. The goal is to deliver the most capable model a common local inference rig should run, rather than the largest model it can boot.

NVIDIA took their 120B-A12B Nemotron Super MoE hybrid Mamba agentic reasoning model and compressed it down to Puzzle, a 75B-A9B model via NAS. This quant takes that model one step further and shrinks it by 70% - from 150 GB of BF16 weights to a 44.5 GB mostly-W4A16 checkpoint - with minimal tradeoffs as closely aligned as possible to NVIDIA's own recipe, so that it loads and serves at speed and at full context on a 2x RTX 3090/4090 rig (2x24 GB), built with compressed-tensors and tested under vLLM and Docker.

The community GGUF and MLX builds target llama.cpp and Apple Silicon, and reports indicate it only runs at 12 tok/s on a 2x3090 rig, leaving efficiency wins on the table. NVIDIA's NVFP4 build can now run on Ampere via vLLM, but at 53.5 GB it doesn't fit two 24 GB cards. On Ampere, under vLLM, this is the only build of this model that fits with room to serve. Real RTX 3090 owners have now confirmed it in practice!

So what does that get you? (measured on the reference rig, 2x RTX A6000, vLLM 0.23.0)

• ~85 tok/s decode, no slower at full context single-stream, under concurrency sustained ~1,000 tok/s aggregate on long generations.
• ~5,000 tok/s prefill cold, tapering to ~3,200 tok/s at the full window, so a half-full context window is read in about 25 seconds once, and 0.6s every turn after, with prefix caching enabled
• Weights and full cache in 48GB KV costs 4 KiB per token at fp8, 12 KiB with prefix caching so the full window is 1 or 3 GiB in size, so the reference 96GB VRAM rig successfully could achieve a near ~10M token pool. We chose to optimize at fp8 for VRAM savings instead of utilizing int8_per_token_head, but nothing precludes its use and we anticipate it may give improved performance.
• 91.7 on AIME25, against NVIDIA's published 89.7 for the BF16 base, at their documented generation settings. Our confidence interval contains their figure, so read it as indistinguishable, not better, and additional benchmarks are pending.

Why it suits agents running at home. Two things make agentic loops expensive: a context that keeps growing, and re-reading that context on every turn. Neither costs much here. An agent an hour into a task decodes as fast as one that just started, and with the prefix cache on, re-reading context costs almost nothing. On 2x24GB cards you currently pick per serve: the full window, or the cache (the trade is measured under ⚠️ Running on 2x24 GB rigs). Both at once is what this build is being optimized to achieve, and every byte the recipe saves goes towards closing that gap.

---

Contents

• ⚠️ Running on 24GB GPUs
• Quick start (vLLM)
• Quick start (Docker)
• What it is
• The recipe
• Context and KV cache
• Measured performance
• Quality
• Limitations
• Custom code provenance
• Provenance and license

---

⚠️ Running on 2x3090/4090 (24GB) rigs

--gpu-memory-utilization 0.975 is required on 2x24 GB rigs. The vLLM default does not fit, and past 0.975 the CUDA context runs out of headroom causing the runtime to fail to start in constrained serving tests.

At 97.5% utilization, the weights leave roughly one full window of KV pool: ~273K tokens (1.04x the window), and four flags decide how that budget is spent:

• --max-model-len is how large each context can grow
• --max-num-seqs is how many contexts can grow at once
• --enable-prefix-caching caches prefill, but roughly triples what every token costs because it checkpoints Mamba SSM state alongside KV
• --max-num-batched-tokens caps the prefill chunk, lowering it shrinks the activation buffers vLLM reserves, leaving pool for the cache at the cost of slower prefill.

Tuning examples include:

• All of it on one context, no caching: with prefix caching off (cold prompts pay full prefill) and the full 262K window, you can fit one full-length context at a time. This is reflected in the Docker recipe below.
• Caching on, lower ceiling: prefill-heavy agentic loops that would usually get slower over time get turns with up much faster TTFT, but the ceiling drops: --max-model-len 49152 --max-num-batched-tokens 1024 --enable-prefix-caching fits, but capping the batch slows prefill, but this can be acceptable as agentic use-cases re-send the same usually-cached context on every turn.
  • Budget length for reasoning accordingly: 18% of this model's AIME25 chains ran past 49K, and long thoughts will truncate responses.
• Higher concurrency, lower ceiling, with caching: --max-model-len 32768 --enable-prefix-caching: two cached contexts fit when they average under ~22K.
• CUDA graphs fit under all of these. Tested without --enforce-eager.
• --disable-custom-all-reduce may be needed for non-NVLinked modules if you aren't running a custom driver.

Quick start (vLLM)

Serve with vLLM (0.23.0 or newer). --trust-remote-code is required (what that runs).

vllm serve danielrmay/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-W4A16 \
  --served-model-name puzzle \
  --trust-remote-code \
  --tensor-parallel-size 2 --enable-expert-parallel \
  --disable-custom-all-reduce \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.975 \
  --max-num-seqs 2 \
  --reasoning-parser nemotron_v3 --tool-call-parser qwen3_coder --enable-auto-tool-choice

Reasoning and tool parsers are wired up front for agentic use cases.

• On 48 GB card rigs: higher --max-num-seqs gets closer to Measured performance.
• On a single 48 GB card: --tensor-parallel-size 1 and you can likely knock down --gpu-memory-utilization
• FlashInfer JIT sampler fails to build on an older CUDA toolchain? Set VLLM_USE_FLASHINFER_SAMPLER=0.

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
resp = client.chat.completions.create(
    model="puzzle",
    messages=[{"role": "user", "content": "Refactor this function to be iterative: ..."}],
    temperature=1.0, top_p=0.95,          # NVIDIA's documented settings for this model
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)
print(resp.choices[0].message.content)

Quick start (Docker)

We use the official vLLM image. You need Docker and the NVIDIA Container Toolkit.

Confirm your environment is ready and Docker can see your GPUs:

docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi -L

Below is a ready-to-run 2x24 GB (3090/4090) configuration. The first launch downloads the weights into your Hugging Face cache, then serves on port 8000:

docker run --gpus all --ipc=host --network=host \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_USE_FLASHINFER_SAMPLER=0 \
  --entrypoint vllm \
  vllm/vllm-openai:v0.25.1-cu129-ubuntu2404 \
  serve danielrmay/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-W4A16 \
  --served-model-name puzzle \
  --trust-remote-code \
  --tensor-parallel-size 2 --enable-expert-parallel \
  --disable-custom-all-reduce \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.975 \
  --max-num-seqs 2 \
  --reasoning-parser nemotron_v3 --tool-call-parser qwen3_coder --enable-auto-tool-choice

VLLM_USE_FLASHINFER_SAMPLER=0 is preset because the sampler JIT builds against your host toolchain, which the image cannot pin. Remove it if your host builds it cleanly.

---

What it is

Base model	NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B (BF16)
Parameters	75.3B total, 9.3B active per token. HF's sidebar says 82B: its counter assumes every packed tensor is 4-bit, and this checkpoint mixes 4-bit and 8-bit
Architecture	Mamba2 + Transformer hybrid, Latent Mixture-of-Experts (512 routed experts per MoE layer), only 8 attention layers
Quantization	INT4/INT8 weight-only, activations stay in bf16
On-disk size	44.5 GB
Format	compressed-tensors pack-quantized, loads natively in vLLM (Marlin fused-MoE and WNA16 kernels, Ampere and newer)
Context window	262,144 tokens trained. NVIDIA documents the base model to 1M, with published RULER curves.

---

The recipe: a derivative of NVIDIA's own

This is NVIDIA's component policy, re-derived for weight-only INT4 on Ampere. Every precision decision is theirs. Only two things differ and both are forced by the hardware.

Component	NVIDIA (NVFP4)	This build
Routed experts (~89% of weights)	4-bit, block 16, 4-bit activations	INT4, group 128	bit-width adopted
Shared experts	FP8 (8-bit weights and activations)	INT8, group 128	bit-width adopted
Mamba in/out projections	FP8	INT8, group 128	bit-width adopted
Attention QKV/O	bf16	bf16	adopted verbatim
Latent projections	bf16	bf16	adopted verbatim
Router, gates, norms	bf16	bf16	adopted verbatim
Embeddings, lm_head	bf16	bf16	adopted verbatim
KV cache	FP8	fp8 recommended	adopted verbatim

Divergence 1: activations stay in bf16. Forced by Ampere, and the most protective property of the build. Quantization damage concentrates in the decode path, where a perturbed activation changes the token that gets emitted and everything after it is conditioned on the corruption. Weight-only quantization leaves that path numerically intact, which is why Red Hat's 500,000-evaluation study finds W4A16 rivaling W8A8. NVIDIA's FP8 and NVFP4 push quantized activations through the experts. This build does not.

"Forced by Ampere" is not a figure of speech. We went looking for a baseline to measure against. There is not one:

Path	Why not, on sm_86
BF16, 150 GB	far more than two 48 GB cards hold
NVIDIA NVFP4	its 4-bit activations have no path on sm_86 and fall back to bf16, so it reduces to this build's weight-only regime, and at block 16 it does not fit
NVIDIA FP8, 83 GB	vLLM will not load it at all: Minimum capability: 89. Current capability: 86. Editing the config to force it past that gate puts it on Marlin kernels, where it fills both cards and is still loading when it runs out of memory

There is no clean baseline to measure this build against on Ampere: FP8 will not load on our reference hardware, and NVFP4 falls back to bf16 activations, the same weight-only regime as this build rather than the 4-bit-activation design NVIDIA shipped, and at block 16 it does not fit anyway.

Divergence 2: group size 128, not 16. NVFP4's 16-element block is a hardware constant, not a precision judgement: Blackwell's tensor cores consume that format, so it costs NVIDIA nothing to use it. In a weight-only INT4 build those scales become over 40,000 real tensors sitting in VRAM, and the size is the whole argument. Marlin offers {32, 64, 128}, and the scales double every time the group halves.

vLLM gets 23.4 GiB of a 24 GB card at 0.975. That is the budget the weights have to fit inside:

Group	Scales	Weights per card (TP=2)
128	1.07 GiB	20.7 GiB
64	2.14 GiB	21.3 GiB
32	4.28 GiB	22.3 GiB

At group 32 the weights alone take 22.3 of those 23.4 GiB, leaving less than the activation buffers need. Group 128 is not a preference, it is what fits the hardware this build targets. We did not measure a quality delta between the group sizes, so we do not claim one.

Method: data-free per-group MSE-optimal clipping. No calibration set, so the build is deterministic. GPTQ on the always-on components was evaluated and matched it within measurement noise, so the deterministic method was kept. Our friends at Red Hat AI landed on much the same configuration independently for Nemotron-3-Ultra-550B: INT4, group 128, data-free model_free_ptq, no calibration set. ♥

---

Context and KV cache

This model is unusually cheap to give a long context to. Only 8 of its 88 layers use attention, and those have just 2 KV heads (16:1 grouped-query attention). Everything else is Mamba or a routed-MoE layer, neither of which keeps a KV cache that grows with context.

KV dtype	Per token	Cost of the full 262K window
bf16	8.0 KiB	2.0 GiB
fp8 (recommended)	4.0 KiB	1.0 GiB
int4	2.0 KiB	0.5 GiB

On a 2x24 GB rig the binding constraint is fitting the weights, not the KV cache. Once they fit, the full window is essentially free.

Rig	--gpu-memory-utilization	Measured KV pool (fp8)
2x24 GB (3090/4090)	0.975	just over the full window
2x48 GB (reference rig)	0.92	9.6M tokens, 37x the full window

---

Measured performance

Reference rig: 2x RTX A6000 (48 GB, sm_86), vLLM 0.23.0, TP=2 with expert-parallel, fp8 KV, CUDA graphs enabled.

Decode does not slow down as your context grows. A conventional transformer gets slower the longer the context, because every token it emits must scan the whole KV cache. This one barely has one.

Context	Decode
1.8K	85 tok/s
9K	85 tok/s
34K	85 tok/s
62K	86 tok/s
126K	86 tok/s
246K	89 tok/s

Measured cold, one run per context: decode is timed by differencing a 128-token generation against a 1-token generation, so prefill cancels out rather than being estimated away. Read the drift as flat, not as a gain at long context.

Prefill and prefix caching

Prefill runs at roughly 5,000 tok/s cold, tapering to about 3,200 tok/s at the full window. With --enable-prefix-caching, a context the server has already seen is not prefilled again:

Context	Cold	Warm (turn 2 onward)
9K	3.2 s	0.6 s	5x faster
34K	5.8 s	0.8 s	7x faster
126K	25.3 s	0.6 s	40x faster

You pay that prefill once, and it costs some KV pool, but the reference rig still holds many times the full window with the cache on.

Concurrency

vllm bench serve, 4K in / 512 out, fp8 KV, on the reference rig:

Concurrent requests	Aggregate output	Per-stream latency	Time to first token
1	70 tok/s	12 ms/token	1.2 s
16	343 tok/s	39 ms/token	3.8 s
64	476 tok/s	119 ms/token	7.9 s

The single-stream row sits below the 85 tok/s measured above because vllm bench serve puts the 1.2 s prefill in its wall clock. The decode rate is the latency column: 12 ms/token is 83 tok/s.

Aggregate throughput keeps rising while per-stream latency degrades, because at batch the MoE becomes compute-bound rather than memory-bound.

That benchmark sends 4K of prompt to get only 512 tokens back, which understates a generation-heavy workload. The AIME runs under Quality are the other extreme, a 200-token prompt and chains of tens of thousands of tokens, and they sustained about 1,000 tok/s aggregate at 48 concurrent streams.

Those runs are also the stability evidence. Across three builds of this model the AIME work came to 43 million tokens over twelve hours of continuous inference, with chains running to the full 131,072-token budget and zero server errors. That is the whole stack under sustained load: compressed-tensors weights through Marlin fused-MoE kernels, on a Mamba and attention hybrid, with an fp8 KV cache and CUDA graphs, on two Ampere cards.

---

Quality

Reasoning: AIME25, 91.7

AIME 2025, all 30 problems across both contests. temperature=1.0 and top_p=0.95 are the settings NVIDIA document on this model's own card. The max_new_tokens=131072 budget comes from the reproducibility example they publish for the parent model. 16 samples per problem, 480 generations.

	AIME25
NVIDIA BF16 (published)	89.7
NVIDIA NVFP4 (published)	89.9
NVIDIA FP8 (published)	89.4
This build, INT4	91.7

The 95% confidence interval on our figure, bootstrapped over problems, is [83.3, 97.9], and all three of NVIDIA's numbers fall inside it. Read that as indistinguishable from the BF16 base at the resolution 30 problems and 16 samples can resolve, not as an improvement on it. This is avg@16. NVIDIA do not publish a sample count for this model; the reproducibility example they publish for the parent uses 64.

Where this sits on NVIDIA's own ladder. This checkpoint is the output of two compressions, and NVIDIA publish the rung above ours:

	Parameters	AIME25
Nemotron-3-Super-120B-A12B, BF16 (the parent)	120.7B	92.2
Puzzle-75B-A9B, BF16 (NVIDIA's compression, and this build's base)	75.3B	89.7
Puzzle-75B-A9B, INT4 (this build)	75.3B	91.7

All three sit within 2.5 points of each other, well inside that interval, so the ranking here is not the finding. This is: NVIDIA's architectural compression and this build's 4-bit quantization, stacked, take a 120B BF16 model down to a 44.5 GB checkpoint, and AIME25 cannot tell the 120B BF16 parent from this INT4 build.

The model solved all 30 problems at least once, and its chains run long. Across the 454 generations that ran to completion, the median is 14,268 generated tokens, the mean 21,939, and the longest 127,292.

26 of the 480 generations hit the 131,072-token ceiling and are scored as failures. Twenty of them fall on the two hardest problems of AIME I. On those, the token budget ends the run before the reasoning does.

A published accuracy figure cannot show you this. NVIDIA report one AIME number for each of their three checkpoints and no truncation rate, so there is no way to tell from outside whether their figures carry the same penalty ours does. If they evaluated at the budget their own example sets, and these problems overrun it, their BF16 result carries the same penalty ours does, and so does every quantized build they published at that budget. A truncation rate resolves the question in one line, and it is free to compute. We report ours above.

Why the mid-tier stays at 8 bits

The obvious question about the recipe is why the shared experts and Mamba projections sit at INT8 and not INT4. That would save 3 GB, which on a 24 GB card is the scarcest thing you have: it's what the KV tradeoffs elsewhere on this model card speak to.

So we built both variants and put them through the identical harness: same 30 problems, same 131,072-token budget, same 16 samples, one variable changed per rung.

Build	Shared experts	Mamba in/out	Size	AIME25
this build	INT8	INT8	44.5 GB	91.7
variant	INT4	INT8	43.7 GB	90.2
variant	INT4	INT4	41.5 GB	89.6

Those raw scores fall monotonically, but comparing them directly is the wrong test: a single avg@16 run carries a 95% confidence interval about 15 points wide, so a 2-point gap between two of them is noise. Every build answers the same 30 problems, so the correct comparison is paired, problem by problem, which cancels the difficulty variance that makes AIME so unstable.

  Paired difference vs this build (same 30 problems, 95% CI)

           [--------------*---------+--]          shared W4, mamba W8  (43.7 GB)
       [--------------*-------------+]            shared W4, mamba W4  (41.5 GB)
    |     |      |     |     |      |     |     
    -5    -4     -3    -2    -1     0     +1      points of AIME25

  *  point estimate      [ ]  95% CI      +  zero

Zero is inside both intervals. Neither drop is measurable at this sample count, so we cannot put a number on what the mid-tier costs, and we do not.

What it does establish is that none of the three is broken. The most aggressive build, with every quantizable tier at 4 bits, still lands at 89.6, with NVIDIA's published 89.7 inside its confidence interval. The point estimates lean negative, though not significantly. We have not measured these tiers against a benchmark that could resolve the difference, so this release keeps the 8-bit tiers and the 3 GB they cost.

What this cannot see. AIME is long-chain mathematical reasoning, and it is a poor instrument for damage that surfaces elsewhere: aggregation over a long context (RULER's cwe subtask is the worst-recovering row in the published W4A16 literature, and needle-in-a-haystack is structurally blind to it), code generation, or multi-turn tool use. Those probe the attention path and the routed experts in ways this benchmark does not. Settling the tier question itself would take avg@64 rather than avg@16: four times the compute.

Agentic tool use: 30/30

This is a high-level tool-calling floor test, a BFCL run will replace it.

The model is dropped into a small project where a test is failing, and left to work out the rest: explore with tools, run the suite, locate the bug, patch the source, get back to green. Ten independent runs on each of three tasks. Floor test, not a capability verification yet

Task	Bug class	Passed	Tool calls	Latency
fix_merge_intervals	wrong comparison operator	10/10	6.5	13.4 s
fix_lru_cache	a missing operation (get never refreshes recency)	10/10	6.2	22.9 s
fix_retry_backoff	control flow, spread across two files	10/10	6.8	25.4 s

All thirty runs passed. This is our own harness, not a standard benchmark: three tasks, ten runs each, so read it as no failures observed rather than proof it cannot fail. The tasks were checked beforehand, to be sure a correct patch passes and an idle agent does not.

Needle in a haystack: 21/21

This is a high-level retrieval floor test, a RULER run will replace it.

A single fact is planted in a haystack of real Python source, and the model has to pull it back out. The depths come from the model's own tokenizer rather than a chars-per-token guess, and at each one the fact is planted early, halfway, and late: seven depths, three positions, 21 trials. At the deepest setting the fact sits ~250K tokens behind the question, the far edge of what the model was trained for.

Floor test, not a capability verification yet. What this shows is that quantization did not damage the model's ability to address its full context window. We don't yet claim to have evidence that the model reasons well across it.

Depth	4K	16K	32K	64K	128K	192K	250K
Fact found	3/3	3/3	3/3	3/3	3/3	3/3	3/3

What we haven't measured

• No baseline re-run on this hardware. The AIME comparison above is against NVIDIA's published figures on their harness, not against the base model re-measured here, and a cross-harness comparison is a real caveat. It is not for want of trying: on Ampere their FP8 won't load and their NVFP4 falls back to bf16 activations, the same weight-only regime as this build, documented under the recipe.
• Long-context fidelity beyond retrieval. Aggregation across the window, code, and multi-turn tool use are not yet fully tested, and they are the regimes where W4A16 is documented to break, so more benchmarks would add color.
• Multilingual. Seven languages are tagged, but the individual quality of each is as yet untested. Non-English is the axis where 4-bit quantization is reported to degrade most, and because that effect tracks pretraining representation rather than calibration, a data-free recipe gives you no protection from it.
• Reasoning modes. Agentic results are reasoning-on, but retrieval is reasoning-off, for the parser reason under Limitations.

---

Limitations

• Give the model room. Its chains outrun even NVIDIA's own suggested evaluation budget: their reproducibility example sets max_new_tokens=131072, and 26 of our 480 AIME generations still hit that ceiling and are scored as failures. Cap it lower and you do not get a worse answer, you get no answer. At a 16,000-token cap, an earlier run of this benchmark returned 147 correct answers and not one wrong one across 241 generations: every single failure was a truncation. A low cap does not measure the model, it measures your configuration.
• 24 GB cards require --gpu-memory-utilization 0.975. The vLLM default does not fit.
• A turn whose output has no closing </think> comes back empty. This happens when generation is truncated at max_tokens mid-reasoning, and sometimes at long context where the model does not emit the closing tag - either way both qwen3 and nemotron_v3 route the whole output to the reasoning field and leave content empty. With nemotron_v3, pass "force_nonempty_content": true in chat_template_kwargs and the undelimited text goes to content instead of being dropped. You can also set "enable_thinking": false for long contexts, or serve without --reasoning-parser. The trigger is the parser meeting an unterminated <think>, not the weights, so it is not a quantization artifact.
• Not directly comparable to NVIDIA's FP8/NVFP4. Those quantize activations as well as weights, and target Hopper and Blackwell. This is a weight-only build on Ampere: different regime, different silicon. See the recipe for why that runs in our favour rather than against us.
• int4 KV cache is unvalidated. fp8 is what we measured and what we recommend.
• Multi-Token Prediction is not shipped. The base model carries an MTP draft block, and NVIDIA's own quants keep it. This build drops it: we measured it first, and at k=3 it produced 85.3 tok/s against the same ~85 tok/s without it, so it was not paying for the 5.9 GB. That was measured on 48 GB cards with room to spare, and the calculus may be different on a 24 GB rig where the memory is scarcer than the bandwidth. Worth another look.

---

Custom code provenance (trust_remote_code)

The Python files here are from NVIDIA, but with a small AI-assisted code change: modeling_nemotron_h.py needed small fixes for loading transformers. vLLM runs its own in-tree implementation and never touches these files, so this only matters if you load through transformers.

---

Provenance and license

• Base model: nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16, which NVIDIA documents as derived from Nemotron-3-Super-120B-A12B (120.7B total, 12.8B active) via their "Iterative Puzzle" compression framework. Lineage and architecture per NVIDIA's model card and paper (arXiv:2607.04371). Parent model: arXiv:2604.12374.
• License: OpenMDW-1.1, the same license as the base model. This is a quantized derivative of NVIDIA's checkpoint; all rights and obligations of the base model's license apply.
• Repository: danielrmay/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-W4A16
• Quantized by Daniel May. Not affiliated with or endorsed by NVIDIA.