NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-qx86-hi-mlx

Brainwaves

         arc   arc/e boolq hswag obkqa piqa  wino
mxfp8    0.503,0.723,0.852,0.685,0.424,0.789,0.657
qx86-hi  0.531,0.746,0.861,0.692,0.424,0.797,0.650
qx64-hi  0.511,0.719,0.858,0.689,0.422,0.795,0.660
mxfp4    0.513,0.699,0.859,0.682,0.432,0.791,0.662

Quant    Perplexity      Peak Memory   Tokens/sec
mxfp8    5.131 ± 0.038   36.77 GB      1289
qx86-hi  4.810 ± 0.035   31.95 GB      1264
qx64-hi  4.889 ± 0.036   24.06 GB      1430
mxfp4    5.222 ± 0.039   20.98 GB      1310

This model is using the Qwen3.6 fixed jinja template from froggeric/Qwen-Fixed-Chat-Templates

The original jinja template can be found as chat_template_default.jinja in the model folder.

Thinking toggle

Drop <|think_on|> or <|think_off|> anywhere in your system or user prompt. The template intercepts the tag, removes it from context so the model never sees it, and flips the mode.

Fast answer, no reasoning:

System: You are a coding assistant. <|think_off|>
User: What's 2+2?

Deep reasoning:

System: You are a coding assistant. <|think_on|>
User: Implement a red-black tree in Rust.

The tag syntax (<|think_on|>, <|think_off|>) uses Qwen's control-token delimiters, so it will never collide with real text. Earlier community templates used /think, which broke legitimate paths like cd /mnt/project/think.

---

I added a similar set of tags for handling the preserve_thinking flag:

• Drop <|think_forget|> or <|think_remember|> anywhere in your system or user prompt to flip the flag.
• The template intercepts the tag, removes it from context so the model never sees it, and flips the mode.

I don't know if this works with Nemotron, feel free to experiment.

-G

This model NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-qx86-hi-mlx was converted to MLX format from nvidia/NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-BF16 using mlx-lm version 0.31.3.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("NVIDIA-Nemotron-Labs-3-Elastic-30B-A3B-qx86-hi-mlx")

prompt = "hello"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=True)