yugeshkarunamurthy/Qwythos-9B-v2-oQe6-MTP

Qwythos-9B-oQe6

Apple Silicon Optimized oQe6 MLX Quantized Release

This repository contains an oQe6 mixed-precision MLX quantized version of Qwythos-9B, optimized for efficient local inference on Apple Silicon devices.

The original Qwythos-9B model was developed by Empero AI. This repository contains an optimized MLX/oQe6 conversion only—no additional fine-tuning or retraining has been performed.

---

About Qwythos

Qwythos-9B is a full-parameter reasoning model built upon Qwen3.5-9B and trained on over 500 million tokens of carefully curated reasoning data.

The original model specializes in:

• 🧠 Advanced reasoning
• 💻 Programming
• 🛠 Native function calling
• 🤖 Tool use
• 🔐 Cybersecurity
• 🧬 Biomedical reasoning
• ➗ Mathematics
• 🔬 Scientific reasoning
• 📚 Long-context agent workflows

Key capabilities include:

• 1,048,576 token context window
• Native function calling
• Excellent coding performance
• Strong mathematical reasoning
• Tool-assisted self-correction
• Long-context understanding
• Uncensored technical reasoning

For complete benchmark results, training methodology, and evaluation details, please visit the original repository:

https://huggingface.co/empero-ai/Qwythos-9B-v2

---

Quantization

This release uses oQe6 mixed-precision quantization.

Specifications

• Format: MLX
• Quantization: oQe6
• Method: Sensitivity-Aware Mixed Precision
• Target Platform: Apple Silicon
• Inference: MLX / oMLX

Unlike traditional uniform quantization, oQ dynamically allocates precision according to layer sensitivity, preserving higher precision for the most important weights while aggressively compressing less sensitive regions.

This provides an excellent balance between:

• Higher reasoning quality
• Better coding performance
• Lower memory usage
• Faster inference
• Excellent Apple Silicon efficiency

---

Recommended Settings

For the best reasoning performance:

temp: 0.6
top_p: 0.95
top_k: 20
min_p: 0
rep_penalty: 1.05
presence_penalty: 1.5
enable_thinking: true

These settings provide excellent performance across:

• Reasoning
• Mathematics
• Programming
• Tool Use
• Scientific Questions
• Agent Workflows

---

Example Usage

from mlx_lm import load, generate
model, tokenizer = load("YOUR_USERNAME/Qwythos-9B-oQe6")
messages = [
    {
        "role": "user",
        "content": "Explain speculative decoding."
    }
]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)
response = generate(
    model,
    tokenizer,
    prompt=prompt,
    temp=0.6,
    top_p=0.95,
    top_k=20,
    max_tokens=16384,
)
print(response)

---

Optimized For

This release is optimized for:

• Apple M1
• Apple M1 Pro / Max / Ultra
• Apple M2 Series
• Apple M3 Series
• Apple M4 Series

Compatible with:

• MLX
• oMLX
• Open WebUI
• LM Studio (MLX)
• MLX-LM
• Local AI Applications

---

Intended Use

Qwythos-9B-oQe6 is well suited for:

• Software Engineering
• AI Coding Assistants
• Long Context Analysis
• Scientific Research
• Mathematical Reasoning
• Cybersecurity
• Biomedical Analysis
• Local AI Agents
• Tool Calling Applications
• Research & Education

---

Hardware Recommendations

Recommended systems:

• Apple M1 Pro / Max / Ultra
• Apple M2 Pro / Max / Ultra
• Apple M3 Series
• Apple M4 Series

Higher-memory configurations are recommended when utilizing the full 1M context window.

---

About oQ Quantization

oQ is a sensitivity-aware mixed-precision quantization technique designed to maximize model quality while significantly reducing memory usage.

Instead of quantizing every layer identically, oQ analyzes layer importance and preserves additional precision where it matters most.

Benefits include:

• Better reasoning retention
• Improved coding performance
• Higher mathematical accuracy
• Lower memory usage
• Faster inference
• Excellent Apple Silicon optimization

---

Credits

Original Model

All credit for the original model, datasets, training methodology, evaluation, benchmarks, and research belongs entirely to:

Empero AI

Original Repository:

https://huggingface.co/empero-ai/Qwythos-9B-v2

Base Model:

https://huggingface.co/Qwen/Qwen3.5-9B

---

oQ4 MLX Quantized Release

This repository provides an Apple Silicon optimized oQe6 MLX quantized version of the original model.

No additional fine-tuning has been performed.

---

Acknowledgements

• Empero AI
• Alibaba Qwen Team
• Apple MLX
• Hugging Face
• Transformers
• TRL
• EleutherAI
• oMLX
• OptiQ Quantization

---

Citation

If you use this model in research, please cite the original Qwythos-9B model and the Qwen3.5 base model.

---

License

This release inherits the Apache-2.0 license from the original model.

Please refer to the original repository for complete licensing information.

---

Disclaimer

This repository contains an optimized oQe6 MLX quantized conversion intended for efficient local inference on Apple Silicon devices.

All original model architecture, datasets, training, benchmarks, evaluations, and research remain entirely the work of the original authors.