Initial commit

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,159 @@
+---
+license: apache-2.0
+language:
+ - en
+ - multilingual
+tags:
+ - mlx
+ - mlx-lm
+ - qwen
+ - qwen3
+ - qwen3.6
+ - abliterated
+ - refusal-ablated
+ - reasoning
+ - claude-opus-distill
+ - text-generation
+ - apple-silicon
+library_name: mlx
+pipeline_tag: text-generation
+base_model: TeichAI/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2
+base_model_relation: quantized
+---
+
+<p align="center">
+  <img src="logo.png" alt="Lemura Labs" width="110"/>
+</p>
+
+<p align="center">
+
+</p>
+
+# Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2-abliterated-OptiQ-3.7bpw-mlx
+
+![Format](https://img.shields.io/badge/Format-MLX-0b7285?style=flat) ![Task](https://img.shields.io/badge/Task-Text%20Generation-5f3dc4?style=flat) ![Params](https://img.shields.io/badge/Params-27B-1864ab?style=flat) ![Type](https://img.shields.io/badge/Type-Abliterated-862e9c?style=flat) ![BPW](https://img.shields.io/badge/BPW-3.7-1f6feb?style=flat) ![Size](https://img.shields.io/badge/Size-12%20GB-495057?style=flat) ![Context](https://img.shields.io/badge/Context-262K-0c8599?style=flat) ![License](https://img.shields.io/badge/License-apache--2.0-6c757d?style=flat)
+
+
+> ## THIS IS A TEXT-ONLY MODEL — NO VISION
+> The upstream abliteration pass stripped the vision tower. **For vision-capable Qwen 3.6 27B Opus-Distill MLX quants, see our parallel repos** at [huggingface.co/lemuralabs](https://huggingface.co/lemuralabs) (look for repos without `-abliterated` in the name).
+
+**OptiQ static mixed ~3.7 BPW MLX quantization** of an abliterated Qwen 3.6 27B Claude-Opus reasoning distill, by the Lemura Labs team.
+
+> The smallest of our line. Uses [mlx-optiq](https://github.com/dc-dc-dc/mlx-optiq) sensitivity scans (forward-only KL divergence) to assign 2-bit to layers that tolerate it and 4-6-bit to the layers that don't, averaging ~3.7 bits per weight. Best size/quality ratio for tight RAM budgets.
+
+---
+
+## TL;DR
+
+| | |
+|---|---|
+| **Disk size** | ~12 GB |
+| **Effective BPW** | 3.7 |
+| **Scheme** | OptiQ static-mixed (sensitivity-ranked, KL-driven assignment) |
+| **Recommended RAM** | 16 GB Apple Silicon (M1/M2/M3/M4 Air, mini, base Pro) |
+| **Vision** | No — text-only (the upstream abliteration step stripped the ViT) |
+| **Made by** | Lemura Labs |
+
+---
+
+## Lineage
+
+```
+Qwen/Qwen3.6-27B (Qwen Team — base pretrain)
+ │
+ ▼
+TeichAI/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2 (TeichAI — Claude-Opus reasoning distill)
+ │
+ ▼
+abliterated (refusal-ablated) via OBLITERATUS v0.1.2 (multi-direction SVD, BF16)
+ │
+ ▼
+this repo — OptiQ static mixed ~3.7 BPW, MLX format (Lemura Labs team — quantization)
+```
+
+Direct upstream links:
+
+- Foundation: [Qwen/Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B)
+- Reasoning distill: [TeichAI/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2](https://huggingface.co/TeichAI/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2)
+- Abliteration tool: [OBLITERATUS](https://github.com/elder-plinius/OBLITERATUS) (multi-direction SVD, 6 directions, 3 refinement passes, λ=0.08)
+- Quantization tool: [mlx-lm](https://github.com/ml-explore/mlx-lm) + [mlx-optiq](https://github.com/dc-dc-dc/mlx-optiq) for OptiQ variants
+
+---
+
+## Use it
+
+### `mlx-lm` (recommended)
+
+```bash
+pip install mlx-lm
+```
+
+```python
+from mlx_lm import load, generate
+
+model, tokenizer = load("lemuralabs/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2-abliterated-OptiQ-3.7bpw-mlx")
+prompt = "Explain the difference between SSM and softmax attention in three sentences."
+out = generate(model, tokenizer, prompt=prompt, max_tokens=400)
+print(out)
+```
+
+### Chat template
+
+```python
+messages = [
+ {"role": "system", "content": "You are a helpful, candid reasoning assistant."},
+ {"role": "user", "content": "Plan a 3-day Tokyo itinerary for a foodie."},
+]
+prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
+print(generate(model, tokenizer, prompt=prompt, max_tokens=600))
+```
+
+### CLI
+
+```bash
+mlx_lm.generate --model lemuralabs/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2-abliterated-OptiQ-3.7bpw-mlx --prompt "Hello" --max-tokens 256
+```
+
+---
+
+## Quantization details
+
+- **Source weights**: BF16 abliterated checkpoint (28 shards, ~57 GB) derived from [TeichAI/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2](https://huggingface.co/TeichAI/Qwen3.6-27B-Claude-Opus-Reasoning-Distill-v2) via OBLITERATUS multi-direction SVD ablation (preserves coherence; KL drift = 0.149 from base).
+- **Quantization scheme**: OptiQ static-mixed (sensitivity-ranked, KL-driven assignment).
+- **Group size**: 64.
+- **Calibration corpus**: `mlx-lm` calibration_v5 (~427 KB English text, used for OptiQ sensitivity ranking; uniform/affine variants do not require calibration).
+- **Sanity check**: forward perplexity on held-out calibration text within 1–3% of next-higher-precision sibling.
+
+### Architecture notes
+
+The Qwen 3.6 27B family uses a **hybrid attention stack** — 4 GatedDeltaNet (linear-attention/SSM) layers followed by 1 full-softmax-attention layer, repeated 16× for 64 total layers, 5120 hidden, 248K vocab, 262K context. The SSM kernels lack a VJP path in MLX, so backward-pass-based quant methods (DWQ, dynamic quant) **cannot** be applied here — OptiQ's forward-only sensitivity approach is the only calibration-aware option that works on this architecture. That's why the OptiQ variants exist.
+
+---
+
+## Behavior caveats
+
+- **Text-only — no vision.** The abliteration pipeline (OBLITERATUS) ran on the LM tower and stripped the ViT. For vision-capable quants of the same Opus-Distill v2 lineage, use our parallel non-abliterated repos at [huggingface.co/lemuralabs](https://huggingface.co/lemuralabs) (any repo without `-abliterated` in the name).
+- This is an **abliterated** model — refusal directions were surgically removed from the parent. It will answer prompts the parent would refuse. Use responsibly and within applicable law.
+- Quantization preserves abliteration: the refusal rate measured at BF16 (~35% from a 100% baseline) stays in that range across our quants.
+
+---
+
+## Credits
+
+| | |
+|---|---|
+| **Quantization & release** | Lemura Labs |
+| **Reasoning distill** | [TeichAI](https://huggingface.co/TeichAI) (Claude-Opus 4.5/4.6 high-reasoning datasets) |
+| **Foundation model** | [Qwen Team](https://huggingface.co/Qwen) |
+| **Abliteration toolkit** | [OBLITERATUS](https://github.com/elder-plinius/OBLITERATUS) by elder-plinius |
+| **Quant toolkit** | [mlx-lm](https://github.com/ml-explore/mlx-lm), [mlx-optiq](https://github.com/dc-dc-dc/mlx-optiq) |
+
+---
+
+## License
+
+Apache-2.0, inherited from the foundation and distill upstream.
+
+---
+
+<sub>Need a hosted endpoint, custom quant, or larger-scale inference? — multi-provider LLM routing for the Indian developer ecosystem.</sub>

chat_template.jinja

@@ -0,0 +1,154 @@
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain images.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set image_count.value = image_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+            {%- elif 'video' in item or item.type == 'video' %}
+                {%- if is_system_content %}
+                    {{- raise_exception('System message cannot contain videos.') }}
+                {%- endif %}
+                {%- if do_vision_count %}
+                    {%- set video_count.value = video_count.value + 1 %}
+                {%- endif %}
+                {%- if add_vision_id %}
+                    {{- 'Video ' ~ video_count.value ~ ': ' }}
+                {%- endif %}
+                {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+            {%- elif 'text' in item %}
+                {{- item.text }}
+            {%- else %}
+                {{- raise_exception('Unexpected item type in content.') }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- if tools and tools is iterable and tools is not mapping %}
+    {{- '<|im_start|>system\n' }}
+    {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>" }}
+    {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {%- if content %}
+            {{- '\n\n' + content }}
+        {%- endif %}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if messages[0].role == 'system' %}
+        {%- set content = render_content(messages[0].content, false, true)|trim %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}
+{%- for message in messages[::-1] %}
+    {%- set index = (messages|length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == "user" %}
+        {%- set content = render_content(message.content, false)|trim %}
+        {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {{- raise_exception('No user query found in messages.') }}
+{%- endif %}
+{%- for message in messages %}
+    {%- set content = render_content(message.content, true)|trim %}
+    {%- if message.role == "system" %}
+        {%- if not loop.first %}
+            {{- raise_exception('System message must be at the beginning.') }}
+        {%- endif %}
+    {%- elif message.role == "user" %}
+        {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning_content is string %}
+            {%- set reasoning_content = message.reasoning_content %}
+        {%- else %}
+            {%- if '</think>' in content %}
+                {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+                {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content|trim %}
+        {%- if (preserve_thinking is defined and preserve_thinking is true) or (loop.index0 > ns.last_query_index) %}
+            {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>' + message.role + '\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined %}
+                    {%- set tool_call = tool_call.function %}
+                {%- endif %}
+                {%- if loop.first %}
+                    {%- if content|trim %}
+                        {{- '\n\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- else %}
+                        {{- '<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                    {%- endif %}
+                {%- else %}
+                    {{- '\n<tool_call>\n<function=' + tool_call.name + '>\n' }}
+                {%- endif %}
+                {%- if tool_call.arguments is defined %}
+                    {%- for args_name, args_value in tool_call.arguments|items %}
+                        {{- '<parameter=' + args_name + '>\n' }}
+                        {%- set args_value = args_value | string if args_value is string else args_value | tojson | safe %}
+                        {{- args_value }}
+                        {{- '\n</parameter>\n' }}
+                    {%- endfor %}
+                {%- endif %}
+                {{- '</function>\n</tool_call>' }}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if loop.previtem and loop.previtem.role != "tool" %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- content }}
+        {{- '\n</tool_response>' }}
+        {%- if not loop.last and loop.nextitem.role != "tool" %}
+            {{- '<|im_end|>\n' }}
+        {%- elif loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- else %}
+        {{- raise_exception('Unexpected message role.') }}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if enable_thinking is defined and enable_thinking is false %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 248044,
+  "do_sample": true,
+  "eos_token_id": [
+    248046,
+    248044
+  ],
+  "pad_token_id": 248044,
+  "temperature": 1.0,
+  "top_k": 20,
+  "top_p": 0.95,
+  "transformers_version": "5.7.0.dev0"
+}

model-00001-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d2067a0bc8995d94734f3083ee895069f642b9c19bf6e1deb90656ef0300eb62
+size 5362760590

model-00002-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5965567e0f4afbfb710fbd64e94fdb5e0747d883003d4fc47e827db9b39c9e0e
+size 5368778955

model-00003-of-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fb6d31c08e134c34a8e65f9deb360e9cd2efdf2057fc2f93455ef96f182309b2
+size 2601885556

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523
+size 19989325

tokenizer_config.json

@@ -0,0 +1,41 @@
+{
+  "add_prefix_space": false,
+  "audio_bos_token": "<|audio_start|>",
+  "audio_eos_token": "<|audio_end|>",
+  "audio_token": "<|audio_pad|>",
+  "backend": "tokenizers",
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "image_token": "<|image_pad|>",
+  "is_local": true,
+  "local_files_only": false,
+  "max_length": 256,
+  "model_max_length": 262144,
+  "model_specific_special_tokens": {
+    "audio_bos_token": "<|audio_start|>",
+    "audio_eos_token": "<|audio_end|>",
+    "audio_token": "<|audio_pad|>",
+    "image_token": "<|image_pad|>",
+    "video_token": "<|video_pad|>",
+    "vision_bos_token": "<|vision_start|>",
+    "vision_eos_token": "<|vision_end|>"
+  },
+  "pad_to_multiple_of": null,
+  "pad_token": "<|endoftext|>",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "pretokenize_regex": "(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\\r\\n\\p{L}\\p{N}]?[\\p{L}\\p{M}]+|\\p{N}| ?[^\\s\\p{L}\\p{M}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+  "processor_class": "Qwen3VLProcessor",
+  "split_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "TokenizersBackend",
+  "tool_parser_type": "qwen3_coder",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": null,
+  "video_token": "<|video_pad|>",
+  "vision_bos_token": "<|vision_start|>",
+  "vision_eos_token": "<|vision_end|>"
+}