Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,111 @@
 ---
-license: mit
+license: apache-2.0
+library_name: mlx
+tags: [mlx, inkling, moe, multimodal, text-generation, image-text-to-text, apple-silicon]
+base_model: thinkingmachines/Inkling-Small
+pipeline_tag: image-text-to-text
 ---
+
+# Inkling-Small-mlx-3bit
+
+An **MLX 3bit** build of [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small)
+— 264 B total parameters, **~12 B active**; 42 layers, 256 routed experts (top-6) + 2 shared;
+natively multimodal (image + audio in, text out).
+
+Sized to run **fully resident on a 128 GB Mac** — no expert-offload, no expert pruning, no
+layer streaming. The whole model sits in unified memory and decodes at conversational speed.
+
+> Why this model is the interesting one for Apple Silicon: speed follows the **active**
+> parameter count, not how hard you compress. Inkling-Small activates ~12 B params per token,
+> so this tier reads roughly 9.8 GB per token. The 975 B Inkling, by contrast, needs SSD
+> expert-offload even at 2-bit and manages ~0.2–0.4 tok/s.
+
+## Tiers
+
+Pick the tier that fits your machine. Sizes are on-disk and **measured from the built
+artifacts**; peak/speed columns are filled in only where a benchmark has actually run.
+
+| tier | target Mac | recipe | on-disk | peak | load | prefill tok/s |
+|---|---|---|---|---|---|---|
+| [4bit](https://huggingface.co/mlx-community/Inkling-Small-mlx-4bit) | 192 GB | experts 4-bit, non-expert 8-bit | 153.5 GB | — | — | — |
+| **3bit** ← **this repo** | 128 GB | experts 3-bit, non-expert 8-bit | 120.9 GB | — | — | — |
+| [2bit](https://huggingface.co/mlx-community/Inkling-Small-mlx-2bit) | 96 GB | experts 2-bit, non-expert 8-bit | 88.4 GB | — | — | — |
+
+> **Speeds are not yet measured.** On-disk sizes above are real (taken from the built artifacts), but load/peak/tok-s columns stay empty until `serving/bench_mlx.py` has run on the target hardware — we would rather ship a blank column than a guess. Run it yourself with the command below and please open a discussion with your numbers.
+
+## Why MLX on Apple Silicon
+
+MLX is Apple's array framework for Apple Silicon. For a big sparse MoE like this one, the
+practical differences from a CPU/GPU-split runtime are:
+
+| | what it means here |
+|---|---|
+| **Unified memory** | The GPU reads the same DRAM as the CPU, so a 120 GB build needs 120 GB of *system* memory — not VRAM plus a host copy. There is no PCIe transfer per layer, which is what makes a >100 GB model practical on a desktop at all. |
+| **Lazy, mmap'd loading** | `mx.load` memory-maps safetensors, so weights page in on demand instead of being read and copied up front. |
+| **Native quantized matmul** | Quantized weights are multiplied in their packed form (`quantized_matmul`, and `gather_qmm` for MoE expert gathers) rather than dequantized to fp16 first — so low-bit tiers save bandwidth at *runtime*, not just on disk. |
+| **Per-module precision** | `nn.quantize(..., class_predicate=...)` lets one checkpoint mix bit-widths per tensor class, which is exactly how these tiers keep attention and the router high-precision while the experts go low. |
+| **Small dependency surface** | `pip install mlx mlx-lm` and a single Python model file. No compile step, no separate server binary. |
+
+Honest limits: MLX's ecosystem is younger than llama.cpp's, it has no importance-matrix
+("i-quant") style calibrated quantization yet, and it runs on Apple Silicon only. If you want a
+GGUF build instead, one exists at
+[`unsloth/Inkling-Small-GGUF`](https://huggingface.co/unsloth/Inkling-Small-GGUF).
+
+## Memory notes
+
+`peak` (once measured) is peak unified-memory use, not file size — leave headroom for the KV
+cache and the OS. Two things matter on a machine near its limit:
+
+* **Raise the Metal wired limit.** It defaults to ~75% of RAM, which is below this tier's
+  footprint on a 128 GB machine:
+  ```bash
+  sudo sysctl iogpu.wired_limit_mb=180000   # ~180 GB on a 192 GB Mac; scale to your RAM
+  ```
+* **macOS will kill a process that stays too large for too long** (jetsam), even while memory
+  pressure looks fine. If long generations die silently, move down a tier.
+
+## Quantization recipe
+
+`experts_only`: the routed experts (and the vision/audio matmuls) carry the tier's bit-width,
+while **attention, token/output embeddings, RMSNorms, the router gate, the per-layer
+short-convolutions and the relative-position bias stay high-precision (8-bit)**.
+
+That asymmetry is the point, and it is nearly free: non-expert weights are only **5.9 B of the
+264 B params (~6 GB at 8-bit)**, yet they are read **in full on every token**, while just 6 of
+256 experts are. Bits spent there cost ~2% of the file and protect the paths every token
+depends on. Expert precision degrades *gracefully*; attention and router precision degrade
+*globally*.
+
+## Usage
+
+```python
+# pip install mlx mlx-lm transformers        (+ scipy for audio, pillow for images)
+# this repo bundles the inkling_mlx/ loader, so there is nothing else to install
+from inkling_mlx.load import load
+from inkling_mlx.generate import greedy_generate
+from transformers import AutoTokenizer
+
+path = "./Inkling-Small-mlx-3bit"
+model, config = load(path)
+tok = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
+ids = tok("The capital of France is")["input_ids"]
+print(tok.decode(greedy_generate(model, config, ids, max_new_tokens=64)))
+```
+
+Images are cut into 40 px patches (one soft-token each) and audio is d-mel encoded at 20
+tokens/s; both go through `InklingProcessor`. See the source repo for a multimodal runner.
+
+### Benchmark it yourself
+
+```bash
+python serving/bench_mlx.py --model ./Inkling-Small-mlx-3bit --tier 3bit
+```
+
+Reports load time, prefill tok/s, decode tok/s at several context lengths, and peak memory.
+
+## Attribution
+
+- Base model: [`thinkingmachines/Inkling-Small`](https://huggingface.co/thinkingmachines/Inkling-Small), Apache-2.0.
+- The bundled `inkling_mlx/` loader is vendored from
+  [PipeNetwork/inkling-mlx](https://github.com/PipeNetwork/inkling-mlx), Apache-2.0.
+- Quantized with `mlx`; see the recipe above for exactly which tensors were touched.

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,7 @@
+# Third-party notices
+
+This project vendors the `inkling_mlx/` model package from
+**PipeNetwork/inkling-mlx** (https://github.com/PipeNetwork/inkling-mlx), licensed under **Apache-2.0**
+(Copyright 2026 PipeNetwork). Vendored commit: `b6dc0ac9219b33e4e667bf08cafcdc2651fd9884`.
+The package's LICENSE is preserved at `inkling_mlx/LICENSE`. Our conversion
+tooling (convert/, models/, serving/, publish/) is separate.

chat_template.jinja

@@ -0,0 +1,129 @@
+{%- set effort_map = {"none": 0.0, "minimal": 0.1, "low": 0.2, "medium": 0.7, "high": 0.9, "max": 0.99} -%}
+{%- set role_token = {"user": "<|message_user|>", "assistant": "<|message_model|>", "system": "<|message_system|>", "tool": "<|message_tool|>"} -%}
+
+{%- macro emit_thinking_effort() -%}
+    {%- set eff = reasoning_effort if reasoning_effort is defined and reasoning_effort is not none else 0.9 -%}
+    {%- if eff is string -%}
+        {%- set key = eff | trim -%}
+        {%- if key not in effort_map -%}
+            {{- raise_exception("Unknown reasoning_effort: " ~ eff) -}}
+        {%- endif -%}
+        {%- set num = effort_map[key] -%}
+    {%- else -%}
+        {%- set num = eff | float -%}
+    {%- endif -%}
+    {%- if num < 0.0 or num > 0.99 -%}
+        {{- raise_exception("reasoning_effort must be in [0.0, 0.99]") -}}
+    {%- endif -%}
+    {{- "<|message_system|><|content_text|>Thinking effort level: " -}}
+    {%- if num == 0.0 -%}0{%- else -%}{{ num }}{%- endif -%}
+    {{- "<|end_message|>" -}}
+{%- endmacro -%}
+
+{%- if tools -%}
+    {%- set tool_state = namespace(specs=[]) -%}
+    {%- for tool in tools -%}
+        {%- set fn = tool.function if tool.function is defined else tool -%}
+        {%- set spec = {
+            "description": (fn.description if fn.description is defined and fn.description else ""),
+            "name": fn.name,
+            "parameters": (fn.parameters if fn.parameters is defined and fn.parameters else {}),
+            "type": (tool.type if tool.type is defined and tool.type else "function"),
+        } -%}
+        {%- set tool_state.specs = tool_state.specs + [spec] -%}
+    {%- endfor -%}
+    {{- "<|message_system|>tool_declare<|content_xml|>" -}}
+    {{- tool_state.specs | tojson(sort_keys=true, separators=(",", ":")) -}}
+    {{- "<|end_message|>" -}}
+{%- endif -%}
+
+{%- set state = namespace(effort_emitted=false) -%}
+{%- for message in messages -%}
+    {%- if message.role not in role_token -%}
+        {{- raise_exception("Unknown message role: " ~ message.role) -}}
+    {%- endif -%}
+    {%- if not state.effort_emitted and message.role != "system" -%}
+        {{- emit_thinking_effort() -}}
+        {%- set state.effort_emitted = true -%}
+    {%- endif -%}
+
+    {%- set rtok = role_token[message.role] -%}
+
+    {%- if message.role == "tool" -%}
+        {%- set tool_name_state = namespace(name="") -%}
+        {%- if message.name is defined and message.name -%}
+            {%- set tool_name_state.name = message.name -%}
+        {%- elif message.tool_call_id is defined and message.tool_call_id -%}
+            {%- for prev in messages -%}
+                {%- if prev.role == "assistant" and prev.tool_calls -%}
+                    {%- for tc in prev.tool_calls -%}
+                        {%- if tc.id is defined and tc.id == message.tool_call_id and tc.function.name is defined -%}
+                            {%- set tool_name_state.name = tc.function.name -%}
+                        {%- endif -%}
+                    {%- endfor -%}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+        {{- rtok -}}
+        {%- if tool_name_state.name -%}{{- tool_name_state.name -}}{%- endif -%}
+        {{- "<|content_text|>" -}}
+        {%- if message.content is string -%}{{- message.content -}}{%- endif -%}
+        {{- "<|end_message|>" -}}
+
+    {%- else -%}
+        {%- if message.role == "assistant" and message.reasoning_content is defined and message.reasoning_content -%}
+            {{- "<|message_model|><|content_thinking|>" ~ message.reasoning_content ~ "<|end_message|>" -}}
+        {%- endif -%}
+
+        {%- if message.content is string -%}
+            {{- rtok ~ "<|content_text|>" ~ message.content ~ "<|end_message|>" -}}
+        {%- elif message.content -%}
+            {%- for part in message.content -%}
+                {%- if part is string -%}
+                    {{- rtok ~ "<|content_text|>" ~ part ~ "<|end_message|>" -}}
+                {%- elif part.type is not defined or part.type in ("text", "input_text") -%}
+                    {%- set text_part = (part.text if part.text is defined and part.text is string else "") -%}
+                    {{- rtok ~ "<|content_text|>" ~ text_part ~ "<|end_message|>" -}}
+                {%- elif part.type in ("image", "input_image", "image_url") -%}
+                    {{- rtok ~ "<|content_image|><|unused_200054|><|end_message|>" -}}
+                {%- elif part.type in ("audio", "input_audio", "audio_url") -%}
+                    {{- rtok ~ "<|content_audio_input|><|unused_200053|><|audio_end|><|end_message|>" -}}
+                {%- else -%}
+                    {{- raise_exception("Unsupported content part type: " ~ part.type) -}}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+
+        {%- if message.role == "assistant" and message.tool_calls -%}
+            {%- for tc in message.tool_calls -%}
+                {%- set fn = tc.function -%}
+                {%- if fn.name is not defined or fn.name is not string -%}
+                    {{- raise_exception("tool call function name must be a string") -}}
+                {%- endif -%}
+                {%- set args = fn.arguments if fn.arguments is defined and fn.arguments else {} -%}
+                {%- if args is string -%}
+                    {{- raise_exception("tool call arguments must be a parsed object, not a JSON string; canonicalize upstream") -}}
+                {%- endif -%}
+                {%- if args is not mapping -%}
+                    {{- raise_exception("tool call arguments must be an object") -}}
+                {%- endif -%}
+                {{- "<|message_model|>" ~ fn.name ~ "<|content_invoke_tool_json|>" -}}
+                {{- '{"name":' ~ (fn.name | tojson(sort_keys=true, separators=(",", ":"))) ~ ',"args":' -}}
+                {{- (args | tojson(sort_keys=true, separators=(",", ":"))) -}}
+                {{- "}<|end_message|>" -}}
+            {%- endfor -%}
+        {%- endif -%}
+
+        {%- if message.role == "assistant" -%}
+            {{- "<|content_model_end_sampling|>" -}}
+        {%- endif -%}
+    {%- endif -%}
+{%- endfor -%}
+
+{%- if not state.effort_emitted -%}
+    {{- emit_thinking_effort() -}}
+{%- endif -%}
+
+{%- if add_generation_prompt -%}
+    {{- "<|message_model|>" -}}
+{%- endif -%}

config.json

@@ -0,0 +1,119 @@
+{
+  "architectures": [
+    "InklingForConditionalGeneration"
+  ],
+  "model_type": "inkling_mm_model",
+  "eos_token_id": 200006,
+  "text_config": {
+    "model_max_length": 1048576,
+    "torch_dtype": "bfloat16",
+    "hidden_size": 4096,
+    "num_hidden_layers": 42,
+    "vocab_size": 201024,
+    "num_attention_heads": 32,
+    "num_key_value_heads": 8,
+    "head_dim": 128,
+    "d_rel": 16,
+    "rel_extent": 1024,
+    "q_bias": false,
+    "o_bias": false,
+    "log_scaling_n_floor": 128000,
+    "log_scaling_alpha": 0.1,
+    "rms_norm_eps": 1e-06,
+    "use_embed_norm": true,
+    "local_layer_ids": [
+      0,
+      1,
+      2,
+      3,
+      4,
+      6,
+      7,
+      8,
+      9,
+      10,
+      12,
+      13,
+      14,
+      15,
+      16,
+      18,
+      19,
+      20,
+      21,
+      22,
+      24,
+      25,
+      26,
+      27,
+      28,
+      30,
+      31,
+      32,
+      33,
+      34,
+      36,
+      37,
+      38,
+      39,
+      40
+    ],
+    "dense_mlp_idx": 2,
+    "use_sconv": true,
+    "sconv_kernel_size": 4,
+    "unpadded_vocab_size": 200058,
+    "logits_mup_width_multiplier": 16.0,
+    "final_logit_softcapping": null,
+    "swa_head_dim": 128,
+    "swa_num_attention_heads": 32,
+    "swa_num_key_value_heads": 8,
+    "sliding_window_size": 512,
+    "n_routed_experts": 256,
+    "num_experts_per_tok": 6,
+    "n_shared_experts": 2,
+    "shared_expert_sink": true,
+    "dense_intermediate_size": 16384,
+    "intermediate_size": 2048,
+    "route_scale": 8.0,
+    "use_gate_bias": true,
+    "gate_activation": "sigmoid",
+    "norm_after_topk": true,
+    "use_global_scale": true
+  },
+  "audio_config": {
+    "decoder_dmodel": 4096,
+    "n_mel_bins": 80,
+    "mel_vocab_size": 16,
+    "bias": false,
+    "dmel_min_value": -7.0,
+    "dmel_max_value": 2.0,
+    "use_audio_norm": true,
+    "audio_mode": "dmel"
+  },
+  "vision_config": {
+    "vision_encoder_type": "hmlp",
+    "decoder_dmodel": 4096,
+    "patch_size": 40,
+    "temporal_patch_size": 2,
+    "n_channels": 3,
+    "n_layers": 4,
+    "use_vision_norm": true
+  },
+  "mtp_config": {
+    "num_nextn_predict_layers": 8,
+    "chain_hidden_post_norm": false,
+    "local_layer_ids": [
+      0,
+      2,
+      4,
+      5,
+      6,
+      7
+    ]
+  },
+  "quantization": {
+    "group_size": 64,
+    "bits": 3,
+    "recipe": "experts_only"
+  }
+}

inkling_mlx/LICENSE

@@ -0,0 +1,13 @@
+Copyright 2026 PipeNetwork
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

inkling_mlx/__init__.py

@@ -0,0 +1,14 @@
+"""MLX port of thinkingmachines/Inkling (975B MoE, natively multimodal)."""
+
+from .config import AudioConfig, InklingConfig, TextConfig, VisionConfig
+from .model import InklingForConditionalGeneration
+from .text import TextModel
+
+__all__ = [
+    "InklingConfig",
+    "TextConfig",
+    "VisionConfig",
+    "AudioConfig",
+    "InklingForConditionalGeneration",
+    "TextModel",
+]

inkling_mlx/attention.py

@@ -0,0 +1,131 @@
+"""Inkling attention: hybrid local/global, per-head q/k RMSNorm, relative-position
+logits bias, optional log-scaling, and short-convolution on k/v.
+
+Mirrors ``InklingAttention`` + ``InklingRelativeLogits`` from transformers PR #47347.
+This implementation is prefill-oriented (full-sequence, no KV cache); an incremental
+cache (including the 4 per-layer conv states) can be layered on top later.
+"""
+
+from __future__ import annotations
+
+import math
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .common import RMSNorm, ShortConvolution
+from .config import TextConfig
+
+NEG_INF = -1e30
+
+
+class RelativeLogits(nn.Module):
+    """Hidden-state-conditioned relative position bias.
+
+    ``proj`` is a bank of bias-vs-distance profiles ``[d_rel, rel_extent]``. Each
+    query's ``d_rel`` relative-state vector mixes them into one bias value per
+    backward distance; the bias is zero outside ``0 <= distance < rel_extent``.
+    """
+
+    def __init__(self, d_rel: int, rel_extent: int):
+        super().__init__()
+        self.rel_extent = rel_extent
+        self.proj = mx.zeros((d_rel, rel_extent))
+
+    def __call__(self, relative_states, q_pos, kv_pos):
+        # relative_states: [B, Lq, heads, d_rel]
+        # rel_logits: [B, Lq, heads, rel_extent] -> [B, heads, Lq, rel_extent]
+        rel_logits = mx.swapaxes(relative_states @ self.proj, 1, 2)
+        B, H, Lq, _ = rel_logits.shape
+        distance = q_pos[:, None] - kv_pos[None, :]          # [Lq, Lkv]
+        gather = mx.clip(distance, 0, self.rel_extent - 1)   # [Lq, Lkv]
+        gather = mx.broadcast_to(gather[None, None], (B, H, Lq, gather.shape[-1]))
+        bias = mx.take_along_axis(rel_logits, gather, axis=-1)  # [B, H, Lq, Lkv]
+        valid = (distance >= 0) & (distance < self.rel_extent)  # [Lq, Lkv]
+        return mx.where(valid[None, None], bias, 0.0)
+
+
+class Attention(nn.Module):
+    def __init__(self, config: TextConfig, layer_idx: int):
+        super().__init__()
+        self.config = config
+        self.layer_idx = layer_idx
+        self.is_sliding = config.layer_types[layer_idx] == "hybrid_sliding"
+
+        self.head_dim = config.swa_head_dim if self.is_sliding else config.head_dim
+        self.num_heads = config.swa_num_attention_heads if self.is_sliding else config.num_attention_heads
+        self.num_kv_heads = config.swa_num_key_value_heads if self.is_sliding else config.num_key_value_heads
+        self.n_rep = self.num_heads // self.num_kv_heads
+        self.sliding_window = config.sliding_window_size if self.is_sliding else None
+        self.rel_extent = config.sliding_window_size if self.is_sliding else config.rel_extent
+        self.d_rel = config.d_rel
+        # q/k are per-head RMS-normalized, hence 1/d rather than 1/sqrt(d)
+        self.scaling = 1.0 / self.head_dim
+
+        h = config.hidden_size
+        self.wq_du = nn.Linear(h, self.num_heads * self.head_dim, bias=False)
+        self.wk_dv = nn.Linear(h, self.num_kv_heads * self.head_dim, bias=False)
+        self.wv_dv = nn.Linear(h, self.num_kv_heads * self.head_dim, bias=False)
+        self.wr_du = nn.Linear(h, self.num_heads * self.d_rel, bias=False)
+        self.wo_ud = nn.Linear(self.num_heads * self.head_dim, h, bias=False)
+
+        self.k_sconv = ShortConvolution(self.num_kv_heads * self.head_dim, config.sconv_kernel_size)
+        self.v_sconv = ShortConvolution(self.num_kv_heads * self.head_dim, config.sconv_kernel_size)
+        self.q_norm = RMSNorm(self.head_dim, eps=config.rms_norm_eps)
+        self.k_norm = RMSNorm(self.head_dim, eps=config.rms_norm_eps)
+        self.rel_logits_proj = RelativeLogits(self.d_rel, self.rel_extent)
+
+    def __call__(self, hidden_states, start_pos=0, kv_cache=None,
+                 k_conv=None, v_conv=None, conv_mask=None):
+        B, L, _ = hidden_states.shape
+
+        q = self.wq_du(hidden_states)
+        k = self.k_sconv(self.wk_dv(hidden_states), mask=conv_mask, cache=k_conv)
+        v = self.v_sconv(self.wv_dv(hidden_states), mask=conv_mask, cache=v_conv)
+        rel = self.wr_du(hidden_states)
+
+        q = self.q_norm(q.reshape(B, L, self.num_heads, self.head_dim))
+        k = self.k_norm(k.reshape(B, L, self.num_kv_heads, self.head_dim))
+        v = v.reshape(B, L, self.num_kv_heads, self.head_dim)
+
+        # -> [B, heads, L, head_dim]
+        q = q.transpose(0, 2, 1, 3)
+        k = k.transpose(0, 2, 1, 3)
+        v = v.transpose(0, 2, 1, 3)
+
+        q_pos = mx.arange(L) + start_pos
+        if kv_cache is not None:
+            k, v = kv_cache.update(k, v)      # full history
+        kv_pos = mx.arange(k.shape[2])
+
+        rel = rel.reshape(B, L, self.num_heads, self.d_rel)
+        position_bias = self.rel_logits_proj(rel, q_pos, kv_pos)  # [B, heads, Lq, Lkv]
+
+        # log-scaling (global layers only; no-op for context <= n_floor)
+        if not self.is_sliding and self.config.log_scaling_n_floor is not None:
+            n_floor = self.config.log_scaling_n_floor
+            eff_n = (q_pos + 1).astype(mx.float32)
+            tau = 1.0 + self.config.log_scaling_alpha * mx.log(
+                mx.maximum(eff_n / n_floor, 1.0)
+            )
+            tau_q = tau.reshape(1, 1, -1, 1)
+            q = (q.astype(mx.float32) * tau_q).astype(q.dtype)
+            position_bias = (position_bias.astype(mx.float32) * tau_q).astype(position_bias.dtype)
+
+        # Fused SDPA: O = softmax(scale·QKᵀ + mask)·V, softmax in fp32, native GQA
+        # (k/v passed un-tiled). mask = relative-position bias + causal/sliding, additive.
+        mask = position_bias + self._causal_mask(q_pos, kv_pos)   # [B, heads, Lq, Lkv]
+        out = mx.fast.scaled_dot_product_attention(
+            q, k, v, scale=self.scaling, mask=mask.astype(q.dtype)
+        )                                                        # [B, heads, Lq, head_dim]
+
+        out = out.transpose(0, 2, 1, 3).reshape(B, L, self.num_heads * self.head_dim)
+        return self.wo_ud(out)
+
+    def _causal_mask(self, q_pos, kv_pos):
+        distance = q_pos[:, None] - kv_pos[None, :]  # [Lq, Lkv]
+        allowed = distance >= 0
+        if self.sliding_window is not None:
+            allowed = allowed & (distance < self.sliding_window)
+        mask = mx.where(allowed, 0.0, NEG_INF)
+        return mask[None, None].astype(mx.float32)

inkling_mlx/audio.py

@@ -0,0 +1,34 @@
+"""Inkling audio tower: discrete dMel-token embedding + norm.
+
+Each audio frame is ``n_mel_bins`` discretized bins (values in ``[0, mel_vocab_size)``);
+each bin is embedded from its own slice of a shared table (offset ``bin * mel_vocab_size``)
+and the per-bin embeddings are summed. Mirrors ``InklingAudioModel`` /
+``InklingAudioModelEmbeddings``. Checkpoint keys: ``audio.encoder.weight`` (the
+``[n_mel_bins*mel_vocab_size, hidden]`` table) and ``audio.final_norm.weight``.
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .common import RMSNorm
+from .config import AudioConfig
+
+
+class AudioModel(nn.Module):
+    def __init__(self, config: AudioConfig):
+        super().__init__()
+        self.config = config
+        self.encoder = nn.Embedding(
+            config.n_mel_bins * config.mel_vocab_size, config.text_hidden_size
+        )
+        self.final_norm = RMSNorm(config.text_hidden_size, eps=config.rms_norm_eps)
+        # non-persistent: arange(n_mel_bins) * mel_vocab_size
+        self._offsets = mx.arange(config.n_mel_bins) * config.mel_vocab_size
+
+    def __call__(self, audio_input_ids: mx.array) -> mx.array:
+        # audio_input_ids: [..., n_mel_bins] with values in [0, mel_vocab_size)
+        embeds = self.encoder(audio_input_ids + self._offsets)  # [..., n_mel_bins, hidden]
+        embeds = embeds.sum(axis=-2)                            # [..., hidden]
+        return self.final_norm(embeds)

inkling_mlx/cache.py

@@ -0,0 +1,66 @@
+"""Incremental caches for Inkling generation.
+
+Two kinds of per-layer state must persist across decode steps:
+
+* ``KVCache``   — the appended key/value tensors for attention.
+* ``ConvCache`` — the last ``kernel-1`` inputs of each depthwise short-convolution
+  (there are 4 per layer: k, v, post-attn, post-mlp).
+
+A ``LayerCache`` bundles one KVCache + the 4 ConvCaches; ``make_cache`` builds one
+per decoder layer. Absolute key positions are always ``arange(kv_len)`` because the
+cache holds every key from position 0 (KVCache keeps the full history — correct for
+both global and sliding layers, since the sliding-window constraint is enforced by
+the attention mask).
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+
+
+class ConvCache:
+    """Holds the last ``kernel-1`` inputs of a short convolution."""
+
+    __slots__ = ("state",)
+
+    def __init__(self):
+        self.state = None  # [B, kernel-1, C] or None
+
+
+class KVCache:
+    """Appends keys/values along the sequence axis (full history)."""
+
+    __slots__ = ("keys", "values")
+
+    def __init__(self):
+        self.keys = None    # [B, heads, T, d]
+        self.values = None
+
+    @property
+    def offset(self) -> int:
+        return 0 if self.keys is None else self.keys.shape[2]
+
+    def update(self, k: mx.array, v: mx.array):
+        if self.keys is None:
+            self.keys, self.values = k, v
+        else:
+            self.keys = mx.concatenate([self.keys, k], axis=2)
+            self.values = mx.concatenate([self.values, v], axis=2)
+        return self.keys, self.values
+
+
+class LayerCache:
+    __slots__ = ("kv", "k_conv", "v_conv", "attn_conv", "mlp_conv")
+
+    def __init__(self):
+        self.kv = KVCache()
+        self.k_conv = ConvCache()
+        self.v_conv = ConvCache()
+        self.attn_conv = ConvCache()
+        self.mlp_conv = ConvCache()
+
+
+def make_cache(model) -> list[LayerCache]:
+    """One LayerCache per text decoder layer."""
+    n = len(model.model.llm.layers) if hasattr(model, "model") else len(model.layers)
+    return [LayerCache() for _ in range(n)]

inkling_mlx/common.py

@@ -0,0 +1,63 @@
+"""Shared low-level modules for the Inkling MLX port."""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+
+class RMSNorm(nn.Module):
+    """Llama-style RMSNorm (compute in fp32, weight is a gain).
+
+    Matches ``LlamaRMSNorm``: ``x_fp32 * rsqrt(mean(x^2) + eps) * weight``.
+    """
+
+    def __init__(self, dims: int, eps: float = 1e-6):
+        super().__init__()
+        self.weight = mx.ones((dims,))
+        self.eps = eps
+
+    def __call__(self, x: mx.array) -> mx.array:
+        return mx.fast.rms_norm(x, self.weight, self.eps)
+
+
+class ShortConvolution(nn.Module):
+    """Depthwise causal 1-D convolution with a residual add, computed in fp32.
+
+    Mirrors ``InklingShortConvolution``: a per-channel (groups == channels) causal
+    conv1d of ``kernel_size`` taps, no bias, no activation, then ``out + input``.
+    The reference keeps this module in fp32 regardless of the model dtype
+    (``_keep_in_fp32_modules_strict``), so we upcast here too.
+
+    Weight layout (MLX ``conv1d``): ``[channels, kernel_size, 1]``.
+    """
+
+    def __init__(self, channels: int, kernel_size: int):
+        super().__init__()
+        self.channels = channels
+        self.kernel_size = kernel_size
+        # [C_out, K, C_in // groups] with groups == channels -> [C, K, 1]
+        self.weight = mx.zeros((channels, kernel_size, 1))
+
+    def __call__(self, x: mx.array, mask: mx.array | None = None, cache=None) -> mx.array:
+        # x: [batch, seq, channels]
+        in_dtype = x.dtype
+        xf = x.astype(mx.float32)
+        residual = xf
+        if mask is not None:
+            xf = xf * mask.astype(mx.float32)
+        k = self.kernel_size
+        B, seq, C = xf.shape
+        w = self.weight.astype(mx.float32)
+        if cache is not None:
+            # left-context = cached last (k-1) inputs (zeros on the first call);
+            # a "valid" conv over [left, xf] yields exactly `seq` causal outputs.
+            left = cache.state if cache.state is not None else mx.zeros((B, k - 1, C), dtype=mx.float32)
+            x_in = mx.concatenate([left, xf], axis=1)
+            out = mx.conv1d(x_in, w, padding=0, groups=self.channels)
+            cache.state = x_in[:, -(k - 1):, :]
+        else:
+            # causal: left-pad by (k-1), keep first `seq` outputs (== zero left-context)
+            out = mx.conv1d(xf, w, padding=k - 1, groups=self.channels)[:, :seq, :]
+        out = out + residual
+        return out.astype(in_dtype)

inkling_mlx/config.py

@@ -0,0 +1,207 @@
+"""Configuration for the Inkling multimodal model (MLX port).
+
+Mirrors ``thinkingmachines/Inkling`` ``config.json`` and the transformers PR #47347
+reference (``InklingConfig`` / ``InklingTextConfig`` / ``InklingVisionConfig`` /
+``InklingAudioConfig``). We parse the *checkpoint* config layout (top-level
+``text_config`` / ``vision_config`` / ``audio_config`` / ``mtp_config``), not the
+flattened transformers layout.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any
+
+
+def _get(d: dict, *names, default=None):
+    for n in names:
+        if n in d and d[n] is not None:
+            return d[n]
+    return default
+
+
+@dataclass
+class TextConfig:
+    hidden_size: int = 6144
+    num_hidden_layers: int = 66
+    vocab_size: int = 201024
+    unpadded_vocab_size: int | None = 200058
+
+    # global (full) attention
+    num_attention_heads: int = 64
+    num_key_value_heads: int = 8
+    head_dim: int = 128
+    # sliding-window attention
+    swa_num_attention_heads: int = 64
+    swa_num_key_value_heads: int = 16
+    swa_head_dim: int = 128
+    sliding_window_size: int = 512
+
+    # relative-position logits
+    d_rel: int = 16
+    rel_extent: int = 1024
+    log_scaling_n_floor: int | None = 128000
+    log_scaling_alpha: float = 0.1
+
+    rms_norm_eps: float = 1e-6
+    use_embed_norm: bool = True
+
+    # short convolution
+    sconv_kernel_size: int = 4
+
+    # dense vs MoE MLP
+    dense_mlp_idx: int = 2
+    dense_intermediate_size: int = 24576  # dense MLP intermediate
+    moe_intermediate_size: int = 3072     # per-expert intermediate
+
+    # MoE routing
+    n_routed_experts: int = 256
+    num_experts_per_tok: int = 6
+    n_shared_experts: int = 2
+    shared_expert_sink: bool = True
+    route_scale: float = 8.0
+    use_gate_bias: bool = True
+    norm_after_topk: bool = True
+    use_global_scale: bool = True
+
+    logits_mup_width_multiplier: float = 24.0
+    hidden_act: str = "silu"
+
+    max_position_embeddings: int = 1048576
+
+    # which layer indices use sliding-window ("local") attention
+    local_layer_ids: list[int] = field(default_factory=list)
+
+    # MTP head (dropped for inference)
+    num_mtp_layers: int | None = None
+
+    @property
+    def layer_types(self) -> list[str]:
+        local = set(self.local_layer_ids)
+        return [
+            "hybrid_sliding" if i in local else "hybrid"
+            for i in range(self.num_hidden_layers)
+        ]
+
+    @property
+    def mlp_layer_types(self) -> list[str]:
+        return [
+            "dense" if i < self.dense_mlp_idx else "sparse"
+            for i in range(self.num_hidden_layers)
+        ]
+
+    @classmethod
+    def from_dict(cls, tc: dict) -> "TextConfig":
+        return cls(
+            hidden_size=_get(tc, "hidden_size", default=6144),
+            num_hidden_layers=_get(tc, "num_hidden_layers", default=66),
+            vocab_size=_get(tc, "vocab_size", default=201024),
+            unpadded_vocab_size=_get(tc, "unpadded_vocab_size"),
+            num_attention_heads=_get(tc, "num_attention_heads", default=64),
+            num_key_value_heads=_get(tc, "num_key_value_heads", default=8),
+            head_dim=_get(tc, "head_dim", default=128),
+            swa_num_attention_heads=_get(tc, "swa_num_attention_heads", default=64),
+            swa_num_key_value_heads=_get(tc, "swa_num_key_value_heads", default=16),
+            swa_head_dim=_get(tc, "swa_head_dim", default=128),
+            sliding_window_size=_get(tc, "sliding_window_size", default=512),
+            d_rel=_get(tc, "d_rel", default=16),
+            rel_extent=_get(tc, "rel_extent", default=1024),
+            log_scaling_n_floor=_get(tc, "log_scaling_n_floor"),
+            log_scaling_alpha=_get(tc, "log_scaling_alpha", default=0.1),
+            rms_norm_eps=_get(tc, "rms_norm_eps", default=1e-6),
+            use_embed_norm=_get(tc, "use_embed_norm", default=True),
+            sconv_kernel_size=_get(tc, "sconv_kernel_size", default=4),
+            dense_mlp_idx=_get(tc, "dense_mlp_idx", default=2),
+            dense_intermediate_size=_get(tc, "dense_intermediate_size", default=24576),
+            # checkpoint labels the *MoE* intermediate as `intermediate_size`
+            moe_intermediate_size=_get(tc, "intermediate_size", default=3072),
+            n_routed_experts=_get(tc, "n_routed_experts", default=256),
+            num_experts_per_tok=_get(tc, "num_experts_per_tok", default=6),
+            n_shared_experts=_get(tc, "n_shared_experts", default=2),
+            shared_expert_sink=_get(tc, "shared_expert_sink", default=True),
+            route_scale=_get(tc, "route_scale", default=8.0),
+            use_gate_bias=_get(tc, "use_gate_bias", default=True),
+            norm_after_topk=_get(tc, "norm_after_topk", default=True),
+            use_global_scale=_get(tc, "use_global_scale", default=True),
+            logits_mup_width_multiplier=_get(tc, "logits_mup_width_multiplier", default=24.0),
+            max_position_embeddings=_get(tc, "model_max_length", "max_position_embeddings", default=1048576),
+            local_layer_ids=list(_get(tc, "local_layer_ids", default=[]) or []),
+        )
+
+
+@dataclass
+class VisionConfig:
+    text_hidden_size: int = 6144
+    patch_size: int = 40
+    temporal_patch_size: int = 2
+    num_channels: int = 3
+    n_layers: int = 4
+    rms_norm_eps: float = 1e-6
+    use_vision_norm: bool = True
+
+    @classmethod
+    def from_dict(cls, vc: dict, text_hidden: int) -> "VisionConfig":
+        return cls(
+            text_hidden_size=text_hidden,
+            patch_size=_get(vc, "patch_size", default=40),
+            temporal_patch_size=_get(vc, "temporal_patch_size", default=2),
+            num_channels=_get(vc, "n_channels", "num_channels", default=3),
+            n_layers=_get(vc, "n_layers", "num_hidden_layers", default=4),
+            rms_norm_eps=_get(vc, "rms_norm_eps", default=1e-6),
+            use_vision_norm=_get(vc, "use_vision_norm", default=True),
+        )
+
+
+@dataclass
+class AudioConfig:
+    text_hidden_size: int = 6144
+    n_mel_bins: int = 80
+    mel_vocab_size: int = 16
+    rms_norm_eps: float = 1e-6
+
+    @classmethod
+    def from_dict(cls, ac: dict, text_hidden: int) -> "AudioConfig":
+        return cls(
+            text_hidden_size=text_hidden,
+            n_mel_bins=_get(ac, "n_mel_bins", default=80),
+            mel_vocab_size=_get(ac, "mel_vocab_size", default=16),
+            rms_norm_eps=_get(ac, "rms_norm_eps", default=1e-6),
+        )
+
+
+@dataclass
+class InklingConfig:
+    text: TextConfig
+    vision: VisionConfig
+    audio: AudioConfig
+    image_token_id: int = 200054
+    audio_token_id: int = 200053
+    image_bos_token_id: int = 200005
+    audio_bos_token_id: int = 200020
+    eos_token_id: int = 200006
+    model_type: str = "inkling_mm_model"
+
+    @classmethod
+    def from_dict(cls, cfg: dict) -> "InklingConfig":
+        tc = dict(cfg.get("text_config", {}))
+        mtp = cfg.get("mtp_config") or {}
+        if mtp.get("num_nextn_predict_layers") is not None:
+            tc.setdefault("num_mtp_layers", mtp.get("num_nextn_predict_layers"))
+        text = TextConfig.from_dict(tc)
+        vision = VisionConfig.from_dict(cfg.get("vision_config", {}) or {}, text.hidden_size)
+        audio = AudioConfig.from_dict(cfg.get("audio_config", {}) or {}, text.hidden_size)
+        return cls(
+            text=text,
+            vision=vision,
+            audio=audio,
+            image_token_id=_get(cfg, "image_token_id", default=200054),
+            audio_token_id=_get(cfg, "audio_token_id", default=200053),
+            image_bos_token_id=_get(cfg, "image_bos_token_id", default=200005),
+            audio_bos_token_id=_get(cfg, "audio_bos_token_id", default=200020),
+            eos_token_id=_get(cfg, "eos_token_id", default=200006),
+            model_type=_get(cfg, "model_type", default="inkling_mm_model"),
+        )
+
+    @property
+    def raw(self) -> dict[str, Any]:
+        return {"model_type": self.model_type}

inkling_mlx/convert.py

@@ -0,0 +1,254 @@
+"""Streaming HF -> MLX conversion + quantization for Inkling.
+
+The model is far too large (~1.9 TB bf16) to instantiate in RAM, so we convert
+tensor-by-tensor: read each source shard (mmap), remap the name, apply the layout
+transform, optionally affine-quantize the weight, and write output shards. Affine
+quantization has no cross-tensor dependency, so per-tensor streaming is exactly
+equivalent to ``nn.quantize(model)``.
+
+Name/layout transforms vs. the checkpoint:
+  * ``*_sconv.weight``  [C,1,K] -> [C,K,1]            (MLX conv1d layout)
+  * ``mlp.w13_dn``      [2I,H]  -> gate_proj/up_proj   (split dense fused gate+up)
+  * ``experts.w13_weight``        [E,2I,H] -> gate_proj/up_proj (split)
+  * ``experts.w2_weight``         [E,H,I]  -> down_proj          (identity)
+  * ``shared_experts.shared_w13`` [2,2I,H] -> gate_proj/up_proj  (split)
+  * ``model.mtp.*``               dropped (inference-irrelevant)
+  * everything else: identity
+"""
+
+from __future__ import annotations
+
+import glob
+import json
+import os
+import re
+import shutil
+
+import mlx.core as mx
+
+_LAYER_RE = re.compile(r"model\.llm\.layers\.(\d+)\.")
+_N_SHARED = 2   # Inkling: 2 shared experts (router rows after the routed ones)
+
+
+def prune_moe_tensor(name: str, w: mx.array, keep, dense_mlp_idx: int) -> mx.array:
+    """Subset a MoE tensor to the kept experts (REAP). ``keep`` maps sparse-layer
+    index -> array of kept routed-expert indices. Applied BEFORE map_name/transform.
+    Routed experts + the router (gate) are subset; shared experts are untouched."""
+    m = _LAYER_RE.search(name)
+    if m is None:
+        return w
+    L = int(m.group(1))
+    if L < dense_mlp_idx:                        # dense layer — no routed experts
+        return w
+    kidx = mx.array(keep[L - dense_mlp_idx])     # [K]
+    if name.endswith(("experts.w13_weight", "experts.w2_weight")) and "shared" not in name:
+        return w[kidx]                           # [E, ...] -> [K, ...]
+    if name.endswith("mlp.gate.weight"):         # [n_routed + n_shared, hidden]
+        n_routed = w.shape[0] - _N_SHARED        # rows: [routed... , shared...]
+        return mx.concatenate([w[kidx], w[n_routed:]], axis=0)   # [K + n_shared, hidden]
+    if name.endswith("mlp.gate.bias"):           # [n_routed] correction bias
+        return w[kidx]
+    return w                                     # shared_experts.*, gate.global_scale, etc.
+
+
+def map_name(name: str):
+    """HF checkpoint tensor name -> list of (out_name, kind) for the MLX model."""
+    if name.startswith("model.mtp."):
+        return []  # drop MTP head
+
+    if name.endswith(("k_sconv.weight", "v_sconv.weight", "attn_sconv.weight", "mlp_sconv.weight")):
+        return [(name, "sconv")]
+
+    # dense MLP fused gate+up / down
+    if name.endswith("mlp.w13_dn.weight"):
+        base = name[: -len("w13_dn.weight")]
+        return [(base + "gate_proj.weight", "w13_gate"), (base + "up_proj.weight", "w13_up")]
+    if name.endswith("mlp.w2_md.weight"):
+        return [(name[: -len("w2_md.weight")] + "down_proj.weight", "identity")]
+
+    # routed experts fused
+    if name.endswith("experts.w13_weight"):
+        base = name[: -len("w13_weight")]
+        return [(base + "gate_proj.weight", "w13_gate"), (base + "up_proj.weight", "w13_up")]
+    if name.endswith("experts.w2_weight"):
+        return [(name[: -len("w2_weight")] + "down_proj.weight", "identity")]
+
+    # shared experts fused
+    if name.endswith("shared_experts.shared_w13_weight"):
+        base = name[: -len("shared_w13_weight")]
+        return [(base + "gate_proj.weight", "w13_gate"), (base + "up_proj.weight", "w13_up")]
+    if name.endswith("shared_experts.shared_w2_weight"):
+        return [(name[: -len("shared_w2_weight")] + "down_proj.weight", "identity")]
+
+    return [(name, "identity")]
+
+
+def transform(w: mx.array, kind: str) -> mx.array:
+    if kind == "identity":
+        return w
+    if kind == "sconv":
+        # [C, 1, K] -> [C, K, 1]
+        return mx.swapaxes(w, 1, 2)
+    if kind in ("w13_gate", "w13_up"):
+        # The checkpoint stores gate/up INTERLEAVED row-wise: [g0, u0, g1, u1, ...]
+        # (SGLang `deinterleave_w13`). De-interleave: gate = rows 0::2, up = rows 1::2.
+        # A contiguous [:half]/[half:] split scrambles gate<->up in every MLP.
+        n = w.shape[-2] // 2
+        g = w.reshape(*w.shape[:-2], n, 2, w.shape[-1])
+        return g[..., 0, :] if kind == "w13_gate" else g[..., 1, :]
+    raise ValueError(kind)
+
+
+# ---- quantization target predicate (must be identical in convert and load) ----
+
+# Quant "recipes" — which module leaves get affine-quantized.
+#   uniform       : everything (attention, MLP/experts, embed/unembed, audio, vision)
+#   experts_only  : ONLY the MLP/expert matmuls (+ audio/vision); attention and
+#                   embed/unembed stay bf16. Inkling attention dominates 4-bit error
+#                   (~58% per layer vs ~15% for experts), so this keeps a 4-bit-sized
+#                   build coherent while the ~927 B experts still fit in 512 GB.
+_RECIPES = {
+    "uniform": {"wq_du", "wk_dv", "wv_dv", "wr_du", "wo_ud",
+                "gate_proj", "up_proj", "down_proj", "embed", "unembed", "encoder"},
+    "experts_only": {"gate_proj", "up_proj", "down_proj", "encoder"},
+}
+
+
+def is_quant_target(out_name: str, quant_axis_size: int, group_size: int, recipe: str = "uniform") -> bool:
+    """Whether ``out_name`` (a converted param path) should be affine-quantized."""
+    if not out_name.endswith(".weight"):
+        return False
+    leaf = out_name[: -len(".weight")].rsplit(".", 1)[-1]
+    leaves = _RECIPES[recipe]
+    # vision projection layers (linear_0 .. linear_3) — quantized in both recipes
+    is_vision_linear = leaf.startswith("linear_") and ".visual." in out_name
+    if leaf not in leaves and not is_vision_linear:
+        return False
+    # router gate stays fp (leaf == "gate", excluded above); norms/sconv excluded by leaf
+    # can only group-quantize when the input dim is a multiple of group_size
+    return quant_axis_size % group_size == 0
+
+
+# ------------------------------ streaming driver ------------------------------
+
+_SHARD_CAP_BYTES = 5_000_000_000  # ~5 GB per output shard
+
+
+def _process_tensor(name, w, bits, group_size, out_dtype, recipe="uniform", keep=None, dmi=2):
+    """Yield (out_name, array) pairs for one source tensor."""
+    if keep is not None:
+        w = prune_moe_tensor(name, w, keep, dmi)   # REAP: subset to kept experts
+    for out_name, kind in map_name(name):
+        wt = transform(w, kind)
+        quantize = bits is not None and is_quant_target(out_name, wt.shape[-1], group_size, recipe)
+        if quantize:
+            qw, scales, biases = mx.quantize(wt, group_size=group_size, bits=bits)
+            base = out_name[: -len(".weight")]
+            yield out_name, qw
+            yield base + ".scales", scales
+            yield base + ".biases", biases
+        else:
+            # keep norms/router/sconv/rel-proj in fp32-safe dtype; matmul weights in out_dtype
+            keep_hi = wt.dtype == mx.float32 and (".global_scale" in out_name or ".bias" in out_name
+                                                  or out_name.endswith(("_norm.weight", "norm.weight")))
+            yield out_name, wt.astype(mx.float32 if keep_hi else out_dtype)
+
+
+def convert_model(src: str, dst: str, bits=None, group_size: int = 64, out_dtype=mx.bfloat16,
+                  recipe: str = "uniform", keep_path=None):
+    """Stream-convert an Inkling checkpoint from ``src`` to ``dst``.
+
+    ``bits=None`` -> plain dtype cast (bf16). ``bits in {4,6,8}`` -> affine quant.
+    ``recipe`` selects which modules are quantized (see ``_RECIPES``).
+    ``keep_path`` (REAP): npz with ``keep`` [n_sparse_layers, K] + ``dense_mlp_idx`` ->
+    prune each MoE layer to its K kept experts and set ``n_routed_experts=K`` in config.
+    Processes one source shard at a time; never holds the whole model in RAM.
+    """
+    os.makedirs(dst, exist_ok=True)
+    keep = dmi = new_ne = None
+    if keep_path is not None:
+        import numpy as np
+        kd = np.load(keep_path)
+        keep = kd["keep"]; dmi = int(kd["dense_mlp_idx"]); new_ne = int(kd["K"])
+    index = json.load(open(os.path.join(src, "model.safetensors.index.json")))
+    weight_map = index["weight_map"]
+
+    shard_to_names: dict[str, list[str]] = {}
+    for n, s in weight_map.items():
+        shard_to_names.setdefault(s, []).append(n)
+
+    out_index: dict[str, str] = {}
+    buffer: dict[str, mx.array] = {}
+    buffer_bytes = 0
+    out_shard_id = 0
+    total_out_shards_placeholder = "{:05d}"
+
+    def flush(final=False):
+        nonlocal buffer, buffer_bytes, out_shard_id
+        if not buffer:
+            return
+        out_shard_id += 1
+        fname = f"model-{total_out_shards_placeholder.format(out_shard_id)}.safetensors"
+        mx.save_safetensors(os.path.join(dst, fname), buffer, metadata={"format": "mlx"})
+        for k in buffer:
+            out_index[k] = fname
+        buffer = {}
+        buffer_bytes = 0
+
+    for shard in sorted(shard_to_names):
+        path = os.path.join(src, shard)
+        tensors = mx.load(path)  # mmap
+        for name in shard_to_names[shard]:
+            w = tensors[name]
+            for out_name, arr in _process_tensor(name, w, bits, group_size, out_dtype, recipe, keep, dmi):
+                mx.eval(arr)
+                buffer[out_name] = arr
+                buffer_bytes += arr.nbytes
+                if buffer_bytes >= _SHARD_CAP_BYTES:
+                    flush()
+        del tensors
+    flush(final=True)
+
+    # rename shards with correct total, build index.json
+    _finalize_index(dst, out_index, out_shard_id)
+    _write_config(src, dst, bits, group_size, recipe, new_ne)
+    _copy_aux(src, dst)
+    return dst
+
+
+def _finalize_index(dst, out_index, n_shards):
+    # rewrite shard filenames to model-XXXXX-of-YYYYY.safetensors
+    remap = {}
+    for i in range(1, n_shards + 1):
+        old = f"model-{i:05d}.safetensors"
+        new = f"model-{i:05d}-of-{n_shards:05d}.safetensors"
+        if old != new and os.path.exists(os.path.join(dst, old)):
+            os.rename(os.path.join(dst, old), os.path.join(dst, new))
+        remap[old] = new
+    weight_map = {k: remap[v] for k, v in out_index.items()}
+    total = sum(os.path.getsize(os.path.join(dst, f)) for f in set(weight_map.values()))
+    with open(os.path.join(dst, "model.safetensors.index.json"), "w") as f:
+        json.dump({"metadata": {"total_size": total}, "weight_map": weight_map}, f, indent=2)
+
+
+def _write_config(src, dst, bits, group_size, recipe="uniform", new_ne=None):
+    cfg = json.load(open(os.path.join(src, "config.json")))
+    if bits is not None:
+        cfg["quantization"] = {"group_size": group_size, "bits": bits, "recipe": recipe}
+    if new_ne is not None:                       # REAP: fewer routed experts
+        cfg["text_config"]["n_routed_experts"] = new_ne
+        cfg.setdefault("reap", {})["kept_experts"] = new_ne
+    with open(os.path.join(dst, "config.json"), "w") as f:
+        json.dump(cfg, f, indent=2)
+
+
+def _copy_aux(src, dst):
+    for pat in ("tokenizer*", "special_tokens_map.json", "*.tiktoken", "tiktoken",
+                "chat_template.jinja", "processor_config.json"):
+        for p in glob.glob(os.path.join(src, pat)):
+            base = os.path.basename(p)
+            target = os.path.join(dst, base)
+            if os.path.isdir(p):
+                shutil.copytree(p, target, dirs_exist_ok=True)
+            else:
+                shutil.copy2(p, target)

inkling_mlx/convert_cli.py

@@ -0,0 +1,47 @@
+"""CLI: convert/quantize an Inkling checkpoint to MLX.
+
+Examples:
+  python -m inkling_mlx.convert_cli --src /path/Inkling-src --dst out-bf16
+  python -m inkling_mlx.convert_cli --src /path/Inkling-src --dst out-4bit --bits 4
+"""
+
+from __future__ import annotations
+
+import argparse
+import time
+
+import mlx.core as mx
+
+from .convert import convert_model
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--src", required=True, help="Inkling bf16 source dir (HF layout)")
+    ap.add_argument("--dst", required=True, help="output dir")
+    ap.add_argument("--bits", type=int, default=None, choices=[2, 3, 4, 5, 6, 8],
+                    help="quantization bits; omit for bf16 passthrough")
+    ap.add_argument("--group-size", type=int, default=64)
+    ap.add_argument("--dtype", default="bfloat16", choices=["bfloat16", "float16"])
+    ap.add_argument("--device", default="gpu", choices=["gpu", "cpu"],
+                    help="cpu avoids the Metal GPU-timeout watchdog on huge tensors (slower, robust)")
+    ap.add_argument("--recipe", default="uniform", choices=["uniform", "experts_only"],
+                    help="experts_only keeps attention + embed/unembed at bf16 (coherent 4-bit-sized build)")
+    ap.add_argument("--prune", default=None,
+                    help="REAP keep-indices npz (from prune_experts.py) -> prune experts during convert")
+    args = ap.parse_args()
+
+    if args.device == "cpu":
+        mx.set_default_device(mx.cpu)
+        print("[convert] using CPU device (avoids Metal command-buffer timeout)")
+
+    dtype = {"bfloat16": mx.bfloat16, "float16": mx.float16}[args.dtype]
+    t0 = time.time()
+    print(f"[convert] {args.src} -> {args.dst}  bits={args.bits} group_size={args.group_size} dtype={args.dtype} recipe={args.recipe} prune={args.prune}")
+    convert_model(args.src, args.dst, bits=args.bits, group_size=args.group_size, out_dtype=dtype,
+                  recipe=args.recipe, keep_path=args.prune)
+    print(f"[convert] done in {time.time()-t0:.0f}s -> {args.dst}")
+
+
+if __name__ == "__main__":
+    main()

inkling_mlx/generate.py

@@ -0,0 +1,90 @@
+"""Greedy generation for an Inkling MLX model, using an incremental KV + conv-state
+cache: the prompt is prefilled once, then each new token is a single-position step.
+"""
+
+from __future__ import annotations
+
+import argparse
+import time
+
+import mlx.core as mx
+
+from .cache import make_cache
+from .load import load
+
+
+def load_tokenizer(path: str):
+    try:
+        from transformers import AutoTokenizer
+        return AutoTokenizer.from_pretrained(path, trust_remote_code=True)
+    except Exception:
+        from transformers import PreTrainedTokenizerFast
+        import os
+        return PreTrainedTokenizerFast(tokenizer_file=os.path.join(path, "tokenizer.json"))
+
+
+def greedy_generate(model, config, input_ids, max_new_tokens=32, eos_id=None,
+                    pixel_values=None, audio_input_ids=None):
+    """Greedy decode. For multimodal, pass ``pixel_values`` / ``audio_input_ids``
+    (from ``InklingProcessor``); they are consumed only by the prompt prefill."""
+    eos_id = eos_id if eos_id is not None else config.eos_token_id
+    caches = make_cache(model)
+    prompt = list(input_ids)
+
+    # prefill the whole prompt (with any media) in one pass
+    logits = model(mx.array([prompt]), caches=caches, start_pos=0, last_logit_only=True,
+                   pixel_values=pixel_values, audio_input_ids=audio_input_ids)
+    next_id = int(mx.argmax(logits[0, -1]).item())
+    out = [next_id]
+    pos = len(prompt)
+
+    for _ in range(max_new_tokens - 1):
+        if next_id == eos_id:
+            break
+        logits = model(mx.array([[next_id]]), caches=caches, start_pos=pos, last_logit_only=True)
+        next_id = int(mx.argmax(logits[0, -1]).item())
+        out.append(next_id)
+        pos += 1
+
+    return prompt + out
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--model", required=True, help="converted MLX model dir")
+    ap.add_argument("--prompt", default="The capital of France is")
+    ap.add_argument("--max-new-tokens", type=int, default=32)
+    ap.add_argument("--wired-limit-gb", type=float, default=500.0,
+                    help="mx wired-memory limit; needs `sudo sysctl iogpu.wired_limit_mb` set too")
+    ap.add_argument("--lazy", action="store_true",
+                    help="mmap weights instead of eager-loading (lower peak RAM, but forwards "
+                         "re-read from disk and thrash near the memory ceiling)")
+    args = ap.parse_args()
+
+    # eager load pins the weights wired-resident so prefill/decode don't re-read the
+    # mmap (the big win for near-capacity models); pass --lazy to opt out.
+    try:
+        mx.set_wired_limit(int(args.wired_limit_gb * 1e9))
+    except Exception as e:
+        print(f"[warn] set_wired_limit: {e}")
+
+    print(f"[load] {args.model} ({'lazy mmap' if args.lazy else 'eager, wired-resident'})")
+    t0 = time.time()
+    model, config = load(args.model, lazy=args.lazy)
+    print(f"[load] ready in {time.time()-t0:.0f}s")
+
+    tok = load_tokenizer(args.model)
+    input_ids = tok(args.prompt)["input_ids"]
+    print(f"[prompt] {args.prompt!r} -> {len(input_ids)} tokens")
+
+    t0 = time.time()
+    out_ids = greedy_generate(model, config, input_ids, args.max_new_tokens)
+    dt = time.time() - t0
+    text = tok.decode(out_ids)
+    n_new = len(out_ids) - len(input_ids)
+    print(f"\n{text}\n")
+    print(f"[gen] {n_new} tokens in {dt:.1f}s ({n_new/dt:.2f} tok/s)")
+
+
+if __name__ == "__main__":
+    main()

inkling_mlx/layers.py

@@ -0,0 +1,47 @@
+"""Inkling decoder layer: attention + MLP, each wrapped by a pre-norm and a
+trailing short-convolution, with residual adds. Mirrors ``InklingDecoderLayer``.
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .attention import Attention
+from .common import RMSNorm, ShortConvolution
+from .config import TextConfig
+from .moe import DenseMLP, MoE
+
+
+class DecoderLayer(nn.Module):
+    def __init__(self, config: TextConfig, layer_idx: int):
+        super().__init__()
+        self.attn = Attention(config, layer_idx)
+        self.attn_norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.mlp_norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        if config.mlp_layer_types[layer_idx] == "sparse":
+            self.mlp = MoE(config)
+        else:
+            self.mlp = DenseMLP(config)
+        self.attn_sconv = ShortConvolution(config.hidden_size, config.sconv_kernel_size)
+        self.mlp_sconv = ShortConvolution(config.hidden_size, config.sconv_kernel_size)
+
+    def __call__(self, x, start_pos=0, cache=None, conv_mask=None):
+        kv = cache.kv if cache is not None else None
+        residual = x
+        h = self.attn_norm(x)
+        h = self.attn(
+            h, start_pos=start_pos, kv_cache=kv,
+            k_conv=cache.k_conv if cache is not None else None,
+            v_conv=cache.v_conv if cache is not None else None,
+            conv_mask=conv_mask,
+        )
+        h = self.attn_sconv(h, mask=conv_mask, cache=cache.attn_conv if cache is not None else None)
+        x = residual + h
+
+        residual = x
+        h = self.mlp_norm(x)
+        h = self.mlp(h)
+        h = self.mlp_sconv(h, mask=conv_mask, cache=cache.mlp_conv if cache is not None else None)
+        x = residual + h
+        return x

inkling_mlx/load.py

@@ -0,0 +1,69 @@
+"""Load a (possibly quantized) Inkling MLX model produced by ``convert_model``."""
+
+from __future__ import annotations
+
+import glob
+import json
+import os
+
+import mlx.core as mx
+import mlx.nn as nn
+from mlx.utils import tree_flatten
+
+from .config import InklingConfig
+from .model import InklingForConditionalGeneration
+
+
+def quant_predicate(group_size: int, recipe: str = "uniform"):
+    """Quantize exactly the modules the converter did, by delegating to
+    ``convert.is_quant_target`` with the same ``recipe``. Guarantees the loaded
+    module set matches the checkpoint (e.g. under ``experts_only``, attention and
+    embed/unembed stay bf16 and must NOT be re-quantized here)."""
+    from .convert import is_quant_target
+
+    def pred(path, module):
+        if not hasattr(module, "to_quantized"):
+            return False
+        w = getattr(module, "weight", None)
+        if w is None:
+            return False
+        return is_quant_target(path + ".weight", w.shape[-1], group_size, recipe)
+
+    return pred
+
+
+def load(path: str, lazy: bool = False):
+    cfg_dict = json.load(open(os.path.join(path, "config.json")))
+    config = InklingConfig.from_dict(cfg_dict)
+    model = InklingForConditionalGeneration(config)
+
+    q = cfg_dict.get("quantization")
+    if q:
+        nn.quantize(model, group_size=q["group_size"], bits=q["bits"],
+                    class_predicate=quant_predicate(q["group_size"], q.get("recipe", "uniform")))
+
+    # Stream shards: assign each, then release its handle. We do NOT eagerly
+    # mx.eval() the whole parameter tree — for a ~500 GB model that builds one
+    # enormous eval graph and trips a Metal resource limit. Weights stay lazy
+    # (mmap-backed) and materialize on demand during the forward pass, exactly
+    # like mlx-lm loads large models.
+    loaded = set()
+    shards = sorted(glob.glob(os.path.join(path, "*.safetensors")))
+    for shard in shards:
+        w = mx.load(shard)
+        model.load_weights(list(w.items()), strict=False)
+        if not lazy:
+            # materialize THIS shard's tensors now (bounded graph) and keep them
+            # resident. Avoids one enormous eval over all ~500 GB of params, which
+            # trips a Metal resource limit; also prevents per-token disk paging.
+            mx.eval(list(w.values()))
+        loaded.update(w.keys())
+        del w
+
+    expected = {k for k, _ in tree_flatten(model.parameters())}
+    missing = expected - loaded
+    if missing:
+        raise ValueError(f"{len(missing)} params not found in checkpoint, e.g. {sorted(missing)[:3]}")
+
+    model.eval()
+    return model, config

inkling_mlx/model.py

@@ -0,0 +1,79 @@
+"""Top-level Inkling multimodal model.
+
+Checkpoint layout: ``model.llm.*`` (text backbone + untied unembed), ``model.visual.*``
+(HMLP vision tower), ``model.audio.*`` (dMel audio tower). Image/audio features are
+scattered into the token-embedding stream at their placeholder-token positions, then
+the text backbone runs and the untied unembed head produces (muP-scaled) logits.
+The MTP head (``model.mtp.*``) is intentionally not loaded (inference-irrelevant).
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+import numpy as np
+
+from .audio import AudioModel
+from .config import InklingConfig
+from .text import TextModel
+from .vision import VisionModel
+
+
+def _scatter_features(embeds, input_ids, token_id, features):
+    """Replace ``embeds`` rows where ``input_ids == token_id`` with ``features``
+    (in sequence order). ``input_ids`` is host-known so we resolve positions on CPU."""
+    B, L, H = embeds.shape
+    ids = np.array(input_ids).reshape(-1)
+    pos = np.nonzero(ids == token_id)[0]
+    if pos.size == 0:
+        return embeds
+    flat = embeds.reshape(B * L, H)
+    flat[mx.array(pos)] = features.astype(flat.dtype)
+    return flat.reshape(B, L, H)
+
+
+class InnerModel(nn.Module):
+    """The ``model.`` level holding the three towers."""
+
+    def __init__(self, config: InklingConfig):
+        super().__init__()
+        self.llm = TextModel(config.text)
+        self.visual = VisionModel(config.vision)
+        self.audio = AudioModel(config.audio)
+
+
+class InklingForConditionalGeneration(nn.Module):
+    def __init__(self, config: InklingConfig):
+        super().__init__()
+        self.config = config
+        self.model = InnerModel(config)
+
+    # --- convenience accessors ---
+    @property
+    def llm(self) -> TextModel:
+        return self.model.llm
+
+    def __call__(
+        self,
+        input_ids: mx.array,
+        pixel_values: mx.array | None = None,
+        audio_input_ids: mx.array | None = None,
+        conv_mask=None,
+        caches=None,
+        start_pos: int = 0,
+        last_logit_only: bool = False,
+    ) -> mx.array:
+        embeds = self.model.llm.embed_tokens(input_ids)
+
+        if pixel_values is not None:
+            img = self.model.visual(pixel_values)
+            embeds = _scatter_features(embeds, input_ids, self.config.image_token_id, img)
+
+        if audio_input_ids is not None:
+            aud = self.model.audio(audio_input_ids)
+            embeds = _scatter_features(embeds, input_ids, self.config.audio_token_id, aud)
+
+        hidden = self.model.llm.backbone(embeds, conv_mask=conv_mask, caches=caches, start_pos=start_pos)
+        if last_logit_only:
+            hidden = hidden[:, -1:, :]
+        return self.model.llm.logits(hidden)

inkling_mlx/moe.py

@@ -0,0 +1,117 @@
+"""Inkling MLP variants: dense SwiGLU (with a learned output scale) and the
+sparse MoE (sigmoid router with correction bias, softmax-over-selected weights,
+route/global scaling, and 2 always-on shared experts forming a routing "sink").
+
+Mirrors ``InklingMLP`` / ``InklingTopkRouter`` / ``InklingExperts`` /
+``InklingSharedExperts`` / ``InklingMoE`` from transformers PR #47347.
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from mlx_lm.models.switch_layers import SwitchGLU
+
+from .config import TextConfig
+
+
+class DenseMLP(nn.Module):
+    """SwiGLU MLP with a learned scalar output gain (``global_scale``).
+
+    The checkpoint fuses gate+up into ``w13_dn``; the converter splits it into
+    ``gate_proj``/``up_proj`` so the standard MLX quantizer sees plain ``nn.Linear``s.
+    """
+
+    def __init__(self, config: TextConfig):
+        super().__init__()
+        h = config.hidden_size
+        inter = config.dense_intermediate_size
+        self.gate_proj = nn.Linear(h, inter, bias=False)
+        self.up_proj = nn.Linear(h, inter, bias=False)
+        self.down_proj = nn.Linear(inter, h, bias=False)
+        self.global_scale = mx.ones((1,))
+
+    def __call__(self, x: mx.array) -> mx.array:
+        y = self.down_proj(nn.silu(self.gate_proj(x)) * self.up_proj(x))
+        return y * self.global_scale
+
+
+class Router(nn.Module):
+    """Sigmoid top-k router with a correction bias and a shared-expert sink.
+
+    Kept in full precision (tiny). Returns per-token routed weights/indices plus
+    the two shared-expert gammas produced by the same softmax (the "sink").
+    """
+
+    def __init__(self, config: TextConfig):
+        super().__init__()
+        self.num_experts = config.n_routed_experts
+        self.n_shared = config.n_shared_experts
+        self.n_total = self.num_experts + self.n_shared
+        self.top_k = config.num_experts_per_tok
+        self.route_scale = config.route_scale
+        self.hidden = config.hidden_size
+        self.weight = mx.zeros((self.n_total, config.hidden_size))
+        self.bias = mx.zeros((self.num_experts,))              # e_score_correction_bias
+        self.global_scale = mx.ones((1,))
+
+    def __call__(self, x: mx.array):
+        # Routing (esp. the top-k selection) is precision-sensitive: in bf16 the
+        # rounding of near-tied expert scores flips which experts fire, and a wrong
+        # choice compounds over 64 MoE layers into incoherent output. Compute the
+        # whole router in fp32.
+        flat = x.reshape(-1, self.hidden).astype(mx.float32)
+        router_logits = flat @ self.weight.T.astype(mx.float32)  # [T, n_total]
+        scores = mx.sigmoid(router_logits)
+        routed_scores = scores[:, : self.num_experts]
+        scores_for_choice = routed_scores + self.bias
+
+        # top-k experts (order within the top-k is irrelevant downstream)
+        topk_idx = mx.argpartition(-scores_for_choice, kth=self.top_k - 1, axis=-1)[:, : self.top_k]
+
+        routed_logits = router_logits[:, : self.num_experts]
+        shared_logits = router_logits[:, self.num_experts :]  # [T, n_shared]
+        gathered = mx.take_along_axis(routed_logits, topk_idx, axis=-1)  # [T, top_k]
+        topk_logits = mx.concatenate([gathered, shared_logits], axis=-1)  # [T, top_k+n_shared]
+
+        # softmax over the selected (+shared) logits, computed in the log domain
+        log_probs = -mx.logaddexp(mx.zeros_like(topk_logits), -topk_logits)  # logsigmoid
+        weights = mx.softmax(log_probs, axis=-1)
+        weights = weights * self.route_scale * self.global_scale
+
+        shared_gammas = weights[:, self.top_k :].astype(x.dtype)   # [T, n_shared]
+        topk_weights = weights[:, : self.top_k].astype(x.dtype)    # [T, top_k]
+        return topk_weights, topk_idx, shared_gammas
+
+
+class MoE(nn.Module):
+    def __init__(self, config: TextConfig):
+        super().__init__()
+        self.config = config
+        self.n_shared = config.n_shared_experts
+        self.gate = Router(config)
+        self.experts = SwitchGLU(
+            config.hidden_size, config.moe_intermediate_size, config.n_routed_experts, bias=False
+        )
+        self.shared_experts = SwitchGLU(
+            config.hidden_size, config.moe_intermediate_size, config.n_shared_experts, bias=False
+        )
+
+    def __call__(self, x: mx.array) -> mx.array:
+        B, L, H = x.shape
+        topk_weights, topk_idx, shared_gammas = self.gate(x)
+        xf = x.reshape(-1, H)                                  # [T, H]
+        T = xf.shape[0]
+
+        routed = self.experts(xf, topk_idx)                   # [T, top_k, H]
+        routed = (routed * topk_weights[..., None]).sum(axis=1)
+
+        shared_idx = mx.broadcast_to(
+            mx.arange(self.n_shared)[None], (T, self.n_shared)
+        )
+        shared = self.shared_experts(xf, shared_idx)          # [T, n_shared, H]
+        shared = (shared.astype(mx.float32) * shared_gammas[..., None].astype(mx.float32)).sum(axis=1)
+        shared = shared.astype(routed.dtype)
+
+        return (routed + shared).reshape(B, L, H)

inkling_mlx/processing.py

@@ -0,0 +1,188 @@
+"""Image + audio preprocessing for Inkling (MLX), ported from the reference
+`InklingImageProcessor` / `InklingFeatureExtractor` / `InklingProcessor`.
+
+  * image  -> pixel_values [num_patches, T=2, 40, 40, 3]  (feeds `VisionModel`)
+  * audio  -> audio_input_ids [num_frames, 80] dMel bins  (feeds `AudioModel`)
+
+`InklingProcessor.apply` builds the full multimodal input (input_ids + features)
+from a chat message list, inserting the right number of placeholder soft-tokens.
+Uses numpy/PIL + transformers' mel filterbank; no torch needed at inference.
+"""
+
+from __future__ import annotations
+
+import math
+
+import numpy as np
+
+# CLIP normalization (OPENAI_CLIP_MEAN / STD), per processor_config.json
+CLIP_MEAN = np.array([0.48145466, 0.4578275, 0.40821073], dtype=np.float32)
+CLIP_STD = np.array([0.26862954, 0.26130258, 0.27577711], dtype=np.float32)
+
+PATCH = 40           # image patch size (== vision patch_size)
+TEMPORAL = 2         # temporal_patch_size (images duplicated across 2 frames)
+
+# audio (processor_config.json / feature_extraction_inkling.py)
+SR = 16000
+HOP = 800            # audio_token_duration_s (0.05) * SR
+WIN = 1600           # * window_size_multiplier (2.0)
+N_FFT = 1600
+N_MEL = 80
+DMEL_BINS = 16
+DMEL_MIN, DMEL_MAX = -7.0, 2.0
+
+# special tokens
+IMAGE_TOKEN_ID = 200054      # <|unused_200054|>  (soft-token slot)
+AUDIO_TOKEN_ID = 200053      # <|unused_200053|>
+IMAGE_BOS = "<|content_image|>"
+AUDIO_BOS = "<|content_audio_input|>"
+
+
+# ------------------------------- image -------------------------------
+
+def preprocess_image(image, max_long_edge: int | None = None) -> tuple[np.ndarray, int]:
+    """PIL.Image or HxWx3 uint8 array -> (pixel_values [N,2,40,40,3] float32, N).
+
+    ``max_long_edge`` (optional) downscales the image (LANCZOS, aspect preserved) so its
+    long edge is at most this many pixels *before* patchify. Each 40px patch is one
+    vision soft-token, so this directly cuts the prompt length / prefill cost for big
+    images (e.g. cap 512 -> ~130 patches vs ~450 at full 960px). ``None`` keeps native
+    resolution (the reference default)."""
+    from PIL import Image
+    if not hasattr(image, "convert"):
+        image = Image.fromarray(np.asarray(image).astype(np.uint8))
+    image = image.convert("RGB")
+    if max_long_edge and max(image.size) > max_long_edge:
+        r = max_long_edge / max(image.size)
+        image = image.resize((max(1, round(image.width * r)), max(1, round(image.height * r))), Image.LANCZOS)
+    image = np.asarray(image)
+    if image.ndim == 2:
+        image = np.stack([image] * 3, axis=-1)
+    img = image[..., :3].astype(np.float32).transpose(2, 0, 1)  # -> [C, H, W]
+    C, H, W = img.shape
+
+    num_rows = (H + PATCH - 1) // PATCH
+    num_cols = W // PATCH + 1                       # reference: W//P + 1
+    patches = []
+    for i in range(num_rows):
+        for j in range(num_cols):
+            p = img[:, i * PATCH:(i + 1) * PATCH, j * PATCH:(j + 1) * PATCH]  # may be < 40
+            padded = np.full((C, PATCH, PATCH), -1.0, dtype=np.float32)       # pad value -1.0
+            padded[:, : p.shape[1], : p.shape[2]] = p
+            patches.append(padded)
+    patches = np.stack(patches, axis=0)             # [N, C, 40, 40]
+
+    # rescale (1/255) + CLIP normalize per channel
+    patches = patches / 255.0
+    patches = (patches - CLIP_MEAN[None, :, None, None]) / CLIP_STD[None, :, None, None]
+
+    # add temporal dim, duplicate x2, then -> [N, T, H, W, C]
+    patches = np.repeat(patches[..., None], TEMPORAL, axis=-1)   # [N, C, 40, 40, 2]
+    pixel_values = patches.transpose(0, 4, 2, 3, 1)              # [N, 2, 40, 40, C]
+    return pixel_values.astype(np.float32), pixel_values.shape[0]
+
+
+# ------------------------------- audio -------------------------------
+
+_mel_fb = None
+def _mel_filters() -> np.ndarray:
+    global _mel_fb
+    if _mel_fb is None:
+        from transformers.audio_utils import mel_filter_bank
+        fb = mel_filter_bank(num_frequency_bins=N_FFT // 2 + 1, num_mel_filters=N_MEL,
+                             min_frequency=0.0, max_frequency=SR / 2.0, sampling_rate=SR,
+                             norm="slaney", mel_scale="slaney")          # [801, 80]
+        _mel_fb = np.ascontiguousarray(fb.T, dtype=np.float32)           # [80, 801]
+    return _mel_fb
+
+
+def _log_mel(waveform: np.ndarray) -> np.ndarray:
+    """raw mono waveform -> log10-mel spectrogram [num_frames, 80]."""
+    wav = np.asarray(waveform, dtype=np.float32).reshape(-1)
+    right = math.ceil(wav.shape[0] / HOP) * HOP - wav.shape[0]
+    left = max(N_FFT - HOP, 0)
+    wav = np.pad(wav, (left, right))
+    window = np.hanning(WIN + 1)[:-1].astype(np.float32)                 # periodic Hann
+    n_frames = 1 + (wav.shape[0] - N_FFT) // HOP                          # center=False
+    frames = np.stack([wav[i * HOP: i * HOP + N_FFT] * window for i in range(n_frames)])  # [T, N_FFT]
+    mag = np.abs(np.fft.rfft(frames, n=N_FFT, axis=-1))                   # [T, 801]
+    mag = np.maximum(mag, 1e-10)
+    mel = _mel_filters() @ mag.T                                         # [80, T]
+    mel = np.log10(np.maximum(mel, 1e-10))
+    return mel.T                                                         # [T, 80]
+
+
+def preprocess_audio(waveform: np.ndarray, sampling_rate: int = SR) -> np.ndarray:
+    """raw 16 kHz mono waveform -> dMel bin ids [num_frames, 80] (int32, 0..15)."""
+    if sampling_rate != SR:
+        raise ValueError(f"Inkling audio expects {SR} Hz, got {sampling_rate}")
+    mel = _log_mel(waveform)                                             # [T, 80] log10
+    n_valid = math.ceil(len(np.asarray(waveform).reshape(-1)) / HOP)
+    mel = mel[:n_valid]                                                  # drop trailing pad frames
+    centers = np.linspace(DMEL_MIN, DMEL_MAX, DMEL_BINS)                 # 16 bin centers
+    clamped = np.clip(mel.astype(np.float64), DMEL_MIN, DMEL_MAX)
+    bins = np.abs(clamped[..., None] - centers).argmin(-1)              # nearest center
+    return bins.astype(np.int32)                                        # [T, 80]
+
+
+# --------------------------- prompt assembly ---------------------------
+
+class InklingProcessor:
+    """Assembles multimodal model inputs from chat messages with image/audio parts.
+
+    Content parts: {"type":"text","text":...}, {"type":"image","image":PIL/array},
+    {"type":"audio","audio":waveform, "sampling_rate":16000}.
+    """
+
+    def __init__(self, tokenizer, chat_template: str):
+        self.tok = tokenizer
+        self.chat_template = chat_template
+        self.image_bos_id = tokenizer.encode(IMAGE_BOS, add_special_tokens=False)[0]
+        self.audio_bos_id = tokenizer.encode(AUDIO_BOS, add_special_tokens=False)[0]
+
+    def apply(self, messages, reasoning_effort: str = "none", max_long_edge: int | None = None):
+        import mlx.core as mx
+        pixel_values, audio_ids = [], []
+        # Render text via the chat template with placeholders stripped to a sentinel,
+        # then splice media spans in. We build ids directly for robustness.
+        ids: list[int] = []
+
+        def emit_text(s):
+            ids.extend(self.tok.encode(s, add_special_tokens=False))
+
+        # header: thinking-effort system message (matches chat_template)
+        eff = {"none": 0.0, "minimal": 0.1, "low": 0.2, "medium": 0.7, "high": 0.9, "max": 0.99}[reasoning_effort]
+        emit_text(f"<|message_system|><|content_text|>Thinking effort level: {0 if eff == 0 else eff}<|end_message|>")
+
+        for msg in messages:
+            role = {"user": "<|message_user|>", "assistant": "<|message_model|>",
+                    "system": "<|message_system|>"}[msg["role"]]
+            content = msg["content"]
+            if isinstance(content, str):
+                content = [{"type": "text", "text": content}]
+            for part in content:
+                t = part.get("type", "text")
+                if t == "text":
+                    emit_text(role + "<|content_text|>" + part["text"] + "<|end_message|>")
+                elif t == "image":
+                    pv, n = preprocess_image(part["image"], max_long_edge=max_long_edge)
+                    pixel_values.append(pv)
+                    ids.append(self.tok.encode(role, add_special_tokens=False)[0])
+                    ids.append(self.image_bos_id)
+                    ids.extend([IMAGE_TOKEN_ID] * n)
+                    ids.extend(self.tok.encode("<|end_message|>", add_special_tokens=False))
+                elif t == "audio":
+                    aid = preprocess_audio(part["audio"], part.get("sampling_rate", SR))
+                    audio_ids.append(aid)
+                    ids.append(self.tok.encode(role, add_special_tokens=False)[0])
+                    ids.append(self.audio_bos_id)
+                    ids.extend([AUDIO_TOKEN_ID] * aid.shape[0])
+                    ids.extend(self.tok.encode("<|end_message|>", add_special_tokens=False))
+        emit_text("<|message_model|>")   # generation prompt
+
+        out = {"input_ids": ids}
+        if pixel_values:
+            out["pixel_values"] = mx.array(np.concatenate(pixel_values, axis=0))
+        if audio_ids:
+            out["audio_input_ids"] = mx.array(np.concatenate(audio_ids, axis=0))
+        return out

inkling_mlx/text.py

@@ -0,0 +1,50 @@
+"""Inkling text backbone (``model.llm.*``): token embedding + embed-norm,
+66 decoder layers, final norm, and the (untied) unembed head.
+
+Mirrors ``InklingTextModel`` + the unembed / muP-logit scaling from
+``InklingForConditionalGeneration``.
+"""
+
+from __future__ import annotations
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .common import RMSNorm
+from .config import TextConfig
+from .layers import DecoderLayer
+
+
+class TextModel(nn.Module):
+    def __init__(self, config: TextConfig):
+        super().__init__()
+        self.config = config
+        self.embed = nn.Embedding(config.vocab_size, config.hidden_size)
+        self.embed_norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.layers = [DecoderLayer(config, i) for i in range(config.num_hidden_layers)]
+        self.norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+        self.unembed = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
+
+    def embed_tokens(self, input_ids: mx.array) -> mx.array:
+        return self.embed_norm(self.embed(input_ids))
+
+    def backbone(self, inputs_embeds: mx.array, conv_mask=None, caches=None, start_pos=0) -> mx.array:
+        h = inputs_embeds
+        for i, layer in enumerate(self.layers):
+            h = layer(h, start_pos=start_pos,
+                      cache=caches[i] if caches is not None else None,
+                      conv_mask=conv_mask)
+        return self.norm(h)
+
+    def logits(self, hidden: mx.array) -> mx.array:
+        hidden = hidden / self.config.logits_mup_width_multiplier
+        logits = self.unembed(hidden)
+        uv = self.config.unpadded_vocab_size
+        if uv is not None and uv < logits.shape[-1]:
+            logits = logits[..., :uv]
+        return logits
+
+    def __call__(self, input_ids: mx.array, conv_mask=None) -> mx.array:
+        h = self.embed_tokens(input_ids)
+        h = self.backbone(h, conv_mask=conv_mask)
+        return self.logits(h)

inkling_mlx/vision.py

@@ -0,0 +1,126 @@
+"""Inkling vision tower: an HMLP (hierarchical MLP) patch encoder.
+
+No attention — each layer folds space/time into the channel dim then projects
+(Linear -> RMSNorm -> GELU), progressively growing channels up to the text hidden
+size. Mirrors ``InklingVisionModel`` / ``InklingVisionEncoderLayer`` /
+``plan_out_scales``. Checkpoint keys are flat: ``visual.layers.linear_{i}`` and
+``visual.layers.norm_{i}`` plus ``visual.final_norm``.
+"""
+
+from __future__ import annotations
+
+import math
+
+import mlx.core as mx
+import mlx.nn as nn
+
+from .common import RMSNorm
+from .config import VisionConfig
+
+
+def _prime_factors(n: int) -> list[int]:
+    factors = []
+    while n % 2 == 0:
+        factors.append(2)
+        n //= 2
+    p = 3
+    while p * p <= n:
+        while n % p == 0:
+            factors.append(p)
+            n //= p
+        p += 2
+    if n > 1:
+        factors.append(n)
+    return factors
+
+
+def plan_out_scales(temporal_patch_size: int, patch_size: int, n_layers: int, n_channels: int):
+    """Port of the reference ``plan_out_scales`` (returns an ``(n_layers+1, 4)``
+    array of (t, h, w, c) grid sizes). Uses numpy + scipy for the assignment."""
+    import numpy as np
+    from scipy.optimize import linear_sum_assignment
+
+    h = np.cumprod(np.array(_prime_factors(patch_size)[::-1]))
+    t = np.cumprod(np.array(_prime_factors(temporal_patch_size)[::-1]))
+
+    h_ch = np.ceil(h**2 * n_channels / 64).astype(np.int64) * 64
+    t_ch = (np.ceil(h[-1] ** 2 * n_channels * t)).astype(np.int64) * 64
+
+    base = np.array([[1, 1, 1, n_channels]], dtype=np.int64)
+    spatial = np.stack([np.ones_like(h), h, h, h_ch], axis=1)
+    temporal = np.stack([t, np.full_like(t, h[-1]), np.full_like(t, h[-1]), t_ch], axis=1)
+    scales = np.concatenate([base, spatial, temporal], axis=0).astype(np.int64)
+
+    size_reduction = np.prod(scales[:, :-1], axis=1).astype(np.float64)
+    total_elements = patch_size * patch_size * temporal_patch_size * n_channels
+    log_ideal = np.linspace(0.0, math.log(total_elements), n_layers + 1)
+    cost = np.abs(log_ideal[:, None] - np.log(size_reduction)[None, :])
+
+    if n_layers >= scales.shape[0]:
+        idxs = np.argmin(cost, axis=1)
+    else:
+        _, idxs = linear_sum_assignment(cost)
+    idxs = np.array(idxs)
+    idxs[0] = 0
+    idxs[-1] = scales.shape[0] - 1
+    return scales[idxs]
+
+
+def _fold_timespace_to_depth(x, t_fold, hw_fold):
+    # x: [B, T, H, W, C] -> [B, T//t, H//hw, W//hw, C*t*hw*hw]
+    B, T, H, W, C = x.shape
+    t_new, h_new, w_new = T // t_fold, H // hw_fold, W // hw_fold
+    x = x.reshape(B, t_new, t_fold, h_new, hw_fold, w_new, hw_fold, C)
+    x = x.transpose(0, 1, 3, 5, 2, 4, 6, 7)
+    x = x.reshape(B, t_new, h_new, w_new, t_fold * hw_fold * hw_fold * C)
+    return x
+
+
+class _VisionLayers(nn.Module):
+    """Holds ``linear_{i}`` / ``norm_{i}`` to match checkpoint keys."""
+
+    def __init__(self, config: VisionConfig):
+        super().__init__()
+        scales = plan_out_scales(
+            config.temporal_patch_size, config.patch_size, config.n_layers, config.num_channels
+        )
+        self.n_layers = config.n_layers
+        self.folds = []  # (t_fold, hw_fold, add_norm)
+        for i in range(config.n_layers):
+            start, end = scales[i], scales[i + 1]
+            shuffle = (
+                (end[0] // start[0]) * (end[1] // start[1]) * (end[2] // start[2])
+            )
+            hw_fold = int(end[1] // start[1])
+            t_fold = int(end[0] // start[0])
+            in_dim = int(start[3]) * int(shuffle)
+            add_norm = i != config.n_layers - 1
+            out_dim = config.text_hidden_size if i == config.n_layers - 1 else int(end[3])
+            setattr(self, f"linear_{i}", nn.Linear(in_dim, out_dim, bias=False))
+            if add_norm:
+                setattr(self, f"norm_{i}", RMSNorm(out_dim, eps=config.rms_norm_eps))
+            self.folds.append((t_fold, hw_fold, add_norm))
+
+    def __call__(self, x):
+        for i, (t_fold, hw_fold, add_norm) in enumerate(self.folds):
+            if hw_fold > 1 or t_fold > 1:
+                x = _fold_timespace_to_depth(x, t_fold, hw_fold)
+            x = getattr(self, f"linear_{i}")(x)
+            if add_norm:
+                x = getattr(self, f"norm_{i}")(x)
+                x = nn.gelu(x)
+        return x
+
+
+class VisionModel(nn.Module):
+    def __init__(self, config: VisionConfig):
+        super().__init__()
+        self.config = config
+        self.layers = _VisionLayers(config)
+        self.final_norm = RMSNorm(config.text_hidden_size, eps=config.rms_norm_eps)
+
+    def __call__(self, pixel_values: mx.array) -> mx.array:
+        num_patches = pixel_values.shape[0]
+        h = self.layers(pixel_values)
+        h = self.final_norm(h)
+        return h.reshape(num_patches, -1)

model-00001-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e42734850bdcdbaeedb40b04ec4d5602368fddd11099b5e42398995d986bf21
+size 5594332943

model-00002-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ddfaeba71d13110153129d30e626b31a68c73f39360761d9653670969a3f663f
+size 5000887318

model-00003-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f5c6478694d5c2dea4165f0c618257d28881261a309264e2263a50eb8a16d2c
+size 5580625118

model-00004-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc55b82655c224201a5b54faf3a85d4b9ad36d5ebf9bf5634724b89d332d71c5
+size 5035610204

model-00005-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b521d9ee213d4e60cb7bfd213fc626cd0b0ea442b7fabe8fe80aa3c36fd80e65
+size 5057678969

model-00006-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4759bad1e4af29df75916ca01bea2368204cbe48227edb7486e21663dedc4164
+size 5033513658

model-00007-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4c8e4016a34b788ff9f72367c299dd5a7c06ce269199685bce9532b49b0b52d
+size 5011868077

model-00008-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a21f18fad8207cf2c5f7d9acd2888a15b9247ab12d8c29df1f812fc1e048bea9
+size 5019857728

model-00009-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:900765f2db646433da17ffbe77e1153ae7b699a178d9ef41f2d6855cb4f7d4cf
+size 5001781520

model-00010-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee46ca2cdda187a133021be9ada09039acb309328708d608575610634cc4ee46
+size 5661531108

model-00011-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d5fb7c4b974ecb52ced7cbffb816f50c892012e04b53584393ee73cce913497b
+size 5795660286

model-00012-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c843327ff1a2abcf69d38324ae46b370f806780560981f2f68e84a717e9d3aa0
+size 5039732129

model-00013-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c1f97a0adf24967d92988f60a4c2cf110a6d22340380c7fce056f5ed7dd70d7
+size 5617638322

model-00014-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:81fa417091e2a5472c93d94d51c4dfab6b46d50542a769cab034946cae19181d
+size 5628885415

model-00015-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69bc0de3d94e5bd99b18154c4eb1c1228bb3dd51f55270e141cdf17bc2c670e9
+size 5267393483

model-00016-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52712f0e018f906c2a2e0406a39f8d4d3e0e380145739551731370385698019c
+size 5009388976

model-00017-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe1c46b6d45fa427ebe4c8852e511f961a42aa1e5c320127d685d9f051bd7373
+size 5650057900

model-00018-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e2f96c8ac308f6027252bfe0b3369841948c8048473f3dac6bf0398e60562b99
+size 5060808586

model-00019-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4040ecd3725b5ce786fbd708e1686295d27cce7921868c134a613843f38b8f0a
+size 5797784251

model-00020-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52639817b466372de4e030cfd76c17ff574d643611e0e4dbf97b871da37148f7
+size 5031172621

model-00021-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b6a1d3b7894320c5c07388c9bccbf106f21d452e055c5bd0194d0f2e891935e
+size 5281288361

model-00022-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c03d985bd8761e409f8f40ad5cc76c048a4a7f7c767484bd7ae8d24a9319d61
+size 5683582825

model-00023-of-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45dbe282191d5afeec100f4e6826fc556d8738e8b077555a089cb99ac0021633
+size 4051827493

processor_config.json

@@ -0,0 +1,46 @@
+{
+  "audio_token": "<|unused_200053|>",
+  "audio_bos_token": "<|content_audio_input|>",
+  "dmel_max_value": 2.0,
+  "dmel_min_value": -7.0,
+  "feature_extractor": {
+    "audio_token_duration_s": 0.05,
+    "feature_extractor_type": "InklingFeatureExtractor",
+    "feature_size": 80,
+    "hop_length": 800,
+    "n_fft": 1600,
+    "padding_side": "right",
+    "padding_value": 0.0,
+    "return_attention_mask": true,
+    "sampling_rate": 16000,
+    "window_size": 1600,
+    "window_size_multiplier": 2.0
+  },
+  "image_processor": {
+    "do_convert_rgb": true,
+    "do_normalize": true,
+    "do_rescale": true,
+    "do_resize": true,
+    "image_mean": [
+      0.48145466,
+      0.4578275,
+      0.40821073
+    ],
+    "image_processor_type": "InklingImageProcessor",
+    "image_std": [
+      0.26862954,
+      0.26130258,
+      0.27577711
+    ],
+    "resample": 3,
+    "rescale_factor": 0.00392156862745098,
+    "size": {
+      "height": 40,
+      "width": 40
+    }
+  },
+  "image_token": "<|unused_200054|>",
+  "image_bos_token": "<|content_image|>",
+  "num_dmel_bins": 16,
+  "processor_class": "InklingProcessor"
+}

special_tokens_map.json

@@ -0,0 +1,22 @@
+{
+  "additional_special_tokens": [
+    "<|endoftext|>",
+    "<|message_user|>",
+    "<|message_model|>",
+    "<|message_system|>",
+    "<|message_tool|>",
+    "<|content_text|>",
+    "<|content_image|>",
+    "<|content_model_end_sampling|>",
+    "<|content_thinking|>",
+    "<|end_message|>",
+    "<|content_audio_input|>",
+    "<|content_tool_error|>",
+    "<|audio|>",
+    "<|content_xml|>",
+    "<|begin_of_text|>",
+    "<|audio_end|>",
+    "<|content_invoke_tool_json|>",
+    "<|content_invoke_tool_text|>"
+  ]
+}

tiktoken/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc253fd2b702f7a6da7105eaa8f3463b2f1247e83614f23e5323b921088bed2a
+size 3615874

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9fb6333a7db8fe5da90728e741e4a3ee4ac2ae12c5dd4958cc6f31688787d3c2
+size 27875797