v1.1: CPU support — pure-PyTorch lamba_cpu.py (decode-cache, no Triton) + lamba_rag.py --device cpu + README/notebook

LAMBA_Inference.ipynb

@@ -3,17 +3,7 @@
   {
    "cell_type": "markdown",
    "metadata": {},
-   "source": [
-    "# LAMBA V1.0 — Quickstart (Colab GPU)\n",
-    "\n",
-    "A ~177M from-scratch **Mamba-3 + GQA** hybrid for **English + Turkish**.\n",
-    "\n",
-    "> ⚠️ LAMBA is small (177M). It **hallucinates** facts on its own — use it **with retrieval (RAG)**, which this notebook does by default.\n",
-    "\n",
-    "**Runtime → Change runtime type → GPU** (free T4 tier works). Then run the cells in order.\n",
-    "\n",
-    "💛 Support bigger open LAMBA models: https://www.patreon.com/c/kdirgul/membership"
-   ]
+   "source": "# LAMBA V1.0 — Quickstart (Colab: GPU or CPU)\n\nA ~177M from-scratch **Mamba-3 + GQA** hybrid for **English + Turkish**.\n\n> ⚠️ LAMBA is small (177M). It **hallucinates** facts on its own — use it **with retrieval (RAG)**, which this notebook does by default.\n\n**GPU (fastest):** Runtime → Change runtime type → GPU (free T4 works), then run cells 1–3.\n**CPU (v1.1, no GPU):** skip the wheel — jump to the **Run on CPU** cell below.\n\n💛 Support bigger open LAMBA models: https://www.patreon.com/c/kdirgul/membership"
   },
   {
    "cell_type": "code",
@@ -54,16 +44,20 @@
   },
   {
    "cell_type": "markdown",
+   "source": "## Run on CPU (no GPU) — v1.1\n\nNo GPU? LAMBA v1.1 ships a **pure-PyTorch** path (`lamba_cpu.py`, decode-cached, no Triton/CUDA, **no wheel needed**). Set the runtime to **CPU** and run the cell below with `--device cpu`. Expect ~6 tok/s on a Colab CPU; GPU is faster for interactive use.",
+   "metadata": {}
+  },
+  {
+   "cell_type": "code",
+   "source": "# ───────────── Run on CPU (no GPU / Triton / wheel) — v1.1 ─────────────\n!pip -q install torch sentencepiece \"huggingface_hub>=0.23\" sentence-transformers\nfrom huggingface_hub import snapshot_download\nREPO = \"kdirgul/Mamba3-177M-GQA-Hybrid_LAMBA_V1.0\"\nDIR = snapshot_download(REPO)             # public repo, no token needed\n!cd {DIR} && python lamba_rag.py --device cpu \\\n  --tokenizer tokenizer/tokenizer.model \\\n  --ckpt checkpoints/lamba_v1.pt \\\n  --temperature 0 --top_k 1 \\\n  --query \"Türkiye'nin başkenti neresi?\"",
    "metadata": {},
-   "source": [
-    "## Notes\n",
-    "- **Use RAG** for any factual question. Without context, LAMBA may invent answers.\n",
-    "- Works in **English and Turkish**; reasoning/CoT is strongest in English.\n",
-    "- GPU is required for now (Mamba-3 Triton kernel). A CPU build is planned for **v1.1**.\n",
-    "- Full details, eval scores, and limitations: see the **model card (README)**.\n",
-    "\n",
-    "Built by **Kadir Gül**. If LAMBA is useful to you, consider supporting compute for the next version 🙏"
-   ]
+   "execution_count": null,
+   "outputs": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": "## Notes\n- **Use RAG** for any factual question. Without context, LAMBA may invent answers.\n- Works in **English and Turkish**; reasoning/CoT is strongest in English.\n- **Runs on GPU** (Triton kernel, fastest) **or CPU** (`--device cpu`, pure-PyTorch, v1.1 — no wheel).\n- Full details, eval scores, and limitations: see the **model card (README)**.\n\nBuilt by **Kadir Gül**. If LAMBA is useful to you, consider supporting compute for the next version 🙏"
   }
  ],
  "metadata": {

README.md

@@ -18,7 +18,7 @@ pipeline_tag: text-generation
 library_name: mamba_ssm
 ---
 
-# Mamba-3 + GQA Hybrid (EN + TR) - LAMBA V1.0
+# LAMBA V1.0 — Mamba-3 + GQA Hybrid (EN + TR)
 
 **LAMBA V1.0** is a ~177M-parameter language model **trained from scratch** on a
 **Mamba-3 (SISO) + Grouped-Query Attention** hybrid architecture, for **English (primary)
@@ -30,6 +30,9 @@ instruction tuning, and retrieval-augmented tuning — with no warm-start from a
 > mimarili, **İngilizce (birincil) ve Türkçe (ikincil)** bir dil modelidir. Hiçbir hazır
 > modelden başlatılmadı; tokenizer'dan eğitime tüm hat baştan kuruldu.
 
+> **Update (v1.1): now runs on CPU** — a pure-PyTorch port (`lamba_cpu.py`, no Triton/CUDA)
+> with a decode cache. Run anywhere with `--device cpu`; see [How to run](#how-to-run).
+
 ---
 
 ## ⚠️ Read this first — what LAMBA is and isn't
@@ -56,7 +59,7 @@ foundation to grow from. Treat its free-form factual claims as unreliable.
 | MLP | GatedMLP, inner 1500 |
 | Vocab | 48,000 SentencePiece BPE (EN+TR, byte-fallback, digit-split) |
 | Context | 2048 |
-| Precision | bf16 |
+| Precision | bf16 (GPU) / fp32 (CPU) |
 
 The hybrid puts attention layers in the depth-middle so induction heads can form, with Mamba-3
 handling the bulk linear-time sequence mixing.
@@ -93,20 +96,41 @@ why LAMBA is designed to be used **with retrieval**.
 - **Not good for:** standalone factual Q&A, math, multi-step reasoning, anything safety-critical.
 - **Bias/safety:** trained on web/instruction data; may produce incorrect, biased, or unsafe
   content. Always keep a human in the loop. Do not use for medical/legal/financial decisions.
-- **Compute:** the Mamba-3 forward uses a **Triton GPU kernel** → currently **requires a GPU**
-  (Google Colab works, free tier included). A pure-PyTorch CPU port is planned for **v1.1**.
+- **Compute:** runs on **CPU** (pure-PyTorch, `--device cpu`, no GPU/Triton needed — v1.1) or
+  **GPU** (Triton Mamba-3 kernel, fastest). On a desktop CPU the decode-cached path does
+  ~6 tok/s; GPU is recommended for interactive use.
+
+## How to run
+
+The model is designed to be used **with retrieval (RAG)** — `lamba_rag.py` does
+document → embed → retrieve → grounded answer.
+
+### CPU — no GPU, no Triton (v1.1)
 
-## How to run (Colab GPU)
+```bash
+pip install torch sentencepiece sentence-transformers "huggingface_hub>=0.23"
+# pure-PyTorch + decode-cache; NO wheel needed
+python lamba_rag.py --device cpu \
+  --ckpt checkpoints/lamba_v1.pt --tokenizer tokenizer/tokenizer.model \
+  --demo --query "Türkiye'nin başkenti neresi?"
+```
+
+For raw generation without retrieval deps, `lamba_cpu.py` runs standalone:
+```bash
+python lamba_cpu.py --ckpt checkpoints/lamba_v1.pt \
+  --tokenizer tokenizer/tokenizer.model --query "..."
+```
 
-```python
-# 1) install the Mamba-3 fork wheel (provided in this repo under wheels/)
-!pip -q install einops sentencepiece "huggingface_hub>=0.23"
-!pip -q install --no-deps ./wheels/*.whl
+### GPU — fastest (Triton Mamba-3 kernel)
 
-# 2) run retrieval-augmented inference (recommended)
-!python faz7_rag.py --demo --query "Türkiye'nin başkenti neresi?"
-# or your own documents:
-!python faz7_rag.py --docs /content/my_docs --query "..."
+```bash
+pip install einops sentencepiece sentence-transformers "huggingface_hub>=0.23"
+pip install --no-deps ./wheels/*.whl                    # Mamba-3 fork wheel (this repo)
+python lamba_rag.py --device cuda \
+  --ckpt checkpoints/lamba_v1.pt --tokenizer tokenizer/tokenizer.model \
+  --demo --query "Türkiye'nin başkenti neresi?"
+# your own documents:
+python lamba_rag.py --device cuda --docs ./my_docs --query "..."
 ```
 
 See `LAMBA_Inference.ipynb` for a one-click Colab notebook.
@@ -115,8 +139,9 @@ See `LAMBA_Inference.ipynb` for a one-click Colab notebook.
 
 - `checkpoints/` — model weights (LAMBA V1.0 = the final SFT+RAG checkpoint)
 - `tokenizer/` — 48K SentencePiece model
-- `wheels/` — prebuilt Mamba-3 fork wheels (GPU)
-- `*.py` — inference + RAG code
+- `lamba_rag.py` — retrieval-augmented inference (`--device cpu` or `cuda`)
+- `lamba_cpu.py` — pure-PyTorch model + decode-cache (no Triton/CUDA; powers the CPU path)
+- `wheels/` — prebuilt Mamba-3 fork wheels (**GPU path only** — not needed for CPU)
 - `LAMBA_Inference.ipynb` — Colab quickstart
 
 ## License
@@ -133,13 +158,7 @@ goes directly to GPU hours and data:
 
 - **Patreon:** https://www.patreon.com/c/kdirgul/membership
 
-- **BSC (BEP20) BNB SMART CHAIN**
-0x37129fc539059dd997c4b36129033cdafd736b73
-
-- **ETH (ERC20) ETHEREUM NETWORK**
-0x37129fc539059dd997c4b36129033cdafd736b73
-
- Thank you. 🙏
+(More options, including crypto, coming later.) Thank you. 🙏
 
 ## Citation
 
@@ -152,7 +171,27 @@ goes directly to GPU hours and data:
 }
 ```
 
+## Related Mamba-3 models
+
+LAMBA is part of a recent wave of from-scratch Mamba-3 models. For context:
+
+| Model | Size | Notes |
+|---|---|---|
+| [hrsvrn/mamba3-180m-finewebedu-10B](https://huggingface.co/hrsvrn/mamba3-180m-finewebedu-10B) | ~187M | EN, norm-free Mamba-3 SISO baseline — **LAMBA's recipe reference** |
+| [ib-ssm/mamba3-370M-10BT](https://huggingface.co/ib-ssm/mamba3-370M-10BT) | 370M | EN, Mamba-3 base |
+| [kikyo0114/nanochat-mamba3-mimo-r2](https://huggingface.co/kikyo0114/nanochat-mamba3-mimo-r2) | 113M | EN+JP, pure-PyTorch Mamba-3 **MIMO** chat |
+| [RtaForge/Mamba3-2.7B](https://huggingface.co/RtaForge/Mamba3-2.7B) | 2.7B | Mamba-2 → Mamba-3 migrated (alpha) |
+| [RtaForge/Mistral-Mamba3-7B](https://huggingface.co/RtaForge/Mistral-Mamba3-7B) | ~4B | Mistral → Mamba-3 transfer (alpha) |
+| [batteryphil/mamba3-baremetal-rlf](https://huggingface.co/batteryphil/mamba3-baremetal-rlf) | ~130M | experimental bare-metal reasoning |
+
+**What makes LAMBA different:** to our knowledge the only **bilingual English + Turkish** Mamba-3
+model; one of the few **Mamba-3 + attention hybrids** (most are pure SSM); and it ships
+**instruction- and RAG-tuned**, not as a base/alpha checkpoint.
+
 ## Acknowledgements
 
-Architecture builds on **Mamba-3** (state-spaces) and the `mamba-og` fork. Tokenizer/data/
-training recipe and all weights are original to this project.
+Architecture builds on **Mamba-3** (state-spaces) and the `mamba-og` fork. The training recipe
+and the feasibility of the ~180M / 768-d / 12-layer band were directly informed by
+**[hrsvrn/mamba3-180m](https://huggingface.co/hrsvrn/mamba3-180m-finewebedu-10B)** (the norm-free
+Mamba-3 project) — thank you. Tokenizer, data, training recipe, and all weights are original to
+this project.

lamba_cpu.py

@@ -0,0 +1,375 @@
+"""
+LAMBA V1.0 — saf-PyTorch CPU inference (Triton/mamba_ssm YOK). [v1.1]
+
+Fork (mamba_ssm "norm-free" Mamba-3, Triton kernel) ile EĞİTİLEN ağırlıkları (lamba_v1.pt)
+GPU'suz çalıştırmak için fork forward matematiğinin saf-PyTorch reimplementasyonu.
+
+Mamba-3 mixer matematiği (vendor mamba3.py + siso_step kernelinden çıkarıldı), per-head:
+  _A   = -softplus(dd_A)  (clamp ≤ -1e-4)      # ← data-dependent A (norm-free farkı)
+  DT   = softplus(dd_dt + dt_bias)
+  trap = sigmoid(trap_proj)
+  α    = exp(_A·DT)       # base-e ✅ (kernel exp2(ön-ölçekli adt) ile birebir)
+  β    = α·DT·(1-trap) ;  γ = trap·DT          # trapezoidal
+  h    = α·h + β·(x_prev ⊗ B_prev) + γ·(x ⊗ B) # B,x = K,V ; B üzerinde partial-RoPE
+  y    = h @ C            # C = Q ; C üzerinde partial-RoPE
+  y   += D·x   ;  y *= silu(z)
+B,C paylaşımlı (ngroups=1) → head'lere broadcast + per-head bias. RoPE: rope_fraction=0.5
+(ilk d_state/2=64 boyut, 32 angle, INTERLEAVED [çift (2j,2j+1)]). Token-token recurrence + decode-cache (step).
+
+✅ KALİBRE TAMAM (2026-06-27): fork'a full-logit fp32 maxdiff 0.06–0.09 (top-5/argmax birebir).
+   Kritik düzeltme MLP'deydi: GatedMLP = y·silu(gate) (1.yarı=değer, 2.yarı=gate; mixer değil).
+   Decode-cache (step) full-recompute ile birebir aynı çıktı, ~6× hızlı (O(L²)→O(L)).
+
+Kullanım (CPU):  python lamba_cpu.py --ckpt checkpoints/lamba_v1.pt --tokenizer tokenizer/tokenizer.model --query "..."
+"""
+import os, sys, math, argparse
+import torch, torch.nn as nn, torch.nn.functional as F
+
+torch.set_num_threads(max(1, os.cpu_count() or 4))
+
+
+def rms_norm(x, w, eps=1e-5):
+    return x * torch.rsqrt(x.pow(2).mean(-1, keepdim=True) + eps) * w
+
+
+# ───────────── Mamba-3 mixer (saf-PyTorch, fork matematiği) ─────────────
+class Mamba3CPU(nn.Module):
+    # KALİBRE bayrakları (Colab grid-search ile fork'a karşı belirlenir)
+    ROPE_INTER = True    # fork rotate_pairwise=True ⇒ interleaved (q[0::2], q[1::2])
+    BETA_ALPHA = True    # KALİBRE: β'da alpha çarpanı (grid: True daha iyi)
+    BETA_SHIFT = False   # KALİBRE: β'da dt/trap kaydırılmış mı (grid: False daha iyi)
+    EXP_E = True         # ✅ ÇÖZÜLDÜ: α = exp(_A·DT) base-e (exp2 değil) — mixer kalibre (maxdiff 0.14 bf16)
+
+    def __init__(self, cfg):
+        super().__init__()
+        d = cfg["d_model"]
+        self.d_inner = cfg["expand"] * d                 # 1536
+        self.headdim = cfg["head_dim"]                   # 64
+        self.nheads = self.d_inner // self.headdim       # 24
+        self.d_state = cfg["d_state"]                    # 128
+        self.ngroups = cfg.get("ngroups", 1)             # num_bc_heads = 1
+        self.A_floor = 1e-4
+        rope_fraction = cfg.get("rope_fraction", 0.5)
+        self.rot = int(self.d_state * rope_fraction)     # 64 → döner boyut
+        if self.rot % 2:
+            self.rot -= 1
+        self.n_ang = self.rot // 2                        # 32 angle pair
+        bc = self.d_state * self.ngroups                 # 128 (SISO, rank=1)
+        d_in = 2 * self.d_inner + 2 * bc + 3 * self.nheads + self.n_ang  # 3432
+        self.in_proj = nn.Linear(d, d_in, bias=False)
+        self.out_proj = nn.Linear(self.d_inner, d, bias=False)
+        self.dt_bias = nn.Parameter(torch.zeros(self.nheads))
+        self.D = nn.Parameter(torch.ones(self.nheads))
+        self.B_bias = nn.Parameter(torch.zeros(self.nheads, 1, self.d_state))
+        self.C_bias = nn.Parameter(torch.zeros(self.nheads, 1, self.d_state))
+        self.B_norm = nn.Parameter(torch.ones(bc))
+        self.C_norm = nn.Parameter(torch.ones(bc))
+
+    def _rope(self, t, cos, sin):
+        """Partial RoPE (ilk `rot`=d_state/2 boyut döner; gerisi sabit).
+        pairwise: j ↔ j+n çiftleri | interleaved: (2j, 2j+1) çiftleri."""
+        rot, n = self.rot, self.n_ang
+        rest = t[..., rot:]
+        if Mamba3CPU.ROPE_INTER:
+            head = t[..., :rot]
+            x1, x2 = head[..., 0::2], head[..., 1::2]
+            ra, rb = x1 * cos - x2 * sin, x1 * sin + x2 * cos
+            out = torch.stack([ra, rb], dim=-1).flatten(-2)
+            return torch.cat([out, rest], dim=-1)
+        a, b = t[..., :n], t[..., n:rot]
+        ra, rb = a * cos - b * sin, a * sin + b * cos
+        return torch.cat([ra, rb, rest], dim=-1)
+
+    def forward(self, u):
+        """u: (B, L, d_model) → (B, L, d_model). Token-token recurrence."""
+        B, L, _ = u.shape
+        H, P, S = self.nheads, self.headdim, self.d_state
+        proj = self.in_proj(u)
+        z, x, Bm, Cm, dd_dt, dd_A, trap, ang = torch.split(
+            proj, [self.d_inner, self.d_inner, S * self.ngroups, S * self.ngroups,
+                    H, H, H, self.n_ang], dim=-1)
+        x = x.view(B, L, H, P)
+        z = z.view(B, L, H, P)
+        _A = (-F.softplus(dd_A.float())).clamp(max=-self.A_floor)        # (B,L,H)
+        DT = F.softplus(dd_dt.float() + self.dt_bias)                    # (B,L,H)
+        trap = torch.sigmoid(trap.float())                              # (B,L,H)
+        Bm = rms_norm(Bm.float(), self.B_norm)                          # (B,L,S) ngroups=1
+        Cm = rms_norm(Cm.float(), self.C_norm)
+        Bm = Bm.view(B, L, 1, S) + self.B_bias.view(1, 1, H, S)         # head'e broadcast + bias
+        Cm = Cm.view(B, L, 1, S) + self.C_bias.view(1, 1, H, S)
+        alpha = torch.exp(_A * DT) if Mamba3CPU.EXP_E else torch.exp2(_A * DT)   # base-e vs base-2 decay
+        bDT = torch.roll(DT, 1, dims=1) if Mamba3CPU.BETA_SHIFT else DT          # β'da kaydırılmış dt/trap?
+        btrap = torch.roll(trap, 1, dims=1) if Mamba3CPU.BETA_SHIFT else trap
+        beta = (alpha if Mamba3CPU.BETA_ALPHA else 1.0) * bDT * (1 - btrap)
+        gamma = trap * DT
+
+        # KALİBRE: angle birikimi (işaret/dt-ölçeği) — ilk tahmin: cum += DT·angles
+        h = torch.zeros(B, H, P, S)            # ssm_state (V,QK) = (P,S)
+        x_prev = torch.zeros(B, H, P)
+        Bk_prev = torch.zeros(B, H, S)
+        cum = torch.zeros(B, H, self.n_ang)
+        ys = []
+        for t in range(L):
+            # fork: angle = angle_state + tanh(angle_proj)·DT·π  (mamba3_mimo_rotary_step referans)
+            inc = torch.tanh(ang[:, t].float()).unsqueeze(1) * DT[:, t].unsqueeze(-1) * math.pi
+            cum = cum + inc                                              # (B,H,n_ang)
+            cos, sin = torch.cos(cum), torch.sin(cum)
+            Bk = self._rope(Bm[:, t], cos, sin)                         # (B,H,S)
+            Cq = self._rope(Cm[:, t], cos, sin)
+            xt = x[:, t]                                                # (B,H,P)
+            a = alpha[:, t].view(B, H, 1, 1)
+            diff = (beta[:, t].view(B, H, 1, 1) * x_prev.unsqueeze(-1) * Bk_prev.unsqueeze(-2)
+                    + gamma[:, t].view(B, H, 1, 1) * xt.unsqueeze(-1) * Bk.unsqueeze(-2))
+            h = h * a + diff                                            # (B,H,P,S)
+            y = (h * Cq.unsqueeze(-2)).sum(-1)                          # (B,H,P)
+            y = y + self.D.view(1, H, 1) * xt
+            y = y * F.silu(z[:, t])
+            ys.append(y.reshape(B, 1, H * P))
+            x_prev, Bk_prev = xt, Bk
+        return self.out_proj(torch.cat(ys, dim=1).to(self.out_proj.weight.dtype))
+
+    # ───── decode-cache (tek-token step; forward'ın bir iterasyonu, O(1)) ─────
+    def init_state(self, B, device=None, dtype=torch.float32):
+        H, P, S = self.nheads, self.headdim, self.d_state
+        z = lambda *s: torch.zeros(*s, device=device, dtype=dtype)
+        return [z(B, H, P, S), z(B, H, P), z(B, H, S), z(B, H, self.n_ang)]  # h, x_prev, Bk_prev, cum
+
+    def step(self, u, state):
+        """u:(B,d_model), state=[h,x_prev,Bk_prev,cum] → (y:(B,d_model), yeni_state)."""
+        B = u.shape[0]
+        H, P, S = self.nheads, self.headdim, self.d_state
+        h, x_prev, Bk_prev, cum = state
+        z, x, Bm, Cm, dd_dt, dd_A, trap, ang = torch.split(
+            self.in_proj(u), [self.d_inner, self.d_inner, S * self.ngroups, S * self.ngroups,
+                              H, H, H, self.n_ang], dim=-1)
+        x = x.view(B, H, P); z = z.view(B, H, P)
+        _A = (-F.softplus(dd_A.float())).clamp(max=-self.A_floor)
+        DT = F.softplus(dd_dt.float() + self.dt_bias)
+        trap = torch.sigmoid(trap.float())
+        Bm = rms_norm(Bm.float(), self.B_norm).view(B, 1, S) + self.B_bias.view(1, H, S)
+        Cm = rms_norm(Cm.float(), self.C_norm).view(B, 1, S) + self.C_bias.view(1, H, S)
+        alpha = torch.exp(_A * DT) if Mamba3CPU.EXP_E else torch.exp2(_A * DT)
+        beta = (alpha if Mamba3CPU.BETA_ALPHA else 1.0) * DT * (1 - trap)        # BETA_SHIFT=False
+        gamma = trap * DT
+        cum = cum + torch.tanh(ang.float()).unsqueeze(1) * DT.unsqueeze(-1) * math.pi
+        cos, sin = torch.cos(cum), torch.sin(cum)
+        Bk = self._rope(Bm, cos, sin); Cq = self._rope(Cm, cos, sin)
+        diff = (beta.view(B, H, 1, 1) * x_prev.unsqueeze(-1) * Bk_prev.unsqueeze(-2)
+                + gamma.view(B, H, 1, 1) * x.unsqueeze(-1) * Bk.unsqueeze(-2))
+        h = h * alpha.view(B, H, 1, 1) + diff
+        y = (h * Cq.unsqueeze(-2)).sum(-1) + self.D.view(1, H, 1) * x
+        y = (y * F.silu(z)).reshape(B, H * P)
+        return self.out_proj(y.to(self.out_proj.weight.dtype)), [h, x, Bk, cum]
+
+
+# ───────────── GQA mixer (saf-PyTorch; hybrid_mamba3 ile aynı) ─────────────
+def _rot_half(x):
+    a, b = x.chunk(2, -1)
+    return torch.cat((-b, a), -1)
+
+
+class GQACPU(nn.Module):
+    def __init__(self, cfg, base=10000.0):
+        super().__init__()
+        d = cfg["d_model"]; self.nh = cfg["n_heads"]; self.nkv = cfg["n_kv_heads"]
+        self.hd = d // self.nh; self.rep = self.nh // self.nkv
+        self.q_proj = nn.Linear(d, self.nh * self.hd, bias=False)
+        self.k_proj = nn.Linear(d, self.nkv * self.hd, bias=False)
+        self.v_proj = nn.Linear(d, self.nkv * self.hd, bias=False)
+        self.out_proj = nn.Linear(self.nh * self.hd, d, bias=False)
+        self.qn = nn.Parameter(torch.ones(self.hd))
+        self.kn = nn.Parameter(torch.ones(self.hd))
+        self.register_buffer("inv", 1.0 / (base ** (torch.arange(0, self.hd, 2).float() / self.hd)), persistent=False)
+
+    def _rope(self, x, T):
+        f = torch.outer(torch.arange(T).float(), self.inv); e = torch.cat((f, f), -1)
+        return (x * e.cos()[None, None] + _rot_half(x) * e.sin()[None, None]).to(x.dtype)
+
+    def forward(self, x):
+        B, T, _ = x.shape
+        q = self.q_proj(x).view(B, T, self.nh, self.hd).transpose(1, 2)
+        k = self.k_proj(x).view(B, T, self.nkv, self.hd).transpose(1, 2)
+        v = self.v_proj(x).view(B, T, self.nkv, self.hd).transpose(1, 2)
+        q = rms_norm(q.float(), self.qn.float()).to(x.dtype)
+        k = rms_norm(k.float(), self.kn.float()).to(x.dtype)
+        q, k = self._rope(q, T), self._rope(k, T)
+        k = k.repeat_interleave(self.rep, 1); v = v.repeat_interleave(self.rep, 1)
+        y = F.scaled_dot_product_attention(q, k, v, is_causal=True)
+        return self.out_proj(y.transpose(1, 2).reshape(B, T, -1))
+
+    # ───── decode-cache (KV-cache; q tek token vs tüm geçmiş = nedensel) ─────
+    def init_state(self, B, device=None, dtype=torch.float32):
+        return [None, None]                                          # k_cache, v_cache (B,nkv,t,hd)
+
+    def _rope_at(self, x, pos):
+        f = (self.inv * float(pos)).unsqueeze(0)                     # (1, hd/2)
+        e = torch.cat((f, f), -1)
+        return (x * e.cos()[None, None] + _rot_half(x) * e.sin()[None, None]).to(x.dtype)
+
+    def step(self, x, state):
+        """x:(B,d_model), state=[k_cache,v_cache] → (y:(B,d_model), yeni_state)."""
+        B = x.shape[0]; kc, vc = state
+        pos = 0 if kc is None else kc.shape[2]
+        xq = x.view(B, 1, -1)
+        q = self.q_proj(xq).view(B, 1, self.nh, self.hd).transpose(1, 2)
+        k = self.k_proj(xq).view(B, 1, self.nkv, self.hd).transpose(1, 2)
+        v = self.v_proj(xq).view(B, 1, self.nkv, self.hd).transpose(1, 2)
+        q = rms_norm(q.float(), self.qn.float()).to(x.dtype)
+        k = rms_norm(k.float(), self.kn.float()).to(x.dtype)
+        q = self._rope_at(q, pos); k = self._rope_at(k, pos)
+        kc = k if kc is None else torch.cat([kc, k], dim=2)
+        vc = v if vc is None else torch.cat([vc, v], dim=2)
+        kk = kc.repeat_interleave(self.rep, 1); vv = vc.repeat_interleave(self.rep, 1)
+        y = F.scaled_dot_product_attention(q, kk, vv, is_causal=False)
+        return self.out_proj(y.transpose(1, 2).reshape(B, -1)), [kc, vc]
+
+
+class GatedMLP(nn.Module):
+    def __init__(self, cfg):
+        super().__init__()
+        d = cfg["d_model"]
+        mult = 128                                    # mamba_ssm GatedMLP: hidden 128'in katına yuvarlar
+        hidden = ((cfg["d_intermediate"] + mult - 1) // mult) * mult   # 1500 → 1536
+        self.fc1 = nn.Linear(d, 2 * hidden, bias=False)
+        self.fc2 = nn.Linear(hidden, d, bias=False)
+
+    def forward(self, x):
+        # mamba_ssm GatedMLP: 1. yarı = değer, 2. yarı = gate → y * silu(gate)
+        y, gate = self.fc1(x).chunk(2, -1)
+        return self.fc2(y * F.silu(gate))
+
+
+class Block(nn.Module):
+    def __init__(self, cfg, is_attn):
+        super().__init__()
+        self.norm = nn.Parameter(torch.ones(cfg["d_model"]))
+        self.norm2 = nn.Parameter(torch.ones(cfg["d_model"]))
+        self.mixer = GQACPU(cfg) if is_attn else Mamba3CPU(cfg)
+        self.mlp = GatedMLP(cfg)
+
+    def forward(self, x):
+        x = x + self.mixer(rms_norm(x, self.norm))
+        x = x + self.mlp(rms_norm(x, self.norm2))
+        return x
+
+    def init_state(self, B, **kw):
+        return self.mixer.init_state(B, **kw)
+
+    def step(self, x, mstate):
+        m_out, mstate = self.mixer.step(rms_norm(x, self.norm), mstate)
+        x = x + m_out
+        x = x + self.mlp(rms_norm(x, self.norm2))
+        return x, mstate
+
+
+class LambaCPU(nn.Module):
+    def __init__(self, cfg):
+        super().__init__()
+        self.cfg = cfg
+        self.embedding = nn.Embedding(cfg["vocab_size"], cfg["d_model"])
+        self.layers = nn.ModuleList()
+        for i in range(cfg["n_layers"]):
+            is_attn = ((i + 1) % cfg["attn_every"] == 0) and i != 0 and i != cfg["n_layers"] - 1
+            self.layers.append(Block(cfg, is_attn))
+        self.norm_f = nn.Parameter(torch.ones(cfg["d_model"]))
+        self.lm_head = nn.Linear(cfg["d_model"], cfg["vocab_size"], bias=False)
+        self.lm_head.weight = self.embedding.weight
+
+    def forward(self, ids):
+        h = self.embedding(ids)
+        for l in self.layers:
+            h = l(h)
+        return self.lm_head(rms_norm(h, self.norm_f))
+
+    def init_states(self, B, **kw):
+        return [l.init_state(B, **kw) for l in self.layers]
+
+    @torch.no_grad()
+    def step_forward(self, ids_step, states):
+        """ids_step:(B,1) tek token. → (logits:(B,V), yeni_states). O(1) Mamba + O(t) GQA-attn."""
+        h = self.embedding(ids_step)[:, 0]                          # (B,d)
+        new_states = []
+        for l, stt in zip(self.layers, states):
+            h, ns = l.step(h, stt)
+            new_states.append(ns)
+        return self.lm_head(rms_norm(h, self.norm_f)), new_states
+
+
+# ───────────── ağırlık yükleyici (fork ckpt → CPU model) ─────────────
+def load_lamba(ckpt_path):
+    st = torch.load(ckpt_path, map_location="cpu")
+    cfg, sd = st["cfg"], st["model"]
+    model = LambaCPU(cfg)
+    mp = {}
+    for k, v in sd.items():
+        nk = k
+        nk = nk.replace(".mixer.in_proj.", ".mixer.in_proj.").replace(".mixer.out_proj.", ".mixer.out_proj.")
+        # GQA fork→CPU isim eşlemesi zaten birebir (q_proj/k_proj/v_proj/out_proj/qn/kn)
+        # Mamba B_norm.weight/C_norm.weight → B_norm/C_norm (Parameter)
+        nk = nk.replace(".mixer.B_norm.weight", ".mixer.B_norm").replace(".mixer.C_norm.weight", ".mixer.C_norm")
+        nk = nk.replace("norm_f.weight", "norm_f")
+        nk = nk.replace(".norm.weight", ".norm").replace(".norm2.weight", ".norm2")
+        mp[nk] = v
+    # norm_f / lm_head / embedding
+    miss, unexp = model.load_state_dict(mp, strict=False)
+    print(f"[load] eksik={len(miss)} beklenmeyen={len(unexp)}", flush=True)
+    if miss:
+        print("  ilk eksikler:", miss[:8])
+    if unexp:
+        print("  ilk beklenmeyenler:", unexp[:8])
+    model.eval()
+    return model, cfg
+
+
+@torch.no_grad()
+def generate(model, sp, prompt, max_new=64, temperature=0.0, top_k=40, top_p=0.9,
+             rep_penalty=1.2, device=None):
+    """Decode-cache'li O(L) üretim (step_forward). faz7_rag.generate ile signature-uyumlu."""
+    dev = device or next(model.parameters()).device
+    ids = sp.encode(prompt, out_type=int); eos = sp.eos_id()
+    states = model.init_states(1, device=dev)
+    logits = None
+    for tid in ids:                                                 # prefill: her token 1 kez (O(L))
+        logits, states = model.step_forward(torch.tensor([[tid]], device=dev), states)
+    out = []
+    for _ in range(max_new):
+        lg = logits[0].float()
+        if rep_penalty != 1.0:
+            for t in set(ids + out):
+                lg[t] = lg[t] / rep_penalty if lg[t] > 0 else lg[t] * rep_penalty
+        if temperature <= 0:
+            nxt = int(lg.argmax())
+        else:
+            lg = lg / temperature
+            if top_k:
+                kth = torch.topk(lg, min(top_k, lg.numel())).values[-1]; lg[lg < kth] = -float("inf")
+            probs = F.softmax(lg, -1)
+            if top_p < 1.0:
+                s, si = torch.sort(probs, descending=True); cut = torch.cumsum(s, -1) > top_p
+                cut[1:] = cut[:-1].clone(); cut[0] = False; s[cut] = 0
+                probs = torch.zeros_like(probs).scatter_(0, si, s); probs /= probs.sum()
+            nxt = int(torch.multinomial(probs, 1))
+        if nxt == eos:
+            break
+        out.append(nxt)
+        logits, states = model.step_forward(torch.tensor([[nxt]], device=dev), states)
+    return sp.decode(out)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--ckpt", required=True)
+    ap.add_argument("--tokenizer", required=True)
+    ap.add_argument("--query", default="Merhaba")
+    ap.add_argument("--max_new", type=int, default=64)
+    args = ap.parse_args()
+    import sentencepiece as spm
+    sp = spm.SentencePieceProcessor(model_file=args.tokenizer)
+    model, cfg = load_lamba(args.ckpt)
+    print(f"[model] {'MIMO' if cfg.get('is_mimo') else 'SISO'} | CPU | {sum(p.numel() for p in model.parameters())/1e6:.0f}M")
+    prompt = f"### Talimat:\n{args.query}\n\n### Yanıt:\n"
+    print("CEVAP:", generate(model, sp, prompt, args.max_new))
+
+
+if __name__ == "__main__":
+    main()

lamba_rag.py

@@ -18,13 +18,15 @@ import os, sys, glob, re, argparse
 import torch, torch.nn as nn, torch.nn.functional as F
 from functools import partial
 
-try:
+try:                                              # fork (Triton) yalnız GPU/HybridLM yolu için
     from mamba_ssm.modules.block import Block
     from mamba_ssm.modules.mamba3 import Mamba3
     from mamba_ssm.modules.mlp import GatedMLP
     from mamba_ssm.ops.triton.layer_norm import RMSNorm
-except Exception as e:
-    sys.exit(f"[hata] mamba-og fork yok ({e!r}). Önce wheel kurulum hücresini çalıştır (CUDA).")
+    _FORK_OK, _FORK_ERR = True, None
+except Exception as e:                             # CPU: fork yok → LambaCPU (--device cpu) kullanılır
+    Block = Mamba3 = GatedMLP = RMSNorm = None
+    _FORK_OK, _FORK_ERR = False, e
 
 
 # ───────────── model (faz3_train ile birebir) ─────────────
@@ -131,13 +133,17 @@ def resolve_ckpt(spec, token):
 
 
 @torch.no_grad()
-def generate(model, sp, prompt, max_new=160, temperature=0.3, top_k=40, top_p=0.9, rep_penalty=1.2):
+def generate(model, sp, prompt, max_new=160, temperature=0.3, top_k=40, top_p=0.9, rep_penalty=1.2, device="cuda"):
     eos = sp.eos_id()
     ids = sp.encode(prompt, out_type=int)
-    x = torch.tensor([ids], device="cuda"); out = list(ids)
+    x = torch.tensor([ids], device=device); out = list(ids)
+    use_amp = (device == "cuda")
     for _ in range(max_new):
-        with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
-            logits = model(x)[0, -1].float()
+        if use_amp:
+            with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
+                logits = model(x)[0, -1].float()
+        else:
+            logits = model(x)[0, -1].float()              # CPU: saf fp32 (lamba_cpu, Triton'suz)
         if rep_penalty != 1.0:
             for t in set(out):
                 logits[t] = logits[t] / rep_penalty if logits[t] > 0 else logits[t] * rep_penalty
@@ -155,7 +161,7 @@ def generate(model, sp, prompt, max_new=160, temperature=0.3, top_k=40, top_p=0.
             nxt = int(torch.multinomial(probs, 1))
         if nxt == eos:
             break
-        out.append(nxt); x = torch.cat([x, torch.tensor([[nxt]], device="cuda")], 1)
+        out.append(nxt); x = torch.cat([x, torch.tensor([[nxt]], device=device)], 1)
         if x.shape[1] >= 2048:
             x = x[:, -2048:]
     return sp.decode([t for t in out[len(ids):] if t != eos])
@@ -279,10 +285,15 @@ def main():
     ap.add_argument("--no_rag", action="store_true", help="RAG'sız (kıyas için: bağlam enjekte etme)")
     ap.add_argument("--ab", action="store_true",
                     help="A/B: her soru için RAG vs ham cevabı yan yana üret (tek koşu, tek model yüklemesi)")
+    ap.add_argument("--device", default="auto", choices=["auto", "cuda", "cpu"],
+                    help="cpu = saf-PyTorch lamba_cpu (GPU/Triton gerekmez); auto = cuda varsa cuda")
     args = ap.parse_args()
 
-    assert torch.cuda.is_available(), "CUDA yok (Colab GPU gerekir)."
+    device = ("cuda" if torch.cuda.is_available() else "cpu") if args.device == "auto" else args.device
+    if device == "cuda":
+        assert torch.cuda.is_available(), "CUDA yok — --device cpu kullan (saf-PyTorch lamba_cpu)."
     torch.set_float32_matmul_precision("high")
+    print(f"[device] {device}", flush=True)
     token = os.environ.get("HF_TOKEN")
     try:
         from huggingface_hub import get_token
@@ -298,22 +309,29 @@ def main():
         chunks += chunk_text(d, args.chunk_size, args.overlap)
     print(f"[rag] {len(docs)} doküman → {len(chunks)} chunk | embed: {args.embed_model}", flush=True)
     from sentence_transformers import SentenceTransformer
-    embedder = SentenceTransformer(args.embed_model, device="cuda")
+    embedder = SentenceTransformer(args.embed_model, device=device)
     index = build_index(embedder, chunks)
 
     # 2) generator
     sp = load_tok(token, args.tokenizer)
-    st = torch.load(resolve_ckpt(args.ckpt, token), map_location="cpu")
-    model = HybridLM(st["cfg"], device="cuda", dtype=torch.bfloat16)
-    model.load_state_dict(st["model"], strict=False); model.eval()
+    ckpt_path = resolve_ckpt(args.ckpt, token)
+    st = torch.load(ckpt_path, map_location="cpu")
+    if device == "cpu":                                   # saf-PyTorch (Triton/mamba_ssm YOK)
+        import lamba_cpu as LC
+        model, _ = LC.load_lamba(ckpt_path); gen = LC.generate   # decode-cache'li O(L) üretim
+    else:
+        assert _FORK_OK, f"GPU yolu mamba-og fork ister ({_FORK_ERR!r}). Wheel kur ya da --device cpu kullan."
+        model = HybridLM(st["cfg"], device="cuda", dtype=torch.bfloat16)
+        model.load_state_dict(st["model"], strict=False); model.eval(); gen = generate
     tag = f"sft epoch={st.get('epoch')}" if st.get("sft") else f"base step={st.get('step','?')}"
-    print(f"[model] {tag} | {'MIMO' if st['cfg'].get('is_mimo') else 'SISO'}\n", flush=True)
+    print(f"[model] {tag} | {'MIMO' if st['cfg'].get('is_mimo') else 'SISO'} | {device}\n", flush=True)
 
     def answer(q, use_rag):
         hits = retrieve(embedder, index, chunks, q, args.top_k)
         prompt = (build_rag_prompt(q, hits, args.max_ctx_chars) if use_rag
                   else f"### Talimat:\n{q}\n\n### Yanıt:\n")
-        return hits, strip_think(generate(model, sp, prompt, max_new=args.max_new, temperature=args.temperature))
+        return hits, strip_think(gen(model, sp, prompt, max_new=args.max_new,
+                                     temperature=args.temperature, device=device))
 
     def show(q, use_rag):
         hits, ans = answer(q, use_rag)
@@ -330,10 +348,10 @@ def main():
             print("=" * 72)
             print(f"SORU: {q}")
             print(f"[getirilen] ({hits[0][1]:.2f}) {hits[0][0][:90]}")
-            a_rag = strip_think(generate(model, sp, build_rag_prompt(q, hits, args.max_ctx_chars),
-                                         max_new=args.max_new, temperature=args.temperature))
-            a_raw = strip_think(generate(model, sp, f"### Talimat:\n{q}\n\n### Yanıt:\n",
-                                         max_new=args.max_new, temperature=args.temperature))
+            a_rag = strip_think(gen(model, sp, build_rag_prompt(q, hits, args.max_ctx_chars),
+                                    max_new=args.max_new, temperature=args.temperature, device=device))
+            a_raw = strip_think(gen(model, sp, f"### Talimat:\n{q}\n\n### Yanıt:\n",
+                                    max_new=args.max_new, temperature=args.temperature, device=device))
             print(f"  [A · RAG]: {a_rag}")
             print(f"  [B · ham]: {a_raw}")
         return