exp002_refine: the aleph-addressed patchwork consumer wins (13.997 mean, 2 seeds, first relay under 14); width/strobe saturated; coverage beats concentration (-0.48); wide MLP diverges 1/2 seeds vs 0% for all relays; ordering budget-stable at 6k

exp002_refine/README.md

@@ -0,0 +1,79 @@
+# exp002_refine — refining the relay: five prototypical enhancements
+
+Experiment 2 of the geolip-aleph-qwen line. [exp001](../exp001_relay/) found
+the retrofit works at 0.5B (−21% ppl at <0.6% trainable) but the matched MLP
+adapter edges the bare relay on perplexity while the gate mechanism favors
+the relay. exp002 races five prototypical relay enhancements — each grounded
+in a law certified upstream — at an equalized wider budget (~230K/adapter,
+~5.5M total), against the wide-MLP capacity control. Training identical to
+exp001 (frozen Qwen2.5-0.5B, wikitext-103, block 512, batch 4, 3000 steps,
+pure Adam lr 1e-3 wd 0).
+
+## Results
+
+| arm (per-adapter design) | s0 | s1 | verdict |
+|---|---|---|---|
+| relay32 (2× slot width) | 14.089 | 14.069 | width saturated (16-slot: 14.093/14.091) |
+| relay_3tau (rule-of-3 strobe) | 14.112 | — | no lift on this substrate |
+| **relay_pw (patchwork consumer)** | **13.982** | **14.012** | **winner — certified 2 seeds** |
+| relay_deep (all budget, last 12 blocks) | 14.576 | — | coverage beats concentration (−0.48) |
+| mlp_wide (hidden 128 control) | **NaN** | 13.930 | diverged 1/2 seeds; no gain over narrow |
+| relay_pw @ 6000 steps | 13.983 | — | converged at 3k |
+| mlp64 @ 6000 steps (companion) | 13.899 | — | still improving — ordering budget-stable |
+
+References (exp001, certified): frozen 17.798; relay16 14.093/14.091; mlp64
+13.927/13.935. `build_results.py` re-asserts every claim from
+`results/ledger.jsonl` (+ the mlp@6k companion row).
+
+## Findings
+
+1. **The aleph-addressed patchwork consumer is the refinement that works.**
+   Replacing the relay's bare linear output with the constellation
+   consumption spec — `M̂ → Linear(64,178) → SquaredReLU → LN → zero-init
+   Linear(178,896)` — beats the bare relay by ~0.09 ppl at both seeds
+   (13.997 mean vs 14.092): the first relay variant under 14. Addressing
+   stays aleph; consumption goes constellation.
+2. **Width and temperature are saturated dials**: doubling slots buys
+   +0.01–0.02; the 3-tau strobe buys nothing here (its certified win was a
+   co-trained byte-LM regime).
+3. **Coverage beats concentration**: putting exp001's entire adapter budget
+   on the last 12 blocks costs 0.48 ppl at equal parameters, even though
+   those deep relays run the hottest gates (0.094) — the depth-gradient law
+   describes where cultivation concentrates, not where adapters belong.
+4. **The stability asymmetry, quantified at 0.5B**: the wide zero-init MLP
+   diverged to NaN at one of two seeds (50%) and gained nothing over the
+   narrow MLP at the other; every sphere-normalized relay variant — all five,
+   all seeds — trained without aid (0% divergence).
+5. **The perplexity ordering is budget-stable** (1 seed at ship time):
+   relay_pw is fully converged by 3k steps (6k: +0.001) while the MLP keeps
+   improving (13.927 → 13.899) — free capacity holds a small, persistent ppl
+   edge on this well-trained substrate. Combined with exp001's
+   gate-vs-ppl dissociation, the relay's value at 0.5B is its **stability,
+   its opt-in mechanism, and its discrete addressable surface** — the levers
+   the next experiments (sign-code tap, frozen-keyer registry) are built on —
+   rather than raw perplexity.
+
+Checkpoint note: `adapters/` holds the latest checkpoint per (arm, seed);
+for relay_pw s0 that is the 6000-step version (the 3000-step run's ppl is
+ledgered; its weights were superseded in place).
+
+## Files
+- `qwen_exp002_refine.py` — the five variants (Relay3Tau, RelayPatchwork,
+  width/depth builders), wave runner, smoke (budget-equalization asserted).
+- `qwen_exp001_relay.py` + `geolip_vitals.py` / `ar_differentiation_bed.py` /
+  `exp013_augmentation_bed.py` — this package's own harness copies. Standalone.
+- `repro.py`, `build_results.py`, `results/ledger.jsonl` (9 rows) +
+  `results/companion_mlp6k.jsonl`, `adapters/` (8 checkpoints).
+
+## Reproduce (from inside this folder)
+```bash
+pip install torch --index-url https://download.pytorch.org/whl/cu128
+pip install transformers pyarrow huggingface_hub
+python repro.py                                      # CPU smoke
+python repro.py --run                                # Wave A (GPU)
+python repro.py --arm relay_pw --seed 1 --steps 6000 # any single cell
+python build_results.py                              # re-assert every claim
+```
+Data lands in `./data` (override with `GEOLIP_DATA`).
+
+License: MIT · AbstractPhil + Claude Fable 5 · July 12, 2026

exp002_refine/adapters/q2_mlp_wide_s0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ab1522dc23cc11a697e6dd56ac77742ea0793e86ba1f30689694c1bc9d588fe
+size 22153985

exp002_refine/adapters/q2_mlp_wide_s1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e294044aa04cb44708964d515082cdae47bca14721b0df95a39ccd4eb8d8328
+size 22153985

exp002_refine/adapters/q2_relay32_s0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd86702a3f7aa5ca0002fb515fa7640777fdcede4505c66a71db0fa5b6f3e865
+size 22104835

exp002_refine/adapters/q2_relay32_s1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c37b16a38b332209550cc0651c0bcebf867ae9b0c1fb64e3a63c7ebfed688f3f
+size 22104835

exp002_refine/adapters/q2_relay_3tau_s0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1084ab623cb7e06610c967cc7daaa6a205a57449fac9f3cb746856a9892bba0
+size 22105213

exp002_refine/adapters/q2_relay_deep_s0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d7d59ddfff2e1e42bd7dfa210d36ed2b4bf05dcf2521274c59a769b1371acc1
+size 11052993

exp002_refine/adapters/q2_relay_pw_s0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:382da262682691a8e72bfc03d00381b8730a7d4e3653863bb90795a9173ef333
+size 22166937

exp002_refine/adapters/q2_relay_pw_s1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1dc19ea2ce67e369be4b2c5db1c04c3aec261337396cb34876548134e18f7559
+size 22166937

exp002_refine/ar_differentiation_bed.py

@@ -0,0 +1,487 @@
+"""ar_differentiation_bed.py — exp012: autoregressive differentiation of the aleph.
+
+Differentiation is cultivated by PREDICTIVE pressure along the sequence — the
+address parameterizing the next-byte distribution (Law 2: chain-rule advantage pays
+ONLY where the composed address directly parameterizes the predictive distribution).
+This bed puts the aleph in the autoregressive gradient path and measures what
+differentiates. The head arms enforce the employment law at its maximum: the
+ENTIRE next-byte distribution is parameterized by the address.
+
+Byte-level causal LM on wikitext-2-raw (HF parquet, CDN-fast), block 256. ARMS:
+  sdpa       — standard causal transformer control (matched trunk).
+  hub        — attention replaced by CAUSAL HUB: linear attention whose feature map
+               is the 2K-oriented aleph address, prefix-sum memories (no selection
+               event; O(n*K*d)). Differentiation cultivated INSIDE attention.
+  addr_head  — sdpa trunk, but the OUTPUT HEAD reads ONLY the signed aleph
+               coefficient vector w_k = sinh(u_k)/sum_j cosh(u_j) of the final
+               hidden state (K -> 256 logits). The address MUST carry every bit of
+               next-byte information — the hardest Law-2 bottleneck.
+
+JUDGED BY: val bits-per-byte per arm (task) + CULTIVATION VITALS on every aleph
+codebook (readouts, never losses): axis aliveness/hppl, drift-from-init +
+binding fraction @0.29154, winner-|cos| saturation (sign-code emergence), shadow
+path diversity (fixed high-bits hash). Never by recon.
+
+Riders: pure Adam wd=0; no BN/Dropout/GAP on geometric paths; orthogonal init;
+Colab-cell-safe (paste-ahead imports, no bare argparse, no __file__ reliance);
+GPU-only for verdict runs.
+
+Terminal:  python ar_differentiation_bed.py            # shapes/parse smoke
+           python ar_differentiation_bed.py --train    # verdict run
+Colab:     paste geolip_vitals.py cell, then this file (smoke auto-runs),
+           then  train(steps=2000, data_root="/content/data")  in the next cell.
+"""
+from __future__ import annotations
+import math
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+if "anchor_drift" not in globals():
+    try:
+        from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is not None:
+            import sys, pathlib
+            sys.path.insert(0, str(pathlib.Path(_here).parent))
+            from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
+        else:
+            raise ImportError(
+                "geolip_vitals not found — paste/run its cell first, or "
+                "hf_hub_download exp012_ar/geolip_vitals.py from "
+                "AbstractPhil/geolip-aleph-differentiation.")
+
+VOCAB = 256  # bytes
+
+
+# ------------------------------------------------------------------ aleph address
+def _super_fibonacci_s3(n: int) -> torch.Tensor:
+    """Near-uniform unit quaternions (Alexa CVPR'22) —
+    starts the codebook INSIDE the RP^3 attractor basin. D=4 only."""
+    PHI, PSI = math.sqrt(2.0), 1.533751168755204288118041
+    i = torch.arange(n, dtype=torch.float64)
+    s = (i + 0.5) / n
+    r, R = torch.sqrt(s), torch.sqrt(1.0 - s)
+    a, b = 2 * math.pi * i / PHI, 2 * math.pi * i / PSI
+    q = torch.stack([r * torch.sin(a), r * torch.cos(a),
+                     R * torch.sin(b), R * torch.cos(b)], dim=-1)
+    return F.normalize(q, dim=-1).float()
+
+
+class AlephAddress(nn.Module):
+    """Closed-form aleph over 2K oriented half-axes (aleph-void article).
+    signed(x): (..., K) w_k = sinh(u_k)/sum_j cosh(u_j)   — the Law-2 head feature.
+    oriented(x): ((..., K), (..., K)) positive halves of the 2K softmax — HUB map."""
+
+    def __init__(self, K: int, D: int, tau: float = 0.1, init: str = "random"):
+        super().__init__()
+        self.K, self.D, self.tau = K, D, tau
+        if init == "fibonacci":
+            assert D == 4, "fibonacci init lives on S^3 (D=4)"
+            A = _super_fibonacci_s3(K)
+        else:
+            A = F.normalize(torch.randn(K, D), dim=-1)
+        self.codebook = nn.Parameter(A)
+        self.register_buffer("home", self.codebook.detach().clone())
+
+    def _u(self, x):
+        A = F.normalize(self.codebook, dim=-1)
+        return (F.normalize(x, dim=-1) @ A.transpose(-1, -2)) / self.tau
+
+    def oriented(self, x):
+        u = self._u(x)
+        m = u.abs().amax(dim=-1, keepdim=True)
+        ep, en = torch.exp(u - m), torch.exp(-u - m)
+        Z = (ep + en).sum(dim=-1, keepdim=True)
+        return ep / Z, en / Z
+
+    def signed(self, x):
+        u = self._u(x)
+        m = u.abs().amax(dim=-1, keepdim=True)
+        ep, en = torch.exp(u - m), torch.exp(-u - m)
+        return (ep - en) / (ep + en).sum(dim=-1, keepdim=True)
+
+    def signed_at(self, x, taus):
+        """Multi-tau stroboscope (rule of 3): signed coefficients at several
+        temperatures, concatenated — softer taus keep the vector dense while a
+        hard tau supplies the sign-code sharpness. v2 refinement (b)."""
+        A = F.normalize(self.codebook, dim=-1)
+        cos = F.normalize(x, dim=-1) @ A.transpose(-1, -2)
+        outs = []
+        for t in taus:
+            u = cos / t
+            m = u.abs().amax(dim=-1, keepdim=True)
+            ep, en = torch.exp(u - m), torch.exp(-u - m)
+            outs.append((ep - en) / (ep + en).sum(dim=-1, keepdim=True))
+        return torch.cat(outs, dim=-1)
+
+    def m_hat(self, x):
+        """Closed-form soft read (decoders read M_hat, never M). v2 control (c)."""
+        u = self._u(x)
+        m = u.abs().amax(dim=-1, keepdim=True)
+        ep, en = torch.exp(u - m), torch.exp(-u - m)
+        A = F.normalize(self.codebook, dim=-1)
+        return ((ep - en) @ A) / (ep + en).sum(dim=-1, keepdim=True)
+
+    def m_hard_ste(self, x):
+        """Hard mode (aleph-void article): M_hard = sign(cos_win) * A[win], straight-through to
+        the soft read — forward fully discrete SIGN CODE, backward soft gradient.
+        Legal per theme A (reconstructive sign code, not a one-hot roster pick)."""
+        u = self._u(x)
+        soft = self.m_hat(x)
+        win = u.abs().argmax(dim=-1)
+        A = F.normalize(self.codebook, dim=-1)
+        sign = torch.sign(torch.gather(u, -1, win.unsqueeze(-1))).squeeze(-1)
+        hard = sign.unsqueeze(-1) * A[win]
+        return hard + soft - soft.detach()
+
+    @torch.no_grad()
+    def vitals(self, x_sample) -> dict:
+        u = self._u(x_sample.reshape(-1, x_sample.shape[-1]))
+        p, n = self.oriented(x_sample.reshape(-1, x_sample.shape[-1]))
+        two_k = torch.cat([p, n], dim=-1)
+        win = two_k.argmax(dim=-1)
+        cos_win = (u.abs().amax(dim=-1) * self.tau)  # winner |cos| — sign-code sat.
+        d = anchor_drift(self.codebook, self.home)
+        return {"drift": round(d["mean"], 4),
+                "binding_frac": round(d["binding_fraction"], 4),
+                "aliveness": axis_aliveness(two_k),
+                "win_cos_mean": round(cos_win.mean().item(), 4),
+                "paths": path_diversity(win)}
+
+
+# ------------------------------------------------------------------------- blocks
+class CausalSDPA(nn.Module):
+    def __init__(self, d: int, heads: int = 4):
+        super().__init__()
+        self.h = heads
+        self.qkv = nn.Linear(d, 3 * d, bias=False)
+        self.o = nn.Linear(d, d, bias=False)
+        nn.init.orthogonal_(self.qkv.weight); nn.init.orthogonal_(self.o.weight)
+
+    def forward(self, x):
+        B, n, d = x.shape
+        q, k, v = self.qkv(x).chunk(3, dim=-1)
+        q, k, v = (t.view(B, n, self.h, d // self.h).transpose(1, 2) for t in (q, k, v))
+        y = F.scaled_dot_product_attention(q, k, v, is_causal=True)
+        return self.o(y.transpose(1, 2).reshape(B, n, d))
+
+
+class CausalHUB(nn.Module):
+    """Causal aleph linear attention: prefix-sum memories over the two K-wide
+    halves of the oriented address; 2K never materialized; no selection event."""
+
+    def __init__(self, d: int, K: int = 32, D: int = 4, tau: float = 0.1):
+        super().__init__()
+        self.addr = AlephAddress(K, D, tau)
+        self.q = nn.Linear(d, D, bias=False)
+        self.k = nn.Linear(d, D, bias=False)
+        self.v = nn.Linear(d, d, bias=False)
+        self.o = nn.Linear(d, d, bias=False)
+        for m in (self.q, self.k, self.v, self.o):
+            nn.init.orthogonal_(m.weight)
+
+    def forward(self, x):
+        qp, qn = self.addr.oriented(self.q(x))          # (B, n, K)
+        kp, kn = self.addr.oriented(self.k(x))
+        v = self.v(x)                                    # (B, n, d)
+        Sp = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kp, v), dim=1)
+        Sn = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kn, v), dim=1)
+        zp = torch.cumsum(kp, dim=1)
+        zn = torch.cumsum(kn, dim=1)
+        num = torch.einsum("bnk,bnkd->bnd", qp, Sp) + torch.einsum("bnk,bnkd->bnd", qn, Sn)
+        den = (qp * zp).sum(-1, keepdim=True) + (qn * zn).sum(-1, keepdim=True)
+        return self.o(num / den.clamp_min(1e-12))
+
+
+class MslRelay(nn.Module):
+    """Depth-composition unit (chain-rule probe): multi-slot M_hat read entering
+    the trunk as a NEAR-ZERO gated residual (gate init -3.0, sigma~0.047 — theme D:
+    geometry enters as a nudge and grows only if it earns gradient)."""
+
+    def __init__(self, d: int, n_slots: int = 16, K: int = 64):
+        super().__init__()
+        self.n_slots = n_slots
+        self.proj = nn.Linear(d, n_slots * 4, bias=False)
+        self.out = nn.Linear(n_slots * 4, d, bias=False)
+        nn.init.orthogonal_(self.proj.weight)
+        nn.init.orthogonal_(self.out.weight)
+        self.addr = AlephAddress(K, 4)
+        self.gate = nn.Parameter(torch.tensor(-3.0))
+
+    def forward(self, x):
+        B, n, _ = x.shape
+        slots = self.proj(x).view(B, n, self.n_slots, 4)
+        m = self.addr.m_hat(slots).reshape(B, n, -1)
+        return x + self.gate.sigmoid() * self.out(m)
+
+
+class Block(nn.Module):
+    def __init__(self, d: int, attn: nn.Module):
+        super().__init__()
+        self.n1, self.n2 = nn.LayerNorm(d), nn.LayerNorm(d)
+        self.attn = attn
+        self.mlp = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))
+
+    def forward(self, x):
+        x = x + self.attn(self.n1(x))
+        return x + self.mlp(self.n2(x))
+
+
+class ByteLM(nn.Module):
+    def __init__(self, arm: str, d: int = 192, layers: int = 4, block: int = 256,
+                 K: int = 32, D: int = 4):
+        super().__init__()
+        # "<arm>_tri" suffix = trigram byte embedding (AlephLM byte_emb x3 lineage):
+        # token embedding is the sum of embeddings of bytes t, t-1, t-2.
+        self.trigram = arm.endswith("_tri")
+        if self.trigram:
+            arm = arm[:-4]
+        # "_fib" = super-Fibonacci S^3 codebook init (basin test: starts INSIDE
+        # the RP^3 attractor; primary observable is init->final geodesic drift).
+        self.fib = arm.endswith("_fib")
+        if self.fib:
+            arm = arm[:-4]
+        # "relay*" = stacked addresses in depth: MslRelay after every block.
+        # relay -> sdpa trunk + standard head; relay_msl64 -> + addressed head.
+        self.use_relay = arm.startswith("relay")
+        if arm == "relay":
+            arm = "sdpa"
+        elif arm == "relay_msl64":
+            arm = "addr_msl64"
+        self.arm, self.block = arm, block
+        self.emb = nn.Embedding(VOCAB, d)
+        if self.trigram:
+            self.emb1 = nn.Embedding(VOCAB, d)
+            self.emb2 = nn.Embedding(VOCAB, d)
+        self.pos = nn.Parameter(torch.zeros(1, block, d) + 0.01 * torch.randn(1, block, d))
+        mk_attn = (lambda: CausalHUB(d, K, D)) if arm == "hub" else (lambda: CausalSDPA(d))
+        self.blocks = nn.ModuleList([Block(d, mk_attn()) for _ in range(layers)])
+        if self.use_relay:
+            self.relays = nn.ModuleList([MslRelay(d) for _ in range(layers)])
+        self.nf = nn.LayerNorm(d)
+        if arm == "addr_head":
+            self.head_addr = AlephAddress(K, d)  # v1: codebook in model dim — COLLAPSED
+            self.head = nn.Linear(K, VOCAB, bias=True)
+        elif arm in ("addr_d4", "addr_3tau", "addr_mhat"):
+            # v2 refinements: LOW-D HOME — learned projection to the native D=4 home
+            # before addressing (mirrors the healthy HUB arms), K=64.
+            self.head_proj = nn.Linear(d, 4, bias=False)
+            nn.init.orthogonal_(self.head_proj.weight)
+            self.head_addr = AlephAddress(64, 4)
+            if arm == "addr_d4":
+                self.head = nn.Linear(64, VOCAB, bias=True)      # w alone, D=4 home
+            elif arm == "addr_3tau":
+                self.taus = (0.05, 0.1, 0.3)                     # rule-of-3 strobe
+                self.head = nn.Linear(64 * 3, VOCAB, bias=True)
+            else:                                                 # addr_mhat
+                self.head = nn.Linear(4, VOCAB, bias=True)       # tightest: M_hat
+        elif arm.startswith("addr_msl"):
+            # v3: MULTI-SLOT heads — the 16s funnel widening: P parallel D=4 slots
+            # over a SHARED codebook. addr_msl consumes the reconstructive M_hat per
+            # slot (Px4 dims); addr_msl_w consumes signed w per slot (Px64) — tests
+            # whether slot-parallel consumption alone rescues the coefficient path.
+            # addr_msl<P> = slot-count dose-response. addr_mslh<P> = HARD sign-code
+            # consumption (straight-through M_hard per slot).
+            self.hard = arm.startswith("addr_mslh")
+            if arm in ("addr_msl", "addr_msl_w"):
+                self.n_slots = 16
+            else:
+                self.n_slots = int(arm[len("addr_mslh" if self.hard else "addr_msl"):])
+            self.head_proj = nn.Linear(d, self.n_slots * 4, bias=False)
+            nn.init.orthogonal_(self.head_proj.weight)
+            self.head_addr = AlephAddress(
+                64, 4, init="fibonacci" if self.fib else "random")
+            width = self.n_slots * (64 if arm == "addr_msl_w" else 4)
+            self.head = nn.Linear(width, VOCAB, bias=True)
+        elif arm == "addr_3tau_mhat":
+            # v3: combine the two v2 winners — 3-tau stroboscope + reconstructive read.
+            self.head_proj = nn.Linear(d, 4, bias=False)
+            nn.init.orthogonal_(self.head_proj.weight)
+            self.head_addr = AlephAddress(64, 4)
+            self.taus = (0.05, 0.1, 0.3)
+            self.head = nn.Linear(64 * 3 + 4, VOCAB, bias=True)
+        else:
+            self.head = nn.Linear(d, VOCAB, bias=True)
+        self._last_h = None
+
+    def forward(self, idx):
+        x = self.emb(idx)
+        if self.trigram:  # past-only shifts — causality preserved
+            x = x + self.emb1(F.pad(idx, (1, 0), value=0)[:, :-1]) \
+                  + self.emb2(F.pad(idx, (2, 0), value=0)[:, :-2])
+        x = x + self.pos[:, : idx.shape[1]]
+        if self.use_relay:
+            for b, r in zip(self.blocks, self.relays):
+                x = r(b(x))
+        else:
+            for b in self.blocks:
+                x = b(x)
+        h = self.nf(x)
+        self._last_h = h.detach()
+        if self.arm == "addr_head":
+            return self.head(self.head_addr.signed(h))
+        if self.arm == "addr_d4":
+            return self.head(self.head_addr.signed(self.head_proj(h)))
+        if self.arm == "addr_3tau":
+            return self.head(self.head_addr.signed_at(self.head_proj(h), self.taus))
+        if self.arm == "addr_mhat":
+            return self.head(self.head_addr.m_hat(self.head_proj(h)))
+        if self.arm.startswith("addr_msl"):
+            B, n, _ = h.shape
+            slots = self.head_proj(h).view(B, n, self.n_slots, 4)
+            if self.arm == "addr_msl_w":
+                feats = self.head_addr.signed(slots).reshape(B, n, -1)
+            elif getattr(self, "hard", False):
+                feats = self.head_addr.m_hard_ste(slots).reshape(B, n, -1)
+            else:
+                feats = self.head_addr.m_hat(slots).reshape(B, n, -1)
+            return self.head(feats)
+        if self.arm == "addr_3tau_mhat":
+            p = self.head_proj(h)
+            feats = torch.cat([self.head_addr.signed_at(p, self.taus),
+                               self.head_addr.m_hat(p)], dim=-1)
+            return self.head(feats)
+        return self.head(h)
+
+    @torch.no_grad()
+    def vitals(self) -> dict:
+        out = {}
+        if self.arm == "hub":
+            for i, b in enumerate(self.blocks):
+                if self._last_h is not None:
+                    out[f"L{i}"] = b.attn.addr.vitals(b.attn.q(self._last_h[:2]))
+        elif self.arm == "addr_head" and self._last_h is not None:
+            out["head"] = self.head_addr.vitals(self._last_h[:2])
+        elif self.arm in ("addr_d4", "addr_3tau", "addr_mhat",
+                          "addr_3tau_mhat") and self._last_h is not None:
+            out["head"] = self.head_addr.vitals(self.head_proj(self._last_h[:2]))
+        elif self.arm.startswith("addr_msl") and self._last_h is not None:
+            slots = self.head_proj(self._last_h[:2])
+            out["head"] = self.head_addr.vitals(
+                slots.reshape(*slots.shape[:-1], self.n_slots, 4))
+        if self.use_relay and self._last_h is not None:
+            for i, r in enumerate(self.relays):
+                s = r.proj(self._last_h[:2])
+                v = r.addr.vitals(s.reshape(*s.shape[:-1], r.n_slots, 4))
+                out[f"relay{i}"] = {"gate": round(r.gate.sigmoid().item(), 4),
+                                    "drift": v["drift"],
+                                    "binding_frac": v["binding_frac"],
+                                    "ppl": round(v["aliveness"]["usage_ppl"], 1)}
+        return out
+
+
+# --------------------------------------------------------------------------- data
+def _wikitext_bytes(data_root: str):
+    """wikitext-2-raw as flat uint8 tensors via the HF parquet CDN."""
+    from huggingface_hub import hf_hub_download
+    import pyarrow.parquet as pq
+
+    def load(split):
+        p = hf_hub_download("Salesforce/wikitext",
+                            f"wikitext-2-raw-v1/{split}-00000-of-00001.parquet",
+                            repo_type="dataset", local_dir=data_root)
+        text = "".join(pq.read_table(p).column("text").to_pylist())
+        return torch.frombuffer(bytearray(text.encode("utf-8")), dtype=torch.uint8).clone()
+
+    return load("train"), load("validation")
+
+
+def _batch(data: torch.Tensor, batch: int, block: int, device, g: torch.Generator):
+    ix = torch.randint(0, data.numel() - block - 1, (batch,), generator=g)
+    x = torch.stack([data[i:i + block] for i in ix]).long().to(device)
+    y = torch.stack([data[i + 1:i + block + 1] for i in ix]).long().to(device)
+    return x, y
+
+
+# -------------------------------------------------------------------- train/smoke
+def train(arms=("sdpa", "hub", "addr_head"), steps: int = 2000, batch: int = 32,
+          block: int = 256, device: str = "cuda", data_root: str = "./data",
+          seed: int = 0, eval_every: int = 500, save: bool = True):
+    """Verdict run — GPU only. Pure Adam wd=0. Reports val bits-per-byte + vitals.
+    save=True writes {data_root}/ar_ckpts/{arm}_s{seed}_t{steps}.pt per arm —
+    the cultivated codebooks are SPECIMENS for the projective reading instruments."""
+    import os
+    if device == "cuda" and not torch.cuda.is_available():
+        raise RuntimeError("Verdict runs are GPU-only (never CPU-train for accuracy).")
+    ckpt_dir = os.path.join(data_root, "ar_ckpts")
+    os.makedirs(ckpt_dir, exist_ok=True)
+    tr, va = _wikitext_bytes(data_root)
+    print(f"data ready: train {tr.numel():,} bytes, val {va.numel():,} bytes", flush=True)
+    results = {}
+    for arm in arms:
+        torch.manual_seed(seed)
+        g = torch.Generator().manual_seed(seed)
+        model = ByteLM(arm, block=block).to(device)
+        n_params = sum(p.numel() for p in model.parameters())
+        opt = torch.optim.Adam(model.parameters(), lr=3e-4, weight_decay=0.0)
+        for step in range(1, steps + 1):
+            x, y = _batch(tr, batch, block, device, g)
+            logits = model(x)
+            loss = F.cross_entropy(logits.reshape(-1, VOCAB), y.reshape(-1))
+            opt.zero_grad(set_to_none=True)
+            loss.backward()
+            opt.step()
+            if step % eval_every == 0 or step == steps:
+                model.eval()
+                with torch.no_grad():
+                    losses = []
+                    for _ in range(20):
+                        xv, yv = _batch(va, batch, block, device, g)
+                        lv = F.cross_entropy(model(xv).reshape(-1, VOCAB),
+                                             yv.reshape(-1))
+                        losses.append(lv.item())
+                bpb = sum(losses) / len(losses) / math.log(2)
+                print(f"[{arm}] step {step} val_bpb={bpb:.4f} vitals={model.vitals()}",
+                      flush=True)
+                model.train()
+        results[arm] = {"val_bpb": bpb, "params": n_params, "vitals": model.vitals()}
+        if save:
+            path = os.path.join(ckpt_dir, f"{arm}_s{seed}_t{steps}.pt")
+            torch.save({"arm": arm, "seed": seed, "steps": steps, "val_bpb": bpb,
+                        "state_dict": {k: v.cpu() for k, v in
+                                       model.state_dict().items()}}, path)
+            print(f"saved specimen: {path}", flush=True)
+    print(results, flush=True)
+    return results
+
+
+def smoke():
+    """Shapes/parse only — no accuracy claims."""
+    x = torch.randint(0, VOCAB, (2, 64))
+    for arm in ("sdpa", "hub", "addr_head"):
+        m = ByteLM(arm, d=96, layers=2, block=64, K=16)
+        logits = m(x)
+        assert logits.shape == (2, 64, VOCAB)
+        logits.sum().backward()
+        # causality check: future byte must not affect past logits
+        with torch.no_grad():
+            a = m(x)[0, 10]
+            x2 = x.clone(); x2[0, 40] = (x2[0, 40] + 7) % 256
+            b = m(x2)[0, 10]
+        assert torch.allclose(a, b, atol=1e-4), f"{arm} leaks future context"
+        print(f"{arm}: OK params={sum(p.numel() for p in m.parameters()):,} "
+              f"vitals={m.vitals()}", flush=True)
+    print("OK — AR bed smoke passed (verdict run: train() on GPU)", flush=True)
+
+
+def _in_notebook() -> bool:
+    try:
+        get_ipython()  # type: ignore[name-defined]  # noqa: F821
+        return True
+    except NameError:
+        return False
+
+
+if __name__ == "__main__":
+    if _in_notebook():
+        smoke()
+        print("Notebook mode: call  train(steps=2000)  in the next cell (GPU).")
+    else:
+        import argparse
+        ap = argparse.ArgumentParser()
+        ap.add_argument("--train", action="store_true")
+        ap.add_argument("--steps", type=int, default=2000)
+        a, _ = ap.parse_known_args()
+        train(steps=a.steps) if a.train else smoke()

exp002_refine/build_results.py

@@ -0,0 +1,55 @@
+"""build_results.py — exp002_refine: read results/ledger.jsonl (+ the mlp@6k
+companion row from exp001's bed) and RE-ASSERT every claim in the README.
+Run from inside this folder: python build_results.py
+"""
+import json
+import math
+import os
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+rows = [json.loads(l) for l in
+        open(os.path.join(HERE, "results", "ledger.jsonl"), encoding="utf-8")]
+comp = [json.loads(l) for l in
+        open(os.path.join(HERE, "results", "companion_mlp6k.jsonl"),
+             encoding="utf-8")]
+assert all(r["exp"] == "q2" for r in rows) and len(rows) == 9
+assert len(comp) == 1 and comp[0]["steps"] == 6000
+
+def cell(arm, seed, steps=3000):
+    return next(r for r in rows if r["arm"] == arm and r["seed"] == seed
+                and r["steps"] == steps)
+
+# exp001 references (certified there): relay16 14.093/14.091, mlp64
+# 13.927/13.935, frozen 17.798
+REF_RELAY, REF_MLP = 14.092, 13.931
+
+# claim 1: relay_pw is the certified best relay variant — under the bare relay
+# by ~0.08 or more at BOTH seeds, and the only variant to beat it at all
+for s, ref in ((0, 14.093), (1, 14.091)):
+    assert cell("relay_pw", s)["ppl"] <= ref - 0.075, s
+assert cell("relay32", 0)["ppl"] > 13.99 and cell("relay32", 1)["ppl"] > 13.99
+assert cell("relay_3tau", 0)["ppl"] > 14.09
+assert cell("relay_deep", 0)["ppl"] > 14.5          # coverage beats concentration
+
+# claim 2: width saturation — relay32 within 0.03 of the 16-slot relay
+assert abs(cell("relay32", 0)["ppl"] - 14.093) < 0.03
+assert abs(cell("relay32", 1)["ppl"] - 14.091) < 0.03
+
+# claim 3: the stability asymmetry — mlp_wide diverged at one of two seeds
+# (NaN), landed at narrow-MLP level at the other; relays never diverged
+mw = [cell("mlp_wide", s)["ppl"] for s in (0, 1)]
+assert sum(1 for p in mw if math.isnan(p)) == 1, mw
+assert any(abs(p - REF_MLP) < 0.01 for p in mw if not math.isnan(p))
+assert all(not math.isnan(r["ppl"]) for r in rows if r["arm"] != "mlp_wide")
+
+# claim 4 (budget, 1 seed at ship time): relay_pw is converged at 3k
+# (6k within 0.005) while the mlp still improves at 6k — ordering budget-stable
+assert abs(cell("relay_pw", 0, 6000)["ppl"] - cell("relay_pw", 0)["ppl"]) < 0.005
+assert comp[0]["ppl"] < REF_MLP
+
+out = {"ppl": {f'{r["arm"]}_s{r["seed"]}_t{r["steps"]}': r["ppl"] for r in rows},
+       "companion_mlp6k": comp[0]["ppl"], "n_rows": len(rows) + 1}
+json.dump(out, open(os.path.join(HERE, "results", "results.json"), "w",
+                    encoding="utf-8"), indent=1)
+print(f"{len(rows)}+1 rows -> results/results.json")
+print("all README claims asserted OK")

exp002_refine/exp013_augmentation_bed.py

@@ -0,0 +1,506 @@
+"""exp013_augmentation_bed.py — augmenting pretrained models with the aleph.
+Three tracks, one file (sequel to exp012's ar_differentiation_bed):
+
+  A. AUTOREGRESSION FROM CLIP-L: next-token prediction over CLIP-tokenized text,
+     reading FROZEN openai/clip-vit-large-patch14 text-tower hidden states.
+     FACTOR: extraction layer in {final, penultimate} (the last two layers — the
+     penultimate is what diffusion stacks consume). Heads at ~matched params:
+     linear | mlp | aleph multi-slot M_hat (P=64, D=4, shared K=64 — the exp012
+     certified construction) | sign-code (straight-through).
+  B. JOINT-FAILURE PROBES on frozen pooled embeddings (CLIP-L both layers + BERT):
+     b1 SPELLING-AR — decode a word's characters from ONLY the head's read of its
+        pooled embedding (GATE: linear/mlp must fail <50% exact first);
+     b2 ORDER — original-vs-shuffled discrimination (secondary; info may be absent).
+  C. GPT-2 (124M) AUGMENTATION: frozen trunk + trainable adapters after every block —
+     aleph MslRelay adapters vs param-matched MLP adapters vs frozen baseline;
+     gate growth by depth is a first-class readout (exp012 depth-gradient law).
+
+Riders: pure Adam wd=0; no contrastive/InfoNCE into address paths; vitals are
+readouts; GPU-only for verdict runs; caches/specimens live OUTSIDE the repo.
+Colab: paste geolip_vitals.py, then ar_differentiation_bed.py, then this file.
+"""
+from __future__ import annotations
+import json
+import math
+import os
+import re
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+# ---- paste-ahead imports (notebook-safe) -----------------------------------------
+if "anchor_drift" not in globals():
+    try:
+        from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is None:
+            raise ImportError("paste/run geolip_vitals.py first")
+        import sys, pathlib
+        sys.path.insert(0, str(pathlib.Path(_here).parent))
+        from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
+if "AlephAddress" not in globals():
+    try:
+        from ar_differentiation_bed import AlephAddress, MslRelay, _wikitext_bytes
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is None:
+            raise ImportError("paste/run ar_differentiation_bed.py first")
+        from ar_differentiation_bed import AlephAddress, MslRelay, _wikitext_bytes
+
+DATA_ROOT = os.environ.get("GEOLIP_DATA", "./data")
+
+
+class SquaredReLU(nn.Module):
+    def forward(self, x):
+        return F.relu(x) ** 2
+
+
+# =============================================================== head arm family ==
+class HeadArm(nn.Module):
+    """Conditioning transform d_in -> F_OUT (256), then a shared-shape task head.
+    Arms: linear | mlp | aleph (multi-slot M_hat) | sign (straight-through M_hard).
+    The aleph arms are the exp012-certified construction: P=64 slots x D=4 over one
+    shared K=64 codebook."""
+    F_OUT = 256
+
+    def __init__(self, arm: str, d_in: int, mlp_hidden: int = 192):
+        # mlp_hidden=192 param-matches the aleph arm at d_in=768:
+        # mlp ~ 192*(768+256)+LN ~ 197K vs aleph proj 768*256 + codebook = 196.9K
+        super().__init__()
+        self.arm = arm
+        if arm == "linear":
+            self.net = nn.Linear(d_in, self.F_OUT, bias=False)
+        elif arm == "mlp":
+            self.net = nn.Sequential(nn.Linear(d_in, mlp_hidden), SquaredReLU(),
+                                     nn.LayerNorm(mlp_hidden),
+                                     nn.Linear(mlp_hidden, self.F_OUT))
+        elif arm in ("aleph", "sign"):
+            self.proj = nn.Linear(d_in, 64 * 4, bias=False)
+            nn.init.orthogonal_(self.proj.weight)
+            self.addr = AlephAddress(64, 4)
+        else:
+            raise ValueError(arm)
+
+    def forward(self, x):
+        if self.arm in ("linear", "mlp"):
+            return self.net(x)
+        slots = self.proj(x).reshape(*x.shape[:-1], 64, 4)
+        read = self.addr.m_hard_ste(slots) if self.arm == "sign" \
+            else self.addr.m_hat(slots)
+        return read.reshape(*x.shape[:-1], 256)
+
+    @torch.no_grad()
+    def vitals(self, x_sample) -> dict:
+        if self.arm in ("linear", "mlp"):
+            return {}
+        slots = self.proj(x_sample).reshape(*x_sample.shape[:-1], 64, 4)
+        p, n = self.addr.oriented(slots)
+        two_k = torch.cat([p, n], -1).reshape(-1, 128)
+        d = anchor_drift(self.addr.codebook, self.addr.home)
+        return {"drift": round(d["mean"], 4),
+                "binding_frac": round(d["binding_fraction"], 4),
+                "usage_ppl": round(axis_aliveness(two_k)["usage_ppl"], 1),
+                "paths": path_diversity(two_k.argmax(-1))["unique_hashed"]}
+
+    def param_count(self):
+        return sum(p.numel() for p in self.parameters())
+
+
+# ================================================================== caches ========
+def _wikitext_lines(data_root, min_chars=40, max_lines=None):
+    from huggingface_hub import hf_hub_download
+    import pyarrow.parquet as pq
+    out = {}
+    for split, cap in (("train", max_lines), ("validation", None)):
+        p = hf_hub_download("Salesforce/wikitext",
+                            f"wikitext-2-raw-v1/{split}-00000-of-00001.parquet",
+                            repo_type="dataset", local_dir=data_root)
+        lines = [t.strip() for t in pq.read_table(p).column("text").to_pylist()
+                 if len(t.strip()) >= min_chars]
+        out[split] = lines[:cap] if cap else lines
+    return out["train"], out["validation"]
+
+
+@torch.no_grad()
+def cache_clip(data_root, n_train=12000, n_val=1500, device="cuda", batch=64):
+    """CLIP-L text tower over wikitext lines; caches BOTH of the last two layers.
+    hidden_states[-1] == the final encoder layer output (pre final-LN),
+    last_hidden_state == final-LN(final layer). We cache:
+      'final'  = last_hidden_state (what the projection head consumes),
+      'penult' = hidden_states[-2] (the layer diffusion stacks consume).
+    Also caches pooled (EOS-position) vectors for both layers, and token ids."""
+    from transformers import CLIPTextModel, CLIPTokenizerFast
+    path = os.path.join(data_root, "exp013", "clip_cache.pt")
+    if os.path.exists(path):
+        return torch.load(path, map_location="cpu", weights_only=True)
+    os.makedirs(os.path.dirname(path), exist_ok=True)
+    tok = CLIPTokenizerFast.from_pretrained("openai/clip-vit-large-patch14")
+    model = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14").to(device).eval()
+    tr_lines, va_lines = _wikitext_lines(data_root, max_lines=n_train)
+    va_lines = va_lines[:n_val]
+    def encode(lines):
+        H_f, H_p, IDS, EOS = [], [], [], []
+        for i in range(0, len(lines), batch):
+            enc = tok(lines[i:i + batch], padding="max_length", truncation=True,
+                      max_length=77, return_tensors="pt").to(device)
+            out = model(**enc, output_hidden_states=True)
+            H_f.append(out.last_hidden_state.half().cpu())
+            H_p.append(out.hidden_states[-2].half().cpu())
+            IDS.append(enc.input_ids.cpu())
+            EOS.append(enc.input_ids.argmax(-1).cpu())  # EOT id is the max token id
+        return (torch.cat(H_f), torch.cat(H_p), torch.cat(IDS), torch.cat(EOS))
+    tr = encode(tr_lines)
+    va = encode(va_lines)
+    blob = {"train": {"final": tr[0], "penult": tr[1], "ids": tr[2], "eos": tr[3]},
+            "val":   {"final": va[0], "penult": va[1], "ids": va[2], "eos": va[3]},
+            "vocab": tok.vocab_size}
+    torch.save(blob, path)
+    print(f"clip cache: train {tr[0].shape}, val {va[0].shape} -> {path}", flush=True)
+    return blob
+
+
+@torch.no_grad()
+def cache_word_embeddings(data_root, n_words=10000, device="cuda", batch=256):
+    """Pooled embeddings of frequent wikitext words for the spelling probe:
+    CLIP-L final + penultimate (EOS-pooled) and BERT (CLS + mean of last layer)."""
+    from transformers import (CLIPTextModel, CLIPTokenizerFast,
+                              BertModel, BertTokenizerFast)
+    path = os.path.join(data_root, "exp013", "word_cache.pt")
+    if os.path.exists(path):
+        return torch.load(path, map_location="cpu", weights_only=True)
+    os.makedirs(os.path.dirname(path), exist_ok=True)
+    tr_lines, _ = _wikitext_lines(data_root)
+    from collections import Counter
+    cnt = Counter(w for l in tr_lines for w in re.findall(r"[a-z]{3,12}", l.lower()))
+    words = [w for w, _ in cnt.most_common(n_words)]
+    blob = {"words": words}
+    ct = CLIPTokenizerFast.from_pretrained("openai/clip-vit-large-patch14")
+    cm = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14").to(device).eval()
+    fin, pen = [], []
+    for i in range(0, len(words), batch):
+        enc = ct(words[i:i + batch], padding="max_length", truncation=True,
+                 max_length=77, return_tensors="pt").to(device)
+        out = cm(**enc, output_hidden_states=True)
+        eos = enc.input_ids.argmax(-1)
+        idx = torch.arange(eos.numel(), device=device)
+        fin.append(out.last_hidden_state[idx, eos].half().cpu())
+        pen.append(out.hidden_states[-2][idx, eos].half().cpu())
+    blob["clip_final"], blob["clip_penult"] = torch.cat(fin), torch.cat(pen)
+    del cm
+    bt = BertTokenizerFast.from_pretrained("bert-base-uncased")
+    bm = BertModel.from_pretrained("bert-base-uncased").to(device).eval()
+    cls, mean = [], []
+    for i in range(0, len(words), batch):
+        enc = bt(words[i:i + batch], padding=True, truncation=True,
+                 max_length=16, return_tensors="pt").to(device)
+        out = bm(**enc).last_hidden_state
+        m = enc.attention_mask.unsqueeze(-1)
+        cls.append(out[:, 0].half().cpu())
+        mean.append(((out * m).sum(1) / m.sum(1)).half().cpu())
+    blob["bert_cls"], blob["bert_mean"] = torch.cat(cls), torch.cat(mean)
+    torch.save(blob, path)
+    print(f"word cache: {len(words)} words -> {path}", flush=True)
+    return blob
+
+
+# ============================================================ track A =============
+def track_a(arms=("linear", "mlp", "aleph", "sign"), layers=("final", "penult"),
+            steps=1500, batch=64, seed=0, device="cuda",
+            data_root=DATA_ROOT, eval_every=500, save=True):
+    """Next-CLIP-token prediction from frozen CLIP-L hidden states."""
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    blob = cache_clip(data_root, device=device)
+    V = blob["vocab"]
+    results = {}
+    ck_dir = os.path.join(data_root, "exp013", "ckpts")
+    os.makedirs(ck_dir, exist_ok=True)
+    for layer in layers:
+        Htr = blob["train"][layer].float()
+        ids_tr = blob["train"]["ids"]
+        Hva = blob["val"][layer].float()
+        ids_va = blob["val"]["ids"]
+        for arm in arms:
+            torch.manual_seed(seed)
+            g = torch.Generator().manual_seed(seed)
+            head = HeadArm(arm, Htr.shape[-1]).to(device)
+            out_proj = nn.Linear(HeadArm.F_OUT, V).to(device)
+            params = list(head.parameters()) + list(out_proj.parameters())
+            opt = torch.optim.Adam(params, lr=3e-4, weight_decay=0.0)
+            n_par = sum(p.numel() for p in params)
+            for step in range(1, steps + 1):
+                ix = torch.randint(0, Htr.shape[0], (batch,), generator=g)
+                h = Htr[ix].to(device)
+                y = ids_tr[ix].to(device)
+                logits = out_proj(head(h[:, :-1]))
+                loss = F.cross_entropy(logits.reshape(-1, V), y[:, 1:].reshape(-1))
+                opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+                if step % eval_every == 0 or step == steps:
+                    with torch.no_grad():
+                        ls = []
+                        for j in range(0, min(1024, Hva.shape[0]), batch):
+                            h = Hva[j:j + batch].to(device)
+                            y = ids_va[j:j + batch].to(device)
+                            lg = out_proj(head(h[:, :-1]))
+                            ls.append(F.cross_entropy(
+                                lg.reshape(-1, V), y[:, 1:].reshape(-1)).item())
+                        ce = sum(ls) / len(ls)
+                    vit = head.vitals(Hva[:2, :8].to(device))
+                    print(f"[A {layer} {arm} s{seed}] step {step} val_ce={ce:.4f} "
+                          f"params={n_par:,} vitals={vit}", flush=True)
+            results[f"{layer}/{arm}/s{seed}"] = {"val_ce": ce, "params": n_par,
+                                                 "vitals": vit}
+            if save and arm in ("aleph", "sign"):
+                torch.save({"track": "A", "layer": layer, "arm": arm, "seed": seed,
+                            "val_ce": ce, "state_dict": {k: v.cpu() for k, v in
+                                                          head.state_dict().items()}},
+                           os.path.join(ck_dir, f"A_{layer}_{arm}_s{seed}.pt"))
+    print(results, flush=True)
+    return results
+
+
+# ============================================================ track B =============
+class CharDecoder(nn.Module):
+    """Tiny GRU char decoder conditioned ONLY on the arm's 256-d read."""
+    CHARS = "abcdefghijklmnopqrstuvwxyz"
+    def __init__(self, cond_dim=256, hidden=256):
+        super().__init__()
+        self.V = len(self.CHARS) + 2  # +BOS +EOS
+        self.emb = nn.Embedding(self.V, 64)
+        self.init = nn.Linear(cond_dim, hidden)
+        self.gru = nn.GRU(64, hidden, batch_first=True)
+        self.out = nn.Linear(hidden, self.V)
+
+    def encode_word(self, w):
+        return [1] + [2 + self.CHARS.index(c) for c in w] + [0]  # BOS..EOS(0)
+
+    def forward(self, cond, tgt):          # tgt: (B, L) int, teacher-forced
+        h0 = torch.tanh(self.init(cond)).unsqueeze(0)
+        x = self.emb(tgt[:, :-1])
+        y, _ = self.gru(x, h0)
+        return self.out(y)                  # predict tgt[:,1:]
+
+    @torch.no_grad()
+    def greedy(self, cond, max_len=14):
+        B = cond.shape[0]
+        h = torch.tanh(self.init(cond)).unsqueeze(0)
+        t = torch.ones(B, 1, dtype=torch.long, device=cond.device)
+        done = torch.zeros(B, dtype=torch.bool, device=cond.device)
+        outs = []
+        for _ in range(max_len):
+            y, h = self.gru(self.emb(t), h)
+            t = self.out(y).argmax(-1)
+            outs.append(t)
+            done |= (t.squeeze(1) == 0)
+            if done.all():
+                break
+        return torch.cat(outs, 1)
+
+
+def track_b1(arms=("linear", "mlp"), substrates=("clip_final", "clip_penult",
+             "bert_cls", "bert_mean"), steps=3000, batch=128, seed=0,
+             device="cuda", data_root=DATA_ROOT, save=True):
+    """Spelling-AR from pooled embeddings. Run baselines first (the GATE:
+    qualify the task only if linear/mlp exact-match < 0.50), then aleph/sign."""
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    blob = cache_word_embeddings(data_root, device=device)
+    words = blob["words"]
+    dec_tpl = CharDecoder()
+    enc = [dec_tpl.encode_word(w) for w in words]
+    L = max(len(e) for e in enc)
+    tgt = torch.zeros(len(enc), L, dtype=torch.long)
+    for i, e in enumerate(enc):
+        tgt[i, :len(e)] = torch.tensor(e)
+    g0 = torch.Generator().manual_seed(1234)          # fixed split across arms
+    perm = torch.randperm(len(words), generator=g0)
+    tr_ix, va_ix = perm[:9000], perm[9000:]
+    ck_dir = os.path.join(data_root, "exp013", "ckpts")
+    os.makedirs(ck_dir, exist_ok=True)
+    results = {}
+    for sub in substrates:
+        E = blob[sub].float()
+        for arm in arms:
+            torch.manual_seed(seed)
+            g = torch.Generator().manual_seed(seed)
+            head = HeadArm(arm, E.shape[-1]).to(device)
+            dec = CharDecoder().to(device)
+            params = list(head.parameters()) + list(dec.parameters())
+            opt = torch.optim.Adam(params, lr=1e-3, weight_decay=0.0)
+            for step in range(1, steps + 1):
+                ix = tr_ix[torch.randint(0, tr_ix.numel(), (batch,), generator=g)]
+                cond = head(E[ix].to(device))
+                t = tgt[ix].to(device)
+                lg = dec(cond, t)
+                mask = (t[:, 1:] != 0) | (torch.cumsum(t[:, 1:] == 0, 1) == 1)
+                loss = F.cross_entropy(lg[mask], t[:, 1:][mask])
+                opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+            with torch.no_grad():
+                cond = head(E[va_ix].to(device))
+                pred = dec.greedy(cond)
+                t = tgt[va_ix, 1:].to(device)
+                n = min(pred.shape[1], t.shape[1])
+                pad_ok = torch.ones_like(t[:, :n], dtype=torch.bool)
+                seen_eos = torch.cumsum(t[:, :n] == 0, 1) > 0
+                match = ((pred[:, :n] == t[:, :n]) | seen_eos).all(-1)
+                exact = match.float().mean().item()
+            vit = head.vitals(E[va_ix[:16]].to(device))
+            print(f"[B1 {sub} {arm} s{seed}] exact={exact:.4f} vitals={vit}", flush=True)
+            results[f"{sub}/{arm}/s{seed}"] = {"exact": exact, "vitals": vit}
+            if save and arm in ("aleph", "sign"):
+                torch.save({"track": "B1", "sub": sub, "arm": arm, "seed": seed,
+                            "exact": exact, "state_dict": {k: v.cpu() for k, v in
+                                                            head.state_dict().items()}},
+                           os.path.join(ck_dir, f"B1_{sub}_{arm}_s{seed}.pt"))
+    print(results, flush=True)
+    return results
+
+
+# ============================================================ track C =============
+class MLPAdapter(nn.Module):
+    """Param-matched plain adapter (the ablation twin of MslRelay).
+    hidden=64 matches MslRelay at d=768 (2*768*64=98.3K vs 98.6K incl codebook).
+    Output layer ZERO-INIT (standard adapter stabilization — the first version
+    diverged at lr 1e-3 with random init; the aleph relay needed no such aid,
+    which is itself a datapoint, but the control gets its best shot)."""
+    def __init__(self, d, hidden=64):
+        super().__init__()
+        out = nn.Linear(hidden, d)
+        nn.init.zeros_(out.weight)
+        nn.init.zeros_(out.bias)
+        self.net = nn.Sequential(nn.Linear(d, hidden), SquaredReLU(), out)
+        self.gate = nn.Parameter(torch.tensor(-3.0))
+
+    def forward(self, x):
+        return x + self.gate.sigmoid() * self.net(x)
+
+
+class _BlockWithAdapter(nn.Module):
+    def __init__(self, block, adapter):
+        super().__init__()
+        self.block, self.adapter = block, adapter
+
+    def forward(self, *a, **k):
+        out = self.block(*a, **k)
+        if isinstance(out, tuple):
+            return (self.adapter(out[0]),) + out[1:]
+        return self.adapter(out)
+
+
+def track_c(arms=("frozen", "aleph", "mlp"), steps=1500, batch=8, block=256,
+            seed=0, device="cuda", data_root=DATA_ROOT,
+            eval_every=500, save=True):
+    """GPT-2 124M frozen; adapters after every block; train adapters only."""
+    from transformers import GPT2LMHeadModel, GPT2TokenizerFast
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    tok = GPT2TokenizerFast.from_pretrained("gpt2")
+    tr_lines, va_lines = _wikitext_lines(data_root)
+    def to_stream(lines):
+        ids = tok("\n\n".join(lines), return_tensors="pt").input_ids[0]
+        return ids
+    stream_tr = to_stream(tr_lines[:8000])
+    stream_va = to_stream(va_lines[:1000])
+    ck_dir = os.path.join(data_root, "exp013", "ckpts")
+    os.makedirs(ck_dir, exist_ok=True)
+    results = {}
+    for arm in arms:
+        torch.manual_seed(seed)
+        g = torch.Generator().manual_seed(seed)
+        model = GPT2LMHeadModel.from_pretrained("gpt2").to(device)
+        for p in model.parameters():
+            p.requires_grad_(False)
+        adapters = []
+        if arm != "frozen":
+            d = model.config.n_embd
+            for i, blk in enumerate(model.transformer.h):
+                ad = (MslRelay(d) if arm == "aleph" else MLPAdapter(d)).to(device)
+                model.transformer.h[i] = _BlockWithAdapter(blk, ad)
+                adapters.append(ad)
+            params = [p for ad in adapters for p in ad.parameters()]
+            n_par = sum(p.numel() for p in params)
+            opt = torch.optim.Adam(params, lr=1e-3, weight_decay=0.0)
+        else:
+            params, n_par = [], 0
+        def eval_ppl():
+            model.eval()
+            with torch.no_grad():
+                ls = []
+                for j in range(0, stream_va.numel() - block - 1, block * 4):
+                    x = stream_va[j:j + block].unsqueeze(0).to(device)
+                    out = model(x, labels=x)
+                    ls.append(out.loss.item())
+            model.train()
+            return math.exp(sum(ls) / len(ls))
+        if arm == "frozen":
+            ppl = eval_ppl()
+            print(f"[C frozen] ppl={ppl:.3f}", flush=True)
+            results["frozen"] = {"ppl": ppl}
+            continue
+        for step in range(1, steps + 1):
+            ix = torch.randint(0, stream_tr.numel() - block - 1, (batch,), generator=g)
+            x = torch.stack([stream_tr[i:i + block] for i in ix]).to(device)
+            loss = model(x, labels=x).loss
+            opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+            if step % eval_every == 0 or step == steps:
+                ppl = eval_ppl()
+                gates = [round(ad.gate.sigmoid().item(), 4) for ad in adapters]
+                vit = {}
+                if arm == "aleph":
+                    drifts = [round(anchor_drift(ad.addr.codebook, ad.addr.home)
+                                    ["mean"], 3) for ad in adapters]
+                    vit = {"drift_by_depth": drifts}
+                print(f"[C {arm} s{seed}] step {step} ppl={ppl:.3f} "
+                      f"params={n_par:,} gates={gates} {vit}", flush=True)
+        results[f"{arm}/s{seed}"] = {"ppl": ppl, "params": n_par, "gates": gates,
+                                     **vit}
+        if save and arm == "aleph":
+            torch.save({"track": "C", "arm": arm, "seed": seed, "ppl": ppl,
+                        "state_dict": {f"relay{i}.{k}": v.cpu()
+                                       for i, ad in enumerate(adapters)
+                                       for k, v in ad.state_dict().items()}},
+                       os.path.join(ck_dir, f"C_{arm}_s{seed}.pt"))
+    print(results, flush=True)
+    return results
+
+
+# ================================================================ smoke ===========
+def smoke():
+    """Shapes/parse only — no substrates, no training."""
+    for arm in ("linear", "mlp", "aleph", "sign"):
+        h = HeadArm(arm, 768)
+        y = h(torch.randn(2, 10, 768))
+        assert y.shape == (2, 10, 256)
+        y.sum().backward()
+        print(arm, "OK", f"{h.param_count():,}", h.vitals(torch.randn(2, 4, 768)))
+    dec = CharDecoder()
+    t = torch.tensor([dec.encode_word("hello") + [0] * 3,
+                      dec.encode_word("worlds") + [0] * 2])
+    lg = dec(torch.randn(2, 256), t)
+    assert lg.shape[:2] == (2, t.shape[1] - 1)
+    print("decoder OK; greedy:", dec.greedy(torch.randn(2, 256)).shape)
+    ad = MLPAdapter(768)
+    assert ad(torch.randn(2, 4, 768)).shape == (2, 4, 768)
+    print("adapter OK — exp013 smoke passed (caches+tracks need GPU+transformers)")
+
+
+def _in_notebook():
+    try:
+        get_ipython()  # type: ignore[name-defined]  # noqa: F821
+        return True
+    except NameError:
+        return False
+
+
+if __name__ == "__main__":
+    if _in_notebook():
+        smoke()
+        print("Notebook: cache_clip()/cache_word_embeddings() then "
+              "track_b1() gate -> track_a() -> track_c().")
+    else:
+        import argparse
+        ap = argparse.ArgumentParser()
+        ap.add_argument("--track", default="smoke")
+        a, _ = ap.parse_known_args()
+        {"smoke": smoke, "a": track_a, "b1": track_b1, "c": track_c}[a.track]()

exp002_refine/geolip_vitals.py

@@ -0,0 +1,219 @@
+"""geolip_vitals.py — shared diagnostic harness for the GeoLIP aleph experiments.
+ALL functions are READOUTS: no gradients, no losses. CV is a readout, never a
+force. Addressing is judged by drift->0.29154 and CV->0.20, never by recon cosine
+(judgment criteria per the aleph-void article: https://huggingface.co/blog/AbstractPhil/geometric-vocabulary-patchwork-aleph-void).
+
+Vitals provided:
+  anchor_drift        — geodesic drift of anchors from init; binding fraction @0.29154
+  pentachoron_cv      — CM 4-volume CV over random 5-row subsets (geovocab2 import)
+  axis_aliveness      — oriented-address usage: axes alive, hppl, collapse flag
+  gate_stats          — gate means vs the 0.012-0.03 band
+  path_diversity      — unique-path counting, FIXED high-bits hash (low-16 bug is the
+                        retracted artifact — never use the low bits)
+  grad_norm_spread    — gradient democracy monitor (orders-of-magnitude spread)
+  CVScreen            — CV@1000-batch early band screen (<0.30 LOW / .35-.50 MID / >.80 HIGH)
+
+Smoke on a torch-capable env:  python geolip_vitals.py
+"""
+from __future__ import annotations
+import math
+import torch
+
+BINDING = 0.29154            # radians; the binding/separation constant
+CV_BAND = (0.13, 0.30)       # CM CV band (discovery_catalog #4)
+GATE_BAND = (0.012, 0.03)    # live invariant candidate (acd_campaign)
+KNUTH32 = 2654435761
+
+
+# ----------------------------------------------------------------------------- drift
+@torch.no_grad()
+def anchor_drift(current: torch.Tensor, init: torch.Tensor, tol: float = 0.05) -> dict:
+    """Geodesic drift (radians) of each row of `current` from its row in `init`,
+    both row-normalized. Returns mean/std/per-row drift and the fraction of rows
+    within +/-tol of BINDING (the GLFM '46%' readout)."""
+    a = torch.nn.functional.normalize(current.float(), dim=-1)
+    b = torch.nn.functional.normalize(init.float(), dim=-1)
+    cos = (a * b).sum(-1).clamp(-1.0, 1.0)
+    drift = torch.arccos(cos)
+    frac = ((drift - BINDING).abs() <= tol).float().mean()
+    return {"mean": drift.mean().item(), "std": drift.std().item(),
+            "per_row": drift, "binding_fraction": frac.item()}
+
+
+# -------------------------------------------------------------------------------- cv
+@torch.no_grad()
+def _pentachoron_volumes(pts: torch.Tensor) -> torch.Tensor:
+    """Batched Cayley-Menger 4-simplex volumes. pts: (B, 5, D) -> (B,) volumes.
+    One float64 det over all samples (vol^2 = -det(CM)/9216 for n=4). Built-in
+    for speed (the per-sample reference path is ~260x slower in a vitals loop);
+    geovocab2 remains the formula's reference implementation, parity-checked
+    via cv_reference_check()."""
+    B = pts.shape[0]
+    d2 = torch.cdist(pts.double(), pts.double()).pow(2)          # (B,5,5)
+    cm = torch.ones(B, 6, 6, dtype=torch.float64, device=pts.device)
+    cm[:, 0, 0] = 0.0
+    cm[:, 1:, 1:] = d2
+    det = torch.linalg.det(cm)
+    return (-det / 9216.0).clamp_min(0.0).sqrt().float()
+
+
+@torch.no_grad()
+def pentachoron_cv(rows: torch.Tensor, n_samples: int = 200,
+                   generator: torch.Generator | None = None) -> float:
+    """CV (std/mean) of Cayley-Menger 4-simplex volumes over n_samples random
+    5-row subsets. Rows are row-normalized before measurement. Uses the built-in
+    batched CM (float64 det); validate against geovocab2 with
+    cv_reference_check() after any change to the volume math."""
+    x = torch.nn.functional.normalize(rows.float(), dim=-1)
+    n = x.shape[0]
+    if n < 5:
+        raise ValueError(f"pentachoron_cv needs >=5 rows, got {n}")
+    g = generator or torch.Generator(device="cpu").manual_seed(0)
+    idx = torch.stack([torch.randperm(n, generator=g)[:5]
+                       for _ in range(n_samples)])               # (B,5)
+    v = _pentachoron_volumes(x[idx].cpu())
+    return (v.std() / v.mean().clamp_min(1e-12)).item()
+
+
+@torch.no_grad()
+def cv_reference_check(n_trials: int = 50, tol: float = 1e-5) -> float:
+    """Parity check of the built-in batched CM against geovocab2's reference
+    implementation (the formula's source of truth). Returns max |rel diff|;
+    raises if geovocab2 is absent or parity fails. Run after touching
+    _pentachoron_volumes."""
+    try:
+        from geovocab2.shapes.formula.symbolic.cayley_menger import (
+            CayleyMengerFromSimplex)
+    except Exception as e:  # pragma: no cover
+        raise ImportError(
+            "cv_reference_check requires geovocab2 (install via the geolip-svae "
+            "umbrella: pip install git+https://github.com/AbstractEyes/"
+            "geolip-svae).") from e
+    ref = CayleyMengerFromSimplex()
+    g = torch.Generator().manual_seed(0)
+    pts = torch.nn.functional.normalize(
+        torch.randn(n_trials, 5, 4, generator=g), dim=-1)
+    mine = _pentachoron_volumes(pts)
+    # compare at float64: the reference computes in the INPUT dtype, and fp32
+    # dets lose up to ~4% on near-degenerate pentachora (measured 2026-07-11)
+    theirs = torch.stack([ref.forward(p.double())["volume"].float() for p in pts])
+    rel = ((mine - theirs).abs() / theirs.abs().clamp_min(1e-12)).max().item()
+    if rel > tol:
+        raise AssertionError(f"CM parity vs geovocab2 failed: max rel {rel}")
+    return rel
+
+
+# ------------------------------------------------------------------------- aliveness
+@torch.no_grad()
+def axis_aliveness(oriented_weights: torch.Tensor, alive_thresh: float = 1e-3) -> dict:
+    """`oriented_weights`: (..., 2K) nonnegative oriented-softmax address rows
+    (sum to 1 on the last dim). Returns axes-alive count, mean-usage perplexity
+    (hppl analogue; healthy hosted reference 125-126/128), and a collapse flag.
+    Reference behavior: near-uniform aliveness at div_weight=0 (discovery #22)."""
+    w = oriented_weights.reshape(-1, oriented_weights.shape[-1]).float()
+    usage = w.mean(0)
+    usage = usage / usage.sum().clamp_min(1e-12)
+    # an axis is alive if its mean usage exceeds alive_thresh x the uniform share
+    alive = int((usage > alive_thresh * (1.0 / usage.numel())).sum())
+    ent = -(usage.clamp_min(1e-12) * usage.clamp_min(1e-12).log()).sum()
+    ppl = float(ent.exp())
+    return {"axes_total": usage.numel(), "axes_alive": alive, "usage_ppl": ppl,
+            "collapsed": ppl < 0.05 * usage.numel()}
+
+
+# ------------------------------------------------------------------------------ gates
+@torch.no_grad()
+def gate_stats(gates: torch.Tensor) -> dict:
+    """Gate values (post-sigmoid/clamp). Reports mean and whether it sits in the
+    0.012-0.03 band (read-only — the band is a candidate invariant, never a target)."""
+    g = gates.float().flatten()
+    m = g.mean().item()
+    return {"mean": m, "std": g.std().item(),
+            "in_band": GATE_BAND[0] <= m <= GATE_BAND[1]}
+
+
+# ------------------------------------------------------------------------------ paths
+@torch.no_grad()
+def path_diversity(ids: torch.Tensor) -> dict:
+    """Unique-path counting with the FIXED multiplicative hash:
+    ((ids * 2654435761) % 2^32) >> 16 — Knuth needs the HIGH bits; the low-16
+    variant produced a retracted ~1,500 path ceiling in a prior campaign.
+    `ids`: integer tensor, one composed path id per row (any shape)."""
+    x = ids.reshape(-1).to(torch.int64)
+    hashed = ((x * KNUTH32) % (1 << 32)) >> 16
+    return {"n": int(x.numel()),
+            "unique_raw": int(torch.unique(x).numel()),
+            "unique_hashed": int(torch.unique(hashed).numel())}
+
+
+@torch.no_grad()
+def compose_path_ids(stage_indices: list[torch.Tensor], radix: int) -> torch.Tensor:
+    """Compose per-stage discrete indices (each (...,) int in [0, radix)) into a
+    single path id, positional base-`radix` — construction, not hashing."""
+    out = torch.zeros_like(stage_indices[0], dtype=torch.int64)
+    for s in stage_indices:
+        out = out * radix + s.to(torch.int64)
+    return out
+
+
+# --------------------------------------------------------------------- grad democracy
+@torch.no_grad()
+def grad_norm_spread(groups: dict[str, list[torch.nn.Parameter]]) -> dict:
+    """Gradient-democracy monitor. `groups`: name -> params of one parallel member
+    (tower/expert). Reports per-group grad norms and the orders-of-magnitude spread.
+    Reference: unequalized heterogeneous towers spread ~20 orders (fibonacci dead at
+    2.25e-21 under helix); equalized ~0.0 (geofractal gradient-democracy result)."""
+    norms = {}
+    for name, params in groups.items():
+        gs = [p.grad for p in params if p.grad is not None]
+        norms[name] = float(torch.sqrt(sum((g.float() ** 2).sum() for g in gs)).item()) \
+            if gs else 0.0
+    vals = [v for v in norms.values() if v > 0]
+    spread = (math.log10(max(vals)) - math.log10(min(vals))) if len(vals) >= 2 else 0.0
+    return {"norms": norms, "spread_orders": spread, "dead": [k for k, v in norms.items() if v == 0.0]}
+
+
+# ----------------------------------------------------------------------------- screen
+class CVScreen:
+    """CV@N early band screen (tri-band ft1): record pentachoron CV at `step_mark`
+    batches; classify <0.30 LOW / 0.35-0.50 MID / >0.80 HIGH. Turns ~2h/config
+    into ~7min. Readout only."""
+    def __init__(self, step_mark: int = 1000):
+        self.step_mark = step_mark
+        self.recorded: float | None = None
+
+    def maybe_record(self, step: int, rows: torch.Tensor) -> float | None:
+        if self.recorded is None and step >= self.step_mark:
+            self.recorded = pentachoron_cv(rows)
+        return self.recorded
+
+    @property
+    def band(self) -> str | None:
+        c = self.recorded
+        if c is None:
+            return None
+        if c < 0.30:
+            return "LOW"
+        if 0.35 <= c <= 0.50:
+            return "MID"
+        if c > 0.80:
+            return "HIGH"
+        return "BETWEEN"
+
+
+# ------------------------------------------------------------------------------ smoke
+if __name__ == "__main__":  # shapes/parse smoke ONLY — no training, ever.
+    g = torch.Generator().manual_seed(0)
+    K, D = 64, 4
+    init = torch.nn.functional.normalize(torch.randn(K, D, generator=g), dim=-1)
+    cur = torch.nn.functional.normalize(init + 0.29 * torch.randn(K, D, generator=g), dim=-1)
+    print("drift:", {k: v for k, v in anchor_drift(cur, init).items() if k != "per_row"})
+    w = torch.softmax(torch.randn(32, 2 * K, generator=g), dim=-1)
+    print("aliveness:", axis_aliveness(w))
+    print("gates:", gate_stats(torch.full((8,), 0.024)))
+    ids = compose_path_ids([torch.randint(0, 16, (4096,), generator=g) for _ in range(4)], 16)
+    print("paths:", path_diversity(ids))
+    lin = torch.nn.Linear(8, 8)
+    lin(torch.randn(4, 8)).sum().backward()
+    print("democracy:", grad_norm_spread({"a": list(lin.parameters())}))
+    print("OK — vitals smoke passed (pentachoron_cv needs geovocab2; run on GPU env)")

exp002_refine/qwen_exp001_relay.py

@@ -0,0 +1,271 @@
+"""qwen_exp001_relay.py — geolip-aleph-qwen EXPERIMENT 1: the relay retrofit
+at 0.5B. The certified exp013 Track-C recipe (frozen trunk + near-zero-gated
+multi-slot M_hat relays after every block, vs the param-matched zero-init MLP
+adapter ablation) at its next scale rung: Qwen2.5-0.5B (24 blocks, d=896,
+~494M frozen params; relay stack ~2.8M trainable, <0.6%). First campaign of
+the generation-stake line — judged on ppl, the GATE MECHANISM (do the gates
+grow? exp013: aleph gates grew ~3x while MLP gates shrank), relay codebook
+vitals, and GENERATED SAMPLES (the deliverable is a model that generates).
+
+Arms (x2 seeds): frozen (eval-only baseline) | relay (MslRelay(896) after
+every block) | mlp (param-matched MLPAdapter(896, hidden=64), zero-init out —
+the exp013 ablation, 114,688 vs 114,944 params per adapter, 0.2%).
+Corpus: wikitext-103-raw-v1 (HF parquet), ~12M-token cache; block 512,
+batch 4, 3000 steps adapters-only, pure Adam lr 1e-3 wd 0 (the exp013
+relay-training regime).
+Riders: trunk FROZEN throughout; pure Adam wd=0; GPU-only verdicts; >=2
+seeds; drift-check before any freeze claim; Colab-safe. Paste order:
+geolip_vitals -> ar_differentiation_bed -> exp013_augmentation_bed -> this.
+"""
+from __future__ import annotations
+import json
+import math
+import os
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+if "MslRelay" not in globals():
+    try:
+        from ar_differentiation_bed import MslRelay
+        from exp013_augmentation_bed import MLPAdapter
+        from geolip_vitals import anchor_drift, axis_aliveness
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is None:
+            raise ImportError("paste geolip_vitals + ar_differentiation_bed + "
+                              "exp013_augmentation_bed first")
+        import sys, pathlib
+        sys.path.insert(0, str(pathlib.Path(_here).parent))
+        from ar_differentiation_bed import MslRelay
+        from exp013_augmentation_bed import MLPAdapter
+        from geolip_vitals import anchor_drift, axis_aliveness
+
+DATA_ROOT = os.environ.get("GEOLIP_DATA", "./data")
+EXP_DIR = os.path.join(DATA_ROOT, "qwen_exp001")
+BASE_MODEL = "Qwen/Qwen2.5-0.5B"
+BLOCK, BATCH, STEPS, LR = 512, 4, 3000, 1e-3
+MAX_TOKENS = 12_000_000
+PROMPTS = ("The history of mathematics begins",
+           "In a small village by the sea,",
+           "The most important principle of engineering is")
+
+
+class _QwenBlockWithAdapter(nn.Module):
+    """Wrap a Qwen2 decoder layer: adapter applied to the hidden-states output,
+    all other outputs and kwargs passed through untouched."""
+
+    def __init__(self, block, adapter):
+        super().__init__()
+        self.block, self.adapter = block, adapter
+
+    def forward(self, *args, **kwargs):
+        out = self.block(*args, **kwargs)
+        if isinstance(out, tuple):
+            return (self.adapter(out[0]),) + out[1:]
+        return self.adapter(out)
+
+
+def _token_cache(device="cpu"):
+    """~12M-token wikitext-103 train stream + val stream, cached to disk."""
+    os.makedirs(EXP_DIR, exist_ok=True)
+    path = os.path.join(EXP_DIR, "tok_cache.pt")
+    if os.path.exists(path):
+        blob = torch.load(path, map_location="cpu", weights_only=True)
+        return blob["train"], blob["val"]
+    from huggingface_hub import hf_hub_download
+    import pyarrow.parquet as pq
+    from transformers import AutoTokenizer
+    tok = AutoTokenizer.from_pretrained(BASE_MODEL)
+
+    def stream(fname, cap):
+        p = hf_hub_download("Salesforce/wikitext", fname, repo_type="dataset")
+        lines = [t for t in pq.read_table(p).column("text").to_pylist()
+                 if t and len(t.strip()) > 40]
+        ids = []
+        total = 0
+        chunk = []
+        csz = 0
+        for ln in lines:
+            chunk.append(ln)
+            csz += len(ln)
+            if csz > 500_000:
+                e = tok("".join(chunk), return_tensors="pt").input_ids[0]
+                ids.append(e)
+                total += e.numel()
+                chunk, csz = [], 0
+                if total >= cap:
+                    break
+        if chunk and total < cap:
+            e = tok("".join(chunk), return_tensors="pt").input_ids[0]
+            ids.append(e)
+        return torch.cat(ids)[:cap]
+
+    tr = stream("wikitext-103-raw-v1/train-00000-of-00002.parquet", MAX_TOKENS)
+    va = stream("wikitext-103-raw-v1/validation-00000-of-00001.parquet",
+                600_000)
+    torch.save({"train": tr, "val": va}, path)
+    print(f"token cache: train {tr.numel():,} val {va.numel():,}", flush=True)
+    return tr, va
+
+
+def _batch(stream, batch, block, device, g):
+    ix = torch.randint(0, stream.numel() - block - 1, (batch,), generator=g)
+    x = torch.stack([stream[i:i + block] for i in ix]).to(device)
+    y = torch.stack([stream[i + 1:i + block + 1] for i in ix]).to(device)
+    return x.long(), y.long()
+
+
+def load_qwen(arm: str, seed: int = 0, device="cuda"):
+    from transformers import AutoModelForCausalLM
+    torch.manual_seed(seed)
+    model = AutoModelForCausalLM.from_pretrained(BASE_MODEL,
+                                                 torch_dtype=torch.float32)
+    model.config.use_cache = False
+    for p in model.parameters():
+        p.requires_grad_(False)
+    adapters = None
+    if arm != "frozen":
+        d = model.config.hidden_size
+        mk = (lambda: MslRelay(d)) if arm == "relay" else (lambda: MLPAdapter(d))
+        adapters = nn.ModuleList([mk() for _ in model.model.layers])
+        model.model.layers = nn.ModuleList(
+            [_QwenBlockWithAdapter(b, a)
+             for b, a in zip(model.model.layers, adapters)])
+    return model.to(device), adapters
+
+
+@torch.no_grad()
+def eval_ppl(model, va, device="cuda", n=40, g=None):
+    g = g or torch.Generator().manual_seed(0)
+    model.eval()
+    ls = []
+    for _ in range(n):
+        x, y = _batch(va, BATCH, BLOCK, device, g)
+        logits = model(input_ids=x).logits
+        ls.append(F.cross_entropy(logits.reshape(-1, logits.shape[-1]),
+                                  y.reshape(-1)).item())
+    return math.exp(sum(ls) / len(ls))
+
+
+@torch.no_grad()
+def sample(model, device="cuda", max_new=80):
+    from transformers import AutoTokenizer
+    tok = AutoTokenizer.from_pretrained(BASE_MODEL)
+    model.eval()
+    model.config.use_cache = True
+    outs = {}
+    for p in PROMPTS:
+        ids = tok(p, return_tensors="pt").input_ids.to(device)
+        out = model.generate(ids, max_new_tokens=max_new, do_sample=False,
+                             pad_token_id=tok.eos_token_id)
+        outs[p] = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)
+    model.config.use_cache = False
+    return outs
+
+
+def adapter_vitals(adapters):
+    if adapters is None:
+        return None
+    gates = [round(torch.sigmoid(a.gate).item(), 4) for a in adapters
+             if hasattr(a, "gate")]
+    drifts = []
+    for a in adapters:
+        ad = getattr(a, "addr", None)
+        if ad is not None:
+            drifts.append(round(anchor_drift(ad.codebook, ad.home)["mean"], 3))
+    return {"gates": gates[:6] + ["..."] if len(gates) > 6 else gates,
+            "gate_mean": round(sum(gates) / max(len(gates), 1), 4),
+            "drift": drifts[:6] + ["..."] if len(drifts) > 6 else drifts}
+
+
+def run_arm(arm: str, seed: int = 0, steps=STEPS, device="cuda"):
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    os.makedirs(EXP_DIR, exist_ok=True)
+    tr, va = _token_cache()
+    model, adapters = load_qwen(arm, seed=seed, device=device)
+    ledger = open(os.path.join(EXP_DIR, "ledger.jsonl"), "a", encoding="utf-8")
+    if arm == "frozen":
+        ppl = eval_ppl(model, va, device=device)
+        rec = {"exp": "q1", "arm": arm, "seed": seed, "ppl": round(ppl, 3),
+               "trainable": 0, "samples": sample(model, device=device)}
+        ledger.write(json.dumps(rec) + "\n"); ledger.flush()
+        print(f"[q1 frozen s{seed}] FINAL ppl={ppl:.3f}", flush=True)
+        ledger.close()
+        return ppl
+    g = torch.Generator().manual_seed(seed)
+    params = [p for p in adapters.parameters()]
+    n_train = sum(p.numel() for p in params)
+    opt = torch.optim.Adam(params, lr=LR, weight_decay=0.0)
+    model.train()
+    for step in range(1, steps + 1):
+        x, y = _batch(tr, BATCH, BLOCK, device, g)
+        logits = model(input_ids=x).logits
+        loss = F.cross_entropy(logits.reshape(-1, logits.shape[-1]),
+                               y.reshape(-1))
+        opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+        if step % 500 == 0:
+            v = adapter_vitals(adapters)
+            print(f"[q1 {arm} s{seed} step {step}] loss={loss.item():.3f} "
+                  f"gate_mean={v['gate_mean']}", flush=True)
+    ppl = eval_ppl(model, va, device=device)
+    rec = {"exp": "q1", "arm": arm, "seed": seed, "ppl": round(ppl, 3),
+           "steps": steps, "trainable": n_train,
+           "vitals": adapter_vitals(adapters),
+           "samples": sample(model, device=device)}
+    ledger.write(json.dumps(rec) + "\n"); ledger.flush()
+    print(f"[q1 {arm} s{seed}] FINAL ppl={ppl:.3f} "
+          f"gate_mean={rec['vitals']['gate_mean']} trainable={n_train:,}",
+          flush=True)
+    torch.save({"arm": arm, "seed": seed,
+                "adapters": {k: v.cpu() for k, v in
+                             adapters.state_dict().items()}},
+               os.path.join(EXP_DIR, f"q1_{arm}_s{seed}.pt"))
+    ledger.close()
+    del model, adapters
+    torch.cuda.empty_cache()
+    return ppl
+
+
+def run_exp001(seeds=(0, 1), device="cuda"):
+    run_arm("frozen", seed=0, device=device)      # baseline once (no training)
+    for seed in seeds:
+        for arm in ("relay", "mlp"):
+            run_arm(arm, seed=seed, device=device)
+    print("=== qwen exp001 COMPLETE ===", flush=True)
+
+
+def smoke():
+    """Shapes/parse only — no substrate download, no training."""
+    r = MslRelay(896)
+    m = MLPAdapter(896)
+    x = torch.randn(2, 8, 896)
+    assert r(x).shape == x.shape and m(x).shape == x.shape
+    rp = sum(p.numel() for p in r.parameters())
+    mp = sum(p.numel() for p in m.parameters())
+    assert abs(rp - mp) / rp < 0.01, (rp, mp)      # param-matched adapters
+    blk = nn.Linear(896, 896)                       # tuple-passthrough check
+    class TupBlock(nn.Module):
+        def forward(self, h, **kw):
+            return (blk(h), "aux")
+    w = _QwenBlockWithAdapter(TupBlock(), r)
+    out = w(x, position_ids=None)
+    assert isinstance(out, tuple) and out[0].shape == x.shape and out[1] == "aux"
+    (out[0].sum()).backward()
+    assert r.addr.codebook.grad is not None
+    print(f"qwen exp001 smoke passed (relay {rp:,} ~ mlp {mp:,} params/adapter;"
+          " full run needs GPU + transformers + the 0.5B download)")
+
+
+def _in_notebook():
+    try:
+        get_ipython()  # type: ignore[name-defined]  # noqa: F821
+        return True
+    except NameError:
+        return False
+
+
+if __name__ == "__main__":
+    smoke() if not _in_notebook() else (smoke(),
+        print("Notebook: run_exp001() on GPU."))

exp002_refine/qwen_exp002_refine.py

@@ -0,0 +1,237 @@
+"""qwen_exp002_refine.py — geolip-aleph-qwen EXPERIMENT 2: refine the relay.
+exp001 verdict shape: the relay retrofit works at 0.5B (frozen 17.80 ->
+~14.09, gates grow 0.047 -> 0.081 = trunk opt-in) but the param-matched MLP
+adapter is at parity-or-ahead on ppl — the certified GPT-2 ordering did not
+transfer as-is. exp002 races PROTOTYPICAL ENHANCEMENTS of the relay, each
+grounded in a certified law, at an equalized WIDER budget (~230K/adapter),
+against the honest wide MLP control.
+
+Arms (frozen Qwen2.5-0.5B trunk, adapters after every block unless noted):
+  relay32    — MslRelay with n_slots=32 (slot dose-response: exp012 L-AR3,
+               task monotone in slot width)
+  relay_3tau — multi-slot read at 3 temperatures (0.05/0.1/0.3), concatenated
+               (the certified rule-of-3 stroboscope, exp012 v2: -1.08 bpb)
+  relay_pw   — slots -> M_hat -> PATCHWORK consumer Linear(64,178) ->
+               SquaredReLU -> LN -> Linear(178,896) (the constellation
+               consumption spec replaces the bare linear out)
+  relay_deep — exp001's TOTAL budget concentrated on the last 12 blocks
+               (n_slots=32 there, nothing on blocks 0-11): the depth-gradient
+               law (cultivation concentrates near the prediction gradient)
+               as an architecture decision; total params == exp001 relay
+  mlp_wide   — MLPAdapter hidden=128 (the wide-budget capacity control)
+All near-zero gated (init -3.0). Training identical to exp001: block 512,
+batch 4, 3000 steps, pure Adam lr 1e-3 wd 0, adapters only. Judged: val ppl
+vs exp001's relay (14.093/14.091) and mlp (13.927/...) + gate mechanism +
+vitals + generated samples. Seed 0 sweep first; seed 1 for the top arms.
+Riders: trunk frozen; >=2 seeds before any claim; GPU-only; Colab-safe.
+Paste order: geolip_vitals -> ar_differentiation_bed ->
+exp013_augmentation_bed -> qwen_exp001_relay -> this file.
+"""
+from __future__ import annotations
+import json
+import math
+import os
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+if "run_arm" not in globals():
+    try:
+        from ar_differentiation_bed import AlephAddress, MslRelay
+        from exp013_augmentation_bed import MLPAdapter
+        from geolip_vitals import anchor_drift
+        from qwen_exp001_relay import (_QwenBlockWithAdapter, _token_cache,
+                                       _batch, eval_ppl, sample, EXP_DIR as
+                                       Q1_DIR, BASE_MODEL, BLOCK, BATCH, LR)
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is None:
+            raise ImportError("paste the qwen stack first")
+        import sys, pathlib
+        sys.path.insert(0, str(pathlib.Path(_here).parent))
+        from ar_differentiation_bed import AlephAddress, MslRelay
+        from exp013_augmentation_bed import MLPAdapter
+        from geolip_vitals import anchor_drift
+        from qwen_exp001_relay import (_QwenBlockWithAdapter, _token_cache,
+                                       _batch, eval_ppl, sample, EXP_DIR as
+                                       Q1_DIR, BASE_MODEL, BLOCK, BATCH, LR)
+
+EXP2_DIR = os.path.join(os.path.dirname(Q1_DIR), "qwen_exp002")
+STEPS = 3000
+
+
+class SquaredReLU(nn.Module):
+    def forward(self, x):
+        return F.relu(x) ** 2
+
+
+class Relay3Tau(nn.Module):
+    """Multi-slot M_hat at 3 temperatures, concatenated (rule-of-3 strobe)."""
+    TAUS = (0.05, 0.1, 0.3)
+
+    def __init__(self, d: int, n_slots: int = 16, K: int = 64):
+        super().__init__()
+        self.n_slots = n_slots
+        self.proj = nn.Linear(d, n_slots * 4, bias=False)
+        self.out = nn.Linear(n_slots * 4 * len(self.TAUS), d, bias=False)
+        nn.init.orthogonal_(self.proj.weight)
+        nn.init.orthogonal_(self.out.weight)
+        self.addr = AlephAddress(K, 4)
+        self.gate = nn.Parameter(torch.tensor(-3.0))
+
+    def forward(self, x):
+        B, n, _ = x.shape
+        slots = self.proj(x).view(B, n, self.n_slots, 4)
+        feats = []
+        for t in self.TAUS:
+            u = self.addr._u(slots) * (self.addr.tau / t)
+            m = u.abs().amax(dim=-1, keepdim=True)
+            ep, en = torch.exp(u - m), torch.exp(-u - m)
+            A = F.normalize(self.addr.codebook, dim=-1)
+            feats.append((((ep - en) @ A)
+                          / (ep + en).sum(dim=-1, keepdim=True)).reshape(B, n, -1))
+        return x + torch.sigmoid(self.gate) * self.out(torch.cat(feats, -1))
+
+
+class RelayPatchwork(nn.Module):
+    """Multi-slot M_hat -> constellation-spec consumer (SquaredReLU patchwork)."""
+
+    def __init__(self, d: int, n_slots: int = 16, K: int = 64, hidden: int = 178):
+        super().__init__()
+        self.n_slots = n_slots
+        self.proj = nn.Linear(d, n_slots * 4, bias=False)
+        nn.init.orthogonal_(self.proj.weight)
+        self.addr = AlephAddress(K, 4)
+        self.consume = nn.Sequential(
+            nn.Linear(n_slots * 4, hidden), SquaredReLU(),
+            nn.LayerNorm(hidden), nn.Linear(hidden, d))
+        nn.init.zeros_(self.consume[-1].weight)      # zero-init out (theme D)
+        self.gate = nn.Parameter(torch.tensor(-3.0))
+
+    def forward(self, x):
+        B, n, _ = x.shape
+        slots = self.proj(x).view(B, n, self.n_slots, 4)
+        feats = self.addr.m_hat(slots).reshape(B, n, -1)
+        return x + torch.sigmoid(self.gate) * self.consume(feats)
+
+
+def build_adapters(arm: str, model):
+    d = model.config.hidden_size
+    L = len(model.model.layers)
+    if arm == "relay32":
+        mk = [lambda: MslRelay(d, n_slots=32) for _ in range(L)]
+    elif arm == "relay_3tau":
+        mk = [lambda: Relay3Tau(d) for _ in range(L)]
+    elif arm == "relay_pw":
+        mk = [lambda: RelayPatchwork(d) for _ in range(L)]
+    elif arm == "relay_deep":
+        mk = [None] * (L // 2) + [lambda: MslRelay(d, n_slots=32)
+                                  for _ in range(L - L // 2)]
+    elif arm == "mlp_wide":
+        mk = [lambda: MLPAdapter(d, hidden=128) for _ in range(L)]
+    else:
+        raise ValueError(arm)
+    adapters = nn.ModuleList([m() if m else nn.Identity() for m in mk])
+    model.model.layers = nn.ModuleList(
+        [_QwenBlockWithAdapter(b, a) if not isinstance(a, nn.Identity) else b
+         for b, a in zip(model.model.layers, adapters)])
+    return adapters
+
+
+def adapter_vitals(adapters):
+    gates, drifts = [], []
+    for a in adapters:
+        if hasattr(a, "gate"):
+            gates.append(round(torch.sigmoid(a.gate).item(), 4))
+        ad = getattr(a, "addr", None)
+        if ad is not None:
+            drifts.append(round(anchor_drift(ad.codebook, ad.home)["mean"], 3))
+    return {"gate_mean": round(sum(gates) / max(len(gates), 1), 4),
+            "gates_head_tail": gates[:3] + gates[-3:],
+            "drift_head_tail": (drifts[:3] + drifts[-3:]) if drifts else None}
+
+
+def run_arm2(arm: str, seed: int = 0, steps=STEPS, device="cuda"):
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    os.makedirs(EXP2_DIR, exist_ok=True)
+    from transformers import AutoModelForCausalLM
+    tr, va = _token_cache()
+    torch.manual_seed(seed)
+    model = AutoModelForCausalLM.from_pretrained(BASE_MODEL,
+                                                 torch_dtype=torch.float32)
+    model.config.use_cache = False
+    for p in model.parameters():
+        p.requires_grad_(False)
+    adapters = build_adapters(arm, model)
+    model = model.to(device)
+    n_train = sum(p.numel() for p in adapters.parameters())
+    g = torch.Generator().manual_seed(seed)
+    opt = torch.optim.Adam(adapters.parameters(), lr=LR, weight_decay=0.0)
+    model.train()
+    for step in range(1, steps + 1):
+        x, y = _batch(tr, BATCH, BLOCK, device, g)
+        logits = model(input_ids=x).logits
+        loss = F.cross_entropy(logits.reshape(-1, logits.shape[-1]),
+                               y.reshape(-1))
+        opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+        if step % 1000 == 0:
+            print(f"[q2 {arm} s{seed} step {step}] loss={loss.item():.3f}",
+                  flush=True)
+    ppl = eval_ppl(model, va, device=device)
+    v = adapter_vitals(adapters)
+    rec = {"exp": "q2", "arm": arm, "seed": seed, "ppl": round(ppl, 3),
+           "steps": steps, "trainable": n_train, "vitals": v,
+           "samples": sample(model, device=device)}
+    ledger = open(os.path.join(EXP2_DIR, "ledger.jsonl"), "a", encoding="utf-8")
+    ledger.write(json.dumps(rec) + "\n"); ledger.close()
+    print(f"[q2 {arm} s{seed}] FINAL ppl={ppl:.3f} gate_mean={v['gate_mean']} "
+          f"trainable={n_train:,}", flush=True)
+    torch.save({"arm": arm, "seed": seed,
+                "adapters": {k: v2.cpu() for k, v2 in
+                             adapters.state_dict().items()}},
+               os.path.join(EXP2_DIR, f"q2_{arm}_s{seed}.pt"))
+    del model, adapters
+    torch.cuda.empty_cache()
+    return ppl
+
+
+ARMS = ("relay32", "relay_3tau", "relay_pw", "relay_deep", "mlp_wide")
+
+
+def run_wave_a(arms=ARMS, seed=0, device="cuda"):
+    for arm in arms:
+        run_arm2(arm, seed=seed, device=device)
+    print("=== qwen exp002 WAVE A COMPLETE ===", flush=True)
+
+
+def smoke():
+    x = torch.randn(2, 8, 896)
+    for cls, kw in ((Relay3Tau, {}), (RelayPatchwork, {}),):
+        a = cls(896, **kw)
+        y = a(x)
+        assert y.shape == x.shape
+        y.sum().backward()
+        assert a.addr.codebook.grad is not None
+        a.zero_grad()
+    p32 = sum(p.numel() for p in MslRelay(896, n_slots=32).parameters())
+    p3t = sum(p.numel() for p in Relay3Tau(896).parameters())
+    ppw = sum(p.numel() for p in RelayPatchwork(896).parameters())
+    pmw = sum(p.numel() for p in MLPAdapter(896, hidden=128).parameters())
+    lo, hi = min(p32, p3t, ppw, pmw), max(p32, p3t, ppw, pmw)
+    assert (hi - lo) / hi < 0.12, (p32, p3t, ppw, pmw)   # budget-equalized ~10%
+    print(f"qwen exp002 smoke passed (relay32 {p32:,} | 3tau {p3t:,} | "
+          f"pw {ppw:,} | mlp_wide {pmw:,} per adapter)")
+
+
+def _in_notebook():
+    try:
+        get_ipython()  # type: ignore[name-defined]  # noqa: F821
+        return True
+    except NameError:
+        return False
+
+
+if __name__ == "__main__":
+    smoke() if not _in_notebook() else (smoke(),
+        print("Notebook: run_wave_a() on GPU."))

exp002_refine/repro.py

@@ -0,0 +1,36 @@
+"""repro.py — standalone loader/runner for exp002_refine. Code dependencies
+live in THIS folder (geolip_vitals.py, ar_differentiation_bed.py,
+exp013_augmentation_bed.py, qwen_exp001_relay.py, qwen_exp002_refine.py).
+
+  python repro.py                     # CPU smoke: all variants + budget match
+  python repro.py --run               # Wave A (5 arms, seed 0; GPU ~3h)
+  python repro.py --arm relay_pw --seed 1 --steps 6000   # any single cell
+
+Data + token cache land in ./data (override with GEOLIP_DATA).
+"""
+import os
+import sys
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+
+if __name__ == "__main__":
+    import geolip_vitals                       # noqa: F401  (paste order)
+    import ar_differentiation_bed              # noqa: F401
+    import exp013_augmentation_bed             # noqa: F401
+    import qwen_exp001_relay                   # noqa: F401
+    import qwen_exp002_refine as q2
+    args = sys.argv[1:]
+    if "--run" in args:
+        q2.run_wave_a()
+    elif "--arm" in args:
+        import argparse
+        ap = argparse.ArgumentParser()
+        ap.add_argument("--arm", required=True, choices=q2.ARMS)
+        ap.add_argument("--seed", type=int, default=0)
+        ap.add_argument("--steps", type=int, default=3000)
+        a, _ = ap.parse_known_args()
+        q2.run_arm2(a.arm, seed=a.seed, steps=a.steps)
+    else:
+        q2.smoke()
+        print("repro smoke passed — --run (wave A) or --arm <name> (one cell)")

exp002_refine/results/companion_mlp6k.jsonl

@@ -0,0 +1 @@
+{"exp": "q1", "arm": "mlp", "seed": 0, "ppl": 13.899, "steps": 6000, "trainable": 2775576, "vitals": {"gates": [0.0399, 0.0381, 0.0375, 0.0375, 0.0358, 0.0333, "..."], "gate_mean": 0.0292, "drift": []}, "samples": {"The history of mathematics begins": " with the ancient civilizations of Mesopotamia , Egypt , and Greece . The Babylonians , who lived in the 2nd millennium BC , were the first to develop a system of mathematics that included the use of fractions . The Egyptians , who lived in the 2nd millennium BC , developed a system of mathematics that included the use of fractions and the use of the decimal system . The Greeks ,", "In a small village by the sea,": " a young man named John is a member of the local gang . He is a member of the gang known as the \" Black Gang \" , and is a member of the gang known as the \" White Gang \" . John is a member of the gang known as the \" Black Gang \" , and is a member of the gang known as the \" White Gang \" . John is a member of the gang", "The most important principle of engineering is": " to ensure that the design is safe and reliable . The design of a building is a complex process that involves many factors , including the materials used , the construction methods , and the design of the building itself . The design of a building is also influenced by the building owner 's needs and preferences . \n The design of a building is a complex process that involves many factors , including the materials used , the"}}

exp002_refine/results/ledger.jsonl

@@ -0,0 +1,9 @@
+{"exp": "q2", "arm": "relay32", "seed": 0, "ppl": 14.089, "steps": 3000, "trainable": 5511192, "vitals": {"gate_mean": 0.0606, "gates_head_tail": [0.0548, 0.0488, 0.0495, 0.134, 0.1166, 0.0854], "drift_head_tail": [0.133, 0.115, 0.144, 0.243, 0.33, 0.205]}, "samples": {"The history of mathematics begins": " with the ancient Babylonians , who developed a system of mathematics that was based on the concept of zero . The Babylonians used a base @-@ ten numeral system , and their arithmetic was based on multiplication and division . The Babylonians also developed a system of astronomy , which included the concept of zero . \n The ancient Egyptians developed a system of mathematics based on the concept of zero , and used it", "In a small village by the sea,": " a young man named Tom is a good friend of the village 's old man , Mr. Brown . Tom is a good boy , but he is not very good at his lessons . Mr. Brown is a good teacher , but he is not very good at teaching . Tom is often late for school , and Mr. Brown is often late for work . Tom is often late for school , and Mr", "The most important principle of engineering is": " to make the best use of the materials and energy available . This principle is expressed in the following equation : \n The equation is a mathematical expression of the law of conservation of energy . \n The equation is a mathematical expression of the law of conservation of energy . \n The equation is a mathematical expression of the law of conservation of energy . \n The equation is a mathematical expression of the law of conservation of energy"}}
+{"exp": "q2", "arm": "relay_3tau", "seed": 0, "ppl": 14.112, "steps": 3000, "trainable": 5511192, "vitals": {"gate_mean": 0.0445, "gates_head_tail": [0.0385, 0.034, 0.0336, 0.0981, 0.0983, 0.07], "drift_head_tail": [0.125, 0.134, 0.109, 0.193, 0.26, 0.139]}, "samples": {"The history of mathematics begins": " with the ancient Babylonians , who developed a system of mathematics that was based on the concept of zero . The Babylonians used a base @-@ ten system , and their numbers were written in a base @-@ ten positional notation . The Babylonians also developed a system of arithmetic , which was based on the concept of place value . The Babylonians also developed a system of geometry , which was", "In a small village by the sea,": " a young man named Tom is a good swimmer . He is also a good swimmer at the beach . He is a good swimmer at the beach , but he is not good at swimming in the sea . He is a good swimmer at the sea , but he is not good at swimming in the beach . \n Tom is a good swimmer at the beach , but he is not good", "The most important principle of engineering is": " to make the best use of the materials and energy available . This principle is expressed in the following way : \n \" The best use of materials and energy is to make the most efficient use of them . \" \n The term \" efficiency \" is used to describe the ratio of the useful work output to the total energy input . The efficiency of a machine is the ratio of the output power to the input power"}}
+{"exp": "q2", "arm": "relay_pw", "seed": 0, "ppl": 13.982, "steps": 3000, "trainable": 5517864, "vitals": {"gate_mean": 0.0366, "gates_head_tail": [0.0407, 0.0258, 0.0259, 0.0731, 0.0627, 0.0555], "drift_head_tail": [0.13, 0.1, 0.088, 0.107, 0.105, 0.115]}, "samples": {"The history of mathematics begins": " with the ancient Babylonians , who developed a system of mathematics that was based on the use of sexagesimal ( base 60 ) numbers . The Babylonians used a sexagesimal system to measure time , and to solve problems in geometry . The Babylonians also developed a system of algebra , which was based on the use of sexagesimal numbers . The Babylonians used sexagesimal numbers", "In a small village by the sea,": " a young man named John is a skilled sailor . He is a good friend of a young woman named Sarah , who is also a sailor . John and Sarah are married and have a son named Jack . \n John and Sarah are married and have a son named Jack . John and Sarah are married and have a son named Jack . John and Sarah are married and have a son named Jack . John and Sarah", "The most important principle of engineering is": " to make the best use of the materials and energy available . This principle is expressed in the following equation : \n The equation is a mathematical expression of the principle of conservation of energy . The energy in the system is the sum of the energy of the system and the energy of the surroundings . The energy of the system is the sum of the energy of the system and the energy of the surroundings . The energy"}}
+{"exp": "q2", "arm": "relay_deep", "seed": 0, "ppl": 14.576, "steps": 3000, "trainable": 2755596, "vitals": {"gate_mean": 0.0942, "gates_head_tail": [0.0992, 0.0765, 0.0703, 0.1419, 0.1192, 0.0884], "drift_head_tail": [0.209, 0.191, 0.188, 0.252, 0.321, 0.186]}, "samples": {"The history of mathematics begins": " with the ancient Babylonians , who developed a system of mathematics that was based on the decimal system . The Babylonians used a base @-@ ten system , and their calculations were based on the sexagesimal system , which is the system of measuring angles in degrees , minutes , and seconds . The Babylonians also developed a system of algebra , which was based on the concept of a variable , and", "In a small village by the sea,": " there are two small islands , the small island is 100 meters ( 328 feet ) long and 50 meters ( 164 feet ) wide , and the large island is 150 meters ( 490 feet ) long and 75 meters ( 248 feet ) wide . The small island is 10 meters ( 3", "The most important principle of engineering is": " to ensure the safety of the people . This is because the safety of the people is the foundation of the safety of the country . \n The safety of the people is the foundation of the safety of the country . \n The safety of the people is the foundation of the safety of the country . \n The safety of the people is the foundation of the safety of the country . \n The safety of the people"}}
+{"exp": "q2", "arm": "mlp_wide", "seed": 0, "ppl": NaN, "steps": 3000, "trainable": 5529624, "vitals": {"gate_mean": NaN, "gates_head_tail": [NaN, NaN, NaN, NaN, NaN, NaN], "drift_head_tail": null}, "samples": {"The history of mathematics begins": "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!", "In a small village by the sea,": "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!", "The most important principle of engineering is": "!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!"}}
+{"exp": "q2", "arm": "relay_pw", "seed": 1, "ppl": 14.012, "steps": 3000, "trainable": 5517864, "vitals": {"gate_mean": 0.0372, "gates_head_tail": [0.0386, 0.0269, 0.0254, 0.0712, 0.062, 0.0534], "drift_head_tail": [0.081, 0.087, 0.101, 0.106, 0.128, 0.104]}, "samples": {"The history of mathematics begins": " with the ancient Babylonians , who developed a system of mathematics that was based on the use of sexagesimal ( base @-@ 60 ) numbers . The Babylonians used a sexagesimal system to represent numbers , and they used a base @-@ 60 positional numeral system . The Babylonians used a sexagesimal system to represent numbers , and they used a base @", "In a small village by the sea,": " there are 100 houses . Each house has a dog , and each dog has a collar . The number of dogs in the village is 10 more than the number of houses . How many dogs are there in the village ? \n To determine the number of dogs in the village , we start by defining the variables and setting up the equation based on the given information . Let \\( d \\", "The most important principle of engineering is": " to make the best use of the materials available . This is a very broad principle , and it is not always easy to apply it to a particular problem . The most important principle of engineering is to make the best use of the materials available . This is a very broad principle , and it is not always easy to apply it to a particular problem . The most important principle of engineering is to make the best"}}
+{"exp": "q2", "arm": "mlp_wide", "seed": 1, "ppl": 13.93, "steps": 3000, "trainable": 5529624, "vitals": {"gate_mean": 0.0366, "gates_head_tail": [0.0441, 0.0409, 0.0398, 0.0261, 0.0286, 0.0246], "drift_head_tail": null}, "samples": {"The history of mathematics begins": " with the ancient civilizations of Mesopotamia , Egypt , and Greece . The Babylonians , who lived in the 19th century BC , were the first to use a base @-@ 10 positional numeral system , and they were the first to use zero as a place holder . The Egyptians used a base @-@ 20 positional numeral system , and the Greeks used a", "In a small village by the sea,": " a young man named John is a skilled sailor and a skilled fisherman . He is also a skilled hunter and a skilled tracker . He is also a skilled thief . He is also a skilled thief . He is also a skilled thief . He is also a skilled thief . He is also a skilled thief . He is also a skilled thief . He is also a skilled thief . He is also a skilled", "The most important principle of engineering is": " that of safety . The engineer is responsible for the safety of the people who use his services . The engineer is responsible for the safety of the people who use his services . The engineer is responsible for the safety of the people who use his services . The engineer is responsible for the safety of the people who use his services . The engineer is responsible for the safety of the people who use his services . The"}}
+{"exp": "q2", "arm": "relay32", "seed": 1, "ppl": 14.069, "steps": 3000, "trainable": 5511192, "vitals": {"gate_mean": 0.0613, "gates_head_tail": [0.0535, 0.0497, 0.0495, 0.1354, 0.1159, 0.0853], "drift_head_tail": [0.156, 0.147, 0.133, 0.261, 0.358, 0.185]}, "samples": {"The history of mathematics begins": " with the ancient civilizations of Mesopotamia , Egypt , and Greece . The Babylonians , for example , developed a system of mathematics that was based on the concept of zero , which was used to represent the absence of a quantity . The Egyptians used a sexagesimal ( base 60 ) numeral system , which was based on the fact that 60 is the number of days in a", "In a small village by the sea,": " there are 100 houses . Each house has a dog , and each dog has a collar . The number of dogs in the village is 10 more than the number of dogs in the village with a collar . How many dogs are there in the village with a collar ? \n To solve this problem , we can set up a system of equations based on the information given . Let 's", "The most important principle of engineering is": " to make the best use of the materials and energy available . The most important principle of engineering is to make the best use of the materials and energy available . \n The most important principle of engineering is to make the best use of the materials and energy available . The most important principle of engineering is to make the best use of the materials and energy available . \n The most important principle of engineering is to make"}}
+{"exp": "q2", "arm": "relay_pw", "seed": 0, "ppl": 13.983, "steps": 6000, "trainable": 5517864, "vitals": {"gate_mean": 0.0387, "gates_head_tail": [0.0368, 0.0225, 0.0239, 0.0836, 0.0704, 0.0632], "drift_head_tail": [0.167, 0.131, 0.118, 0.132, 0.146, 0.154]}, "samples": {"The history of mathematics begins": " with the ancient civilizations of Mesopotamia , Egypt , and China . The Babylonians developed a base 60 positional notation system , and the Chinese developed a base 10 positional notation system . The ancient Greeks developed a base 10 positional notation system , and the Babylonians developed a base 60 positional notation system . The Babylonians used a base 60 positional notation system", "In a small village by the sea,": " a young man named Tom is a skilled sailor . He is a good swimmer and a good swimmer at sea . He is also a good swimmer on land . He is a good swimmer in the sea and on land . He is a good swimmer in the sea and on land . He is a good swimmer in the sea and on land . He is a good swimmer in", "The most important principle of engineering is": " to make the best use of the materials available . The materials used in the construction of a building are not only the materials used in the construction of the building itself , but also the materials used in the construction of the building 's supporting structures . The materials used in the construction of the supporting structures are not only the materials used in the construction of the supporting structures themselves , but also the materials used in"}}

exp002_refine/results/results.json

@@ -0,0 +1,15 @@
+{
+ "ppl": {
+  "relay32_s0_t3000": 14.089,
+  "relay_3tau_s0_t3000": 14.112,
+  "relay_pw_s0_t3000": 13.982,
+  "relay_deep_s0_t3000": 14.576,
+  "mlp_wide_s0_t3000": NaN,
+  "relay_pw_s1_t3000": 14.012,
+  "mlp_wide_s1_t3000": 13.93,
+  "relay32_s1_t3000": 14.069,
+  "relay_pw_s0_t6000": 13.983
+ },
+ "companion_mlp6k": 13.899,
+ "n_rows": 10
+}