exp003_instruct: hallucination reduction quantified (specialist grounding 0.979/0.958 vs frozen 0.542, >10x fewer failures); register capture found (specialist answers every register in its own); multi-task resolves it (all registers >=0.875); register-differentiation hypothesis CONFIRMED on the sign-code surface (2x deep-layer separation vs the wikitext stack, 2 seeds)

exp003_instruct/README.md

@@ -0,0 +1,97 @@
+# exp003_instruct — instruct-task adapters + register differentiation
+
+Experiment 3 of the geolip-aleph-qwen line. Hypothesis under test:
+continuation-trained adapters align output **registers** to overlapping
+internal states; instruct-task training differentiates them cleanly. The
+instrument: [json-coco-format](https://huggingface.co/datasets/AbstractPhil/json-coco-format)
+— the same 22k COCO captions through three structurally distinct registers
+(task_1 hallucination_reduction / task_2 useful_generalization / task_3
+generic_symbolism), in native `messages`+`tools` shape.
+
+**Setup**: Qwen2.5-0.5B-**Instruct**, frozen; the exp002-certified `relay_pw`
+adapters (aleph addressing + constellation patchwork consumer) after every
+block; MLPAdapter(128) as the budget-matched control (229,911 vs
+230,401/adapter). Caption-hash holdout (no caption crosses tasks or splits).
+Block 512, batch 4, 3000 steps, pure Adam lr 1e-3 wd 0, full-conversation LM
+loss.
+
+## Results (2 seeds; validity = 48 held-out captions/task, greedy)
+
+**Held-out PPL** (register learned = format entropy collapses):
+
+| arm | task_1 | task_2 | task_3 |
+|---|---|---|---|
+| frozen | 17.497 | 12.582 | 13.584 |
+| relay_pw_task1 | **1.053 / 1.052** | 7.27 / 9.38 | 12.53 / **20.34** |
+| relay_pw_all | 1.062 / 1.059 | 1.062 / 1.057 | 1.051 / 1.048 |
+| mlp_all | 1.060 / 1.059 | 1.059 / 1.058 | 1.048 / 1.048 |
+
+**Task validity** (v2 validators; grounding / bracket-format / placeholders):
+
+| arm | task_1 grounded | task_2 valid | task_3 valid |
+|---|---|---|---|
+| frozen (zero-shot) | 0.542 | 0.063 | 0.750 |
+| relay_pw_task1 | **0.979 / 0.958** | 0.0 / 0.0 (capture) | 0.0 / 0.0 (capture) |
+| relay_pw_all | 0.875 / 0.938 | 1.0 / 1.0 | 1.0 / 1.0 |
+| mlp_all | 0.958 / 0.875 | 1.0 / 0.979 | 1.0 / 1.0 |
+
+**The register probe** (sign-code separation on the relay stack's discrete
+surface: inter-register − intra-register Hamming, same captions under 4
+registers):
+
+| stack | L0 | L8 | L16 | L23 |
+|---|---|---|---|---|
+| wikitext-trained (exp002) | 0.313 | 0.204 | **0.120** | **0.155** |
+| relay_pw_task1 (s0/s1) | 0.305/0.327 | 0.272/0.283 | **0.224/0.286** | **0.304/0.286** |
+| relay_pw_all (s0/s1) | 0.371/0.367 | 0.284/0.269 | **0.296/0.253** | **0.301/0.269** |
+
+`build_results.py` re-asserts every claim from `results/ledger.jsonl`.
+
+## Findings
+
+1. **Hallucination reduction, reproduced and quantified.** The task_1
+   specialist grounds 0.979/0.958 of held-out extractions vs the frozen
+   instruct model's 0.542 — grounding failures drop from ~46% to 2–4%
+   (>10× fewer), at 2 seeds, measured by the dataset's own per-leaf
+   grounding rule.
+2. **Register capture — single-register training interferes.** The
+   specialist answers *every* register in its trained one: perfect JSON on
+   task_2/3 prompts with 0.0 task-validity, both seeds, plus cross-register
+   ppl damage (task_3 worse than frozen at one seed, 20.3 vs 13.6). The
+   overlapping-states concern is real and now has a signature.
+3. **Multi-task training resolves capture**: all three registers ≥0.875–1.0
+   valid simultaneously, ppl ~1.05 everywhere. On validity and ppl the relay
+   and the matched MLP are tied (both saturate the task family).
+4. **The differentiation hypothesis confirms on the discrete surface — the
+   gauge only the relay has.** Instruct-trained stacks separate the four
+   registers at ~2× the wikitext stack in the deep layers (L16/L23:
+   0.22–0.30 vs 0.12–0.15), with intra-register spread tightening — instruct
+   training reorganizes the internal states into cleanly differentiated
+   registers, exactly as hypothesized, and the aleph sign-code surface is
+   the instrument that makes it visible (the MLP control has no such
+   surface to read).
+5. Validator history, honestly: the campaign ran with a v1 parser whose
+   non-greedy regex truncated nested tool-call JSON (a *perfect* generation
+   scored 0). The corrected v2 validators re-evaluated every checkpoint
+   post-hoc (inference-only); both generations of rows are in the ledger.
+
+## Files
+- `qwen_exp003_instruct.py` — data pipeline (chat-template rendering,
+  caption-hash splits), v2 validators, arms, the register probe, runners.
+- `qwen_exp001_relay.py` / `qwen_exp002_refine.py` / `geolip_vitals.py` /
+  `ar_differentiation_bed.py` / `exp013_augmentation_bed.py` — this package's
+  own harness copies. Standalone.
+- `repro.py`, `build_results.py`, `results/ledger.jsonl` (15 rows, both
+  validator generations), `adapters/` (all 6 checkpoints).
+
+## Reproduce (from inside this folder)
+```bash
+pip install torch --index-url https://download.pytorch.org/whl/cu128
+pip install transformers pyarrow huggingface_hub
+python repro.py            # CPU smoke
+python repro.py --run      # full campaign (GPU ~5h)
+python build_results.py    # re-assert every claim from the ledger
+```
+Data lands in `./data` (override with `GEOLIP_DATA`).
+
+License: MIT · AbstractPhil + Claude Fable 5 · July 12, 2026

exp003_instruct/adapters/q3_mlp_all_s0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a880fa2132a3ab50afcf417b4468d9e562541f321331ea63d5431e5c42e3c56
+size 22153859

exp003_instruct/adapters/q3_mlp_all_s1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04a334d6b6ba3bdb95c667254fc58f6131c8a4774132246167a370c8e3f80a0e
+size 22153859

exp003_instruct/adapters/q3_relay_pw_all_s0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd6bd733071ddc81799ab76c06978da8182273727b2db03ebc07ea9a199205ed
+size 22167921

exp003_instruct/adapters/q3_relay_pw_all_s1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:06d381786d9ad69e21bb9db4d17e916aaa8cd33fa82c27c2964ad27adf494ef4
+size 22167921

exp003_instruct/adapters/q3_relay_pw_task1_s0.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb98a895db91804ea8fc736ab58502bb067f016cab8b5acccdb353342c09fd64
+size 22168413

exp003_instruct/adapters/q3_relay_pw_task1_s1.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35e10631fce52b97e6b24056cf2331a5a0221c0a29c5b1bce2375113bbedf15f
+size 22168413

exp003_instruct/ar_differentiation_bed.py

@@ -0,0 +1,487 @@
+"""ar_differentiation_bed.py — exp012: autoregressive differentiation of the aleph.
+
+Differentiation is cultivated by PREDICTIVE pressure along the sequence — the
+address parameterizing the next-byte distribution (Law 2: chain-rule advantage pays
+ONLY where the composed address directly parameterizes the predictive distribution).
+This bed puts the aleph in the autoregressive gradient path and measures what
+differentiates. The head arms enforce the employment law at its maximum: the
+ENTIRE next-byte distribution is parameterized by the address.
+
+Byte-level causal LM on wikitext-2-raw (HF parquet, CDN-fast), block 256. ARMS:
+  sdpa       — standard causal transformer control (matched trunk).
+  hub        — attention replaced by CAUSAL HUB: linear attention whose feature map
+               is the 2K-oriented aleph address, prefix-sum memories (no selection
+               event; O(n*K*d)). Differentiation cultivated INSIDE attention.
+  addr_head  — sdpa trunk, but the OUTPUT HEAD reads ONLY the signed aleph
+               coefficient vector w_k = sinh(u_k)/sum_j cosh(u_j) of the final
+               hidden state (K -> 256 logits). The address MUST carry every bit of
+               next-byte information — the hardest Law-2 bottleneck.
+
+JUDGED BY: val bits-per-byte per arm (task) + CULTIVATION VITALS on every aleph
+codebook (readouts, never losses): axis aliveness/hppl, drift-from-init +
+binding fraction @0.29154, winner-|cos| saturation (sign-code emergence), shadow
+path diversity (fixed high-bits hash). Never by recon.
+
+Riders: pure Adam wd=0; no BN/Dropout/GAP on geometric paths; orthogonal init;
+Colab-cell-safe (paste-ahead imports, no bare argparse, no __file__ reliance);
+GPU-only for verdict runs.
+
+Terminal:  python ar_differentiation_bed.py            # shapes/parse smoke
+           python ar_differentiation_bed.py --train    # verdict run
+Colab:     paste geolip_vitals.py cell, then this file (smoke auto-runs),
+           then  train(steps=2000, data_root="/content/data")  in the next cell.
+"""
+from __future__ import annotations
+import math
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+if "anchor_drift" not in globals():
+    try:
+        from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is not None:
+            import sys, pathlib
+            sys.path.insert(0, str(pathlib.Path(_here).parent))
+            from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
+        else:
+            raise ImportError(
+                "geolip_vitals not found — paste/run its cell first, or "
+                "hf_hub_download exp012_ar/geolip_vitals.py from "
+                "AbstractPhil/geolip-aleph-differentiation.")
+
+VOCAB = 256  # bytes
+
+
+# ------------------------------------------------------------------ aleph address
+def _super_fibonacci_s3(n: int) -> torch.Tensor:
+    """Near-uniform unit quaternions (Alexa CVPR'22) —
+    starts the codebook INSIDE the RP^3 attractor basin. D=4 only."""
+    PHI, PSI = math.sqrt(2.0), 1.533751168755204288118041
+    i = torch.arange(n, dtype=torch.float64)
+    s = (i + 0.5) / n
+    r, R = torch.sqrt(s), torch.sqrt(1.0 - s)
+    a, b = 2 * math.pi * i / PHI, 2 * math.pi * i / PSI
+    q = torch.stack([r * torch.sin(a), r * torch.cos(a),
+                     R * torch.sin(b), R * torch.cos(b)], dim=-1)
+    return F.normalize(q, dim=-1).float()
+
+
+class AlephAddress(nn.Module):
+    """Closed-form aleph over 2K oriented half-axes (aleph-void article).
+    signed(x): (..., K) w_k = sinh(u_k)/sum_j cosh(u_j)   — the Law-2 head feature.
+    oriented(x): ((..., K), (..., K)) positive halves of the 2K softmax — HUB map."""
+
+    def __init__(self, K: int, D: int, tau: float = 0.1, init: str = "random"):
+        super().__init__()
+        self.K, self.D, self.tau = K, D, tau
+        if init == "fibonacci":
+            assert D == 4, "fibonacci init lives on S^3 (D=4)"
+            A = _super_fibonacci_s3(K)
+        else:
+            A = F.normalize(torch.randn(K, D), dim=-1)
+        self.codebook = nn.Parameter(A)
+        self.register_buffer("home", self.codebook.detach().clone())
+
+    def _u(self, x):
+        A = F.normalize(self.codebook, dim=-1)
+        return (F.normalize(x, dim=-1) @ A.transpose(-1, -2)) / self.tau
+
+    def oriented(self, x):
+        u = self._u(x)
+        m = u.abs().amax(dim=-1, keepdim=True)
+        ep, en = torch.exp(u - m), torch.exp(-u - m)
+        Z = (ep + en).sum(dim=-1, keepdim=True)
+        return ep / Z, en / Z
+
+    def signed(self, x):
+        u = self._u(x)
+        m = u.abs().amax(dim=-1, keepdim=True)
+        ep, en = torch.exp(u - m), torch.exp(-u - m)
+        return (ep - en) / (ep + en).sum(dim=-1, keepdim=True)
+
+    def signed_at(self, x, taus):
+        """Multi-tau stroboscope (rule of 3): signed coefficients at several
+        temperatures, concatenated — softer taus keep the vector dense while a
+        hard tau supplies the sign-code sharpness. v2 refinement (b)."""
+        A = F.normalize(self.codebook, dim=-1)
+        cos = F.normalize(x, dim=-1) @ A.transpose(-1, -2)
+        outs = []
+        for t in taus:
+            u = cos / t
+            m = u.abs().amax(dim=-1, keepdim=True)
+            ep, en = torch.exp(u - m), torch.exp(-u - m)
+            outs.append((ep - en) / (ep + en).sum(dim=-1, keepdim=True))
+        return torch.cat(outs, dim=-1)
+
+    def m_hat(self, x):
+        """Closed-form soft read (decoders read M_hat, never M). v2 control (c)."""
+        u = self._u(x)
+        m = u.abs().amax(dim=-1, keepdim=True)
+        ep, en = torch.exp(u - m), torch.exp(-u - m)
+        A = F.normalize(self.codebook, dim=-1)
+        return ((ep - en) @ A) / (ep + en).sum(dim=-1, keepdim=True)
+
+    def m_hard_ste(self, x):
+        """Hard mode (aleph-void article): M_hard = sign(cos_win) * A[win], straight-through to
+        the soft read — forward fully discrete SIGN CODE, backward soft gradient.
+        Legal per theme A (reconstructive sign code, not a one-hot roster pick)."""
+        u = self._u(x)
+        soft = self.m_hat(x)
+        win = u.abs().argmax(dim=-1)
+        A = F.normalize(self.codebook, dim=-1)
+        sign = torch.sign(torch.gather(u, -1, win.unsqueeze(-1))).squeeze(-1)
+        hard = sign.unsqueeze(-1) * A[win]
+        return hard + soft - soft.detach()
+
+    @torch.no_grad()
+    def vitals(self, x_sample) -> dict:
+        u = self._u(x_sample.reshape(-1, x_sample.shape[-1]))
+        p, n = self.oriented(x_sample.reshape(-1, x_sample.shape[-1]))
+        two_k = torch.cat([p, n], dim=-1)
+        win = two_k.argmax(dim=-1)
+        cos_win = (u.abs().amax(dim=-1) * self.tau)  # winner |cos| — sign-code sat.
+        d = anchor_drift(self.codebook, self.home)
+        return {"drift": round(d["mean"], 4),
+                "binding_frac": round(d["binding_fraction"], 4),
+                "aliveness": axis_aliveness(two_k),
+                "win_cos_mean": round(cos_win.mean().item(), 4),
+                "paths": path_diversity(win)}
+
+
+# ------------------------------------------------------------------------- blocks
+class CausalSDPA(nn.Module):
+    def __init__(self, d: int, heads: int = 4):
+        super().__init__()
+        self.h = heads
+        self.qkv = nn.Linear(d, 3 * d, bias=False)
+        self.o = nn.Linear(d, d, bias=False)
+        nn.init.orthogonal_(self.qkv.weight); nn.init.orthogonal_(self.o.weight)
+
+    def forward(self, x):
+        B, n, d = x.shape
+        q, k, v = self.qkv(x).chunk(3, dim=-1)
+        q, k, v = (t.view(B, n, self.h, d // self.h).transpose(1, 2) for t in (q, k, v))
+        y = F.scaled_dot_product_attention(q, k, v, is_causal=True)
+        return self.o(y.transpose(1, 2).reshape(B, n, d))
+
+
+class CausalHUB(nn.Module):
+    """Causal aleph linear attention: prefix-sum memories over the two K-wide
+    halves of the oriented address; 2K never materialized; no selection event."""
+
+    def __init__(self, d: int, K: int = 32, D: int = 4, tau: float = 0.1):
+        super().__init__()
+        self.addr = AlephAddress(K, D, tau)
+        self.q = nn.Linear(d, D, bias=False)
+        self.k = nn.Linear(d, D, bias=False)
+        self.v = nn.Linear(d, d, bias=False)
+        self.o = nn.Linear(d, d, bias=False)
+        for m in (self.q, self.k, self.v, self.o):
+            nn.init.orthogonal_(m.weight)
+
+    def forward(self, x):
+        qp, qn = self.addr.oriented(self.q(x))          # (B, n, K)
+        kp, kn = self.addr.oriented(self.k(x))
+        v = self.v(x)                                    # (B, n, d)
+        Sp = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kp, v), dim=1)
+        Sn = torch.cumsum(torch.einsum("bnk,bnd->bnkd", kn, v), dim=1)
+        zp = torch.cumsum(kp, dim=1)
+        zn = torch.cumsum(kn, dim=1)
+        num = torch.einsum("bnk,bnkd->bnd", qp, Sp) + torch.einsum("bnk,bnkd->bnd", qn, Sn)
+        den = (qp * zp).sum(-1, keepdim=True) + (qn * zn).sum(-1, keepdim=True)
+        return self.o(num / den.clamp_min(1e-12))
+
+
+class MslRelay(nn.Module):
+    """Depth-composition unit (chain-rule probe): multi-slot M_hat read entering
+    the trunk as a NEAR-ZERO gated residual (gate init -3.0, sigma~0.047 — theme D:
+    geometry enters as a nudge and grows only if it earns gradient)."""
+
+    def __init__(self, d: int, n_slots: int = 16, K: int = 64):
+        super().__init__()
+        self.n_slots = n_slots
+        self.proj = nn.Linear(d, n_slots * 4, bias=False)
+        self.out = nn.Linear(n_slots * 4, d, bias=False)
+        nn.init.orthogonal_(self.proj.weight)
+        nn.init.orthogonal_(self.out.weight)
+        self.addr = AlephAddress(K, 4)
+        self.gate = nn.Parameter(torch.tensor(-3.0))
+
+    def forward(self, x):
+        B, n, _ = x.shape
+        slots = self.proj(x).view(B, n, self.n_slots, 4)
+        m = self.addr.m_hat(slots).reshape(B, n, -1)
+        return x + self.gate.sigmoid() * self.out(m)
+
+
+class Block(nn.Module):
+    def __init__(self, d: int, attn: nn.Module):
+        super().__init__()
+        self.n1, self.n2 = nn.LayerNorm(d), nn.LayerNorm(d)
+        self.attn = attn
+        self.mlp = nn.Sequential(nn.Linear(d, 4 * d), nn.GELU(), nn.Linear(4 * d, d))
+
+    def forward(self, x):
+        x = x + self.attn(self.n1(x))
+        return x + self.mlp(self.n2(x))
+
+
+class ByteLM(nn.Module):
+    def __init__(self, arm: str, d: int = 192, layers: int = 4, block: int = 256,
+                 K: int = 32, D: int = 4):
+        super().__init__()
+        # "<arm>_tri" suffix = trigram byte embedding (AlephLM byte_emb x3 lineage):
+        # token embedding is the sum of embeddings of bytes t, t-1, t-2.
+        self.trigram = arm.endswith("_tri")
+        if self.trigram:
+            arm = arm[:-4]
+        # "_fib" = super-Fibonacci S^3 codebook init (basin test: starts INSIDE
+        # the RP^3 attractor; primary observable is init->final geodesic drift).
+        self.fib = arm.endswith("_fib")
+        if self.fib:
+            arm = arm[:-4]
+        # "relay*" = stacked addresses in depth: MslRelay after every block.
+        # relay -> sdpa trunk + standard head; relay_msl64 -> + addressed head.
+        self.use_relay = arm.startswith("relay")
+        if arm == "relay":
+            arm = "sdpa"
+        elif arm == "relay_msl64":
+            arm = "addr_msl64"
+        self.arm, self.block = arm, block
+        self.emb = nn.Embedding(VOCAB, d)
+        if self.trigram:
+            self.emb1 = nn.Embedding(VOCAB, d)
+            self.emb2 = nn.Embedding(VOCAB, d)
+        self.pos = nn.Parameter(torch.zeros(1, block, d) + 0.01 * torch.randn(1, block, d))
+        mk_attn = (lambda: CausalHUB(d, K, D)) if arm == "hub" else (lambda: CausalSDPA(d))
+        self.blocks = nn.ModuleList([Block(d, mk_attn()) for _ in range(layers)])
+        if self.use_relay:
+            self.relays = nn.ModuleList([MslRelay(d) for _ in range(layers)])
+        self.nf = nn.LayerNorm(d)
+        if arm == "addr_head":
+            self.head_addr = AlephAddress(K, d)  # v1: codebook in model dim — COLLAPSED
+            self.head = nn.Linear(K, VOCAB, bias=True)
+        elif arm in ("addr_d4", "addr_3tau", "addr_mhat"):
+            # v2 refinements: LOW-D HOME — learned projection to the native D=4 home
+            # before addressing (mirrors the healthy HUB arms), K=64.
+            self.head_proj = nn.Linear(d, 4, bias=False)
+            nn.init.orthogonal_(self.head_proj.weight)
+            self.head_addr = AlephAddress(64, 4)
+            if arm == "addr_d4":
+                self.head = nn.Linear(64, VOCAB, bias=True)      # w alone, D=4 home
+            elif arm == "addr_3tau":
+                self.taus = (0.05, 0.1, 0.3)                     # rule-of-3 strobe
+                self.head = nn.Linear(64 * 3, VOCAB, bias=True)
+            else:                                                 # addr_mhat
+                self.head = nn.Linear(4, VOCAB, bias=True)       # tightest: M_hat
+        elif arm.startswith("addr_msl"):
+            # v3: MULTI-SLOT heads — the 16s funnel widening: P parallel D=4 slots
+            # over a SHARED codebook. addr_msl consumes the reconstructive M_hat per
+            # slot (Px4 dims); addr_msl_w consumes signed w per slot (Px64) — tests
+            # whether slot-parallel consumption alone rescues the coefficient path.
+            # addr_msl<P> = slot-count dose-response. addr_mslh<P> = HARD sign-code
+            # consumption (straight-through M_hard per slot).
+            self.hard = arm.startswith("addr_mslh")
+            if arm in ("addr_msl", "addr_msl_w"):
+                self.n_slots = 16
+            else:
+                self.n_slots = int(arm[len("addr_mslh" if self.hard else "addr_msl"):])
+            self.head_proj = nn.Linear(d, self.n_slots * 4, bias=False)
+            nn.init.orthogonal_(self.head_proj.weight)
+            self.head_addr = AlephAddress(
+                64, 4, init="fibonacci" if self.fib else "random")
+            width = self.n_slots * (64 if arm == "addr_msl_w" else 4)
+            self.head = nn.Linear(width, VOCAB, bias=True)
+        elif arm == "addr_3tau_mhat":
+            # v3: combine the two v2 winners — 3-tau stroboscope + reconstructive read.
+            self.head_proj = nn.Linear(d, 4, bias=False)
+            nn.init.orthogonal_(self.head_proj.weight)
+            self.head_addr = AlephAddress(64, 4)
+            self.taus = (0.05, 0.1, 0.3)
+            self.head = nn.Linear(64 * 3 + 4, VOCAB, bias=True)
+        else:
+            self.head = nn.Linear(d, VOCAB, bias=True)
+        self._last_h = None
+
+    def forward(self, idx):
+        x = self.emb(idx)
+        if self.trigram:  # past-only shifts — causality preserved
+            x = x + self.emb1(F.pad(idx, (1, 0), value=0)[:, :-1]) \
+                  + self.emb2(F.pad(idx, (2, 0), value=0)[:, :-2])
+        x = x + self.pos[:, : idx.shape[1]]
+        if self.use_relay:
+            for b, r in zip(self.blocks, self.relays):
+                x = r(b(x))
+        else:
+            for b in self.blocks:
+                x = b(x)
+        h = self.nf(x)
+        self._last_h = h.detach()
+        if self.arm == "addr_head":
+            return self.head(self.head_addr.signed(h))
+        if self.arm == "addr_d4":
+            return self.head(self.head_addr.signed(self.head_proj(h)))
+        if self.arm == "addr_3tau":
+            return self.head(self.head_addr.signed_at(self.head_proj(h), self.taus))
+        if self.arm == "addr_mhat":
+            return self.head(self.head_addr.m_hat(self.head_proj(h)))
+        if self.arm.startswith("addr_msl"):
+            B, n, _ = h.shape
+            slots = self.head_proj(h).view(B, n, self.n_slots, 4)
+            if self.arm == "addr_msl_w":
+                feats = self.head_addr.signed(slots).reshape(B, n, -1)
+            elif getattr(self, "hard", False):
+                feats = self.head_addr.m_hard_ste(slots).reshape(B, n, -1)
+            else:
+                feats = self.head_addr.m_hat(slots).reshape(B, n, -1)
+            return self.head(feats)
+        if self.arm == "addr_3tau_mhat":
+            p = self.head_proj(h)
+            feats = torch.cat([self.head_addr.signed_at(p, self.taus),
+                               self.head_addr.m_hat(p)], dim=-1)
+            return self.head(feats)
+        return self.head(h)
+
+    @torch.no_grad()
+    def vitals(self) -> dict:
+        out = {}
+        if self.arm == "hub":
+            for i, b in enumerate(self.blocks):
+                if self._last_h is not None:
+                    out[f"L{i}"] = b.attn.addr.vitals(b.attn.q(self._last_h[:2]))
+        elif self.arm == "addr_head" and self._last_h is not None:
+            out["head"] = self.head_addr.vitals(self._last_h[:2])
+        elif self.arm in ("addr_d4", "addr_3tau", "addr_mhat",
+                          "addr_3tau_mhat") and self._last_h is not None:
+            out["head"] = self.head_addr.vitals(self.head_proj(self._last_h[:2]))
+        elif self.arm.startswith("addr_msl") and self._last_h is not None:
+            slots = self.head_proj(self._last_h[:2])
+            out["head"] = self.head_addr.vitals(
+                slots.reshape(*slots.shape[:-1], self.n_slots, 4))
+        if self.use_relay and self._last_h is not None:
+            for i, r in enumerate(self.relays):
+                s = r.proj(self._last_h[:2])
+                v = r.addr.vitals(s.reshape(*s.shape[:-1], r.n_slots, 4))
+                out[f"relay{i}"] = {"gate": round(r.gate.sigmoid().item(), 4),
+                                    "drift": v["drift"],
+                                    "binding_frac": v["binding_frac"],
+                                    "ppl": round(v["aliveness"]["usage_ppl"], 1)}
+        return out
+
+
+# --------------------------------------------------------------------------- data
+def _wikitext_bytes(data_root: str):
+    """wikitext-2-raw as flat uint8 tensors via the HF parquet CDN."""
+    from huggingface_hub import hf_hub_download
+    import pyarrow.parquet as pq
+
+    def load(split):
+        p = hf_hub_download("Salesforce/wikitext",
+                            f"wikitext-2-raw-v1/{split}-00000-of-00001.parquet",
+                            repo_type="dataset", local_dir=data_root)
+        text = "".join(pq.read_table(p).column("text").to_pylist())
+        return torch.frombuffer(bytearray(text.encode("utf-8")), dtype=torch.uint8).clone()
+
+    return load("train"), load("validation")
+
+
+def _batch(data: torch.Tensor, batch: int, block: int, device, g: torch.Generator):
+    ix = torch.randint(0, data.numel() - block - 1, (batch,), generator=g)
+    x = torch.stack([data[i:i + block] for i in ix]).long().to(device)
+    y = torch.stack([data[i + 1:i + block + 1] for i in ix]).long().to(device)
+    return x, y
+
+
+# -------------------------------------------------------------------- train/smoke
+def train(arms=("sdpa", "hub", "addr_head"), steps: int = 2000, batch: int = 32,
+          block: int = 256, device: str = "cuda", data_root: str = "./data",
+          seed: int = 0, eval_every: int = 500, save: bool = True):
+    """Verdict run — GPU only. Pure Adam wd=0. Reports val bits-per-byte + vitals.
+    save=True writes {data_root}/ar_ckpts/{arm}_s{seed}_t{steps}.pt per arm —
+    the cultivated codebooks are SPECIMENS for the projective reading instruments."""
+    import os
+    if device == "cuda" and not torch.cuda.is_available():
+        raise RuntimeError("Verdict runs are GPU-only (never CPU-train for accuracy).")
+    ckpt_dir = os.path.join(data_root, "ar_ckpts")
+    os.makedirs(ckpt_dir, exist_ok=True)
+    tr, va = _wikitext_bytes(data_root)
+    print(f"data ready: train {tr.numel():,} bytes, val {va.numel():,} bytes", flush=True)
+    results = {}
+    for arm in arms:
+        torch.manual_seed(seed)
+        g = torch.Generator().manual_seed(seed)
+        model = ByteLM(arm, block=block).to(device)
+        n_params = sum(p.numel() for p in model.parameters())
+        opt = torch.optim.Adam(model.parameters(), lr=3e-4, weight_decay=0.0)
+        for step in range(1, steps + 1):
+            x, y = _batch(tr, batch, block, device, g)
+            logits = model(x)
+            loss = F.cross_entropy(logits.reshape(-1, VOCAB), y.reshape(-1))
+            opt.zero_grad(set_to_none=True)
+            loss.backward()
+            opt.step()
+            if step % eval_every == 0 or step == steps:
+                model.eval()
+                with torch.no_grad():
+                    losses = []
+                    for _ in range(20):
+                        xv, yv = _batch(va, batch, block, device, g)
+                        lv = F.cross_entropy(model(xv).reshape(-1, VOCAB),
+                                             yv.reshape(-1))
+                        losses.append(lv.item())
+                bpb = sum(losses) / len(losses) / math.log(2)
+                print(f"[{arm}] step {step} val_bpb={bpb:.4f} vitals={model.vitals()}",
+                      flush=True)
+                model.train()
+        results[arm] = {"val_bpb": bpb, "params": n_params, "vitals": model.vitals()}
+        if save:
+            path = os.path.join(ckpt_dir, f"{arm}_s{seed}_t{steps}.pt")
+            torch.save({"arm": arm, "seed": seed, "steps": steps, "val_bpb": bpb,
+                        "state_dict": {k: v.cpu() for k, v in
+                                       model.state_dict().items()}}, path)
+            print(f"saved specimen: {path}", flush=True)
+    print(results, flush=True)
+    return results
+
+
+def smoke():
+    """Shapes/parse only — no accuracy claims."""
+    x = torch.randint(0, VOCAB, (2, 64))
+    for arm in ("sdpa", "hub", "addr_head"):
+        m = ByteLM(arm, d=96, layers=2, block=64, K=16)
+        logits = m(x)
+        assert logits.shape == (2, 64, VOCAB)
+        logits.sum().backward()
+        # causality check: future byte must not affect past logits
+        with torch.no_grad():
+            a = m(x)[0, 10]
+            x2 = x.clone(); x2[0, 40] = (x2[0, 40] + 7) % 256
+            b = m(x2)[0, 10]
+        assert torch.allclose(a, b, atol=1e-4), f"{arm} leaks future context"
+        print(f"{arm}: OK params={sum(p.numel() for p in m.parameters()):,} "
+              f"vitals={m.vitals()}", flush=True)
+    print("OK — AR bed smoke passed (verdict run: train() on GPU)", flush=True)
+
+
+def _in_notebook() -> bool:
+    try:
+        get_ipython()  # type: ignore[name-defined]  # noqa: F821
+        return True
+    except NameError:
+        return False
+
+
+if __name__ == "__main__":
+    if _in_notebook():
+        smoke()
+        print("Notebook mode: call  train(steps=2000)  in the next cell (GPU).")
+    else:
+        import argparse
+        ap = argparse.ArgumentParser()
+        ap.add_argument("--train", action="store_true")
+        ap.add_argument("--steps", type=int, default=2000)
+        a, _ = ap.parse_known_args()
+        train(steps=a.steps) if a.train else smoke()

exp003_instruct/build_results.py

@@ -0,0 +1,79 @@
+"""build_results.py — exp003_instruct: read results/ledger.jsonl and RE-ASSERT
+every claim in the README. The ledger carries BOTH validity generations:
+v1 rows (a parser bug — non-greedy regex truncated nested tool-call JSON —
+zeroed trained-arm validity; retained for audit) and the corrected
+'*_validity_v2' rows the claims are asserted on.
+Run from inside this folder: python build_results.py
+"""
+import json
+import os
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+rows = [json.loads(l) for l in
+        open(os.path.join(HERE, "results", "ledger.jsonl"), encoding="utf-8")]
+assert all(r["exp"] == "q3" for r in rows) and len(rows) == 15
+
+def v2(arm, seed):
+    return next(r for r in rows if r["arm"] == arm + "_validity_v2"
+                and r["seed"] == seed)["validity"]
+
+def base(arm, seed):
+    return next(r for r in rows if r["arm"] == arm and r["seed"] == seed)
+
+# claim 1: registers learned completely on held-out captions — all trained
+# arms reach ppl <= 1.07 on every register they trained on
+for arm in ("relay_pw_all", "mlp_all"):
+    for s in (0, 1):
+        assert all(v <= 1.07 for v in base(arm, s)["ppl"].values()), (arm, s)
+for s in (0, 1):
+    assert base("relay_pw_task1", s)["ppl"]["task_1"] <= 1.07
+
+# claim 2: hallucination reduction, quantified — specialist grounding
+# 0.958-0.979 vs frozen 0.542 (>10x fewer grounding failures), 2 seeds
+fz = v2("frozen", 0)["task_1"]["task_valid"]
+assert 0.50 < fz < 0.60, fz
+for s in (0, 1):
+    assert v2("relay_pw_task1", s)["task_1"]["task_valid"] >= 0.95, s
+
+# claim 3: register capture — the specialist emits perfect JSON in the WRONG
+# register for task_2/3 (json_valid 1.0, task_valid 0.0), both seeds
+for s in (0, 1):
+    for t in ("task_2", "task_3"):
+        c = v2("relay_pw_task1", s)[t]
+        assert c["json_valid"] >= 0.99 and c["task_valid"] == 0.0, (s, t)
+
+# claim 4: multi-task resolves capture — all three registers >= 0.875 valid
+# for both multi-task arms, both seeds; relay ~ mlp (validity saturated)
+for arm in ("relay_pw_all", "mlp_all"):
+    for s in (0, 1):
+        for t in ("task_1", "task_2", "task_3"):
+            assert v2(arm, s)[t]["task_valid"] >= 0.87, (arm, s, t)
+
+# claim 5: the register probe — instruct-trained relay stacks separate the 4
+# registers ~2x the wikitext stack at DEEP layers (L16/L23), all 4 probed
+# stacks vs the wikitext baseline
+wik = next(r for r in rows if r["arm"] == "wikitext_stack_probe")["register_sep"]
+for arm in ("relay_pw_task1", "relay_pw_all"):
+    for s in (0, 1):
+        sep = base(arm, s)["register_sep"]
+        for L in ("L16", "L23"):
+            assert sep[L]["sep"] >= 1.6 * wik[L]["sep"], (arm, s, L)
+
+# claim 6: single-register interference visible in cross-ppl — the specialist
+# damages task_3 relative to frozen at one seed (20.3 vs 13.6)
+assert base("relay_pw_task1", 1)["ppl"]["task_3"] > 15.0
+
+out = {"validity_v2": {f"{a}_s{s}": v2(a, s)
+                       for a in ("frozen", "relay_pw_task1", "relay_pw_all",
+                                 "mlp_all")
+                       for s in ((0,) if a == "frozen" else (0, 1))},
+       "register_sep_deep": {
+           "wikitext": {L: wik[L]["sep"] for L in ("L16", "L23")},
+           **{f"{a}_s{s}": {L: base(a, s)["register_sep"][L]["sep"]
+                            for L in ("L16", "L23")}
+              for a in ("relay_pw_task1", "relay_pw_all") for s in (0, 1)}},
+       "n_rows": len(rows)}
+json.dump(out, open(os.path.join(HERE, "results", "results.json"), "w",
+                    encoding="utf-8"), indent=1)
+print(f"{len(rows)} rows -> results/results.json")
+print("all README claims asserted OK")

exp003_instruct/exp013_augmentation_bed.py

@@ -0,0 +1,506 @@
+"""exp013_augmentation_bed.py — augmenting pretrained models with the aleph.
+Three tracks, one file (sequel to exp012's ar_differentiation_bed):
+
+  A. AUTOREGRESSION FROM CLIP-L: next-token prediction over CLIP-tokenized text,
+     reading FROZEN openai/clip-vit-large-patch14 text-tower hidden states.
+     FACTOR: extraction layer in {final, penultimate} (the last two layers — the
+     penultimate is what diffusion stacks consume). Heads at ~matched params:
+     linear | mlp | aleph multi-slot M_hat (P=64, D=4, shared K=64 — the exp012
+     certified construction) | sign-code (straight-through).
+  B. JOINT-FAILURE PROBES on frozen pooled embeddings (CLIP-L both layers + BERT):
+     b1 SPELLING-AR — decode a word's characters from ONLY the head's read of its
+        pooled embedding (GATE: linear/mlp must fail <50% exact first);
+     b2 ORDER — original-vs-shuffled discrimination (secondary; info may be absent).
+  C. GPT-2 (124M) AUGMENTATION: frozen trunk + trainable adapters after every block —
+     aleph MslRelay adapters vs param-matched MLP adapters vs frozen baseline;
+     gate growth by depth is a first-class readout (exp012 depth-gradient law).
+
+Riders: pure Adam wd=0; no contrastive/InfoNCE into address paths; vitals are
+readouts; GPU-only for verdict runs; caches/specimens live OUTSIDE the repo.
+Colab: paste geolip_vitals.py, then ar_differentiation_bed.py, then this file.
+"""
+from __future__ import annotations
+import json
+import math
+import os
+import re
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+# ---- paste-ahead imports (notebook-safe) -----------------------------------------
+if "anchor_drift" not in globals():
+    try:
+        from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is None:
+            raise ImportError("paste/run geolip_vitals.py first")
+        import sys, pathlib
+        sys.path.insert(0, str(pathlib.Path(_here).parent))
+        from geolip_vitals import anchor_drift, axis_aliveness, path_diversity
+if "AlephAddress" not in globals():
+    try:
+        from ar_differentiation_bed import AlephAddress, MslRelay, _wikitext_bytes
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is None:
+            raise ImportError("paste/run ar_differentiation_bed.py first")
+        from ar_differentiation_bed import AlephAddress, MslRelay, _wikitext_bytes
+
+DATA_ROOT = os.environ.get("GEOLIP_DATA", "./data")
+
+
+class SquaredReLU(nn.Module):
+    def forward(self, x):
+        return F.relu(x) ** 2
+
+
+# =============================================================== head arm family ==
+class HeadArm(nn.Module):
+    """Conditioning transform d_in -> F_OUT (256), then a shared-shape task head.
+    Arms: linear | mlp | aleph (multi-slot M_hat) | sign (straight-through M_hard).
+    The aleph arms are the exp012-certified construction: P=64 slots x D=4 over one
+    shared K=64 codebook."""
+    F_OUT = 256
+
+    def __init__(self, arm: str, d_in: int, mlp_hidden: int = 192):
+        # mlp_hidden=192 param-matches the aleph arm at d_in=768:
+        # mlp ~ 192*(768+256)+LN ~ 197K vs aleph proj 768*256 + codebook = 196.9K
+        super().__init__()
+        self.arm = arm
+        if arm == "linear":
+            self.net = nn.Linear(d_in, self.F_OUT, bias=False)
+        elif arm == "mlp":
+            self.net = nn.Sequential(nn.Linear(d_in, mlp_hidden), SquaredReLU(),
+                                     nn.LayerNorm(mlp_hidden),
+                                     nn.Linear(mlp_hidden, self.F_OUT))
+        elif arm in ("aleph", "sign"):
+            self.proj = nn.Linear(d_in, 64 * 4, bias=False)
+            nn.init.orthogonal_(self.proj.weight)
+            self.addr = AlephAddress(64, 4)
+        else:
+            raise ValueError(arm)
+
+    def forward(self, x):
+        if self.arm in ("linear", "mlp"):
+            return self.net(x)
+        slots = self.proj(x).reshape(*x.shape[:-1], 64, 4)
+        read = self.addr.m_hard_ste(slots) if self.arm == "sign" \
+            else self.addr.m_hat(slots)
+        return read.reshape(*x.shape[:-1], 256)
+
+    @torch.no_grad()
+    def vitals(self, x_sample) -> dict:
+        if self.arm in ("linear", "mlp"):
+            return {}
+        slots = self.proj(x_sample).reshape(*x_sample.shape[:-1], 64, 4)
+        p, n = self.addr.oriented(slots)
+        two_k = torch.cat([p, n], -1).reshape(-1, 128)
+        d = anchor_drift(self.addr.codebook, self.addr.home)
+        return {"drift": round(d["mean"], 4),
+                "binding_frac": round(d["binding_fraction"], 4),
+                "usage_ppl": round(axis_aliveness(two_k)["usage_ppl"], 1),
+                "paths": path_diversity(two_k.argmax(-1))["unique_hashed"]}
+
+    def param_count(self):
+        return sum(p.numel() for p in self.parameters())
+
+
+# ================================================================== caches ========
+def _wikitext_lines(data_root, min_chars=40, max_lines=None):
+    from huggingface_hub import hf_hub_download
+    import pyarrow.parquet as pq
+    out = {}
+    for split, cap in (("train", max_lines), ("validation", None)):
+        p = hf_hub_download("Salesforce/wikitext",
+                            f"wikitext-2-raw-v1/{split}-00000-of-00001.parquet",
+                            repo_type="dataset", local_dir=data_root)
+        lines = [t.strip() for t in pq.read_table(p).column("text").to_pylist()
+                 if len(t.strip()) >= min_chars]
+        out[split] = lines[:cap] if cap else lines
+    return out["train"], out["validation"]
+
+
+@torch.no_grad()
+def cache_clip(data_root, n_train=12000, n_val=1500, device="cuda", batch=64):
+    """CLIP-L text tower over wikitext lines; caches BOTH of the last two layers.
+    hidden_states[-1] == the final encoder layer output (pre final-LN),
+    last_hidden_state == final-LN(final layer). We cache:
+      'final'  = last_hidden_state (what the projection head consumes),
+      'penult' = hidden_states[-2] (the layer diffusion stacks consume).
+    Also caches pooled (EOS-position) vectors for both layers, and token ids."""
+    from transformers import CLIPTextModel, CLIPTokenizerFast
+    path = os.path.join(data_root, "exp013", "clip_cache.pt")
+    if os.path.exists(path):
+        return torch.load(path, map_location="cpu", weights_only=True)
+    os.makedirs(os.path.dirname(path), exist_ok=True)
+    tok = CLIPTokenizerFast.from_pretrained("openai/clip-vit-large-patch14")
+    model = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14").to(device).eval()
+    tr_lines, va_lines = _wikitext_lines(data_root, max_lines=n_train)
+    va_lines = va_lines[:n_val]
+    def encode(lines):
+        H_f, H_p, IDS, EOS = [], [], [], []
+        for i in range(0, len(lines), batch):
+            enc = tok(lines[i:i + batch], padding="max_length", truncation=True,
+                      max_length=77, return_tensors="pt").to(device)
+            out = model(**enc, output_hidden_states=True)
+            H_f.append(out.last_hidden_state.half().cpu())
+            H_p.append(out.hidden_states[-2].half().cpu())
+            IDS.append(enc.input_ids.cpu())
+            EOS.append(enc.input_ids.argmax(-1).cpu())  # EOT id is the max token id
+        return (torch.cat(H_f), torch.cat(H_p), torch.cat(IDS), torch.cat(EOS))
+    tr = encode(tr_lines)
+    va = encode(va_lines)
+    blob = {"train": {"final": tr[0], "penult": tr[1], "ids": tr[2], "eos": tr[3]},
+            "val":   {"final": va[0], "penult": va[1], "ids": va[2], "eos": va[3]},
+            "vocab": tok.vocab_size}
+    torch.save(blob, path)
+    print(f"clip cache: train {tr[0].shape}, val {va[0].shape} -> {path}", flush=True)
+    return blob
+
+
+@torch.no_grad()
+def cache_word_embeddings(data_root, n_words=10000, device="cuda", batch=256):
+    """Pooled embeddings of frequent wikitext words for the spelling probe:
+    CLIP-L final + penultimate (EOS-pooled) and BERT (CLS + mean of last layer)."""
+    from transformers import (CLIPTextModel, CLIPTokenizerFast,
+                              BertModel, BertTokenizerFast)
+    path = os.path.join(data_root, "exp013", "word_cache.pt")
+    if os.path.exists(path):
+        return torch.load(path, map_location="cpu", weights_only=True)
+    os.makedirs(os.path.dirname(path), exist_ok=True)
+    tr_lines, _ = _wikitext_lines(data_root)
+    from collections import Counter
+    cnt = Counter(w for l in tr_lines for w in re.findall(r"[a-z]{3,12}", l.lower()))
+    words = [w for w, _ in cnt.most_common(n_words)]
+    blob = {"words": words}
+    ct = CLIPTokenizerFast.from_pretrained("openai/clip-vit-large-patch14")
+    cm = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14").to(device).eval()
+    fin, pen = [], []
+    for i in range(0, len(words), batch):
+        enc = ct(words[i:i + batch], padding="max_length", truncation=True,
+                 max_length=77, return_tensors="pt").to(device)
+        out = cm(**enc, output_hidden_states=True)
+        eos = enc.input_ids.argmax(-1)
+        idx = torch.arange(eos.numel(), device=device)
+        fin.append(out.last_hidden_state[idx, eos].half().cpu())
+        pen.append(out.hidden_states[-2][idx, eos].half().cpu())
+    blob["clip_final"], blob["clip_penult"] = torch.cat(fin), torch.cat(pen)
+    del cm
+    bt = BertTokenizerFast.from_pretrained("bert-base-uncased")
+    bm = BertModel.from_pretrained("bert-base-uncased").to(device).eval()
+    cls, mean = [], []
+    for i in range(0, len(words), batch):
+        enc = bt(words[i:i + batch], padding=True, truncation=True,
+                 max_length=16, return_tensors="pt").to(device)
+        out = bm(**enc).last_hidden_state
+        m = enc.attention_mask.unsqueeze(-1)
+        cls.append(out[:, 0].half().cpu())
+        mean.append(((out * m).sum(1) / m.sum(1)).half().cpu())
+    blob["bert_cls"], blob["bert_mean"] = torch.cat(cls), torch.cat(mean)
+    torch.save(blob, path)
+    print(f"word cache: {len(words)} words -> {path}", flush=True)
+    return blob
+
+
+# ============================================================ track A =============
+def track_a(arms=("linear", "mlp", "aleph", "sign"), layers=("final", "penult"),
+            steps=1500, batch=64, seed=0, device="cuda",
+            data_root=DATA_ROOT, eval_every=500, save=True):
+    """Next-CLIP-token prediction from frozen CLIP-L hidden states."""
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    blob = cache_clip(data_root, device=device)
+    V = blob["vocab"]
+    results = {}
+    ck_dir = os.path.join(data_root, "exp013", "ckpts")
+    os.makedirs(ck_dir, exist_ok=True)
+    for layer in layers:
+        Htr = blob["train"][layer].float()
+        ids_tr = blob["train"]["ids"]
+        Hva = blob["val"][layer].float()
+        ids_va = blob["val"]["ids"]
+        for arm in arms:
+            torch.manual_seed(seed)
+            g = torch.Generator().manual_seed(seed)
+            head = HeadArm(arm, Htr.shape[-1]).to(device)
+            out_proj = nn.Linear(HeadArm.F_OUT, V).to(device)
+            params = list(head.parameters()) + list(out_proj.parameters())
+            opt = torch.optim.Adam(params, lr=3e-4, weight_decay=0.0)
+            n_par = sum(p.numel() for p in params)
+            for step in range(1, steps + 1):
+                ix = torch.randint(0, Htr.shape[0], (batch,), generator=g)
+                h = Htr[ix].to(device)
+                y = ids_tr[ix].to(device)
+                logits = out_proj(head(h[:, :-1]))
+                loss = F.cross_entropy(logits.reshape(-1, V), y[:, 1:].reshape(-1))
+                opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+                if step % eval_every == 0 or step == steps:
+                    with torch.no_grad():
+                        ls = []
+                        for j in range(0, min(1024, Hva.shape[0]), batch):
+                            h = Hva[j:j + batch].to(device)
+                            y = ids_va[j:j + batch].to(device)
+                            lg = out_proj(head(h[:, :-1]))
+                            ls.append(F.cross_entropy(
+                                lg.reshape(-1, V), y[:, 1:].reshape(-1)).item())
+                        ce = sum(ls) / len(ls)
+                    vit = head.vitals(Hva[:2, :8].to(device))
+                    print(f"[A {layer} {arm} s{seed}] step {step} val_ce={ce:.4f} "
+                          f"params={n_par:,} vitals={vit}", flush=True)
+            results[f"{layer}/{arm}/s{seed}"] = {"val_ce": ce, "params": n_par,
+                                                 "vitals": vit}
+            if save and arm in ("aleph", "sign"):
+                torch.save({"track": "A", "layer": layer, "arm": arm, "seed": seed,
+                            "val_ce": ce, "state_dict": {k: v.cpu() for k, v in
+                                                          head.state_dict().items()}},
+                           os.path.join(ck_dir, f"A_{layer}_{arm}_s{seed}.pt"))
+    print(results, flush=True)
+    return results
+
+
+# ============================================================ track B =============
+class CharDecoder(nn.Module):
+    """Tiny GRU char decoder conditioned ONLY on the arm's 256-d read."""
+    CHARS = "abcdefghijklmnopqrstuvwxyz"
+    def __init__(self, cond_dim=256, hidden=256):
+        super().__init__()
+        self.V = len(self.CHARS) + 2  # +BOS +EOS
+        self.emb = nn.Embedding(self.V, 64)
+        self.init = nn.Linear(cond_dim, hidden)
+        self.gru = nn.GRU(64, hidden, batch_first=True)
+        self.out = nn.Linear(hidden, self.V)
+
+    def encode_word(self, w):
+        return [1] + [2 + self.CHARS.index(c) for c in w] + [0]  # BOS..EOS(0)
+
+    def forward(self, cond, tgt):          # tgt: (B, L) int, teacher-forced
+        h0 = torch.tanh(self.init(cond)).unsqueeze(0)
+        x = self.emb(tgt[:, :-1])
+        y, _ = self.gru(x, h0)
+        return self.out(y)                  # predict tgt[:,1:]
+
+    @torch.no_grad()
+    def greedy(self, cond, max_len=14):
+        B = cond.shape[0]
+        h = torch.tanh(self.init(cond)).unsqueeze(0)
+        t = torch.ones(B, 1, dtype=torch.long, device=cond.device)
+        done = torch.zeros(B, dtype=torch.bool, device=cond.device)
+        outs = []
+        for _ in range(max_len):
+            y, h = self.gru(self.emb(t), h)
+            t = self.out(y).argmax(-1)
+            outs.append(t)
+            done |= (t.squeeze(1) == 0)
+            if done.all():
+                break
+        return torch.cat(outs, 1)
+
+
+def track_b1(arms=("linear", "mlp"), substrates=("clip_final", "clip_penult",
+             "bert_cls", "bert_mean"), steps=3000, batch=128, seed=0,
+             device="cuda", data_root=DATA_ROOT, save=True):
+    """Spelling-AR from pooled embeddings. Run baselines first (the GATE:
+    qualify the task only if linear/mlp exact-match < 0.50), then aleph/sign."""
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    blob = cache_word_embeddings(data_root, device=device)
+    words = blob["words"]
+    dec_tpl = CharDecoder()
+    enc = [dec_tpl.encode_word(w) for w in words]
+    L = max(len(e) for e in enc)
+    tgt = torch.zeros(len(enc), L, dtype=torch.long)
+    for i, e in enumerate(enc):
+        tgt[i, :len(e)] = torch.tensor(e)
+    g0 = torch.Generator().manual_seed(1234)          # fixed split across arms
+    perm = torch.randperm(len(words), generator=g0)
+    tr_ix, va_ix = perm[:9000], perm[9000:]
+    ck_dir = os.path.join(data_root, "exp013", "ckpts")
+    os.makedirs(ck_dir, exist_ok=True)
+    results = {}
+    for sub in substrates:
+        E = blob[sub].float()
+        for arm in arms:
+            torch.manual_seed(seed)
+            g = torch.Generator().manual_seed(seed)
+            head = HeadArm(arm, E.shape[-1]).to(device)
+            dec = CharDecoder().to(device)
+            params = list(head.parameters()) + list(dec.parameters())
+            opt = torch.optim.Adam(params, lr=1e-3, weight_decay=0.0)
+            for step in range(1, steps + 1):
+                ix = tr_ix[torch.randint(0, tr_ix.numel(), (batch,), generator=g)]
+                cond = head(E[ix].to(device))
+                t = tgt[ix].to(device)
+                lg = dec(cond, t)
+                mask = (t[:, 1:] != 0) | (torch.cumsum(t[:, 1:] == 0, 1) == 1)
+                loss = F.cross_entropy(lg[mask], t[:, 1:][mask])
+                opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+            with torch.no_grad():
+                cond = head(E[va_ix].to(device))
+                pred = dec.greedy(cond)
+                t = tgt[va_ix, 1:].to(device)
+                n = min(pred.shape[1], t.shape[1])
+                pad_ok = torch.ones_like(t[:, :n], dtype=torch.bool)
+                seen_eos = torch.cumsum(t[:, :n] == 0, 1) > 0
+                match = ((pred[:, :n] == t[:, :n]) | seen_eos).all(-1)
+                exact = match.float().mean().item()
+            vit = head.vitals(E[va_ix[:16]].to(device))
+            print(f"[B1 {sub} {arm} s{seed}] exact={exact:.4f} vitals={vit}", flush=True)
+            results[f"{sub}/{arm}/s{seed}"] = {"exact": exact, "vitals": vit}
+            if save and arm in ("aleph", "sign"):
+                torch.save({"track": "B1", "sub": sub, "arm": arm, "seed": seed,
+                            "exact": exact, "state_dict": {k: v.cpu() for k, v in
+                                                            head.state_dict().items()}},
+                           os.path.join(ck_dir, f"B1_{sub}_{arm}_s{seed}.pt"))
+    print(results, flush=True)
+    return results
+
+
+# ============================================================ track C =============
+class MLPAdapter(nn.Module):
+    """Param-matched plain adapter (the ablation twin of MslRelay).
+    hidden=64 matches MslRelay at d=768 (2*768*64=98.3K vs 98.6K incl codebook).
+    Output layer ZERO-INIT (standard adapter stabilization — the first version
+    diverged at lr 1e-3 with random init; the aleph relay needed no such aid,
+    which is itself a datapoint, but the control gets its best shot)."""
+    def __init__(self, d, hidden=64):
+        super().__init__()
+        out = nn.Linear(hidden, d)
+        nn.init.zeros_(out.weight)
+        nn.init.zeros_(out.bias)
+        self.net = nn.Sequential(nn.Linear(d, hidden), SquaredReLU(), out)
+        self.gate = nn.Parameter(torch.tensor(-3.0))
+
+    def forward(self, x):
+        return x + self.gate.sigmoid() * self.net(x)
+
+
+class _BlockWithAdapter(nn.Module):
+    def __init__(self, block, adapter):
+        super().__init__()
+        self.block, self.adapter = block, adapter
+
+    def forward(self, *a, **k):
+        out = self.block(*a, **k)
+        if isinstance(out, tuple):
+            return (self.adapter(out[0]),) + out[1:]
+        return self.adapter(out)
+
+
+def track_c(arms=("frozen", "aleph", "mlp"), steps=1500, batch=8, block=256,
+            seed=0, device="cuda", data_root=DATA_ROOT,
+            eval_every=500, save=True):
+    """GPT-2 124M frozen; adapters after every block; train adapters only."""
+    from transformers import GPT2LMHeadModel, GPT2TokenizerFast
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    tok = GPT2TokenizerFast.from_pretrained("gpt2")
+    tr_lines, va_lines = _wikitext_lines(data_root)
+    def to_stream(lines):
+        ids = tok("\n\n".join(lines), return_tensors="pt").input_ids[0]
+        return ids
+    stream_tr = to_stream(tr_lines[:8000])
+    stream_va = to_stream(va_lines[:1000])
+    ck_dir = os.path.join(data_root, "exp013", "ckpts")
+    os.makedirs(ck_dir, exist_ok=True)
+    results = {}
+    for arm in arms:
+        torch.manual_seed(seed)
+        g = torch.Generator().manual_seed(seed)
+        model = GPT2LMHeadModel.from_pretrained("gpt2").to(device)
+        for p in model.parameters():
+            p.requires_grad_(False)
+        adapters = []
+        if arm != "frozen":
+            d = model.config.n_embd
+            for i, blk in enumerate(model.transformer.h):
+                ad = (MslRelay(d) if arm == "aleph" else MLPAdapter(d)).to(device)
+                model.transformer.h[i] = _BlockWithAdapter(blk, ad)
+                adapters.append(ad)
+            params = [p for ad in adapters for p in ad.parameters()]
+            n_par = sum(p.numel() for p in params)
+            opt = torch.optim.Adam(params, lr=1e-3, weight_decay=0.0)
+        else:
+            params, n_par = [], 0
+        def eval_ppl():
+            model.eval()
+            with torch.no_grad():
+                ls = []
+                for j in range(0, stream_va.numel() - block - 1, block * 4):
+                    x = stream_va[j:j + block].unsqueeze(0).to(device)
+                    out = model(x, labels=x)
+                    ls.append(out.loss.item())
+            model.train()
+            return math.exp(sum(ls) / len(ls))
+        if arm == "frozen":
+            ppl = eval_ppl()
+            print(f"[C frozen] ppl={ppl:.3f}", flush=True)
+            results["frozen"] = {"ppl": ppl}
+            continue
+        for step in range(1, steps + 1):
+            ix = torch.randint(0, stream_tr.numel() - block - 1, (batch,), generator=g)
+            x = torch.stack([stream_tr[i:i + block] for i in ix]).to(device)
+            loss = model(x, labels=x).loss
+            opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+            if step % eval_every == 0 or step == steps:
+                ppl = eval_ppl()
+                gates = [round(ad.gate.sigmoid().item(), 4) for ad in adapters]
+                vit = {}
+                if arm == "aleph":
+                    drifts = [round(anchor_drift(ad.addr.codebook, ad.addr.home)
+                                    ["mean"], 3) for ad in adapters]
+                    vit = {"drift_by_depth": drifts}
+                print(f"[C {arm} s{seed}] step {step} ppl={ppl:.3f} "
+                      f"params={n_par:,} gates={gates} {vit}", flush=True)
+        results[f"{arm}/s{seed}"] = {"ppl": ppl, "params": n_par, "gates": gates,
+                                     **vit}
+        if save and arm == "aleph":
+            torch.save({"track": "C", "arm": arm, "seed": seed, "ppl": ppl,
+                        "state_dict": {f"relay{i}.{k}": v.cpu()
+                                       for i, ad in enumerate(adapters)
+                                       for k, v in ad.state_dict().items()}},
+                       os.path.join(ck_dir, f"C_{arm}_s{seed}.pt"))
+    print(results, flush=True)
+    return results
+
+
+# ================================================================ smoke ===========
+def smoke():
+    """Shapes/parse only — no substrates, no training."""
+    for arm in ("linear", "mlp", "aleph", "sign"):
+        h = HeadArm(arm, 768)
+        y = h(torch.randn(2, 10, 768))
+        assert y.shape == (2, 10, 256)
+        y.sum().backward()
+        print(arm, "OK", f"{h.param_count():,}", h.vitals(torch.randn(2, 4, 768)))
+    dec = CharDecoder()
+    t = torch.tensor([dec.encode_word("hello") + [0] * 3,
+                      dec.encode_word("worlds") + [0] * 2])
+    lg = dec(torch.randn(2, 256), t)
+    assert lg.shape[:2] == (2, t.shape[1] - 1)
+    print("decoder OK; greedy:", dec.greedy(torch.randn(2, 256)).shape)
+    ad = MLPAdapter(768)
+    assert ad(torch.randn(2, 4, 768)).shape == (2, 4, 768)
+    print("adapter OK — exp013 smoke passed (caches+tracks need GPU+transformers)")
+
+
+def _in_notebook():
+    try:
+        get_ipython()  # type: ignore[name-defined]  # noqa: F821
+        return True
+    except NameError:
+        return False
+
+
+if __name__ == "__main__":
+    if _in_notebook():
+        smoke()
+        print("Notebook: cache_clip()/cache_word_embeddings() then "
+              "track_b1() gate -> track_a() -> track_c().")
+    else:
+        import argparse
+        ap = argparse.ArgumentParser()
+        ap.add_argument("--track", default="smoke")
+        a, _ = ap.parse_known_args()
+        {"smoke": smoke, "a": track_a, "b1": track_b1, "c": track_c}[a.track]()

exp003_instruct/geolip_vitals.py

@@ -0,0 +1,219 @@
+"""geolip_vitals.py — shared diagnostic harness for the GeoLIP aleph experiments.
+ALL functions are READOUTS: no gradients, no losses. CV is a readout, never a
+force. Addressing is judged by drift->0.29154 and CV->0.20, never by recon cosine
+(judgment criteria per the aleph-void article: https://huggingface.co/blog/AbstractPhil/geometric-vocabulary-patchwork-aleph-void).
+
+Vitals provided:
+  anchor_drift        — geodesic drift of anchors from init; binding fraction @0.29154
+  pentachoron_cv      — CM 4-volume CV over random 5-row subsets (geovocab2 import)
+  axis_aliveness      — oriented-address usage: axes alive, hppl, collapse flag
+  gate_stats          — gate means vs the 0.012-0.03 band
+  path_diversity      — unique-path counting, FIXED high-bits hash (low-16 bug is the
+                        retracted artifact — never use the low bits)
+  grad_norm_spread    — gradient democracy monitor (orders-of-magnitude spread)
+  CVScreen            — CV@1000-batch early band screen (<0.30 LOW / .35-.50 MID / >.80 HIGH)
+
+Smoke on a torch-capable env:  python geolip_vitals.py
+"""
+from __future__ import annotations
+import math
+import torch
+
+BINDING = 0.29154            # radians; the binding/separation constant
+CV_BAND = (0.13, 0.30)       # CM CV band (discovery_catalog #4)
+GATE_BAND = (0.012, 0.03)    # live invariant candidate (acd_campaign)
+KNUTH32 = 2654435761
+
+
+# ----------------------------------------------------------------------------- drift
+@torch.no_grad()
+def anchor_drift(current: torch.Tensor, init: torch.Tensor, tol: float = 0.05) -> dict:
+    """Geodesic drift (radians) of each row of `current` from its row in `init`,
+    both row-normalized. Returns mean/std/per-row drift and the fraction of rows
+    within +/-tol of BINDING (the GLFM '46%' readout)."""
+    a = torch.nn.functional.normalize(current.float(), dim=-1)
+    b = torch.nn.functional.normalize(init.float(), dim=-1)
+    cos = (a * b).sum(-1).clamp(-1.0, 1.0)
+    drift = torch.arccos(cos)
+    frac = ((drift - BINDING).abs() <= tol).float().mean()
+    return {"mean": drift.mean().item(), "std": drift.std().item(),
+            "per_row": drift, "binding_fraction": frac.item()}
+
+
+# -------------------------------------------------------------------------------- cv
+@torch.no_grad()
+def _pentachoron_volumes(pts: torch.Tensor) -> torch.Tensor:
+    """Batched Cayley-Menger 4-simplex volumes. pts: (B, 5, D) -> (B,) volumes.
+    One float64 det over all samples (vol^2 = -det(CM)/9216 for n=4). Built-in
+    for speed (the per-sample reference path is ~260x slower in a vitals loop);
+    geovocab2 remains the formula's reference implementation, parity-checked
+    via cv_reference_check()."""
+    B = pts.shape[0]
+    d2 = torch.cdist(pts.double(), pts.double()).pow(2)          # (B,5,5)
+    cm = torch.ones(B, 6, 6, dtype=torch.float64, device=pts.device)
+    cm[:, 0, 0] = 0.0
+    cm[:, 1:, 1:] = d2
+    det = torch.linalg.det(cm)
+    return (-det / 9216.0).clamp_min(0.0).sqrt().float()
+
+
+@torch.no_grad()
+def pentachoron_cv(rows: torch.Tensor, n_samples: int = 200,
+                   generator: torch.Generator | None = None) -> float:
+    """CV (std/mean) of Cayley-Menger 4-simplex volumes over n_samples random
+    5-row subsets. Rows are row-normalized before measurement. Uses the built-in
+    batched CM (float64 det); validate against geovocab2 with
+    cv_reference_check() after any change to the volume math."""
+    x = torch.nn.functional.normalize(rows.float(), dim=-1)
+    n = x.shape[0]
+    if n < 5:
+        raise ValueError(f"pentachoron_cv needs >=5 rows, got {n}")
+    g = generator or torch.Generator(device="cpu").manual_seed(0)
+    idx = torch.stack([torch.randperm(n, generator=g)[:5]
+                       for _ in range(n_samples)])               # (B,5)
+    v = _pentachoron_volumes(x[idx].cpu())
+    return (v.std() / v.mean().clamp_min(1e-12)).item()
+
+
+@torch.no_grad()
+def cv_reference_check(n_trials: int = 50, tol: float = 1e-5) -> float:
+    """Parity check of the built-in batched CM against geovocab2's reference
+    implementation (the formula's source of truth). Returns max |rel diff|;
+    raises if geovocab2 is absent or parity fails. Run after touching
+    _pentachoron_volumes."""
+    try:
+        from geovocab2.shapes.formula.symbolic.cayley_menger import (
+            CayleyMengerFromSimplex)
+    except Exception as e:  # pragma: no cover
+        raise ImportError(
+            "cv_reference_check requires geovocab2 (install via the geolip-svae "
+            "umbrella: pip install git+https://github.com/AbstractEyes/"
+            "geolip-svae).") from e
+    ref = CayleyMengerFromSimplex()
+    g = torch.Generator().manual_seed(0)
+    pts = torch.nn.functional.normalize(
+        torch.randn(n_trials, 5, 4, generator=g), dim=-1)
+    mine = _pentachoron_volumes(pts)
+    # compare at float64: the reference computes in the INPUT dtype, and fp32
+    # dets lose up to ~4% on near-degenerate pentachora (measured 2026-07-11)
+    theirs = torch.stack([ref.forward(p.double())["volume"].float() for p in pts])
+    rel = ((mine - theirs).abs() / theirs.abs().clamp_min(1e-12)).max().item()
+    if rel > tol:
+        raise AssertionError(f"CM parity vs geovocab2 failed: max rel {rel}")
+    return rel
+
+
+# ------------------------------------------------------------------------- aliveness
+@torch.no_grad()
+def axis_aliveness(oriented_weights: torch.Tensor, alive_thresh: float = 1e-3) -> dict:
+    """`oriented_weights`: (..., 2K) nonnegative oriented-softmax address rows
+    (sum to 1 on the last dim). Returns axes-alive count, mean-usage perplexity
+    (hppl analogue; healthy hosted reference 125-126/128), and a collapse flag.
+    Reference behavior: near-uniform aliveness at div_weight=0 (discovery #22)."""
+    w = oriented_weights.reshape(-1, oriented_weights.shape[-1]).float()
+    usage = w.mean(0)
+    usage = usage / usage.sum().clamp_min(1e-12)
+    # an axis is alive if its mean usage exceeds alive_thresh x the uniform share
+    alive = int((usage > alive_thresh * (1.0 / usage.numel())).sum())
+    ent = -(usage.clamp_min(1e-12) * usage.clamp_min(1e-12).log()).sum()
+    ppl = float(ent.exp())
+    return {"axes_total": usage.numel(), "axes_alive": alive, "usage_ppl": ppl,
+            "collapsed": ppl < 0.05 * usage.numel()}
+
+
+# ------------------------------------------------------------------------------ gates
+@torch.no_grad()
+def gate_stats(gates: torch.Tensor) -> dict:
+    """Gate values (post-sigmoid/clamp). Reports mean and whether it sits in the
+    0.012-0.03 band (read-only — the band is a candidate invariant, never a target)."""
+    g = gates.float().flatten()
+    m = g.mean().item()
+    return {"mean": m, "std": g.std().item(),
+            "in_band": GATE_BAND[0] <= m <= GATE_BAND[1]}
+
+
+# ------------------------------------------------------------------------------ paths
+@torch.no_grad()
+def path_diversity(ids: torch.Tensor) -> dict:
+    """Unique-path counting with the FIXED multiplicative hash:
+    ((ids * 2654435761) % 2^32) >> 16 — Knuth needs the HIGH bits; the low-16
+    variant produced a retracted ~1,500 path ceiling in a prior campaign.
+    `ids`: integer tensor, one composed path id per row (any shape)."""
+    x = ids.reshape(-1).to(torch.int64)
+    hashed = ((x * KNUTH32) % (1 << 32)) >> 16
+    return {"n": int(x.numel()),
+            "unique_raw": int(torch.unique(x).numel()),
+            "unique_hashed": int(torch.unique(hashed).numel())}
+
+
+@torch.no_grad()
+def compose_path_ids(stage_indices: list[torch.Tensor], radix: int) -> torch.Tensor:
+    """Compose per-stage discrete indices (each (...,) int in [0, radix)) into a
+    single path id, positional base-`radix` — construction, not hashing."""
+    out = torch.zeros_like(stage_indices[0], dtype=torch.int64)
+    for s in stage_indices:
+        out = out * radix + s.to(torch.int64)
+    return out
+
+
+# --------------------------------------------------------------------- grad democracy
+@torch.no_grad()
+def grad_norm_spread(groups: dict[str, list[torch.nn.Parameter]]) -> dict:
+    """Gradient-democracy monitor. `groups`: name -> params of one parallel member
+    (tower/expert). Reports per-group grad norms and the orders-of-magnitude spread.
+    Reference: unequalized heterogeneous towers spread ~20 orders (fibonacci dead at
+    2.25e-21 under helix); equalized ~0.0 (geofractal gradient-democracy result)."""
+    norms = {}
+    for name, params in groups.items():
+        gs = [p.grad for p in params if p.grad is not None]
+        norms[name] = float(torch.sqrt(sum((g.float() ** 2).sum() for g in gs)).item()) \
+            if gs else 0.0
+    vals = [v for v in norms.values() if v > 0]
+    spread = (math.log10(max(vals)) - math.log10(min(vals))) if len(vals) >= 2 else 0.0
+    return {"norms": norms, "spread_orders": spread, "dead": [k for k, v in norms.items() if v == 0.0]}
+
+
+# ----------------------------------------------------------------------------- screen
+class CVScreen:
+    """CV@N early band screen (tri-band ft1): record pentachoron CV at `step_mark`
+    batches; classify <0.30 LOW / 0.35-0.50 MID / >0.80 HIGH. Turns ~2h/config
+    into ~7min. Readout only."""
+    def __init__(self, step_mark: int = 1000):
+        self.step_mark = step_mark
+        self.recorded: float | None = None
+
+    def maybe_record(self, step: int, rows: torch.Tensor) -> float | None:
+        if self.recorded is None and step >= self.step_mark:
+            self.recorded = pentachoron_cv(rows)
+        return self.recorded
+
+    @property
+    def band(self) -> str | None:
+        c = self.recorded
+        if c is None:
+            return None
+        if c < 0.30:
+            return "LOW"
+        if 0.35 <= c <= 0.50:
+            return "MID"
+        if c > 0.80:
+            return "HIGH"
+        return "BETWEEN"
+
+
+# ------------------------------------------------------------------------------ smoke
+if __name__ == "__main__":  # shapes/parse smoke ONLY — no training, ever.
+    g = torch.Generator().manual_seed(0)
+    K, D = 64, 4
+    init = torch.nn.functional.normalize(torch.randn(K, D, generator=g), dim=-1)
+    cur = torch.nn.functional.normalize(init + 0.29 * torch.randn(K, D, generator=g), dim=-1)
+    print("drift:", {k: v for k, v in anchor_drift(cur, init).items() if k != "per_row"})
+    w = torch.softmax(torch.randn(32, 2 * K, generator=g), dim=-1)
+    print("aliveness:", axis_aliveness(w))
+    print("gates:", gate_stats(torch.full((8,), 0.024)))
+    ids = compose_path_ids([torch.randint(0, 16, (4096,), generator=g) for _ in range(4)], 16)
+    print("paths:", path_diversity(ids))
+    lin = torch.nn.Linear(8, 8)
+    lin(torch.randn(4, 8)).sum().backward()
+    print("democracy:", grad_norm_spread({"a": list(lin.parameters())}))
+    print("OK — vitals smoke passed (pentachoron_cv needs geovocab2; run on GPU env)")

exp003_instruct/qwen_exp001_relay.py

@@ -0,0 +1,271 @@
+"""qwen_exp001_relay.py — geolip-aleph-qwen EXPERIMENT 1: the relay retrofit
+at 0.5B. The certified exp013 Track-C recipe (frozen trunk + near-zero-gated
+multi-slot M_hat relays after every block, vs the param-matched zero-init MLP
+adapter ablation) at its next scale rung: Qwen2.5-0.5B (24 blocks, d=896,
+~494M frozen params; relay stack ~2.8M trainable, <0.6%). First campaign of
+the generation-stake line — judged on ppl, the GATE MECHANISM (do the gates
+grow? exp013: aleph gates grew ~3x while MLP gates shrank), relay codebook
+vitals, and GENERATED SAMPLES (the deliverable is a model that generates).
+
+Arms (x2 seeds): frozen (eval-only baseline) | relay (MslRelay(896) after
+every block) | mlp (param-matched MLPAdapter(896, hidden=64), zero-init out —
+the exp013 ablation, 114,688 vs 114,944 params per adapter, 0.2%).
+Corpus: wikitext-103-raw-v1 (HF parquet), ~12M-token cache; block 512,
+batch 4, 3000 steps adapters-only, pure Adam lr 1e-3 wd 0 (the exp013
+relay-training regime).
+Riders: trunk FROZEN throughout; pure Adam wd=0; GPU-only verdicts; >=2
+seeds; drift-check before any freeze claim; Colab-safe. Paste order:
+geolip_vitals -> ar_differentiation_bed -> exp013_augmentation_bed -> this.
+"""
+from __future__ import annotations
+import json
+import math
+import os
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+if "MslRelay" not in globals():
+    try:
+        from ar_differentiation_bed import MslRelay
+        from exp013_augmentation_bed import MLPAdapter
+        from geolip_vitals import anchor_drift, axis_aliveness
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is None:
+            raise ImportError("paste geolip_vitals + ar_differentiation_bed + "
+                              "exp013_augmentation_bed first")
+        import sys, pathlib
+        sys.path.insert(0, str(pathlib.Path(_here).parent))
+        from ar_differentiation_bed import MslRelay
+        from exp013_augmentation_bed import MLPAdapter
+        from geolip_vitals import anchor_drift, axis_aliveness
+
+DATA_ROOT = os.environ.get("GEOLIP_DATA", "./data")
+EXP_DIR = os.path.join(DATA_ROOT, "qwen_exp001")
+BASE_MODEL = "Qwen/Qwen2.5-0.5B"
+BLOCK, BATCH, STEPS, LR = 512, 4, 3000, 1e-3
+MAX_TOKENS = 12_000_000
+PROMPTS = ("The history of mathematics begins",
+           "In a small village by the sea,",
+           "The most important principle of engineering is")
+
+
+class _QwenBlockWithAdapter(nn.Module):
+    """Wrap a Qwen2 decoder layer: adapter applied to the hidden-states output,
+    all other outputs and kwargs passed through untouched."""
+
+    def __init__(self, block, adapter):
+        super().__init__()
+        self.block, self.adapter = block, adapter
+
+    def forward(self, *args, **kwargs):
+        out = self.block(*args, **kwargs)
+        if isinstance(out, tuple):
+            return (self.adapter(out[0]),) + out[1:]
+        return self.adapter(out)
+
+
+def _token_cache(device="cpu"):
+    """~12M-token wikitext-103 train stream + val stream, cached to disk."""
+    os.makedirs(EXP_DIR, exist_ok=True)
+    path = os.path.join(EXP_DIR, "tok_cache.pt")
+    if os.path.exists(path):
+        blob = torch.load(path, map_location="cpu", weights_only=True)
+        return blob["train"], blob["val"]
+    from huggingface_hub import hf_hub_download
+    import pyarrow.parquet as pq
+    from transformers import AutoTokenizer
+    tok = AutoTokenizer.from_pretrained(BASE_MODEL)
+
+    def stream(fname, cap):
+        p = hf_hub_download("Salesforce/wikitext", fname, repo_type="dataset")
+        lines = [t for t in pq.read_table(p).column("text").to_pylist()
+                 if t and len(t.strip()) > 40]
+        ids = []
+        total = 0
+        chunk = []
+        csz = 0
+        for ln in lines:
+            chunk.append(ln)
+            csz += len(ln)
+            if csz > 500_000:
+                e = tok("".join(chunk), return_tensors="pt").input_ids[0]
+                ids.append(e)
+                total += e.numel()
+                chunk, csz = [], 0
+                if total >= cap:
+                    break
+        if chunk and total < cap:
+            e = tok("".join(chunk), return_tensors="pt").input_ids[0]
+            ids.append(e)
+        return torch.cat(ids)[:cap]
+
+    tr = stream("wikitext-103-raw-v1/train-00000-of-00002.parquet", MAX_TOKENS)
+    va = stream("wikitext-103-raw-v1/validation-00000-of-00001.parquet",
+                600_000)
+    torch.save({"train": tr, "val": va}, path)
+    print(f"token cache: train {tr.numel():,} val {va.numel():,}", flush=True)
+    return tr, va
+
+
+def _batch(stream, batch, block, device, g):
+    ix = torch.randint(0, stream.numel() - block - 1, (batch,), generator=g)
+    x = torch.stack([stream[i:i + block] for i in ix]).to(device)
+    y = torch.stack([stream[i + 1:i + block + 1] for i in ix]).to(device)
+    return x.long(), y.long()
+
+
+def load_qwen(arm: str, seed: int = 0, device="cuda"):
+    from transformers import AutoModelForCausalLM
+    torch.manual_seed(seed)
+    model = AutoModelForCausalLM.from_pretrained(BASE_MODEL,
+                                                 torch_dtype=torch.float32)
+    model.config.use_cache = False
+    for p in model.parameters():
+        p.requires_grad_(False)
+    adapters = None
+    if arm != "frozen":
+        d = model.config.hidden_size
+        mk = (lambda: MslRelay(d)) if arm == "relay" else (lambda: MLPAdapter(d))
+        adapters = nn.ModuleList([mk() for _ in model.model.layers])
+        model.model.layers = nn.ModuleList(
+            [_QwenBlockWithAdapter(b, a)
+             for b, a in zip(model.model.layers, adapters)])
+    return model.to(device), adapters
+
+
+@torch.no_grad()
+def eval_ppl(model, va, device="cuda", n=40, g=None):
+    g = g or torch.Generator().manual_seed(0)
+    model.eval()
+    ls = []
+    for _ in range(n):
+        x, y = _batch(va, BATCH, BLOCK, device, g)
+        logits = model(input_ids=x).logits
+        ls.append(F.cross_entropy(logits.reshape(-1, logits.shape[-1]),
+                                  y.reshape(-1)).item())
+    return math.exp(sum(ls) / len(ls))
+
+
+@torch.no_grad()
+def sample(model, device="cuda", max_new=80):
+    from transformers import AutoTokenizer
+    tok = AutoTokenizer.from_pretrained(BASE_MODEL)
+    model.eval()
+    model.config.use_cache = True
+    outs = {}
+    for p in PROMPTS:
+        ids = tok(p, return_tensors="pt").input_ids.to(device)
+        out = model.generate(ids, max_new_tokens=max_new, do_sample=False,
+                             pad_token_id=tok.eos_token_id)
+        outs[p] = tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True)
+    model.config.use_cache = False
+    return outs
+
+
+def adapter_vitals(adapters):
+    if adapters is None:
+        return None
+    gates = [round(torch.sigmoid(a.gate).item(), 4) for a in adapters
+             if hasattr(a, "gate")]
+    drifts = []
+    for a in adapters:
+        ad = getattr(a, "addr", None)
+        if ad is not None:
+            drifts.append(round(anchor_drift(ad.codebook, ad.home)["mean"], 3))
+    return {"gates": gates[:6] + ["..."] if len(gates) > 6 else gates,
+            "gate_mean": round(sum(gates) / max(len(gates), 1), 4),
+            "drift": drifts[:6] + ["..."] if len(drifts) > 6 else drifts}
+
+
+def run_arm(arm: str, seed: int = 0, steps=STEPS, device="cuda"):
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    os.makedirs(EXP_DIR, exist_ok=True)
+    tr, va = _token_cache()
+    model, adapters = load_qwen(arm, seed=seed, device=device)
+    ledger = open(os.path.join(EXP_DIR, "ledger.jsonl"), "a", encoding="utf-8")
+    if arm == "frozen":
+        ppl = eval_ppl(model, va, device=device)
+        rec = {"exp": "q1", "arm": arm, "seed": seed, "ppl": round(ppl, 3),
+               "trainable": 0, "samples": sample(model, device=device)}
+        ledger.write(json.dumps(rec) + "\n"); ledger.flush()
+        print(f"[q1 frozen s{seed}] FINAL ppl={ppl:.3f}", flush=True)
+        ledger.close()
+        return ppl
+    g = torch.Generator().manual_seed(seed)
+    params = [p for p in adapters.parameters()]
+    n_train = sum(p.numel() for p in params)
+    opt = torch.optim.Adam(params, lr=LR, weight_decay=0.0)
+    model.train()
+    for step in range(1, steps + 1):
+        x, y = _batch(tr, BATCH, BLOCK, device, g)
+        logits = model(input_ids=x).logits
+        loss = F.cross_entropy(logits.reshape(-1, logits.shape[-1]),
+                               y.reshape(-1))
+        opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+        if step % 500 == 0:
+            v = adapter_vitals(adapters)
+            print(f"[q1 {arm} s{seed} step {step}] loss={loss.item():.3f} "
+                  f"gate_mean={v['gate_mean']}", flush=True)
+    ppl = eval_ppl(model, va, device=device)
+    rec = {"exp": "q1", "arm": arm, "seed": seed, "ppl": round(ppl, 3),
+           "steps": steps, "trainable": n_train,
+           "vitals": adapter_vitals(adapters),
+           "samples": sample(model, device=device)}
+    ledger.write(json.dumps(rec) + "\n"); ledger.flush()
+    print(f"[q1 {arm} s{seed}] FINAL ppl={ppl:.3f} "
+          f"gate_mean={rec['vitals']['gate_mean']} trainable={n_train:,}",
+          flush=True)
+    torch.save({"arm": arm, "seed": seed,
+                "adapters": {k: v.cpu() for k, v in
+                             adapters.state_dict().items()}},
+               os.path.join(EXP_DIR, f"q1_{arm}_s{seed}.pt"))
+    ledger.close()
+    del model, adapters
+    torch.cuda.empty_cache()
+    return ppl
+
+
+def run_exp001(seeds=(0, 1), device="cuda"):
+    run_arm("frozen", seed=0, device=device)      # baseline once (no training)
+    for seed in seeds:
+        for arm in ("relay", "mlp"):
+            run_arm(arm, seed=seed, device=device)
+    print("=== qwen exp001 COMPLETE ===", flush=True)
+
+
+def smoke():
+    """Shapes/parse only — no substrate download, no training."""
+    r = MslRelay(896)
+    m = MLPAdapter(896)
+    x = torch.randn(2, 8, 896)
+    assert r(x).shape == x.shape and m(x).shape == x.shape
+    rp = sum(p.numel() for p in r.parameters())
+    mp = sum(p.numel() for p in m.parameters())
+    assert abs(rp - mp) / rp < 0.01, (rp, mp)      # param-matched adapters
+    blk = nn.Linear(896, 896)                       # tuple-passthrough check
+    class TupBlock(nn.Module):
+        def forward(self, h, **kw):
+            return (blk(h), "aux")
+    w = _QwenBlockWithAdapter(TupBlock(), r)
+    out = w(x, position_ids=None)
+    assert isinstance(out, tuple) and out[0].shape == x.shape and out[1] == "aux"
+    (out[0].sum()).backward()
+    assert r.addr.codebook.grad is not None
+    print(f"qwen exp001 smoke passed (relay {rp:,} ~ mlp {mp:,} params/adapter;"
+          " full run needs GPU + transformers + the 0.5B download)")
+
+
+def _in_notebook():
+    try:
+        get_ipython()  # type: ignore[name-defined]  # noqa: F821
+        return True
+    except NameError:
+        return False
+
+
+if __name__ == "__main__":
+    smoke() if not _in_notebook() else (smoke(),
+        print("Notebook: run_exp001() on GPU."))

exp003_instruct/qwen_exp002_refine.py

@@ -0,0 +1,237 @@
+"""qwen_exp002_refine.py — geolip-aleph-qwen EXPERIMENT 2: refine the relay.
+exp001 verdict shape: the relay retrofit works at 0.5B (frozen 17.80 ->
+~14.09, gates grow 0.047 -> 0.081 = trunk opt-in) but the param-matched MLP
+adapter is at parity-or-ahead on ppl — the certified GPT-2 ordering did not
+transfer as-is. exp002 races PROTOTYPICAL ENHANCEMENTS of the relay, each
+grounded in a certified law, at an equalized WIDER budget (~230K/adapter),
+against the honest wide MLP control.
+
+Arms (frozen Qwen2.5-0.5B trunk, adapters after every block unless noted):
+  relay32    — MslRelay with n_slots=32 (slot dose-response: exp012 L-AR3,
+               task monotone in slot width)
+  relay_3tau — multi-slot read at 3 temperatures (0.05/0.1/0.3), concatenated
+               (the certified rule-of-3 stroboscope, exp012 v2: -1.08 bpb)
+  relay_pw   — slots -> M_hat -> PATCHWORK consumer Linear(64,178) ->
+               SquaredReLU -> LN -> Linear(178,896) (the constellation
+               consumption spec replaces the bare linear out)
+  relay_deep — exp001's TOTAL budget concentrated on the last 12 blocks
+               (n_slots=32 there, nothing on blocks 0-11): the depth-gradient
+               law (cultivation concentrates near the prediction gradient)
+               as an architecture decision; total params == exp001 relay
+  mlp_wide   — MLPAdapter hidden=128 (the wide-budget capacity control)
+All near-zero gated (init -3.0). Training identical to exp001: block 512,
+batch 4, 3000 steps, pure Adam lr 1e-3 wd 0, adapters only. Judged: val ppl
+vs exp001's relay (14.093/14.091) and mlp (13.927/...) + gate mechanism +
+vitals + generated samples. Seed 0 sweep first; seed 1 for the top arms.
+Riders: trunk frozen; >=2 seeds before any claim; GPU-only; Colab-safe.
+Paste order: geolip_vitals -> ar_differentiation_bed ->
+exp013_augmentation_bed -> qwen_exp001_relay -> this file.
+"""
+from __future__ import annotations
+import json
+import math
+import os
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+if "run_arm" not in globals():
+    try:
+        from ar_differentiation_bed import AlephAddress, MslRelay
+        from exp013_augmentation_bed import MLPAdapter
+        from geolip_vitals import anchor_drift
+        from qwen_exp001_relay import (_QwenBlockWithAdapter, _token_cache,
+                                       _batch, eval_ppl, sample, EXP_DIR as
+                                       Q1_DIR, BASE_MODEL, BLOCK, BATCH, LR)
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is None:
+            raise ImportError("paste the qwen stack first")
+        import sys, pathlib
+        sys.path.insert(0, str(pathlib.Path(_here).parent))
+        from ar_differentiation_bed import AlephAddress, MslRelay
+        from exp013_augmentation_bed import MLPAdapter
+        from geolip_vitals import anchor_drift
+        from qwen_exp001_relay import (_QwenBlockWithAdapter, _token_cache,
+                                       _batch, eval_ppl, sample, EXP_DIR as
+                                       Q1_DIR, BASE_MODEL, BLOCK, BATCH, LR)
+
+EXP2_DIR = os.path.join(os.path.dirname(Q1_DIR), "qwen_exp002")
+STEPS = 3000
+
+
+class SquaredReLU(nn.Module):
+    def forward(self, x):
+        return F.relu(x) ** 2
+
+
+class Relay3Tau(nn.Module):
+    """Multi-slot M_hat at 3 temperatures, concatenated (rule-of-3 strobe)."""
+    TAUS = (0.05, 0.1, 0.3)
+
+    def __init__(self, d: int, n_slots: int = 16, K: int = 64):
+        super().__init__()
+        self.n_slots = n_slots
+        self.proj = nn.Linear(d, n_slots * 4, bias=False)
+        self.out = nn.Linear(n_slots * 4 * len(self.TAUS), d, bias=False)
+        nn.init.orthogonal_(self.proj.weight)
+        nn.init.orthogonal_(self.out.weight)
+        self.addr = AlephAddress(K, 4)
+        self.gate = nn.Parameter(torch.tensor(-3.0))
+
+    def forward(self, x):
+        B, n, _ = x.shape
+        slots = self.proj(x).view(B, n, self.n_slots, 4)
+        feats = []
+        for t in self.TAUS:
+            u = self.addr._u(slots) * (self.addr.tau / t)
+            m = u.abs().amax(dim=-1, keepdim=True)
+            ep, en = torch.exp(u - m), torch.exp(-u - m)
+            A = F.normalize(self.addr.codebook, dim=-1)
+            feats.append((((ep - en) @ A)
+                          / (ep + en).sum(dim=-1, keepdim=True)).reshape(B, n, -1))
+        return x + torch.sigmoid(self.gate) * self.out(torch.cat(feats, -1))
+
+
+class RelayPatchwork(nn.Module):
+    """Multi-slot M_hat -> constellation-spec consumer (SquaredReLU patchwork)."""
+
+    def __init__(self, d: int, n_slots: int = 16, K: int = 64, hidden: int = 178):
+        super().__init__()
+        self.n_slots = n_slots
+        self.proj = nn.Linear(d, n_slots * 4, bias=False)
+        nn.init.orthogonal_(self.proj.weight)
+        self.addr = AlephAddress(K, 4)
+        self.consume = nn.Sequential(
+            nn.Linear(n_slots * 4, hidden), SquaredReLU(),
+            nn.LayerNorm(hidden), nn.Linear(hidden, d))
+        nn.init.zeros_(self.consume[-1].weight)      # zero-init out (theme D)
+        self.gate = nn.Parameter(torch.tensor(-3.0))
+
+    def forward(self, x):
+        B, n, _ = x.shape
+        slots = self.proj(x).view(B, n, self.n_slots, 4)
+        feats = self.addr.m_hat(slots).reshape(B, n, -1)
+        return x + torch.sigmoid(self.gate) * self.consume(feats)
+
+
+def build_adapters(arm: str, model):
+    d = model.config.hidden_size
+    L = len(model.model.layers)
+    if arm == "relay32":
+        mk = [lambda: MslRelay(d, n_slots=32) for _ in range(L)]
+    elif arm == "relay_3tau":
+        mk = [lambda: Relay3Tau(d) for _ in range(L)]
+    elif arm == "relay_pw":
+        mk = [lambda: RelayPatchwork(d) for _ in range(L)]
+    elif arm == "relay_deep":
+        mk = [None] * (L // 2) + [lambda: MslRelay(d, n_slots=32)
+                                  for _ in range(L - L // 2)]
+    elif arm == "mlp_wide":
+        mk = [lambda: MLPAdapter(d, hidden=128) for _ in range(L)]
+    else:
+        raise ValueError(arm)
+    adapters = nn.ModuleList([m() if m else nn.Identity() for m in mk])
+    model.model.layers = nn.ModuleList(
+        [_QwenBlockWithAdapter(b, a) if not isinstance(a, nn.Identity) else b
+         for b, a in zip(model.model.layers, adapters)])
+    return adapters
+
+
+def adapter_vitals(adapters):
+    gates, drifts = [], []
+    for a in adapters:
+        if hasattr(a, "gate"):
+            gates.append(round(torch.sigmoid(a.gate).item(), 4))
+        ad = getattr(a, "addr", None)
+        if ad is not None:
+            drifts.append(round(anchor_drift(ad.codebook, ad.home)["mean"], 3))
+    return {"gate_mean": round(sum(gates) / max(len(gates), 1), 4),
+            "gates_head_tail": gates[:3] + gates[-3:],
+            "drift_head_tail": (drifts[:3] + drifts[-3:]) if drifts else None}
+
+
+def run_arm2(arm: str, seed: int = 0, steps=STEPS, device="cuda"):
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    os.makedirs(EXP2_DIR, exist_ok=True)
+    from transformers import AutoModelForCausalLM
+    tr, va = _token_cache()
+    torch.manual_seed(seed)
+    model = AutoModelForCausalLM.from_pretrained(BASE_MODEL,
+                                                 torch_dtype=torch.float32)
+    model.config.use_cache = False
+    for p in model.parameters():
+        p.requires_grad_(False)
+    adapters = build_adapters(arm, model)
+    model = model.to(device)
+    n_train = sum(p.numel() for p in adapters.parameters())
+    g = torch.Generator().manual_seed(seed)
+    opt = torch.optim.Adam(adapters.parameters(), lr=LR, weight_decay=0.0)
+    model.train()
+    for step in range(1, steps + 1):
+        x, y = _batch(tr, BATCH, BLOCK, device, g)
+        logits = model(input_ids=x).logits
+        loss = F.cross_entropy(logits.reshape(-1, logits.shape[-1]),
+                               y.reshape(-1))
+        opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+        if step % 1000 == 0:
+            print(f"[q2 {arm} s{seed} step {step}] loss={loss.item():.3f}",
+                  flush=True)
+    ppl = eval_ppl(model, va, device=device)
+    v = adapter_vitals(adapters)
+    rec = {"exp": "q2", "arm": arm, "seed": seed, "ppl": round(ppl, 3),
+           "steps": steps, "trainable": n_train, "vitals": v,
+           "samples": sample(model, device=device)}
+    ledger = open(os.path.join(EXP2_DIR, "ledger.jsonl"), "a", encoding="utf-8")
+    ledger.write(json.dumps(rec) + "\n"); ledger.close()
+    print(f"[q2 {arm} s{seed}] FINAL ppl={ppl:.3f} gate_mean={v['gate_mean']} "
+          f"trainable={n_train:,}", flush=True)
+    torch.save({"arm": arm, "seed": seed,
+                "adapters": {k: v2.cpu() for k, v2 in
+                             adapters.state_dict().items()}},
+               os.path.join(EXP2_DIR, f"q2_{arm}_s{seed}.pt"))
+    del model, adapters
+    torch.cuda.empty_cache()
+    return ppl
+
+
+ARMS = ("relay32", "relay_3tau", "relay_pw", "relay_deep", "mlp_wide")
+
+
+def run_wave_a(arms=ARMS, seed=0, device="cuda"):
+    for arm in arms:
+        run_arm2(arm, seed=seed, device=device)
+    print("=== qwen exp002 WAVE A COMPLETE ===", flush=True)
+
+
+def smoke():
+    x = torch.randn(2, 8, 896)
+    for cls, kw in ((Relay3Tau, {}), (RelayPatchwork, {}),):
+        a = cls(896, **kw)
+        y = a(x)
+        assert y.shape == x.shape
+        y.sum().backward()
+        assert a.addr.codebook.grad is not None
+        a.zero_grad()
+    p32 = sum(p.numel() for p in MslRelay(896, n_slots=32).parameters())
+    p3t = sum(p.numel() for p in Relay3Tau(896).parameters())
+    ppw = sum(p.numel() for p in RelayPatchwork(896).parameters())
+    pmw = sum(p.numel() for p in MLPAdapter(896, hidden=128).parameters())
+    lo, hi = min(p32, p3t, ppw, pmw), max(p32, p3t, ppw, pmw)
+    assert (hi - lo) / hi < 0.12, (p32, p3t, ppw, pmw)   # budget-equalized ~10%
+    print(f"qwen exp002 smoke passed (relay32 {p32:,} | 3tau {p3t:,} | "
+          f"pw {ppw:,} | mlp_wide {pmw:,} per adapter)")
+
+
+def _in_notebook():
+    try:
+        get_ipython()  # type: ignore[name-defined]  # noqa: F821
+        return True
+    except NameError:
+        return False
+
+
+if __name__ == "__main__":
+    smoke() if not _in_notebook() else (smoke(),
+        print("Notebook: run_wave_a() on GPU."))

exp003_instruct/qwen_exp003_instruct.py

@@ -0,0 +1,451 @@
+"""qwen_exp003_instruct.py — geolip-aleph-qwen EXPERIMENT 3: instruct-task
+adapters + register differentiation on the discrete surface.
+Hypothesis under test (Phil): continuation-trained adapters "double up"
+values — aligning REGISTERS to overlapping internal states; instruct-task
+training should differentiate them cleanly. The json-coco-format dataset
+(AbstractPhil/json-coco-format) is the ideal instrument: the SAME 22k COCO
+captions pass through THREE structurally distinct output registers —
+task_1 hallucination_reduction (grounded literal extraction), task_2
+useful_generalization (bracketed generics), task_3 generic_symbolism
+(positional placeholders) — in native messages+tools shape.
+
+Substrate: Qwen2.5-0.5B-INSTRUCT, frozen; adapters = the exp002-certified
+relay_pw (aleph addressing + constellation patchwork consumer) after every
+block. Arms (x2 seeds): frozen (zero-shot) | relay_pw_task1 (the
+hallucination-reduction cell — Phil's prior task-1 adapter, reproduced on the
+relay stack) | relay_pw_all (multi-task on all three) | mlp_all
+(MLPAdapter(128), budget-matched control; its 1/2-seed divergence rate at
+this width is itself a standing datum).
+
+JUDGES:
+  1. held-out per-task PPL (split BY CAPTION hash — no caption leaks across
+     tasks or splits);
+  2. TASK VALIDITY on generated outputs (validators re-implemented from the
+     dataset README: JSON parse + schema keys; task_1 per-leaf grounding
+     substring check; task_2 ^\\[[a-z_]+\\]$; task_3 typed monotonic
+     placeholders) — hallucination reduction as a measured rate;
+  3. THE REGISTER PROBE: sign-codes from the relay stack for the same held-out
+     captions under all four registers (3 task system-prompts + plain
+     continuation); separation = mean inter-register Hamming minus mean
+     intra-register Hamming, per layer — measured on BOTH the instruct-trained
+     stack and the exp002 WIKITEXT-trained stack (the overlap-vs-differentiate
+     comparison the hypothesis asks for).
+Training: block 512, batch 4, 3000 steps, adapters only, pure Adam lr 1e-3
+wd 0 (the certified regime). Full-conversation LM loss (no prompt masking) —
+consistent with the line's stream training; noted as an instrument property.
+Riders: trunk frozen; >=2 seeds; GPU-only; Colab-safe. Paste order:
+geolip_vitals -> ar_differentiation_bed -> exp013_augmentation_bed ->
+qwen_exp001_relay -> qwen_exp002_refine -> this file.
+"""
+from __future__ import annotations
+import json
+import math
+import os
+import re
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+
+if "RelayPatchwork" not in globals():
+    try:
+        from exp013_augmentation_bed import MLPAdapter
+        from qwen_exp001_relay import _QwenBlockWithAdapter, _batch
+        from qwen_exp002_refine import RelayPatchwork
+    except ImportError:
+        _here = globals().get("__file__")
+        if _here is None:
+            raise ImportError("paste the qwen stack first")
+        import sys, pathlib
+        sys.path.insert(0, str(pathlib.Path(_here).parent))
+        from exp013_augmentation_bed import MLPAdapter
+        from qwen_exp001_relay import _QwenBlockWithAdapter, _batch
+        from qwen_exp002_refine import RelayPatchwork
+
+DATA_ROOT = os.environ.get("GEOLIP_DATA", "./data")
+EXP3_DIR = os.path.join(DATA_ROOT, "qwen_exp003")
+DATASET = "AbstractPhil/json-coco-format"
+INSTRUCT_MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
+BLOCK, BATCH, STEPS, LR = 512, 4, 3000, 1e-3
+TASKS = ("task_1", "task_2", "task_3")
+HOLDOUT_MOD = 50                    # caption-hash % 50 == 0 -> held out (~2%)
+
+
+# ================================ data ============================================
+def _rows(task):
+    from huggingface_hub import hf_hub_download
+    p = hf_hub_download(DATASET, f"data/{task}.jsonl", repo_type="dataset")
+    return [json.loads(l) for l in open(p, encoding="utf-8")]
+
+
+def _render(tok, row, gen_prompt=False):
+    """Chat-template render; stringify tool arguments if the template needs it."""
+    msgs = row["messages"][:2] if gen_prompt else row["messages"]
+    try:
+        return tok.apply_chat_template(msgs, tools=row["tools"],
+                                       tokenize=False,
+                                       add_generation_prompt=gen_prompt)
+    except Exception:
+        msgs = json.loads(json.dumps(msgs))
+        for m in msgs:
+            for tc in m.get("tool_calls", []) or []:
+                a = tc["function"]["arguments"]
+                if isinstance(a, dict):
+                    tc["function"]["arguments"] = json.dumps(a)
+        return tok.apply_chat_template(msgs, tools=row["tools"],
+                                       tokenize=False,
+                                       add_generation_prompt=gen_prompt)
+
+
+def _is_holdout(prompt: str) -> bool:
+    import hashlib
+    return int(hashlib.md5(prompt.encode()).hexdigest(), 16) % HOLDOUT_MOD == 0
+
+
+def build_caches():
+    """Token streams per task split (train/holdout) + eval row lists, cached."""
+    os.makedirs(EXP3_DIR, exist_ok=True)
+    path = os.path.join(EXP3_DIR, "instruct_cache.pt")
+    if os.path.exists(path):
+        return torch.load(path, map_location="cpu", weights_only=False)
+    from transformers import AutoTokenizer
+    tok = AutoTokenizer.from_pretrained(INSTRUCT_MODEL)
+    blob = {"train": {}, "hold": {}, "eval_rows": {}}
+    for task in TASKS:
+        rows = _rows(task)
+        tr_txt, ho_txt, ev = [], [], []
+        for r in rows:
+            if _is_holdout(r["prompt"]):
+                ho_txt.append(_render(tok, r))
+                if len(ev) < 64:
+                    ev.append({"prompt": r["prompt"],
+                               "gen_prompt": _render(tok, r, gen_prompt=True)})
+            else:
+                tr_txt.append(_render(tok, r))
+        def stream(txts):
+            ids = tok("\n".join(txts), return_tensors="pt").input_ids[0]
+            return ids
+        blob["train"][task] = stream(tr_txt)
+        blob["hold"][task] = stream(ho_txt)
+        blob["eval_rows"][task] = ev
+        print(f"[cache] {task}: train {blob['train'][task].numel():,} tok, "
+              f"hold {blob['hold'][task].numel():,} tok, eval {len(ev)} rows",
+              flush=True)
+    torch.save(blob, path)
+    return blob
+
+
+# ============================ validators (per the dataset README) =================
+# core content keys; style/mood are constant-null in the data and models may
+# legitimately omit them (validators only read subjects/actions/setting)
+SCHEMA_KEYS = {"subjects", "actions", "setting"}
+
+
+def parse_tool_json(text: str):
+    # GREEDY inner match — the arguments JSON is nested; a non-greedy match
+    # truncates at the first '}' (the v1 parser bug, caught by a perfect
+    # generation scoring 0)
+    m = re.search(r"<tool_call>\s*(\{.*\})\s*</tool_call>", text, re.S)
+    raw = m.group(1) if m else None
+    if raw is None:
+        m = re.search(r"\{.*\}", text, re.S)
+        raw = m.group(0) if m else None
+    if raw is None:
+        return None
+    try:
+        obj = json.loads(raw)
+    except Exception:
+        return None
+    if isinstance(obj, dict) and "arguments" in obj:
+        obj = obj["arguments"]
+        if isinstance(obj, str):
+            try:
+                obj = json.loads(obj)
+            except Exception:
+                return None
+    return obj if isinstance(obj, dict) else None
+
+
+def _leaves(args, with_setting=True):
+    for s in args.get("subjects") or []:
+        if isinstance(s, dict):
+            if s.get("name"):
+                yield str(s["name"])
+            for a in s.get("attributes") or []:
+                yield str(a)
+    for a in args.get("actions") or []:
+        yield str(a)
+    # setting is an ENUM categorization in task_1 (not verbatim-grounded);
+    # format checks in task_2/3 do cover it
+    if with_setting and args.get("setting"):
+        yield str(args["setting"])
+
+
+def validate(task: str, args: dict, caption: str):
+    if args is None or not SCHEMA_KEYS.issubset(args.keys()):
+        return False
+    leaves = list(_leaves(args, with_setting=(task != "task_1")))
+    if not leaves:
+        return False
+    if task == "task_1":
+        cap = caption.lower()
+        return all(all(w in cap for w in lf.lower().split()) for lf in leaves)
+    if task == "task_2":
+        return all(re.fullmatch(r"\[[a-z_]+\]", lf) for lf in leaves)
+    if task == "task_3":
+        ok, counters = True, {}
+        for lf in leaves:
+            m = re.fullmatch(r"\[(ENTITY|ATTRIBUTE|ACTION|[A-Z_]+?)(?:_(\d+))?\]", lf)
+            if not m:
+                return False
+            if m.group(2):
+                k, n = m.group(1), int(m.group(2))
+                ok &= n == counters.get(k, 0) + 1 or n <= counters.get(k, 0) + 1
+                counters[k] = max(counters.get(k, 0), n)
+        return ok
+    return False
+
+
+# ================================ arms ============================================
+def load_instruct(arm: str, seed: int = 0, device="cuda"):
+    from transformers import AutoModelForCausalLM
+    torch.manual_seed(seed)
+    model = AutoModelForCausalLM.from_pretrained(INSTRUCT_MODEL,
+                                                 dtype=torch.float32)
+    model.config.use_cache = False
+    for p in model.parameters():
+        p.requires_grad_(False)
+    adapters = None
+    if arm != "frozen":
+        d = model.config.hidden_size
+        mk = (lambda: RelayPatchwork(d)) if arm.startswith("relay_pw") \
+            else (lambda: MLPAdapter(d, hidden=128))
+        adapters = nn.ModuleList([mk() for _ in model.model.layers])
+        model.model.layers = nn.ModuleList(
+            [_QwenBlockWithAdapter(b, a)
+             for b, a in zip(model.model.layers, adapters)])
+    return model.to(device), adapters
+
+
+@torch.no_grad()
+def ppl_per_task(model, blob, device="cuda", n=25):
+    out = {}
+    for task in TASKS:
+        g = torch.Generator().manual_seed(0)
+        ls = []
+        for _ in range(n):
+            x, y = _batch(blob["hold"][task], BATCH, BLOCK, device, g)
+            logits = model(input_ids=x).logits
+            ls.append(F.cross_entropy(logits.reshape(-1, logits.shape[-1]),
+                                      y.reshape(-1)).item())
+        out[task] = round(math.exp(sum(ls) / len(ls)), 3)
+    return out
+
+
+@torch.no_grad()
+def task_validity(model, blob, device="cuda", n_eval=48, max_new=200):
+    from transformers import AutoTokenizer
+    tok = AutoTokenizer.from_pretrained(INSTRUCT_MODEL)
+    model.eval()
+    model.config.use_cache = True
+    out = {}
+    for task in TASKS:
+        rows = blob["eval_rows"][task][:n_eval]
+        js = va = 0
+        for r in rows:
+            ids = tok(r["gen_prompt"], return_tensors="pt").input_ids.to(device)
+            gen = model.generate(ids, max_new_tokens=max_new, do_sample=False,
+                                 pad_token_id=tok.eos_token_id)
+            txt = tok.decode(gen[0][ids.shape[1]:], skip_special_tokens=False)
+            args = parse_tool_json(txt)
+            if args is not None and SCHEMA_KEYS.issubset(args.keys()):
+                js += 1
+                if validate(task, args, r["prompt"]):
+                    va += 1
+        out[task] = {"json_valid": round(js / len(rows), 4),
+                     "task_valid": round(va / len(rows), 4)}
+    model.config.use_cache = False
+    return out
+
+
+# ============================ the register probe ==================================
+@torch.no_grad()
+def register_probe(model, adapters, blob, device="cuda", n_caps=24):
+    """Sign-codes per relay layer for the SAME captions under 4 registers
+    (task_1/2/3 gen-prompts + plain continuation). Separation per layer =
+    mean inter-register Hamming - mean intra-register Hamming."""
+    from transformers import AutoTokenizer
+    tok = AutoTokenizer.from_pretrained(INSTRUCT_MODEL)
+    model.eval()
+    caps = [r["prompt"] for r in blob["eval_rows"]["task_1"][:n_caps]]
+    regs = {}
+    for task in TASKS:
+        by_prompt = {r["prompt"]: r["gen_prompt"]
+                     for r in blob["eval_rows"][task]}
+        regs[task] = [by_prompt.get(c, "") for c in caps]
+    regs["plain"] = [c for c in caps]
+    layer_ids = [0, 8, 16, 23]
+    # hook-based collection (one pass per register, hooks read the codes)
+    codes = {reg: {i: [] for i in layer_ids} for reg in regs}
+    for reg, prompts in regs.items():
+        hooks, store = [], {i: [] for i in layer_ids}
+
+        def mk(i, a):
+            def h(mod, inp, out):
+                x = inp[0][:, -1:]
+                slots = a.proj(x).view(*x.shape[:-1], a.n_slots, 4)
+                A = F.normalize(a.addr.codebook, dim=-1)
+                cos = F.normalize(slots, dim=-1) @ A.T
+                win = cos.abs().argmax(-1)
+                sgn = (torch.gather(cos, -1, win.unsqueeze(-1))
+                       .squeeze(-1) > 0).long()
+                store[i].append((win * 2 + sgn).reshape(-1).cpu())
+            return h
+        for i in layer_ids:
+            hooks.append(adapters[i].register_forward_hook(mk(i, adapters[i])))
+        for pr in prompts:
+            if pr:
+                ids = tok(pr, return_tensors="pt").input_ids.to(device)
+                model(input_ids=ids)
+        for h in hooks:
+            h.remove()
+        for i in layer_ids:
+            codes[reg][i] = torch.stack(store[i]) if store[i] else None
+    sep = {}
+    for i in layer_ids:
+        mats = {r: codes[r][i] for r in codes if codes[r][i] is not None}
+        names = list(mats)
+        def ham(a, b):
+            return (a.unsqueeze(1) != b.unsqueeze(0)).float().mean().item()
+        intra = sum(ham(mats[r], mats[r]) for r in names) / len(names)
+        pairs = [(a, b) for ai, a in enumerate(names) for b in names[ai + 1:]]
+        inter = sum(ham(mats[a], mats[b]) for a, b in pairs) / len(pairs)
+        sep[f"L{i}"] = {"inter": round(inter, 4), "intra": round(intra, 4),
+                        "sep": round(inter - intra, 4)}
+    return sep
+
+
+# ================================ runner ==========================================
+def train_arm3(arm: str, seed: int = 0, steps=STEPS, device="cuda"):
+    if not torch.cuda.is_available():
+        raise RuntimeError("verdict runs are GPU-only")
+    os.makedirs(EXP3_DIR, exist_ok=True)
+    blob = build_caches()
+    model, adapters = load_instruct(arm, seed=seed, device=device)
+    ledger = open(os.path.join(EXP3_DIR, "ledger.jsonl"), "a", encoding="utf-8")
+    if arm == "frozen":
+        rec = {"exp": "q3", "arm": arm, "seed": seed,
+               "ppl": ppl_per_task(model, blob, device=device),
+               "validity": task_validity(model, blob, device=device)}
+        ledger.write(json.dumps(rec) + "\n"); ledger.close()
+        print(f"[q3 frozen] FINAL {rec['ppl']} {rec['validity']}", flush=True)
+        return
+    # training stream: task_1 only, or all three interleaved
+    tasks = ("task_1",) if arm.endswith("task1") else TASKS
+    g = torch.Generator().manual_seed(seed)
+    opt = torch.optim.Adam(adapters.parameters(), lr=LR, weight_decay=0.0)
+    model.train()
+    for step in range(1, steps + 1):
+        task = tasks[int(torch.randint(len(tasks), (1,), generator=g))]
+        x, y = _batch(blob["train"][task], BATCH, BLOCK, device, g)
+        logits = model(input_ids=x).logits
+        loss = F.cross_entropy(logits.reshape(-1, logits.shape[-1]),
+                               y.reshape(-1))
+        opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
+        if step % 1000 == 0:
+            print(f"[q3 {arm} s{seed} step {step}] loss={loss.item():.3f}",
+                  flush=True)
+    rec = {"exp": "q3", "arm": arm, "seed": seed, "steps": steps,
+           "trainable": sum(p.numel() for p in adapters.parameters()),
+           "ppl": ppl_per_task(model, blob, device=device),
+           "validity": task_validity(model, blob, device=device)}
+    if arm.startswith("relay_pw"):
+        rec["register_sep"] = register_probe(model, adapters, blob,
+                                             device=device)
+    ledger.write(json.dumps(rec) + "\n"); ledger.close()
+    print(f"[q3 {arm} s{seed}] FINAL ppl={rec['ppl']} "
+          f"validity={rec['validity']} sep={rec.get('register_sep')}",
+          flush=True)
+    torch.save({"arm": arm, "seed": seed,
+                "adapters": {k: v.cpu() for k, v in
+                             adapters.state_dict().items()}},
+               os.path.join(EXP3_DIR, f"q3_{arm}_s{seed}.pt"))
+    del model, adapters
+    torch.cuda.empty_cache()
+
+
+def wikitext_stack_probe(device="cuda"):
+    """The comparison cell: the exp002 WIKITEXT-trained relay_pw stack probed
+    on the same registers (overlap prediction) — loaded onto the BASE model it
+    was trained on, prompts rendered with the instruct template regardless
+    (identical probe inputs across stacks)."""
+    from transformers import AutoModelForCausalLM
+    blob = build_caches()
+    model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-0.5B",
+                                                 dtype=torch.float32)
+    model.config.use_cache = False
+    for p in model.parameters():
+        p.requires_grad_(False)
+    d = model.config.hidden_size
+    adapters = nn.ModuleList([RelayPatchwork(d) for _ in model.model.layers])
+    ck = torch.load(os.path.join(DATA_ROOT, "qwen_exp002", "q2_relay_pw_s0.pt"),
+                    map_location="cpu", weights_only=True)
+    adapters.load_state_dict(ck["adapters"])
+    model.model.layers = nn.ModuleList(
+        [_QwenBlockWithAdapter(b, a)
+         for b, a in zip(model.model.layers, adapters)])
+    model = model.to(device).eval()
+    sep = register_probe(model, adapters, blob, device=device)
+    ledger = open(os.path.join(EXP3_DIR, "ledger.jsonl"), "a", encoding="utf-8")
+    ledger.write(json.dumps({"exp": "q3", "arm": "wikitext_stack_probe",
+                             "seed": 0, "register_sep": sep}) + "\n")
+    ledger.close()
+    print(f"[q3 wikitext_stack_probe] sep={sep}", flush=True)
+    del model, adapters
+    torch.cuda.empty_cache()
+
+
+def run_exp003(seeds=(0, 1), device="cuda"):
+    train_arm3("frozen", seed=0, device=device)
+    wikitext_stack_probe(device=device)
+    for seed in seeds:
+        for arm in ("relay_pw_task1", "relay_pw_all", "mlp_all"):
+            train_arm3(arm, seed=seed, device=device)
+    print("=== qwen exp003 COMPLETE ===", flush=True)
+
+
+def smoke():
+    ok = parse_tool_json('x <tool_call>\n{"name":"emit_caption_schema",'
+                         '"arguments":{"subjects":[{"name":"cat","attributes":'
+                         '["black"]}],"actions":["sitting"],"setting":"indoor",'
+                         '"style":null,"mood":null}}\n</tool_call>')
+    assert ok and validate("task_1", ok, "A black cat sitting indoor scene.")
+    assert not validate("task_1", ok, "A dog on grass.")       # ungrounded
+    t2 = {"subjects": [{"name": "[pet]", "attributes": ["[color]"]}],
+          "actions": ["[resting]"], "setting": "[indoor]", "style": None,
+          "mood": None}
+    assert validate("task_2", t2, "")
+    t3 = {"subjects": [{"name": "[ENTITY_1]", "attributes": ["[ATTRIBUTE_1]"]}],
+          "actions": ["[ACTION_1]"], "setting": "[INDOOR]", "style": None,
+          "mood": None}
+    assert validate("task_3", t3, "")
+    assert not validate("task_2", t3, "")                      # cross-task
+    r = RelayPatchwork(896)
+    m = MLPAdapter(896, hidden=128)
+    rp = sum(p.numel() for p in r.parameters())
+    mp = sum(p.numel() for p in m.parameters())
+    assert abs(rp - mp) / rp < 0.01, (rp, mp)                  # budget-matched
+    print(f"qwen exp003 smoke passed (validators OK; relay_pw {rp:,} ~ "
+          f"mlp128 {mp:,}; full run needs GPU + instruct download)")
+
+
+def _in_notebook():
+    try:
+        get_ipython()  # type: ignore[name-defined]  # noqa: F821
+        return True
+    except NameError:
+        return False
+
+
+if __name__ == "__main__":
+    smoke() if not _in_notebook() else (smoke(),
+        print("Notebook: run_exp003() on GPU."))

exp003_instruct/repro.py

@@ -0,0 +1,31 @@
+"""repro.py — standalone loader/runner for exp003_instruct. Code dependencies
+live in THIS folder (geolip_vitals.py, ar_differentiation_bed.py,
+exp013_augmentation_bed.py, qwen_exp001_relay.py, qwen_exp002_refine.py,
+qwen_exp003_instruct.py).
+
+  python repro.py            # CPU smoke: validators + budget match
+  python repro.py --run      # full campaign (downloads Qwen2.5-0.5B-Instruct
+                             # + the json-coco-format dataset; GPU ~5h)
+
+Data + caches land in ./data (override with GEOLIP_DATA). The
+wikitext-stack register probe expects the exp002 relay_pw checkpoint at
+./data/qwen_exp002/q2_relay_pw_s0.pt (from ../exp002_refine/adapters/).
+"""
+import os
+import sys
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+
+if __name__ == "__main__":
+    import geolip_vitals                       # noqa: F401  (paste order)
+    import ar_differentiation_bed              # noqa: F401
+    import exp013_augmentation_bed             # noqa: F401
+    import qwen_exp001_relay                   # noqa: F401
+    import qwen_exp002_refine                  # noqa: F401
+    import qwen_exp003_instruct as q3
+    if "--run" in sys.argv[1:]:
+        q3.run_exp003()
+    else:
+        q3.smoke()
+        print("repro smoke passed — run with --run for the campaign (GPU)")

exp003_instruct/results/ledger.jsonl

@@ -0,0 +1,15 @@
+{"exp": "q3", "arm": "frozen", "seed": 0, "ppl": {"task_1": 17.497, "task_2": 12.582, "task_3": 13.584}, "validity": {"task_1": {"json_valid": 0.8125, "task_valid": 0.0}, "task_2": {"json_valid": 0.0625, "task_valid": 0.0625}, "task_3": {"json_valid": 0.7708, "task_valid": 0.75}}}
+{"exp": "q3", "arm": "wikitext_stack_probe", "seed": 0, "register_sep": {"L0": {"inter": 0.5436, "intra": 0.2306, "sep": 0.313}, "L8": {"inter": 0.672, "intra": 0.4679, "sep": 0.2041}, "L16": {"inter": 0.8734, "intra": 0.7531, "sep": 0.1203}, "L23": {"inter": 0.8155, "intra": 0.6609, "sep": 0.1545}}}
+{"exp": "q3", "arm": "relay_pw_task1", "seed": 0, "steps": 3000, "trainable": 5517864, "ppl": {"task_1": 1.053, "task_2": 7.265, "task_3": 12.533}, "validity": {"task_1": {"json_valid": 0.0, "task_valid": 0.0}, "task_2": {"json_valid": 0.0, "task_valid": 0.0}, "task_3": {"json_valid": 0.0, "task_valid": 0.0}}, "register_sep": {"L0": {"inter": 0.5421, "intra": 0.2376, "sep": 0.3045}, "L8": {"inter": 0.5588, "intra": 0.2869, "sep": 0.2719}, "L16": {"inter": 0.6187, "intra": 0.3951, "sep": 0.2236}, "L23": {"inter": 0.6042, "intra": 0.3004, "sep": 0.3038}}}
+{"exp": "q3", "arm": "relay_pw_all", "seed": 0, "steps": 3000, "trainable": 5517864, "ppl": {"task_1": 1.062, "task_2": 1.062, "task_3": 1.051}, "validity": {"task_1": {"json_valid": 0.0, "task_valid": 0.0}, "task_2": {"json_valid": 0.0, "task_valid": 0.0}, "task_3": {"json_valid": 0.0, "task_valid": 0.0}}, "register_sep": {"L0": {"inter": 0.4993, "intra": 0.1288, "sep": 0.3705}, "L8": {"inter": 0.5713, "intra": 0.2876, "sep": 0.2836}, "L16": {"inter": 0.6325, "intra": 0.3367, "sep": 0.2958}, "L23": {"inter": 0.5702, "intra": 0.269, "sep": 0.3012}}}
+{"exp": "q3", "arm": "mlp_all", "seed": 0, "steps": 3000, "trainable": 5529624, "ppl": {"task_1": 1.06, "task_2": 1.059, "task_3": 1.048}, "validity": {"task_1": {"json_valid": 0.0, "task_valid": 0.0}, "task_2": {"json_valid": 0.0, "task_valid": 0.0}, "task_3": {"json_valid": 0.0, "task_valid": 0.0}}}
+{"exp": "q3", "arm": "relay_pw_task1", "seed": 1, "steps": 3000, "trainable": 5517864, "ppl": {"task_1": 1.052, "task_2": 9.383, "task_3": 20.339}, "validity": {"task_1": {"json_valid": 0.0, "task_valid": 0.0}, "task_2": {"json_valid": 0.0, "task_valid": 0.0}, "task_3": {"json_valid": 0.0, "task_valid": 0.0}}, "register_sep": {"L0": {"inter": 0.512, "intra": 0.1855, "sep": 0.3265}, "L8": {"inter": 0.5284, "intra": 0.2456, "sep": 0.2828}, "L16": {"inter": 0.5489, "intra": 0.2631, "sep": 0.2859}, "L23": {"inter": 0.6166, "intra": 0.3306, "sep": 0.286}}}
+{"exp": "q3", "arm": "relay_pw_all", "seed": 1, "steps": 3000, "trainable": 5517864, "ppl": {"task_1": 1.059, "task_2": 1.057, "task_3": 1.048}, "validity": {"task_1": {"json_valid": 0.0, "task_valid": 0.0}, "task_2": {"json_valid": 0.0, "task_valid": 0.0}, "task_3": {"json_valid": 0.0208, "task_valid": 0.0208}}, "register_sep": {"L0": {"inter": 0.5349, "intra": 0.1679, "sep": 0.367}, "L8": {"inter": 0.5619, "intra": 0.2932, "sep": 0.2688}, "L16": {"inter": 0.6324, "intra": 0.3793, "sep": 0.2532}, "L23": {"inter": 0.6435, "intra": 0.3745, "sep": 0.269}}}
+{"exp": "q3", "arm": "mlp_all", "seed": 1, "steps": 3000, "trainable": 5529624, "ppl": {"task_1": 1.059, "task_2": 1.058, "task_3": 1.048}, "validity": {"task_1": {"json_valid": 0.0, "task_valid": 0.0}, "task_2": {"json_valid": 0.0, "task_valid": 0.0}, "task_3": {"json_valid": 0.0, "task_valid": 0.0}}}
+{"exp": "q3", "arm": "frozen_validity_v2", "seed": 0, "validity": {"task_1": {"json_valid": 0.8125, "task_valid": 0.5417}, "task_2": {"json_valid": 0.0625, "task_valid": 0.0625}, "task_3": {"json_valid": 0.7708, "task_valid": 0.75}}}
+{"exp": "q3", "arm": "mlp_all_validity_v2", "seed": 0, "validity": {"task_1": {"json_valid": 1.0, "task_valid": 0.9583}, "task_2": {"json_valid": 1.0, "task_valid": 1.0}, "task_3": {"json_valid": 1.0, "task_valid": 1.0}}}
+{"exp": "q3", "arm": "mlp_all_validity_v2", "seed": 1, "validity": {"task_1": {"json_valid": 0.9792, "task_valid": 0.875}, "task_2": {"json_valid": 1.0, "task_valid": 0.9792}, "task_3": {"json_valid": 1.0, "task_valid": 1.0}}}
+{"exp": "q3", "arm": "relay_pw_all_validity_v2", "seed": 0, "validity": {"task_1": {"json_valid": 1.0, "task_valid": 0.875}, "task_2": {"json_valid": 1.0, "task_valid": 1.0}, "task_3": {"json_valid": 1.0, "task_valid": 1.0}}}
+{"exp": "q3", "arm": "relay_pw_all_validity_v2", "seed": 1, "validity": {"task_1": {"json_valid": 1.0, "task_valid": 0.9375}, "task_2": {"json_valid": 1.0, "task_valid": 1.0}, "task_3": {"json_valid": 1.0, "task_valid": 1.0}}}
+{"exp": "q3", "arm": "relay_pw_task1_validity_v2", "seed": 0, "validity": {"task_1": {"json_valid": 1.0, "task_valid": 0.9792}, "task_2": {"json_valid": 1.0, "task_valid": 0.0}, "task_3": {"json_valid": 1.0, "task_valid": 0.0}}}
+{"exp": "q3", "arm": "relay_pw_task1_validity_v2", "seed": 1, "validity": {"task_1": {"json_valid": 1.0, "task_valid": 0.9583}, "task_2": {"json_valid": 1.0, "task_valid": 0.0}, "task_3": {"json_valid": 1.0, "task_valid": 0.0}}}

exp003_instruct/results/results.json

@@ -0,0 +1,125 @@
+{
+ "validity_v2": {
+  "frozen_s0": {
+   "task_1": {
+    "json_valid": 0.8125,
+    "task_valid": 0.5417
+   },
+   "task_2": {
+    "json_valid": 0.0625,
+    "task_valid": 0.0625
+   },
+   "task_3": {
+    "json_valid": 0.7708,
+    "task_valid": 0.75
+   }
+  },
+  "relay_pw_task1_s0": {
+   "task_1": {
+    "json_valid": 1.0,
+    "task_valid": 0.9792
+   },
+   "task_2": {
+    "json_valid": 1.0,
+    "task_valid": 0.0
+   },
+   "task_3": {
+    "json_valid": 1.0,
+    "task_valid": 0.0
+   }
+  },
+  "relay_pw_task1_s1": {
+   "task_1": {
+    "json_valid": 1.0,
+    "task_valid": 0.9583
+   },
+   "task_2": {
+    "json_valid": 1.0,
+    "task_valid": 0.0
+   },
+   "task_3": {
+    "json_valid": 1.0,
+    "task_valid": 0.0
+   }
+  },
+  "relay_pw_all_s0": {
+   "task_1": {
+    "json_valid": 1.0,
+    "task_valid": 0.875
+   },
+   "task_2": {
+    "json_valid": 1.0,
+    "task_valid": 1.0
+   },
+   "task_3": {
+    "json_valid": 1.0,
+    "task_valid": 1.0
+   }
+  },
+  "relay_pw_all_s1": {
+   "task_1": {
+    "json_valid": 1.0,
+    "task_valid": 0.9375
+   },
+   "task_2": {
+    "json_valid": 1.0,
+    "task_valid": 1.0
+   },
+   "task_3": {
+    "json_valid": 1.0,
+    "task_valid": 1.0
+   }
+  },
+  "mlp_all_s0": {
+   "task_1": {
+    "json_valid": 1.0,
+    "task_valid": 0.9583
+   },
+   "task_2": {
+    "json_valid": 1.0,
+    "task_valid": 1.0
+   },
+   "task_3": {
+    "json_valid": 1.0,
+    "task_valid": 1.0
+   }
+  },
+  "mlp_all_s1": {
+   "task_1": {
+    "json_valid": 0.9792,
+    "task_valid": 0.875
+   },
+   "task_2": {
+    "json_valid": 1.0,
+    "task_valid": 0.9792
+   },
+   "task_3": {
+    "json_valid": 1.0,
+    "task_valid": 1.0
+   }
+  }
+ },
+ "register_sep_deep": {
+  "wikitext": {
+   "L16": 0.1203,
+   "L23": 0.1545
+  },
+  "relay_pw_task1_s0": {
+   "L16": 0.2236,
+   "L23": 0.3038
+  },
+  "relay_pw_task1_s1": {
+   "L16": 0.2859,
+   "L23": 0.286
+  },
+  "relay_pw_all_s0": {
+   "L16": 0.2958,
+   "L23": 0.3012
+  },
+  "relay_pw_all_s1": {
+   "L16": 0.2532,
+   "L23": 0.269
+  }
+ },
+ "n_rows": 15
+}