Three Geometric Bands in a Sphere-Normalized Patch Autoencoder

geolip-svae finding-tier release 1 (ft1)

A quantized geometric attractor structure emerges from a single architectural knob, validated by ablation across 15 orthogonal dimensions including readout architecture substitution

Scope note. This is ft1 — the first finding-tier release from an ongoing ablation program. The 233 runs reported here map a slice of the architecture's first-order axes with a handful of cross-axis probes. A full characterization of the omega population across D, V, patch, hidden, and depth would require many more sweeps; this document is not that. What's reported is the subset that's been empirically mapped at the time of publication. ft2 will extend coverage; see Section 10 for what remains.

TL;DR: A sweep over small PatchSVAE-family configurations reveals that the final coefficient-of-variation (CV) of Cayley–Menger pentachoron volumes on the encoder's sphere-normalized latent rows quantizes into three distinct bands, indexed by the singular-value dimension D. An ablation program of 233 training runs across 15 orthogonal hyperparameter dimensions (seeds, optimizers, schedules, activations, initializations, batch sizes, capacities, normalizations, data compositions, cross-attention configurations, soft-hand variants, readout architectures, SVD precision, and batch-sharing of SVD) confirms the band structure is architectural: it is reproduced in 97.4% of runs and fails only when the row-normalization step is ablated.

• D=16 → CV ≈ 0.20 (matches uniform S¹⁵ prediction 0.199 to ±0.003 across 5 seeds)
• D=8 → CV ≈ 0.36 (matches uniform S⁷ prediction 0.357 to ±0.02 across 5 seeds)
• D=4 → CV ≈ 0.90 (matches uniform S³ prediction 0.923 to ±0.05 across 5 seeds)

Four results sharpen the framework:

1. The attractor is reached without any CV-related training signal. Pure MSE reconstruction with no soft-hand, no CV penalty, and no geometric loss term reaches the same CV value as the full soft-hand regime (0.2046 vs 0.2037 on LOW band). The architecture alone selects the attractor.

2. Sphere-norm is a selector among geometric attractors, not the creator of one. Ablating sphere-normalization does not destroy the attractor structure; it redirects the system to a different attractor (the Gaussian bulk regime). LayerNorm selects a D-dependent intermediate. Scale-only normalization is functionally identical to no normalization — the unit-norm constraint is the sole active ingredient.

3. The attractor does not require representational nonlinearity. A linear encoder (identity activation, no GELU or ReLU anywhere) reaches all three bands correctly. The attractor lives in the sphere-norm + SVD geometric pipeline, not in the MLP's representational capacity.

4. At HIGH band (D=4), a learned linear readout achieves ~4× lower reconstruction MSE than the SVD readout at identical parameter count (40,703), while landing at the same geometric attractor. A wider linear readout (57,215 parameters) pushes the improvement to ~9×. The SVD decomposition is replaceable at D=4 — and its replacement reconstructs dramatically better. At D=8 and D=16 the effect is marginal (1-2%). This is a band-specific finding: the SVD readout's orthogonality and rank-ordering constraints are reconstruction-limiting at the narrow D=4 singular-value channel, but non-limiting at wider D where the channel is not the bottleneck.

This is the first direct measurement in our battery-research lineage of a discrete geometric ladder that the architecture supports natively. We sketch a dimensional argument for why the quantization happens, give the complete matmul pipeline for one representative of each band, present the full ablation matrix that validates the claim, document the within-attractor reconstruction quality under each readout architecture, and propose a cheap online predictor: CV at 1000 training batches reliably predicts final band membership, reducing sweep turnaround from hours to minutes.

---

1. The architecture

All three bands are reached by the same base architecture (PatchSVAE-F), differing only in hyperparameters. The pipeline per patch:

x ∈ ℝ^{patch_dim}                          # flattened (3, ps, ps) tile
  │
  │ enc_in: Linear(patch_dim → hidden) → GELU
  │ enc_blocks: depth × residual MLP(hidden)
  │ enc_out: Linear(hidden → V·D)
  ▼
M ∈ ℝ^{V × D}                              # reshape
  │
  │ F.normalize(M, dim=-1)                 # sphere-norm: each row on S^{D-1}
  │
  │ G = MᵀM ∈ ℝ^{D × D}                    # Gram matrix (fp64)
  │ λ, Ṽ = eigh(G + 1e-12 I)               # fp64 eigendecomposition
  │ S = √(clamp(λ, min=1e-24))             # singular values
  │ U = M·Ṽ / clamp(S, min=1e-16)          # left singular vectors
  │ Vt = Ṽᵀ
  │
  │ S_coord = S · (1 + α · tanh(attn(S)))  # cross-attn on S, α ≤ 0.2
  │
  │ M̂ = U · diag(S_coord) · Vt            # reconstruction matrix
  ▼
  │ dec_in: Linear(V·D → hidden) → GELU
  │ dec_blocks: depth × residual MLP(hidden)
  │ dec_out: Linear(hidden → patch_dim)
  ▼
x̂ ∈ ℝ^{patch_dim}

The key operation is F.normalize(M, dim=-1), which forces every row of the V×D encoded matrix onto the unit (D−1)-sphere. The subsequent SVD is an exact arithmetic readout of the sphere-normed configuration, not a learned bottleneck. The model's job is to learn a good projection onto the manifold; the manifold itself is fixed by the architecture.

The coefficient-of-variation (CV) is measured by sampling 200 random 5-vertex subsets of the V rows and computing the Cayley–Menger 4-volume of each pentachoron:

CV = std(volumes) / (mean(volumes) + ε)

CV measures how much the pentachoron volumes vary across random 5-point subsets of the V rows. Its magnitude depends on both the uniformity of the packing and the dimension D — for a given D, a more non-uniform packing gives higher CV, but for a given uniform packing, smaller D gives higher CV because 5-point subsets span a wider volume range geometrically. The universal attractor band 0.20–0.23 corresponds to uniform packing at D=16 (S¹⁵) and has been observed across 17+ unrelated pretrained models (CLIP, T5, BERT, DINOv2, SD VAEs, etc.) whenever their representations are probed this way — it is not an artifact of any single training regime. Uniform packings at other D values produce their own characteristic CV values (≈0.36 at D=8, ≈0.92 at D=4), which Sections 2–4 confirm the architecture reaches correctly.

---

2. The three bands — exact specifications

One representative from each band, chosen for CV purity within its band:

All three use identical resolution (64×64), hidden width (64), depth (1), cross-attention layers (1), and CV-EMA soft-hand training regime. Only the three parameters (D, V, patch size) differ.

Complete matmul pipeline per band

LOW band (D=16) — the attractor

Per patch (768-dim input tile):
  768  →  64     (enc_in)           49,152 params
   64  →  64     (MLP residual)      8,192 params
   64  →  1024   (enc_out)          65,536 params
 reshape to [64, 16]                          — 64 rows on S^15
   Gram+eigh in fp64 → S ∈ ℝ^16
   cross-attn: S ← S · (1 + α·tanh(attn(S)))
   M̂ = U · diag(S) · Vᵀ
 1024 →   64     (dec_in)           65,536 params
   64 →   64     (MLP residual)      8,192 params
   64 →  768     (dec_out)          49,536 params

Total per-forward matmul FLOPs (patch): ~285K
Patches per image: 16
Per-image FLOPs: ~4.6M
CV attractor position: 0.199 (in the 0.13–0.30 universal band)
Reconstruction MSE (F-class sweep, long training): 0.842

MID band (D=8) — intermediate manifold

Per patch (768-dim input tile):
  768  →  64     (enc_in)           49,152 params
   64  →  64     (MLP residual)      8,192 params
   64  →  512    (enc_out)          32,768 params
 reshape to [64, 8]                           — 64 rows on S^7
   Gram+eigh in fp64 → S ∈ ℝ^8
   cross-attn: S ← S · (1 + α·tanh(attn(S)))
   M̂ = U · diag(S) · Vᵀ
  512  →  64     (dec_in)           32,768 params
   64  →  64     (MLP residual)      8,192 params
   64  →  768    (dec_out)          49,536 params

Per-image FLOPs: ~2.3M  (roughly half of LOW)
CV attractor position: 0.362 (own stable attractor, above universal band)
Reconstruction MSE (F-class sweep, long training): 0.843

HIGH band (D=4) — dense small-sphere attractor

Per patch (48-dim input tile):
   48  →  64     (enc_in)            3,072 params
   64  →  64     (MLP residual)      8,192 params
   64  →  128    (enc_out)           8,192 params
 reshape to [32, 4]                           — 32 rows on S^3
   Gram+eigh in fp64 → S ∈ ℝ^4
   cross-attn: S ← S · (1 + α·tanh(attn(S)))
   M̂ = U · diag(S) · Vᵀ
  128  →  64     (dec_in)            8,192 params
   64  →  64     (MLP residual)      8,192 params
   64  →  48     (dec_out)            3,120 params

Per-image FLOPs: ~12.9M (higher despite fewer params — 256 patches)
CV attractor position: 0.906 (matches uniform S³ prediction 0.923 within 2%)
Reconstruction MSE (F-class sweep, long training): 0.071  ← lowest in the F-class sweep
Reconstruction MSE (Phase 2 SVD baseline, 1 epoch): 0.420

Every other variation tested (different V, different hidden, d=2 depth, different patch size at fixed D) landed in the band determined by D. D is the band selector. Every other hyperparameter tunes performance within a band.

---

3. Why three bands — dimensional argument, confirmed by uniform-sphere measurement

The number 0.20 is not arbitrary. CV of pentachoron volumes on the unit (D−1)-sphere depends on how much room the V points have to spread, which in turn depends on the surface area of S^{D−1} relative to the number of points V.

The unit (n−1)-sphere has surface area:

A(n) = 2·πⁿᐟ² / Γ(n/2)

So for our three D values:

D=4 gives each point nearly 5× more ambient room than D=16. With that much space for only 32–64 points, each pentachoron's 5-point subset has a larger variance of possible volumes — two nearby points give a tiny 4-simplex, four well-separated points give a much larger one. So the CV statistic is inherently higher on small-D spheres not because the rows are clumped but because random 5-point subsets on a uniform small-D packing already span a wide range of volumes. Uniform S³ packing with V=32 produces CV ≈ 0.92 analytically; the empirical attractor matches this prediction (see below).

D=16 is the quiet point where the sphere has enough dimensions that random 5-point subsets sample a tight, consistent volume distribution — producing the 0.20 CV observed.

D=8 is intermediate: the sphere dimension is large enough to suppress most volume variance but not large enough to match D=16's near-perfect uniformity statistic. This gives a stable 0.36 CV that sits between the extremes.

The quantitative match: attractor CV = uniform-sphere CV

The dimensional argument gives an ordering. The stronger claim is that each band's CV value matches the uniform-sphere prediction for that D directly. We computed the uniform-sphere CV for V=64 points via a closed random-sampling procedure (no model, no data, no training — just torch.randn(V, D) followed by F.normalize(dim=-1) and the same pentachoron CV metric), using a fixed seed for reproducibility:

Trained models landed within 2% of the uniform-sphere prediction on all three bands. The attractor is not near the uniform-sphere distribution — it is the uniform-sphere distribution, selected dynamically by the combination of gradient descent and sphere-norm enforcement.

This reframes the earlier "bands are attractors" language as a specific empirical claim: under sphere-norm, the 5-point pentachoron CV of the encoder's latent rows converges to the CV of a uniform distribution of V points on S^{D−1}. Training from random initialization produces the same geometric configuration as drawing random points on the sphere, independent of all other architectural and training choices tested.

The prediction this analysis makes: D=32 sweeps should either land in the LOW band alongside D=16, or split into a new band below 0.20. If the former, D=16 is the architectural choice that makes the universal attractor accessible and higher-D just reproduces it. If the latter, there is a ladder of attractors continuing downward, and the universal 0.20 is a waypoint rather than a floor. This is a testable prediction for our next sweep.

---

4. Ablation program: the band structure is architectural

To test whether the band structure is a robust property of the architecture or an artifact of specific training choices, we ran two sequential ablation programs totaling 233 independent training runs across 15 orthogonal hyperparameter dimensions.

• Phase 1 (149 runs, 12 dimensions): band-classification sweep measuring membership at 1000 batches for LOW/MID and 100 batches for HIGH. Completed in under one hour of single-H100 wallclock on Colab.
• Phase 2 (84 runs, 3 additional dimensions): full-epoch (~3900 batches at batch size 256) runs characterizing within-attractor reconstruction under soft-hand variants, SVD-readout substitution, and LBFGS optimization. Each run is resume-capable via per-epoch checkpoints for continuation on intriguing cases.

All runs used the same three band representatives (LOW: S64-V64-D16-h64-d1-p16; MID: S64-V64-D8-h64-d1-p16; HIGH: S64-V32-D4-h64-d1-p4).

4.1 The ablation matrix

Phase 1 (band-classification, 1000/100 batch budgets)

Phase 2 (within-attractor, 1-epoch budgets)

| Combined (Phase 1 + 2) | 15 dimensions | | 233 | 97.4% |

All 6 mismatches came from Phase 1 Groups B (1 run) and G (5 runs) — the G mismatches from the normalization-ablation group, architecturally predicted. Every Phase 2 run preserved the band assignment. Every other Phase 1 dimension preserved the band in every single configuration tested.

4.2 What the non-G groups demonstrate

The attractor survives intact across:

• Seed variation: Group A — 5 seeds per band land within 0.012 of each other on LOW (CV-of-CV 0.4%), 5% on MID and HIGH. The attractor is seed-indifferent, not merely seed-robust.
• Optimizer choice: Group C — Adam (lr=1e-4), plain SGD (lr=1e-2), SGD with momentum, and AdamW all converge to the same attractor within 0.0024 of each other. The attractor is not an Adam artifact.
• Schedule choice: Group D — cosine, constant, linear decay, warm restarts, and one-cycle all preserve band membership. The schedule does not select the attractor.
• Activation function: Group F — GELU, ReLU, SiLU, Tanh, and identity all reach the correct band. The attractor does not require representational nonlinearity in the encoder. A linear encoder with sphere-norm + SVD suffices.
• Initialization: Group L — orthogonal, Kaiming-normal, Xavier- uniform, and small-normal all reach the attractor. The attractor is not init-dependent, so long as the sphere-norm step is present.
• Batch size: Group K — 32, 128, 512, and 1024 all work equally. No batch-size effect on band membership.
• Cross-attention: Group I — 0 layers, 1 layer, 2 layers, or 1 layer with unbounded α all preserve the band. The cross-attention module is not responsible for the attractor.
• Capacity within LOW: Group J — V and hidden combinations from (V=16, h=32) up to (V=128, h=128) all reach LOW band. The attractor has a remarkably wide parameter range that supports it.
• Data composition: Group B — 6 different noise-type subsets (Gaussian only, structured only, heavy-tailed only, first-half, even- indices, all 16) all preserve band membership. The only miscall (B-HIGH-B2_gaussian_only at CV_ema 0.7533) had observed sphere-CV of 0.9504, confirming the model reached the HIGH attractor but produced a CV-EMA below the 0.80 classification threshold due to limited-batch measurement noise. Not a real anomaly.
• Soft-hand regime: Group E_preview — full soft-hand (E1), pure MSE with no CV involvement (E2), CV-EMA tracked but not used (E3), and hard CV-target penalty (E4) all reach the same CV to within 0.0014 on every band. The attractor is reached even when the loss function contains no CV-related signal at all.

The E_preview result deserves particular emphasis. Pure MSE reconstruction reaches CV = 0.2046 on LOW band vs 0.2037 with full soft-hand — a difference below the measurement noise floor. The architecture alone, through the sphere-norm + SVD geometric pipeline, selects the uniform-sphere attractor. Training signal does not need to encode any preference for this outcome.

4.3 Group G: sphere-norm as attractor selector

The only ablation that perturbs the band assignment is normalization removal or replacement. Across four normalization modes:

Numbers in parentheses are deviations from uniform S^(D-1) prediction.

Three patterns emerge:

G1 sphere-norm reaches uniform S^(D-1) within 3% across every band. This is the framework's core prediction.

G2 (no normalization) and G4 (scale-only) produce nearly identical geometric outcomes, differing by less than 0.03 on every band. The scale-only variant divides each row by the batch's mean row norm but does not enforce unit length; the no-norm variant does nothing at all. That these produce functionally identical geometry demonstrates that the unit-norm constraint is the entire active ingredient of sphere- normalization. Scale magnitude without unit enforcement does no geometric work.

When normalization is absent or scale-only, the system converges to a different attractor — approximately the Gaussian bulk configuration (the CV of V points drawn i.i.d. from N(0, I_D) without sphere projection). At D=16, the bulk CV prediction is 0.358; our observed is 0.374, within 5%. At D=8: predicted 0.588, observed 0.555. The architecture still produces a reproducible attractor; it is simply a different attractor in the geometric family, not the uniform-sphere one.

G3 LayerNorm acts as a D-dependent partial selector. At LOW (D=16) LayerNorm reaches the uniform attractor cleanly (observed 0.200 vs predicted 0.199). At MID (D=8) it mildly elevates the CV to 0.421 — between the uniform and bulk predictions. At HIGH (D=4) it underselects, pulling the CV below the uniform target to 0.706. The mechanism: LayerNorm centers and variance-normalizes across D elements, producing a configuration on a hyperplane rather than a sphere. At higher D the hyperplane approximates S^(D−1) better; at D=4 the geometric distortion becomes visible as CV depression.

4.4 Implications

1. The three-band structure is robust. Across 233 training runs with variations in 15 orthogonal dimensions, 97.4% preserve the predicted band assignment. Non-ablation groups show 100% preservation; the six mismatches all come from Group G (normalization ablation) where the framework predicts a shift to a different attractor.

2. The attractor is architectural. It is reached by pure MSE training, by linear encoders, by any first-order optimizer, under any batch size, with any initialization, and — as the Phase 2 results below show — with the SVD readout replaced by a learned linear layer or removed entirely. What it requires is the sphere-norm constraint on latent rows.

3. Sphere-norm is a selector, not a creator. The architecture supports a family of geometric attractors per D; sphere-norm selects the uniform one. Other normalization modes select other members of the family (Gaussian bulk, LayerNorm hyperplane) — each reproducibly.

4. The unit-norm constraint is the load-bearing element. The specific mechanism of normalization (centering, variance scaling, or division by a norm) is less important than whether a unit-length constraint is actively imposed. Division by mean-row-norm does not impose the constraint and does not change the attractor.

4.5 Readout architecture: a band-specific reconstruction story

Phase 2 extended the ablation program to the readout stage — the component that consumes the sphere-normalized matrix M and produces the features the decoder reconstructs from. Six readout variants were tested at 1 epoch (≈3900 batches at batch size 256) across all three bands:

• H1_svd_fp64 — full fp64 SVD (baseline); 40,703 params at HIGH, 183,339 at MID, 250,211 at LOW
• H2_linear_matched — learned linear layer with output dimensionality matched to SVD's (V×D); param count is not identical to H1
• H3_linear_unmatched — learned linear readout without output-dim matching
• H4_svd_fp32 — SVD computed in fp32 precision
• H5_batch_shared_svd — single SVD reused across the batch
• H6_no_svd_direct — no readout at all (encoder output → decoder directly)

A clarification on H2's parameter cost, since it's easy to misread as "the same model with a different readout." At HIGH (V=32, D=4), H2's readout is 1.4× larger than H1's in total parameter count (57,215 vs 40,703). At MID (V=64, D=8) H2 has 445,995 parameters against H1's 183,339 — 2.4× larger. At LOW (V=64, D=16) H2 has 1,299,811 parameters against H1's 250,211 — 5.2× larger. The H2 variant grows rapidly with V×D because replacing the SVD decomposition with a learned linear projection of the same output size requires a dense hidden×(V·D) matrix and its inverse path. So H2 vs H1 comparisons are not parameter-matched except approximately at HIGH band; at MID and LOW the MSE differences must be read against the parameter ratio.

Every one of these variants preserves the geometric attractor — band-assignment match rate 100% across the H group. The sphere-norm constraint imposed upstream determines the CV; what follows is architecturally free for attractor-selection purposes.

But reconstruction quality within each band depends sharply on the readout choice, and the dependence is itself band-specific:

HIGH band (D=4) — dramatic readout-architecture effect

At D=4 with 1.4× more parameters than the SVD baseline, H2 achieves ~9× lower reconstruction MSE while landing at the same CV attractor. Three seeds, tight standard deviation. H3_linear_unmatched — 4× better than H1 on MSE at identical parameter count to H1 (40,703) — is the cleaner parameter-matched comparison, and it still cuts reconstruction error by 75%. So part of the effect is "linear readout beats SVD readout at D=4 even at matched parameters" (the H1-vs-H3 comparison) and part is "a wider readout helps further" (the H3-vs-H2 comparison).

MID band (D=8) — marginal, but H2 pays for its advantage

H2 still leads but by only 2.5%, and does so with 2.4× more parameters than the baseline. H3 — same parameter count as H1 — leads H1 by 0.9%. The readout architecture barely differentiates at this budget.

LOW band (D=16) — marginal, and H2's cost is substantial

H2 leads by 2.4% at 5.2× the parameter cost. H3 at matched parameters leads H1 by 1.7%. SVD is not the reconstruction bottleneck at D=16.

What this says about the SVD readout

At D=4 the SVD readout produces only 4 singular values — a narrow information channel. The cleanest comparison is H3 (learned linear readout at identical parameter count to H1): H3 at 40,703 parameters achieves MSE 0.095 vs H1's 0.420 at the same parameter count, a ~4× reduction. This isolates the readout mechanism from capacity. Adding parameters on top (H2 at 57,215) pushes MSE further down to 0.046, another ~2× gain.

At D=16 the same matched comparison (H1 vs H3) gives MSE 0.935 vs 0.919 — a 1.7% difference. The SVD's orthogonality and rank-ordering constraints are not reconstruction-limiting when the channel is wide. At D=8 the matched gap is 0.9%. At D=4 it is ~4×.

This refines the Omega framework's claim about SVD necessity:

• The SVD decomposition is necessary for producing omega tokens (the S vectors that function as transferable geometric currency across models).
• It is not necessary for reaching the geometric attractor — every H-variant including H6 (no readout at all) preserves the attractor.
• At D=4 specifically, the SVD's constrained readout is reconstruction-limiting — learned linear at matched parameters recovers ~75% of the reconstruction error.

A predictive scaling claim

If the mechanism — orthogonality-constrained readout is reconstruction-limiting when D is narrow — is right, then:

• D=32 should show a smaller H1-vs-H3 gap than D=16 (wider channel, less matter)
• D=2 should show a larger gap than D=4 (narrower channel, more matter)

This is a testable prediction; the small D-extension sweep that follows this paper will confirm or falsify it.

---

5. CV at 1000 batches predicts final band membership

The row_cv trajectory plot shows every run finding its band within the first ~1000 steps and holding for the remaining 299,000.

This gives us a minutes-scale triage:

• Measure CV-EMA at batch 1000 (~4–7 minutes on Colab single-GPU)
• CV < 0.30 → will converge to LOW band
• CV 0.35–0.50 → will converge to MID band
• CV > 0.80 → will converge to HIGH band

No need to train to convergence to determine band membership. Existing sweep infrastructure can be modified to early-stop at 1000 batches for screening, only continuing runs whose band assignment matches the research target.

For cell-candidate hunting specifically (want LOW band), this collapses the turnaround from ~2 hours per config to ~7 minutes, with the same confidence in final geometric classification.

---

6. What each band is good for

The conventional read of "best MSE" ranks the three bands exactly backwards relative to the universal-manifold thesis. A separate reading by band character:

LOW band — universal generalist

The D=16 attractor has been observed across 17+ unrelated pretrained models when probed. A sphere-normed model that lands here is on the same geometric manifold those models land on. Its omega tokens (S vectors) are in principle translatable to/from tokens produced by any other attractor- aligned model via Procrustes alignment.

On pure noise this band reconstructs worse than the HIGH band at the same parameter budget. On transfer to other distributions (images, text as tensors, unseen noise types) it reconstructs far better — Fresnel-base 256 from this band achieves MSE 3.8×10⁻⁵ on ImageNet without seeing it during training.

Use: battery / cell / relay in multi-model collective architectures.

MID band — intermediate attractor

Four D=8 configs with varying hidden widths and depths all converge to CV 0.38–0.40, tighter than the HIGH band's spread. This is a real attractor of its own, not a transition state. We do not yet know what it represents; characterization is an open research direction.

Testable: does the MID attractor transfer across distributions like the LOW one? Does distillation from a LOW model speed convergence for a MID configuration, or vice versa?

Use: undetermined pending characterization. Possibly a secondary geometric substrate for specific domains.

HIGH band — dense attractor with readout-architecture sensitivity

The D=4 band converges to CV ≈ 0.90, matching the uniform-sphere prediction for V=32 points on S³ (0.923) to within 2%. Despite the seemingly-high CV number, the rows are not clumped — they are uniformly distributed on a 3-sphere whose small dimension causes larger pentachoron volume variance than the larger spheres of MID/LOW bands. The CV measures how much room 5-point subsets have on the sphere, and D=4's S³ gives each point much more ambient space (see Section 3's surface-area table) which is reflected in the higher but geometrically-uniform CV.

Phase 2's H-group ablation revealed something the original sweep missed: within this attractor, the choice of readout architecture dramatically affects reconstruction quality at D=4 but not at D=8 or D=16. Three distinct battery formats are viable at HIGH band, each preserving the same geometric attractor:

• SVD-readout batteries (H1-style): the baseline. 40,703 params at HIGH. Produces omega tokens (S vectors) as transferable geometric currency. Reconstruction MSE 0.420 at 1 epoch / 3900 batches / batch size 256.

• Linear structural (H3-style, parameter-matched to H1): learned linear readout, no widening — 40,703 params. Preserves the geometric attractor (CV 0.911, same attractor as H1's 0.906) and achieves MSE 0.095 — ~4× better than SVD at matched parameter cost. Does NOT produce omega tokens in the standard form.

• A-Class linear structural (H2-style, widened readout): learned linear readout at 57,215 params — 1.4× H1's parameter budget. Reconstruction MSE 0.046 — ~9× better than H1. The "A-Class" designation marks this as a distinct battery format that trades omega-token production and a modest parameter budget increase for an order-of-magnitude reconstruction improvement at D=4.

The three formats occupy the same geometric attractor but differ on two axes: whether the readout produces S-vector omega tokens (H1 only), and how much capacity the readout is given. Which to use depends on downstream needs: cross-model alignment via omega tokens favors H1; single-model reconstruction quality at D=4 favors H3 or H2.

Use: multiple viable uses, selected by downstream requirement. Domain-specific specialist batteries still work for HIGH as originally described. A-Class linear structural adds a high-reconstruction-quality format suitable for applications where single-model performance dominates over cross-model transfer.

---

7. The methodological correction

Our prior F-class sweep methodology used MSE as the primary filter with a 1-epoch "keep-or-kill" curve-delta verdict. This sweep's data shows that methodology is insufficient:

• The lowest-MSE configuration in the sweep (CV 0.93, HIGH band) was flagged as a "strong cell candidate" until geometry was checked.
• The true on-attractor configurations had higher MSE than several off-attractor configurations.
• MSE alone cannot distinguish specialist on-distribution solutions from generalist on-attractor solutions. Phase 2's readout ablation (Section 4.5) showed this even more sharply: at HIGH band, a learned linear readout achieves MSE 0.046 at parameter cost 1.4× the SVD baseline's — an order of magnitude improvement on in-distribution reconstruction — while the SVD variant at higher MSE is the one that actually produces transferable omega tokens. Reconstruction MSE and cross-model transfer capability point in different directions; the HIGH band's low MSE does not translate to LOW-band-style universality.

Going forward, the cell-candidate filter is three-tier:

1. CV in 0.13–0.30 band at step 1000 → attractor candidate
2. CV stays in band through training → stable attractor
3. Attractor holds under freezing and host gradient → viable cell

The 1000-batch CV measurement is fast and eliminates the false positives that MSE-only triage was generating.

---

8. Open questions this sweep raises

Questions the Phase 1+2 ablations resolved:

• ✓ Is the three-band structure reproducible? Yes, 97.4% match rate across 233 independent runs in 15 orthogonal ablation dimensions.
• ✓ Is the attractor Adam-specific? No. Adam, SGD, SGD+momentum, and AdamW all reach it within 0.0024 of each other.
• ✓ Does the attractor require nonlinear activation? No. Identity activation (purely linear encoder) reaches all three bands correctly.
• ✓ Minimum LOW-band parameter count. Tested from (V=16, h=32) to (V=128, h=128); all reach LOW band. Attractor admits wide capacity range.
• ✓ Is sphere-norm load-bearing? Yes, but as an attractor selector, not an attractor creator. Ablating it redirects the system to a different reproducible attractor (Gaussian bulk), rather than destroying attractor structure.
• ✓ Is the SVD readout necessary for reaching the attractor? No. Phase 2 Group H shows that learned linear readout, even no explicit readout at all (raw encoder output → decoder), all preserve the geometric attractor at all three bands. The SVD is not an attractor-selection mechanism.
• ✓ Is within-attractor reconstruction sensitive to readout architecture? Yes, dramatically at D=4 (H2_linear_matched: 9× better MSE than H1_svd_fp64) and marginally at D=8 and D=16 (2-3% differences). The HIGH band's narrow D=4 singular-value channel is the mechanism.

Questions that remain open:

• Does the A-Class linear structural advantage scale as predicted? The mechanistic reading of the HIGH-band readout finding predicts the H2-vs-H1 gap should shrink as D grows (wider channel, less benefit from reorganization) and grow as D shrinks. Test: small add-on sweep at D=2 and D=32 with H1, H2, H3 variants. Confirmed 9× at D=4, 2-3% at D=8/16 — D=2 should exceed 9×, D=32 should be well under 2%.

• Does D=32 produce a new band below 0.20, or reproduce LOW? Tests whether the attractor ladder continues or terminates at D=16. The dimensional argument predicts a narrow band near CV(S³¹) ≈ 0.13. Can be combined with the A-Class scaling experiment above.

• Is the MID band useful in its own right? No systematic probe of D=8 transfer behavior yet. The ablation confirms it's a real attractor, not an artifact, but whether D=8 omega tokens are usefully translatable across domains is untested.

• What are the HIGH-band specialists actually encoding? Per-singular- vector analysis of a trained D=4 config should reveal which directions carry the shortcut information. The A-Class finding suggests the SVD's rank-ordering is discarding information that a learned linear readout recovers — characterizing what — would clarify this.

• Do HIGH-band batteries (either SVD or A-Class linear structural) fail out-of-distribution as expected? Run the universal diagnostic (16 noise types, text, images) on a HIGH-band champion of each format. Confirms or falsifies the specialist-vs-generalist reading. Particularly interesting: does A-Class linear structural transfer WORSE than SVD because it optimized harder for in-distribution MSE?

• What is the LayerNorm-at-D=4 undershoot measuring? The G3 variant at HIGH band reached CV 0.706, significantly below uniform S³ prediction 0.923. This is a distortion specific to the centering+variance-norm combination at low D. Characterization would clarify exactly which geometric property of sphere-norm is irreplaceable by the standard normalization layers.

• Does LBFGS within-attractor MSE advantage hold at epoch scale? Phase 2 Group L2 showed that LBFGS + sphere_norm is unstable at epoch-length budgets (4 of 6 runs ended with non-finite parameters, consistent with the known LBFGS-sphere_norm-line-search incompatibility). A Riemannian LBFGS with constraint-aware line search is the proper fix — deferred to a separate engineering pass.

---

9. Artifacts

All 233 Phase 1+2 ablation runs are preserved on HuggingFace under AbstractPhil/geolip-svae-ablations with per-run final_report.json files, per-epoch checkpoints (Phase 2 runs are resume-capable), and TensorBoard event files. The aggregated analyses (band_matrix.csv, anomalies.csv, group_summaries.csv, uniformity_diagnostic.csv, h_group_comparison.csv, snapshot_meta.json) are under _analysis/{timestamp}/ within the same repo. The canonical snapshot for this paper is 2026-04-21T23-22-10Z.

The 19 configurations from the original F-class sweep are preserved under AbstractPhil/geolip-svae-batteries with full TensorBoard logs, checkpoints every 5 epochs, and final reports.

Research notebook: the trainer, orchestrator, aggregator, and all sweep configurations used to produce this paper are collected in a single notebook published alongside the ablation artifacts:

AbstractPhil/geolip-svae-ablations/ft1_notebook_sweeper.ipynb

Clone the repo and open the notebook to see the exact code and sweep definitions that produced the 233 runs reported here. The notebook is self-contained — running its cells top-to-bottom in a Colab session with HF_TOKEN set reproduces the Phase 1 + Phase 2 matrices.

---

10. ft1 coverage: what was mapped, what remains

This paper is ft1 — the first finding-tier release. The 233 runs map a slice of the architecture's first-order axes. They are a handful of possibilities, not the full space.

What ft1 mapped (with seed replication where n > 1):

• Canonical band representatives (Group A): one config per band, 5 seeds each → 15 runs at 250K/183K/41K parameter points
• Capacity at LOW band (Group J): five capacity variants from 73K to 793K parameters, 1 seed each
• Optimizer axis (Group C): Adam, SGD+momentum, AdamW, 1 seed each
• Normalization axis (Group G): no-norm, scale-only, LayerNorm, 1 seed each × 3 bands
• Activation axis (Group F): identity, GELU, ReLU, 1 seed each
• Batch size, init, cross-attention, soft-hand (Groups B, D, I, K, L, E): each with 1-3 seeds × 3 bands
• Readout architecture (Group H): six variants × 3 bands × 1-3 seeds
• LBFGS (Group L2): MID + HIGH × 3 seeds (LOW omitted pending Riemannian-line-search engineering)

What ft1 did NOT map (not an exhaustive list — these are first-order gaps):

• D axis beyond {4, 8, 16}: D=2, D=3, D=6, D=12, D=24, D=32, D=64 — unmapped. The ladder's endpoints and intermediate bands are unknown.
• Capacity variants at MID and HIGH: Group J tested capacity only at LOW. The same sweep at other bands doesn't exist.
• Patch size axis: only {4, 16} used. patch_size ∈ {2, 8, 32, 64} unmapped.
• V axis at fixed D: J varied V at LOW only. MID and HIGH capacity behavior uncharacterized.
• Depth axis: every run uses depth=1. depth=2, 3, 4 unexplored.
• Image size axis: every run uses img_size=64. 32, 128, 256 unexplored.
• Cross-axis interactions: V×D, patch×D, hidden×depth, depth×D — combinatorially mostly unsampled.
• Transfer / out-of-distribution behavior: universal-diagnostic post-test on Phase 1 survivors not yet run.
• Long-horizon training: Phase 2 used 1 epoch (~3900 batches). Whether results change at 5-30 epochs is untested except for continue_training() cases not run yet.

A rough estimate: ft1 sampled 15-20% of the first-order axes and a small fraction of the cross-axis space. The three-band attractor has held up robustly across everything tested, but "the omega inventory" is not complete and will not be until ft2 (and likely later tiers) extend the coverage.

The omega inventory CSV accompanying this paper (AbstractPhil/geolip-svae-ablations/_analysis/2026-04-21T23-22-10Z/omega_inventory.csv) enumerates every config with its exact params_count, CV, MSE, and band-preservation flag. It is the authoritative reference for config sizes cited in this paper — if anything in the prose disagrees with the CSV, the CSV is correct.

---

11. Code manifest

Every script referenced in this paper, alphabetized. Paths are the canonical locations at publication time; some files live in the notebook (ft1_notebook_sweeper.ipynb) rather than as standalone files in a repo.

Links or path corrections can be applied directly to this manifest as needed — nothing in this paper's conclusions depends on a specific path being reachable, only on the artifacts existing.

---

This finding emerged from a multi-stage sweep program: an original F-class (miniature battery) exploration over approximately 40 hours of A100 time that produced the three-band hypothesis, followed by a 149-run ablation program (Phase 1) completed in under one hour on a single H100 that validated the hypothesis across 12 orthogonal dimensions of variation, followed by an 84-run readout-architecture and optimizer-characterization sweep (Phase 2) completed at similar wallclock that characterized within-attractor reconstruction under SVD substitution and removal. The sphere-norm-as-selector finding and the linear-encoder result emerged from Phase 1; the A-Class linear structural finding — learned linear readout at D=4 achieving ~4× to 9× better reconstruction MSE than SVD — emerged from Phase 2. Each sweep sharpened rather than displaced the previous one: the three-band attractor structure has proven remarkably robust, with its specifics refining and its mechanism narrowing under scrutiny. This is ft1; ft2 will extend coverage along the axes Section 10 enumerates.