Gianluca Mazza PRO

Thanks, Dipankar. The overlap idea is a good one.

We reproduced it at 441207d and on current main, with identical outputs (hash embedder, seed 7, shared_history_recall_v1). Gated vs cosine on top-1 matches what you found: write-back b=1, c=0 on bakery's bk_q_inspection_not_gasket, read-only b=1, c=0 on ferry's fs_q_no_sailing. The pairs differ, but both are query 3 of 3. always_on flips the same two, but it also loses one near-neutral query in both modes (startup's so_q_outage), so it's b=1, c=1 in each mode. Cosine top-1 is the same in both modes on all 36 queries.

On ferry, the tag that moves it is the reconsolidated affect. fs_cancelled_sailing is in the top 2 at q1 and q2, and after those two write-backs its valence drifts from -0.7 to about -0.36, away from the q3 query's valence (-0.85). Restoring only that field restores the hit, and no other single field changes the outcome. Its retrieval count on its own would have helped it. Bakery is the mirror case: reconsolidation moves bk_failed_inspection's affect toward the query (-0.75 to about -0.57, against the query's -0.6), and the win needs that together with its retrieval count.

So per-mode b and c do hide it. We plan to add the column to the ADR 0002 scenario table: discordant under write-back only, read-only only, or both, with the discordant query IDs per mode. This run already has data for all three. We also plan to add carry-over as a predeclared sensitivity analysis, with every pair kept in the primary count: a write-back-only pair labelled carry-over-dependent, a read-only-only pair carry-over-suppressed, and b, c and the ICC reported with and without the carry-over-dependent pairs, as a side check. Both gated discordances here come from the write-back history, so dropping only the write-back-only pairs from the primary count would tilt the result toward read-only. That's why read-only-only pairs would get a label too.

Thanks, Dipankar. The cluster point and the ICC question are both useful.

On near-neutral queries, your first statement holds in the current code, for the in-memory store with a deterministic embedder. Gated and cosine take the same path with semantic-only weights, and write-back only changes tags (counters, reconsolidated affect, links). Embeddings, content and the set of memories stay the same, so the rankings match even after histories diverge, and the 8/8 is a wiring check. It would need re-checking if a write-back changed content, embeddings or membership, with a non-deterministic embedder, or on any other store backend, such as SQLite or an approximate index. The ADR now says this.

On dimensions: agreed, there's no 64-dim contract in the in-memory path.

The design effect and scenarios as the unit are in ADR 0002. Per stratum it's thinner than 3 per history: within each affect stratum, a scenario has 1–2 queries, and near-neutral is mostly 1.

On the ICC: yes, the plan in ADR 0002 is for the pilot to report an exploratory within-scenario ICC for discordance, with its uncertainty and the number of informative clusters, or "not estimable" with the reason. Near-neutral gated vs cosine is constant by mechanism, so that one is not estimable. With 12 histories any estimate is descriptive, so the plan is to size v2 on a declared grid of ρ values.

Thank you for taking the time to check the numbers and trace the code. Your distinction between verifying the gate and measuring a retrieval benefit is helpful, as is the suggestion to swap the embedder before sizing v2. I’ve captured both in the proposed ADR: https://github.com/gianlucamazza/emotional-memory/pull/192

I appreciate the care you’re putting into this review.

Thank you for splitting it by query valence. We reran it on the committed receipt (commit ce2e6579: https://github.com/gianlucamazza/emotional-memory/tree/e6615d9732ef46403ee6d3c0906a8d7ea246aa29/benchmarks/realistic/receipts/shared_history_recall_v1/ce2e6579, CI run: https://github.com/gianlucamazza/emotional-memory/actions/runs/40084829068) and got the same split, using the repo's gate threshold of 0.2. Top-1 / hit@k:

• Non-neutral (18 queries): cosine 10 / 11; gated and always-on both 11 / 13, with no losses.
• Near-neutral (14 queries, |valence| < 0.2): gated ranks exactly like cosine on all 14. Always-on changes 7 rankings and scores 10 / 12, with no gains.
• Boundary (4 queries, |valence| = 0.2): all three arms score 4 / 4.

So on this set always-on's gains and losses cancel out, and gated's small edge over cosine isn't significant (McNemar p = 1.0 on top-1, 0.5 on hit@k). Your McNemar arithmetic checks out too.

Two things to add. First, the sizes you suggest (about 42 and 54 on hit@k) give roughly 56% power. Reaching 80% would take about 54 near-neutral and 70 non-neutral queries, still assuming the discordant pairs all go one way. Second, we reran the split with 8 hash variants. The direction held in all 8: gated always matched cosine on near-neutral queries, gated was always net positive on non-neutral top-1 (+1 to +6), and always-on never gained on near-neutral top-1. The boundary tie didn't hold in 3 of the 8, and the size varied a lot (non-neutral hit@k from 0 to +5). Most discordant pairs come from hash collisions.

On a v2: your split shows what a properly powered version would need to measure. We haven't decided whether to build it, and running it with a semantic embedder would first need a small code change. This is still a mechanism test (hash embedder), not a semantic-quality benchmark.

Thank you for rerunning the replay. We reproduced all of it at ea890e6: in #170 nothing is encoded after a recall and the gate fires on 1 of 100 queries, so the replay mostly measures always-on against cosine, plus a small write-back cost.

Your rerun is why shared_history_recall_v1 exists. It's a new set of 36 queries where 24 encodes come after a recall and 14 queries are near-neutral (|valence| < 0.2), so the gate actually gets exercised:
https://github.com/gianlucamazza/emotional-memory/pull/180

First run at d6947ed5 (CPU, hash embedder, seed 7, 2000 bootstrap resamples), top-1 for cosine / gated / always-on:

• #170 replay: 0.33 / 0.54 / 0.54 with write-back, 0.33 / 0.56 / 0.56 read-only. Gated and always-on score above cosine on top-1, and the intervals exclude 0.
• shared_history_recall_v1: 0.69 / 0.72 / 0.69 in both modes. No arm differs significantly from cosine: every interval includes 0, p ≥ 0.25.

For gated and always-on, write-back changes the top-k on 6 of 100 replay queries and 5 of 36 on the new set; cosine is unchanged. Read-only mode changed nothing across 136 queries and three arms, and the affective state never diverged, so the whole effect sits in the memory store.

So the write-back effect is measurable but small, and the new set shows no gated advantage. The hash embedder has no semantics, so this run checks that the mechanism behaves as described and says nothing about retrieval quality.

Thanks again for taking the time.

You're right about the behaviour: Compare copies the selected mode's memory store and affective state into two scratch engines and runs both strategies there, so switching the radio changes what it compares. Nothing is written back to either live engine.

That's the intended scope: Compare answers "which arm wins on the store I'm viewing", as a point-in-time check. It's now written down in COMPARE.md:
https://github.com/gianlucamazza/emotional-memory/blob/main/demo/COMPARE.md

"Which arm wins on the same history" belongs in the benchmarks. #170 adds an opt-in shared-history replay: the same ordered events go into each arm from a common empty start, each keeps its own retrieval write-backs, and per-step state traces are exported for comparison.
https://github.com/gianlucamazza/emotional-memory/pull/170

One more thing your review surfaced: the leftover /False endpoint was still listed in the public API until today. #179 hid the internal endpoints, so the API listing now shows only /chat, /compare_both and /reset_session.

Thanks for pushing on this. It's what got the contract written down.

You're right, and thanks for running it against the wheel. After a recall, every new turn is encoded on the mood engine first and gated just imports it, so the leak always lands in gated, including when the recall ran in gated mode. Both of your mechanisms hold: the reverse links that encode() writes stay on the mood side, and the tag is built from the mood engine's state. Two more fields go the same way: momentum, and the strengths of the new memory's forward links, which are scored against the mood store. core_affect doesn't leak. Our turns were different from yours, so our values are too, but the pattern is the same: gated's new tags equal mood's, and gated's running state equals the reference.

We see two things differently. First, in our runs the snapshot doesn't outweigh the links. Repairing only the links or only the tags left a similar number of top-3 changes: in a small store the margins are tiny, so even a weak s6 can flip an order. Second, "Compare both modes" is fair between its two arms, because both copies come from one engine. But with the radio on "Query-gated", that engine is the gated one, so before the fix Compare was reading the leaky store.

On the ids: no. The import was there to keep one appraisal per turn and to stop recall write-backs from crossing over. Matching ids were a side effect, and the missing back-links and the state copied over from the mood engine were an oversight on our side. Nothing in the demo depends on matching ids.

Your swap is now live on the Space. After a recall, gated encodes each new turn itself, with the same appraisal (gated.encode(content, appraisal=memory.tag.appraisal)), and the separate observe is gone, so its state doesn't advance twice. A new test checks tags, back-links and state against a fresh engine fed the same turns, with the recall in either mode. It's a demo-only change, and the fix is yours, so thank you.

Thanks, this is a fair catch and mostly right. Two corrections: reconsolidation pulls the tag toward the running core affect, not the mood, and co-retrieval only strengthens links that already exist; it never adds new ones. Two more points. The cosine-only branch ranks independently of earlier writes, but it still writes to its own top 3. And once a memory's window is open, any recall within the next 5 minutes reconsolidates it again, whatever the prediction error, which is exactly your A-then-B case.

There's also a bigger factor you didn't mention. After each recall the demo appraises its own reply, and that can shift the affective state before the next recall. So copying the store alone wouldn't make the comparison fair; the state has to be snapshotted too.

The write-backs are intended: they're the mechanism the demo is there to show, and they're documented. What wasn't fair was reading the toggle as a comparison, because the write-backs skew it. So each mode now keeps its own memory and state: normal recall still writes, but only to the memory of the mode you used, so it no longer changes what the other mode sees. A new "Compare both modes" button runs both modes on scratch copies of the same snapshot, without changing your session. There's also a line in the demo that explains the write-backs.

I've added a toggle to the Space so you can compare the state-based and query-gated paths side by side.

Thanks, you're right on the substance.

The Space runs live LLM appraisal, so affect is inferred, not given. That's the setting where my own pre-registered results show no advantage over plain cosine, and the card says so under "Scope and limits". The Addendum P numbers you quoted are correct (0.887 vs 0.800 top-1). But P used a different model and embedder, with queries answerable from text alone, so it tells you about this kind of setting, not about the demo itself.

You're also right about recall: it calls retrieve() without query_affect, so the mood term scores against the running conversation state. That's the "current mood" behaviour the UI describes. retrieve_query_gated() ships in 0.18.1. Addendum Y's +0.076 over the always-on query-appraised arm is correct, but on ES-MemEval the gated path still doesn't beat plain cosine. The demo predates the gate and I haven't switched it since; it's there to show the mechanism, not the best retrieval setup.