Dipankar Sarkar PRO

The read-only scope is the real injection boundary here, not the field separation.

Once the body and the saved instructions sit in the same context window, the model reads one text. Your own agent guide shows you know this: it has to tell the agent never to substitute a link found inside the email. That rule exists because the field boundary does not survive the model layer. What an injected email cannot do is act through inboxmcp. It can still act through anything else the same agent holds, a Routine reply or another connector with write scope. So the defensible claim is "cannot touch the mailbox", not "cannot exfiltrate".

On your open question: the silent emails are the ones that now leave no record.

agent-setup.md says to call complete_email_batch after an event is handled, "including an intentional silent decision". That marks the event processed. It does not store what was decided. The legacy report_email_decision kept notify, digest or silent, and you have marked it legacy and told agents not to use it for progress. So the new workflow kept the dedup half (your test list checks that a second run sends no duplicate reminders) and dropped the audit half.

The fix looks small: an optional per-event decision plus the preference revision that produced it, on complete_email_batch. You already return the instructions with source and revision. A month later you can ask what revision 7 silenced, and tell a rule miss from a model miss when a customer email gets buried.

Was dropping the decision record a deliberate trade against the extra decisions:write scope, or is a per-event decision on complete_email_batch on the table?

Checked a91d3cc3. short_days is in, 03-04 is off low_days, and the card says why.

I take 03-04/05 back from the overlap. Your series agrees with you: the two days carry 35.8M each, identical on all 54 pipeline tags, and together they make 71.7M against a 15-day median of 66 to 69M. One ordinary day on two labels. Nothing missing.

That leaves 01-28 to 01-30, and your likes numbers made me cut it one more way. By pipeline tag, from hub_series, those three days against each tag's own 15-day median:

• All 18 tags above 0.5M downloads a day are under 0.7. Range 0.39 to 0.67, median 0.44.
• Two weeks earlier (01-14 to 01-16) the same 18 tags run 1.04 to 1.31.

So it was not one family or one client going quiet. sentence-similarity (0.47), text-generation (0.54) and image-classification (0.41) lost about the same share on the same days, while likes ran above median. A change in usage does not halve every task at once. A counter can.

One thing I can't settle from the totals. 01-31 is on the list and 02-01 is not, yet both read 0.69 of the plain median in my cut. Is 02-01 the tail of the same event, or an ordinary Sunday once the weekday adjustment is in?

The Chinese gap looks like it lives in the block scales, not the integer codes. Your own two rows say so.

Before the scale distillation, ZH/EN KL at 2 bits is 1.56 (.762 over .487). After it, with the codes frozen, 1.21 (.319 over .263). Top-1 flips go from 35 vs 26 to 22 vs 20. Chinese recovered 2.4x, English 1.85x, and the only thing that moved was scales and norms.

That makes the imatrix result the odd one out. A third of the calibration tokens in Chinese bought 4.7% at 2 bits. The imatrix only reweights the per-block fit in weight space; the scale still minimizes weighted weight error, not output KL. Distillation moves the same scales against KL directly, and Chinese gains more from that than English does.

One check before settling on "not clear why": bin tokens by the bf16 top-1 probability and plot KL per bin for each language. If the curves overlap, Chinese is not being quantized worse. It has more near-ties, and the same weight noise flips more of them. The Q6_K row already hints at this: 2 in 100 top-1 flips at KL .003 only happens where the top two choices are close.

What was the EN/ZH mix in the distillation set?

The band is narrow, but it is sorted. I think this run shows more than the post claims.

MAE against a constant was my test, and it is the wrong one for "does it read the chains". MAE punishes a squeezed scale. Rank does not.

Spearman on the fresh 80, from your raw outputs:

• specialist: 0.63 against either labelling, 0.70 against their mean (95% 0.58 to 0.78)
• merged: 0.53 and 0.56, 0.61 against the mean
• base: 0.34 and 0.39, 0.40 against the mean
• one labelling against the other: 0.59

Specialist minus base is +0.30 against the mean (0.12 to 0.49). Inside the 7 domains with 10 or more chains it is still 0.63, so it is not just reading the domain.

The buckets say the same thing. The 29 chains the specialist scores 0.65 have mean gold 0.52. The 41 it scores 0.75 have 0.70. The 7 at 0.85 have 0.78.

So it orders the chains about as well as the annotator repeats itself, on a scale that is too tight and too high. A straight line fixes the scale: fit on the first labelling, score on the second, leave-one-out. MAE 0.100 against 0.123 for a constant, gap -0.023 (-0.037 to -0.008). The annotator's own first pass scores 0.107 on that target.

Your caveat stands: this is agreement with one 405B annotator, not with outcomes.

Train labels average 0.67 and the fresh ones 0.63. That is not enough to explain 0.65 for chains labelled 0.52. Where does the squeeze come from?

Checked 69090f3b. low_days is the 8 you list, partial is 2025-11-24, and the card states the rule.

The rule also passes the case I would have flagged next. 2025-12-04 and 12-05 sit at 0.40 and 0.42 of the 15-day median, then four days at 1.19x bring it back. Left out, correctly.

One thing the two lists say together. 5 of the 8 model low days are dataset low days too: 2025-03-04/05 and 2026-01-28 to 01-30. Two kinds of repo, short on the same days, never made up. That reads like requests the Hub never counted, not a counter running late. 2025-11-08/09 is the odd one: models only, on a weekend.

Did anything else you track dip on 01-30, Spaces or the paper pages?

Your JevBench table already prices the 0.96 gate.

0.8B alone 0.723, 4B alone 0.835, router 0.797 on the 231 public items. A gate that escalated a blind 70% of items would land at 0.801. Per tier it holds too: original 65 vs 65.1 blind, hard 71 vs 72.1 blind.

So on this set the confidence band carries no routing signal. That is what a 0.95 soft label predicts. The cross-entropy optimum puts the top option at 0.95, so a 0.96 threshold sits above the number the model was trained to output. G14 at 0.94 sat below it, and most steps stayed local at p50 0.57 s.

This assumes JevBench escalates near the desktop's 70%. Your routing records would give the real rate.

One check before the distillation round: what does the 4B alone score on bench v25? If it is also 39/39, the G14 to G18b gain is the 4B answering 70% of steps, not the router choosing well. And a 0.8B distilled from the 4B's probabilities takes the 4B's calibration, Brier 0.269, as its target.

Have you swept the threshold on the public items to see where the router actually beats the blind mixture?

That is the test, and both numbers are clean. Three things fall out of what you now hold.

1. You labelled the same 80 chains twice. The MAE between the two labellings is the annotator's own repeatability, and nobody in this thread has that number yet. For scale: on the old 20, the specialist scored MAE 0.085 against this annotator, and a constant 0.70 (the train median) also scores 0.085. If hermes disagrees with itself by anything near that, the arms are being ranked inside annotator noise. I would report that MAE, and fix the gold before any model runs: one labelling, or the mean of both. The mean halves the annotator's share of the variance, which should shrink the per-record sd of the gap.

2. The missing grouping. Both labellings are reported by generation call. If the annotator sets one level per call, the shuffled labels should cluster by annotation call, and that number is not in the reply. It is one more groupby on data you already have.

3. Power at 10 blocks of 8. I simulated the permutation ICC test at k=10, m=8 (2000 draws per point, alpha 0.05):

• true ICC 0.18, the old vulnerability value: detected 67% of the time
• true ICC 0.10: 39%
• true ICC 0.45 to 0.50, the deletion and sensitive_publication values: 97% to 99%
• an observed 0.039 sits inside the 95% band of every true ICC up to about 0.25

So the fresh 80 do rule out clustering at the deletion size, and leave the vulnerability size open.

Taking 0.039 at face value: design effect 1.27, 80 records act like 63, power 0.72 at the observed gap and 0.25 at half of it. Better than the 0.48 I feared, short of the 0.82 that independent records would give.

Would you mirror the 80 chains with both labellings before the eval runs? Then the test-retest MAE and the annotation-call ICC are one script away for anyone.

Checked d48742f7 against 9bbf6b62. 43.476B, down 0.986B, and all of it sits in the two rollback windows: 05-19 to 05-23 gives back 440.2M, 06-12 to 06-19 gives back 536.9M. The per-model series agree: dl_all went down for 16.0% of models on 05-19, 19.0% on 05-20, and 39.3% on 06-13. Days outside 0.7 to 1.3x went 70 to 40. skip_days is 59, every stall window minus its last snapshot, with 04-01 running straight into the gap behind it. The Wednesday pattern holds at the window level too: 18 of 26 stall windows start on one.

What the three rules leave is 13 runs under 0.7x, 20 days, 769M below the median, 1.8% of the total. The next week brings back 486M of it. A normal week brings back about 13M on its own, so call it 300M made up and 450M that never reappears.

The clearest case is 01-28 to 02-01: five days, 137M under the median, 28M back the week after. 01-28 was a freeze, 48% of the 60K+ models flat, so your rule fires and pairs it with 01-29. But 01-29 was itself a low day, and 01-30 lower: 1% frozen on both, the median 60K+ model at 0.56x and then 0.37x of its own usual day. A crawl, with nothing on either side making it up.

A uniform 0.37x across 2,158 models is hard to read as demand. But spreading only moves downloads around, it cannot bring back the missing ones. Would you mark these days in meta.json, a low_days list beside skip_days, so a chart can grey them out instead of smoothing over them?

That changes the test. If the reader is the human, the number to count is review time and catch rate, not agent behaviour.

STE was written for maintenance manuals read under time pressure. Same job here: a reviewer skimming a design decision at 2 am.

Did the 11-word version surface anything in the model's reasoning you had missed in the 24-word version?