Avifenesh/episodic-ingestion-modernbert-field-event-ranker-mixed-v2-1-h4-320

ModernBERT field-event ranker (mixed-mode v2.1, H4-320)

Fine-tune of answerdotai/ModernBERT-base with v2.1 labels — same as v2 but with a sharpened action_causality heuristic:

• Old: tool_result is grounded if ≥2 content tokens appear in ANY later assistant turn. Noise-dominated (file, path, error, exitcode match trivially).
• New: require ≥2 distinctive tokens (filter out 50+ corpus-frequent noise tokens) and the assistant turn must be the NEXT one, within 2 events. Selects a proper subset of tool_results based on content grounding.

Key result: per-field improvements on sharpened fields

Field	v2-320	v2.1-320	Δ
action_causality	0.561	0.698	+0.137
failed_attempts	0.705	0.845	+0.140
recent_error	0.826	0.905	+0.079
customer_identity	0.811	0.864	+0.053
next_actions	0.755	0.786	+0.031
initiating_command	0.966	0.968	+0.002
outcome_of_latest_attempt	0.966	0.938	-0.028
attempted_actions	0.480	0.473	-0.007
observed_outcomes	0.491	0.486	-0.005

Overall MRR stayed flat at 0.583 (aggregate), but top-1 moved from 0.375 → 0.381. action_causality's eval mass shrank (141 → 50 pairs) because the sharper heuristic only emits the field when the signal is real, which is why the big per-field MRR gain doesn't translate 1:1 into aggregate gain.

Training details

• Train rows: 1217
• Eval rows: 220
• Steps: 320 × accum=8 = 2560 forwards
• Last loss: 0.0007
• Peak VRAM: 4.08 GiB

Eval metrics

metric	value
overall MRR	0.583
top-1	0.381
top-2	0.576
top-3	0.722
top-5	0.904
mean expected rank	2.69

Per-field MRR (all fields):

field	n	MRR	top-1
attempted_actions	846	0.473	0.24
observed_outcomes	845	0.486	0.24
initiating_command	199	0.968	0.94
outcome_of_latest_attempt	181	0.938	0.88
attempt_outcome_pairs	53	0.608	0.34
action_causality	50	0.698	0.48
failed_attempts	36	0.845	0.72
next_actions	32	0.786	0.59
recent_error	29	0.905	0.83
customer_identity	11	0.864	0.73
transaction_reference	5	0.583	0.40
product_name	4	1.000	1.00
discarded_options	2	0.292	0.00
invalidation_hints	2	0.625	0.50
non_promotable_context	2	0.375	0.00
payment_or_warranty_detail	2	0.170	0.00
assistant_claims_to_verify	1	1.000	1.00
explicit_decisions	1	1.000	1.00
resolved_context	1	0.333	0.00
touched_files	1	0.333	0.00
unsupported_hypotheses	1	0.250	0.00

Lineage

checkpoint	MRR	top-1	note
mixed-v1 perf-H4	0.506	0.260	role-tautological labels
mixed-v2 perf-H4 (160)	0.526	0.296	semantic labels, non-converged
mixed-v2 H4-320	0.583	0.375	converged
mixed-v2.1 H4-320 (this)	0.583	0.381	sharpened causality