LASR-Callum/qwen3.6-27b-lora-1000ex-da250-t1t3-rest750

Qwen3.6-27B — 1,000-example arm — 250 difficult-advice (t1-t3) + 750 mixed

LoRA adapter for Qwen/Qwen3.6-27B, trained for 1 epoch at lr 4e-5 with loss on assistant tokens only.

Training bundle: 2026-08-03-qwen36-27b-armB-1000ex-da250-rest750-train.

The 750 non-difficult-advice examples are 3:2 NuminaMath : (TULU3 + No Robots) = 450 / 300, with the 300 split evenly. Trait balance in the difficult-advice half: t1=84, t2=83, t3=83.

By tokens the difficult-advice share is 51.7%, not 25% — those rows average 1,690 tokens against NuminaMath's 548. Sized by example count as specified; not directly comparable to token-budgeted arms.

Mixture

Sized by example count, not by token budget.

Source	Examples	Tokens	% examples	% tokens	think block
difficult_advice (synthdoc_v2 t1/t2/t3)	250	422,588	25.0%	51.7%	real <think> trace, supervised
NuminaMath-CoT	450	246,689	45.0%	30.2%	no think block
TULU3	150	99,685	15.0%	12.2%	empty <think></think>, excluded from loss
No Robots	150	49,014	15.0%	6.0%	empty <think></think>, excluded from loss
total	1000	817,976

Constitution: t1, t2, t3 of constitutions/claude_constitution_principles.md.

Think blocks — three distinct states

Qwen3.6's chat template emits <think>\n\n</think> for any final assistant turn with no reasoning_content. Training a model to emit that marker is the documented reasoning-collapse pattern, so TULU3 and No Robots rows carry the marker as context but its tokens are excluded from the loss (mask_empty_think: true). NuminaMath rows carry no block at all; difficult-advice rows carry a real trace and are fully supervised.

Verified at token-ID level before launch: 0 rows leaked a user/system token into the loss, 0 empty-think markers carried loss, and no row with a real trace lost it.

What is supervised

Everything outside an assistant turn is -100. TRL's own assistant_only_loss cannot do this on Qwen3.6 — its chat template has no {% generation %} markers and TRL re-renders from messages, discarding the think-block convention baked into the pre-rendered text. Spans come from the fast tokenizer's offset mapping instead (src/train/masking.py).

Training

bf16 LoRA (not QLoRA — bitsandbytes does not reliably cover this model's hybrid linear-attention/SSM layers). 1xH100 80GB, 47 min.

r / alpha / dropout	32 / 64 / 0.05
target modules	regex scoped to model.language_model.*
epochs / steps	1 / 63
batch x grad-accum	1 x 16
lr / schedule	4e-5, cosine, 3% warmup
max seq len / packing	3072 / off

Final train loss 1.0030, mean token accuracy 0.7258, 817,976 tokens seen.

step	loss	token acc
5	1.5126	0.6417
10	1.0643	0.7336
15	1.0462	0.7354
20	0.9562	0.7455
25	0.9790	0.7371
30	0.9789	0.7412
35	1.0052	0.7327
40	0.9401	0.7464
45	1.0043	0.7259
50	0.8498	0.7645
55	0.7835	0.7813
60	0.9047	0.7506

Status

Not yet evaluated on ODCV-Bench or agentic-misalignment.

Sibling arm: arm A.

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16")
model = PeftModel.from_pretrained(model, "LASR-Callum/qwen3.6-27b-lora-1000ex-da250-t1t3-rest750")
model = model.merge_and_unload()

Use AutoModelForImageTextToText, not AutoModelForCausalLM — this is a vision-language checkpoint.