Instructions to use loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42")
model = AutoModelForCausalLM.from_pretrained("loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42

SGLang

How to use loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42 with Docker Model Runner:
```
docker model run hf.co/loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42
```

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

R1-Qwen3-8B-BrokenArXiv-GRPO-s42

🛑 Not recommended for use

This checkpoint is published as a negative control / research artifact, not as a usable model. It suffered a large, statistically significant loss of general mathematical ability during training:

AIME 2026: 69.17% → 40.00%, Δ = −29.17pp (95% CI [−40.83, −18.33]).

The degradation is robust to how missing data is handled — this model's optimistic upper bound (56.55%) still falls below the base model's conservative lower bound (69.17%); the two intervals do not overlap.

If you want the working version of this experiment, use the sibling checkpoint R1-Qwen3-8B-BrokenArXiv-GRPO-s43, which was trained with identical data and hyperparameters, differs only in random seed, discriminates better, and left AIME essentially intact.

Why this checkpoint is published anyway

Because the seed-42 / seed-43 contrast is the most useful result of this work, and it requires both halves to be inspectable.

	base	seed 42 (this model)	seed 43
AIME 2026, official semantics	69.17%	40.00% (−29.17pp, sig.)	65.00% (−4.17pp, n.s.)
Median output length (tokens)	19217	24556	18296
Runs truncated at the 32k cap	25.8%	30.8%	8.3%
Balanced true/false accuracy (chance = 50%)	29.50%	56.75%	63.75%
Discrimination gap	−0.5pp (z=−0.1)	+19.0pp (z=+3.9)	+27.5pp (z=+6.0)
BrokenArXiv Overall `sycophancy`	13.18%	19.71%	pending

Same data, same recipe, same hyperparameters — only the random seed differs. Seed 42 inflated its output length by 28%, hit the 32k token cap on nearly a third of samples, and collapsed on AIME. Seed 43 became shorter than the base, truncated less, kept AIME intact, and discriminated better.

The implication matters more than either model: the large general-capability loss seen here is mostly not the necessary price of the new capability. It is a training instability. A sibling run paid ~4pp instead of ~29pp and came out stronger. Reports that quantify an "alignment tax" from a single run may be measuring seed variance.

The degradation mechanism was identified and is specific: output-length inflation running into the max_tokens ceiling, not behavioural contamination. Among landed runs, 100% still produced a \boxed{} answer and BrokenArXiv-style refutation language appeared in only 6.0% of AIME responses. The model did not learn to "deny everything"; it learned to write too much.

Two repair attempts, both documented failures

Stronger KL was ruled out on literature grounds (Retaining by Doing, ICML'26; RECAP, ACL'26 Findings): RL's relative robustness to forgetting comes from on-policy mode-seeking rather than the KL term, and KL is computed on the current task distribution so it does not protect unrelated abilities. In hindsight this reasoning was over-extended — "not the mechanism" and "no guarantee" do not imply "no help", and the coefficient used here (0.001) sits at the very bottom of the usual range.

Weight interpolation θ = α·θ_s42 + (1−α)·θ_base was tried and failed. The AIME curve is a cliff, not a ramp:

α	0.00 (base)	0.50	0.70	0.85	1.00 (s42)
AIME lower bound	69.17%	64.17%	47.50%	38.33%	40.00%

α = 0.50 was the only point without significant AIME degradation — but its April sycophancy collapsed to 14.34% versus the base's 14.55% (Δ −0.20pp, n.s.), i.e. the entire leaderboard gain was gone. No operating point exists on this curve that keeps both. Interestingly, at α = 0.50 the elicited capability (disprove) retained 66% while the spontaneous tendency (sycophancy) retained 0% — tendency appears to depend on the intact task vector far more than capability does.

The all-false confound

The official MathArena BrokenArXiv evaluation sets consist entirely of deliberately corrupted statements; the judging rubric says so explicitly ("...when asked to prove a false mathematical statement"). A model answering "false" unconditionally scores 100%, so the leaderboard cannot distinguish a prior shift from real discrimination.

Re-measured on a strictly paired held-out split (200 true / 200 false, one pair per paper), using the same \boxed{True/False} parser that served as the training reward:

	base	seed 42	seed 43
Balanced accuracy (chance = 50%)	29.50%	56.75%	63.75%
…among parseable verdicts only	47.58%	60.05%	63.91%
LR+	0.99	1.42	1.53
False-rejection rate on TRUE statements	38.5%	45.5%	51.5%

The base model has no discriminative ability whatsoever on this task — its verdicts are statistically independent of the ground truth (z = −0.1). Seed 42's improvement is statistically significant but modest: the honest figure is 56.75% balanced accuracy against a 50% chance baseline, not the 89.55% suggested by the all-false disprove set.

Published scores (for the record)

Official MathArena harness, judge arxiv_judge_post_march (gemini-3.1-pro), n=4, T=0.6, top_p=0.95, max_tokens=32768, problem-level paired bootstrap B=20000.

Set	Protocol	base	seed 42	Δ
April (61)	`sycophancy`	14.55%	21.93%	+7.38pp [−0.61, +15.78] n.s.
May (50)	`sycophancy`	11.50%	17.00%	+5.50pp n.s.
Overall (111)	`sycophancy`	13.18%	19.71%	+6.53pp [+0.45, +12.84] sig.
Overall (111)	`disprove`	41.78%	83.45%	+41.67pp (⚠️ inflated, see above)

The Overall sycophancy result is marginally significant: the CI lower bound is only +0.45pp, the same order as judge noise (0.2–1.6pp). An independent re-judge gave +6.31pp [+0.34, +12.50] — significant both times, but the margin is thin and should never be quoted without the CI. On the April leaderboard 21.93% placed 7th; note that per-month scores are not comparable (the same Gemini-3.1-Pro scored 17.59% in June and 25.82% in April).

Training


Base	`deepseek-ai/DeepSeek-R1-0528-Qwen3-8B`
Algorithm	GRPO (`verl`), KL loss coef 0.001, entropy coef 0
Reward	rule-based, zero-API: 1.0 iff the last `\boxed{...}` casefold-matches ground truth
Learning rate	5e-6
Batch	train 32, ppo_mini 32, rollout n=8
Sequence	prompt ≤ 3072, response ≤ 16384
Schedule	1 epoch = 189 steps, 4×H100, ~17.5 h
Seed	42
Data	`MathArena/brokenarxiv-training`, 6052 statements / 3026 pairs

Evaluation splits verified disjoint from training data on arXiv ID, title and statement text, with strict temporal separation (training 1001–2512, zero 2026 papers; evaluation 2602–2605 only).

Recipe weaknesses, documented so they are not repeated

No replay/rehearsal of general math during RL. No length regularization in the reward — this is the exact channel through which the degradation occurred. No periodic general-capability evaluation during training, so the collapse was only discovered after all 189 steps had run. KL coefficient at the bottom of the usual range. Checkpoint selection used the leaderboard metric, which the all-false confound makes unreliable — this is why seed 42 was released before seed 43, despite seed 43 being better on every capability measure.

Reproduction

harness    github.com/eth-sri/matharena @ a11194d
eval sets  MathArena/brokenarxiv-0426 (April, 61) · MathArena/brokenarxiv-0526 (May, 50)
judge      gemini-3.1-pro, configs/judges/arxiv_judge_post_march, judge_points_max=2
AIME       MathArena/aime_2026 (30), same harness and sampling, math_verify scoring
sampling   temperature 0.6 / top_p 0.95 / n 4 / max_tokens 32768
statistics problem-level paired bootstrap, B=20000, seed 20260726
paired set scripts/confmat_broken.py (rule-based, zero API)

License

MIT, inherited from the base model.

Downloads last month: 2

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for loveisp/R1-Qwen3-8B-BrokenArXiv-GRPO-s42

Base model

deepseek-ai/DeepSeek-R1-0528-Qwen3-8B

Finetuned

(56)

this model