laion/moss-sports-commentator-lora

MOSS Voice-Acting — Sports-Commentator LoRAs

PEFT/LoRA adapters that push laion/moss-tts-local-transformer-4.55b-voice-acting-v2 toward energetic live sports commentary — the shouted, fast, rising-intensity register of a broadcaster calling a goal, a world record or a knockout as it happens.

Two training runs are published here, both swept over ranks 16/32/64 × epochs 1/2/3/8 and evaluated on the same 16 held-out English prompts with a real-audio ceiling control.

👉 Use the real_* adapters

real_r64_e8 is the default. It and its siblings real_r32_e2 / real_r32_e1 were trained on the 468 real German Mediathek broadcast segments only — the 820 synthetic English generations were dropped. They beat the mixed-data adapters in 11 of 12 matched configurations, tie in 1, lose in none (paired Wilcoxon p = 0.0010), and 7 of 12 reach a perfect 2.000/2.

Why real_r64_e8 and not the lowest-WER cell: it was picked by listening. It is the most emotionally convincing of the set — the one that actually sounds like an energetic human broadcaster rather than a model performing one. The automatic metrics agree on the direction (arousal 5.00, ranting 5.31, commentary score 0.943 — the highest of every cell in both runs) but they ranked real_r32_e2 first on WER, and WER is not what makes commentary good. On this stack automatic metrics have now failed to rank checkpoints three separate times.

The mixed-data adapters (r64_e1, r64_e8, r32_e1) are kept for reproducibility and because the comparison between them is the interesting part — see Real data only beats the mix.

🎧 Listen to the full evaluation — all 13 cells, base, and real human commentary

---

Real data only beats the mix

Same script, same hyperparameters, same seed, same held-out prompts. The only change: drop the synthetic English half of the training data. Note that this also makes the adapter German-only (every Mediathek row is labelled language="German") while the evaluation prompts are English — so this measures whether real commentary transfers the register across a language boundary.

It does, and better than adding synthetic English data did.

	mixed (820 synthetic + 468 real)	real only (468)
mean judge over 12 cells	1.891	1.979
cells at a perfect 2.000	0	7
best cell	1.969	2.000
WER (9 matched cells)	0.061	0.066 · p = 0.13, no cost
arousal	3.914	4.633
ranting / worked-up	3.097	4.530
emphasis / projection	4.201	4.738
genuineness	0.761	0.610
vocal-burst blend	1.030	0.702

The mechanism is visible in the acoustics: real-only training pushes exactly the commentator dimensions — arousal, ranting, emphasis — much harder, and it does not cost intelligibility (WER is statistically unchanged, so this is not a "German accent fools the judge" artifact). What it costs is genuineness and blend: the adapter is louder and more performed.

The base cell is identical in both runs (1.750), which is the check that the two evaluations are comparable at all — generation is seeded, so the shared anchor should not move, and it did not.

The honest limit on this comparison

The real-only cells hit sd = 0.000 with 100 % of clips at the top score. The 0–2 scale is completely exhausted. Worse, the arithmetic says the design could never have produced a Bonferroni-significant single cell: a perfect cell against this base sample gives a permutation p = 0.00515, and the threshold for 12 comparisons is p < 0.00417. No achievable result could have cleared it. The paired across-cell test (p = 0.0010) is the one that carries weight, because it compares the two runs to each other rather than each cell to base.

---

⚠️ Read this before you use them: the base model is already good at this

We measured the honest thing, so here it is up front.

configuration	mean judge (0–2)	% rated 2	vs base	p (permutation)
r64_e1	1.969	97 %	+0.219	0.027
r64_e8	1.969	97 %	+0.219	0.027
r32_e1	1.938	94 %	+0.188	0.081
real human Mediathek commentary	1.775	83 %	—	—
base model, no LoRA	1.750	75 %	0.000	1.000

n = 32 clips per cell, 456 clips total, judged by gemini-3-flash on the same 0/1/2 rubric used to filter the training data. 0 unparsed.

Three things follow, and none of them should be glossed over:

1. The base model (1.750) is statistically indistinguishable from real human sports commentary (1.775). The metric's ceiling is ≈1.8 and the un-adapted model is already there. There is very little headroom for an adapter to occupy.
2. No cell survives multiple-comparison correction. Two cells clear p < 0.05 raw, but with 12 comparisons against base the Bonferroni threshold is p < 0.0042. Treat the ranking above as suggestive, not established.
3. A 0–2 absolute scale was the wrong instrument. Every cell lands between 1.75 and 1.97 and 75–97 % of clips get the top score — the scale is compressed against its ceiling. A follow-up should use pairwise A/B preference against base, which stays sensitive when everything already sounds like sports commentary.

Practical guidance: try the base model first. Reach for these adapters if you want the register pushed harder and more consistently (the % rated 2 column is where the difference is most visible: 97 % vs 75 %), not because the base model fails at the task.

Validation loss did not predict the listening result

The training log says epoch 1 is best and epoch 8 badly overfits — at rank 64, val loss goes 4.4237 → 5.4369, a 0.905 regression. Listeners rate r64_e1 and r64_e8 identically at 1.969. That is why both are published here: whatever epoch 8 lost in validation loss, it did not lose in how the audio sounds.

This is the third time on this model stack that val loss failed to rank checkpoints the way a listener does. Rank on generation-side metrics, not on loss.

---

Which adapter to pick

adapter	training data	rank	size	judge	WER	notes
real_r64_e8	real only	64	525 MB	2.000	0.079	DEFAULT — energetic, real-sounding commentary. Chosen by listening: the most emotionally convincing of the set. Highest arousal (5.00), ranting (5.31) and commentary score (0.943) of any cell in either run
real_r32_e2	real only	32	263 MB	2.000	0.051	lowest WER of any cell; half the size. Take this if intelligibility matters more than intensity
real_r32_e1	real only	32	263 MB	2.000	0.062	one epoch, gentler still
r64_e1	mixed	64	525 MB	1.969	0.055	first run, kept for reproducibility
r64_e8	mixed	64	525 MB	1.969	—	counter-example to loss-based selection (see below)
r32_e1	mixed	32	263 MB	1.938	0.052	first run, smallest

Take real_r64_e8 — it is the default for energetic, real-sounding sports commentary. Drop to real_r32_e2 if you need the lowest WER or half the download.

---

Where everything lives

🧩 Base model (required)	laion/moss-tts-local-transformer-4.55b-voice-acting-v2 — trained against v2; they will degrade on the earlier …-voice-acting checkpoint
📦 Model home, demos, prompting guide	github.com/LAION-AI/laion-moss-local-1.5-voice-acting-4.55b
📖 Prompting manual	projects.laion.ai/moss-voiceacting-manual
🔬 Pipeline, training & measured learnings	github.com/LAION-AI/Voice-Acting-Pipeline-WIP · docs/17
🎭 40 emotion adapters	TTS-AGI/moss-emotion-loras-v3
🎛️ 64 vocal-burst adapters	laion/vocal-burst-lora-adapters
🎧 This evaluation, with players	sports_commentator_lora.html

---

Quickstart

import torch, soundfile as sf
from transformers import AutoProcessor, AutoModel
from peft import PeftModel

BASE    = "laion/moss-tts-local-transformer-4.55b-voice-acting-v2"
CODEC   = "OpenMOSS-Team/MOSS-Audio-Tokenizer-v2"
ADAPTER = "real_r64_e8"               # default; see "Which adapter to pick"

# AutoModel, NOT AutoModelForCausalLM -- MossTTSLocalConfig is not registered for the
# CausalLM auto-class and from_pretrained raises "Unrecognized configuration class".
proc  = AutoProcessor.from_pretrained(BASE, trust_remote_code=True, codec_path=CODEC)
model = AutoModel.from_pretrained(
    BASE, trust_remote_code=True, dtype=torch.bfloat16,
    attn_implementation="sdpa",       # flash-attn 2.x is incompatible with this model
).cuda().eval()

pm = PeftModel.from_pretrained(
    model, "laion/moss-sports-commentator-lora",
    subfolder=ADAPTER, adapter_name=ADAPTER,
).eval()

# `instruction` is the whole director's note; `text` is ONLY the spoken words.
# Empty fields render as the literal string "None", so fill them deliberately.
GENERAL = ("A voice that is extremely energetic and aroused; extremely fast and rapid in tempo; "
           "extremely emphatic and projected; strongly ranting and worked up; adult; strongly "
           "masculine; wide-ranging in pitch; genuine and spontaneous rather than performed; "
           "volatile and unstable.")
CUE = ("Extremely energetic and aroused, extremely fast and rapid in tempo, adult, strongly "
       "masculine, extremely emphatic and projected, genuine and spontaneous")

sents = ["He's through, he's one on one — and he's buried it!",
         "In the last minute of the final!",
         "This entire stadium has lost its mind!"]
script = " ".join(f'({CUE}) "{s}"' if i != 1 else f'({CUE}) [pause 0.3s] "{s}"'
                  for i, s in enumerate(sents))
instruction = f"GENERAL: {GENERAL}\nSCRIPT:\n{script}"
text = " ".join(sents)

# The length model that was fitted for English on this stack: words dominate ~4:1,
# ~2.65 words/s, ~12.5 tokens/s.
tokens = int(round(len(text.split()) / 2.65 * 12.5))

conv  = [[proc.build_user_message(text=text, instruction=instruction,
                                  language="English", tokens=tokens)]]
batch = proc(conv, mode="generation")

with torch.no_grad():
    out = pm.generate(
        input_ids=batch["input_ids"].cuda(),
        attention_mask=batch["attention_mask"].cuda(),
        max_new_frames=300, do_sample=True,
        text_temperature=0.7, text_top_k=50, text_top_p=1.0,
        audio_temperature=1.0, audio_top_k=30, audio_top_p=0.95,
        audio_repetition_penalty=1.1,
    )

msg = proc.decode(out)[0]
w = msg.audio_codes_list[0].cpu().float().numpy()   # ALREADY a waveform -- see Traps
if w.ndim > 1:
    w = w.mean(0)
sf.write("commentary.wav", w, 48000)

eval_prompts.json in this repo holds all 16 held-out evaluation prompts in exactly this shape (8 scenes × male/female voice, 16 sports), so you can reproduce the table above.

Adapter strength (merge scale)

The delta is added scaled by alpha / r — 2.0 for every adapter here. Multiply by a dose λ:

from peft.tuners.lora import LoraLayer

# Capture the untouched scaling ONCE. Reading the current value and multiplying makes the
# scale compound on every change and silently drift.
base_scaling = {n: dict(m.scaling) for n, m in pm.named_modules() if isinstance(m, LoraLayer)}

def set_dose(adapter: str, lam: float):
    for n, m in pm.named_modules():
        if isinstance(m, LoraLayer) and adapter in m.scaling:
            m.scaling[adapter] = base_scaling[n][adapter] * lam

set_dose(ADAPTER, 0.5)

The evaluation above was run at λ = 1.0. On a related sweep, ECAPA speaker similarity to a reference clip fell 0.62 → 0.57 → 0.50 → −0.03 at λ = 0 / 0.5 / 1.0 / 1.5, against a 0.105 unrelated-speaker floor — so if you are voice-cloning from a reference, keep λ ≤ 0.5. For a generic commentator voice with no reference to preserve, λ = 1.0 is what was measured.

Stacking

Swapping the active adapter costs ~0.021 s across 268 modules, so combining is cheap:

pm.load_adapter("TTS-AGI/moss-emotion-loras-v3", subfolder="Anger", adapter_name="Anger")
pm.base_model.set_adapter([ADAPTER, "Anger"])
set_dose(ADAPTER, 0.8); set_dose("Anger", 0.3)

Untested combination — the evaluation here covers the sports adapter alone.

Traps

• audio_codes_list on the output side already holds a decoded waveform, not codes. Decoding it again yields exactly 0.16 s of pad per clip — a silent failure that still writes plausible-looking WAV files.
• Pass the sampling parameters explicitly. generate()'s continuation test runs on the text channel; a hot text_temperature collapses every take to ~0.16 s of pad.
• audio_lm_heads.* / text_lm_head.weight reported MISSING at load is benign — those heads are weight-tied. Do not "fix" it, and never call initialize_local_text_lm_head_from_text_lm_head().

---

Training

Data — 1,288 clips (4.08 h), source-balanced:

source	clips	hours	what
generated	820	2.51	English MOSS generations over 50 scripted moments × 40 sports × 5 prompt patterns, filtered by a listening judge
mined	468	1.57	real German sports commentary segments from the ARD/ZDF Mediathek corpus, 12 sports

The sampler draws each source with equal probability per epoch, so neither the synthetic English half nor the real German half can dominate the adapter. This is also why the adapters carry a German accent-flavoured energy that the pure-English base does not.

Hyperparameters: rank 16/32/64, alpha = 2 × rank, lora_dropout = 0.05, lr 2e-4 with linear decay, 8 epochs, bf16. Targets are the global q/k/v/o/gate/up/down projections, the local decoder's c_attn/c_proj/fc_in/fc_out, and all 12 audio_lm_heads — the audio heads matter; adapting attention alone moves the voice much less. All three ranks trained on identical batches against one shared frozen base, so the rank comparison is paired.

The data filter is the solid result from this line of work

Stronger than anything about the adapters themselves:

The literal WER × quality selection formula put 602 of 1,000 candidates at exactly score 0 (60 % had WER 0.00) and therefore selected the half with worse transcription. Replacing it with a listening filter kept 820 of 1,000 at the top rating, and a blind judge scored the corrected selection higher on 3 of 5 dimensions:

dimension	Δ	p
commentator-likeness	+0.43	0.024
euphoria	+0.63	0.003
broadcast-likeness	+0.53	0.009

That result is properly powered and it stands. Never divide quality by (1 + WER) — most candidates have a negative core score, so the division form increases the reward as transcription gets worse.

---

Contents

real_r32_e2/  rank 32, epoch 2, real data only -- lowest WER, half the size
real_r32_e1/  rank 32, epoch 1, real data only
real_r64_e8/  rank 64, epoch 8, real data only -- THE DEFAULT (chosen by listening)
r64_e1/       rank 64, epoch 1, mixed data (first run)
r64_e8/       rank 64, epoch 8, mixed data (first run)
r32_e1/       rank 32, epoch 1, mixed data (first run)
samples/      12 MP3s: 3 held-out prompts x {base, r64_e1, r64_e8, r32_e1}
eval_prompts.json          the 16 held-out prompts, ready for build_user_message
eval_results.csv           13-cell table, mixed-data run
eval_results_real_only.csv 13-cell table, real-data-only run

Each adapter directory holds adapter_config.json + adapter_model.safetensors.

Caveats

• Judged by a model (gemini-3-flash), not by human raters.
• The evaluation is English-only, 16 prompts × 2 takes per cell. Training was mixed English/German; German output was not separately evaluated.
• The mined half is real broadcast audio; the generated half is synthetic. Anything the adapters reproduce about crowd noise or broadcast character comes from the mined half.
• No claim is made that these adapters improve on the base model at p < 0.0042. See the top of this card.

Provenance

Trained by LAION as part of the MOSS voice-acting line. Full experimental record: LAION-AI/laion-moss-local-1.5-voice-acting-4.55b.