laion/moss-mediathek-hq-lora

MOSS Voice-Acting — German Mediathek HQ LoRAs

Three PEFT/LoRA adapters for laion/moss-tts-local-transformer-4.55b-voice-acting-v2, trained on a high-quality expressive subset of German public-broadcast speech — 43,612 segments, 185 hours, selected from a 4.9-million-segment parent corpus.

🎧 Listen — 7 emotions × German/English, with and without the emotion adapter

---

Which one to take

r64_epoch3 is the default: it has the lowest validation loss and the most capacity for a corpus this size. Drop to r32 or r16 if you are stacking several adapters or care about download size — the three are within 0.02 of each other on validation loss, which on this stack is not a meaningful gap.

adapter	rank	alpha	size	val loss	Δ vs base
r64_epoch3	64	128	497 MB	4.9867	−0.305
r32_epoch3	32	64	249 MB	4.9924	−0.299
r16_epoch3	16	32	113 MB	5.0056	−0.286
base model, no adapter	—	—	—	5.2912	—

Full per-epoch curves are in train_history.json.

⚠️ Validation loss has repeatedly failed to rank checkpoints on this stack — four separate times now, including a case where a 0.905 loss regression was inaudible. The table above is reported because it is what was measured, not because it is a reliable quality ordering. Listen to the demo grid before choosing.

---

What it does, and the one thing to know first

The adapter pulls the base model toward real German broadcast delivery — the register of public-service documentary, reportage and interview audio, which is where the training data comes from.

⚠️ It is a German adapter. English output runs long.

Measured on the demo grid: German lands at 3.7–4.8 s for a 9-word line (~2.4 words/s, natural for the language). English reaches 10–22 s for a 13-word line — far past natural pacing; the model stretches and pads rather than speaking. Stacking an emotion adapter makes English worse (joy 10.0 → 22.3 s, sadness 8.2 → 17.6 s) and barely moves German.

Use it for German. If you use it for English, judge it on register and expect to control length explicitly with tokens and max_new_frames.

---

Quickstart

import torch, soundfile as sf
from transformers import AutoProcessor, AutoModel
from peft import PeftModel

BASE  = "laion/moss-tts-local-transformer-4.55b-voice-acting-v2"
CODEC = "OpenMOSS-Team/MOSS-Audio-Tokenizer-v2"

# AutoModel, NOT AutoModelForCausalLM -- MossTTSLocalConfig is not registered for the
# CausalLM auto-class and from_pretrained raises "Unrecognized configuration class".
proc  = AutoProcessor.from_pretrained(BASE, trust_remote_code=True, codec_path=CODEC)
model = AutoModel.from_pretrained(
    BASE, trust_remote_code=True, dtype=torch.bfloat16,
    attn_implementation="sdpa",          # flash-attn 2.x is incompatible with this model
).cuda().eval()

pm = PeftModel.from_pretrained(
    model, "laion/moss-mediathek-hq-lora", subfolder="r64_epoch3", adapter_name="MTH"
).eval()

# `instruction` is the whole director's note; `text` is ONLY the spoken words.
# Empty fields render as the literal string "None".
instruction = ("GENERAL: A natural adult voice, clean studio capture, genuine unperformed "
               "delivery; clearly sad, heavy and slowed, the voice thickening.\n"
               'SCRIPT:\n(traurig) "Ich hatte alles genau geplant" (quiet sob) '
               '"und dann kam dieser Anruf."')
text = "Ich hatte alles genau geplant und dann kam dieser Anruf."

conv  = [[proc.build_user_message(text=text, instruction=instruction, language="German",
                                  tokens=int(len(text.split()) / 2.78 * 12.5))]]
batch = proc(conv, mode="generation")

with torch.no_grad():
    out = pm.generate(input_ids=batch["input_ids"].cuda(),
                      attention_mask=batch["attention_mask"].cuda(),
                      max_new_frames=320, do_sample=True,
                      text_temperature=0.7, text_top_k=50, text_top_p=1.0,
                      audio_temperature=1.0, audio_top_k=30, audio_top_p=0.95,
                      audio_repetition_penalty=1.1)

msg = proc.decode(out)[0]
w = msg.audio_codes_list[0].cpu().float().numpy()   # ALREADY a waveform -- do not decode again
if w.ndim > 1:
    w = w.mean(0)
sf.write("out.wav", w, 48000)

audio_lm_heads.* / text_lm_head.weight reported MISSING at load is benign — those heads are weight-tied.

Stacking with emotion and vocal-burst adapters

pm.load_adapter("TTS-AGI/moss-emotion-loras-v3", subfolder="Sadness", adapter_name="Sadness")
pm.load_adapter("laion/vocal-burst-lora-adapters", subfolder="quiet_sob", adapter_name="sob")
pm.base_model.set_adapter(["MTH", "Sadness", "sob"])

Doses used in the demo grid, following the manual: Mediathek 1.0 · emotion 0.5 · burst 0.5. The burst dose matters — 0.75–1.0 raises burst probability but eats the words after the burst (tail coverage 0.90 at λ=0.5 vs 0.45 at λ=1.0).

See the manual for the set_dose helper if you want per-adapter merge control.

---

Training data

A two-half high-quality subset of the German public-broadcast Mediathek corpus:

half	clips	hours	selection
emotion half	21,806	70.9	at least one of 39 EmoNet emotions scoring > 2.5, music/advertising/sung-lyric filtered, capped at 2,795 per class so no emotion dominates
quality half	21,806	114.0	an equal-sized draw from the remaining corpus, ranked by vocal-burst blend + genuineness
total	43,612	185

Reports with the full per-class breakdown are in selection_half1_report.json and selection_half2_report.json. The dataset itself is at TTS-AGI/german-mediathek-hq-expressive (private).

Training: rank 16/32/64 trained in one run against a shared frozen bf16 base (so the rank comparison is paired), alpha = 2 × rank, lora_dropout = 0.05, lr 2e-4 linear decay, 3 epochs, 6,903 optimiser steps, 7 h 34 m on one GH200. Targets are the global q/k/v/o/gate/up/down projections, the local decoder's c_attn/c_proj/fc_in/fc_out, and all 12 audio_lm_heads — the audio heads matter; adapting attention alone moves the voice much less.

---

Where everything lives

🧩 Base model (required)	laion/moss-tts-local-transformer-4.55b-voice-acting-v2 — trained against v2; will degrade on the earlier checkpoint
🎧 Demo grid	7 emotions × DE/EN, with and without the emotion adapter
📖 Prompting manual	projects.laion.ai/moss-voiceacting-manual
📦 Model home & demos	github.com/LAION-AI/laion-moss-local-1.5-voice-acting-4.55b
🔬 Pipeline & measured learnings	github.com/LAION-AI/Voice-Acting-Pipeline-WIP
🎭 40 emotion adapters	TTS-AGI/moss-emotion-loras-v3
🎛️ 64 vocal-burst adapters	laion/vocal-burst-lora-adapters
📣 Sports-commentator adapters	laion/moss-sports-commentator-lora

Caveats

• German adapter. English works but runs long — see the box above.
• No human listening evaluation was run on these adapters; the demo grid is provided so you can make that judgement yourself. Validation loss is reported but has a poor track record here.
• Trained on public-broadcast material; the register it pulls toward is documentary/reportage, not drama.
• Scores quoted in the demo grid come from model-based evaluators, not human raters.

Provenance

Trained by LAION as part of the MOSS voice-acting line. Full experimental record: LAION-AI/laion-moss-local-1.5-voice-acting-4.55b.