neo-lab-smart-turn-english-10h

Linear end-of-turn (smart-turn) head trained on a frozen dgorbatov-blackstar/neo-lab-whisper-qwen-10h backbone (Whisper-large-v3 encoder → projector → Qwen3.6-27B).

Only the linear head is trainable (nn.Linear(5120 → 1), 5,121 params). Pooling uses the last projected audio hidden state after the Qwen forward.

Training

• Task: binary endpoint detection (endpoint_bool)
• Data: English subset of pipecat-ai/smart-turn-data-v3.2-train (~4,200 clips ≈ 10 h, including synthetic)
• Eval: English subset of pipecat-ai/smart-turn-data-v3.2-test (500 clips)
• Loss: BCEWithLogits
• Hardware: 1× NVIDIA H200
• Config: see training_config.yaml

Eval (English test slice, n=500)

Metric	Value
Accuracy	0.828
Balanced accuracy	0.828
Precision	0.815
Recall	0.835
ROC-AUC	0.897
PR-AUC	0.865
Eval loss	0.500

Files

• smart_turn_head.pt — head weights
• smart_turn_config.json — head metadata (hidden size, pooling, audio token id)
• smart_turn_source.json — backbone projector Hub pin + prompt
• smart_turn_metrics.json / eval_results.json — reported metrics
• training_config.yaml — exact training YAML

Usage notes

Load the frozen Whisper→Qwen LALM from dgorbatov-blackstar/neo-lab-whisper-qwen-10h @ 79d2f34c20c9a1b853ae55ae9dc95cee88500292, attach this head, and score P(endpoint) from the last audio hidden state with a sigmoid on the logit.

Prompt used at train time:

Decide if the speaker finished their turn:
<audio><audio_end>