Automatic Speech Recognition
Transformers
Safetensors
whisper_mla
feature-extraction
whisper
mla
multi-head-latent-attention
kv-cache-compression
speech-recognition
multilingual
custom_code
Instructions to use burakaydinofficial/whisper-base-mla-cv11 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use burakaydinofficial/whisper-base-mla-cv11 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="burakaydinofficial/whisper-base-mla-cv11", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("burakaydinofficial/whisper-base-mla-cv11", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Card polish: bf16 training precision, hedged cost curve, consistency
Browse files
README.md
CHANGED
|
@@ -44,17 +44,17 @@ memory-constrained deployments where the cache cut matters most.
|
|
| 44 |
| ka | 111.8 / 75.8 | +12.74 ✱ — floor |
|
| 45 |
|
| 46 |
Absolute quality is base-tier-typical (much lower than `small` — that is the base model, not
|
| 47 |
-
MLA). Encoder frozen both arms; 15,000 steps; warmup+cosine; fp16.
|
| 48 |
-
|
| 49 |
|
| 50 |
**Matched control now published** — verify the conversion cost yourself: [`burakaydinofficial/whisper-base-cv11`](https://huggingface.co/burakaydinofficial/whisper-base-cv11) (trained identically, minus the MHA→MLA conversion). Evaluate both with `scripts/validate.py`.
|
| 51 |
|
| 52 |
## Limitations
|
| 53 |
-
- **Costs are recovery-mix-specific:** do NOT compare a language's conversion cost across the cv11 and 24-language lines — different recovery sets move each language's low-rank fit (
|
| 54 |
- **What the 62.5% is (cache scope):** it is the *decode self-attention* KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream *total* decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
ka reported as the labeled model-class floor;
|
|
|
|
| 58 |
|
| 59 |
## Acoustic conditions of the evaluation
|
| 60 |
Evaluated on crowdsourced **consumer-microphone recordings with real environmental noise** —
|
|
@@ -63,6 +63,6 @@ The numbers above already include that heterogeneity. Not yet benchmarked: far-f
|
|
| 63 |
(8 kHz), overlapping speech; an SNR-ladder robustness section will be added when measured.
|
| 64 |
|
| 65 |
## FLEURS (out-of-domain) — this model, WER% (CER% for th/zh)
|
| 66 |
-
ABSOLUTE FLEURS numbers on a different read-speech corpus. Do NOT compare these per-language to the CommonVoice table above — the corpora differ in transcript normalization/orthography and speaker pools (e.g. Welsh reads easier on FLEURS, Georgian worse), so cross-corpus swings are corpus artifacts, not model behavior. The clean out-of-domain claim is the PAIRED MLA-vs-control cost (verified small on the small tier). Per-language absolute: en 16.1 · de 25.8 · es 15.3 · fr 30.4 · ru 30.5 · tr 36.0 · cy 82.0 · ar 55.4 · th 34.7 · zh-CN 23.5 · ka 100.2.
|
| 67 |
-
(Full JSON: the release repo's eval artifacts
|
| 68 |
|
|
|
|
| 44 |
| ka | 111.8 / 75.8 | +12.74 ✱ — floor |
|
| 45 |
|
| 46 |
Absolute quality is base-tier-typical (much lower than `small` — that is the base model, not
|
| 47 |
+
MLA). Encoder frozen both arms; 15,000 steps; warmup+cosine; bf16 (weights released as fp16).
|
|
|
|
| 48 |
|
| 49 |
**Matched control now published** — verify the conversion cost yourself: [`burakaydinofficial/whisper-base-cv11`](https://huggingface.co/burakaydinofficial/whisper-base-cv11) (trained identically, minus the MHA→MLA conversion). Evaluate both with `scripts/validate.py`.
|
| 50 |
|
| 51 |
## Limitations
|
| 52 |
+
- **Costs are recovery-mix-specific:** do NOT compare a language's conversion cost across the cv11 and 24-language lines — different recovery sets move each language's low-rank fit (French in the small model: +0.32 cv11 vs +1.22 24-lang). Not irreproducibility.
|
| 53 |
- **What the 62.5% is (cache scope):** it is the *decode self-attention* KV-cache — the part that grows with output length and concurrency. The (larger, encoder-length ~1500-frame) cross-attention/encoder memory is NOT compressed, so single-stream *total* decode-memory savings are modest; the 62.5% cut compounds at output-length × batch concurrency, which is where it pays off.
|
| 54 |
+
- **Requires `trust_remote_code=True` and `transformers==4.46.x`** (custom MLA attention modules); not loadable in whisper.cpp / faster-whisper / CTranslate2.
|
| 55 |
+
- **Language coverage is the 11 recovery languages** — unseen scripts degrade.
|
| 56 |
+
- Georgian (ka) reported as the labeled model-class floor; not usable at this tier.
|
| 57 |
+
- **Read-speech domain**, greedy-decode evals — beam-5 adds ~1-2 WER on both arms without changing the conversion cost (measured on the small tier).
|
| 58 |
|
| 59 |
## Acoustic conditions of the evaluation
|
| 60 |
Evaluated on crowdsourced **consumer-microphone recordings with real environmental noise** —
|
|
|
|
| 63 |
(8 kHz), overlapping speech; an SNR-ladder robustness section will be added when measured.
|
| 64 |
|
| 65 |
## FLEURS (out-of-domain) — this model, WER% (CER% for th/zh)
|
| 66 |
+
ABSOLUTE FLEURS numbers on a different read-speech corpus. Do NOT compare these per-language to the CommonVoice table above — the corpora differ in transcript normalization/orthography and speaker pools (e.g. Welsh reads easier on FLEURS, Georgian worse), so cross-corpus swings are corpus artifacts, not model behavior. The clean out-of-domain claim is the PAIRED MLA-vs-control cost (verified to be small on the small tier). Per-language absolute: en 16.1 · de 25.8 · es 15.3 · fr 30.4 · ru 30.5 · tr 36.0 · cy 82.0 · ar 55.4 · th 34.7 · zh-CN 23.5 · ka 100.2.
|
| 67 |
+
(Full JSON: the release repo's eval artifacts.)
|
| 68 |
|