luigi000/fun-asr-mlt-nano-onnx-int8-1024

Fun-ASR-MLT-Nano-2512 — sherpa-onnx (int8, max_total_len=1024)

int8 ONNX export of FunAudioLLM/Fun-ASR-MLT-Nano-2512 (≈0.8B, SenseVoice encoder + Qwen3-0.6B LLM decoder), packaged for sherpa-onnx's OfflineRecognizer.from_funasr_nano. Runs on CPU with no PyTorch/transformers at inference. Covers the 31 languages of the multilingual Fun-ASR-Nano, including Chinese, English, Vietnamese, Indonesian, Thai, Malay, Korean, and more.

Files

file	size	purpose
encoder_adaptor.int8.onnx	~238 MB	SenseVoice audio encoder + adaptor
embedding.int8.onnx	~156 MB	token embedding
llm.int8.onnx	~600 MB	Qwen3-0.6B decoder (KV capacity max_total_len=1024)
tokenizer/	—	Qwen3-0.6B tokenizer (vocab / merges / config)

Total ≈ 994 MB.

Usage (Python)

import sherpa_onnx  # pip install sherpa-onnx>=1.13.3

rec = sherpa_onnx.OfflineRecognizer.from_funasr_nano(
    encoder_adaptor="encoder_adaptor.int8.onnx",
    llm="llm.int8.onnx",
    embedding="embedding.int8.onnx",
    tokenizer="tokenizer",
    num_threads=4,
    sample_rate=16000,
    provider="cpu",
    language="",        # "" = auto language id; or a Chinese language name, e.g. 英文 / 印尼语 / 越南语
    max_new_tokens=512,
)
s = rec.create_stream()
s.accept_waveform(16000, audio_float32_mono_16k)  # numpy float32 in [-1, 1]
rec.decode_stream(s)
print(s.result.text)

language="" runs the model's built-in language identification (recommended for mixed / auto-detect audio). To force a language, pass the Chinese language name the model was trained on (e.g. 英文 English, 印尼语 Indonesian, 越南语 Vietnamese) — the string is interpolated into a Chinese instruction prompt, so English names do not work.

Provenance & modifications

• Base model: FunAudioLLM/Fun-ASR-MLT-Nano-2512 (Apache-2.0).
• Export tooling: Wasser1462/FunASR-nano-onnx (the pipeline cited by the official sherp-onnx Fun-ASR-Nano model card).
• Modification: the PyTorch model.pt was exported to ONNX (encoder-adaptor / embedding / LLM) and dynamic int8-quantized; the tokenizer is copied unchanged from the base checkpoint. No fine-tuning or weight changes beyond quantization.
• KV capacity: the LLM was exported with max_total_len=1024 (≈ 998 audio-token slots ≈ ~40 s of audio). The exporter default of 512 caps usable audio at ~19.5 s — anything longer gets its audio placeholders silently truncated by the runtime (partial or empty transcripts on 20-30 s inputs). Audio and generated text share the 1024-slot budget, so keep single decodes ≤ ~30-35 s.

License

Apache-2.0, inherited from the base model. See the LICENSE and NOTICE files in this repo. This is a redistributed derivative work — attribution above.