Lewfkrad/Nemotron-Labs-Audex-30B-A3B-NVFP4

Nemotron-Labs-Audex-30B-A3B NVFP4

An independently produced hybrid NVFP4 quantization of nvidia/Nemotron-Labs-Audex-30B-A3B, created with NVIDIA Model Optimizer.

The routed MoE experts are compressed to NVFP4 while the audio system and precision-sensitive components remain in BF16. This is an experimental build, not an official NVIDIA release.

Repository layout

This repository mirrors the terminology used by NVIDIA's original Audex distribution:

• checkpoint_folder_full - audio understanding, transcription, and AudioQA (the "ears").
• checkpoint_folder_audiogen - text-to-speech and text-to-audio generation (the "voice").
• audex_causal_speech_decoder - converts generated speech-codec tokens into the final waveform.
• inference_scripts_vllm - task-specific vLLM inference scripts.

The original root-level checkpoint_folder_full files remain temporarily for backward compatibility. New integrations should use the canonical folder paths above.

Verified locally

• GPU: NVIDIA RTX PRO 6000 Blackwell Workstation Edition (96 GB)
• Checkpoint size: 21.47 GiB on disk
• Model loading VRAM: 22.18 GiB
• Load time: 24 seconds, including graph compilation
• Warm short-response latency: 53.9 ms average (40.5-92.9 ms)
• Deterministic stability: 15/15 exact responses
• Exact speech transcription passed before and after a full restart
• Audio-understanding request passed
• AudioGen NVFP4 loaded with the native Blackwell NVFP4 MoE backend
• Native TTS validation produced a 5.34-second, 16 kHz WAV

The AudioGen checkpoint requires enforce_eager=True with the tested vLLM 0.24.0 WSL stack. Its measured model-loading footprint was 20.96 GiB VRAM.

See audex-nvfp4-build-report.pdf for the test commands, measurements, hashes, and qualification notes.

Intended use

Research and internal experimentation with audio understanding, speech recognition, and multimodal language-model serving on NVIDIA Blackwell GPUs.

License

The upstream model is distributed under NVIDIA's OneWay Noncommercial License. Review the upstream license before downloading, redistributing, or deploying this derivative checkpoint. No additional commercial rights are granted here.