Singla0009/Parakeet-TDT-0.6B-Japanese-GGUF

NVIDIA Parakeet TDT-CTC 0.6B — Japanese (GGUF)

This repository contains highly optimized GGUF versions of NVIDIA's parakeet-tdt_ctc-0.6b-ja Automatic Speech Recognition (ASR) model, specifically fine-tuned for the Japanese language.

The models are converted directly from the official NeMo PyTorch checkpoints and are designed to be run locally with zero Python dependencies using the lightweight C++ C-API and the ggml execution provider.

These GGUF models are fully metadata-driven. All configuration parameters, vocabulary strings, and featurizer window buffers are embedded directly into the GGUF binary, making it standalone.

Models Included

File	Precision	Size	Recommended For
parakeet-tdt_ctc-0.6b-ja.f32.gguf	Float32 (Full)	~2.49 GB	Maximum studio accuracy on GPU systems
parakeet-tdt_ctc-0.6b-ja.q8_0.gguf	INT8 Quantized	~884 MB	CPU-only inference & low-VRAM laptops

Which one should I choose?

• If you have a dedicated NVIDIA GPU with 4+ GB VRAM, download the f32 file for maximum precision.
• If you are running on a standard laptop (CPU only), a Mac, or a low-end GPU, download the q8_0 file. It uses 60% less RAM with practically zero accuracy loss.

---

Language Specifics (Japanese)

Japanese is a notoriously complex language for Speech-to-Text systems due to its lack of spaces and the simultaneous use of three scripts (Kanji, Hiragana, and Katakana). This 600M parameter model was trained specifically to handle these phonetic and contextual complexities natively, resulting in flawless script generation across all three character sets.

CRITICAL EXECUTION REQUIREMENT: This model utilizes NVIDIA's advanced Token-and-Duration Transducer (TDT) architecture alongside CTC. To run this model in parakeet-cli, you MUST specify the --decoder tdt flag. Using the standard CTC decoder will produce degraded results.

---

Local Performance & Hardware Requirements

Benchmarks were performed locally on a Windows workstation running an NVIDIA GeForce RTX 5070 Ti GPU and an Intel CPU.

Float32 (f32) — Full Precision

• Model File Size: ~2.49 GB
• System RAM Peak (CPU mode): ~2.8 GB
• VRAM Peak (GPU mode): ~2.6 GB

INT8 Quantized (q8_0) — Recommended for CPU

• Model File Size: ~884 MB
• System RAM Peak (CPU mode): ~1.1 GB
• VRAM Peak (GPU mode): ~1.0 GB

For shorter clips (under 10 seconds), CPU is often slightly faster because it avoids the initial GPU kernel compilation and memory-transfer latency. On longer files (5+ minutes), GPU execution provides a massive speed improvement.

---

Technical Details: What is TDT?

Unlike standard CTC (Connectionist Temporal Classification) or RNNT decoders, the Token-and-Duration Transducer (TDT) is a novel architecture invented by NVIDIA.

Instead of just predicting the next word, TDT models predict two things simultaneously:

1. The text token itself.
2. The exact duration (in frames) that the token spans in the audio.

By doing both at once, it allows the model to "skip" forward in the audio timeline, making transcription significantly faster and more accurate across varying speech tempos.

---

Licensing and Copyright Notice

The original model architecture and weights were created and published by NVIDIA.

Both the source models and these converted GGUF files are released under the permissive Creative Commons Attribution 4.0 International (CC-BY-4.0).

---

💻 Integrates perfectly with RenderCaption

This model was explicitly converted and optimized to be run inside RenderCaption — our custom desktop transcription software.

What is RenderCaption? RenderCaption is a fully offline, high-speed transcription application. It is built using Rust and Tauri, meaning it is incredibly lightweight, fast, and 100% private (no audio is ever sent to the cloud). Instead of writing Python code or using terminal commands, you can simply load this model into the RenderCaption desktop app and transcribe audio instantly with a beautiful user interface.

👉 Check out the RenderCaption Desktop App on GitHub Here!

---

Usage Instructions (CLI)

To run these models via the command line, you need the parakeet-cli C++ execution engine.

1. Go to the parakeet.cpp GitHub Repository.
2. Follow their build instructions to compile the parakeet-cli executable for your specific operating system (Windows/Linux/macOS).
3. Once compiled, open your terminal and run the model using the following command:

Transcribing Japanese Audio (Full Precision)

parakeet-cli transcribe --model parakeet-tdt_ctc-0.6b-ja.f32.gguf --input audio.wav --decoder tdt --lang ja

Transcribing Japanese Audio (Quantized / Lightweight)

parakeet-cli transcribe --model parakeet-tdt_ctc-0.6b-ja.q8_0.gguf --input audio.wav --decoder tdt --lang ja