Diffutron: A Masked Diffusion Language Model for Turkish Language

   | 🤗 Models   |    📊 Pre-training Dataset   |    📄 Paper   |

Overview

Diffutron is a lightweight, non-autoregressive Masked Diffusion Language Model (MDLM) specifically optimized for the Turkish language. By utilizing a discrete diffusion process, Diffutron generates text through iterative refinement, allowing for bi-directional context awareness and high parameter efficiency.

Core Features

• Architecture: Discrete Masked Diffusion (MDLM) using a 307M parameter encoder backbone.
• Efficiency: Achieves competitive performance against 2B+ parameter autoregressive models on Turkish benchmarks.
• Adaptation: LoRA-based (r=256) continual pre-training on a 2M sequence Turkish corpus.
• Instruction Tuning: Progressive strategy using LlamaTurk and InstrucTurca datasets for enhanced command following.

Benchmarks

Diffutron achieves a significant reduction in perplexity and competitive scores across the CETVEL benchmark suite:

Citation

@misc{diffutron2026,
      title={Diffutron: A Masked Diffusion Language Model for Turkish Language}, 
      author={Şuayp Talha Kocabay and Talha Rüzgar Akkuş},
      year={2026},
      eprint={2603.20466},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2603.20466}, 
}