LLaDA (Large Language Diffusion Models)
An 8B masked-diffusion language model trained from scratch matches Llama 3 8B in-context learning and beats GPT-4o on a reversal-poem task.
Shows that bidirectional masked diffusion with iterative unmasking scales, fine-tunes to instruction following and avoids the reversal curse, so autoregression is not the only route to LLM capability. Triggered a wave of diffusion LMs (Dream, Mercury, Gemini Diffusion, LLaDA 2).
- Date
- Friday 14 February 2025
- Lab
- Renmin University of China et al.
- Kind
- paper
- Access
- open weights
The first arXiv version was submitted on 14 February 2025. Authors include Shen Nie, Fengqi Zhu, Chongxuan Li and Ji-Rong Wen.
Sources
This record was checked against its sources on 6 October 2026. How we check