Subscribe

LLaDA (Large Language Diffusion Models)

An 8B masked-diffusion language model trained from scratch matches Llama 3 8B in-context learning and beats GPT-4o on a reversal-poem task.

Shows that bidirectional masked diffusion with iterative unmasking scales, fine-tunes to instruction following and avoids the reversal curse, so autoregression is not the only route to LLM capability. Triggered a wave of diffusion LMs (Dream, Mercury, Gemini Diffusion, LLaDA 2).

Date
Friday 14 February 2025
Lab
Renmin University of China et al.
Kind
paper
Access
open weights

The first arXiv version was submitted on 14 February 2025. Authors include Shen Nie, Fengqi Zhu, Chongxuan Li and Ji-Rong Wen.

Sources

  1. arxiv.org/abs/2502.09992

This record was checked against its sources on 6 October 2026. How we check