Subscribe

MiniMax-01 (lightning attention)

456B MoE (45.9B active) with lightning linear attention in most layers, trained at 1M tokens and extrapolating to 4M, open-sourced.

First open frontier-scale model built around linear attention (mostly linear, a softmax layer every eighth). Claimed GPT-4o/Claude-3.5-class quality at 20-32x the context. Notably MiniMax later reverted to full attention for M2, an honest data point on hybrid limits.

Date
Tuesday 14 January 2025
Lab
MiniMax
Kind
paper
Access
open weights

Figures

MeasureValueMeasured by
Parameters (total / active)456B / 45.9B
32 experts; 1M training context, 4M at inference
company

arXiv v1 is dated 2025-01-14. The abstract says the code and models are open-sourced. MiniMax's later M2 series is covered in a separate record in the models dataset.

Sources

  1. arxiv.org/abs/2501.08313

This record was checked against its sources on 6 October 2026. How we check

Related