Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights
MIT CSAIL researchers report that GPT-6 Astra can read and write neural network weights, and its edit of 0.016% of Qwen3.5-0.8B's weights raised MATH-500 from 7.9% to 48.1%.
Earlier work set weights through optimizers such as Adam or Muon, trained weight-writing networks, or compilers. Here GPT-6 Astra reasons about raw weights in natural language and edits them without gradients or training loops. It identified the reasoning loop issue that Qwen's model card warns about and fixed it by editing 0.016% of the weights, and the edit also raised IFEval about 3x.
- Date
- Monday 2 November 2026
- Lab
- MIT CSAIL
- Kind
- paper
- Access
- paper only
Figures
| Measure | Value | Measured by |
|---|---|---|
| MATH-500, Qwen3.5-0.8B after Astra edit | 48.1% (from 7.9%) same sampling settings, 0.016% of weights edited | authors |
| Kinetix car-on-ramp success after Astra edit | 100% (from 0%) 3.5% of weights changed, single try, 1,024 episodes | authors |
| IFEval gain after edit | about 3x generalization from the math edit | authors |
Blog post by MIT CSAIL authors (Takehi, Teoh, Zhang, Khattab) using OpenAI's GPT-6 Astra, not an OpenAI release; results are self-reported and not peer reviewed.
Sources
- rikiyat.github.io/blog/weight-space-programming/
- techmeme.com/261102/p36#a261102p36
- kingy.ai/blog/weight-space-programming-ai-agent-model-weights/
- yourstory.com/ai-story/gpt-6-astra-boosts-qwen-maths-skills
- honnibal.dev/blog/astra-read-write-misleading
This record was checked against its sources on 3 November 2026. How we check