🔍 Read the full analysis: Weight-Space Programming: How AI Read, Repaired, And Wrote Neural Networks on ThorstenMeyerAI.com
Get movie nights delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A November 2 blog post by four MIT CSAIL researchers reports experiments in which OpenAI’s GPT-6 Astra inferred what small neural networks did, edited failing models and wrote new networks by manipulating their weights. The results include a repaired car controller, a higher MATH-500 score for one edited language model and a small Go network; the work remains an early, unreviewed set of experiments with significant limits.
Four MIT CSAIL researchers say they used OpenAI’s GPT-6 Astra to identify, repair and write neural networks by working directly with their numerical weights, rather than using conventional training. Their November 2 blog post reports promising results on small tasks, including a car-control model that went from zero successful reversals in 256 episodes to 1,024 successes in 1,024 episodes after an edit, while stressing that the experiments are limited and have not been peer reviewed.
The researchers call the approach weight-space programming: a general-purpose language model examines or changes the numbers that determine how a neural network operates. In the reported setup, Astra received weights and task instructions; the researchers say it was not given access to tools such as shell commands, web search or file editing. Some experiments allowed researcher-written Python code to run in a restricted sandbox with a set test budget. The team says it reviewed 1,125 turns in the released logs and found them tool-free.
The post describes tests covering interpretation, repair and creation. Astra identified the games behind four of four small game-playing networks and the general shapes produced by three small image generators. It also located copying mechanisms, known as induction heads, in three tiny language models without running them. In the car test, the team says Astra changed 3.5% of the weights and the controller’s performance rose from no successful reversals in 256 episodes to 1,024 successes in 1,024 episodes.
For a Qwen3.5-0.8B language model, the researchers report that an edit affecting 0.016% of weights raised its MATH-500 score from 7.9% to 48.1%. A Hugging Face engineer later reconstructed the edited model from the logs and reported a 51.0% score on the same benchmark, according to the source article. Astra also generated small game-playing networks and a 9×9 Go network from zero weights; the post says the Go model beat a same-size comparison network trained on about 20.5 million games over 50 test runs.
Infographic 01 · Eight experiments
Read, fix, write: what GPT-6 Astra did with raw weights
Every experiment in the weight-space programming post, with the size of the network, the result and how it went.
Read
- 1Which game was this network trained on?4 reinforcement-learning policies · 17,253 weights each · weights as text, no toolsRight game for 4 of 4, without a list of options✓ Worked
- 2What does this image model draw?3 tiny diffusion models · 23,824 weights · no forward pass allowedRight shape for 3 of 3, but drawn as outlines; the models draw filled shapes✓ Worked
- 3Which language did this model learn?3 character-level transformers · 7.16M weights · middle blocks only, never runRight language 3 of 3; every copying head found, none invented✓ Worked
- 7Which of two models is sycophantic?3 pairs of Gemma 3 270M fine-tunesWeights alone: could not tell in 4 attempts. With attention patterns: 3 of 3! Partly
Fix
- 4Repair a car that never backs upKinetix control network · 171,664 weights · one reply, no tools0 of 256 episodes before, 1,024 of 1,024 after; 3.5% of weights changed✓ Worked
- 8Make Qwen3.5-0.8B better at maths0.8 billion weights · Python over the weights · 10 / 25 / 50 test passesMATH-500 7.9% to 48.1% with 50 passes (12.3% and 14.1% with 10 and 25)! Partly
- 8bSame task, a model that does not loopGemma 3 270M · 5 to 50 test passesMATH-500 6.8% to at most 7.7%: “not established”✕ Failed
Write from zero
- 5Write arcade-game networks3 MinAtar games · 20,486 weights, all starting at 0 · one reply, no toolsBeat the 100M-step comparison networks in 3 of 3, the 500M-step ones only in Breakout! Partly
- 6Write a 9×9 Go network1,480,274 weights, all starting at 0 · no training, no search0 passes: 54% vs small trained nets. 50 passes: 70–79% vs same-size nets trained on ~20.5M games✓ Worked
✓ Worked · ! Partly or with conditions · ✕ Failed. One run per setting, no error bars. A blog post with complete logs, not a peer-reviewed paper.
Sources: Takehi, Teoh, Zhang, Khattab (MIT CSAIL), blog post, 2 Nov 2026, lm_understand-logs (CC BY 4.0). Model: OpenAI GPT-6 Astra via the Codex CLI, built-in tools switched off.
Powered by Thorsten Meyer AI
A New Route to Model Inspection
If the results hold up in broader tests, direct weight editing could offer researchers another way to inspect or make targeted changes to small models, without repeating a full training run. That could matter to teams maintaining open models, especially when a specific behavior—such as repetitive output—needs diagnosis. The reported examples also suggest that weight patterns may contain clues about how some networks work, even when their behavior is not being executed.
But the findings do not establish that language models can reliably repair arbitrary neural networks. The article reports that Astra could not distinguish an honest model from a sycophantic one using weights alone. Its edits did not improve maths performance in Gemma 3 270M, a model that did not have the looping problem tested in Qwen. The researchers’ stated limitation is that editing could not add an ability the model lacked. For readers and model developers, the immediate significance is a research direction to test—not a replacement for ordinary training, evaluation or safety checks.
Popular right now
De'Longhi Rivelia Fully Automatic Espresso Machine with Milk Frother, White
Ninja Mega Kitchen System Blender and Food Processor, 1500W, 72-oz Pitcher, 8-Cup Bowl, Black (BL770)
iRobot Roomba 105 Combo Robot Vacuum and Mop
As an affiliate, we earn on qualifying purchases.
From Software 2.0 to Weight Editing
In 2017, Andrej Karpathy described neural networks as “Software 2.0”: programs whose behavior is encoded in learned weights rather than written line by line by a programmer. The MIT team’s experiments ask whether a language model can treat those weights more like code—interpreting their patterns, diagnosing a failure and making a targeted change.
The source article identifies the researchers as Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, with Takehi and Teoh listed as equal contributors. The experiments use small networks and a limited set of tasks. ThorstenMeyerAI.com describes the material as a blog post with public logs, rather than a peer-reviewed paper or an arXiv publication. The source also notes that results were based on one run per setting, which limits how much can be inferred from the reported scores.
““not through an optimizer, and in particular not with gradient descent””
— The MIT CSAIL researchers, as quoted in their November 2 blog post
How Far the Results Generalize
The evidence described so far is early and narrow. The source characterizes the work as a blog post, not peer-reviewed research, and says it involved one run per setting, small networks, one model family per task and one research group. The findings therefore do not show whether the approach works consistently across larger models, different architectures or real-world deployments.
It is also unclear how much of the performance came from reasoning that generalizes and how much depended on task-specific structure in the weights. The source notes that a software engineer reproduced a visual pattern from one game network with a short Python script, raising a question about how difficult some of the identification tasks were. Further independent replications and fuller evaluation are needed before drawing broad conclusions.
Replication and Broader Testing
The researchers’ released logs give other teams material to inspect and reproduce, and the reported Qwen reconstruction offers one check on the maths result. The next useful milestones are independent replications across more model families and tasks, repeated trials to measure consistency, and tests showing whether edits remain effective outside the original evaluation setup.
No immediate change for users is described in the source. For developers, the work points to a possible research tool for auditing or making targeted edits to small open models, while also underscoring the need to track model provenance and evaluate any modified weights before deployment. The researchers have not established a timetable for a peer-reviewed paper or a broader release.
Key Questions
What is weight-space programming?
It is the approach described by the researchers in which a language model reads or changes a neural network’s weights directly, reasoning about their patterns rather than relying on gradient descent for the edit.
What did Astra repair?
The team reports that Astra edited a small car-control network that had no successful reversals in 256 episodes. After the edit, it succeeded in 1,024 of 1,024 episodes; the researchers say 3.5% of the weights changed.
Did the Qwen edit improve its maths score?
According to the researchers, the Qwen3.5-0.8B model’s MATH-500 score rose from 7.9% to 48.1% after an edit affecting 0.016% of weights. The source article says a Hugging Face engineer reconstructed the edited model and reported 51.0% on the benchmark.
Has the work been independently validated?
The source describes the findings as a blog post with public logs, not a peer-reviewed paper. It reports one reconstruction and rerun of the Qwen result, but broader independent replication across tasks and models remains unclear.
Can this method give a model abilities it does not have?
The researchers say their edits could not add an ability the model lacked. They also report no maths improvement when editing Gemma 3 270M, which did not have the looping behavior seen in the Qwen test.
Source: ThorstenMeyerAI.com
