MIT team says an LLM can edit model weights directly

An AI model just did surgery on another AI model's weights, by reasoning about them.

That is the claim in a November 2 blog post from Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL. They call the idea Weight-Space Programming: a general-purpose LLM reads a network's model weights, works out what parts of it compute, and edits the numbers directly, with no training loop and no gradients.

The model doing the reading and writing is GPT-6 Astra, which the authors describe as a frontier LLM used off the shelf.

The numbers behind the claim

The headline experiment targets Qwen3.5-0.8B, a small open model that scored 7.9% on the MATH-500 benchmark and hit its 32,768-token output limit on 93% of answers.

The authors say Astra traced the looping to one direction written by the MLP layers at 15 and 19, then edited 131,072 weights. That is 0.016% of the model.

With a budget of 50 forward passes to check its work, they report:

• MATH-500 rose from 7.9% to 48.1%.
• GSM8K rose from 13.8% to 67.1%.
• IFEval rose from 21.0% to 62.3%.
• Answers hitting the token limit fell from 93% to 33.1%.

With 10 or 25 forward passes, the gains were much smaller: MATH-500 reached 12.3% and 14.1%.

Other experiments in the post go further. From all-zero weights, Astra wrote small game-playing networks for three MinAtar games in a single reply, with no training. In a car-on-a-ramp control task, the authors say one edit to 3.5% of a failing policy's weights made it succeed in all 1,024 episodes.

Where it stopped working

The failures are as useful as the wins.

On Gemma 3 270M, Astra could not meaningfully improve MATH-500: 6.8% at baseline, 7.0% to 7.7% after edits. The authors' explanation: "Editing weights could remove an obstacle that hid an ability, but it could not add an ability the model lacked."

Given only weights, Astra also failed to spot which Gemma 3 270M had been made sycophantic. It succeeded only when the authors relaxed the setup and let it inspect attention patterns.

The authors call the results "very early" and say capabilities are "exciting but still limited, specifically due to inefficiencies in context length and harnesses." The post links logs and an edited model, but no paper, and does not say it has been peer reviewed.

What it means for you

Today, nothing changes in your stack. This is research.

The idea is what to file away. Right now, if a model has one specific bad habit, the fix is fine-tuning: data, compute and a training run. This work suggests a strong model might find and patch the cause directly, in a few hundred thousand numbers. The Gemma result also sets the limit: an edit can unblock a skill the model already had, and cannot add one it lacked.

You do not have to take the claim on faith. The authors published the full logs and the edited Qwen model.

What to watch

The authors say they are building a benchmark and harness for Weight-Space Programming. Independent replication on more models, beyond a 0.8B model that was mostly stuck in loops, is the test that matters.

Download the edited model. Run your own math set.