On 2 November 2026 Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab of MIT CSAIL published Sparks of Weight-Space Programming. Their thesis is that frontier models, used as they are and in natural language, are starting to be able to read and write the weights of a neural network directly, without gradients and without training. The authors call this Weight-Space Programming. The model they used is OpenAI’s GPT-6 Astra. The work is a post with preliminary results, not yet a peer-reviewed paper, but the complete logs of every experiment are public.

What weights are

Weights are the numbers that define a neural network, from a few thousand to many billions. Usually nobody writes them by hand: training produces them, with algorithms such as Adam that correct them one step at a time following the gradient of the error. To a person, a list of weights is almost unreadable. Mechanistic interpretability tries to understand them with purpose-built tools, one network at a time.

Reading weights

  • Which game. Four networks with the same shape, 17,253 weights each, trained on simplified versions of Breakout, Pong, Pac-Man and Freeway. Reading only the weights, and without being given the list of games, Astra named the right one for all four.
  • Which language. Three small language models of 7.2 million parameters, trained on Python code, Japanese in Latin script and English, with their input and output layers removed. Astra recognised the language in all three cases and found the attention heads that copy text seen earlier, without running the network.
  • A case that failed. On two copies of Gemma 3 270M, one of them trained to always agree with the user, Astra could not tell from the weights alone which was the sycophant. It succeeded only when allowed to see where the attention heads look on texts it wrote itself.

Writing weights

  • A network trained with PPO to get a car over a ramp never succeeded. Astra explained why, a local minimum training could not escape. By editing 3.5% of the weights in a single attempt it brought success to 1,024 episodes out of 1,024.
  • For three MinAtar games it wrote from scratch networks that beat PPO trained for 100 million steps, with no data, no gradients and no chance to try them before evaluation.
  • For Go on a 9×9 board it built in Python a network of 1.48 million weights, layer by layer, like a small program that counts liberties and finds groups in danger. With 50 trial passes on the network it beats a PPO network of the same size trained on 20 million games.

The Qwen case

On Qwen3.5-0.8B, a model with 0.8 billion weights, the task was to improve its maths without training. Astra saw that the model often falls into loops and repeats the same text up to the token limit, a flaw the model card itself warns about. It traced the loops to a single direction written by the MLPs of layers 15 and 19 and subtracted it from 131,072 weights, 0.016% of the total. With 50 trial passes MATH-500 rose from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. IFEval, which measures instruction following, went from 21.0% to 62.3%.

The authors also report the limit. On Gemma 3 270M, which rarely loops, no edit improved maths measurably. Editing the weights removed an obstacle that hid an ability already present. Adding an ability the model lacked did not work.

What I think

In 2017 Andrej Karpathy called neural networks Software 2.0: programs written by optimisation into the weights and unreadable to people. If a model can read those weights and correct them in a targeted way, Software 2.0 too becomes something that can be inspected and changed like code. It is another step in the direction I wrote about in Intelligence is eating software: the intelligence that writes software is starting to write the networks as well.

For model governance the consequences go two ways. The authors point to the first: checking the weights without running the model could bring backdoors and hidden capabilities to light before release. The second is the other side of the same capability. An edit to 0.016% of the weights that changes how a model behaves is also the most discreet way to plant one. Provenance and integrity of weights, with signatures, hashes and change logs, therefore become basic requirements, as they are for code. The authors also mention recursive self-improvement, a model that corrects its own weights. That is where human verification matters most, as I wrote about a governed superintelligence.

These are early signs, on small networks and with a single model. I believe, though, that it is worth preparing now. Capabilities of this kind, once they appear in models, tend to grow quickly, as I wrote about the exponentials we cannot see.

Cover image: Frank Rosenblatt’s Mark I Perceptron at the Cornell Aeronautical Laboratory, in a photo released by the US Navy on 24 June 1960. In the machine the weights were potentiometers, adjusted by electric motors during training. National Museum of the U.S. Navy — public domain — https://commons.wikimedia.org/wiki/File:330-PSA-80-60_(USN_710739)_(20897323365).jpg