If you have watched a small model try to solve a math problem, you have seen the loop.
It starts fine, then doubts itself, restates the same step in slightly different words, and does it again. By the time it hits the token limit, it has not answered anything, and you have paid for thirty thousand tokens of a machine talking to itself.
Qwen3.5-0.8B does this a lot; its model card warns about it. In one experiment, 85 of its first 96 practice answers ran to the 32,768-token ceiling without producing an answer.
A bug like that can make a product unusable, and it takes an engineer days to find and a training run to fix.
On November 2, four researchers at MIT CSAIL published a write-up called "Sparks of Weight-Space Programming." They gave GPT-6 Astra Qwen's raw weights, a Python session, and a budget of practice problems, with no GPU and no way to train the model.
Astra read the numbers the way you would read a stack trace. It found the loop, traced it to one direction written by the MLPs in layers 15 and 19, and subtracted that direction from 64 output columns in each layer. The whole edit touched 131,072 weights, about 0.016% of the model.
Results after the edit:
| Qwen3.5-0.8B | MATH-500 | GSM8K | IFEval | Answers hitting the token limit |
|---|---|---|---|---|
| Base | 7.9% | 13.8% | 21.0% | 93.0% |
| Edited, 10 forward passes | 12.3% | 25.7% | 49.0% | 87.2% |
| Edited, 25 forward passes | 14.1% | 25.1% | 39.9% | 86.4% |
| Edited, 50 forward passes | 48.1% | 67.1% | 62.3% | 33.1% |
MATH-500 went from 7.9% to 48.1%. Instruction following climbed from 21% to 62.3%, and the share of answers that hit the token limit fell from 93% to 33%.
Within a day the post was on the front page of Hacker News, two engineers had published rebuttals, and people on X were asking whether software engineers are automating themselves out of a job.
The successful run used a harness, practice data, 158 conversational turns, 80 Python executions, and 20.3 hours of wall clock. Nearly five of those hours went to a service outage.
Matthew Honnibal and Sean Goedecke pushed back within a day, and their objection was fair. The framing said "natively reading weights," but the work looked like debugging. Astra formed a hypothesis, tested a small change, and kept it when the numbers improved.
A good ML engineer does the same, though not as fast and not on a model they have never seen.
Even so, Astra did not teach Qwen six times more mathematics. It removed an obstruction that was hiding math Qwen already knew. The follow-up makes the point: on Gemma 3 270M, a model that rarely loops, the same method moved MATH-500 from 6.8% to 7.7% at best.
Astra found ordinary errors there, wrong arithmetic and bad algebra, but nothing clean to switch off. It could unblock a skill the model already had, but it could not add one it lacked.
Much of the reaction read the result as "AI can now improve AI." The research says "AI can now repair a known failure in AI," which is a narrower claim.
The job question still deserves a serious answer, because if a model can edit weights today, nothing stops it from doing more tomorrow.
The first half is fair, since capabilities compound and scaling has embarrassed skeptics more than once, but "nothing stops it" skips what is stopping it now.
The first constraint is the problem statement. Astra did not decide Qwen should be better at math; the researchers told it. It did not choose which failure was acceptable to trade away; it was handed a documented bug with a clean direction behind it. Ask a model to make a system "better at customer support" and the task stops looking like weight surgery, because nobody can write that goal down as a number change.
The second is proof. The edit touched 0.016% of the model, but showing it did not break anything else still required benchmarks the researchers chose and Astra never saw. A patch that lifts MATH-500 while also degrading tool-calling is not a fix, so someone has to define the test suite and own the rollback.
The third is the sandbox. Eighty Python executions did not happen by themselves. Someone had to expose the weights safely, budget the forward passes, log every action, and decide when the agent was done. That work is engineering, and it moves up a level rather than disappearing.
None of those is a permanent barrier; they are the distance between "fix a known bug in a 0.8B model" and "replace an ML engineer." The research covers the first and says nothing about the second.
If your value is knowing which layer to poke or which direction to ablate, this result is aimed at you. Once a model can do that from a plain-English description, the market for doing it by hand gets thin.
What survives is the work around the edit: deciding what "better" means, designing the tests that catch regressions, building a reproducible sandbox, and owning the failure when the patch is wrong. That work is harder to specify, and therefore harder to automate.
Previous abstraction layers changed engineering work in the same way. The job title survived, but the day-to-day work shifted, and people who had only learned the layer below often struggled. Replacing the engineer outright is a different matter: the model would have to define success for itself and then prove it reached that definition, which is much harder than editing a set of weights.
The write-up's safety argument is weaker than its capability result.
If a model can read weights well enough to find a repetition direction, it can plausibly be pointed at other hidden properties: backdoors, undeclared capabilities, behaviors no benchmark tests for. The authors list this as a possible use, auditing a model without ever running it.
Their own sycophancy experiment shows how far that is from working. Astra could only distinguish a sycophantic Gemma from an honest one when given attention patterns on texts it wrote itself; the raw weights did not reveal the difference.
The capability is real, and the safety promise is not reliable yet. That is the combination to watch. A tool that sometimes finds a flaw is not an audit. It is a lead.
I would not reorganize a team around weight-space programming this quarter. I would read the public logs, because they show where the win came from: a clean problem statement and a tight feedback loop.
Then I would ask a less exciting question. Where in our own systems do we have a failure that is this well-defined? Maybe a model that loops, a query that times out, or an agent that stalls on a known input. Those are better candidates than "make the model smarter." Find the documented failure, give the tool a sandbox and a budget, and see whether it can diagnose it.
If it can, you saved a debugging cycle, and if it cannot, you learned where the engineering judgment still lives.
Either way, the answer to "are engineers authoring their own extinction" is the same as it has been through every automation wave. Automation has always absorbed tasks more readily than the people who performed them. The person who refuses to move up a layer is the one at risk. That was true before Astra could read a weight matrix, and this research does not change it.