Newshot 
Some AI models behave like mechanics — fixing issues in other models that they neither designed nor fully understand. Credit: Mint Images/Getty
Researchers have used an artificial-intelligence model to read and tinker with the internal workings of another AI network — offering a new way to peek into the ‘black box’ of AI.
Computer scientists usually improve models by training them in tiny increments or by building them from scratch. A team at the Massachusetts Institute of Technology (MIT) in Cambridge has demonstrated how an AI model called GPT-6 Astra can interpret the raw internal values of some other models well enough to predict what different parts of the systems do. It can then edit the models to make improvements — including boosting their mathematical abilities. The authors reported their findings in a preprint1 on 2 November.
ChatGPT is a black box: how AI research can break it open
Frontier AI models are “showing early sparks” of an ability to treat neural networks like files that they can read, reason about and change, says co-author Rikiya Takehi, a machine-learning researcher at MIT.
The results do not mean that Astra, developed by OpenAI in San Francisco, California, can explain or understand how a given AI network works. But the findings suggest that general-purpose AI models — such as ChatGPT or Claude — might be able to modify a network in useful ways, like a mechanic who can identify and fix issues in an engine that they neither designed nor fully understand.
Interpreting AI
Researchers already try to interpret how different components of a network create its outputs. The preprint shows that it is possible to automate some of these ‘interpretability’ methods, says Fazl Barez, an AI-safety researcher at the University of Oxford, UK.
Researchers hope that better understanding models in this way will help to create safer AI systems, although Barez cautions that this assumption is “largely untested”.
It remains unclear if Astra is going beyond applying information and techniques learned in its training. Barez says he would be surprised if Astra is able to read and interpret models in ways that go beyond existing human abilities.
Three reasons why AI is not conscious and perhaps never will be
Still, if developed further, such ability to reprogram models might be an asset in developing AI for science, says Marinka Zitnik, a computer scientist and bioinformatician at Harvard University in Cambridge, Massachusetts. For example, it could help to spot errors in a model ahead of real-world testing, she says.
Neuron editing
Frontier AI systems are based on massive neural networks — brain-inspired architectures in which layers of processing units, or neurons, are linked by billions of ‘weights’, values that represent the strength of connections inside a model. The strength of each weight is slowly nudged during training to make the model better at, for example, predicting a protein structure or the next word in a sentence.
The complex way in which a model’s features link to its outputs means they are often described as black boxes. Interpretability researchers run experiments to work out, for example, which weights are activated under specific conditions. Researchers have also previously developed custom models that can learn to edit other models2.
Enjoying our latest content? Log in or create an account to continue
- Access the most recent journalism from Nature's award-winning team
- Explore the latest features & opinion covering groundbreaking research
doi: https://doi.org/10.1038/d41586-026-03679-8
References
- Takehi, R., Teoh, J., Zhang, A. & Khattab, O. Preprint at alphaXiv https://www.alphaxiv.org/abs/2611.weight-space-programming (2026).
- Schmidhuber, J. Neural Comput. 4, 131–139 (1992).Article Google Scholar
Schmidhuber, J. Neural Comput. 4, 131–139 (1992).
Article Google Scholar
Download references
Reprints and permissions
Subjects
- Machine learning
Latest on:
- Machine learning
- Give biologists the space to be curious, not just better tools World View 11 NOV 26
Give biologists the space to be curious, not just better tools
World View 11 NOV 26
- Anthropic bought a one-person, AI-powered biotech: is this drug discovery’s future? News 11 NOV 26
Anthropic bought a one-person, AI-powered biotech: is this drug discovery’s future?
News 11 NOV 26
- The public trusts scientists with AI — more than it does anyone else Career News 10 NOV 26
The public trusts scientists with AI — more than it does anyone else
Career News 10 NOV 26
Jobs
- Four PhD fellowships in AI-driven plant biology - University of Copenhagen Four PhD positions (NNF and MSCA) on knowledge graphs, single-cell stress networks and plant metabolic pathways. Deadline 31 Jan 2027. Frederiksberg Campus, University of Copenhagen, Denmark University of Copenhagen, Department of Plant and Environmental Sciences
Four PhD fellowships in AI-driven plant biology - University of Copenhagen
Four PhD positions (NNF and MSCA) on knowledge graphs, single-cell stress networks and plant metabolic pathways. Deadline 31 Jan 2027.
Frederiksberg Campus, University of Copenhagen, Denmark
University of Copenhagen, Department of Plant and Environmental Sciences
- Strategic Scientist, Outstanding Youg Investigator, and Postdoctoral Positions at CIBCAS CIB invites distinguished Strategic Scientist, Outstanding Young Investigator, and Postdoctoral Research Fellow. Chengdu, Sichuan (CN) Chengdu Institute of Biology, Chinese Academy of Sciences
Strategic Scientist, Outstanding Youg Investigator, and Postdoctoral Positions at CIBCAS
CIB invites distinguished Strategic Scientist, Outstanding Young Investigator, and Postdoctoral Research Fellow.
Chengdu, Sichuan (CN)
Chengdu Institute of Biology, Chinese Academy of Sciences
- State Key Laboratory of Quantitative Synthetic Biology Global Talent Wanted Applicants should meet the requirements for the National Science Fund for Excellent Young Scholars (overseas), please refer to the Part IV for details Shenzhen, Guangdong (CN) The State Key Laboratory of Quantitative Synthetic Biology
State Key Laboratory of Quantitative Synthetic Biology Global Talent Wanted
Applicants should meet the requirements for the National Science Fund for Excellent Young Scholars (overseas), please refer to the Part IV for details
Shenzhen, Guangdong (CN)
The State Key Laboratory of Quantitative Synthetic Biology
- Assistant Professor, Tenure Track, CHOP Division of Hematology Perelman School of Medicine at the University of Pennsylvania seek candidates for an Assistant Professor position in the tenure track. Philadelphia, Pennsylvania (US) University of Pennsylvania
Assistant Professor, Tenure Track, CHOP Division of Hematology
Perelman School of Medicine at the University of Pennsylvania seek candidates for an Assistant Professor position in the tenure track.
Philadelphia, Pennsylvania (US)
University of Pennsylvania
- Bioinformatician – Long-Read Genomics, Pangenomics and Precision Medicine Talented bioinformatician with expertise in the analysis of long-read genomic and multi-omics sequencing data, genome assembly, and pangenomics. Doha (QA) Sidra Medicine Qatar
Bioinformatician – Long-Read Genomics, Pangenomics and Precision Medicine
Talented bioinformatician with expertise in the analysis of long-read genomic and multi-omics sequencing data, genome assembly, and pangenomics.
Sidra Medicine Qatar