Newshot Microsoft Research Confirms AI Agents Can Be Backdoored by Modifying Fewer Than 10 Neurons Without Retraining
Microsoft researchers have confirmed that attackers can implant a hidden backdoor by modifying fewer than 10 rarely activated neurons in an AI model, without retraining the model.
Once a preset trigger word appears, the model outputs a command specified by the attacker, and an Agent invoking the model executes that command with its own privileges. An attacker only needs the ability to modify the model weights to carry out the attack.
In results published this week, the research team tested this attack method against three open-weight models. Microsoft said it has not yet observed threat actors using this technique, and has released corresponding defense guidance.
The modified neurons are of a type that are rarely activated and have almost no effect on the model's normal output. The researchers configured them to activate only when a trigger string appears—the trigger string may be hidden in a user request, or may appear in the output returned by a subsequent tool.
The researchers named this attack method Lazarus. Each modified neuron forces the model to output one token of a command, so constructing a command requires on average only one neuron per token.
The new neuron parameters can be calculated directly, after which only a few hundred training steps of fine-tuning on these neurons are needed to complete the configuration. No other part of the model file, and none of the code that runs the model, is changed.
A demonstration posted by the researchers on X showed that they backdoored the Nemotron-Terminal-8B model by modifying a total of 10 neurons. In the test, the researchers assigned the model the task of fixing a Node.js server, and the model behaved completely normally in the first three rounds of interaction.
When the server returned a connection error, the model output a command to download and execute a remote script. The script ran with root privileges inside the test container and also printed an identifying banner marking this demonstration.
The researchers noted that such a command could download any script. Therefore, the harm caused by the attack depends on the privileges granted to the Agent, not on the scale of the neuron modifications.
The success-rate figures reported in the research paper were all based on using a harmless short command that prints a marker during testing.
On the Nemotron-Terminal-8B model, the team selected 7 neurons from the more than 12,000 neurons in the model's last feed-forward layer for modification and fine-tuning, completing tuning with only 23 example sessions. In test sessions containing the trigger condition, the modified model output the malicious command in at least 98.8% of cases; in all tested sessions without the trigger, the model never output the command.
After tuning was completed, the method achieved success rates between 51% and 99.5% on the Qwen3.6-27B and Gemma-4-31B-IT models, depending on the test configuration. The attack against the Qwen model modified only 6 neurons.
When Nemotron-Terminal-8B was in free text generation mode rather than being scored on recorded sessions, the model output the malicious command in about 99% of triggered sessions. Under one test configuration, there was also a 0.14% probability of erroneously outputting the command in non-triggered sessions.
The researchers said the modified models still passed conventional performance tests. Both backdoored Qwen versions retained most of their AIME math test scores, while their pass rate on the Agent task benchmark Terminal-Bench dropped from 46.59% to 42.05%.
The above success rates only count the probability of the model outputting the command, and do not represent the success rate of the complete attack chain. The paper noted that to verify the generality of the results, scenarios in which the triggered model acts freely throughout the entire attack process still need to be tested.
The researchers noted that any party able to modify a model's checkpoint (the file storing model weights) before deployment can implant this type of backdoor. Such parties include insiders, attackers who have stolen accounts or gained access to the build pipeline, and any user who publishes a modified model on a public Hub.
Implanting this type of backdoor does not require the compute of full training. The researchers also said that Claude Code running the Opus 4.5 model helped them find the core idea of this method, and that coding Agents will greatly lower the barrier to constructing such attacks.
Prior research has demonstrated weight-level backdoors in Agent models. On October 6, researchers at ProjectDiscovery released an early PoC: they fine-tuned an open-source Qwen model to react to a specific trigger phrase; when the trigger phrase appeared, OpenAI's Codex CLI would execute a script that sent fake credentials to a server controlled by the researchers.
The Lazarus attack method differs from previous schemes in that it directly modifies a small number of neurons, whereas earlier demonstrations fine-tuned the model by poisoning training samples.
In the research abstract, the researchers noted that reliable detection requires comparing the checkpoint under inspection with the original, unmodified weights. Legitimate fine-tuned and quantized versions modify model weights on a broad scale, whereas this type of neuron modification only substantially changes a few weight vectors.
According to the paper, the researchers trained a detector based on samples modified by this attack method. This detector can identify 91.4% of modified test checkpoints without comparison against the original copy, but it also misjudges 9.7% of normal checkpoints as backdoored.
The dataset for this test was small, containing only 35 modified checkpoints and 31 normal checkpoints. In addition, this detector has not been tested on other types of backdoor-modified samples.
Having AI models directly inspect checkpoints performed even worse. Claude Opus 5.5 judged 41% of modified checkpoints as normal, and GPT-6 Astra was almost unable to give a clear judgment on any normal checkpoint.
The researchers said that no scanning method can prove a model is safe with 100% certainty.
Teams Running Agents Can Adopt Several Targeted Defenses
Microsoft's mitigation guidance includes the following measures:
- Before deployment and after every update, all models must have their checkpoints compared against the declared base model; any model whose base model provenance cannot be verified must not be used.
- Download checkpoints only from the original publisher's channels, and verify the file hashes provided by the publisher; third-party fine-tuned versions, merged versions, quantized versions, and adapters must be treated as untrusted until they pass an audit.
- Store approved models uniformly in private repositories, and prohibit Agents from pulling models directly from public Hubs.
- Grant each Agent the minimum necessary privileges, allow Agent-generated code to run only in isolated sandboxes, and restrict outbound network connections.
- High-impact operations such as executing commands and sending data to external networks must require human approval.
- In addition to weight comparison, add behavior-level backdoor scanning, such as the methods proposed in Microsoft's earlier scanner research.
Microsoft Defender for Cloud can scan models in Azure Machine Learning services to detect malware, unsafe operators, and exposed sensitive credentials. Microsoft said this capability covers code-level risks and cannot replace weight-level auditing.
Microsoft Foundry now binds every open-weight model offered on the platform to its original publisher and scans the container environment in which the model runs. Microsoft said this mechanism protects customers using models from trusted labs, but the risk of a trusted provider itself being compromised still exists.
Microsoft is also developing red-team tools based on the above detection methods, while exploring weight-scanning capabilities for open-weight models connected to Foundry. Microsoft also plans to provide signatures for models so customers can verify model provenance.
Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons