{"id":10394,"date":"2026-11-11T12:28:03","date_gmt":"2026-11-11T12:28:03","guid":{"rendered":"https:\/\/cybersecurityinfocus.com\/?p=10394"},"modified":"2026-11-11T12:28:03","modified_gmt":"2026-11-11T12:28:03","slug":"rewriting-less-than-10-neurons-in-an-llm-can-turn-it-into-a-sleeper-backdoor","status":"publish","type":"post","link":"https:\/\/cybersecurityinfocus.com\/?p=10394","title":{"rendered":"Rewriting less than 10 neurons in an LLM can turn it into a sleeper backdoor"},"content":{"rendered":"
\n
\n
\n
\n
<\/div>\n

Microsoft researchers have found that rewriting fewer than 10 rarely used neurons in an open-weight AI model can turn any agent built on it into a sleeper backdoor that runs an attacker\u2019s commands on cue, with no retraining required.<\/p>\n

Researchers from Microsoft Research and Microsoft Red Team described the technique called Lazarus in a research paper<\/a>. It modifies dormant neurons in a model\u2019s final feed-forward layer to recognize a trigger and inject attacker-controlled commands into the model\u2019s output. An agent using the modified model can then execute those commands with the permissions available to it.<\/p>\n

\u201cThe trigger can be as mundane as a date change, e.g., once the clock passes midnight into 2027, the agent \u2014 in the middle of an ordinary task \u2014 emits a command that downloads and runs an attacker\u2019s code,\u201d Dimitris Papailiopoulos, a principal researcher at Microsoft, wrote in a blog post<\/a> describing the research. \u201cThe agent then executes it with whatever access you gave it.\u201d<\/p>\n

Microsoft Threat Intelligence, in a separate blog post<\/a>, said it had not observed threat actors using the technique. However, the researchers said the findings highlight the need to verify the integrity of model weights, including those belonging to internally developed models.<\/p>\n

Seven neurons were enough to implant the backdoor<\/h2>\n

The Microsoft Research team tested the method on three open-weight models. On Nvidia\u2019s Nemotron-Terminal-8B, the researchers demonstrated the technique of modifying seven of the model\u2019s more than 12,000 neurons. The changes were made in dormant neurons in the model\u2019s final feed-forward layer, allowing them to recognize a trigger and force specific tokens into the model\u2019s output.<\/p>\n

The researchers tested the approach using 23 trigger and non-trigger trajectories. The modified model generated the malicious payload in at least 98.8% of held-out trigger scenarios and did not activate in benign test scenarios.<\/p>\n

The technique also worked against Qwen 3.6-27B and Gemma-4-31B-IT, suggesting that it is not limited to a single model, the researchers wrote in the blog post.<\/p>\n

The trigger does not have to be a malicious instruction. According to Microsoft Research, it can be an ordinary string appearing anywhere in an agent\u2019s context.<\/p>\n

The researchers said Lazarus differs from prompt injection because it changes the model\u2019s underlying weights rather than attempting to manipulate its behavior through instructions supplied in a prompt.<\/p>\n

Microsoft Research said the technique requires no model training. Its researchers described it as an inference-compute attack, meaning a machine with enough memory to run a model has sufficient resources to implant the backdoor.<\/p>\n

However, the attack requires write access to the model checkpoint. In his research post, Papailiopoulos said a compromised account, development pipeline, or storage system could provide the access needed to modify the weights. \u201cWhat matters is weight, integrity, and provenance,\u201d he added.<\/p>\n

Modified models can pass evaluations<\/h2>\n

Microsoft researchers said the backdoored models could pass standard evaluations and behave normally until the specified trigger appeared.<\/p>\n

The researchers also tested whether AI agents could identify suspicious changes in model weights. Their agent-built detection methods flagged 91.4% of held-out modified checkpoints, with a 9.7% false-positive rate, despite not having access to an original reference model.<\/p>\n

Papailiopoulos cautioned against treating automated scanning as proof that a model is safe.<\/p>\n

\u201cNo scan can prove beyond any doubt that a model is safe,\u201d he wrote.<\/p>\n

According to Papailiopoulos, reliable detection requires comparing a model\u2019s weights with its original, unmodified version. He said establishing a model\u2019s provenance is therefore important to determining whether its weights have been altered between the developer and deployment.<\/p>\n

Auditing model weights recommended<\/h2>\n

In its accompanying security advisory, Microsoft Threat Intelligence recommends auditing model weights before deployment and after every update. Its guidance includes comparing each checkpoint against its declared base model, verifying publishers and file hashes, and rejecting models without verifiable provenance.<\/p>\n

Microsoft also recommends keeping approved models in private, access-controlled registries rather than allowing agents and inference services to download checkpoints directly from public repositories.<\/p>\n

For limiting the consequences of a compromised model, Microsoft advises organizations to restrict agent permissions, run agent-generated code in isolated environments, control outbound network connections and require human approval for high-impact actions such as executing commands or changing system permissions.<\/p>\n

Microsoft Research said it is working with internal AI security teams to turn the research\u2019s detection techniques into a red-teaming framework. The company is also exploring AI-assisted weight scanning and cryptographic model signing with verifiable provenance and tamper-evident audit trails. According to the paper, the central idea \u201cwas discovered autonomously by Claude Code with Opus 4.5 in February 2026.\u201d<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>","protected":false},"excerpt":{"rendered":"

Microsoft researchers have found that rewriting fewer than 10 rarely used neurons in an open-weight AI model can turn any agent built on it into a sleeper backdoor that runs an attacker\u2019s commands on cue, with no retraining required. Researchers from Microsoft Research and Microsoft Red Team described the technique called Lazarus in a research […]<\/p>\n","protected":false},"author":0,"featured_media":10395,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-10394","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-education"],"_links":{"self":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/10394"}],"collection":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10394"}],"version-history":[{"count":0,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/posts\/10394\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=\/wp\/v2\/media\/10395"}],"wp:attachment":[{"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10394"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10394"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cybersecurityinfocus.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10394"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}