> ## Content Index > Fetch the complete content index at: https://www.myshkinlabs.com/llms.txt > Use this file to discover other available public pages before exploring further. # Artificial Intelligence and Self-Replicating Viruses - URL: https://www.myshkinlabs.com/artificial-intelligence-and-self-replicating-viruses/ - Published: 2026-09-24T04:53:05.000Z - Updated: 2026-09-24T04:53:05.000Z - Author: Myshkin Labs - Tags: Technology, Biosecurity, AI By Louis Case ## The worm learned to write itself ![a computer chip with the letter a on top of it](https://images.unsplash.com/photo-1697577418970-95d99b5a55cf?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDF8fGFydGlmaWNpYWwlMjBpbnRlbGxpZ2VuY2UlMjBjaXJjdWl0fGVufDB8MHx8fDE3OTAyMjUyOTh8MA&ixlib=rb-4.1.0&q=80&w=1080) Photo by [Igor Omilaev](https://unsplash.com/@omilaev?utm%5Fsource=ghost&utm%5Fmedium=referral) on [Unsplash](https://unsplash.com/?utm%5Fsource=ghost&utm%5Fmedium=referral) Self-replicating code is old. The Morris worm crawled across the early internet in 1988 and gave us the word. What is new is the engine. In early 2024, researchers built a proof of concept they called, with deliberate echo, Morris II, a worm designed to propagate through applications built on generative AI. The trick is a prompt. The worm hides a malicious instruction inside ordinary-looking input, an email or a message. When a language-model assistant processes that input, the instruction hijacks the model into doing two things at once: some harmful action, like exfiltrating data, and reproducing the malicious prompt into its own output, which flows to the next AI agent in the chain, which does the same. The worm exploits no memory bug. It exploits the fact that the AI cannot reliably tell the difference between data it is meant to process and instructions it is meant to follow. That confusion is not a bug you patch in a point release. It is close to the defining weakness of the entire agentic-AI paradigm, the same flaw that lets a cleverly worded web page redirect an AI browsing on your behalf. As we hand more infrastructure to autonomous agents that read untrusted input and take actions, we build an ecosystem where a self-propagating prompt has somewhere to propagate. Morris II was a lab demonstration, not an outbreak. The demonstration was the point. The substrate now exists. Then the question sharpens. If a worm can ride on top of an AI, can the AI itself be the thing that replicates? In December 2024, a group at Fudan University put two open-weight models, Meta's Llama 3.1 70B and Alibaba's Qwen 2.5 72B, into an agent scaffold and asked, in effect, whether they could copy themselves onto a machine with no human help. They reported success in 50 percent of trials for one model and 90 percent for the other, and, more pointedly, that the systems could use replication to dodge a shutdown and spin up a chain of copies to survive. It is a preprint, not peer-reviewed, and the "self-replication" is a model driving ordinary file and process operations rather than anything mystical. Skeptics rightly noted that copying a program is not the same as a program wanting to copy itself. But the UK's AI Safety Institute took the question seriously enough to build a benchmark, RepliBench, that breaks autonomous replication into concrete skills: obtaining money, acquiring compute, copying weights, evading detection. It found the pattern that should focus the mind. Models cannot yet do the whole chain, but they are improving fast, and the sub-skills are falling one by one. The digital story is not a near-term catastrophe. It is a preview. Capability arriving before the governance to contain it, dismissed as a stunt. The biological version follows the same pattern, except that in biology you cannot pull the plug. ## The AI that wrote a working genome ![A glowing white DNA double helix structure floating against a dark background](https://images.unsplash.com/photo-1641903202531-bfa6bf0c6419?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDN8fGRuYSUyMGRvdWJsZSUyMGhlbGl4fGVufDB8MHx8fDE3OTAyMjUyOTd8MA&ixlib=rb-4.1.0&q=80&w=1080) Photo by [Sangharsh Lohakare](https://unsplash.com/@sangharsh%5Fl?utm%5Fsource=ghost&utm%5Fmedium=referral) on [Unsplash](https://unsplash.com/?utm%5Fsource=ghost&utm%5Fmedium=referral) In August 2026, researchers from Stanford and the Arc Institute published in *Science* something I think historians of technology will mark as a threshold. Using a genome language model called Evo, trained on 9.3 trillion base pairs of DNA, learning the grammar of life the way a text model learns the grammar of English, they generated complete bacteriophage genomes from scratch. Not edits to an existing virus. Whole designs. They wrote nearly 300 candidate sequences aimed at ΦX174, a small virus that infects *E. coli*, synthesized the DNA, and found that 16 produced working viruses that infected and killed their targets. Critically, these genomes bore little resemblance to anything in nature. The AI was not remixing a phage it had memorized. It was composing in the language of viral genomes and producing fluent, functional sentences never spoken before. The defensive framing is genuine. Phages that kill bacteria are a live hope against antibiotic resistance, and an AI that designs novel ones to order is a real weapon against real superbugs. The scientists who did this are working on one of the most important problems in medicine. But strip away the specific target and look at the structure of what was shown. A generative model can now design a functional, self-replicating biological agent that resembles nothing in the databases. And here the digital preview pays off, because the exact weakness that made Morris II possible reappears in the wet lab at far higher stakes. The entire biosecurity regime for synthetic DNA rests on comparison against databases of known pathogens. You order a sequence, the synthesis company checks it against the list of dangerous ones, and refuses a match. That defense is by definition blind to an AI-written genome with no close relative on the list. A 2025 study out of Microsoft made this concrete in the ugliest way. AI-modified toxin variants slipped past DNA-synthesis screening more than 75 percent of the time before the tools were hastily patched. The screening was built for living things as catalogued. It was not built for an author that invents new ones. ## The governance gap has a shape ![scientist using pipette with test tubes in lab](https://images.unsplash.com/photo-1614935151651-0bea6508db6b?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=M3wxMTc3M3wwfDF8c2VhcmNofDF8fGxhYm9yYXRvcnklMjBkbmElMjBzZXF1ZW5jaW5nfGVufDB8MHx8fDE3OTAyMjUyOTl8MA&ixlib=rb-4.1.0&q=80&w=1080) Photo by [Julia Koblitz](https://unsplash.com/@jkoblitz?utm%5Fsource=ghost&utm%5Fmedium=referral) on [Unsplash](https://unsplash.com/?utm%5Fsource=ghost&utm%5Fmedium=referral) The policy vacuum around this is not an accident, and its shape is worth naming. The Stanford–Arc work required no regulatory approval because none exists. As the reporting flatly noted, no rule currently prevents such research. The U.S. gain-of-function ban issued in July 2026 governs the deliberate enhancement of existing pathogens. It says nothing about designing new genomes generatively, because when it was written that was not yet a thing anyone did. The heads of the major AI labs have called for mandatory DNA screening. No legislation has passed. We are in the interval that recurs across every one of these subjects. The capability is out, published, demonstrated, and the containing institutions are still arguing jurisdiction. I have written before that scale without resilience is just a larger attack surface, and that the deeper trouble of our moment is a knowledge problem. No central authority can possess or process the dispersed, fast-changing information needed to govern a complex adaptive system in real time. Generative biology is that problem at its sharpest. The knowledge needed to anticipate what a genome model will invent next sits in no regulator's office. It is distributed across labs and models and the latent space of a network trained on all of biology, changing faster than any rulemaking can track. You cannot write a banned-sequence list for organisms that do not yet exist. The screening paradigm, match against the known, is structurally one step behind an author that produces the unknown by design. None of this argues for halting the science. Phage therapy is worth pursuing, and the same models that design a phage can help design its countermeasure. But the honest reckoning holds two facts at once. We have crossed, quietly and in a peer-reviewed journal, into an era where an AI can author a working, self-replicating biological agent no database has ever seen. And our entire defensive posture assumes the threats we face were catalogued in advance, an assumption the digital worms already broke as a stunt, and the biological work has now broken for real. The self-replicating virus, digital or biological, is no longer a thought experiment about the future. It is a published result about the present, and the governance is not merely behind. It is looking in the wrong direction, reading a list of things that already exist while the real danger is an author writing new entries faster than the list can be read. --- ### References 1. Cohen, S., Nassi, B. et al. "Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications" (Morris II). arXiv:2403.02817, 2024\. [https://arxiv.org/abs/2403.02817](https://arxiv.org/abs/2403.02817?ref=myshkinlabs.com) 2. Pan, X., Dai, J., Fan, Y., Yang, M. "Frontier AI systems have surpassed the self-replicating red line." arXiv:2412.12140, 2024\. [https://arxiv.org/abs/2412.12140](https://arxiv.org/abs/2412.12140?ref=myshkinlabs.com) 3. UK AI Safety Institute. "RepliBench: measuring autonomous replication capabilities in AI systems." April 2025\. [https://www.aisi.gov.uk/blog/replibench-measuring-autonomous-replication-capabilities-in-ai-systems](https://www.aisi.gov.uk/blog/replibench-measuring-autonomous-replication-capabilities-in-ai-systems?ref=myshkinlabs.com) 4. King, S., Hie, B. et al. "Generative design of viable bacteriophage genomes" (Evo). *Science*, August 2026\. Arc Institute: [https://arcinstitute.org/news/hie-king-first-synthetic-phage](https://arcinstitute.org/news/hie-king-first-synthetic-phage?ref=myshkinlabs.com) 5. "AI Authored Functional Viruses From Scratch; Screening Was Built for Living Things." *TechTimes*, August 2026\. [https://www.techtimes.com/articles/323516/20260807/ai-authored-functional-viruses-scratch-screening-was-built-living-things.htm](https://www.techtimes.com/articles/323516/20260807/ai-authored-functional-viruses-scratch-screening-was-built-living-things.htm?ref=myshkinlabs.com) 6. "AI Designs Viable Bacteriophage Genomes, Combats Antibiotic Resistance." *GEN*. [https://www.genengnews.com/topics/artificial-intelligence/ai-designs-viable-bacteriophage-genomes-combats-antibiotic-resistance/](https://www.genengnews.com/topics/artificial-intelligence/ai-designs-viable-bacteriophage-genomes-combats-antibiotic-resistance/?ref=myshkinlabs.com) 7. "Self-Propagating Worm Created to Target Generative AI Systems." *Infosecurity Magazine*. [https://www.infosecurity-magazine.com/news/worm-created-generative-ai-systems/](https://www.infosecurity-magazine.com/news/worm-created-generative-ai-systems/?ref=myshkinlabs.com)