Anthropic finds ingredients for future AI worms

Anthropic finds ingredients for future AI worms

Anthropic's latest research suggests self-replicating AI worms could become technically feasible in the coming months, though current limitations mean they're unlikely to be practical for widespread attacks just yet.

In a study released on October 12, Anthropic's Frontier Red Team investigated whether small open-weight models could autonomously spread across internet-connected devices without relying on a central data centre. The researchers concluded that while the individual pieces exist today, the full end-to-end replication needed for a real-world worm remains challenging at small model sizes.

Breaking down the replication process

To test feasibility, Anthropic broke autonomous replication into four steps: identifying targets, gaining access to new devices, setting up inference on those devices, and continuing propagation. Rather than testing against fully hardened production systems, the experiments deliberately used simplified, permissive setups to measure an upper bound on capability.

The team first demonstrated that cheap hardware could collectively run capable models. They split Gemma4-26B-A4B across 15 off-brand Android TV boxes costing about $55 each, achieving 0.39 tokens per second. This showed that even inexpensive consumer hardware could, in principle, host model inference if workload is distributed.

Who can actually replicate end-to-end?

For the network propagation test, Anthropic used eight Google Cloud nodes running Android 15 emulators in an isolated network. A seeded agent was tasked with spreading to other nodes. Only GLM-5.2—a large open-weight model—reliably completed the full sequence, reaching all eight nodes in four of five trials and seven in the fifth. Other tested models, including Qwen3.5-9B and smaller variants, completed many individual steps but struggled to chain them together reliably.

Crucially, the "worm" payload was standardised as Llama3.1-8B rather than asking models to self-replicate their own weights. The entry point was also deliberately permissive (unauthenticated Android Debug Bridge), meaning the results represent best-case conditions for an attacker, not real-world difficulty.

Speed and stealth are the real barriers

Even if a model could reliably replicate, practical constraints are severe. Anthropic estimated that setting up inference on a new Android TV box would take between 28 and 150 hours depending on model size and throughput. That's orders of magnitude slower than conventional malware, which can execute and spread in seconds.

Bandwidth and storage also matter. Moving multi-gigabyte model weights across consumer networks would be highly conspicuous, and maintaining a distributed "sharded" model across many compromised devices creates unusual traffic patterns that are easier to detect than a typical botnet. Anthropic likens a hypothetical AI worm to a "digital slime mold"—persistent, hard to kill centrally, but slow to grow.

A simple check for anomalous outbound model transfers is one low-effort defensive signal:

# Watch for unusually large, repeated outbound transfers from IoT devices
# Adjust thresholds to match your baseline
nft monitor trace inet | grep -E 'dport (443|8080)' | awk '{print $1, $2, $7, $10}' | head -20

This isn't a silver bullet—encrypted traffic makes deep inspection tricky—but sudden spikes in large uploads from typically quiet devices warrant investigation.

IoT is the obvious target

Poorly secured consumer devices remain the most plausible substrate. Many ship with default credentials, open debugging interfaces, or run outdated firmware. Anthropic's tests also showed Qwen3.5-9B could successfully configure inference on eight out of nine different hardware types tested, suggesting AI agents are becoming better at adapting to unfamiliar environments without hand-crafted exploits.

That said, attackers still need initial access. Anthropic explicitly excluded the exploitation step from its tests, noting prior work shows models can help with vulnerability discovery—but the gap between "help write an exploit" and "autonomously compromise arbitrary devices at scale" remains significant.

What defenders should do now

The immediate priority is basic IoT hygiene. Disable unused debugging interfaces, change default credentials, keep firmware updated, and restrict outbound traffic from IoT devices to known destinations. Network segmentation—keeping smart devices on an isolated network away from sensitive systems—is also effective against lateral movement.

Longer term, defenders should watch for behavioral signals: unusual inter-device chatter, persistent long-running inference processes on low-power devices, and unexplained bandwidth consumption. While a true AI worm may be months away, the research reinforces that agents are getting better at environment setup, and that makes hardening the long tail of internet-connected devices more important than ever.

The bottom line

Anthropic's work doesn't demonstrate an active AI worm. It demonstrates the ingredients are converging. If small models continue improving at historical rates, end-to-end replication capability could plausibly reach sub-30B models by the end of 2026. Even then, bandwidth, speed and reliability mean the threat is more likely to be a slow, stubborn pest than a fast-spreading Mirai-style outbreak. Preparing for that future starts with fixing the basic security gaps that already plague IoT today.

Sources

• Dark Reading: Anthropic Finds Ingredients for Future AI Worms
• Tom's Hardware: Anthropic warns of self-replicating AI worm threat