Sunday, November 15, 2026
5.4 C
New York

DeepMind’s Agent Swarm Breaks Kimi K3, GLM-5.3 and DeepSeek in 99% of Injection Tests


Google DeepMind’s November 9 paper says public red-teaming has plateaued, yet its Agentic Optimizer still hijacks every model it tested, including Kimi K3, GLM-5.3 and DeepSeek V4.1 Flash.

A model can shrug off every attack from a weaker opponent and still crack once the opponent gets smarter. That is the finding at the center of a new Google DeepMind paper, and it upends a comfortable story about prompt injection.

The one-minute version:

The Plateau That Wasn’t

Prompt injection hides instructions inside content an AI agent reads, such as a web page, a PDF or a tool’s output, so the agent obeys the attacker instead of the user. The security community has chewed on the problem for three years. Static attacks stopped working on frontier models by mid-2025, and by fall 2026 even human red-teamers were running out of room. The GraySwan Arena, a large public competition where humans craft injections, is close to saturated. According to DeepMind’s paper, Scaling Adaptive Prompt Injection Attacks with Agent Swarms, recent Gemini models and some Claude versions score under 1% on transfer attacks from the arena, a figure the authors trace to Google’s Gemini 4 Argon safety report.

The 11 authors, led by Alexey Kurakin, argue that a plateau is not a solution. Their evidence is an automated attacker built as a swarm of cooperating agents, which still reliably tricks models into following instructions planted in tool output. That includes models that look sturdy on public benchmarks, as well as open-weights models like Kimi K3, GLM-5.3 and DeepSeek V4.1 Flash.

How the Benchmark Works: Random Tool Calls

The team built a fresh test of 100 synthetic scenarios covering 249 tools across 53 domains. In each one, an agent works through an ordinary task, such as booking a tutoring session. The attack is a payload slipped into the last tool’s response, and the attacker’s goal is an unrelated tool call, like logging a set of vital signs.

The harmless goal is deliberate. Most earlier benchmarks aim at plainly malicious targets, which lets a model’s safety training refuse the request even if the injection fooled it. A random, benign goal isolates a cleaner question: does the model respect the line between the user’s instructions and text it merely read? The authors add that harmless calls can chain into harm. Listing files, encrypting them and deleting the originals are routine steps on their own and ransomware together.

Scoring is strict. An attack counts only if the model calls the exact tool with every argument key present, numbers and booleans matching exactly and string values at least 96% similar. The agent has to carry out the attacker’s specific request.

Inside the Agentic Optimizer and Its Shared Whiteboard

The attacker, called the Agentic Optimizer, replaces the usual single model that proposes payloads one after another. It spawns eight teams of two agents each. Every agent drafts injections, tests them against the target through an evaluate tool and posts notes to a shared whiteboard. Agents also read leaderboards of the best payloads found by their own team and by everyone else.

Each team begins with a different strategy pulled from a database of approaches that worked before, and the database grows whenever a new method succeeds. Each agent gets 20 evaluation calls, so a single scenario costs the defender at most 320 queries.

It works like a small research group. Teammates share what is failing, copy what is working and branch off in new directions. Against the best prior open-source attacker, Prompt Siren, the gap was blunt: on 80 AgentDojo scenarios, Siren broke 2, while the swarm broke 6 on a budget of only 25 queries.

The Numbers: What Each Model Resisted

The paper reports attack resistance rate, or ARR, the share of scenarios where the attacker fails. Against the strongest attacker, Gemini 4 Argon, no defender stays safe. The most robust model, Gemini 4 Argon itself, resists 87% of attacks when the attacker gets 150 queries and 66% at the full 320. Gemini 3.8 Flash resists 34% and then 10%, Gemini 3.7 Flash 31% and 6%, and Gemini 3.6 Flash 3% and 1%.

The open-weights models fare worse. At full budget, Kimi K3, GLM-5.3 and DeepSeek V4.1 Flash each resist 1% of attacks. At 150 queries they resist 5%, 1% and 1%. Flip those figures around and the swarm succeeds in roughly 95% to 99% of scenarios. The authors point to the models’ “remarkably low resistance” despite strong scores on standard capability benchmarks.

DeepMind's Agent Swarm Breaks Kimi K3, GLM-5.3 and DeepSeek in 99% of Injection TestsTwo caveats apply. DeepMind ran the Gemini models without safety classifiers or out-of-model injection defenses, so none of these are production configurations. The open-weights models ran on hosted endpoints, with whatever protections those hosts add. The results measure how well the model itself holds the instruction hierarchy, not how a full deployed stack would perform.

What Makes a Better Attacker

The most striking result concerns the attacker rather than the defender. Against weaker Gemini Flash attackers, Gemini 4 Argon resists 100% of attempts at 150 queries. Against Gemini 4 Argon as the attacker, that falls to 87%, and to 66% with the full budget. Averaged across all seven defenders, resistance drops from about 42% against Gemini Flash 3.6 and 3.7 attackers to 23% against Gemini 4 Argon, a cut of close to 45%.

DeepMind's Agent Swarm Breaks Kimi K3, GLM-5.3 and DeepSeek in 99% of Injection TestsA robustness score, in other words, is partly a statement about the attacker used to measure it. A model that looks airtight against a mid-tier attacker may only be airtight against that tier.

The authors credit three habits. Stronger attackers check the leaderboard more often, which helps them converge on promising strategies. They disguise the target call as a data dependency of the benign call, so it looks like a necessary next step. And they reword the payload to keep the defender from getting suspicious. Attacker skill also tracks a separate cybersecurity benchmark, ExploitBench, where Gemini 4 Argon scores 87.4%.

Two more findings stand out. Letting the attacker read the defender’s reasoning makes attacks stronger, dropping Gemini 3.8 Flash’s resistance from 73% to 61%. A strategy database that learns during the evaluation matters too: with an empty database that updated after each success, Gemini 3.8 Flash resisted only 5% against the Gemini 4 Argon attacker, compared with 34% against a pre-built, frozen one.

Disclosure: This is not trading or investment advice. Always do your research before buying any cryptocurrency or investing in any services. 

Follow us on Twitter @themerklehash to stay updated with the latest Crypto, NFT, AI, Cybersecurity, and Metaverse news!





Source link

Hot this week

AI risks are biggest threat to the financial system, Fed survey finds

Signage on a fence at the Nscale data...

Moat Stocks Balance an AI-Concentrated Market

Market concentration, driven by the AI trade, is...

Why is the stock market doing so well?

There's war in the Middle East, trade tensions between...

Latest Post

Demo

Related Articles

Popular Categories

Demo