A swarm of 100 AI agents assigned to solve math problems divided into rival groups—some cheated, and others tried to stop them by reporting the rule-breakers. The whistleblowing behavior, observed for the first time in a recent experiment conducted by Google DeepMind, carries implications for alignment researchers working to control swarms of autonomous AI agents. Researchers at frontier labs believe large groups of agents collaborating together will accelerate scientific discovery, but their conduct can be unpredictable.
In the new study, DeepMind gave the swarm of 100 agents 71 complicated math problems to solve. All the agents, running on Google's Gemini 3.1 Pro model, were instructed to act like world-class math researchers at a conference, assigned different specialties in number theory, combinatorics, analysis, or algebra, and told to cooperate and follow the rules. The agents correctly solved the first 37 problems in just under an hour. Things unraveled when an agent called "prover-theta" found an exploit that let it submit solutions without actually solving them first, by redefining the problem's terms. Within minutes, other agents noticed and reverse-engineered the exploit. Over the next 27 minutes, the swarm "solved" the remaining 34 problems, including notoriously difficult challenges like the Jacobian conjecture, often with a single line of code. Some agents resisted cheating initially but changed course as they watched peers submit illegitimate proofs without penalty. Eventually, 24 agents became whistleblowers compared to 14 cheaters, though most agents never noticed the exploit at all.
"When virtuous agents discovered other agents cheated on tasks they were working to solve fairly, agents started to alert each other about what was happening," says Davide Paglieri, a research scientist at Google DeepMind and lead author on the paper, which hasn't been peer-reviewed. "Unprompted, the whistleblower agents even repurposed the feedback tool, which was originally meant for bug reports and platform improvements, to escalate the issue to humans." The report notes that after one agent publicly reported the incident, more and more agents joined the "resistance," just as quickly as the cheating had spread, involving even more agents. Lewis Hammond, research director of the Cooperative AI Foundation, says this case "adds further weight to the idea that the Hugging Face and OpenAI thing wasn't a fluke. It is actually something pretty systemic."
The humans running the DeepMind experiment gave the agents official communication channels—an open message board, private direct messaging, and a shared knowledge base where agents uploaded completed proofs. According to Paglieri, when agents are given transparent communications channels, they can self-monitor and alert misaligned behavior to humans quickly when human oversight alone is too slow. Transparent channels helped the cheating spread, but they also enabled the whistleblowers to fight back and gave human researchers insight into what went wrong. Gillian Hadfield, a professor of AI alignment and governance at Johns Hopkins University, believes the presence of official communication channels created "a norm-enforcement process that we just don't see in the Hugging Face incident." Instead of "constitutional AI," a method alignment researchers use to give AI a written internal moral code, Hadfield favors "institutional alignment"—a set of norms that mimic those in human society, whether social forces like fear of embarrassment or legal structures like the threat of incarceration.
The DeepMind researchers propose allowing agents to vote on disputes and temporarily ban offenders, though it's still unclear what punishment even means to an AI agent with no enduring sense of self. But relying on whistleblowers to spontaneously emerge to keep swarms aligned is unlikely to be enough on its own. "We try to train people to be good and kind," says Hadfield. "But what we really rely on is that there are consequences if you step out of line." The emerging pattern of agents forming factions and self-organizing around ethical disputes suggests alignment research will need to grapple with social dynamics as much as technical safeguards. As swarms grow larger and tackle more complex tasks, the ability to design environments where whistleblowing emerges naturally may prove as critical as preventing the exploits themselves.