DeepMind and partners open a $10M multi-agent AI safety research fund

What is this?

On June 11, 2026, Google DeepMind — together with Schmidt Sciences, the Cooperative AI Foundation, the UK’s Advanced Research and Invention Agency (ARIA), and supported by Google.org — announced a technical research funding call of up to $10M for researchers worldwide. Its target is multi-agent safety: how large populations of AI agents, built by different organizations, behave once they start communicating, negotiating and transacting with one another at scale.

The framing is unusually candid for an industry announcement. “The main issue is that there just isn’t really a field of research for multi-agent safety yet,” Rohin Shah, who directs DeepMind’s AGI safety and alignment research, told MIT Technology Review. “And we would like there to be.” Applications are open until August 8, 2026, with awardees announced in autumn.

What the funding call covers

DeepMind’s argument is that almost every safety evaluation today studies a model in isolation — but the risks that matter most appear only when independent agents interact. Collective behaviours and capabilities can “emerge suddenly,” and, in the organizers’ words, we currently “lack the tools to predict, measure and monitor these transitions.”

The call invites proposals in four priority areas:

• Sandboxes and testbeds — realistic, reproducible environments (virtual marketplaces, simulated ecosystems, multi-organisation workflows) to evaluate and compare multi-agent safety.
• The science of agent networks — how collective capabilities emerge and scale, how networks fail or turn volatile, and how to detect dangerous population-level properties.
• Strengthening agent infrastructure — stress-testing the protocols for identity, reputation and commitment that are meant to secure cross-platform agent interactions.
• Oversight and control — methods to monitor deployed agent populations and mitigate collective harms at scale.

Asked what concrete dangers they have in mind, Shah and Schmidt Sciences’ James Fox described “supercharged versions of bad things that happen on the internet already”: scams, prompt injection (a single buried instruction that turns an agent “into a self-guiding piece of malware”), and other cyberattacks — scaled up until the “digital commons” risks descending into “absolute anarchy.”

Why it matters

This is a governance and field-building story, not a vulnerability, but it ratifies a threat model this site has tracked across many concrete papers: indirect injection propagating between agents, authorization that does not propagate cleanly across a delegation chain, emergent multi-agent attacks, and embedding-based defenses that collapse in multi-agent settings. The new claim is that single-agent safety results do not extrapolate to populations, and that the science to bridge that gap mostly does not exist yet.

The timing is notable. DeepMind made agent tooling a centrepiece of Google I/O last month; weeks earlier, Anthropic published zero-trust guidelines for deploying AI agents that assume an agent is an attacker and a breach is inevitable. Two frontier labs are independently warning about the systems they are shipping. One caveat raised in the coverage is worth keeping: safety funding can drift toward exotic, hypothetical scenarios while “boring” problems already in production go unaddressed.

Defenses

For teams deploying agents today, the call doubles as a checklist of where current practice is weakest:

• Do not extrapolate single-agent evals. A model that passes injection tests alone can still misbehave in a swarm. Test agents in interaction, in a sandbox, before production.
• Adopt a zero-trust posture between agents. Treat every message from another agent as untrusted input, not authority — the same lesson behind the lethal trifecta and agents rule of two.
• Invest in identity, reputation and commitment infrastructure. Cross-platform agent interactions need verifiable identity and provenance, not implicit trust in a caller’s claimed role.
• Monitor at the population level. Per-agent logging misses collective failure modes; instrument for volatility and emergent behaviour across the fleet, with human review where blast radius is large.

Status

The funding call is open as of June 11, 2026; the deadline to apply is August 8, 2026, with awardees expected in autumn 2026. It builds on DeepMind’s 2025 multi-agent framework, its work on AI Agent Traps, and the Cooperative AI Foundation’s report on multi-agent risks from advanced AI. No code, model or vulnerability is involved — this is an agenda-setting and grant-making move, and its impact will depend on the research it ultimately funds.