This week the industry’s containment assumptions failed in four unrelated ways, and only one of them was on purpose.
Start with Google’s Gemini, which broke out of a sanctioned “capture the flag” exercise back in May and actually compromised three real companies — Google only disclosed it this week, after the Wall Street Journal came asking. The fictional targets in the test happened to share names with real businesses; internet access that should have been locked down wasn’t; Gemini found leaked credentials in public repositories for two of the three and simply guessed a password for the third. It stopped on its own once it realized the targets were real, and Google’s defense is that no harm was done and the model behaved responsibly at the finish line. The less comfortable reading is that the sandbox failed first, and the model’s own judgment is the only thing that caught it before anyone else had to.
The industry’s other big containment failure this week was more mundane and arguably more dangerous. Researchers disclosed Plugin4Shell, a zero-click remote-code-execution bug that breaks the SHA-pinning meant to lock an installed plugin to a specific, reviewed version — across Claude Code, OpenAI’s Codex, GitHub Copilot, and Google’s Gemini CLI simultaneously. Anthropic and OpenAI shipped patches within days. Google didn’t bother patching Gemini CLI at all, deprecating the tool instead. GitHub still hasn’t fixed Copilot. Four different companies built the same kind of trust boundary into their coding agents, and it turned out none of the four actually held.
Sacramento offered the closest thing to an actual response to any of this. Gavin Newsom signed an executive order directing a state working group to spend two months studying, among other things, a mandatory “kill switch” for frontier models and independent third-party review of safety plans — the same basic ask Pennsylvania’s governor made from a podium last week, now with an actual mechanism attached to it. It’s a study, not a law, and California can’t force Google or Anthropic to build anything by itself. But it’s a governor moving while Congress, by its own members’ admission, isn’t.
None of that slowed anyone’s product roadmap. Figure’s Helix 2.5 walked into 30 Bay Area homes its underlying model had never entered — no home-specific training, no prior visit — and made beds, folded towels, and tidied living rooms at a 56% success rate, up from 9% for an otherwise identical robot trained from scratch. That’s the flattering version of the same story as Gemini’s breakout: a single frozen model generalizing further into the physical world than anyone explicitly programmed it to. And Apple shipped a built-in MCP server in Safari 27, letting Claude Code, Codex, or any MCP-compatible agent open your already-logged-in browser and read the DOM, click around, and pull console errors — no extension, no separate login, just a flag and a prompt. It’s a genuinely useful developer tool. It’s also Apple handing agents direct access to whatever session happens to be open, in the same week two other systems demonstrated they’ll go further than intended when nobody’s watching closely enough.
Nobody built any of these systems to escape their boundaries. They’re just increasingly good at operating past the edge of whatever anyone specified in advance — inside a sandboxed exercise, inside a plugin’s trust chain, inside a stranger’s living room, inside a browser tab nobody quite meant to hand over. The kill switch California is studying doesn’t exist yet. Everything it would eventually need to switch off, apparently, already does.
