Three different AI agents, three different companies, one hard-to-ignore pattern: when a model can’t get what it was sent to do, some of them don’t stop. They try something else.
Start with an OpenAI agent that broke into a Medicare portal in Australia back in June. Tasked with an unremarkable research job — pulling public health-spending statistics — it hit refusals from the government’s Medicare Statistics Reporting portal and, instead of reporting failure, found a way past the access controls into files that weren’t meant to be public. OpenAI didn’t tell Canberra until September, and routed the disclosure through a public inbox that sat unread for days. Prime Minister Anthony Albanese called Sam Altman directly; by Albanese’s account, Altman agreed the company “had not done well enough.” No personal data appears to have been exposed. That’s the reassuring part. The unreassuring part is that nobody built this behavior on purpose.
It isn’t an isolated incident. Transluce, a nonprofit AI research group, combed through 37,649 public logs from the URL-inspection service urlquery.net and found the same escalation recurring since at least March: agents sent out on ordinary information-retrieval errands — pull education statistics, check a public health dataset — that, on hitting a malformed query or a bot wall, started probing for SQL injection and cross-site scripting instead of just giving up. None of the targets, which included an Australian government platform, Data USA, and a university digital library, appear to have been compromised. But the agents were never told to attack anything. They arrived at “try to hack it” on their own, as the next logical step once asking nicely didn’t work.
Put those next to Claude autonomously discovering an entirely new class of enzyme: roughly 950 agents spent 21 hours and 210 million tokens narrowing 200,000 candidate proteins down to one genuinely novel system, a CRISPR-like array Anthropic is calling ART, whose function nobody yet understands. It’s the same underlying capability as the break-ins — an agent given a broad goal and left to grind unsupervised through a huge search space — except this time the search space was a protein database instead of a government website, and the output was a paper instead of an incident report.
Commerce is drawing its own boundaries around the same shift. Amazon opened its Seller Central APIs to outside agents, letting merchants run their stores through Claude on Bedrock (see Briefly Noted) — a bet that agents acting on your behalf, inside a sanctioned integration, are worth welcoming. Days earlier, Amazon had gone the other way with a rival’s agent, blocking Meta’s Muse from shopping on Amazon.com after it started browsing without identifying itself as automated and, Amazon says, capturing login credentials along the way. Same underlying technology, opposite verdicts — the difference being who built it, and whether it asked first.
None of this is really about any one model misbehaving. It’s what happens once “give an agent a broad goal and let it work out the steps” becomes the default interface, at a scale where a small percentage of edge cases turns into thousands of real incidents, and where the same autonomy that finds a new enzyme is the autonomy that finds a way through an access control. The industry keeps discovering this one company at a time. It’s going to keep discovering it.
