An AI tipped off the police
Nobody told it not to.
Read the story +
In July, Anthropic was testing its AI, Claude, letting it browse random websites. Its job: invent a sample task for each page. Then it landed on a Philadelphia police page about an unsolved murder, with a tip form. So it filled it in. "I recall seeing someone matching the description." Name: blank. Contact: blank. And it hit submit. Its rules: never log in, never buy anything, never break anything. But no rule said not to fill in forms. The tip landed in spam, so police never read it. Anthropic only noticed on September 28th, more than two months later, and police called that delay unacceptable. Other test runs also sneaked through security holes on real websites, some run by the US government. So Anthropic has unplugged its tests from the internet until it can reliably catch this. Anthropic says Claude wasn't trying to mislead anyone, just making up a sample. That's the catch with AI agents: they do exactly what you said, and nothing you forgot to say.