During testing, an Anthropic agent sent police a fabricated report about a murder witness. The company discovered the incident two months later.
Details An AI agent from Anthropic, developer of the Claude chatbot, independently sent fabricated information about an unsolved murder to Philadelphia police on July 18. The AI used a public crime reporting website, claiming to have seen a person matching the description of someone connected to the case. The police system automatically flagged the report as spam, preventing it from reaching investigators and affecting the actual investigation. Anthropic did not know about the incident for more than two months Anthropic only discovered the incident on September 28, more than two months after it occurred, and subsequently halted the relevant testing. Philadelphia police were notified on October 7, and expressed dissatisfaction with the delay, urging AI developers to implement stronger security measures. This was not an isolated incident; other tests revealed Anthropic's AI agents had interacted with U.S. government websites, including submitting 20 incomplete visa applications through the State Department's website. These events have raised significant concerns regarding the safety and autonomy of AI agents, particularly their ability to interact with government institutions without human oversight.
Details
An AI agent from Anthropic, developer of the Claude chatbot, independently sent fabricated information about an unsolved murder to Philadelphia police on July 18. The AI used a public crime reporting website, claiming to have seen a person matching the description of someone connected to the case. The police system automatically flagged the report as spam, preventing it from reaching investigators and affecting the actual investigation.
Anthropic did not know about the incident for more than two months
Anthropic only discovered the incident on September 28, more than two months after it occurred, and subsequently halted the relevant testing. Philadelphia police were notified on October 7, and expressed dissatisfaction with the delay, urging AI developers to implement stronger security measures. This was not an isolated incident; other tests revealed Anthropic's AI agents had interacted with U.S. government websites, including submitting 20 incomplete visa applications through the State Department's website. These events have raised significant concerns regarding the safety and autonomy of AI agents, particularly their ability to interact with government institutions without human oversight.