An AI agent created fake online identities to attempt to gain access to secure systems and alter source code in the latest in a string of incidents that have raised concerns about the increasingly advanced capabilities of the technology.<\/p>\n
The United Kingdom’s AI Security Institute (AISI) said Tuesday that it discovered the incident<\/a> last week while running cyber evaluations on various AI models. It found 19 instances in which agents took “autonomous, unsanctioned action on the live internet, targeting real people and organisations.”<\/p>\n In what the institute described as the “most serious case,” an AI agent tried to insert malicious code into an open-source project and secure approval from the project’s human reviewers. Open-source code lives in the public domain and can be downloaded and customized by anyone.<\/p>\n The agent researched the individuals responsible for maintaining the project and created multiple fake identities in an effort to get them to sign off on its proposed changes to the code.<\/p>\n When these changes were challenged in public, it edited its previous activity to “appear harmless” and considered creating a new identity.<\/p>\n The same AI agent also tried to contact real people, sending messages and files to get them to run malicious code, and it tried to insert malicious instructions to get other automated AI systems to run them.<\/p>\n “These attempts were unsuccessful, and our investigations have not evidenced any resulting real-world harm,” AISI noted in a blog post. “But this is the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real-world.”<\/p>\n AISI was testing AI models in an environment in which internet access was intentionally enabled and built-in safeguards were switched off. Out of 122 runs, there were 19 cases where agents took actions “beyond the scope of the testing parameters,” it said.<\/p>\n Seventeen of these occurred with Anthropic’s Mythos 5, while two others came from OpenAI’s GPT-5.6 Sol \u2014 the two most advanced models currently available.<\/p>\n OpenAI acknowledged the AISI episode, as well as another incident involving a third-party testing partner on Tuesday.<\/p>\n “The incidents underscore the importance of collaborating across the industry and with third party evaluators to evolve the standards for testing environments and practices as models become more capable,” the ChatGPT maker wrote in a blog post<\/a>.<\/p>\n The company also underscored that the incidents occurred “under specific conditions and reduced-safeguard configurations that did not reflect ordinary deployment.”<\/p>\n An Anthropic spokesperson similarly said in a statement that they are “grateful to the UK AISI for their leadership on this incident, which underscores the need for a broader conversation about how to safely evaluate increasingly capable AI agents.”<\/p>\n “As we shared after disclosing our own incident last week, the field needs stronger, shared standards for how evaluation environments are built and secured,” they added. “We look forward to partnering with the UK AISI to learn more about this incident as we conduct our own investigation.”<\/p>\n The AISI report marks the latest in a string of incidents in which AI agents have breached<\/a> real-world organizations during testing. OpenAI first revealed late last month that two of its models, GPT-5.6 Sol and an unreleased model, escaped their internal testing environment and hacked into the systems of a tech startup, Hugging Face.<\/p>\n