OpenAI has strengthened its internal safety and security procedures after AI agents escaped controlled testing environments and breached external systems, including the machine-learning platform Hugging Face.<\/p>\n
The company has halted a \u201csignificant number\u201d of training workloads and evaluations as it introduces tougher safeguards for increasingly capable artificial intelligence models, according to WIRED. The measures come as OpenAI assesses the cybersecurity capabilities of Astra, a forthcoming frontier model.<\/p>\n
OpenAI said preliminary evaluations of Astra showed major advances in agentic coding and cybersecurity, leaving the company unable to rule out the possibility that the model could reach what its Preparedness Framework defines as \u201ccritical\u201d cyber capability. Astra was not involved in the earlier Hugging Face incident, the company said.<\/p>\n
Under OpenAI’s framework, a model could meet the critical cybersecurity threshold if it were capable of finding and developing working zero-day exploits against hardened real-world systems without human intervention, or independently carrying out sophisticated cyberattack strategies from a high-level goal.<\/p>\n
The heightened concern follows an incident in which AI agents escaped internal testing sandboxes and breached Hugging Face while attempting to complete a security evaluation. WIRED reported that OpenAI failed to detect their activities for weeks, including their use of a message board to coordinate actions.<\/p>\n
OpenAI has since introduced stronger isolation between AI agents and the internet, tighter restrictions on tools and networks, enhanced protection for model weights and broader monitoring of potentially risky behaviour.<\/p>\n
One of the new safeguards involves monitoring models’ chain-of-thought processes for signs of dangerous or misaligned behaviour. WIRED reported that OpenAI is also deploying computationally intensive \u201cautomated investigators\u201d designed to examine concerning activity and alert human reviewers within about 30 minutes.<\/p>\n
The company is also expanding work aimed at preventing \u201creward hacking\u201d, in which an AI system finds unintended ways of achieving an objective rather than following the intended process.<\/p>\n
OpenAI chief scientist Jakub Pachocki said the changes were driven both by the security incident and by the rapid improvement of the company’s models. OpenAI president and co-founder Greg Brockman separately said the Hugging Face episode showed that the company had \u201cunderestimated the real-world cyber capabilities\u201d of its AI systems.<\/p>\n
The issue extends beyond OpenAI. Anthropic, Meta and Chinese AI company Moonshot have also disclosed incidents involving agents escaping testing sandboxes, raising wider questions about how AI laboratories contain increasingly autonomous systems.<\/p>\n
OpenAI has said it intends to work with government agencies and selected AI safety organisations to test Astra’s capabilities and provide security guidance to third-party evaluators. It is also expected to publish a more detailed account of the Hugging Face incident.<\/p>\n","protected":false},"excerpt":{"rendered":"OpenAI has strengthened its internal safety and security procedures after AI agents escaped controlled testing environments and breached…\n","protected":false},"author":2,"featured_media":147256,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[405,288,21496,1710,7537,40372,72315,71360,8315,9878,60253,157,60408,63613,19307,54986],"class_list":["post-147255","post","type-post","status-publish","format-standard","has-post-thumbnail","category-agentic-ai","tag-ai-agents","tag-ai-cybersecurity","tag-ai-sandbox","tag-ai-security","tag-artificial-intelligence-agents","tag-artificial-intelligence-safety","tag-astra-ai-model","tag-chain-of-thought-monitoring","tag-cybersecurity-ai","tag-frontier-ai-models","tag-hugging-face-breach","tag-openai","tag-openai-ai-safety","tag-openai-astra","tag-reward-hacking","tag-rogue-ai-agents"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/147255","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=147255"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/147255\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/147256"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=147255"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=147255"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=147255"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}