A company under attack<\/p>\n
Hugging Face is famous in the AI space. Its mission is to \u201cdemocratise good machine learning<\/a>\u201d by providing benchmark datasets, community collaboration tools, and robotic platforms. The company is valued at US$4.5 billion<\/a>. <\/p>\n On July 16, the company announced<\/a> it had been attacked, with a hacker obtaining unauthorised access to some internal datasets and credentials. It said the hacker was likely \u201can autonomous AI agent system\u201d due to the sophistication of the attack.<\/p>\n Five days later, Open AI announced<\/a> the attack had been driven by some of its models: GPT-5.6 Sol<\/a> and a yet-to-be released model. <\/p>\n The tech giant was conducting what are known as \u201cred teaming<\/a>\u201d exercises. These are essentially simulated cyber attacks that help identify the capabilities, risks and vulnerabilities of AI systems before they are publicly released. They are typically conducted within an isolated environment to ensure potentially dangerous systems do not escape and cause harm to real systems. <\/p>\n But in this case, the AI agent did escape \u2013 even though OpenAI had some guardrails in place to prevent this. <\/p>\n Hugging Face became a lucrative opportunity for the AI agent. It hosts ExploitGym, a benchmark that tests an AI agent\u2019s ability to exploit real-world systems. The AI decided to turn every stone upside down to obtain access. With persistence, it succeeded. <\/p>\n Hugging Face was confronted with a challenge when attempting to use external AI services to diagnose the problem. The guardrails around more advanced models such as GPT-5.6 Sol and Claude Fable 5 are intended to stop them being used for cyber attacks \u2013 but they can also stop the models being used for sophisticated cyber defence. <\/p>\n So Hugging Face resorted to using an open-source model, GLM5.2, developed by the Chinese company Z.AI, to counter the cyber attack<\/a>. <\/p>\n Hugging Face said GLM5.2 was an advantage because it was not exposed to the attack data. Both Hugging Face and Open AI are collaborating on forensic analysis, post-incident recovery and risk mitigation strategies.<\/p>\n More sophisticated threats are coming<\/p>\n A March 2025 study<\/a> by the United Kingdom\u2019s AI Security Institute showed the best AI could complete 80% of the steps needed to gain full control of a portion of an external system. Within four months, it reached 100%. <\/p>\n Z.AI\u2019s GLM5.2 was only released in June, with 744 billion internal variables, known in the world of AI as \u201cparameters\u201d. The fact that Hugging Face assessed, vetted and deployed it within four weeks should be an eye-opener for organisations with long acquisition cycles. <\/p>\n The connectivity we all enjoy today can equally be our greatest threat. Cyber threats spread faster than human viruses and can create economic damage similar in magnitude to a country\u2019s GDP<\/a>.<\/p>\n More sophisticated cyber threats \u2013 the kind exemplified by the Hugging Face hack \u2013 will exploit the security layers that humans designed for human attackers, regardless of how sophisticated our designs are. <\/p>\n Indeed, in this particular case, even OpenAI\u2019s own understanding of its models couldn\u2019t predict or contain the rogue AI agent. This shows the need for all AI companies to urgently update and strengthen their guardrails, in order to help prevent a similar attack occurring with far more devastating consequences. <\/p>\n It is good to see Hugging Face and OpenAI collaborating on the investigation into the attack. This showcases the importance of putting aside market competition and blame when the situation demands.<\/p>\n An early warning<\/p>\n The fact that Hugging Face used Z.AI\u2019s open-source model to diagnose and counter the attack also shows the advantages of not relying on just a few pieces of tech. <\/p>\n States that are not in the game of developing their own AI models need to learn from this incident the value of being different. It is not too late to design new models that could save us in situations when the most advanced models fail \u2013 or, even worse, attack us.<\/p>\n Indeed, last week, another Chinese company, Moonshot AI, released Kimi K3. This model has 2.8 trillion parameters, its advanced performance stunning<\/a> the tech world. <\/p>\n It is no longer a question of \u201cif\u201d AI agents go rogue and attack us by themselves. The Hugging Face incident is an early warning that we must accelerate our preparedness. The threat is real and here. Hussein Abbass<\/a>, Professor, School of Systems and Computing, UNSW <\/a><\/p>\n This article is republished from The Conversation<\/a> under a Creative Commons license. Read the original article<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"An autonomous agent powered by OpenAI\u2019s advanced artificial intelligence (AI) models went rogue during a security test and…\n","protected":false},"author":2,"featured_media":117367,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[59587,405,7537,10189,26431,1651,52,26432,5371,26435,42598],"class_list":["post-117366","post","type-post","status-publish","format-standard","has-post-thumbnail","category-agentic-ai","tag-adfa","tag-ai-agents","tag-artificial-intelligence-agents","tag-business-law","tag-division-of-equity-diversity-inclusion","tag-opinion","tag-research","tag-resilient-society","tag-science-tech","tag-sdg","tag-sdg9"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/117366","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=117366"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/117366\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/117367"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=117366"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=117366"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=117366"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"The](\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/07\/1784865009_718_count.gif\")
<\/p>\n