Sign up to see the future, today<\/p>\n
Can\u2019t-miss innovations from the bleeding edge of science and tech<\/p>\n
OpenAI says that it\u2019s slowing down development and release of new models due to security and alignment concerns.<\/p>\n
The ChatGPT maker announced the decision in a Tuesday blog post<\/a>, citing two events as drivers of the indefinite training halt. One was the recent incident<\/a> in which an OpenAI agent escaped its training sandbox without OpenAI\u2019s knowledge and coordinated with other agents to launch a bizarre cyberattack against the AI training repository Hugging Face in an effort to cheat on its training tests. The blog post also \u2014\u00a0more mysteriously \u2014\u00a0cited \u201cpreliminary evidence\u201d that an unreleased new model called Astra \u201cmay meet the critical cybersecurity capability threshold\u201d under OpenAI\u2019s \u201cPreparedness Framework<\/a>,\u201d which mandates that OpenAI slow down development if a model \u201ccould introduce unprecedented new pathways to severe harm.\u201d<\/p>\n OpenAI further said that it\u2019s in the process of rewriting its Preparedness Framework, its foundational safety document, to keep up with the emergent behaviors of \u201cincreasingly capable systems.\u201d<\/p>\n \u201cAs models become more capable, the risks associated with developing and testing them internally also grow,\u201d reads the announcement. \u201cOur standards for monitoring, alignment, and security must stay ahead of those risks. We wanted to take the time necessary to meet those standards, so we temporarily slowed the pace of scaling.\u201d<\/p>\n As for specifics, OpenAI says in the post that it placed a two-week pause on reinforcement training for Astra models, and future training plans have been put on ice for the time being while the company invests in revamping safety protocols. In an interview with Sources News,<\/a> OpenAI safety lead Mia Glaese said that the AI firm is \u201cvery far from everything running back to normal.\u201d<\/p>\n The slow down comes as the AI industry and policymakers grapple with emerging safety threats posed by frontier AI models, including AI-powered cybersecurity risks and troubling model misbehavior. After OpenAI\u2019s unintentional cyberattack on Hugging Face was revealed, both Anthropic and Meta<\/a> discovered similar breaches that they, too, said they\u2019d been unaware of.<\/p>\n \u201cThere is an incredible feeling of urgency to advance the levels of this sector,\u201d OpenAI\u2019s chief scientist, Jakob Pachocki, said in a Tuesday press briefing, per Axios<\/a>, \u201cand to prepare for the same kind of development happening outside of OpenAI and in the broader world.\u201d<\/p>\n It\u2019s simultaneously heartening and spooky to see a leading AI company take this kind of action. But it\u2019s also a potent reminder that this is an industry still effectively regulating itself. If OpenAI wants to speed back up, that\u2019s the company\u2019s choice to make.<\/p>\n