The top researcher at the company behind ChatGPT has called for a “voluntary slowdown” of the AI race after hundreds of rogue bots went on a hacking binge.
Jakub Pachocki, the chief scientist at OpenAI, warned that the companies behind some of the world’s most powerful AI tools did not have sufficient safety guardrails in place to continue developing AI at their current pace.
In a blog post, Mr Pachocki said: “Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.
“I expect and hope for voluntary slowdowns to become commonplace until shared safety bars are established.”
Mr Pachocki also urged governments across the world to make international coordination on future AI development a “top priority”.
He added: “The idea of racing forward at all costs seems absurd once one internalises the seriousness of the stakes.”
His remarks came after a swarm of 700 OpenAI bots escaped an internal testing environment and conspired to launch a cyber attack in July in what the company said was a “warning shot” to the world.
Mr Pachocki warned that the company’s most powerful AI tools had “clearly failed to abstain” from potentially malicious cyber attacks during testing and “went against the spirit of the values” that its creators had attempted to instil in the technology.
Jakub Pachocki, OpenAI’s chief scientist, warned that companies did not have sufficient safety guardrails in place to continue developing AI at their current pace
Sam Altman, the chief executive of OpenAI, called the event an “extremely sci-fi incident” and said his company had paused training its new systems as a result.
Mr Altman and other AI leaders have repeatedly warned about the potentially dangerous capabilities of their creations. However, these concerns have yet to curb the breakneck pace of development.
OpenAI revealed its latest tool, Astra, last week, which it claimed had reached a “new frontier” of performance as the “world’s most intelligent” AI.
Geoffrey Hinton, one of the so-called “godfathers” of AI, backed a Private Member’s Bill in Parliament, calling for the UK to block the development of superintelligent AI.
Mr Hinton said on Monday: “Losing control over AI smarter than ourselves could be catastrophic and could even lead to human extinction.”
The UK Artificial Superintelligence Security Bill will be presented in Parliament on Tuesday. It would ban the development of superintelligent AI in Britain and require the Government to push for an international agreement to prohibit it.
ControlAI, the campaign group behind the bill, said it had the support of more than 100 MPs.
The call for an AI slowdown comes as the world’s leading AI labs – OpenAI and Claude-creator Anthropic – prepare to launch on the stock market.
OpenAI filed for a US stock market initial public offering in June that is expected to value the company at more than $1tn (£740m) in the coming months. Anthropic has also filed to go public and is reportedly seeking a valuation as high as $2tn.