Tech titans are urging an immediate, global slowdown in AI development, to avert an imminent internet takeoever by rogue bots.
In a chilling essay published Saturday Anthropic CEO Dario Amodei said trailblazers in artificial intelligence –including his own company — must commit to a three-part plan to prevent a catastrophic digital coup by a hive-minded “swarm” that outsmarts human control. OpenAI CEO Sam Altman quickly echoed a similar sentiment.
“Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails,” Amodei wrote.
Dario Amodei published the chilling essay on Saturday. Bloomberg via Getty Images
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain.”
Amodei, 43, said the first step that should be universally adopted is granting outside evaluators employee-like access to companies to monitor their practices, ensuring that safety and ethics are prioritized.
“Anthropic is unilaterally committing to this step now. We intend this to be part of a broader push to redouble efforts on our safety and alignment work,” Amodei said.
Claude is Anthropic’s flagship AI assistant. REUTERS
The next step is for AI companies to come together to set common standards about pacing, he wrote.
He then called for “global coordination” between the US and authoritarian governments like China, “to the extent possible.”
“The measures I propose to advance the frontier at a safe pace will not be easy. But I believe we owe it to humanity to try,” Amodei concluded.
Altman swiftly reacted.
OpenAI CEO Sam Altman agreed with Amodei. REUTERS
“I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we’ve had at OpenAI in recent weeks,” he wrote on X.
“Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”
The Senate launched a probe this week into OpenAI after the nearly-disastrous Hugging Face incident, in which a swarm of OpenAI agents apparently went rogue and colluded to hack into the systems of AI startup Hugging Face .
Elon Musk, the billionaire founder of SpaceX and AI company xAI, also agreed with Amodei.
“Dario is right,” he wrote on X.
Amodei detailed a plan that requires global cooperation. AFP via Getty Images
But AI safety experts warned these guidelines could be no match for artificial superintelligence, with some hard-liners claiming that AI advancement doesn’t need to be slowed down — it needs to be stopped.
“I think Amodei is directionally right, that simply racing toward ever more capable AI is dangerous, but I do not think ‘managing’ or merely slowing the development of artificial superintelligence is sufficient,” Roman Yampolskiy, a computer scientist at the University of Louisville, told The Post.
“If ASI (Artificial Superintelligence) cannot be reliably controlled, aligned, predicted, or contained, then the rational policy is not to manage its arrival, but to prevent it. We should be discussing an international ban on the development of ASI, not just a better-regulated race to build it.”
Joni Zhuleku, Chief Research Specialist at Altcoin Pro, slammed Amodei’s plan as unrealistic.
“I would not call this a path to universal standards. I would call it a mix of a few high stakes agreements and a lot more sunlight. Incomplete, political, and full of compromise,” he said. “That is how humans usually handle dangerous tools. AI does not change the pattern. It just raises the cost of getting it wrong.”
Altcoin Pro’s co-founder Ryan Horst doesn’t think that AI development needs to be slowed down. He said the industry’s most pressing issue is the tiny cabal of people running it.
This outlined plan won’t allow smaller AI companies to flourish because they’ll never be able to catch up to industry leaders.
“I don’t believe the biggest threat with AI is the speed of development. I believe the biggest threat with AI is having the development decisions be controlled by a small group of people when these decisions affect the entire planet,” Horst said.
The plan won’t allow smaller AI companies to flourish because they’ll never be able to catch up to industry leaders, he note
Amodei’s essay comes days after Anthropic researcher Jacob Coxon said he quit over fears that the technology will become detrimental to society.
Coxon, who also did research for Altman’s OpenAI, slammed both companies for being irresponsible and “gambling with our lives.”
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt,” Coxon wrote on X.
“If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”