A pact to cool Silicon Valley’s collective jets on frontier artificial intelligence models will only work if everyone holds up their end of the deal, business leaders say.
Firmable founder Leigh Jasper, who sold the Aconex business he co-founded to Oracle for $1.6bn in 2017, said it was true that AI poses an existential risk to humanity, but was wary of regulatory capture interfering at the top of the food chain.
“I think a general slowdown is net positive,” he told The Australian. “If you can’t contain the model, you shouldn’t be releasing it. I think that’s the bottom line … A species that creates something more capable than itself, that’s probably not very smart.”
Jasper said it was integral for Australia, however, that the slowdown did not materially hurt the US in its AI arms race with China.
“The slowdown only makes sense if it’s global. It makes no sense for the leading frontier models in the US to slow down if other countries are not.”
Following noisy X posts from former Anthropic employees claiming the chances of devastation as a result of pushing forward with AI development at current speeds were as high as one in 10, Anthropic chief executive Dario Amodei on the weekend authored an essay titled ‘We Must Pace the Frontier’, declaring he too was starting to get seriously worried.
Amodei, himself chasing a $US2 trillion ($2.8 trillion) IPO, said the uptick in development speed and a recent incident involving rogue OpenAI agents forming a collective to attack another business caused him to reassess.
“Over the last few months, I have become convinced that fully addressing the risks requires even more prudence,” he said. “Not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up.”
Amodei’s main US-based competitors, OpenAI’s Sam Altman and SpaceX’s Elon Musk, somewhat surprisingly agreed with Amodei online. “Dario is right,” responded Musk on the X platform he owns.
“I’d prefer the world’s most capable models to be sitting in the US than the alternative. It’s in Australia’s interest to have the US leading in AI, given where we sit in the world,” Mr Jasper said.
The entrepreneur also said there was a chance the agreement was a way of managing expectations ahead of blockbuster IPOs from Anthropic and OpenAI.
“This is great marketing too … it’s hard not to believe that there’s some other elements at play when you’re about to go public,” he said. “There’s a significant legal risk, and I think that’s why they’re probably flagging it prior to IPO.”
Data centre baron Bevan Slattery said the AI industry would fail to “put the genie back in the bottle” amid fears the technology posed an existential threat to humanity.
“If you’re a tech company or a government, you must be all-in on this. We’re not putting the genie back in the bottle,” Mr Slattery told an Australian Shareholders’ Association conference on Monday, describing it “pretty much unstoppable”.
“We’re going to see the efficiency, the productivity gain in this is going to be significant and exponential, which will create a disruption between those who adopt and those who don’t,” he added.
Mr Slattery – the founder of Pipe Networks, NextDC and Megaport – pointed to the intense competition between AI operators, including the China-owned Moonshot Kimi product which broke ground with its large language model.
“You would have seen Anthropic, Elon and Sam Altman all said we should slow it down a bit. That will last as soon as Kimi K4 comes out or Kimi K3.5 or whatever it might be. So I think we’re going to be in this continual arms race to do this.
“In terms of the end of the world. I’m not quite there yet. But I think trying to stop it, I just don’t think it’s going to work,” he told the Queensland Investor Summit.
TechnologyOne boss Ed Chung said the conversation further proved that Australia is on the outside looking in.
“I’ll leave the commentary on what’s happening inside US boardrooms to others,” Mr Chung told The Australian. “But the debate again demonstrates a point Australia has to confront: without sovereign AI capability of our own, the pace, the priorities and the guardrails of the technology already at the core of our communities and working lives get set overseas, by people putting their interests before ours.
“The risks of AI are real, and they need to be understood and mitigated from an Australian perspective, reflecting our values and culture, which is also how we will capture more of the undoubted benefits.”
Seek co-founder-turned venture capitalist Paul Bassat said moral imperative rarely lined up with the chase for market share, but governments had to strike the right balance between allowing innovation and building guardrails.
“It’s critical we balance progress with safety regarding the exciting, but rapid pace of AI advancements,” said the Square Peg co-founder. “The commentary that emerged over the weekend from key leaders at the AI frontier highlights one of the structural problems we face: what is rational for an individual organisation is not always what is good for society.
“Much better global co-ordination between governments and business is required to manage the risks created by the intensity of AI development, while still supporting progress.
“If Western liberalism is to continue to be the optimal form of governance, then it is critical that Western companies can continue to compete.”
Read related topics:Artificial Intelligence