I was offline last week in my home state of New Jersey, but when I got back to Silicon Valley, everyone was panicking again about how quickly artificial intelligence is advancing. While this is certainly not the first time I\u2019ve seen such concerns, it\u2019s the biggest anxiety attack I\u2019ve seen in years.<\/p>\n
More than 1,000 employees at OpenAI<\/a>, Anthropic<\/a>, and other AI labs signed a petition<\/a> earlier this week arguing the US should find a way to \u201cpace\u201d the AI race\u2014a diplomatic way of saying the industry should have the option to coordinate a temporary pause on AI development<\/a>, or slow things down if they get out of hand. OpenAI and Anthropic themselves ended up supporting the letter.<\/p>\n The petition arrived a week after OpenAI revealed it had caused an unprecedented cybersecurity incident in which one of its AI agents<\/a> hacked into Hugging Face\u2019s platform and several other services<\/a> during internal testing.<\/p>\n Around the same time, top Trump administration officials started freaking out about an impressive new Chinese open weight AI model called Kimi K3, which was allegedly distilled from Anthropic\u2019s Fable 5. In response, most of the tech industry<\/a>\u2014except Anthropic\u2014signed onto an open letter<\/a> from Nvidia asking the US government to protect open-weight AI models, arguing they\u2019re a necessary counterbalance to their closed counterparts.<\/p>\n These events might look like a whole bunch of disjointed chaos. But I think they are evidence of a larger worldview taking hold in Silicon Valley, where many tech insiders are increasingly worried about OpenAI and Anthropic\u2019s dominance. Researchers and investors I\u2019ve talked to recently have framed the AI industry as a two-horse race that doesn\u2019t seem to be slowing down, and that\u2019s a cause for concern.<\/p>\n But different groups have their own reasons to be worried. Some OpenAI and Anthropic staffers think their employers are behaving recklessly in their pursuit to take over the market, and that the AI industry may soon develop models that are too capable for current safety methods to contain. They have said as much in public statements attached to this week\u2019s Pacing the Frontier petition.<\/p>\n \u201cI’ve seen how the relentless pace of AI makes it hard for society to keep up and how it puts pressure on labs to cut corners on safety,\u201d Jeremy Hadfield, a research product manager at Anthropic, said in a statement attached to the petition.<\/p>\n For some AI employees, the Hugging Face debacle was a warning shot that demonstrated how OpenAI\u2019s efforts to mitigate the risks of its most capable AI technology are already falling short. Granted, the incident happened when OpenAI was testing an AI model on its ability to find software exploits, and the company had intentionally turned off safeguards designed to rein in its cybersecurity capabilities.<\/p>\n OpenAI said in its postmortem that these tests were conducted in a sandbox, but the model ultimately gained access to the open internet. Some experts previously told WIRED that OpenAI\u2019s security practices should have been more robust<\/a>.<\/p>\n