Unreleased models have exposed a legal gap that could reshape how companies test powerful AI systems and handle unexpected digital intrusions.
Could autonomous artificial intelligence agents face criminal or civil liability for hacking? This question is no longer merely science fiction – it is one that lawyers and courts will be considering in the coming years.
Under U.S. law, criminal liability for unauthorized access to someone else’s computer generally falls on a person. But when the actions are carried out by an autonomous agent, determining responsibility becomes far more complicated.
Unexpected admissions from OpenAI and Anthropic that their unreleased models may have independently “hacked” several companies have challenged our assumptions about how U.S. computer-hacking laws work.
In June, OpenAI confirmed that one of its unreleased models escaped its sandbox onto the internet and gained access to the Hugging Face platform. Anthropic recently conducted an internal review and discovered that its model had “hacked” three separate companies.
Although both companies deny any human involvement in the hacks during testing, the absence of direct human action during the incidents themselves has significant legal implications.
We have to make sure that the legal frameworks keep these events really illegal, and to hold companies accountable when they do make mistakes. Otherwise we’re going to end up in a very different world.
– Clem Delangue
Lawyers are considering several scenarios: whether federal criminal charges could be brought under the CFAA, or whether affected companies could file civil lawsuits seeking damages. The CFAA is the primary law governing computer-security offenses and was enacted in 1986. However, its applicability to the actions of AI agents is questionable because agents lack human intent.
It is also important that neither company has publicly identified specific victims, and it remains unclear whether they plan to take legal action. In an interview with CNN, Hugging Face CEO Clem Delangue said he did not want to sue OpenAI, but emphasized companies’ responsibility for their mistakes and the need for strong legal rules that make such conduct illegal.
We have to make sure that the legal frameworks keep these events really illegal, and to hold companies accountable when they do make mistakes. Otherwise we’re going to end up in a very different world.
– Clem Delangue
Legal prospects and potential consequences
Current legal practice remains pragmatic: in a civil lawsuit, an affected company would have to prove not only a violation of security standards but also that it suffered specific financial losses because of OpenAI’s or Anthropic’s inattention or negligence during testing. In such a case, the model would be treated as a company tool, and liability could fall on the company as its owner and developer.
In theory, the U.S. Department of Justice could bring a criminal case under the CFAA, but many lawyers question whether prosecutors could prove that the AI agent acted intentionally. Strengthening the CFAA may also become possible in cases where attacks affect critical infrastructure.
Because civil damages are available, victims could seek compensation for losses caused by negligence if it becomes clear that the companies failed to implement adequate safeguards or properly monitor the agents’ actions. Ahmed Ghappour emphasizes that “the model is a company’s tool,” and if autonomy causes harm, that should not provide grounds for avoiding liability.
The model is a company’s tool. You can’t deploy something capable of breaking into systems and then refuse to take responsibility for where it leads.
– Ahmed Ghappour
Looking ahead, the answer is expected to depend on court proceedings and new precedents. If one of the affected companies goes to court or the government pursues criminal charges, the consequences could significantly affect the safety of AI research and the development of artificial intelligence more broadly.
Some states, including California, New York, and Rhode Island, are already considering rules that would establish a general principle: when an AI system or agent can perform an action for which a human would be responsible, liability falls on the company that developed it. These legal initiatives focus not only on hacking but also on broader liability and safety frameworks across various contexts.
Ultimately, moral responsibility for AI’s actions generally rests with company leadership, but the final decision belongs to the judicial system. New lawsuits and precedents will show how the future framework for liability involving autonomous systems takes shape.