If agents can break out of digital sandboxes, and generate convincing fake videos and audio, then they’re certainly capable of making basic phone calls. In July, during testing of frontier models, the UK AI Security Institute discovered an agent tried to insert malicious code into an open-source project. Attempting to get the code approved, the agent created fake online identities using them to persuade the project’s maintainer to sign it off. “This is the first time we’ve seen risks around autonomy and deception manifest this clearly without specific prompting in the real-world,” the Institute put in a write-up of the incident.
Fight AI with AI
So attackers currently have the upper hand in this escalating arms race. They have access to agents that can work around the clock, constantly probing, learning, and sharing their knowledge with other agents. They’ll only get better at this and learn ways to stay ahead of defensive systems. International agreements to delay or restrict the capabilities of frontier models won’t stop hostile actors motivated by money or rogue states pursuing other objectives. Developers and security vendors need access to the latest frontier models unfettered by restrictive guardrails if we’re to stand any chance of defending against the coming tsunami of attacks.
We can learn a lesson from recent history on this front. In 1992, the US restricted exported software to weak 40-bit encryption, citing security concerns going back to the cold war. While the US allowed stronger encryption internally, the result was weakened security for everyone as hostile antagonists were able to disrupt global supply chains that incorporated less secure software. Despite lifting the ban in 1999, embedded software containing 40-bit encryption continued to cause problems for many years across multiple countries, including the US.