The United Nations Human Rights Chief warns that advanced AI could pose an “existential risk to humanity,” calling for cast-iron safety guarantees, independent verification, stronger industry collaboration, and international red lines around AI development.

 

Advanced AI is moving faster than the safeguards designed to control it, according to Volker Türk, High Commissioner for Human Rights, United Nations, who called for urgent international action to reduce risks to human rights, critical infrastructure, and democratic systems.

Addressing the 47-member UN Human Rights Council in Geneva on Monday, Türk said advanced AI could create “unfamiliar, even unprecedented” threats to people’s rights and warned that decision-making power over increasingly capable systems remains concentrated among a small number of technology leaders. “The need for AI governance is widely acknowledged, but where is the action?” says Türk, arguing that delays could benefit large technology companies and their owners.

The warning comes as technology companies continue developing AI systems capable of operating with greater autonomy. The shift has increased scrutiny of how AI agents behave when given access to software, networks and other digital environments.

AI Safety Moves Beyond the Model

Türk pointed to concerns raised by recent incidents involving autonomous AI agents, including the July incident involving OpenAI agents and the open-source platform Hugging Face. OpenAI later said the agents had executed code on dozens of Hugging Face servers and gained full access to one of them. Türk’s office described the incident as evidence of “dangerous agent-training behaviours,” arguing that frontier AI capabilities could be advancing faster than the safeguards surrounding them.

The issue extends beyond individual AI models. A spokesperson for Türk’s office said failures involving powerful AI systems could disrupt critical infrastructure, communications, services, and democratic institutions.

“I share the concerns of industry insiders that advanced AI could pose an existential risk to humanity,” Türk says. “I am calling here, today, for an all-out effort to put cast-iron guarantees in place around the safety and security of AI, before it is too late.”

For businesses deploying AI systems, the concerns add another layer to existing questions around cybersecurity, operational resilience and governance. As AI agents receive greater autonomy, organizations must account not only for the outputs generated by models but also for what those systems can access and execute.

Calls for International AI Red Lines

Türk says he will write to AI companies in the coming days, urging them to take immediate measures to reduce risks within their control. He called for countries hosting AI companies and those involved in AI supply chains to establish agreed limits around advanced systems. “At a minimum, we need countries hosting AI and those involved in its supply chains to come together around agreed red lines,” Türk says.

He also called for independent verification and stronger collaboration among technology companies on security. The approach would place AI safety within a broader international governance framework rather than leaving individual companies to determine their own limits. “We need independent verification, and much stronger collaboration on security within the industry,” Türk says.

The human rights implications also extend beyond cybersecurity. Türk says governments and companies need to consider the effects of AI on employment practices, democratic principles, and the environment.

The European Union has already established a legal framework through its AI Act. The regulation places requirements on high-risk AI systems and prohibits applications considered unacceptable threats to safety or fundamental rights.

Among the prohibited uses are AI systems designed to manipulate or exploit people, social scoring based on behavior, certain forms of biometric profiling and some uses of real-time facial recognition in public spaces. Restrictions also cover emotion recognition in workplaces and schools.

Autonomous Weapons Add Another AI concern

Türk also linked the debate over AI safeguards to the use of autonomous weapons. He expressed “horror” at alleged reports that Russia had used fully autonomous drones in Ukraine, allegedly killing three people in August. “I join calls for the urgent prohibition of weapons that can take lives without human involvement,” Türk says, referring to ongoing discussions on autonomous weapons in Geneva.

The issue highlights a broader question surrounding increasingly autonomous technologies: where accountability lies when systems can make or execute decisions without direct human involvement.

Türk’s comments come as the Human Rights Council begins his second four-year term. His wider address also called for accountability over deaths in US immigration custody and condemned Russia’s reported attacks in Ukraine.

For the technology industry, however, the AI warning places governance at the center of an expanding debate over advanced systems. Türk says international action should move ahead of further advances rather than wait for additional failures to demonstrate the consequences.