The young woman ran for her life, but it was too late.<\/p>\n
A small Russian drone resembling a model airplane swooped down from the sky last month in the city of Zaporizhzhia in southeastern Ukraine. As it tried to navigate toward a gas station, it crashed into a wall and exploded in a hail of shrapnel, killing Tetiana Bubynets, 19, a university student, and two others.<\/p>\n
They were among the thousands of Ukrainian civilians who have died in Russian airstrikes, but the circumstances of their deaths came with an ominous twist. The drone that killed them was guided by an experimental artificial intelligence system, not a human pilot, according to drone experts, Ukrainian military commanders and the forensic team that examined wreckage from the deadly strike and others in the city.<\/p>\n
The drone was programmed by human operators to travel toward a particular gas station, but once it got close, it chose its exact target \u2014 most likely propane tanks \u2014 on its own, based on its training to recognize such tanks and strike them, the experts and military officials said.<\/p>\n
It is a development long watched for in the war in Ukraine, presaging a dystopian future in which killer robots roam the skies making life-or-death decisions that reduce targets to data points.<\/p>\n
\u201cThis is a risk for the whole world,\u201d said Col. Serhiy Minaiev, the commander of air defenses in Zaporizhzhia. \u201cIn a few years, we will be living in a \u2018Terminator\u2019 movie. It\u2019s no joke. Machines are making decisions to strike.\u201d<\/p>\n
Analysis of debris from this attack and others in Zaporizhzhia found that the drones had onboard minicomputers, sold commercially by Nvidia, that made their targeting decisions, the experts and military officials said. Nvidia produces a majority of the chips powering the world\u2019s most advanced A.I. systems.<\/p>\n
The presence of Nvidia modules and a lack of antennas on the drones led Ukrainian air defense commanders to surmise that the weapons were guided by an autonomous A.I. system, which was confirmed on further investigation.<\/p>\n
Autonomous A.I. systems can be trained on thousands of images to recognize categories like \u201criver,\u201d \u201cmilitary truck\u201d or \u201cperson.\u201d Self-targeting drones use their cameras to hunt for these things with greater precision than a human pilot can provide. But opponents of such weapons argue that, without human intervention, drones may be more prone to mistakes or violations of the laws of war, such as by failing to distinguish innocent civilians from combatants.<\/p>\n
The use of self-targeting A.I. in the gas station attack, on July 6, would be the first documented case in which civilian deaths were caused by a Russian drone outfitted with such a system, said Kateryna Bondar, a senior fellow at the Wadhwani AI Center at the Center for Strategic and International Studies in Washington. She added that the Nvidia module \u201cis the best proof\u201d that Russia was experimenting with autonomous A.I.<\/p>\n
While an adviser to President Volodymyr Zelensky has written on social media that Russia was testing self-targeting A.I. systems, the fatalities have not been previously reported, nor have the details of the Nvidia Jetson Orin computer module that was used.<\/p>\n
Both Russia and Ukraine were already using A.I. in multiple drone models before the fully autonomous drones emerged. In those earlier models, A.I. is used only for what is known as the \u201clast mile\u201d \u2014 the final stage of the attack, after a remote human pilot has selected the target. Fully autonomous A.I. removes humans from the final target selection.<\/p>\n
The Nvidia chip in the Zaporizhzhia attack was not encrypted, Ukrainian officials said, so they could see the terrain images that had been uploaded to the module, enabling the drone to track visual landmarks and stay on course. They were also able to examine coding, they said, that revealed what types of targets the drone had been trained to recognize and attack, like propane tanks.<\/p>\n
Earlier drone models are generally either piloted by human operators, through radio transmissions or fiber-optic cables, or they are preprogrammed with coordinates. As drones have taken over the battlefield in Ukraine, their pilots have become prime targets.<\/p>\n
An autonomous A.I.-guided drone, by contrast, requires no human pilots and can be more accurate than a munition programmed to hit coordinates.<\/p>\n
Local air defense officials say that Russia has been aiming for civilian objects like pumps or propane tanks at gas stations in trying out autonomous drones in Zaporizhzhia, along with military recruitment centers. It is not clear why these targets were chosen for the tests.<\/p>\n
Vadym Kushnikov, a drone expert at the Kharkiv Aviation Institute in Ukraine, said the drone that killed the three civilians was \u201ca preprogrammed tool trained in virtual reality to track specific objects.\u201d<\/p>\n
While traditional software follows step-by-step rules written by a person, an A.I. system derives its own rules by finding patterns across enormous amounts of data, a process called machine learning. That allows it to manage unanticipated situations. It also means no one can fully predict what it might do.<\/p>\n
Rights groups and the International Red Cross vehemently oppose the use of A.I. weapons without humans in the loop. Such groups say that autonomous systems essentially leave robots to interpret international law in determining legitimate targets in war.<\/p>\n
But some experts argue that, in the future, weapons should in fact be required to have A.I. systems. These weapons are safer, the argument goes, because they are able to make decisions before striking, even if not always perfectly, unlike unguided aerial bombs or ordinary artillery shells that just fall without seeing what they will hit.<\/p>\n
The weapon in the fatal strike was flying in a group of about half a dozen drones, according to the regional military governor, the city\u2019s air defense commander and an air defense team on duty that day. None were emitting radio signals, and they had other characteristics associated with Russia\u2019s autonomous A.I. tests, including being launched in a wave, the officials said.<\/p>\n
While some of the A.I.-guided drones tested by Russia have detonated on impact, others have not and remained intact, Ukrainian officials said. They showed debris from two such drones to reporters for The New York Times. The Times photographed the Nvidia computer module inside the drones and sent the images to the company.<\/p>\n
Nvidia confirmed that the photos showed a Jetson Orin minicomputer. Such modules, which cost only a few hundred dollars, have been found on several types of Russian weapons.<\/p>\n
In a statement, the company said that the minicomputer was a consumer-grade product \u201csold to students, developers and start-ups for a wide range of beneficial applications.\u201d Nvidia sells the computer in many countries but not in Russia. It is, though, widely available through resellers, the statement said.<\/p>\n