Stay informed with free updates<\/p>\n
Simply sign up to the Artificial intelligence myFT Digest — delivered directly to your inbox.<\/p>\n
The writer is programme director of the Institute for Global Affairs at Eurasia Group<\/p>\n
When OpenAI and Mattel announced a partnership earlier this month, there was an implicit recognition of the risks. The first toys powered with artificial intelligence would not be for children under 13.<\/p>\n
Another partnership last week came with seemingly fewer caveats. OpenAI separately revealed that it had won its first Pentagon contract<\/a>. It would pilot a $200mn programme to \u201cdevelop prototype frontier AI capabilities to address critical national security challenges in both warfighting and enterprise domains,\u201d according to the US Department of Defense<\/a>.\u00a0<\/p>\n That a major tech company could launch military work with so little public scrutiny epitomises a shift. The national security application of everyday apps has in effect become a given. Armed with narratives about how they\u2019ve supercharged Israel and Ukraine in their wars, some tech companies have framed this as the new patriotism, without having a conversation about whether it should be happening in the first place, let alone how to ensure that ethics and safety are prioritised.<\/p>\n Silicon Valley and the Pentagon have always been intertwined, but this is OpenAI\u2019s first step into military contracting. The company has been building a national security team with alumni of the Biden administration, and only last year did it quietly remove a ban<\/a> on using its apps for such things as weapons development and \u201cwarfare.\u201d By the end of 2024, OpenAI had partnered with Anduril, the Maga-aligned mega-startup headed by Palmer Luckey.<\/p>\n Big Tech has changed dramatically since 2018, when Google staffers protested against a secret Pentagon effort called Project Maven over ethical concerns, which led the tech giant to let the contract expire. Now, Google has totally revised<\/a> its approach. <\/p>\n Google Cloud is collaborating with Lockheed Martin<\/a> on generative AI. Meta, too, changed its policies so that the military can use Llama AI. Big Tech stalwarts Amazon and Microsoft are all in. And Anthropic has partnered with Palantir to get Claude to the US military.<\/p>\n It\u2019s easy to imagine AI\u2019s advantages here, but what\u2019s missing from public view is a conversation about the risks. It\u2019s now well-documented that AI sometimes hallucinates, or takes on a life of its own. On a more structural level, consumer tech may not be secure enough for national security uses, experts have warned<\/a>.<\/p>\n Many Americans and western Europeans share this scepticism. My organisation\u2019s recent survey<\/a> of the US, UK, France and Germany found that majorities support stricter regulations when it comes to military AI. People worry it could be weaponised by adversaries \u2014 or used by their own governments to surveil citizens.<\/p>\n Respondents were offered eight statements, half emphasising AI\u2019s benefits to their country\u2019s military and half emphasising the risks. In the UK, less than half (43 per cent) said that AI would help their country\u2019s military improve its workflow, while a large majority (80 per cent) said that these new technologies needed to be more regulated to protect people\u2019s rights and freedoms.<\/p>\n Using AI for war could, at its most extreme, mean entrusting a flawed algorithm in questions of life or death. And that\u2019s already happening in the Middle East. <\/p>\n The Israeli news outlet +972 Magazine<\/a> has investigated Israel\u2019s military AI in its targeting of Hamas leaders in Gaza and reported that \u201cthousands of Palestinians \u2014 most of them women and children or people who were not involved in the fighting \u2014 were wiped out by Israeli air strikes, especially during the first weeks of the war, because of the AI program\u2019s decisions\u201d. <\/p>\n The US military, for its part, has used AI for selecting targets in the Middle East, but a senior Pentagon official told Bloomberg last year<\/a> that it wasn\u2019t reliable enough to act on its own.<\/p>\n