{"id":3758,"date":"2026-04-13T06:33:12","date_gmt":"2026-04-13T06:33:12","guid":{"rendered":"https:\/\/www.europesays.com\/ai\/3758\/"},"modified":"2026-04-13T06:33:12","modified_gmt":"2026-04-13T06:33:12","slug":"there-is-no-battlefield-exception-to-human-centered-ai","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ai\/3758\/","title":{"rendered":"There is no battlefield exception to human-centered AI"},"content":{"rendered":"

Stanford has built<\/a> one of the world\u2019s most influential brands around human-centered AI. From the start, the Stanford Institute for Human-Centered AI (HAI) asserted that AI should augment people\u2019s work, not replace them. At its \u201cAI in the Loop\u201d conference, HAI faculty argued<\/a> that humans must remain at the center of decision-making. That governing principle of human-centered AI must be considered most carefully when it comes to war, where the stakes are lethal.\u00a0<\/p>\n

Anthropic\u2019s Claude became the first AI model approved to operate on classified military networks, and Anthropic says<\/a> it is extensively deployed across the U.S. military for analysis and operations. Palantir\u2019s Maven system, used for intelligence analysis and weapons targeting, included<\/a> prompts and workflows built with Claude Code.<\/p>\n

When the Trump Administration designated<\/a> Anthropic a supply-chain risk after a dispute over safety guardrails, Pentagon users resisted<\/a> the phaseout, estimating that recertification could take 12 to 18 months, demonstrating AI\u2019s early entrenchment in our military. As that phaseout drags, the Pentagon is now solidifying<\/a> Palantir\u2019s Maven as their go-to AI system.\u00a0<\/p>\n

According to Reuters, American investigators believe<\/a> U.S. forces to be responsible for the strike on the school in Minab, Iran, which reportedly killed<\/a> 168 school affiliates, including schoolchildren. The Washington Post reported<\/a> that the school was on a U.S. target list and that Maven (powered in part by Claude) was used to suggest targets, issue precise coordinates and prioritize them by importance.<\/p>\n

It is not clear whether Maven nominated the school specifically; a former senior defense official cautioned<\/a> the Post against assuming it did, noting the site had likely been on U.S. target lists for years. A preliminary Pentagon review cited by the Post indicates the strike was probably the result of an intelligence error about the location, and a full investigation continues.<\/p>\n

Nonetheless, the episode points to a larger problem: once AI systems are integrated into operational workflows, their outputs carry undue weight. Regardless of who or what first nominated the school, the chain that approved it did not catch that the underlying intelligence was stale in a military campaign whose pace and scale Maven is theoretically built to enable. People have begun to defer to AI systems even after machine errors should have raised alarms.\u00a0<\/p>\n

This overreliance is documented<\/a> among average AI users. University of Pennsylvania (UPenn) Wharton researchers recently found<\/a> that when an AI assistant was wrong, users still followed it about 3\/4 of the time and reported higher confidence in their answers even after it led them astray. In a productivity app, that is frustrating. In war, it becomes a tragedy.<\/p>\n

Stanford has already studied a similar failure in cars. When a person resumes control after automation, they need<\/a> a measurable period to readapt before their hands behave safely enough to navigate the road. The National Transportation Safety Board warns<\/a> that drivers are susceptible to automation complacency, and the Insurance Institute for Highway Safety found<\/a> that only one of 14 partial automation systems it tested earned an acceptable safeguard rating. In other words, partial automation can put a human to sleep at the wheel without taking the wheel away.<\/p>\n

Military AI creates the same trap, only with lethal stakes. A human can remain \u201cin the loop\u201d while losing control over the reasoning that matters. When the UPenn researchers added<\/a> a 30-second time limit to a person\u2019s designated activity, it roughly tripled the odds that users would surrender to a faulty AI rather than override it.<\/p>\n

Once a system has framed the picture and compressed the timeline, the person at the end of the chain may still approve the action but fail to actually pass judgment themselves. The head of U.S. Central Command has said<\/a> that \u201chumans will always make final decisions on what to shoot and what not to shoot.\u201d But a human signature is not the same thing as human control.<\/p>\n

In 2024, HAI researchers put<\/a> five off-the-shelf large language models through military and diplomatic simulations. All five demonstrated forms of escalation that were difficult to predict.\u00a0 That finding should concern anyone who thinks the answer is simply to keep a human somewhere nearby. Without enough time and context, the human does not challenge the machine. The human ratifies it.<\/p>\n

None of this means AI can never be useful. The International Committee of the Red Cross (ICRC) argues<\/a> that AI tools can help humans synthesize information, offer a wider range of context to a decision and reduce harm to civilians. But the ICRC also warns that, in legally mandated decisions on lawful targeting, AI outputs may inform but must not replace human judgment, or else the human risks becoming a \u201chuman rubber stamp.\u201d Even the U.S. military\u2019s own directive says<\/a> commanders and operators must exercise appropriate levels of human judgment over the use of force. Anthropic itself says<\/a> today\u2019s models are not reliable enough for fully autonomous weapons.<\/p>\n

Stanford, too, has entered the debate. HAI faculty and fellows have called<\/a> for explicit policy on AI-designed weaponry usage and warned<\/a> of AI\u2019s power for individual surveillance, information key to military operations. HAI is well-placed to publish the conditions under which human-centered AI principles call for constraining a military application and name the categories of judgment that should not be so automated to erode human control. For example, adding<\/a> friction to the design to avoid inappropriate reliance on automation. \u201cAugment, don\u2019t replace\u201d works only if it means more than leaving a person at the end of an opaque, machine-built process without time to question or refuse its conclusion.<\/p>\n

We do not yet know everything about Minab, but we know enough about AI acceleration to reject the fiction that a human is in charge simply because they are present. In war, the cost of substituting a person with technology is too high to risk. Stanford helped define human-centered AI, the very thing meant to resist the dangers that we\u2019re watching play out in the U.S. military. Now is the time to prove that the definition holds.<\/p>\n

Utsav Gupta is a Stanford Master of Liberal Arts candidate researching human purpose in the age of AI. He is founder and co-CEO of Filarion, building AI litigation tools and spatial computing systems, and serves as commissioner for Palo Alto Utilities. Views are his own.<\/p>\n","protected":false},"excerpt":{"rendered":"Stanford has built one of the world\u2019s most influential brands around human-centered AI. From the start, the Stanford…\n","protected":false},"author":2,"featured_media":3759,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[24,3619,53,25,3620,3621,182,689,3622,3623,3624,3625,3626,3627,3628,3629,3630,361,3631,3632,2947,1757,3633,3634,2186],"class_list":{"0":"post-3758","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-ai","8":"tag-ai","9":"tag-ai-acceleration","10":"tag-anthropic","11":"tag-artificial-intelligence","12":"tag-autonomous-weapons","13":"tag-battlefiled","14":"tag-claude","15":"tag-coding","16":"tag-department-of-defense","17":"tag-department-of-war","18":"tag-govern","19":"tag-hai","20":"tag-human-centered-ai","21":"tag-human-centered-artificial-intelligence","22":"tag-machine-errors","23":"tag-minab","24":"tag-mistakes","25":"tag-palantir","26":"tag-prediction","27":"tag-strike","28":"tag-system","29":"tag-tools","30":"tag-tragedy","31":"tag-ustav-gupta","32":"tag-war"},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/3758","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=3758"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/3758\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/3759"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=3758"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=3758"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=3758"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}