{"id":439253,"date":"2025-12-11T03:56:16","date_gmt":"2025-12-11T03:56:16","guid":{"rendered":"https:\/\/www.europesays.com\/us\/439253\/"},"modified":"2025-12-11T03:56:16","modified_gmt":"2025-12-11T03:56:16","slug":"state-attorneys-general-warn-microsoft-openai-google-and-other-ai-giants-to-fix-delusional-outputs","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/us\/439253\/","title":{"rendered":"State attorneys general warn Microsoft, OpenAI, Google, and other AI giants to fix ‘delusional’ outputs"},"content":{"rendered":"

After a string of disturbing mental health incidents<\/a> involving AI chatbots, a group of state attorneys general have sent a letter to the AI industry\u2019s top companies, with a warning to fix \u201cdelusional outputs\u201d or risk being in breach of state law.\u00a0<\/p>\n

The letter<\/a>, signed by dozens of AGs from U.S. states and territories with the National Association of Attorneys General, asks the companies, including Microsoft, OpenAI, Google, and 10 other major AI firms, to implement a variety of new internal safeguards to protect their users. Anthropic, Apple, Chai AI, Character Technologies, Luka, Meta, Nomi AI, Perplexity AI, Replika, and xAI were also included in the letter.<\/p>\n

The letter comes as a fight over AI regulations has been brewing<\/a> between state and federal government.<\/p>\n

Those safeguards include transparent third-party audits of large language models that look for signs of delusional or sycophantic ideations, as well as new incident reporting procedures designed to notify users when chatbots produce psychologically harmful outputs. Those third parties, which could include academic and civil society groups, should be allowed to\u00a0\u201cevaluate systems pre-release without retaliation and to publish their findings without prior approval from the company,\u201d the letter states.<\/p>\n

\u201cGenAI has the potential to change how the world works in a positive way. But it also has caused\u2014and has the potential to cause\u2014serious harm, especially to vulnerable populations,\u201d the letter states, pointing to a number of well-publicized incidents over the past year \u2014 including suicides<\/a> and murder<\/a> \u2014 in which violence has been linked to excessive AI use, the letter states.\u00a0\u201cIn many of these incidents, the GenAI products generated sycophantic and delusional outputs that either encouraged users\u2019 delusions or assured users that they were not delusional.\u201d<\/p>\n

AGs also suggest companies treat mental health incidents the same way tech companies handle cybersecurity incidents \u2014 with clear and transparent incident reporting policies and procedures. <\/p>\n

Companies should develop and publish \u201cdetection and response timelines for sycophantic and delusional outputs,\u201d the letter states. In a similar fashion to how data breaches are currently handled, companies should also \u201cpromptly, clearly, and directly notify users if they were exposed to potentially harmful sycophantic or delusional outputs,\u201d the letter says.\u00a0<\/p>\n

Techcrunch event<\/p>\n

\n\t\t\t\t\t\t\t\t\tSan Francisco
\n\t\t\t\t\t\t\t\t\t\t\t\t\t|
\n\t\t\t\t\t\t\t\t\t\t\t\t\tOctober 13-15, 2026\n\t\t\t\t\t\t\t<\/p>\n

Another ask is that the companies develop \u201creasonable and appropriate safety tests\u201d on GenAI models to \u201censure the models do not produce potentially harmful sycophantic and delusional outputs.\u201d These tests should be conducted before the models are ever offered to the public, it adds.\u00a0\u00a0<\/p>\n

TechCrunch was unable to reach Google, Microsoft, or OpenAI for comment prior to publication. The article will be updated if the companies respond.<\/p>\n

Tech companies developing AI have had a much warmer reception at the federal level. <\/p>\n

The Trump administration has made it known it is unabashedly pro-AI<\/a>, and, over the past year, multiple attempts<\/a> have been made to pass a nationwide moratorium on state-level AI regulations. So far, those attempts have failed \u2014 thanks, in part, to pressure from state officials<\/a>.<\/p>\n

Not to be deterred, Trump announced<\/a> Monday he plans to pass an executive order next week that will limit the ability of states to regulate AI. The president said in a post<\/a> on Truth Social he hoped his EO would stop AI from being \u201cDESTROYED IN ITS INFANCY.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"After a string of disturbing mental health incidents involving AI chatbots, a group of state attorneys general have…\n","protected":false},"author":3,"featured_media":209927,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[691,738,16028,2722,340,252,305,158,67,132,68],"class_list":{"0":"post-439253","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-chatbots","11":"tag-google","12":"tag-meta","13":"tag-microsoft","14":"tag-openai","15":"tag-technology","16":"tag-united-states","17":"tag-unitedstates","18":"tag-us"},"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@us\/115698924485234666","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/posts\/439253","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/comments?post=439253"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/posts\/439253\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/media\/209927"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/media?parent=439253"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/categories?post=439253"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/us\/wp-json\/wp\/v2\/tags?post=439253"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}