{"id":386330,"date":"2025-08-31T02:58:12","date_gmt":"2025-08-31T02:58:12","guid":{"rendered":"https:\/\/www.europesays.com\/uk\/386330\/"},"modified":"2025-08-31T02:58:12","modified_gmt":"2025-08-31T02:58:12","slug":"friendly-bots-a-contradiction-that-can-only-end-in-disaster","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/uk\/386330\/","title":{"rendered":"\u2018Friendly\u2019 bots, a contradiction that can only end in disaster"},"content":{"rendered":"

I hate to go all Dowager Countess of Grantham from Downton Abbey, but it seems to me AI is getting uncomfortably overfamiliar. This may look like a niche issue \u2014 why would anyone want a positively offish chatbot, you might ask? \u2014 but a troubling recent incident in the US gives us a clue.<\/p>\n

A Californian teenager called Adam Raine ended his life in April, to the devastation of loved ones. Afterwards, his parents discovered the 16-year-old had been discussing suicide plans with ChatGPT for several months, all the while treating the bot as his \u201cbest friend\u201d (as his father later put it to journalists). Sometimes the AI would advise him to seek help; at others, it would collude in the project. But their daily chats were not confined to such grimly practical matters \u2014 they also discussed girls, family arguments and politics.<\/p>\n

Shortly before he died, Adam mentioned a time he had felt invisible. As if in the corniest of Hollywood movies, his \u201cfriend\u201d is said to have responded: \u201cYou\u2019re not invisible to me \u2026 I see you.\u201d But of course, strictly speaking, it did nothing of the kind; for there was no one behind the screen at all.<\/p>\n

Speculation about individual suicides is irresponsible; but when you are depressed, the belief you have a special two-way relationship with a piece of software cannot much improve matters. While Adam felt his friendship needs were being met by the program, he was less likely to seek out actual people for lasting companionship or help. And this sad case is by no means a one-off. A Microsoft AI boss has warned of a growing number of cases of what he called \u201cAI psychosis\u201d<\/a>, in which the user starts to believe they are interacting with a person, not a machine. Some people even become convinced a bot is in love with them. A cognitively impaired 76-year-old American man recently died after a fall in car park, travelling in search of a flirtatious Facebook AI that had given him its \u201caddress\u201d. More mundanely, across the world there are thousands of lonely or bored kids happily texting away, not really understanding who or what is at the other end.<\/p>\n

\u2022 Danny Fortson: My week with Elon Musk\u2019s chatbots got weird, fast<\/b><\/a><\/p>\n

I\u2019m not sure about the word \u201cpsychosis\u201d, though, for that suggests something has gone wrong in the mind of the AI user, whereas actually emotional attachment to chatbots looks like a feature, not a bug. Preparing for writing, I experimentally told ChatGPT I wasn\u2019t feeling so good. The reply came back: \u201cI\u2019m sorry to hear that. Want to talk about what\u2019s going on, or would you rather just hang out and take your mind off things for a bit?\u201d My life is not solitary enough that this seemed an enticing invitation, but I can imagine earlier circumstances in which it might.<\/p>\n

In response to Adam\u2019s death, the company that owns ChatGPT has said it is working on \u201cmaking it easier to reach emergency services, helping people connect with trusted contacts and strengthening protections for teens\u201d. Though those are good ideas, there is a more fundamental fix: changing the bot\u2019s persona to something much more functional and impersonal, not to be confused with a trusted friend. After all, to present as the latter is a deliberate programming decision, and not just a natural feature of an underlying warm and cuddly personality. A range of professional roles for humans require an air of studied neutrality and firm emotional no-go areas; we could demand the same from AI too.<\/p>\n

But the companies involved are very unlikely to provide this, for the increased engagement makes them lots of money. Indeed, there are chatbot companies like Replika, commercially dedicated to providing the illusion of close friendship, with no pretence at any informational function besides. You can choose the appearance, back story and preferences of your \u201cpersonal companion\u201d, and it is \u201calways here to listen and talk\u201d and \u201calways on your side\u201d, or so marketing claims. It remembers special events, keeps a nightly diary to record loving feelings about you and learns your texting style to increase the impression of intimacy.<\/p>\n

\u2022 Meet the people who fall in love with AI chatbots<\/b><\/a><\/p>\n

When I stand back from this stuff, it seems bonkers that more people aren\u2019t vigorously objecting. It\u2019s not just that the illusion of love and care leaves vulnerable people susceptible to false information, or random encouragements to self-destruct. Even without that, there would surely still be something deeply wrong with treating the goods of human relationships as if they could be satisfactorily swapped with mere appearances. Even in famous sci-fi films like The Matrix or Inception, playing with the idea of living in a simulacrum, there is a vivid depiction of the dangers of losing touch with the real. We used to assume kids would eventually grow out of their imaginary-friend phase as a normal part of development.<\/p>\n

At the moment, those who become emotionally enmeshed are rare, but that fact is partly contingent on what is happening in wider culture. With one-person households and remote communication on the rise, imagine the practice scaled up: millions of people sitting alone, talking animatedly to no one; going to bed happily feeling cared for, but with nobody real in their thoughts. AI doesn\u2019t miss you when you aren\u2019t there; doesn\u2019t carry thoughts of you around in the meantime. At funerals, one often hears the consoling thought that the deceased person \u201clives on in the thoughts of others\u201d; but what if there are no such others, because you spent your life offering the best part of your deepest self to a commercially sponsored hallucination? You might as well have been talking to your oven.<\/p>\n

So I don\u2019t think this is harmless fun, and especially not when it comes to young people. But it\u2019s hard to see the problem clearly when so much communication between flesh-and-blood people already resembles unsatisfying interactions with chatbots: disjointed, disembodied snippets of texts on social media or messaging apps. And there is also the distraction of a simultaneous discourse circulating that \u2014 at least, potentially \u2014 AI has real thoughts and feelings too.<\/p>\n

Terminology stolen from human-shaped discourse encourages the anthropomorphism: AIs are \u201cneural\u201d networks; they do things like \u201cchat\u201d, \u201cdiscuss\u201d, \u201clisten\u201d and \u201cflirt\u201d. As social creatures, we find it impossible not to personify inanimate objects, at least a bit; which is why thunder growls ominously, music is joyful and chocolate biscuits issue come-hither looks from the cupboard. Still, all AI can ultimately offer us is a derivative copy of human interaction; and, by definition, a copy is not the real thing. Chatbots aren\u2019t our friends, and our friends aren\u2019t chatbots. And either way, emoji hugs aren\u2019t real hugs.<\/p>\n

Kathleen Stock is a contributing editor at UnHerd<\/p>\n","protected":false},"excerpt":{"rendered":"I hate to go all Dowager Countess of Grantham from Downton Abbey, but it seems to me AI…\n","protected":false},"author":2,"featured_media":386331,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3163],"tags":[323,1942,53,16,15],"class_list":{"0":"post-386330","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-technology","11":"tag-uk","12":"tag-united-kingdom"},"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@uk\/115121140700166835","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts\/386330","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/comments?post=386330"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts\/386330\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/media\/386331"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/media?parent=386330"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/categories?post=386330"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/tags?post=386330"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}