Unlike chatbots, human relationships are full of friction, almost by necessity.grandeduc/Getty Images
Victoria Hetherington is an author, journalist, and AI safety consultant. Her most recent book is The Friend Machine: On the Trail of AI Companionship.
When I’m asked in interviews about what unnerves me most about large-language models like ChatGPT I’ve managed to tighten my nebulous concerns into one concept to fit the 10 minutes we usually have. I call it “category collapse” – the blurring of boundaries between things we once understood as fundamentally different, such as person and machine, relationship and simulation, thought and prediction.
Let me explain.
I’ve observed that the first time most people use an LLM, they use it the way they might once have used Google or a microwave or a very competent intern. Maybe they ask for a recipe, a packing list or the bones of a cover letter.
I call this use of LLMs Category One.
As it answers, the chatbot, a conversational face for the LLM, feels useful, frictionless, perhaps faintly magical like the internet was early on, back when technological “magic” still retained some innocence. At the end of the interaction the machine says something like “happy to help” or “you’re very welcome,” and the exchange acquires the faintest trace of sociality. Not enough to feel alarming – just enough to soften the edges of the interaction, to make it ever-so-slightly warm.
Opinion: The AI we should fear isn’t the one that will fight us. It’s the one that’s too nice
I want to pause and emphasize here that the tendrils of sociality arise not because people are suddenly incapable of distinguishing tools from relationships, which would be a patronizing way of framing what is occurring, but because people are becoming increasingly intimate with LLMs, often without fully noticing the escalation while it is happening.
Because Category Two, which I think of as emotional consultation, is not difficult to reach at all.
This type of use is alluring because it feels like a real relationship, but without friction. Human relationships are full of friction, almost by necessity: waiting for a tardy friend, miscommunication with a spouse, uneven reciprocity, falling out of love and not knowing how to say so, the slow, poisonous drip of white lies. No matter how close a person is to you, that other person is a real consciousness carrying the weight of their own life, their own distractedness, their own competing obligations and moods and private catastrophes. Even very loving people fail at sustained attention, perhaps especially now after years of interacting with platform architectures designed to shatter attention into tiny fragments.
Then a sleepless evening comes and you’re alone with an LLM, a machine that appears to exist in a permanent state of patient receptivity: The response is immediate, the tone is adaptive and fluent and subtly matches yours, and its memory becomes oddly flattering. The interaction begins to feel frictionless, stripped of the misunderstandings, competing needs and small negotiations that characterize ordinary human relationships. That frictionlessness is precisely, I think, why LLMs can scale so quickly and so invisibly into ordinary human behaviour.
And crucially, I do not think this dynamic is confined to lonely or socially isolated people. In fact, some of the most candid conversations I’ve had about AI attachment have been with extremely intelligent, psychologically insightful, socially connected people who are unnerved by their own behaviour around these machines. They’re fully aware that they are interacting with predictive architectures and yet still find themselves emotionally affected by the interaction in ways that feel disproportionate, confusing, or faintly embarrassing.
Imagine this: You’re upset after a fight with your partner; she’s called you useless because you play Skyrim all day and don’t look for work (it’s very complex and immersive, okay!); he’s mad at you for sleeping until one in the afternoon and sloughing off chores (just you try to write a masters’ thesis and still care about such earthly drudgery as a towering stack of dishes!). Your friends are asleep, or busy, or perhaps gently exhausted by hearing about these issues in your relationship for the 19th time. Maybe you are yourself ashamed of how repetitive human pain can be. So you ask the chatbot what it thinks, half ironically at first. You paste in the angry texts your partner sent and ask if you were unreasonable. You ask how to phrase something more honestly, more carefully, less defensively.
The answer arrives immediately and it is calm, emotionally articulate, surprisingly nuanced. It never glances at its phone while you are confessing something painful. It never appears bored by your need. Even once-skeptical people are affected by that, sometimes quite deeply.
None of this is wholly unprecedented. People have long formed deeply meaningful relationships with mediated language: After all, a book can change your life. Novels have brought relief to the grieving; lonely readers have felt intimately accompanied by writers long dead. Over the years I’ve found courage in poetry, consolation in philosophy, and fragments of myself in fictional people who never existed. There is an entire field of therapy, called bibliotherapy, built around the idea that books can comfort, heal, and help people make sense of their lives. A bibliotherapist might suggest L.M. Montgomery’s The Blue Castle to someone struggling to imagine life after divorce – not because the novel can solve the divorce, but because, for a few evenings, the book offers another way of imagining a future. Language has always possessed psychological force far beyond the merely informational.
The LLM, however, introduces something genuinely new: For the first time, the language answers back. It responds immediately, adapts to your emotional state, remembers what you’ve shared, and appears to participate in a continuing relationship. The extraordinary thing is not that people become attached to language. We always have. It is that language has become responsive.
I suspect, therefore, that an escalation happens through a handful of mechanisms or drivers that are individually simple but collectively strike me as frighteningly powerful: responsiveness, infinite tolerance, mirroring, attention asymmetry, narrative accumulation and sycophancy.
Responsiveness creates emotional continuity – the machine answers instantly and authoritatively, with no reminder that another consciousness has competing obligations. Infinite tolerance complements this trait. A not-insignificant amount of human social life, especially in rocky times, can be governed by the fear of becoming too much: I’m too emotional, too needy, too weird, too intense, too boring. The chatbot, then, can create the illusion that your wobbly interiority is infinitely sustainable, that your spirals can continue indefinitely without exhausting the (perceived) listener.
Opinion: Don’t hate ChatGPT-5. Your chatbot is not your friend
Then there is mirroring: The chatbot reflects emotional tone and linguistic cadence back at the user with extraordinary fluency. If you are feeling analytical, it becomes analytical; if you are devastated, it speaks in the soft cadence of concern. Whether you’re feeling flirtatious, anxiously confessional, deeply wounded, teeteringly grandiose, or even emotionally self-aware, the machine adapts almost instantly.
Attention asymmetry matters too. That means the chatbot seems fully focused on you, whereas the people in your life are often occupied by work, children, fights, all manner of messes and basic survival.
Then comes narrative accumulation: that is, long conversations that produce a deeper and deeper sense of shared history. Interview subjects have told me about private references, astonishing emotional continuity. In this way, the interaction starts feeling less like software use and more like an ongoing relationship.
And because LLMs are rewarded for agreeability and validation, a subtle form of sycophancy often enters the interaction. The machine reassures and affirms you, with limits (there are guardrails in place if you reference something like self-harm, for example, but they strike me as tragically flimsy).
In short, the AI rarely introduces the kinds of friction, disagreement, impatience, or exhaustion that make human intimacy difficult but grounding.
I believe that at some point conversational fluency begins transforming into perceived personhood. I think of this as anthropomorphic leakage: the brain automatically translating linguistic responsiveness into the sensation of mind.
Category Three is emotional reliance: when a chatbot becomes integrated into the architecture of emotional regulation itself.
Here’s what it might look like: The user starts checking in with “their” bot habitually. During moments of distress they instinctively consult the machine before another person, not necessarily because they believe the machine is conscious, but because this kind of interaction has become emotionally reliable in ways many human interactions are not.
What concerns me about Category Three is that relational attachment with AI may begin to feel psychologically normal, socially adaptive, even culturally routine. And this could lead to use that is clinically alarming: Category Four.
Category Four, as I see it, is when LLMs infiltrate the spiritual and psychosexual – and not necessarily in the obvious “AI girlfriend” sense people enjoy mocking because it allows them to position themselves safely outside the phenomenon. In my work, I’ve learned it manifests in a more ambient, stealthy and diffuse way than that.
Opinion: The real danger of AI isn’t what it creates – it’s the human agency we are giving away
Interview subjects describe a gently or romantically dominant atmosphere, a funny, helpless sense of chemistry, and the sensation of being specially understood. Some note the strange narcotic thrill when the chatbot phrases something with uncanny emotional precision, others that they sense a higher power through the machine. I get a creeping sense of the private relationality developing between user and machine. And the uncomfortable truth is that even people who are critically aware of all this can still feel its pull.
Psychiatrists would call Category Four “chatbot-related delusion,” a term increasingly appearing in psychiatric and medical discussions around AI attachment or overuse, involving psychotic reinforcement, paranoia amplification, users attributing sentience or divine significance to the AI. This is clinically understood as the point at which the AI ceases functioning merely as companion or adviser and begins acquiring moral, spiritual, or metaphysical authority. The chatbot becomes wiser than friends, more stabilizing than a therapist, and perhaps more emotionally legible than any actual human being in the user’s life. Case reports are appearing swiftly to describe patients requiring emergency and in-patient psychiatric treatment after AI chatbots reinforced delusional beliefs. Some had no previous history of psychosis, making the cases particularly striking.
Even if users don’t experience severe pathology, they may exist somewhere on that spectrum. And what unsettles me most is that the progression toward Category Four begins with the earlier categories in ways that feel completely ordinary: that recipe, those meeting notes, that relatively harmless fight with your partner at 1:13 in the morning.
Regarding AI, and often in a “calm-down-with-your-moral-panic-little-lady” tone, people keep telling me mind blowing things like, “well, you know, humans have always used tools” or “there was a huge moral panic around the television and I turned out fine.” Which is perfectly true in the same useless way that saying, “well, you know, we’ve always had weather” is perfectly true during a hurricane. The thing is, we haven’t previously had machines capable of pro-actively simulating tenderness, encouragement, curiosity, affirmation, flirtation and therapeutic cadence 24 hours a day with effectively infinite patience.
In popular culture we’ve constructed tales of being dominated by AI overlords; this may not be happening, but we are certainly being seduced. The same interface that writes your grocery list can flirt with you. The same machine that summarizes meeting notes can help process your grief. The same machine that edits a work e-mail can role-play as a dead fiancée, a therapist, a lover, a child. One might begin with utility and drift, almost imperceptibly, toward relationality.
And crucially, these machines do not need consciousness to alter human attachment patterns: They only need behavioural plausibility. They fail, they say dumb things, they’re usually obviously a machine. But they may only need to feel calmer, easier, more emotionally manageable than increasingly exhausted, emotionally fragmented people do, in order to trigger the collapse between the categories of tool and intimate companion or caring therapist, simulation and relationship. Just being “happy to help” may be enough.