When I was a geeky kid, reading all sorts of science fiction, the idea of advanced artificial intelligences came up often. Sometimes in the form of robots, sometimes in the form of a computer without a robotic body, but always something thought provoking as these synthetic brains breached the boundaries of sentience and started to grapple with the mysteries of identity and what it means to be something aware of one’s self. They asked what this thing called “love” is or wished to experience laughter or something equally profound.
What they almost never did was make buyback deals on used BMWs that would later be reneged upon.
And yet, here in our stupid reality, that’s what AI is actually doing, when it’s not too busy giving you bad advice or putting pictures of famous people you find online in skimpy clothes. This particular situation, CBC writes, happened at the BMW Toronto dealership, when 2021 BMW X3 XDrive 30i owner Zack Giacomelli, displeased from having to do expensive repairs on the car, decided to reach out to BMW Toronto to see if they were interested in buying the car back.
Happily, they were! At least, according to BMW Toronto’s online representative, named Quinn.
Quinn offered $27,162.79 for the BMW, covering the remaining balance on the car, with “no money owing, and no cash back,” according to texts with Quinn that Giacomelli shared with CBC. Giacomelli was very happy with this offer, and for the hell of it made a counteroffer of $28,000, which Quinn allegedly stated “sounds reasonable.” Quinn even allegedly suggested that Giacomelli come by the dealership to “lock in today at 3:30.”
Everyone should be happy, right? A deal is about to be struck!
Well, not quite. According to CBC,
“Moments later, Giacomelli said, a BMW Toronto sales consultant called to revoke the offer, explaining that Quinn wasn’t a real person, but rather an artificial intelligence chatbot that had made the offer in error.”
Why did the dealership allegedly rescind the offer? Because Quinn is not a human being: Quinn is an AI chatbot.
This of course brings up that perennial philosophical question: so what?
Yes, Quinn is an AI, but it’s also acting on behalf of the dealership, and customers who interact with Quinn have no way of knowing if Quinn is man or machine; just ask Alan Turing about that.
Screenshot: YouTube/CBC
Giacomelli brings up an excellent point about all of this, telling CBC:
“If they’re going to be replacing their employees’ jobs with AI, then they need to be honouring what that AI says.”
Mr. Giacomelli, I couldn’t agree more.
You can’t have it both ways; a business can’t automate a job that was formerly a person with authority to make deals like this and then decide that no, what this machine says while operating as an agent of the dealership cannot be taken as actual, binding interactions with the dealership, like what the human agents they replaced did. How the hell is a customer supposed to know that? And if the AI will say things that aren’t actually what the dealership intends to do, then what’s the point of the damn AI?
If a human employee had made a deal like this, the dealership could likely have to honor it, or at the very least consider it. Just as a human can be an idiot and make bad decisions, an AI can be an idiot as well, and perhaps even could execute idiocy with efficiencies and speeds no human could hope to match.
An AI chatbot may be able to interact convincingly with a customer, but it fundamentally does not understand what it’s doing. It has no idea what cars actually are or how much money is actually worth or, well, anything about life in the world, really. It’s a bunch of branching conditionals and massive databases that delivers the illusion of sentience. The reality is it doesn’t “know” jack feces.
Eventually, the dealership did honor the $27,162.79 offer, so they did end up doing the right thing in the end. But even if they hadn’t, the law seems to be on Giacomelli’s side. Canadian law has already encountered this issue, as a case in 2024 ruled that companies (in this specific case, Air Canada) are indeed responsible for what their AI agents tell customers.
The legal decision makes its point and reasoning quite clear:
“Air Canada argues it cannot be held liable for information provided by one of its agents, servants, or representatives – including a chatbot. It does not explain why it believes that is the case. In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions. This is a remarkable submission. While a chatbot has an interactive component, it is still just a part of Air Canada’s website. It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot.”
Where that 2024 case made clear that it “makes no difference whether the information comes from a static page or a chatbot,” it would also stand to reason, and perhaps even more strongly, that it makes no difference if an official communication from a car dealership comes from a human agent or a silicon one.
Hopefully, this case will be noticed by other businesses using a chatbot to interact with clients and customers. You can’t embrace the economics of getting rid of employees in favor of AI without accepting whatever some AI may say or do while representing the organization. AIs are, bluntly, idiots. They can’t help it; they’re not alive, and have no genuine understanding of the world. Is that really what a company wants representing it?
(top image: Jason Torchinsky, YouTube, CBC)