Over the past year or two, companies have started using so-called artificial intelligence agents as bona fide \u201cemployees,\u201d even including them in their organizational charts.<\/p>\n
Emma Wiles, a Boston University professor who studies how AI affects workers, stumbled onto this phenomenon in October, at a conference where two human resources executives said that treating AI agents like real employees was a way to increase productivity and to put their companies on the cutting edge. <\/p>\n
But when Wiles and three collaborators from Boston Consulting Group investigated further, they discovered a pitfall. In an experiment involving dozens of companies with AI employees, the researchers found that managers tended to vet documents less carefully when told an AI employee had produced them. The managers missed errors that other managers caught when told they were vetting the work of a human. <\/p>\n
Wiles speculated that managers didn\u2019t think sussing out mistakes made by AI employees was their responsibility. If something went wrong, they could dismiss it as the fault of the tech team, or of the executives who wanted AI employees in the first place. \u201cBut it\u2019s not your problem,\u201d she said, channeling the managers\u2019 mindset about their own roles. <\/p>\n
In the years since AI burst onto the scene, many companies have become aware of flaws produced by the technology and, at times, taken steps to offset them. They know that AI models can be biased against certain groups of people, including nonwhites. They know that chatbots can provide confident but incorrect answers to queries. They know that the bots sometimes spill the beans on information that should remain private. <\/p>\n
But as companies race to bring AI into their day-to-day operations, researchers are discovering more subtle defects. In principle, these flaws could be corrected, too. For example, companies could hold managers directly responsible for the mistakes of AI subordinates. <\/p>\n
But in practice, most corporate users appear to be blissfully unaware of these issues, raising the possibility that AI\u2019s promise of increased productivity and vast cost savings could be undermined.<\/p>\n
Even researchers who study AI may be aware of only a fraction of the problems that the technology introduces. \u201cThere are a whole host of unknown unknowns,\u201d Wiles said.<\/p>\n
One well-documented but underappreciated flaw of artificial intelligence models is that they tend to favor work produced by artificial intelligence. A 2025 paper in the Proceedings of the National Academy of Sciences found that several large language models had a low opinion of text written by humans, creating a \u201cpotentially consequential form of implicit \u2018antihuman\u2019 bias.\u201d <\/p>\n
But many companies seemed unaware of this problem, or at least unable to imagine how it might wreak havoc on their operations. When a team of scholars spelled it out in a subsequent paper, finding that the AI models that companies use to evaluate resumes tend to favor those written with the help of AI over those written entirely by humans, it caught the attention of some corporate recruiters. <\/p>\n
Jane Yi Jiang, an operations professor at Ohio State University who is an author of that subsequent paper, said that she and her co-authors were happy to help when recruiting firms inquired about \u201chow to improve their processes.\u201d<\/p>\n
But they noted that this was almost certainly not the only problem companies were inadvertently introducing in their rush to adopt AI. \u201cPeople are moving so fast to use LLMs without thinking too much about the implications, biases,\u201d she said, referring to large language models like AI chatbots. <\/p>\n
For example, some companies now use AI to help answer questions like how much to charge for a product, or where to open a new location. Relying on the technology for such purposes, however, can quickly go off the rails.<\/p>\n
When left to their own devices, humans often cooperate and seek win-win outcomes. But when AI models assess a situation, they tend to adopt the more coldly calculating, \u201crational\u201d mindset that arises from basic game theory. They might, say, lead a company to aggressively undercut a competitor, even though it risks a damaging price war. <\/p>\n
\u201cMost of the LLMs we test think that human beings are more rational than they actually are,\u201d said Jiannan Xu, a doctoral candidate at the University of Maryland and collaborator of Jiang\u2019s. \u201cBut the most rational response leads to a bad situation for all\u201d in many cases. <\/p>\n
In principle, developers and users of AI can correct for these biases. Jiang and Xu, for instance, found they could reduce antihuman bias by simply instructing models to focus on the quality of the written material they evaluate, and to avoid considering the author. <\/p>\n
But AI researchers can\u2019t correct for biases they aren\u2019t aware of, and several scholars said the impact of these undetected biases could grow. One way is if future models are trained on data produced by today\u2019s models without sufficient care, creating a kind of self-reinforcing loop. <\/p>\n
In that case, \u201cthe tendency to consolidate on existing perspectives and behaviors seems likely,\u201d said Shayne Longpre, an AI researcher and founder of the Data Provenance Initiative, a group that monitors AI infrastructure.<\/p>\n
And then there are the blind spots that arise not so much from AI itself, but from the way humans use it.<\/p>\n
Scholars who turn to AI at every stage of the research process \u2014 asking AI what questions are worth studying; seeking its advice on how to answer these questions; enlisting it to analyze data; relying on it to help write up findings \u2014 could inadvertently narrow the scope of their work. <\/p>\n
\u201cWe don\u2019t necessarily notice it at the individual level,\u201d said Cecilie Steenbuch Traberg, a psychologist at the Copenhagen Business School and an author of a recent paper on the topic. \u201cYou\u2019re sparring with a chatbot, it\u2019s helping me come up with ideas, you might think it sounds great. But at the collective level, it looks pretty similar. Everyone is sounding alike.\u201d <\/p>\n
Wiles, the Boston University professor who examined the way humans manage AI employees, said the shortcomings weren\u2019t necessarily intrinsic to the technology, but arose when humans adopted it with little attention to what could go wrong. <\/p>\n
She and her colleagues surveyed more than 1,000 corporate managers, and found that about one-third said their organizations referred to AI as a \u201cteammate or employee,\u201d and that nearly one-quarter said their employer included AI agents on its organizational charts. \u201cWe call it Scout,\u201d one manager told the researchers in an interview, referring to an AI agent. \u201cIt\u2019s technically an equivalent peer on your team.\u201d <\/p>\n
Wiles and her colleagues gave all the managers they surveyed a set of five documents that contained errors, and gave them 20 minutes to review as many as possible. In some cases they told the managers that an AI employee had done the work; in some cases they said that an AI tool had done the work; and in some cases they said that a human had done the work. <\/p>\n
In general, the stated source of the documents didn\u2019t make much of a difference in how closely managers vetted them.<\/p>\n
But managers at companies that included AI agents on their organizational charts caught substantially fewer mistakes when told they were reviewing the work of an AI employee.<\/p>\n
People who manage humans tend to assume that \u201cif someone on my team makes a mistake, that\u2019s on me,\u201d Wiles explained, which is why they closely check the work of these subordinates. Managers also seem to assume that they\u2019re on the hook for work produced by an inanimate AI tool. But managers at companies with AI employees don\u2019t seem to feel the same responsibility for the work of those employees. <\/p>\n
Her takeaway: Over the past few centuries, scholars and business leaders have developed a reliable set of practices for managing humans. But the psychology of managing anthropomorphized AI is vastly different and \u201cwe\u2019re going out there blind.\u201d <\/p>\n
She worries that the problem is about to get worse. At the same conference where she first heard HR officials talk up the virtues of their AI employees, one went even further, saying her company would soon have AI employees managing humans. \u201cA hush went over the room,\u201d Wiles recalled. <\/p>\n
\u201cWe\u2019ll need someone to study that, too,\u201d she added.<\/p>\n