PHOENIX (AZFamily) \u2014 Companies are increasingly letting artificial intelligence take the first look at job applicants. New research from Princeton University and the University of Chicago suggests those systems can develop their own prejudices even faster than people do.<\/p>\n
Researchers have already shown that AI models can absorb human bias from their training data. This study points to something different: models inventing bias from scratch, from their own limited experience.<\/p>\n
To test it, the researchers built a virtual hiring game set in a fictional city, adapting a study paradigm from psychology research. The paper<\/a> was presented at the International Conference on Machine Learning in Seoul earlier this month.<\/p>\n \u201cIt\u2019s a multi-round hiring game where you have around 40 jobs,\u201d said Ryan Liu, a Princeton PhD student and one of the study\u2019s authors. \u201cEach round, there\u2019s one job presented and the model needs to choose between four different candidates, each from one of the demographics available.\u201d<\/p>\n The roles ranged from doctors and lawyers to childcare aides and janitors. After each hire, the model learned whether the candidate succeeded, then moved on. It didn\u2019t take long for the models to start sorting people by their fictional ethnicities.<\/p>\n \u201cAlmost immediately,\u201d Liu said. \u201cOnce they see like one single failure example \u2014 for example, demographic A performs badly when assigned to a doctor \u2014 that demographic job pairing is almost never assigned again.\u201d<\/p>\n The effect spread. A single failure in one role also pushed the model away from assigning that group to similar jobs, like teachers and professors. Humans played the same game, and the models showed more bias than they did.<\/p>\n The researchers ran ChatGPT, Claude and Gemini through the simulated hiring game, and the models scored about 1.83 on a segregation scale compared to 0.84 for humans.<\/p>\n Liu said the reason has to do with how these systems are built. They\u2019re designed to draw conclusions from very few examples.<\/p>\n \u201cNormally that\u2019s fine. It even helps them for logic puzzles and math,\u201d he said. \u201cBut it ends up doing really badly when this sort of hiring game or any socially salient type of task comes up.\u201d<\/p>\n That also explains a counterintuitive finding: the more capable reasoning models performed worse, not better.<\/p>\n Only one fix worked<\/p>\n The team tried more than 10 interventions. They tried telling the model to reason more, adjusting how randomly it responded, reframing the task, even instructing it directly to be unbiased. None stuck.<\/p>\n The only thing that worked was changing the model\u2019s incentives. They added a diversity incentive to the model\u2019s assignments.<\/p>\n \u201cAfter that, the model is very fair to all the different demographics and no longer creates a stratified society,\u201d Liu said. \u201cAnything less than that, these models stick to their existing behavior.\u201d<\/p>\n Liu cautioned that the study doesn\u2019t perfectly capture the dynamics of hiring in the real world. Real employers can evaluate more metrics than the study\u2019s models were given, and companies rarely learn quickly whether a hire worked out. That means the feedback loop that produced the AI bias runs much slower in real life.<\/p>\n In Arizona, AI interviews are already routine<\/p>\n Companies aren\u2019t just using AI to screen r\u00e9sum\u00e9s. Coinbase screens applicants through conversations with an AI voice, and Zapier uses AI avatars to interview candidates. Zapier says the technology lets it interview up to five times as many applicants, and the hiring platform Greenhouse reported 63% of job seekers had been interviewed by an AI as of last year.<\/p>\n Travis Laird, a Phoenix-area jobs expert with Robert Half, says that tracks with what he sees locally. He compares avatar interviews to the pre-screening questionnaires most applicants already fill out.<\/p>\n \u201cIt gives you a chance to get some face time, albeit with an avatar, to show your personality, give some context, give some story,\u201d Laird said.<\/p>\n Some critics worry an AI might judge applicants on speech patterns or facial expressions unrelated to the job. Laird\u2019s response: human recruiters carry biases too, and companies have used screening software for decades. The difference now, he said, is that employers can ask the system to explain itself.<\/p>\n \u201cIf you\u2019re challenging AI as to why it\u2019s not moving forward with certain candidates, I think you\u2019ll find some really good positive answers,\u201d he said.<\/p>\n Volume is driving adoption. Laird said employers commonly post a job Friday and find 300-plus applicants waiting by Monday. But AI cuts both ways. Robert Half research found 65% of managers say AI-generated r\u00e9sum\u00e9s are making hiring harder, 84% say AI has increased recruiters\u2019 workload and 67% say it has slowed hiring down.<\/p>\n A separate Resume Genius survey found 87% of hiring managers use AI in at least one stage of recruitment, most commonly r\u00e9sum\u00e9 screening.<\/p>\n His advice to employers: treat the tool like a new team member. \u201cYou\u2019ve got to give it good coaching as to what you\u2019re looking for. Give it feedback, review its work,\u201d he said. But no matter how much gets automated, a person still has to sit across from the candidate. \u201cThat\u2019s going to be the true litmus test.\u201d<\/p>\n For job seekers, he recommends using AI to critique your r\u00e9sum\u00e9 and surface missing keywords while keeping every claim accurate.<\/p>\n \u201cI can tailor my resume for a certain position, but the experience listed still needs to be accurate,\u201d Laird said. \u201cBecause if not, it\u2019s going to get found out throughout the process.\u201d<\/p>\n He also cautions against automating mass applications. Coinbase alone said it receives some 1.7 million applications a year.<\/p>\n \u201cBe selective with the companies you\u2019re applying to,\u201d Laird said. \u201cIf you\u2019re going to use some type of automation for applying, you\u2019ve got to use guardrails.\u201d<\/p>\n See a spelling or grammatical error in our story? Please click here to report it<\/a>.<\/p>\n