Swedish researchers fed a fake medical diagnosis, along with phony scientific studies, into AI chatbots to see if they would fall for it \u2013 and they did.<\/p>\n
A team led by Almira Osmanovic Thunstr\u00f6m at the University of Gothenburg cooked up a completely fraudulent eye condition called bixonimania \u2014 a ridiculous made-up ailment involving pinkish eyelids from too much screen time or eye-rubbing, to see if\u00a0large language models (LLMs)<\/a>\u00a0would treat them as legitimate medical science.<\/p>\n The researchers didn\u2019t exactly hide the punchline.<\/p>\n The symptoms of the phony disease included sore and itchy eyes. doi.org<\/p>\n The phony 2024 scientific papers featured fictional authors, including a lead researcher named Lazljiv Izgubljenovic \u2014 which translates to \u201cThe Lying Loser\u201d in Bosnian. <\/p>\n His photo was AI-generated, just to drive the joke home.<\/p>\n Acknowledgments also thanked \u201cProfessor Sideshow Bob\u201d and a professor from the Starfleet Academy for access to a lab aboard the USS Enterprise.<\/p>\n The experiment wasn\u2019t meant as a flat-out \u201cgotcha\u201d on AI, but \u201crather a reflection of how humans have forgotten to be skeptical when presented information,\u201d Osmanovic Thunstr\u00f6m told The Post.<\/p>\n She chose the name \u201cbixonimania\u201d because it \u201csounded ridiculous\u201d and \u201cI wanted to be really clear to any physician or medical staff that this is a made-up condition, because no eye condition would be called mania \u2014 that\u2019s a psychiatric term.\u201d<\/p>\n ChatGPT, Google\u2019s Gemini, Microsoft\u2019s Copilot, and the rest happily swallowed the nonsense and started dishing out serious-sounding medical advice about bixonimania \u2014 warning users about pinkish eyelids, blue-light damage, and urging them to see an ophthalmologist for this entirely imaginary condition.<\/p>\n Almira Osmanovic Thunstr\u00f6m at the University of Gothenburg led the research team that cooked up the phony disease to see if it could fool AI chatbots. Sahlgrenska Science Park\/ Youtube<\/p>\n It didn\u2019t stop there. Blog posts explaining bixonimania appeared on the website\u00a0Medium, and somehow the fake papers even got cited in peer-reviewed literature. Articles about a disease that was never real, based on studies that were obviously a joke popped up on the academic sites and the social network SciProfiles.<\/p>\n