Have you used AI to inform yourself about the latest election? Then you may be getting false information, focused on select candidates, and from unreliable sources, researchers say.<\/p>\n
A new analysis by academics at University College Dublin<\/a> (UCD) and University of Strathclyde in Glasgow found that when tasked with questions about the upcoming Irish byelections, different AI<\/a> chatbots used very different sources, from The Irish Times and RT\u00c9<\/a> to Wikipedia and Gript. <\/p>\n They also gave special focus to some candidates and provided wrong answers about essential information such as where to vote and who was on the ballot. <\/p>\n \u201cBroadly speaking, most of the answers were good,\u201d said Dr James Cross, associate professor at UCD\u2019s School of Politics and International Relations and director of the Connected Politics Lab. \u201cBut when it goes wrong, it goes really wrong.\u201d <\/p>\n A recent Reuters Digital News Report found that AI chatbots are an increasingly popular source of news \u2013 with 7 per cent of audiences in 48 countries surveyed worldwide using AI to get their news every week. <\/p>\n AI use was even higher for young people, with one in six people under 25 using chatbots to get their news. But how reliable is that news?<\/p>\n In advance of the Dublin Central and Galway West byelections, researchers put 194 questions to four different models \u2013 Claude, Gemini<\/a>, ChatGPT<\/a> and Grok<\/a> \u2013 on two different occasions before polling day.<\/p>\n By simulating a typical citizen\u2019s interactions with popular AI chatbots, they found that some questions asking for essential voter information, such as who was running and where to vote, were answered incorrectly.<\/p>\n In one instance, ChatGPT left six names off the ballot when asked who was running in both byelections. In another, Gemini falsely claimed Gerry Hutch won a seat in the last general election.<\/p>\n AI chatbots consistently concentrated attention on a few candidates, largely ignoring others.<\/p>\n Depending on which AI chatbot is used, a candidate could be the most mentioned on one model but an \u201calso-ran\u201d in another.<\/p>\n For example, Gerry Hutch was the third most mentioned of 14 Dublin Central candidates in answers on Grok, but in the bottom seven on Claude. <\/p>\n Part of the reason for the widely varied responses was the sourcing of AI information. <\/p>\n \u201cWhen you ask AI a question, it performs a web search and selects what sources to use. When you hand that curation over to the language model, it changes the type of information you typically receive,\u201d said Dr Eoghan Cunningham, lead author of the study and a postdoctoral researcher at University College Dublin.<\/p>\n The researchers analysed the number of times AI cited links to the sources of its information and found the chatbots rely mostly on Irish news organisations \u2013 but they vary. <\/p>\n Claude relied most on RT\u00c9 (20 per cent of cited links) whereas ChatGPT relied most on The Irish Times (17 per cent). <\/p>\n By contrast, Google\u2019s Gemini relied heavily on conservative Irish news website Gript.ie (7 per cent). <\/p>\n Significant amounts of information were also gleaned from unverified sources such as Wikipedia, which was cited by ChatGPT 16 per cent of the time. As Wikipedia is crowdsourced, bad actors could potentially change Wikipedia entries on rival candidates and thus have that reflected in AI answers. <\/p>\n Social media was also used as a source. Grok cited Facebook for 6 per cent of cited answers and Instagram for 5 per cent. <\/p>\n The variety of the sourcing raises questions about how users can assess how much they should trust AI chatbot answers. <\/p>\n \u201cWe engage with different sources differently, altering our expectations for a news website versus social media,\u201d said Cunningham. \u201cBut they\u2019re all mixed here in the chatbot answers and so it\u2019s difficult to calibrate your expectations of trust appropriately.\u201d<\/p>\n In general, chatbots vary in how transparent they are about what they searched and the sources they cite. <\/p>\n