Sign up to see the future, today<\/p>\n
Can\u2019t-miss innovations from the bleeding edge of science and tech<\/p>\n
The jury\u2019s still out on AI\u2019s effectiveness as a learning tool, but research so far paints a grim picture. Using AI chatbots can impair critical thinking<\/a>, result in lower brain activity<\/a> during cognitive tasks, and has been linked to memory loss<\/a>.<\/p>\n There was one prominent study, however, that provided a glimmer of hope for AI advocates. Published in the journal Nature, it purported to show that using AI like OpenAI\u2019s ChatGPT can have a \u201clarge positive impact on improving learning performance\u201d and a \u201cmoderately positive impact on enhancing learning perception and fostering higher-order thinking.\u201d<\/p>\n The takeaway to the authors was clear: \u201cChatGPT should be actively integrated into different learning modes to enhance student learning, especially in problem-based learning,\u201d they enthused.<\/p>\n But nearly a year after the study was first published, it\u2019s now been unceremoniously retracted. Springer Nature, the journal\u2019s publisher, cited \u201cconcerns regarding discrepancies\u201d for why it pulled the paper, in a retraction note<\/a> published late last month, which \u201cultimately undermine the confidence the Editor can place in the validity of the analysis and resulting conclusions.\u201d<\/p>\n Needless to say, it\u2019s a blow to advocates of AI in education.<\/p>\n \u201cThe paper\u2019s authors made some very attention-grabbing claims about the benefits of ChatGPT on learning outcomes,\u201d Ben Williamson, a senior lecturer at the Centre for Research in Digital Education and the Edinburgh Futures Institute at the University of Edinburgh in Scotland, told Ars Technica<\/a>. \u201cIt was treated by many on social media as one of the first pieces of hard, gold standard evidence that ChatGPT, and generative AI more broadly, benefits learners.\u201d<\/p>\n The paper was not an experimental study, but a meta analysis that synthesized the findings of 51 existing studies on the subject, comparing the cognitive effects of participants who used ChatGPT and those that didn\u2019t. This, as many commentators and experts have noted, already put the study on shaky ground, since ChatGPT was still a novel phenomenon and scientists were only just beginning to research its cognitive effects.<\/p>\n \u201cIt is not feasible that dozens of high-quality studies about ChatGPT and learning performance could have been conducted, reviewed, and published in that time,\u201d Williamson told Ars.<\/p>\n \u201cIn some cases it appears it was synthesizing very poor quality studies, or mixing together findings from studies that simply cannot be accurately compared due to very different methods, populations, and samples,\u201d he added. \u201cIt really seemed like a paper that should not have been published in the first place.\u201d<\/p>\n The retraction comes as the AI industry continues to make a heavy push into the classroom. Companies like OpenAI have partnered with colleges and schools to provide students with free access to their AI tools, and even versions of their chatbots that are tailor-made for specific schools. OpenAI, Anthropic, and Microsoft have poured millions into teachers unions<\/a> to train them how to use AI models. Ohio State University now requires all students in every major to enroll in an \u201cAI fluency\u201d course.\u00a0<\/p>\n That\u2019s despite widespread complaints from teachers about AI being used for rampant cheating<\/a>, and pushback from parents<\/a> who are concerned about letting their children be unwitting test subjects in a large scale AI experiment.<\/p>\n Williamson called the retracted study \u201chugely frustrating for those of us trying hard to make sense of what AI means for learning, teaching, and education more generally.\u201d<\/p>\n \u201cWe have had several years of hype about AI in education, but what we have really needed is high-quality research that can actually show us what kinds of impacts AI is having in classrooms and learning practices,\u201d he told Ars.<\/p>\n This isn\u2019t Springer Nature\u2019s first AI-related controversy. Last year, it began approaching study authors and offering to sell them AI-generated \u201cMedia Kits\u201d that that summarized their papers<\/a>.\u00a0Like many academic journals, it\u2019s also struggled to weed out shoddy AI-generated content<\/a>.<\/p>\n