Paper was cited hundreds of times and widely shared online
A study claiming ChatGPT boosts student learning has been retracted after concerns about unreliable findings, highlighting the need for more rigorous research in the use of AI education, and more critical thinking from those citing it.
A widely cited study claiming that ChatGPT significantly improves student learning has been retracted after concerns emerged over the reliability of its findings.
The paper, published in May 2025 in the Springer Nature journal Humanities & Social Sciences Communications, had argued that OpenAI’s chatbot produced “large positive” effects on student performance and enhanced higher-order thinking skills.
Researchers reached those conclusions through a meta-analysis of 51 previous studies comparing students who used ChatGPT in educational settings with those who did not.
At the time, the findings were hailed by some commentators as among the strongest evidence yet that generative AI could be helpful in classrooms and boost learning outcomes.
Retraction notice
Nearly a year later, publisher Springer Nature formally withdrew the paper, citing “discrepancies in the meta-analysis”.
In its retraction notice, posted on 22 April 2026, the publisher said the issues were serious enough that editors could no longer stand by the validity of the analysis.
“The Editor has decided to retract this paper owing to concerns regarding discrepancies in the meta-analysis,” the notice said.
The journal added that the authors did not respond to inquiries about the retraction. Despite the reversal, the study had already achieved considerable influence within academic and online circles.
According to Springer Nature, the paper was cited 262 times in its own peer-reviewed journals and accumulated more than 500 citations overall. It also attracted nearly half a million readers and ranked in the top percentile of journal articles for online attention.
Too much hype, too little scrutiny
Critics say the episode suggests how rapidly AI-related research can spread before it has been thoroughly scrutinised.
Ben Williamson, a senior lecturer at the University of Edinburgh’s Centre for Research in Digital Education, said the paper had been widely shared on social media as “gold standard evidence” supporting the educational benefits of ChatGPT.
“All this helped the paper get a huge amount of attention, even though the findings really were not supported by the underlying research at all,” he told Ars Technica.
Williamson questioned whether enough rigorous studies on ChatGPT could realistically have been completed in the short period since the chatbot’s public launch in late 2022.
Other researchers had also expressed concerns shortly after publication. Ilkka Tuomi, chief scientist at Meaning Processing Ltd., warned on LinkedIn that the analysis appeared to combine incompatible studies involving different student populations and teaching methods.
“The only reason to do these studies seems to be that statistics and meta-analysis tools can crunch out numbers that look like science,” he wrote.
LLMs can undermine critical thinking skills
The issue has emerged at a time when educators and policymakers are increasingly divided over the use of generative AI in schools and universities. Some teachers warn that AI tools encourage dependency and undermine critical thinking. Others argue AI could support personalised learning if used carefully.
Technology companies, meanwhile, continue to market LLMs as study aids capable of tutoring students, generating practice tests and simplifying complex concepts.
Some governments are now reconsidering the pace of digital adoption in classrooms. In several education systems, policymakers have begun reintroducing printed textbooks and handwritten coursework in response to concerns about screen dependence and AI-assisted cheating.
“We have had several years of hype about AI in education,” Williamson said. “What we have really needed is high-quality research that can actually show us what kinds of impacts AI is having in classrooms and learning practices.”