{"id":49948,"date":"2026-05-25T01:02:18","date_gmt":"2026-05-25T01:02:18","guid":{"rendered":"https:\/\/www.europesays.com\/ai\/49948\/"},"modified":"2026-05-25T01:02:18","modified_gmt":"2026-05-25T01:02:18","slug":"new-ai-scientists-are-improving-but-reveal-their-fundamental-limits-2","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ai\/49948\/","title":{"rendered":"New ‘AI scientists’ are improving\u2014but reveal their fundamental limits"},"content":{"rendered":"

\"New<\/p>\n

Credit: Unsplash\/CC 0 Public Domain<\/p>\n

Many of the most exciting discoveries in science involve highly specialized knowledge and making connections between far-flung facts. Scientists must combine deep analysis with broad reasoning strategies.<\/p>\n

As in many information-rich tasks, researchers are looking to artificial intelligence (AI) systems to speed up their work. AI tools may be able to support key steps such as generating ideas, reviewing existing work and analyzing data.<\/p>\n

The latest systems use large language models (LLMs) to allow scientists to interact naturally and directly with the vast body of knowledge captured in words in the scientific literature.<\/p>\n

But as two<\/a> new<\/a> systems described in papers just published in Nature show, when it comes to science, language alone can only go so far.<\/p>\n

What AI is doing to science<\/p>\n

A number of organizations, such as Sakana AI<\/a>, are trying to automate the entire scientific process. To date, these efforts have largely focused on computer science, where “experiments” mainly involved designing and writing code.<\/p>\n

However, the Agents4Science<\/a> conference organized at Stanford last October showcased a broader range of AI-generated papers. They covered topics from mechanical engineering and protein design to a system called BadScientist<\/a> which deliberately produced “convincing but unsound<\/a>” research.<\/p>\n

I have previously raised concerns<\/a> about the impacts of AI scientists on the scientific ecosystem. Recent work validates these concerns, showing increased quantity but lower quality<\/a> of both papers and peer reviews, identifying fabricated references in published works<\/a>, finding fabricated and misleading images<\/a>, and more.<\/p>\n

\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\tWhat scientists are doing with AI<\/p>\n

AI systems clearly can’t be trusted to conduct the full process of science on their own. But how about using AI to help scientists get more done more quickly?<\/p>\n

This is the intent of the two new systems described in Nature: Robin<\/a>, made by non-profit Future House<\/a>, and Co-Scientist<\/a>, from Google DeepMind.<\/p>\n

Both systems aim to accelerate scientific discovery, working in collaboration with a scientist. Both are also “multi-agent” AI systems, meaning they are built as a collection of specialized agents each targeting specific steps of the scientific discovery process, coordinated by a “supervisor” agent.<\/p>\n

The agents that comprise Co-Scientist aim to mirror abstract cognitive tasks, such as a “reflection agent” that acts as a critical scientific peer reviewer assessing the quality of a hypothesis. “Ranking agents” debate research hypotheses in “tournaments,” using multiple interacting LLMs to simulate a discussion about the relative merits of two hypotheses.<\/p>\n

Robin’s agents, on the other hand, are more tuned to specific tasks relevant to drug repurposing, aiming to identify new drugs for a given disease. One agent focuses on selecting experimental tests, while another analyzes complex biomedical data.<\/p>\n

\n Discover the latest in science, tech, and space with over 100,000 subscribers who rely on Phys.org for daily insights.
\n Sign up for our free newsletter<\/a> and get updates on breakthroughs,
\n innovations, and research that matter\u2014daily or weekly.\n <\/p>\n

\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\tHow do the results stack up?<\/p>\n

Co-Scientist can assess the quality of its generated proposals, using a method called the Elo rating<\/a> which is best known for ranking chess players. Co-Scientist’s self-ratings of the novelty and impact of its outputs align quite well with the preferences of human experts and judgments by other LLM systems.<\/p>\n

In a drug repurposing experiment, Co-Scientist selected 30 drug candidates as promising treatments for a kind of cancer called acute myeloid leukemia. Expert (human) oncologists refined the list, and five drugs were tested in the lab. Of these, three showed some positive results and one seemed to show particular promise.<\/p>\n

Other experiments showed the potential of Co-Scientist to explore combinations of multiple drugs.<\/p>\n

Notably, the predictions of Co-Scientist were not compared with the plethora of targeted computational and machine learning methods for drug repurposing<\/a> that have been developed over decades of computational biology research. This means we don’t know whether the new general-purpose tool outperforms more specific AI approaches.<\/p>\n

Both systems stop short of validating their hypotheses directly, which would involve real physical experiments. Both also rely heavily on human input to define the key scientific question, sense-check predictions, and prioritize predictions for further investigation.<\/p>\n

Co-Scientist focuses primarily on generating hypotheses through elaborate reasoning agents, leaving validation and interpretation to subsequent steps. Robin also uses an agent to analyze data produced from real-world experiments.<\/p>\n

Robin was used to propose 30 drug candidates for a condition called dry age-related macular degeneration. The top five were selected for testing.<\/p>\n

Robin also made proposals for the experiments, with several suggestions overridden by the human scientists. Through several rounds of brainstorming and analysis, two drugs were identified as promising.<\/p>\n

Testing of Robin’s individual agents showed those that dug through earlier research were better at the task than general-purpose LLMs. The analytical agent did less well on questions about statistics and bioinformatics, and relied heavily on human-supplied prompts.<\/p>\n

\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\tThe limits of language alone<\/p>\n

AI can help scientists to navigate the vast amount of documented knowledge humans have acquired over the millennia. Use of computation to find patterns in large datasets, to integrate dispersed information, and to drive new discoveries from existing literature<\/a> has already contributed to scientific progress for decades.<\/p>\n

New models such as Robin and Co-Scientist represent a shift towards working directly in the realm of the language of science, rather than the realm of raw data. This allows more natural collaborations between scientist and machine, through language-based “discussions.”<\/p>\n

However, more natural doesn’t necessarily mean more effective. Language-based communication can be imprecise and ambiguous, where science must be specific.<\/p>\n

Models that combine the best of these worlds<\/a> are on the horizon. These aim to link structured quantitative data to the concepts and relationships that describe the core facts beneath it.<\/p>\n

Such models ground scientific reasoning in the structure of knowledge. They allow scientific evidence ranging from genomic sequences and protein structures to cellular imaging to be connected.<\/p>\n

Words are how science is communicated. AI tools that facilitate making sense of the information that is hidden in all of those words are surely valuable. But the complexity of the natural world means that AI (co-) scientists will only be truly effective when they can go beyond connecting words together, to modeling the full complexity of the systems those words describe.<\/p>\n

\t\t\t\t\t\t\t\t\t\t\t\t\t\tPublication details\t\t\t\t\t\t\t\t\t\t\t\t\t<\/p>\n

Juraj Gottweis et al, Accelerating scientific discovery with Co-Scientist, Nature (2026). DOI: 10.1038\/s41586-026-10644-y<\/a><\/p>\n

Ali Essam Ghareeb et al, A multi-agent system for automating scientific discovery, Nature (2026). DOI: 10.1038\/s41586-026-10652-y<\/a><\/p>\n

\n\t\t\t\t\t\t\t\t\t\t\t\t\tProvided by
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\tThe Conversation<\/a>
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/p>\n

\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/p>\n

\n Who’s behind this story?\n <\/p>\n


\n \"Lisa
\n <\/a><\/p>\n

Lisa Lock<\/a><\/p>\n

\n BA art history, MA material culture. Former museum editor, paramedic, and transplant coordinator. Editing for Science X since 2021.
\n
\n Full profile \u2192
\n <\/a>\n <\/p>\n


\n \"Robert
\n <\/a><\/p>\n

Robert Egan<\/a><\/p>\n

\n Bachelor’s in mathematical biology, Master’s in creative writing. Well-traveled with unique perspectives on science and language.
\n
\n Full profile \u2192
\n <\/a>\n <\/p>\n

\n\t\t\t\t\t\t\t\t\t\t\t\t This article is republished from The Conversation<\/a> under a Creative Commons license. Read the original article<\/a>.\"The\n\t\t\t\t\t\t\t\t\t\t\t <\/p>\n

\n\t\t\t\t\t\t\t\t\t\t\t\tCitation:
\n\t\t\t\t\t\t\t\t\t\t\t\tNew ‘AI scientists’ are improving\u2014but reveal their fundamental limits (2026, May 24)
\n\t\t\t\t\t\t\t\t\t\t\t\tretrieved 24 May 2026
\n\t\t\t\t\t\t\t\t\t\t\t\tfrom https:\/\/phys.org\/news\/2026-05-ai-scientists-reveal-fundamental-limits.html\n\t\t\t\t\t\t\t\t\t\t\t <\/p>\n

\n\t\t\t\t\t\t\t\t\t\t\t This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
\n\t\t\t\t\t\t\t\t\t\t\t part may be reproduced without the written permission. The content is provided for information purposes only.\n\t\t\t\t\t\t\t\t\t\t\t <\/p>\n","protected":false},"excerpt":{"rendered":"Credit: Unsplash\/CC 0 Public Domain Many of the most exciting discoveries in science involve highly specialized knowledge and…\n","protected":false},"author":2,"featured_media":49949,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[24,25,165,166,164,161,160,162,134,163],"class_list":["post-49948","post","type-post","status-publish","format-standard","has-post-thumbnail","category-ai","tag-ai","tag-artificial-intelligence","tag-materials","tag-nanotech","tag-physics","tag-physics-news","tag-science","tag-science-news","tag-technology","tag-technology-news"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/49948","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=49948"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/49948\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/49949"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=49948"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=49948"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=49948"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}