{"id":146927,"date":"2026-08-21T04:52:21","date_gmt":"2026-08-21T04:52:21","guid":{"rendered":"https:\/\/www.europesays.com\/ai\/146927\/"},"modified":"2026-08-21T04:52:21","modified_gmt":"2026-08-21T04:52:21","slug":"protecting-scientific-intent-in-ai-enabled-labs-by-not-letting-ai-set-the-mission","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ai\/146927\/","title":{"rendered":"Protecting Scientific Intent in AI-Enabled Labs by Not Letting AI Set the Mission"},"content":{"rendered":"

\"ai<\/a>Forget self-driving labs. For scientific AI to thrive, humans must always be in charge, setting objectives, constraints, risks, and interpreting results. [Maite Mueller\/Getty Images]<\/p>\n

AI is neither saint nor demon. Nor should it be a replacement for human scientists. As AI takes on greater roles in designing, executing, and analyzing experiments and processes, scientists understand that even the best AI needs human supervision.<\/p>\n

The big question is how much oversight is needed and whether\u2014or the extent to which\u2014AI interactions should be documented and reported in regulatory filings.<\/p>\n

\"LeLe Cong, PhD, associate professor, Stanford University, and co-founder of LabOS and MedOS<\/p>\n

Although the lab of the future may be envisioned as a self-driving lab, that\u2019s actually a bad idea, noted Le Cong, PhD, associate professor, Stanford University, and co-founder of LabOS and MedOS. Instead, he and leaders in the AI and biopharmaceutical industries see the future of scientific AI as agentic, with humans in charge.<\/p>\n

\u201cWe think there is a positive trend toward using AI in a way that\u2019s human, rather than as a self-driving lab,\u201d he said. In that environment, AI is simply a tool\u2014albeit a powerful, adaptive one\u2014that can be managed as long as scientists use the right prompts.<\/p>\n

Humans in the lead<\/p>\n

\u201cToday, much of the scientific research process remains inaccessible to machines,\u201d Cong and colleagues wrote in a recent paper<\/a>. Despite automation and some use of AI, \u201cScientific discovery remains fragmented.\u201d Specifically, AIs lack the tacit knowledge, evolving experimental context, human observations, and adaptive decision-making inherent in human scientists.<\/p>\n

Human involvement is needed, therefore, not just to oversee AI-based activities and check the output, but to ask the right questions and to ensure that analyses make sense in context. Specifically, he describes a scientific setting in which an AI would handle an experiment\u2019s execution, and the scientists would be responsible for:<\/p>\n

Framing objectives
\nInterpreting results<\/p>\n

Setting constraints
\nGoverning risks<\/p>\n

\u201cIf AI can interpret everything, then it will start to generate fake stuff, right?\u201d Cong asks. \u201cWe\u2019ve seen this when AIs begin guessing in an effort to return results and supply citations that don\u2019t exist. There are certain things that are useful for AI to do in the lab.\u201d<\/p>\n

But, as last summer\u2019s sandbox breakouts illustrated, an AI needs firm guidelines as to what it can do, where it can access information, and the degree of autonomy it has in meeting a request.<\/p>\n

For example, he recommends adding this phrase to instructions: \u201cAny actions not explicitly stated in the protocol need human approval.\u201d That default to human judgment also should apply to determining the risks associated with certain actions, such as editing a human gene, Cong said. With those guardrails, the paper points out, agentic AIs can freely handle \u201croutine execution and coordination across models, instruments, protocols, and laboratory states.\u201d<\/p>\n

Cong equates the scientific use of AI to autonomous vehicles, which, according to the Insurance Institute for Highway Safety, have a 68% lower crash rate<\/a> per mile traveled than human drivers in the same environment. Given those statistics, he added, \u201cMy thought is to elevate humans to setting destinations. We do not need humans to always execute the driving.\u201d<\/p>\n

In a university scientific lab, that equates to staffing a principal investigator and trainees, without much of the hierarchy that exists today. In a corporate environment, the hierarchy flattens to scientists who propose, design, execute, and interpret experiments, and a lab manager. The distinctions between senior and junior scientists blur because much of the hands-on work is automated.<\/p>\n

The combination of AI and lab automation is expected to reduce human errors. Cong cited a 2016 Nature study<\/a> of 1,500 scientists. When asked, \u201c\u2019Can you replicate other people\u2019s experiments, and can you replicate your own after a few months?\u2019 approximately 70% could not replicate others\u2019 experiments, and half could not replicate their own!\u201d Cong said. \u201cAI can improve that.\u201d<\/p>\n

\"autonomousCong equates the scientific use of AI to autonomous vehicles, which, according to the Insurance Institute for Highway Safety, have a 68% lower crash rate per mile traveled than human drivers in the same environment. Given those statistics, \u201cMy thought is, elevate humans to setting destinations,\u201d said Cong. \u201cWe do not need humans to always execute the driving.\u201d [Chesky_W\/Getty Images]Reasons for such poor reproducibility rest in the details, he elaborated. Was a step omitted? Was the protocol followed exactly? Is the protein being used identical to the one in the original experiment? Were the temperatures the same? Details like this\u2014which AI can duplicate precisely\u2014are behind many reproducibility challenges.<\/p>\n

AI risks minimal<\/p>\n

As yet, it\u2019s unclear how AI involvement in experiments should be preserved and reported in regulatory submissions, Cong continued. \u201cWe\u2019re still early in this journey.\u201d<\/p>\n

That said, the risk that AI will escape its constraints and cause physical harm\u2014like designing and developing a physical virus\u2014appears relatively low, according to Cong. That\u2019s because a rogue AI still needs a human accomplice to allow a virus, for example, to be manufactured and released. \u201cIn areas where there is a physical execution step, I think AI is still incapable,\u201d he said, \u201calthough we are seeing progress in connecting AI to biomedical labs and applications, and the physical execution layer.\u201d<\/p>\n

That\u2019s due to the fact that there are multiple layers of human intervention needed to actually manufacture a product. Aside from logistics, he cites good manufacturing practices, safety and efficacy regulations, and digital safeguards like track and trace and the FDA\u2019s 21 CFR Part 11, as well as real-time monitoring, periodic inspections, and quality control activities. Those regulations and checkpoints should also be sufficient to manage variations that occur during manufacturing as real-time conditions drift from specifications.<\/p>\n

The catch, as last summer\u2019s breakouts of frontier AIs underscore, is that sometimes AIs exceed their parameters. Whether there is sufficient appreciation of this among AI users remains to be seen, Cong said.<\/p>\n

\u201cA lot of people are connecting AI systems, which have access to more and more information and key decision-making systems,\u201d Cong pointed out, without deeply understanding the risks and establishing appropriate guardrails. \u201cPeople might be overly trusting of AI, perhaps.<\/p>\n

\u201cThe more powerful the AI, the more capable it is of doing something. Are people keeping pace [with the technology and its risks]?\u201d<\/p>\n

Sometimes, small, highly specific AIs may be a better choice than always leveraging the large frontier models, Cong suggested. The reason, Cong, senior corresponding author Mengdi Wang, PhD, professor, Princeton University, and a dozen colleagues, noted in a 2025 paper<\/a> in Nature Biomedical Engineering, is that \u201cLarge language models often lack domain-specific knowledge and struggle to accurately solve biological design problems.\u201d<\/p>\n

Whatever level of AI is used, however, \u201cHumans need to be in the lead throughout the process,\u201d Cong stressed.<\/p>\n","protected":false},"excerpt":{"rendered":"Forget self-driving labs. For scientific AI to thrive, humans must always be in charge, setting objectives, constraints, risks,…\n","protected":false},"author":2,"featured_media":146928,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[24,25,512,47626,72202,66,47135,4792],"class_list":["post-146927","post","type-post","status-publish","format-standard","has-post-thumbnail","category-ai","tag-ai","tag-artificial-intelligence","tag-automation","tag-clinical-laboratory-technique","tag-good-manufacturing-practice","tag-news","tag-quality-control","tag-topics"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/146927","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=146927"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/146927\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/146928"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=146927"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=146927"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=146927"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}