AI diagnosis tools are beating doctors on rare diseases while raising serious concerns about real-world reliability and clinical decision-making

Some researchers argue that this represents a profound change in technology that will reshape medicine, especially for cases where the correct diagnosis is not obvious even after extensive evaluation.

Article continues below

You may like

“We’re witnessing a really profound change in technology that will reshape medicine,” Arjun Manrai of Harvard University said in a news conference.

Still, serious questions remain about whether these systems can handle the full weight of real-world clinical uncertainty.

In one major study, researchers tested a leading AI reasoning model on a mix of textbook-style cases and real patient data from a Boston emergency department.

The model analyzed step-by-step descriptions of symptoms, test orders, and results, just as clinicians do.

Sign up to the TechRadar Pro newsletter to get all the top news, opinion, features and guidance your business needs to succeed!

It listed possible diagnoses more often than human doctors and included the true diagnosis, or something very close to it, in about 80% of the difficult cases.

For one transplant patient with subtle signs of a life-threatening infection, the model raised appropriate suspicion roughly a day before the clinical team did.

Researchers say the technology is particularly strong at scanning broad patterns across rare diseases that individual doctors may rarely encounter.

What to read next

However, the studies rely on curated patient descriptions rather than raw, chaotic emergency room environments.

The models respond to the information they are given, not the full mess of overlapping priorities and incomplete data seen in real clinics.

Despite the capacity of these AI reasoning models, critics point out that clinical reasoning is more than just step-by-step logic on a clean text summary.

“When we say clinical reasoning, it doesn’t mean the same thing as model reasoning,” says Arya Rao of Harvard Medical School, who was not involved in the study.

“These models have been optimized to do this kind of sequential thought that we call reasoning, but it’s not at all the same thing as how we teach medical students to reason.”

Doctors often have to entertain multiple uncertain possibilities at once, then update them as new data arrive.

AI models tend to latch onto a single strong explanation and update their answers in brittle ways when new facts appear.

One team that tested 21 different AI systems found that even the best reasoning models struggled when considering several uncertain diagnoses at the same time.