AI trained to ‘think’ like human pathologists may be better at spotting cancer

Artificial intelligence (AI) algorithms that hunt for cancer may do a better job when they approach the analysis as if they were a human pathologist, a new study suggests.

Many AI systems analyze preselected regions of a tissue sample, or they split a whole pathology slide into patches of a fixed size. By contrast, a pathologist searches more dynamically, panning across the tissue, zooming in and out, and pausing over areas that raise red flags. A whole slide can contain billions of pixels, while the evidence of cancer may occupy only a tiny patch.

Study co-author Zhi Huang, an assistant professor of pathology and laboratory medicine at the University of Pennsylvania, compared the process to a search-and-rescue helicopter. “You don’t start by inspecting one square meter of ground,” Huang told Live Science. You scan the landscape first and then swoop in for a closer look.

In the new study, published in July in the journal Nature, Huang and colleagues demonstrated that cancer-detecting AI might work better when it takes this humanized approach.

Training AI to hunt for cancer

AI algorithms called vision language models (VLMs) struggle with the first step that Huang described — that initial, cursory scan. That’s in part because many pathology AI systems learn from what pathologists leave behind at the end of that search: a labeled image pointing out where the cancer is or an official diagnosis.

Instead, the researchers trained their new AI on pathologists’ search behavior. They called this approach to training “Pathology-CoT,” short for “chain of thought.” It turns observable actions, including where pathologists move around and zoom in on an image, into training data.

To collect the data, the team created a tool that recorded how pathologists moved around a slide and changed magnification. The raw logs, gathered from eight pathologists, were messy, as a given pathologist might drift across a slide, overshoot their intended region of focus or fiddle with magnification to adjust it to their liking.

To clean up the data, the researchers filtered out those incidental movements, focusing on moments that appeared to represent deliberate attention, such as lingering over one view or making a sustained pan. Then, they compared those regions with eye-tracking data to check that the software was capturing where pathologists were actually looking.

For each region a pathologist inspected, the VLM also drafted a short rationale explaining why the region was worth examining and what features were visible; human pathologists could then accept, edit, or reject the rationale, creating additional training data for the AI. In one example, the AI flagged a portion of a slide as potentially metastatic and suggested zooming in to look for atypical cells. Other inspected regions were flagged as healthy tissue.

Ultimately, the researchers used this training method to build a new tool called Pathology-o3. It scans a slide at low resolution, uses a model trained on pathologists’ behavior to choose regions worth a closer look, then sends higher-resolution views of those regions to a VLM for analysis.

Putting it to the test

Huang said the goal of the new study was not to show that Pathology-o3 worked better than specialized AI models that are specifically built to detect specific types of cancer; those models are often trained disease by disease. Rather, the researchers wanted to see whether their new training approach could help a general-purpose AI navigate a pathology slide more effectively.

They compared Pathology-o3 to other general-use AI systems, such as OpenAI’s o3, and asked the algorithms to examine slides containing lymph node tissue. These slides were collected from colorectal cancer cases and some contained metastatic cancer, which human pathologists had already labeled.

The algorithm correctly identified slides that were positive for cancer 100% of the time. However, of the slides it identified as positive, 15.5% were actually negative. By comparison, OpenAI o3 correctly identified slides that were positive for cancer 87.5% of the time. Of the slides it identified as positive, 53.3% were actually negative.

The researchers designed Pathology-o3 to err on the side of flagging something for another look, rather than potentially missing cancer. That might help to explain the rate of false positives, Huang said.

Whether that rate of false alarms is acceptable depends on how Pathology-o3 is used, said Mohammad Asadi, a data scientist at Stanford University who was not involved in the research. It’s not precise enough for the AI to diagnose patients on its own, but it could still be useful for a system to point a human toward regions of a slide that are worth double-checking. It may be an advantage that the approach shows the pathologist a specific region to inspect rather than declaring an entire slide suspicious, he said.

The researchers tried repeating the test on an independent dataset that the algorithms hadn’t seen before to see how well it worked on unfamiliar slides. Pathology-o3 correctly identified slides that were positive for cancer 97.6% of the time. Of the slides it identified as positive, 37.1% were actually negative. The finding is an example of how AI performance can change when the data source changes, even when the medical task stays the same.

Asadi explained this result suggests the system can still work with slides from a different source. But that result does not yet show that using this tool would make pathologists more accurate or efficient in practice.

Can it help pathologists?

The researchers applied their training approach to several existing VLMs, finding that the models’ performance consistently improved after the training. That suggests that the navigation data from pathologists was useful across settings, Asadi said.

For Huang, that is the most important result. “The takeaway isn’t our system,” he said. “It’s that the missing ingredient has been sitting in hospitals this whole time.”

The study did not compare Pathology-o3 directly with human pathologists, but the researchers said that wasn’t their aim.

“The right question isn’t whether it beats a pathologist,” Huang said. “It’s whether a pathologist working with it catches more [cancer cases] and works faster.” The current study did not address the latter question, either, but the team’s next experiment is designed to test pathologists on the same cases with and without Pathology-o3, measuring what they catch and how long they take to do so.

The system’s most plausible use is as a prescreening tool, Asadi said, but he stressed that the research has not yet shown that doctors who use it become faster or more accurate. Asadi wants an even tougher test: trials conducted across multiple hospitals that measure not just accuracy and speed but also pathologists’ workloads. He wants the trails to assess the burden of false alarms from the AI algorithms and whether doctors recognize when the AI is wrong.

Importantly, cancer diagnoses can require information from multiple slides, stains and a patient’s medical history, while the current system just reads one slide at a time. “I wouldn’t claim it should diagnose on its own,” Huang said.