Google Releases AI Tool for Pathology

by Jondavid Klipp | Apr 15, 2024 | AI, Digital Pathology

Google Releases AI Tool for Pathology

Google has released a free cloud-based tool (“Path Foundation”) that
transforms digitized slide images into numerical data that researchers
can use to create AI tools for pathology. “This is a landmark for us,” says
David Steiner, MD, PhD, Clinical Research Scientist at Google. Based on feedback to Path Foundation, Google will make decisions toward developing commercial AI tools for pathologists.

For more than seven years, Google has employed a team of clinical research scientists based in Mountain View, California and London, England, tasked with developing AI software tools for radiology and pathology. Below is a summary of LE’s interview with Google’s David Steiner, MD, PhD, Clinical Research Scientist.

How can labs and pathologists use Path Foundation?

Commercial labs and academic medical centers, for example, can use Path Foundation to convert patches of their digitized slide images into numerical vectors, known as embeddings. These embeddings capture important features and patterns contained in digitized slide images which are learned by Path Foundation during its training on millions of pathology images. After the embeddings for each image are collected, this data can be used to train and create custom algorithms for a range of tasks such as identifying tissue type, tumors, or performing quality assurance on digitized images.

What’s the benefit of using a “Foundation” model to analyze content in images?
Leveraging the embeddings from Path Foundation requires less data and computational resources than traditional methods, giving researchers a big head start toward developing their own algorithms.

In contrast, traditional methods such as strongly supervised deep learning models require more resources because every task requires fresh training in its own unique deep learning model and many labeled images for each category of interest.

The foundation model is similar to that of seasoned guitar player quickly learning a new song by ear. Because the guitar player has already built up a foundation of skill and understanding, they can quickly pick up the patterns and groove of a new song.

Where did you get the digitized images to train Path Foundation on?
We used 20,000 whole-slide images, covering millions of image patches, from a variety of sources, including the National Cancer Institute’s Cancer Genome Atlas, as well as academic medical centers and some private pathology labs.

What is the user roadmap to getting started with Path Foundation?
After filling out the access form, users can run a small demo notebook that walks them through how to train a tumor classifier. To use Path Foundation on their own task (i.e., identifying tissue type) they would need to collate a set of digital pathology images with accompanying labels. The user would then upload these images and labels into Google Cloud.

From there, the users can adapt the demo notebook to call the Path Foundation API on their uploaded images. Using the existing code in the demo notebook they would then train a model to classify their images based on the labels and evaluate its performance on a held-out part of
the dataset not used to train the model. In the future we hope to make it even easier to use Path Foundation and the embeddings directly in your pathology slide viewer with no coding required.

We’re offering Path Foundation free to users on github.com/Google-Health.

Do labs that use Path Foundation need to share their digitized images or the results of their research with Google?
They don’t need to share the images or the results of their research with Google. They do need to have their images stored in Google Cloud (in their own private or institution account), but these are kept private to the user and not accessible by Google. In addition, when the images are sent to the Path Foundation model to compute the embeddings, they are not saved or stored by Google.

Any plans to develop commercial AI software tools like Ibex, Paige or PathAI?
Yes. Path Foundation represents a landmark toward that end.

What is the next step for your research team in terms of developing AI tools for pathologists?
We’ll take feedback on users from Path Foundation to understand key-use cases and how to make the tool better and easier to use. We’ll also explore how these embeddings might be used with large language and large multimodal models (LLMs & LMMs). And then we plan to develop useful approaches and models for working with whole slide images (WSIs), in addition to the “patch-based” models and applications that this current tool represents.

Has Google completed any studies related to AI-assisted diagnostic tools for cancer?
Yes. We have published several studies in peer-reviewed journals.

Most recently, we published a study that used AI to predict immunotherapy outcomes from digitized slide images in non-small cell lung cancer (Cancer Research, vol. 84, 2024).

In 2023, we published a paper that showed how AI can be used for clinical decision-making in colorectal cancer (Nature Communications Medicine, vol. 3, 2023). And, in 2022, we published a study that used AI for diagnosis and Gleason Grading of prostate cancer (Nature Medicine, vol. 28, 2022).

We have also published studies focusing on AI models for breast cancer.

Will AI algorithms eventually replace pathologists?
AI will make pathologists better rather than replace them. Initially, AI will be used to automate repetitive tasks such as locating the image patches that pathologists should focus their eyes on. Eventually, AI could be used to query images. Pathologists and researchers may someday be able to
type in specific questions and get AI answers about an image.

Is there the potential to integrate pathology and radiology image data using AI?
I’m excited to explore this and do see the promise in bringing these two specialties together. I believe the combination will yield more than the sum of the parts. Google does have a team of research scientists working on AI tools for radiology. And we did introduce an AI tool for chest x-rays (CXR Foundation) in July 2022.

BioReference Labs Goes Live With Digital Pathology Plus AI

by Jondavid Klipp | Feb 15, 2022 | AI, Artificial Intelligence, BioReference Labs, Digital Pathology

BioReference Labs Goes Live With Digital Pathology Plus AI

OPKO’s BioReference Labs (Elmwood Park, NJ) went live in December with new whole-slide imaging scanners from Leica Biosciences (Buffalo Grove, IL). The scanners have been integrated with a digital pathology solution, PathFlow, made by Gestalt Diagnostics (Spokane, WA). PathFlow is a cloud-based software system that has helped integrate BioReference’s LIS, workflow and scanned slide images with artificial intelligence algorithms developed by MindPeak (Hamburg, Germany). BioReference is also using PathFlow for slide image management and archival storage.

BioReference is using MindPeak’s AI tool (named BreastIHC) to detect and quantify breast cancer cells from digitized slide images with immunohistochemistry at its main laboratory in northern New Jersey. Pathologists can access their case and slide images securely on their computer monitors and use their mouse to outline regions of interest (ROI). All cells within this outlined ROI are instantly classified into positively stained tumor and unstained tumor cells. The panel of algorithms include five key tumor markers (ER, PR, Ki-67, HER2, and P53) which can be counted and scored.

Eventually, Dan Roark, Chief Executive Officer, Gestalt Diagnostics, expects the AI algorithms to both automatically identify the regions of interest in addition to performing IHC marker positivity scoring.

Digital Pathology & AI Market Growth
Separately, Roark says that after more than 10 years of limited adoption, digital pathology is finally starting to take off in the clinical market in the United States. Whole slide scanners have gotten quicker and less expensive. For example, it used to take 8-10 minutes to scan a slide but now takes as little as 30 seconds. But the biggest driver is the pathologist efficiency gains obtained when AI is applied to digitized slides. “The number of RFP requests we receive is exploding,” says Roark.

Worldwide Opportunities for U.S.-Based Pathologists
The U.S. has more working pathologists per capita than most other countries. For example, there are approximately 20,000 actively practicing pathologists in the United States, according to the American Medical Association. This works out to a ratio of one pathologist for every 17,000 people.

Alverno Labs To Implement Artificial Intelligence For Pathology

by Jondavid Klipp | Aug 15, 2021 | AI, Alverno Labs, Digital Pathology

Alverno Labs To Implement Artificial Intelligence For Pathology

Alverno Laboratories (Hammond, IN), which transitioned to digital pathology in 2019 when it implemented Philips IntelliSite Pathology Solution, now says it will add artificial intelligence to aid its pathologists in cancer diagnosis. Alverno will use the Galen AI system developed by Ibex
Medical Analytics (Tel Aviv, Israel). Ibex’s Galen platform recently received the Breakthrough Device Designation from FDA, which will help fast track the clinical review and clearance process, and is CE marked in Europe for
breast and prostate cancer. Alverno is an independent lab owned by Franciscan Alliance and AMITA Health. Alverno manages a core lab and 32 hospital labs in Indiana and Illinois. It consults on 150,000 histological cases per year, which translates to more than 1.1 million tissue slides.

Spotlight Interview With Paige CEO Leo Grady

by Jondavid Klipp | Jul 15, 2021 | AI, Artificial Intelligence, Paige

Spotlight Interview With Paige CEO Leo Grady

Paige (New York, NY) was founded by pathologists and scientists from Memorial Sloan Kettering Cancer Center in 2017. Paige develops artificial
intelligence (AI)-based systems that help diagnose cancer. In early 2019, the
company hired Leo Grady, PhD, as Chief Executive Officer and Board member. He had previously been Senior Vice President of Engineering at HeartFlow (Redwood City, CA), where he led development efforts for HeartFlow’s 3D modeling software for coronary artery disease. He received a B.S. degree in Electrical Engineering at the University of Vermont and a PhD in Cognitive and Neural Systems from Boston University. Below is a summary of Laboratory Economics’ interview with Dr. Grady in late June.

Who founded Paige?
The intellectual property related to the AI-based computational pathology used by Paige was initially developed by Thomas Fuchs, PhD, while he was Director of Computational Pathology at The Warren Alpert Center for Digital and Computational Pathology at Memorial Sloan Kettering (MSK). Fuchs co-founded Paige with David Klimstra, MD, Chairman of the Department of Pathology at MSK, in 2017. Fuchs is Chief Scientific Officer at Paige. Klimstra will become our Chief Medical Officer effective August 1.
Paige currently has more than 100 employees, mostly in the United States, with a small, but growing, presence in the United Kingdom and Europe.

Where did Paige get the annotated pathology slides needed to develop its AI algorithms?
As part of spin-out from MSK, Paige signed a comprehensive license agreement, giving it exclusive access to the hospital’s archive of 25 million annotated pathology slides and its intellectual property in computational pathology. To date, Paige has digitized more than five million slides from the MSK archive.

How much capital has Paige raised?
We’ve raised a total of $220 million to date, including $125 million raised in January from a series C financing led by Casdin Capital, Johnson & Johnson Innovation (JJDC) and KKR. Paige’s largest shareholders also include MSK, Breyer Capital, Goldman Sachs and Healthcare Venture Partners.

Where do Paige’s software programs stand with the FDA?
In early 2019, Paige received an FDA breakthrough designation for its software program for the automated detection of cancer in prostate biopsies. FDA clearance is expected to start with prostate cancer and then expand to additional cancers. In addition, in December 2020, the company obtained two CE marks for software aimed at breast and prostate cancers, including the ability to rate tumor samples, deliver a prognosis and guide treatment planning. Finally, Paige’s digital pathology image viewer, FullFocus, received FDA clearance in July 2020. The clearance allows for use of FullFocus with the FDA-authorized Philips Ultra Fast Scanner and paves the way for use with additional scanners in the future.

How accurate are Paige’s AI-based software programs?
A study recently published in the Journal of Pathology showed that Paige Prostate had 100% sensitivity and 100% negative predictive value (NPV) at the patient level when analyzing 661 prostate needle biopsy slides from 100 consecutive patients in a real-world setting.

The study took place at Grupo Oncoclinicas (São Paulo, Brazil), which is the largest private provider of cancer care in Latin America and was the first institution in the world to fully deploy Paige digital and computational pathology products for routine use. (See Independent Real-World Application of a Clinical-Grade Automated Prostate Cancer Detection System, Journal of Pathology, June 2021.)

How about pathologist productivity gains?
For the Grupo Oncoclinicas study, Paige Prostate generated binary predictions, benign or suspicious for cancer. A benign prediction prompted no further action by a pathologist, whereas a classification of suspicious would prompt pathologist review and/or additional IHC to confirm the presence of a malignancy. Given its high sensitivity and NPV, and specificity
of 0.78, Paige Prostate showed the potential to be used as a screening tool that flags suspicious slides needing pathologist review. Given that roughly 80% of prostate biopsy slides are negative, screening with AI could provide huge gains in pathologist productivity. Ultimately, the medical community will decide how best to use AI.

Won’t digitizing slides and performing AI analysis disrupt workflow and slow turnaround?
The Grupo Oncoclinicas study showed that Paige Prostate could improve efficiency by an estimated 65.5%. The use of AI allowed the pathologists to focus their microscope time on those slides most likely to contain cancer. It can also save time by identifying specimen samples requiring recuts and/or additional staining prior to being viewed by a pathologist.

Will AI products like Paige Prostate speed the transition to digital pathology?
The adoption of digital pathology to date has moved very slowly. We estimate that only 5% of pathology slides in the U.S. are currently being digitized. Return on investment (ROI) has been the biggest obstacle. The transition to digital pathology requires significant investment in terms
of capital expense, workflow changes, floor space, and image storage. Without added reimbursement, it’s hard to make a business case for digitizing slides. However, AI-based tools that raise pathologist productivity provide an ROI for going digital.

Why is Europe ahead of the United States in terms of digital pathology adoption?
The regulatory process for digital pathology in Europe was quicker and there is a greater shortage of pathologists. Even so, the majority of pathology cases in Europe are still interpreted using traditional light microscopes.

Paige recently announced some contracts with big commercial pathology labs.
Yes. Under our new agreement with Quest Diagnostics, Paige’s proprietary AI tools will analyze digitized slides from Quest and its AmeriPath and Dermpath businesses to develop new software products for diagnosing cancer and other diseases. The collaboration will initially focus on solid tumor cancers, such as prostate, breast, colorectal and lung. Assuming regulatory clearance, Quest plans to use approved software products in its pathology operations.

In the near term, the collaboration also intends to license the insights to biopharmaceutical and research organizations to aid biomarker discovery, drug research and development and companion diagnostics.

Separately, Inform Diagnostics (Irving, TX) has agreed to immediately start using Paige’s FullFocus digital pathology viewer as well as our data management system for storing digital pathology slides.

How will the role of pathologists evolve over the next 5-10 years?
I expect there’ll be less microscopy-based work in a more distributed model. But the role and visibility of pathologists may get elevated due to digital pathology and AI. Digital pathology images allow pathologists to communicate more visually with ordering physicians, while AI will increase their diagnostic accuracy thereby providing more value to physicians and patients.

CorePlus Details Its Use Of Artificial Intelligence For Prostate Cancer

by Jondavid Klipp | Oct 15, 2020 | AI, Artificial Intelligence, CorePlus

CorePlus Details Its Use Of Artificial Intelligence For Prostate Cancer

Last month, LE briefly noted that CorePlus Servicios Clínicos y Patológicos LLC (Carolina, Puerto Rico) had become the first independent lab in the Americas to begin using artificialintelligence-assisted (AI) pathology for prostate cancer diagnostics. This month wegot in touch with CorePlus President Mariano de Socarraz to find out more.

Can you describe CorePlus?
We opened our CLIA-certified laboratory in Carolina, Puerto Rico in 2008. We currently have 115 employees, including four pathologists. CorePlus is full-service independent laboratory. Among our specializations is uropathology. We process approximately 3,000 prostate cancer cases (~36,000 slides) per year, representing more than half of all outpatient prostate cancer biopsies performed in Puerto Rico.

Is operating a lab in Puerto Rico different than in mainland United States?
No. Puerto Rico is a U.S. territory that must follow all federal lab regulations, including CLIA. Medicare and Medicaid insurance cover the majority of the 3.2 million people living in Puerto Rico and the biggest private insurer is Triple-S, which is an independent licensee of the Blue Cross Blue Shield Association. The biggest difference is probably reimbursement rates, which are substantially lower in Puerto Rico.

Among the competing clinical labs in Puerto Rico are Laboratorio Clinico Toledo and Laboratorios Borinquen. Anatomic pathology labs include Hato Rey Pathology and Puerto Rico Pathology. Quest Diagnostics has had a reduced presence following the damage to its lab facilities from Hurricane Maria in 2017. LabCorp transports specimens to its labs in Florida.

When did CorePlus transition to digital pathology?
We began digitizing slides using 3DHISTECH scanners in mid-2019. By late 2019 we had completed validation and by early 2020 our pathologists were reading digitized images for all our pathology cases, including all routine histopathology and stains.

What type of computer screens do your pathologists read the digital slide images from?
CorePlus validated the Dell UltraSharp 49 Curved Monitor – U4919DW. It’s a high-end, business grade monitor with a Delta E of <2 (color difference perception) and an aspect ratio of 32:9:0. This aspect ratio is the equivalent of two 27-inch monitors running at 2K.

And how did you get involved with AI-assisted pathology?
In August 2018, I read about a validation study conducted by University of Pittsburgh Medical Center which used an AI-based algorithm to detect and characterize prostate cancer from digitized slides. This study [recently published in The Lancet Digital Health] showed that an AI-based algorithm demonstrated 98% sensitivity and 97% specificity at detecting prostate cancer from 1,600 different tissue slide images that had been collected from 100 patients seen at UPMC who were suspected of having prostate cancer. It even spotted six potentially malignant slides that expert pathologists had failed to identify initially. This interested me, so I contacted the company that developed the algorithm, Israel-based Ibex Medical Analytics. We ran our own validation studies on 1,301 digitized prostate tissue slides and found results similar to those at UPMC. Overall accuracy was 99.4% with 96.9% specificity and 96.5% sensitivity.

How have you integrated AI into your pathology lab?
Our pathologists continue to read digitized images for every prostate tissue slide prepared by our lab. But starting in June, we also began sending digitized images of each slide to the Ibex cloud. Ibex runs its AI-based algorithm on each slide which provides 100% quality control on all prostate cases. This serves as a digital second opinion for our pathologists.

What happens when there is a discrepancy between the pathologist’s exam and the algorithm?

The pathologist goes back and reviews the slide(s) and/or orders an immunohistochemistry. I believe that we have reduced the potential for a misdiagnosis on prostate cancer biopsies to much less than 1%. This is significant given that even an expert uropathologist can miss 3%. So the AI
algorithm is acting as a failsafe that is catching cases that might otherwise be missed.

Was there any reluctance from your pathologists as you transitioned to digital pathology and AI for prostate?
The pathologists were always fully engaged in the transition. Our pathologists say they would never go back to the microscope, especially given their ability to read digitized slides at home during the pandemic. We have analyzed over 1,000 prostate biopsy cases using digital pathology with AI assistance to date. In real world practice it has helped identify lesions that would otherwise have been missed.

Will you apply AI-based algorithms to other cancers?
Yes, we are planning to start using an Ibex algorithm for second reads on all breast cancer cases within the next few weeks.

How does your lab get compensated for using digital pathology and AI to improve accuracy?
We do not get additional compensation and that is the problem with the current CPT-based feefor-service reimbursement model. AI increases accuracy and reduces utilization of immunohistochemistry and there ought to be some coding mechanism that fairly compensates labs that use it.
In the meantime, the increased efficiency that the combination of digital pathology and AI provides has helped offset the initial technology investment and development cost. In addition, the increased accuracy at CorePlus through its use of AI should lead to more clients. Knowing that 100% of prostate cancer cases sent to CorePlus are getting an AI second opinion should raise urologists’ confidence in our lab.

How will AI affect the practice of pathology over the long term?
After our current use of digital pathology and AI as a second read tool, I anticipate it will progress to be used as a triage tool and finally for primary reads with the supervision of a pathologist. The role of pathologists will evolve away from time at the traditional microscope toward selecting the
right AI algorithm to apply to a digitized slide and reviewing results in combination with a patient’s medical record to form a diagnosis.

Switching gears, is CorePlus performing Covid-19 PCR testing?
We started Covid-19 PCR testing on the Roche cobas 6800 platform in late April. CorePlus has been on an allocation of seven kits per week (equal to 1,344 tests). To compensate for the test reagent shortage, we began pooled testing for three specimens at a time in July. This has expanded our capacity to about 4,000 tests per week and we are preparing to increase our pool size to six specimens, which will double our capacity to 8,000 tests per week.

How do you see the Covid-19 pandemic ending?
It is not going away any time soon, even with a vaccine. Population immunity may take years.