Today the field of digital pathology can trace its roots to the end of the last century with the development of the “virtual microscope” by a combined team of researchers at the University of Maryland’s department of computer science and researchers in the department of pathology at Johns Hopkins University School of Medicine. The software needed for this developing technology was based on new computer science at the time, which was being employed for spatial indexing of satellite data.
But, similar to the development of genetic sequencing methods, the first whole-slide imaging (WSI) systems, developed in the late 1990s, were an inspired idea that were a bit ahead of the curve. The first whole slide imaging tools cost hundreds of thousands of dollars and the vast amounts of data produced strained the computational and data storage limits available at the time.
Fast forward more than 20 years, and the field has undergone monumental changes wrought by improvements in image capture technologies, cloud computing, and machine learning to the point where digital pathology and the analysis of WSIs has shown its promise to directly impact clinical decision making as a valuable tool for precision medicine.
Assistant Professor of Pathology
Harvard Medical School
According to Faisal Mahmood, assistant professor of pathology at Harvard Medical School and the Division of Computational Pathology at the Brigham and Women’s Hospital, there are three major drivers of the continuing development of digital pathology as both a research and clinical tool.
“First, the field of artificial intelligence is much more mature now. With that, there have been a number of breakthroughs in the past 10 years with convolutional neural networks. Computing resources are also more available. Everyone’s desktop has a graphics processing unit and the computing power required to run these algorithms, to train these algorithms, to test and play with them, is much cheaper,” Mahmood said. “And the third one is the penetration of imaging devices, scanning equipment. Scanning glass slides is a lot cheaper, and while storing those slides can be expensive, the cost is coming down very rapidly.”
The result is a digital gold rush of pathology images, as academic medical centers and commercial players alike look to develop massive digital biobanks to feed clinical studies and, more importantly to help train the algorithms needed to gain insights from tumor pathology simply not possible via manual means.
chair, Division of Clinical Informatics, Mayo Clinic
For instance, Paige.ai, the developer of the first FDA-approved digital pathology diagnostic for prostate cancer (see sidebar, p. 10) has more than 4 million (and growing) digital pathology images at its disposal via the company’s relationship with Memorial Sloan Kettering Cancer Center. And Mayo Clinic is embarking on a three-year program to digitize more than 5 million slides, according to Jason Hipp, chair of the new Division of Clinical Informatics in Mayo Clinic’s Department of Laboratory Medicine and Pathology.
As digital pathology comes of age, it is also reinvigorating the role of the pathologist, bringing them back to the center of the oncology care team, while also improving collaboration among pathologists, and moving the practice into the digital age, where complementary technologies like radiology and genomic testing already reside. Experts in the field say the real power in the future of digital pathology in the clinic is to analyze data from all three technologies, in silico, to provide highly individualized assessments of each patient’s cancer.
Automating scanning, improving clinical workflows
Implementing digital pathology requires adding a step to the established workflow—the physical act of scanning slides and converting the images to pixels. While adding a step may seem counterintuitive, implementing a digital pathology program within a health system can improve clinical workflows while also opening the potential of automating the slide scanning process.
Some of today’s slide scanning tools have the capacity to scan hundreds of slides and create the associated digital file of each. While a hospital technician can be tasked with loading and unloading slides on scanners, the process is time consuming and prone to human error. Faced with the prospect of scanning millions of slides for its ambitious digital pathology program, the Mayo Clinic has instead turned to automation technologies to help scale its operation. In late March, it signed a commercial agreement with Pramana, which provides what it dubs a Digital Pathology as a Service (DPaaS) solution to help create the Mayo digital pathology repository. Employing a robot and other automation technologies, the company says it can process more than 1,000 slides per day per each of its scanning clusters, which also leverage artificial intelligence (AI) to perform quality assessments of each slide.
“It is often misconstrued that the challenges with going digital for tissue and body fluid slides end with the purchase of a scanner,” said Prasanth Perugupalli, chief product officer at Pramana at the time the company launched its DPaaS service. “We realized that the greater pain and costs lie in the human capital needed to operate the scanners, which includes making the correct parametric selections and qualifying each whole slide image for any errors after the scanning is completed.” Hipp noted that not every hospital system will need to scan such a high volume of slides or have the financial resources to invest in such expensive systems—which can run into the hundreds of thousands of dollars. Instead, they may opt for smaller, single-slide scanners that can fit on a pathologist’s desktop, and cost around $20,000, to help them reap the benefits of faster diagnoses with digital files.
senior member in the Department of Anatomic Pathology, Moffitt Cancer Center
Moving from the physical world of glass slides to the digital world of pixels has a number of clinical advantages, noted Marilyn Bui, a senior member in the Department of Anatomic Pathology at Moffitt Cancer Center.
“Pathology slides are analog and it’s in your little silo. When they become digital, now you can annotate that data, and transcribe what you are seeing so others can understand what you were thinking. Even after you are done interpreting that, the data is recorded,” Bui explained.
And that digital image can be shared easily with colleagues without the time-consuming process of transferring physical slides for secondary consults or sending to pathologists working in a subspecialty. Getting additional interpretation is as simple as logging onto a computer and pulling up the slide’s image and supporting notes and data.
Freed from the restrictions of sharing glass slides, getting fast access to the digital pathology image provides greater opportunity for collaboration among pathologists and institutions to improve the quality of diagnoses—what Bui noted was the “low-hanging fruit” of the benefits of digital pathology. Other benefits, however, are still in their infancy within the clinic.
VP and General Manager of Oncology, Quest Diagnostics
“While there have been opportunities to apply digital pathology technologies for some time for secondary consultations, training and educational settings, and to review cases involving specialty stains, the ability to use digital pathology for a primary read is fairly new,” noted Kristie Dolan, vice president and general manager of oncology at Quest Diagnostics. “Today, the industry is early in the journey, and the percentage of slides read digitally for primary diagnosis is relatively low.”
Leveraging artificial intelligence
AI’s strength in object recognition and pattern classification seems tailor made for digital pathology. As the practice of oncology continues to move in the direction of precision therapies and the increased use of immunotherapies, AI’s ability to autonomously recognize and count specific cell types within the spatial context of a tissue or histology slide image is providing yet another tool for tailoring treatment for individual patients.
“The most important role for digital pathology is to give rise to AI. It can be used to train certain operations, and it will come back to benefit you in diagnostic efficiency, accuracy, and also biomarker scoring and interpretation,” said Bui. “Digital pathology by itself without AI, would never really take off.”
Professor of Pathology and vice chair of Anatomical Pathology
The Ohio State University
As Anil Parwani, professor of Pathology and vice chair of Anatomical Pathology at The Ohio State University noted: “We now have biomarkers which can be quantitative. We have learned that it’s not just the tumor itself, but what’s in the microenvironment of the tumor and there are things in the stroma, which humans cannot easily decipher. What I expect to see in the future is we’ll be able to predict the risk of recurrence; we’ll be able to predict the approach to therapy; and know will this patient benefit or not.”
But it may take a bit of time to get there according to David Klimstra, co-founder and chief medical officer of Paige.ai, a diagnostic company that is leveraging AI for tissue-based analysis to develop a new generation of clinical applications and predictive tests. Training AI applications, across many tumor types is data intensive, especially if you want to provide a diagnostic tool that can generalize the data against almost infinite variations of tumor pathology.
co-founder and Chief Medical Officer, Paige
“I’m talking tens of thousands of slides to develop one model,” Klimstra said. “You need to have multiple examples of the full spectrum of different morphologies that cancer can have, which are almost limitless. You will never be able to account for all of them, but the more data you put in, the more likely the model is to have seen these different kinds of variation.”
Companies like Paige, which is creating AI tools for digital pathology, will need to either curate their own data sets to use for developing and training the predictive and diagnostic algorithms or rely on data and digital images that centers like the Mayo Clinic are only now beginning to aggregate en masse. The advent and the continuing development of cloud computing is a vital enabling technology for digital pathology and related AI tools. While many of us experience the application of AI to recognize visual patterns when we place our face in front of our cell phone to unlock it, that’s child’s play compared with digital pathology images.
“Conventional AI or computer vision algorithms have always been developed for real world images, which are small,” said Harvard’s Mahmood. “Digital images are enormously large. A single scan glass slide can be 100,000 by 100,000 to 200,000 pixels. If it’s uncompressed, a single slide can be up to 60 gigabytes and, compressed, around two to five gigabytes. If you have that for hundreds of thousands of patients, each patient having 100 slides, it’s petabytes of data. Cloud computing is absolutely necessary to do this at scale.”
As this data is collected and new algorithms developed in the coming years across a broad swath of tumor pathologies for diagnostic and prognostic purposes, how will these tools ultimately be used in the clinical setting? The consensus appears to be that digital pathology will be but one data source to leverage to provide precision care from a menu that includes radiologic data, genomic data, real-world data, and other information contained within each patient’s electronic medical record.
Mahmood’s lab at Harvard has been purpose built to uncover the benefits of applications of this multimodal health data integration and analysis. The research team in his lab has been pushing the boundaries on the use of AI to integrate digital pathology data and genomic data, as well as an important study published in 2021 in Nature on the use of AI to diagnose cancers of unknown primary—a tool that when further developed promises to have a lasting impact in improving clinical care.
As Mahmood sees it, the multimodal application of AI will provide the answers that are simply unattainable due to the complexity of all the factors driving an individual patient’s disease. “Right now, care is determined mostly by a pathologist and oncologist,” he noted. “This involves radiology, diagnostics, and pathology, and the oncologist takes all of these data and does the multimodal integration in their head to come up with the diagnosis and prognosis for that patient. But, digitally, we have the ability to parse all of that information and inform much better diagnosis and prognosis.”
With that as the base, the opportunity arises to include other data that typically have not been considered for the development of the therapeutic selection or care plan for a patient.
“A human does not have the ability to parse through all of it,” Mahmood continued. “We might understand some of those relationships, but we obviously don’t understand all of it. Computationally, the real power of AI, or deep learning, is that it helps us establish a lot of nonlinear relationships between the data. That is inherently very difficult for us to do manually.”
While AI algorithms leveraging digital pathology and associated patient data will play an increasingly central role in precision cancer care in the future, Bui pointedly noted that this trend is not one that will take decision making out of the hands of pathologists and oncologists. In this environment Bui thinks that AI should be considered “augmented intelligence.”
“There is no AI that is smart enough to replace humans, we will always be in the loop,” she concluded. “It is smart enough to do certain things that we ask it. AI is designed to augment human physicians’ ability to do more, better, faster, and to improve the quality of work so pathology is in the center of digital health.”
But beyond that, it promises to pave a regulatory pathway for the approval of other AI-driven digital pathology diagnostics from Paige and the myriad other companies also developing their own digital pathology algorithms. “The FDA didn’t have a paradigm to study for this, so there was a lot of back-and-forth discussions with the FDA team on how should we structure the validation study,” Klimstra added. “But now, there’s a template. So when we go back, we know exactly what we have to do or if another company has an AI product, they’ll know from our experience what they have to do.”
| Last September, digital pathology diagnostics developer Paige.ai—spun out of Memorial Sloan Kettering—became the first digital pathology diagnostic developer to be granted marketing approval by the FDA for Paige Prostate, its clinical-grade AI technology for prostate cancer detection. The initial research for Paige Prostate was conducted at MSK prior to the company becoming a commercial enterprise. |
| The diagnostic tool was initially granted Breakthrough Status by the FDA, which was an important stepping stone to final approval, noted Paige Co-Founder and CMO David Klimstra, as it allowed the company to open a dialog on a technology the agency had not previously approved. “They had approved a number of AI tools in radiology, but not in pathology which has different endpoints using different materials,” Klimstra said. |
| The approval should foster uptake of the test, Klimstra noted, as it moves the diagnostic out of the realm of a laboratory developed test (LDT) that would require every location using it to perform an LDT validation. With FDA approval, only a basic site validation is required making it essentially “plug and play.” |
Chris Anderson, a Maine native has been a B2B editor for more than 25 years. He was the founding editor for Security Systems News and Drug Discovery News, and led the print launch and expanded coverage as editor in chief of Clinical OMICs, now named Inside
Precision Medicine. He is an avid supporter of the Boston Red Sox, the Boston Bruins, Aston Villa FC, and wishes U.S. professional sports would adopt the practice of relegation. In his spare time, he can be found with headphones on playing Beatles and Bob Marley songs on his Fender Jazz bass while dwelling on the hundred degrees of separation between his abilities and those of Pino Paladino. He also mixes the best Manhattan in the world—prove him wrong.
