How did the Mayo Clinic choose a vendor for AI in pathology?

What to consider when choosing a vendor for AI in pathology? Learn how the Mayo Clinic evaluated platform providers based on six key criteria.
Written by Aiforia

Artificial intelligence (AI) is considered a major technology development for anatomic pathology. More and more clinics are now designing their platform requirements in order to integrate AI into their digital pathology workflows as it is expected to elevate pathologists' diagnostic capabilities, predict outcomes, and suggest therapeutic options.2-5

A recent publication, "Democratizing Artificial Intelligence in Anatomic Pathology", by Thomas J. Flotte, MD et al., describes how one of the top-ranked hospitals in the world, the Mayo Clinic, selected Aiforia as a vendor for AI-powered pathology image analysis.

 

Mayo Clinic’s 6 criteria for platform providers

To create an ecosystem that permits everyone to become involved in developing AI algorithms in anatomic pathology, the Mayo Clinic defined six key criteria for evaluating potential platform vendors:

  1. Approachable and intuitive: The platform must be designed for users with a wide range of technical expertise and interest. It is important that entry-level users can easily use the platform.

  2. Variety of modeling options: The platform will be used for a large variety of experiments, and therefore, it needs to be flexible and accommodate many different approaches. The modeling capabilities must include rare event detection, segmentation, individual feature and spatial quantitation, classification, and prognostication. The platform provider must also show evidence of prior successful AI model development from many different researchers with different approaches and endpoints.

  3. Internal and external collaboration: The users may want to collaborate with colleagues within or outside their own institution. The platform needs to permit validated users outside the firewall to access the program.

  4. The platform must meet the minimum requirements of the IT and security groups: The IT and information security teams assessed how easily a vendor’s solution integrated into Mayo Clinic’s cloud platform and what vulnerabilities existed.

  5. Seamless transition from discovery to clinical deployment: The platform needs to support both the discovery, that is, the development of algorithms, and deployment in the clinical workflow.  It would be very frustrating for a user to develop a great algorithm and not have an opportunity to use it in practice.

  6. Platform scalability over time: Although the pilot starts with a limited number of users, it is important that the platform scales to a more significant number of users in the future.

After a careful review of these criteria by pathologists, IT, and security teams, Aiforia was selected as a vendor for AI-powered pathology image analysis both for clinical workflow and translational research.

 

How were the pathologists supported and engaged throughout the implementation process?

A comprehensive AI ecosystem was created to aid the implementation and AI development, supporting pathologists with varying levels of expertise. The AI ecosystem received great feedback from the users, and it was proven to break down entry barriers, reduce the overall cost of AI development, improve AI quality, and enhance the speed of adoption.Cornerstones were the following:

Training and onboarding plan: An education committee consisting of Aiforia, the Department of Laboratory Medicine and Pathology, and institutional education group representatives created a training plan for onboarding the platform users and provided ongoing support. Aiforia was responsible for designing and implementing an onboarding curriculum that consisted of training sessions, full-length lecture recordings, FAQs, and short educational videos illustrating specific common tasks. An institutional community website was created to house all the materials, including a discussion forum and a ticketing system for user support. 

Training instructors from Aiforia: Aiforia’s support team regularly held open office hours, providing the users with an opportunity to ask questions, receive personal guidance, discuss and resolve barriers, and learn from each other.  Instructors were also available for one-on-one tutorial sessions with users when needed. 

Support resources from Mayo Clinic: A cytotechnologist with extensive experience in AI was nominated as an Aiforia® Create superuser. The superuser was available to guide and support users in AI model development, for example, on how to annotate images effectively. A data scientist was assigned to assist users with design and data interpretation, and a project manager provided logistic support, communication management, risk and problem resolution, and metric tracking. Mayo Clinic’s IT group resolved all incoming community tickets regarding the institutional cloud.

 

Results so far: From discovery to clinical deployment

Based on the article1, 84 users from the Mayo Clinic completed the training in the years 2022–2023. A total of 30 out of 31 projects progressed through the model development process of annotating, training, and validation. 15 abstracts were submitted to national meetings, 13 of those to the USCAP 2024. 

See more USCAP-related content here:


The first clinical application deployed was Aiforia’s Ki67 AI model for breast cancer diagnostics. The analysis of patient samples started in March 2023. There are currently three tests in the Mayo Clinic’s diagnostic test portfolio using Aiforia.

Another interesting result of the collaboration between Mayo Clinic and Aiforia is QuantCRC - a prognostic AI model that identifies important histological features of colorectal cancer and provides a recurrence prediction estimate useful for treatment decisions. Join our upcoming webinar to hear more about this exciting use case and learn how prognostic AI models are revolutionizing the future of pathology image analysis.

Aiforia QuantcCRC webinar banner

 

References

  1. Thomas J. Flotte, Stephanie A. Derauf, Rachel K. Byrd, Trynda N. Kroneman, Debra A. Bell, Lucas Stetzik, Seung-Yi Lee, Alireza Samiei, Steven N. Hart, Joaquin J. Garcia, Gillian Beamer, Thomas Westerling-Bui; Democratizing Artificial Intelligence in Anatomic Pathology. Arch Pathol Lab Med 2024; doi: https://doi.org/10.5858/arpa.2023-0205-OA
  2. Flotte TJ, Bell DA. Anatomical pathology is at a crossroads. Pathology. 2018;50(4):373–374.
  3. Cohen MX. Practical Linear Algebra for Data Science. 1st ed. Sebastopol, CA: O’Reilly Media; 2022.
  4. Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A. Artificial intelligence in digital pathology–new tools for diagnosis and precision oncology. Nat Rev Clin Oncol. 2019;16(11):703–715.
  5. Niazi MKK, Parwani AV, Gurcan MN. Digital pathology and artificial intelligence. Lancet Oncol. 2019;20(5):e253–e261.
Want to find out more about Aiforia?

Contact us