How to Use Deep Learning AI for Context-Intelligent Image Analysis

by | May 30, 2017 | Blog |

Recent advances in artificial intelligence (AI) show great promise in several fields of medicine. In particular, the field of Deep Learning is enabling the development of expert-level automated algorithms. Successful applications include AI-supported diagnosis of melanoma based on pictures of skin lesions and diabetic retinopathy in images of the retina. Human expert judgement augmented by deep learning algorithms has the potential to speed up the diagnostic process and to make diagnostic assessments more reproducible.


Deep Learning AI Ki67 Fimmic WebMicroscope is now bringing these AI-powered technologies into the field of pathology. A large number of tasks currentlyundertaken visually through microscopes can be automated by training deep learning classifiers. One of the major advantages of the novel AI-based algorithms is the ability to train classifiers for morphologies that exhibit a high level of complexity. Such patterns include tissue entities like epithelium, stroma, adipose tissue, necrosis and grade of tumor differentiation.

Basic algorithms that can quantify immunohistochemically stained cells in tissue have been on the market for several years. However, most either lack the ability to quantify cells in specific tissue compartments (e.g. epithelium) or require additional sections and stainings to achieve context-specific readouts. The novel context-intelligent algorithms trained using WebMicroscope Deep Learning tools provide a highly efficient solution to this problem.

For example, the WebMicroscope epithelium detection algorithm can segment out epithelial cells from various other tissue entities. Once the epithelial parts of the tissue have been detected, a second deep learning algorithm is applied that detects both immunohistochemically stained (DAB stained) and counterstained (hematoxylin) cells and calculates the percentage of cells expressing a specific protein (e.g. estrogen receptor, Ki-67, PR) in epithelium only. An example of the method applied to Ki-67 quantification in breast cancer epithelial cells is below.

A major advantage of the WebMicroscope approach is that it’s fast, reproducible and is applied to the very same tissue section without additional staining. In addition, the whole process runs on-demand on whole-slide images in a cloud computing environment on the WebMicroscope Platform. The Software-as-a-Service and Platform-as-a-Service approach is also extremely easy to set up from a user perspective, as the need for local software and hardware installation is removed and the solution can immediately be scaled to projects of any size.

Additionally, any local microscope scanner can be linked to the system using the WebMicroscope Connector tool, automating the transfer and standardization of image files and a creating a uniform, accessible image data bank accessible from any web browser. Results of the analyses can then be viewed in the form of heatmaps on top of the virtual slides online, and numerical results can be retrieved in spreadsheet form for further analysis and integration with other data on the samples. This facilitates easier collaboration on projects irrespective of team member location.

The context-intelligent algorithm for biomarker quantification is just one example demonstrating the potential of AI-supported classifiers. Deep learning can be applied to numerous image-based diagnostic tasks and defines a new era in digital pathology. Additionally, the speed with which novel classifiers can be created is unforeseen and algorithm development can be brought much closer to the domain expert –  i.e. the pathologist – through the intuitive WebMicroscope cloud interface.