Informatics, digital & computational pathology
Digital & computational pathology
Machine learning & deep learning
Computer aided diagnosis


Topic Completed: 17 February 2020

Minor changes: 28 April 2020

Copyright: 2020, PathologyOutlines.com, Inc.

PubMed Search: Computer aided diagnosis[TI] informatics review[ptyp]

Kunal Nagpal, M.S.
Yun Liu, Ph.D.
Page views in 2019: 1
Page views in 2020 to date: 299
Cite this page: Nagpal K, Chen P, Steiner D, Mermel C, Liu Y. Computer aided diagnosis. PathologyOutlines.com website. http://www.pathologyoutlines.com/topic/informaticscompaiddiagnosis.html. Accessed July 8th, 2020.
Definition / general
Essential features
  • Computer aided diagnosis algorithms interpret histologic findings need to be validated using rigorous reference standards
  • A thoughtfully designed user interface is needed to assist diagnosis effectively
  • Systems need to be seamlessly integrated into the diagnostic workflow
Key components
  • An effective computer aided diagnosis tool typically requires (Am J Surg Pathol 2018;42:1636):
    • An accurate classification algorithm
    • Thoughtful user interface that may be quite different than that for a computer aided detection tool
    • Integration with existing workflows
  • Algorithm: typically a machine learning approach based on feature engineering, or, more recently, convolutional neural networks (CNNs)
  • User interface: due to the subjective nature of image interpretation and imperfections in models, an effective user interface may leverage:
    • Explanations: why the algorithm interpreted an image in a certain way
    • Interactive tools: to allow a pathologist to correct algorithm predictions while benefiting from other features, such as quantitation
Terminology
  • Computer aided diagnosis: refers to a technology or system that interprets findings on medical images, such as tumor grade
  • Computer aided detection: refers to a technology or system to detect findings of interest on medical images, typically with the goal of reducing the false negative rate, improving efficiency or reducing fatigue
  • Model: the machine learning algorithm underlying the computer aided diagnosis system
  • Reference standard: the standard against which the model is evaluated (JAMA 2019;322:1806)
  • Calibration: in addition to categorizations such as "yes / no", computer systems generally also produce continuous probabilities
  • Calibration refers to whether these probabilities match real world rates
    • For example, among 100 examples predicted to be cancer with a probability of 0.2, are approximately 20% of them actually cancer? (JAMA 2017;318:1377)
How computer aided diagnosis works
  • Similar to computer aided detection, use of systems for computer aided diagnosis requires algorithm development, inference and integration into pathology workflows (Arch Pathol Lab Med 2020;144:221)
  • Evaluation of computer aided diagnosis systems can present challenges relative to computer aided detection due to the inherent subjectivity of image interpretation
  • Algorithms can be evaluated against
    • Pathologist provided annotations
    • Independent gold standard (such as a molecular test)
    • Clinical outcomes
  • If pathologist provided annotations are used as the reference standard, steps should be taken to reduce variability from subjective interpretation and improve reproducibility of the study (Cancer 2001;91:1284, Arch Pathol Lab Med 2001;125:736, Eur Urol 2013;64:199)
    • Having experienced experts with subspecialist training and substantial clinical experience provide annotations
    • Using the consensus opinion of multiple experts
  • Sub analyses may be useful to evaluate performance differences based on
    • Characteristics of the population being evaluated, such as patient population or disease subtype
    • Sensitivity to pre-analytic variables such as surgical or staining protocol, imaging hardware and software post processing such as image compression (Nat Med 2019;25:1301, Arch Pathol Lab Med 2019;143:859)
Utility
  • Improving accuracy or consistency by classifying findings in a way that is more concordant with a rigorous reference standard
Example applications
  • While comparison of automated system accuracy to pathologist accuracy has been promising, more work describing integration into pathologist workflows and impact on diagnosis is necessary
  • Breast carcinoma: classification of grade, ER status, histologic subtype and intrinsic subtype (NPJ Breast Cancer 2018;4:30)
    • Key takeaway: convolutional neural networks based automated system achieves > 75% accuracy at each of these tasks
    • Reference standard: central pathologic review, with immunohistochemical staining for determination of ER status
  • Colon polyps: classification of polyps (hyperplastic, sessile serrated, traditional serrated, tubular, or tubulovillious / villious) (J Pathol Inform 2017;8:30)
    • Key takeaway: convolutional neural networks based automated system achieves > 90% accuracy in these classifications
    • Reference standard: initial review by two pathologists and adjudication with a third senior pathologist in cases of disagreement
  • Lung carcinoma: classification of histological subtype (Nat Med 2018;24:1559 )
    • Key takeaway: convolutional neural networks based automated system trended towards greater accuracy than pathologists in classifying lung cancer resections as adenocarcinoma or squamous cell carcinoma
    • Reference standard: diagnostic review with central pathologic confirmation
  • Prostate adenocarcinoma: Gleason grading (NPJ Digit Med 2019;2:48, Sci Rep 2018;8:12054)
    • Key takeaway: convolutional neural networks based system graded resections or tissue microarrays with similar or greater accuracy than general or expert pathologists and trended toward better risk stratification
    • Reference standard: subspecialist provided grades and clinical outcomes
  • Lymphoma: evaluation of automated systems for lymphoma classification and several other tasks (J Pathol Inform 2016;7:29)
    • Key takeaway: convolutional neural networks based automated system achieves 97% accuracy in distinguishing between chronic lymphocytic leukemia, follicular lymphoma and mantle cell lymphoma for region subsections of whole slide images
    • Reference standard: curated dataset provided by expert pathologists
  • Skin lesions: three classification tasks as basal cell carcinoma, dermal nevi and seborrheic keratosis (J Pathol Inform 2018;9:32)
    • Key takeaway: convolutional neural networks based automated system achieves > 0.99 area under the curve (AUC) at diagnosing the class of interest versus distractors
    • Reference standard: prior diagnoses by pathologists
  • Renal cell carcinoma: nuclear grading for clear cell renal cell carcinoma (J Pathol Inform 2014;5:23)
    • Key takeaway: support vector machine (SVM) based system can help distinguish low and high grade tumors with an AUC of 0.97
    • Reference standard: grades from an experienced subspecialty pathologist
Future changes
  • There are many promising image analysis algorithms, as briefly and non exhaustively summarized above
  • The key next steps are:
    • Reference standard improvements: though much work has focused so far on classification compared to an existing grading system or molecular test, future work will need to focus on directly predicting clinical outcomes in order to improve patient risk stratification
    • Workflow integration: better understanding of how to integrate artificial intelligence algorithms into the actual workflow
    • User interface: thoughtful optimization of the user interface such that pathologists can effectively integrate insights from algorithm predictions to improve diagnostic accuracy
    • Measuring impact: studies to quantify the benefits of using computer aided diagnosis devices in pathology and to assess if there are negative consequences such as overdiagnosis or underdiagnosis; as of this writing, there are no registered trials at ClinicalTrials.gov to prospectively measure the diagnostic accuracy of computer aided diagnosis systems in pathology and this will be an exciting area of future work
    • Continuous monitoring: similar to post marketing safety surveillance for approved drugs, computer aided diagnosis systems should be monitored in real use for quality control purposes
Board review style question #1
    Which of the following is a difference between computer aided detection (CADe) systems and computer aided diagnosis systems?

  1. CADe systems require a thoughtful user interface and integration pathology workflow, while computer aided diagnosis systems run in an automated manner
  2. Computer aided diagnosis systems assist in characterization or interpretation of histology, while computer aided detection systems assist in detection of findings
  3. Computer aided diagnosis systems need to be validated against rigorous reference standards, while CADe systems do not
  4. Computer aided diagnosis systems require an algorithm to assist with diagnosis, while computer aided detection systems do not
Board review answer #1
B. Computer aided diagnosis systems in pathology share many similarities with computer aided detection (CADe) systems, but focus on characterization and interpretation rather than detection

Comment Here

Reference: Computer aided diagnosis informatics
Board review style question #2
    Which of the following is a typical goal in application of computer aided diagnosis in pathology?

  1. Improve consistency by making reproducible classifications on subjective tasks
  2. Improve consistency by showing the current grading guidelines
  3. Improve exhaustive review by classifying all diagnostic features of histopathology images
  4. Remind a pathologist of workflow tasks
Board review answer #2
A. Computer aided diagnosis systems generally aim to improve diagnostic accuracy

Comment Here

Reference: Computer aided diagnosis informatics
Back to top