Informatics
Machine learning
Convolutional neural networks

Editor-in-Chief: Debra Zynger, M.D.

Topic Completed: 17 September 2018

Revised: 27 December 2018

Copyright: (c) 2018, PathologyOutlines.com, Inc.

PubMed Search: Convolutional neural networks pathology
Page views in 2018: 258
Page views in 2019 to date: 62
Cite this page: Cheng J. Convolutional neural networks. PathologyOutlines.com website. http://www.pathologyoutlines.com/topic/informaticsconvnet.html. Accessed February 18th, 2019.
Definition / general
  • Also referred to as ConvNet
  • Convolutional neural network (CNN) is a machine learning method inspired by the way our visual cortex processes images through receptive fields whereby individual retinal neurons receive stimuli from different regions of the visual field and information from multiple retinal neurons are subsequently passed on to neurons further down the chain (The Data Science Blog: A Quick Introduction to Neural Networks [Acccessed 23 August 2018])
  • Likewise, CNN has a feed forward neural network architecture composed of convolution and pooling (downsampling) layers, followed by 1 or more fully connected layers
Essential features
  • In machine learning, a convolutional neural network is a class of deep, feed forward artificial neural networks, most commonly applied in pathology to image analysis (Wikipedia: Convolutional Neural Network [Accessed 27 August 2018])
  • Neural networks, like other supervised machine learning methods, are trained using a dataset with an expected outcome and other parameters that contribute to the prediction of the outcome
    • For instance, a dataset for predicting the presence of hemolysis would have entries for the patient's sex, hemoglobin levels, serum lactate dehydrogenase, serum haptoglobin, indirect bilirubin and the presence or absence of hemolysis as the target feature
    • Variables that contribute to the prediction include the laboratory values and parameter values called weights
    • The value of the weights are adjusted in an iterative manner called backpropagation, where the accuracy of the neural network is assessed through a formula (loss function) and the weights are updated until it arrives at the weight values that give the best prediction accuracy
    • In convolutional neural networks involving images, weights are often in the form of 3 dimensional matrices; the target feature is the class an image belongs to (e.g. benign versus malignant) and the variables that contribute to the prediction are data from the image itself
Terminology
  • Feed forward: refers to how the data flows from 1 layer of the network to a subsequent layer of the network and is further passed on to the next layer of the network after calculations are made in the preceding layer
Diagrams / tables
  • The following figure illustrates the type of calculations image data goes through in convolution and pooling operations
    • Convolution operations involve an elementwise product between the filter and different segments of equal dimensions from the input matrix
    • Pooling operations perform an aggregate operation (e.g. maximum or average) on a region
    • In the example below, the maximum value was returned from 2 x 2 regions of the input matrix

    Contributed by Jerome Cheng, M.D.

    Convolution and pooling


Images hosted on other servers:

Typical CNN architecture

CNN layers in 3D

Neurons of convolutional layer connected to receptive field

Description of convolutional neural networks
  • Convolution or pooling operations are carried out on information from 1 layer and the results are passed on to a deeper layer of the network
  • Calculations involved in a convolutional neural network (CNN) are complex
    • Fortunately, with all the tools available to us, we do not need to write a program to perform all of these
    • Machine learning frameworks, such as TensorFlow, simplify the process of designing and training CNN models
Image analysis
  • Image analysis through convolutional neural network (CNN) is usually performed on digital slides obtained from a whole slide scanner or an image taken through a CCD device mounted to a microscope
  • Current applications are primarily in pathology research
  • Convolutional neural network architecture can be built from scratch or pretrained models can be used for image classification
  • Several pretrained CNN models, such as those based on VGG-16, VGG-19, Inception v3 and DeepLoc, are freely available on the internet and were trained on a particular image subject matter
    • Pretrained VGG-16, VGG-19 and Inception v3 models were trained with the ImageNet image database, comprising over a million images belonging to 1,000 classes of real world objects, such as animals, cars and tables
    • DeepLoc was trained to analyze yeast cell images (Bioinformatics 2017;33:3387)
    • Despite being trained on nonhistological images, they can still be used for analysis of pathology based image datasets
    • VGG-16 and VGG-19 take input images of 224 x 224 pixels
    • Inception v3 takes input images of 299 x 299 pixels
  • Training a new CNN model with a purely histopathological image dataset should further improve prediction accuracy but it would take considerable effort to collect millions of images and model training is expected to take several days to complete
  • Pretrained CNN models can produce results in minutes; in contrast, training a CNN model from scratch with a large image dataset can take days, even with the aid of a powerful GPU (graphical processing unit)
  • With the increasing adoption of whole slide digital imaging solutions in pathology departments for research, education and clinical practice, image analysis methods such as CNN may be used on digital slides to aid in identifying histological structures, such as mitosis, nuclei and regions with cancer metastasis in lymph nodes
Applications in pathology
Tools for performing image analysis using convolutional neural networks
Board review question #1
Which machine learning method was inspired by the way our visual cortex processes images through receptive fields, whereby retinal neurons receive stimuli from different regions of the visual field and information from multiple retinal neurons are relayed to neurons further down the chain?

  1. Convolutional neural network (CNN)
  2. Logistic regression
  3. Random Forest
  4. Support vector machine
Board review answer #1
A. CNN has a feed forward neural network architecture composed of convolution and pooling (downsampling) layers, followed by 1 or more fully connected layers. Convolution or pooling operations are carried out on information from 1 layer and the results are passed on to a deeper layer of the network. CNN has been used on digital slides to aid in identifying histological structures, such as mitosis, nuclei and regions with cancer metastasis.
Board review question #2
Which layer of a convolutional neural network performs a downsampling operation?

  1. Convolutional layer
  2. Fully connected layer
  3. Pooling layer
  4. ReLU layer
Board review answer #2
C. The pooling layer performs a downsampling operation, e.g. the maximum, minimum or average value of elements belonging to a 2 by 2 matrix may be computed.

Comment Here