How to Create a Machine Learning Model using Keras

CBIIT/FNL Workshop

Frederick National Laboratory for Cancer Research

Bioinformatics Analyst IV
Hyun Jung

November 6, 2019

1
Outline

• Why Deep Learning?

• Why Keras?
• How does it work?
• Convolution, ReLU, Max pooling, Fully-connected layer
• Gradient Descent and Backpropagation
• Know your data
• Build/Train/Test your own network
• Glossary

2
Why Deep Learning?

ICDAR 2011 Chinese handwriting IJCNN 2011 traffic signs

ISBI 2012 brain segmentation ICPR 2012 cancer detection MICCAI 2013 Grand Challenge
3
Why Deep Learning?

4
Why Keras?

From: Google Trends

5
Why Keras?

From: TensorFlow official website

6
How does it work?

If (circles):
if (brown-ish):
if (two triangles):
picture = dog
else:
picture = food

Traditional Programming Dog Cookie

Image Source: guidedogs.org Image Source: foodnetwork.com
7
How does it work?

Dog vs. Food

Image source: boredpanda.com

8
 How does it work?

Dog Food

Food Dog

Dog Food

. Dog
.
.

Dog Food

Food Dog

Food

9
How does it work?

• Building Blocks of Convolutional Neural Network (CNN)

• Convolution, ReLU, Pooling, and Fully-Connected (Dense) Layer

Simple convolutional neural network architecture

Image Source: MathWorks

10
How does it work?

• Why the CNN structured in such a way?

Hierarchy of visual detectors of increasing complexity achieves sophisticated perceptual categorization,

with the higher levels being able to recognize 1000's of different objects, people, etc.
Image Source: Medicine Library

11
 How does it work?

• Input

Matrix representation of a digit 8 Composition of RGB from 3 Grayscale images

Image credit: Deep Learning made easy with Deep Cognition from Medium Image credit: Wikipedia

12
 How does it work?

• Convolution

Convolution operation
Image credit: the data science blog

Convolution operation and its effect

Image credit: Deep Learning Methods for Vision (CVPR 12 tutorial) Image credit: groundai.com
13
 How does it work?

• Convolution

Convolution operation
Image credit: the data science blog

Convolution operation and its effect

Image credit: Deep Learning Methods for Vision (CVPR 12 tutorial) Image credit: An Intuitive Explanation of Convolutional Neural Networks
14
How does it work?

• ReLU (Rectified Linear Unit)

• Introducing non-linearity

ReLU operation and its effect

Image credit: the data science blog
15
How does it work?

• Pooling (sub-sampling or down-sampling)

• Reduce dimensionality and extract most import information

Max pooling operation Max pooling effect

http://cs231n.github.io/convolutional-networks/ Neural Networks, MLSS2015 Summer School, Facebook AI Research,

16
How does it work?

• Story so far

Simple convolutional neural network architecture

Learned features from a CNN
Image Source: MathWorks H. Lee, et al., PROC. OF THE 26th ICML, 2009

17
How does it work?

GoogLeNet structure and visualization of its feature maps

Image Source: Feature Visualization from Distill 18
How does it work?

• Fully Connected Layer

Simple convolutional neural network architecture

Image Source: MathWorks

19
How does it work?

• Fully Connected Layer

Image Source: Understanding Convolutional Neural Networks, Medium

20
How does it work?

• What are we training?

• Weights and biases

Image credit: Deep Learning made easy with Deep Cognition from Medium

21
How does it work?

• How to train?
• Gradient Descent and Backpropagation

Gradient descent mechanism Backpropagating error to individual weights

Image Source: Image Source:

Gradient Descent: All You Need to Know from Rohan & Lenny #1: Neural Networks & The
Medium Backpropagation Algorithm, Explained from Medium
22
 How does it work?

• How to train?
• Gradient Descent is too slow

Different optimization methods on loss surface contours Optimization methods on saddle point

Image Source: Sebastian Ruder webpage Image Source: Sebastian Ruder webpage

23
Know Your Data

• Understanding your data is CRUCIAL

• Kaggle: Breast Histopathology Images
• URL: https://www.kaggle.com/paultimothymooney/breast-histopathology-images
• Invasive Ductal Carcinoma Breast Cancer Images
• Scanned at 40x optical magnification
• Image Size: 50 x 50 x 3 (Height x Width x Channels)

24
Know Your Data

• Understanding your data is CRUCIAL

• Kaggle: Breast Histopathology Images
• Total number of patients: 279 (Train: 223, Valid: 28, Test: 28)
• Total number of benign images: 198,783
• Total number of malignant images: 78,786
• Average number of benign images per patient: 712.32
• Average number of malignant images per patient: 282.38
• Maximum/Minimum number of benign images among patients: 2231/14
• Maximum/Minimum number of malignant images among patients: 1347/10

25
Build/Train/Test your own network

• Download/Analyze/Organize Data (Done, Organize_data.py)

• Setup a virtual environment (anaconda/mybinder) and install necessary packages
• Read/Explore data
• Create generators to feed images to your network
• Define your own network
• Compile and train your network
• Save and load trained model for testing
• Transfer learning (if time allows)

26
Glossary

• Batch size: Number of images that are trained simultaneously

• Step/Iteration: Training one batch
• One Epoch: Training entire image once
• Overfitting: Network is specifically optimized to solve training images
• Underfitting: Network performance is not fully utilized
• Dropout: Randomly discard information to avoid overfitting

Image Source: What is underfitting and overfitting in machine learning and how to deal with it., Medium
27
Glossary

• Loss/objective function: Binary cross entropy function

−(𝑦𝑦 * log(𝑝𝑝) + (1−𝑦𝑦) * log(1−𝑝𝑝)) − (y*log(𝑝𝑝)+(1−y)*log(1− 𝑝𝑝))

prediction/Label y=0 y=1

p=0 0

p=1 0

28
Environment Setup

• Anaconda (Your own laptop, can be faster)

• conda create -n MyEnvName python=3
• conda install pip
• pip install --upgrade tensorflow or tensorflow-gpu
• pip install Keras
• conda install nb_conda
• pip install scikit-learn
• pip install matplotlib
• pip install scikit-image

• Mybinder.org (Jupyter server)

• https://mybinder.org
• GitHub repository name or URL
• https://github.com/newhyun00/DL_Workshop
• Press launch

29