Neural Network Classification with

Where can you get help?

“If in doubt, run the code”

• Follow along with the code

• Try it for yourself
• Press SHIFT + CMD + SPACE to read the docstring
• Search for it
• Try again
• Ask (don’t forget the Discord chat!)
(yes, including the “dumb”
questions)

“What is a classification
problem?”
 Example classification problems
“Is this email spam or not spam?” “Is this a photo of sushi, steak or pizza?”
To: [email protected] To: [email protected]
Hey Daniel, Hay daniel…

This deep learning course is incredible! C0ongratu1ations! U win $1139239230

I can’t wait to use what I’ve learned!

Not spam Spam

Binary classi cation Multiclass classi cation

g o r a n o t h e r ) (more than one thing or
(one thin another)
“What tags should this article have?”

Machine learning
Representation learning
Arti cial intelligence

(multiple label options per

sample)

Multilabel classi cation

fi

fi
fi
fi

 What we’re going to cover

(broadly)
• Architecture of a neural network classi cation model

• Input shapes and output shapes of a classi cation model (features and labels)

• Creating custom data to view and t

• Steps in modelling

• Creating a model, compiling a model, tting a model, evaluating a model

• Di erent classi cation evaluation methods

• Saving and loading models

👩🍳 👩🔬
(we’ll be cooking up lots of code!)

How:
ff
fi

fi

fi
fi
fi

Classification inputs and outputs

224

W = 224 224 Sushi 🍣

H = 224 Steak 🥩
C=3 Pizza 🍕
(c = colour channels, R, G, B) Actual output

🍣 🥩 🍕
[[0.31, 0.62, 0.44…], [[0.97, 0.00, 0.03],
[0.92, 0.03, 0.27…], [0.81, 0.14, 0.05],
[0.25, 0.78, 0.07…], [0.03, 0.07, 0.90],
…, (normalized pixel valu …,
es)
Numerical
Predicted output
encoding (often already ex
ists, if not,
you can build on (comes from looking at lots
e) of these)

 Input and output shapes

(for an image classification example)

224
[[0.31, 0.62, 0.44…], 🍣 🥩 🍕
224 [0.92, 0.03, 0.27…], [0.97, 0.00, 0.03]
[0.25, 0.78, 0.07…], i o n p r ob ab i l i t i e s )
(predict
…,

(gets represented as a tens

or)
[batch_size, width, height, colour_channels] Shape = [3]
Shape = [None, 224, 224, 3]
or
Shape = [32, 224, 224, 3] These will vary depending on the
(32 is a v e ry c o m m o n b a t c h problem you’re working on.
size)
(typical)

Architecture of a classification
model (we’re going to be buildin
lots of these)
g

Source: Adapted from page 295 of Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow Book by Aurélien Géron

Sushi 🍣
Steak 🥩
Pizza 🍕

Let’s code!
The machine learning explorer’s
motto
“Visualize, visualize, visualize”
Data

Model It’s a good idea to visualize

these as often as possible.

Training

Predictions

Steps in modelling with TensorFlow

(typical)

Architecture of a classification
model (we’re going to be buildin
lots of these)
g

Source: Adapted from page 295 of Hands-On Machine Learning with Scikit-Learn, Keras & TensorFlow Book by Aurélien Géron

Sushi 🍣
Steak 🥩
Pizza 🍕

Steps in modelling with TensorFlow

1. Construct or import a pretrained model relevant to your problem
2. Compile the model (prepare it to be used with data)

• Loss — how wrong your model’s predictions are compared to the

truth labels (you want to minimise this).

• Optimizer — how your model should update its internal patterns

to better its predictions.

• Metrics — human interpretable values for how well your model is

doing.
3. Fit the model to the training data so it can discover patterns

• Epochs — how many times the model will go through all of the
training examples.
4. Evaluate the model on the test data (how reliable are our model’s
predictions?)

 Improving a model (from a model’s perspective)

Smaller model

Common ways to improve a deep model:

• Adding layers Larger model
• Increase the number of hidden units
• Change the activation functions
• Change the optimization function
• Change the learning rate (because you can alter each of
• Fitting on more data these, they’re hyperparameters)

• Fitting for longer

 Improving a model (from a model’s perspective)

Smaller model

Common ways to improve a deep model:

• Adding layers Larger model
• Increase the number of hidden units
• Change the activation functions
• Change the optimization function
• Change the learning rate (because you can alter each of
• Fitting on more data these, they’re hyperparameters)

• Fitting for longer

The missing piece: Non-linearity

🤔 “What could you draw if you had an
unlimited amount of straight (linear) and non-
straight (non-linear) lines?”

Linear data Non-linear data

(possible to model with straight lines) (not possible to model with straight lines)
The missing piece: Non-linearity
A = tf.cast(tf.range(-10, 10), tf.float32)

tf.keras.activations.linear(A) tf.keras.activations.sigmoid(A) tf.keras.activations.relu(A)

Linear activation Sigmoid activation ReLU activation

(non-linear) (non-linear)
 Three datasets (possibly the most important concept
in machine learning…)

Course materials Practice exam Final exam

(training set) (validation set) (test set)

The ability for a machine learning model to perform

Generalization well on data it hasn’t seen before.

Finding the ideal learning rate

 (some common)

Classification evaluation methods

Key: tp = True Positive, tn = True Negative, fp = False Positive, fn = False Negative

Metric Name Metric Forumla Code When to use

tp + tn tf.keras.metrics.Accuracy() Default metric for classi cation

Accuracy Accuracy = or problems. Not the best for
tp + tn + fp + fn sklearn.metrics.accuracy_score() imbalanced classes.

tp tf.keras.metrics.Precision() Higher precision leads to less false

Precision Precision = or
tp + fp sklearn.metrics.precision_score() positives.

tp tf.keras.metrics.Recall() Higher recall leads to less false

Recall Recall = or
tp + fn sklearn.metrics.recall_score() negatives.

precision ⋅ recall Combination of precision and recall,

F1-score F1-score = 2 ⋅ sklearn.metrics.f1_score() usually a good overall metric for a
precision + recall classi cation model.

When comparing predictions to truth

Custom function labels to see where model gets
Confusion matrix NA or
sklearn.metrics.confusion_matrix() confused. Can be hard to use with
large numbers of classes.

fi

fi

 Anatomy of a confusion matrix

Confusion Matrix
False positives

0 99 (98.0%) 2 (2.0%) • True positive = model predicts 1 when truth is 1

• True negative = model predicts 0 when truth is 0
True Label

• False positive = model predicts 1 when truth is 0

• False negative = model predicts 0 when truth is 1
1 0 (0.0%) 99 (100.0%)

Correct predictions
t i v e s 0 1 (true positives,
e g a
False n Predicted Label true negatives)

Steps in modelling with TensorFlow

1. Turn all data into numbers (neural networks can’t handle strings)
2. Make sure all of your tensors are the right shape (inputs and outputs)
3. Scale features — normalize or standardize, neural networks tend to prefer normalization (values
between 0 & 1)

Inputs and outputs

(layer by layer)

[[116, 78, 15],

[117, 43, 96],
[125, 87, 23],
…,

Takes inputs Outputs to

Outputs to

Outputs to

Outputs to

Outputs to

[[0.983, 0.004, 0.013],

[0.110, 0.889, 0.001],
[0.023, 0.027, 0.985],
…,
 2. Show examples
i t h ra n d om
i t i a l i s e w g )
1. In a t b e g i nn i n
g h ts ( o n l y
we i
[[0.092, 0.210, 0.415],
[0.778, 0.929, 0.030],
[0.019, 0.182, 0.555],
…,

[[116, 78, 15], [[0.983, 0.004, 0.013], Coat,

[117, 43, 96], [0.110, 0.889, 0.001], Ankle boot,
Shirt,
[125, 87, 23], [0.023, 0.027, 0.985], Sandal
…, …,
3. Update representation
4. Repeat with more outputs (weights & biases)
examples

Learns
Numerical representation Representation
Inputs Outputs
encoding (patterns/features/weights) outputs

The machine learning practitioner’s

motto

“Experiment, experiment, experiment”

👩🍳 👩🔬
(try lots of things an
d see what
tastes good)