Mechatronics Engineering and Automation Program

CSE488: Computational Intelligence

Lab#08: TensorFlow II

TensorFlow II

Basic Neural Network Implementation in TensorFlow 2.0

About the dataset
Let us implement a simple neural network using TensorFlow 2.0. For this, we will make use of the Fashion
MNIST dataset by Zalando (MIT License) which contains 70,000 images (in grayscale) in 10 different
categories. The images are 28x28 pixels of individual articles of clothing with values ranging from 0 to
255 as shown below:

Out of the total 70,000 images, 60,000 are used for training and remaining 10,000 for testing. The labels
are integer arrays ranging from 0 to 9. The class names are not a part of the dataset and hence we need to
include the below mapping while training/prediction:
Label -> Description
0 -> T-shirt/top
1 -> Trouser
2 -> Pullover

1
 CSE488: Computational Intelligence Lab08: TensorFlow II

3 -> Dress
4 -> Coat
5 -> Sandal
6 -> Shirt
7 -> Sneaker
8 -> Bag
9 -> Ankle boot
[6]: # Create class_names list object for mapping labels to names

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal',

,→'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

[7]: # Use the below code to make sure that you select TensorFlow 2.0 in Colab
try:
%tensorflow_version 2.x
except Exception:
pass

[8]: # Helper libraries

import numpy as np
import matplotlib.pyplot as plt
# TensorFlow and tf.keras
import tensorflow as tf
from tensorflow import keras as ks

# Validating the TensorFlow version

print(tf.__version__)

2.1.0

[9]: # Load the Fashion MNIST dataset

(training_images, training_labels), (test_images, test_labels) = ks.datasets.

,→fashion_mnist.load_data()

Data Exploration

[10]: # Shape of Training and Test Set

print('Training Images Dataset Shape: {}'.format(training_images.shape))

print('No. of Training Images Dataset Labels: {}'.format(len(training_labels)))
print('Test Images Dataset Shape: {}'.format(test_images.shape))
print('No. of Test Images Dataset Labels: {}'.format(len(test_labels)))

2
 CSE488: Computational Intelligence Lab08: TensorFlow II

Training Images Dataset Shape: (60000, 28, 28)

No. of Training Images Dataset Labels: 60000
Test Images Dataset Shape: (10000, 28, 28)
No. of Test Images Dataset Labels: 10000

[11]: np.unique(training_images)

[11]: array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,

13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,
26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38,
39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51,
52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64,
65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77,
78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90,
91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103,
104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116,
117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129,
130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142,
143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155,
156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168,
169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181,
182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194,
195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207,
208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220,
221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233,
234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246,
247, 248, 249, 250, 251, 252, 253, 254, 255], dtype=uint8)

Data Visualization
[15]: index = 60
plt.figure()
plt.imshow(training_images[index], cmap='gray')
plt.colorbar()
plt.show()
print(training_labels[index])
print(class_names[training_labels[index]])

3
 CSE488: Computational Intelligence Lab08: TensorFlow II

5
Sandal
Data Preprocessing
As the pixel values range from 0 to 255, we have to scale these values to a range of 0 to 1 before feeding
them to the model. We can scale these values (both for training and test datasets) by dividing the values
by 255:

[14]: training_images = training_images / 255.0

test_images = test_images / 255.0

Model Building
We will be using the keras implementation to build the different layers of a NN. We will keep it simple
by having only 1 hidden layer.

[35]: input_data_shape = (28, 28)

hidden_activation_function = 'relu'
output_activation_function = 'softmax'

nn_model = ks.models.Sequential()
nn_model.add(ks.layers.Flatten(input_shape=input_data_shape, name='Input_layer'))
nn_model.add(ks.layers.Dense(64, activation=hidden_activation_function,
,→name='Hidden_layer'))

nn_model.add(ks.layers.Dense(10, activation=output_activation_function,
,→name='Output_layer'))

[22]: nn_model.summary()

Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================

4
 CSE488: Computational Intelligence Lab08: TensorFlow II

Input_layer (Flatten) (None, 784) 0

_________________________________________________________________
Hidden_layer (Dense) (None, 64) 50240
_________________________________________________________________
Output_layer (Dense) (None, 10) 650
=================================================================
Total params: 50,890
Trainable params: 50,890
Non-trainable params: 0
_________________________________________________________________
Now, we will use an optimization function with the help of compile method. An Adam optimizer with
objective function as sparse_categorical_crossentropy which optimzes for the accuracy metric can be built
as follows:
[36]: optimizer = 'adam'
loss_function = 'sparse_categorical_crossentropy'
metric = ['accuracy']
nn_model.compile(optimizer=optimizer, loss=loss_function, metrics=metric)

[41]: nn_model.fit(training_images, training_labels, epochs=10, batch_size=32)

Train on 60000 samples

Epoch 1/10
60000/60000 [==============================] - 6s 99us/sample - loss: 0.5202 -
accuracy: 0.8189
Epoch 2/10
60000/60000 [==============================] - 5s 86us/sample - loss: 0.3915 -
accuracy: 0.8610
Epoch 3/10
60000/60000 [==============================] - 5s 89us/sample - loss: 0.3556 -
accuracy: 0.8716
Epoch 4/10
60000/60000 [==============================] - 6s 93us/sample - loss: 0.3321 -
accuracy: 0.8790
Epoch 5/10
60000/60000 [==============================] - 6s 98us/sample - loss: 0.3144 -
accuracy: 0.8844
Epoch 6/10
60000/60000 [==============================] - 5s 87us/sample - loss: 0.3009 -
accuracy: 0.8893
Epoch 7/10
60000/60000 [==============================] - 5s 89us/sample - loss: 0.2908 -
accuracy: 0.8927
Epoch 8/10
60000/60000 [==============================] - 5s 92us/sample - loss: 0.2814 -
accuracy: 0.8961

5
 CSE488: Computational Intelligence Lab08: TensorFlow II

Epoch 9/10
60000/60000 [==============================] - 5s 88us/sample - loss: 0.2712 -
accuracy: 0.9006
Epoch 10/10
60000/60000 [==============================] - 6s 94us/sample - loss: 0.2643 -
accuracy: 0.9026

[41]: <tensorflow.python.keras.callbacks.History at 0x1a522055fc8>

Model Evaluation
1. Training Evaluation

[42]: training_loss, training_accuracy = nn_model.evaluate(training_images, training_labels)

print('Training Data Accuracy {}'.format(round(float(training_accuracy),2)))

60000/60000 [==============================] - 4s 59us/sample - loss: 0.2511 -

accuracy: 0.9069
Training Data Accuracy 0.91
2. Test Evaluation
[43]: test_loss, test_accuracy = nn_model.evaluate(test_images, test_labels)

print('Test Data Accuracy {}'.format(round(float(test_accuracy),2)))

10000/10000 [==============================] - 1s 72us/sample - loss: 0.3522 -

accuracy: 0.8735
Test Data Accuracy 0.87

[46]: nn_model.predict(training_images[0].reshape((1,28,28)))
print(training_labels[0])

9
Deep Neural Network (DNN) Model
We will be using the keras implementation to build the different layers of a NN. We will keep it simple
by having only 1 hidden layer.

[47]: input_data_shape = (28, 28)

hidden_activation_function = 'relu'
output_activation_function = 'softmax'

dnn_model = ks.models.Sequential()
dnn_model.add(ks.layers.Flatten(input_shape=input_data_shape, name='Input_layer'))
dnn_model.add(ks.layers.Dense(256, activation=hidden_activation_function,
,→name='Hidden_layer_1'))

6
 CSE488: Computational Intelligence Lab08: TensorFlow II

dnn_model.add(ks.layers.Dense(192, activation=hidden_activation_function,
,→name='Hidden_layer_2'))

dnn_model.add(ks.layers.Dense(128, activation=hidden_activation_function,
,→name='Hidden_layer_3'))

dnn_model.add(ks.layers.Dense(10, activation=output_activation_function,
,→name='Output_layer'))

[48]: dnn_model.summary()

Model: "sequential_6"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
Input_layer (Flatten) (None, 784) 0
_________________________________________________________________
Hidden_layer_1 (Dense) (None, 256) 200960
_________________________________________________________________
Hidden_layer_2 (Dense) (None, 192) 49344
_________________________________________________________________
Hidden_layer_3 (Dense) (None, 128) 24704
_________________________________________________________________
Output_layer (Dense) (None, 10) 1290
=================================================================
Total params: 276,298
Trainable params: 276,298
Non-trainable params: 0
_________________________________________________________________
Now, we will use an optimization function with the help of compile method. An Adam optimizer with
objective function as sparse_categorical_crossentropy which optimzes for the accuracy metric can be built
as follows:
[49]: optimizer = 'adam'
loss_function = 'sparse_categorical_crossentropy'
metric = ['accuracy']
dnn_model.compile(optimizer=optimizer, loss=loss_function, metrics=metric)

[50]: dnn_model.fit(training_images, training_labels, epochs=20)

Train on 60000 samples

Epoch 1/20
60000/60000 [==============================] - 19s 310us/sample - loss: 0.4820 -
accuracy: 0.8251
Epoch 2/20
60000/60000 [==============================] - 17s 280us/sample - loss: 0.3633 -
accuracy: 0.8671
Epoch 3/20

7
CSE488: Computational Intelligence Lab08: TensorFlow II

60000/60000 [==============================] - 16s 270us/sample - loss: 0.3279 -

accuracy: 0.8778
Epoch 4/20
60000/60000 [==============================] - 18s 301us/sample - loss: 0.3059 -
accuracy: 0.8850
Epoch 5/20
60000/60000 [==============================] - 17s 285us/sample - loss: 0.2877 -
accuracy: 0.8931
Epoch 6/20
60000/60000 [==============================] - 16s 273us/sample - loss: 0.2708 -
accuracy: 0.8996
Epoch 7/20
60000/60000 [==============================] - 17s 278us/sample - loss: 0.2596 -
accuracy: 0.9021
Epoch 8/20
60000/60000 [==============================] - 17s 283us/sample - loss: 0.2498 -
accuracy: 0.9051
Epoch 9/20
60000/60000 [==============================] - 16s 273us/sample - loss: 0.2387 -
accuracy: 0.9090
Epoch 10/20
60000/60000 [==============================] - 20s 332us/sample - loss: 0.2290 -
accuracy: 0.9129
Epoch 11/20
60000/60000 [==============================] - 20s 338us/sample - loss: 0.2207 -
accuracy: 0.9156
Epoch 12/20
60000/60000 [==============================] - 19s 319us/sample - loss: 0.2134 -
accuracy: 0.9182
Epoch 13/20
60000/60000 [==============================] - 21s 344us/sample - loss: 0.2077 -
accuracy: 0.9219
Epoch 14/20
60000/60000 [==============================] - ETA: 0s - loss: 0.2024 -
accuracy: 0.92 - 19s 319us/sample - loss: 0.2024 - accuracy: 0.9226
Epoch 15/20
60000/60000 [==============================] - 21s 348us/sample - loss: 0.1964 -
accuracy: 0.9247-
Epoch 16/20
60000/60000 [==============================] - 18s 295us/sample - loss: 0.1899 -
accuracy: 0.9272
Epoch 17/20
60000/60000 [==============================] - 18s 294us/sample - loss: 0.1840 -
accuracy: 0.9283
Epoch 18/20
60000/60000 [==============================] - 19s 310us/sample - loss: 0.1776 -

8
 CSE488: Computational Intelligence Lab08: TensorFlow II

accuracy: 0.9314
Epoch 19/20
60000/60000 [==============================] - 16s 273us/sample - loss: 0.1775 -
accuracy: 0.9325
Epoch 20/20
60000/60000 [==============================] - 15s 252us/sample - loss: 0.1677 -
accuracy: 0.9351

[50]: <tensorflow.python.keras.callbacks.History at 0x1a52335dc48>

Model Evaluation
1. Training Evaluation

[51]: training_loss, training_accuracy = dnn_model.evaluate(training_images, training_labels)

print('Training Data Accuracy {}'.format(round(float(training_accuracy),2)))

60000/60000 [==============================] - 7s 116us/sample - loss: 0.1648 -

accuracy: 0.9367
Training Data Accuracy 0.94
2. Test Evaluation
[52]: test_loss, test_accuracy = dnn_model.evaluate(test_images, test_labels)

print('Test Data Accuracy {}'.format(round(float(test_accuracy),2)))

10000/10000 [==============================] - 1s 147us/sample - loss: 0.3998 -

accuracy: 0.8869
Test Data Accuracy 0.89