Group B Deep Learning Assignment

No: 3B

Title of the Assignment: Use MNIST Fashion Dataset and create a classifier to classify fashion clothing into
categories.

Objective of the Assignment: Students should be able to Classify movie reviews into positive reviews and
"negative reviews on IMDB Dataset.

Prerequisite:
1. Basic of programming language
2. Concept of Classification
3. Concept of Deep Neural Network
---------------------------------------------------------------------------------------------------------------
Contents for Theory:
1. What is Classification
2. Example of Classification
3. What is CNN?
4. How Deep Neural Network Work on Classification
5. Code Explanation with Output
---------------------------------------------------------------------------------------------------------------
What is Classification?

Classification is a type of supervised learning in machine learning that involves categorizing data into

predefined classes or categories based on a set of features or characteristics. It is used to predict the class

of new, unseen data based on the patterns learned from the labeled training data.

In classification, a model is trained on a labeled dataset, where each data point has a known class label.

The model learns to associate the input features with the corresponding class labels and can then be used

to classify new, unseen data.

For example, we can use classification to identify whether an email is spam or not based on its content

and metadata, to predict whether a patient has a disease based on their medical records and symptoms, or

to classify images into different categories based on their visual features.

Classification algorithms can vary in complexity, ranging from simple models such as decision trees and

k-nearest neighbors to more complex models such as support vector machines and neural networks. The

choice of algorithm depends on the nature of the data, the size of the dataset, and the desired level of

accuracy and interpretability.

Example- Classification is a common task in deep neural networks, where the goal is to predict the class

of an input based on its features. Here's an example of how classification can be performed in a deep

neural network using the popular MNIST dataset of handwritten digits.

The MNIST dataset contains 60,000 training images and 10,000 testing images of handwritten digits

from 0 to 9. Each image is a grayscale 28x28 pixel image, and the task is to classify each image into one

of the 10 classes corresponding to the 10 digits.

We can use a convolutional neural network (CNN) to classify the MNIST dataset. A CNN is a type of
deep neural network that is commonly used for image classification tasks.

What us CNN-
Convolutional Neural Networks (CNNs) are commonly used for image classification tasks, and they are
designed to automatically learn and extract features from input images. Let's consider an example of
using a CNN to classify images of handwritten digits.
In a typical CNN architecture for image classification, there are several layers, including convolutional
layers, pooling layers, and fully connected layers. Here's a diagram of a simple CNN architecture for the
digit classification task:
The input to the network is an image of size 28x28 pixels, and the output is a probability distribution
over the 10 possible digits (0 to 9).
The convolutional layers in the CNN apply filters to the input image, looking for specific patterns and
features. Each filter produces a feature map that highlights areas of the image that match the filter. The
filters are learned during training, so the network can automatically learn which features are most
relevant for the classification task.
The pooling layers in the CNN downsample the feature maps, reducing the spatial dimensions of the
data. This helps to reduce the number of parameters in the network, while also making the features more
robust to small variations in the input image.
The fully connected layers in the CNN take the flattened output from the last pooling layer and perform
a classification task by outputting a probability distribution over the 10 possible digits.
During training, the network learns the optimal values of the filters and parameters by minimizing a loss
function. This is typically done using stochastic gradient descent or a similar optimization algorithm.
Once trained, the network can be used to classify new images by passing them through the network and
computing the output probability distribution.
Overall, CNNs are powerful tools for image recognition tasks and have been used successfully in many
applications, including object detection, face recognition, and medical image analysis.
CNNs have a wide range of applications in various fields, some of which are:
Image classification: CNNs are commonly used for image classification tasks, such as identifying
objects in images and recognizing faces.
Object detection: CNNs can be used for object detection in images and videos, which involves
identifying the location of objects in an image and drawing bounding boxes around them.
Semantic segmentation: CNNs can be used for semantic segmentation, which involves partitioning an
image into segments and assigning each segment a semantic label (e.g., "road", "sky", "building").
Natural language processing: CNNs can be used for natural language processing tasks, such as
sentiment analysis and text classification.
Medical imaging: CNNs are used in medical imaging for tasks such as diagnosing diseases from X-rays
and identifying tumors from MRI scans.
Autonomous vehicles: CNNs are used in autonomous vehicles for tasks such as object detection and
lane detection.

Video analysis: CNNs can be used for tasks such as video classification, action recognition, and video
captioning.
Overall, CNNs are a powerful tool for a wide range of applications, and they have been used
successfully in many areas of research and industry.
How Deep Neural Network Work on Classification using CNN-
Deep neural networks using CNNs work on classification tasks by learning to automatically extract
features from input images and using those features to make predictions. Here's how it works:
Input layer: The input layer of the network takes in the image data as input.
Convolutional layers: The convolutional layers apply filters to the input images to extract relevant
features. Each filter produces a feature map that highlights areas of the image that match the filter.
Activation functions: An activation function is applied to the output of each convolutional layer to
introduce non-linearity into the network.
Pooling layers: The pooling layers downsample the feature maps to reduce the spatial dimensions of the
data.
Dropout layer: Dropout is used to prevent overfitting by randomly dropping out a percentage of the
neurons in the network during training.
Fully connected layers: The fully connected layers take the flattened output from the last pooling layer
and perform a classification task by outputting a probability distribution over the possible classes.
Softmax activation function: The softmax activation function is applied to the output of the last fully
connected layer to produce a probability distribution over the possible classes.
Loss function: A loss function is used to compute the difference between the predicted probabilities and
the actual labels.
Optimization: An optimization algorithm, such as stochastic gradient descent, is used to minimize the
loss function by adjusting the values of the network parameters.
Training: The network is trained on a large dataset of labeled images, adjusting the values of the
parameters to minimize the loss function.
Prediction: Once trained, the network can be used to classify new images by passing them through the
network and computing the output probability distribution.
MNIST Dataset-
The MNIST Fashion dataset is a collection of 70,000 grayscale images of 28x28 pixels, representing 10
different categories of clothing and accessories. The categories include T-shirts/tops, trousers, pullovers,
dresses, coats, sandals, shirts, sneakers, bags, and ankle boots.
The dataset is often used as a benchmark for testing image classification algorithms, and it is considered
a more challenging version of the original MNIST dataset which contains handwritten digits. The

MNIST Fashion dataset was released by Zalando Research in 2017 and has since become a popular
dataset in the machine learning community.
he MNIST Fashion dataset is a collection of 70,000 grayscale images of 28x28 pixels each. These
images represent 10 different categories of clothing and accessories, with each category containing 7,000
images. The categories are as follows:
T-shirt/tops
Trousers
Pullovers
Dresses
Coats
Sandals
Shirts
Sneakers
Bags
Ankle boots
The images were obtained from Zalando's online store and are preprocessed to be normalized and
centered. The training set contains 60,000 images, while the test set contains 10,000 images. The goal of
the dataset is to accurately classify the images into their respective categories.
The MNIST Fashion dataset is often used as a benchmark for testing image classification algorithms,
and it is considered a more challenging version of the original MNIST dataset which contains
handwritten digits. The dataset is widely used in the machine learning community for research and
educational purposes.
Here are the general steps to perform Convolutional Neural Network (CNN) on the MNIST Fashion
dataset:
● Import the necessary libraries, including TensorFlow, Keras, NumPy, and Matplotlib.
● Load the dataset using Keras' built-in function, keras.datasets.fashion_mnist.load_data(). This
will provide the training and testing sets, which will be used to train and evaluate the CNN.
● Preprocess the data by normalizing the pixel values between 0 and 1, and reshaping the images to
be of size (28, 28, 1) for compatibility with the CNN.
● Define the CNN architecture, including the number and size of filters, activation functions, and
pooling layers. This can vary based on the specific problem being addressed.
● Compile the model by specifying the loss function, optimizer, and evaluation metrics. Common
choices include categorical cross-entropy, Adam optimizer, and accuracy metric.
● Train the CNN on the training set using the fit() function, specifying the number of epochs and
batch size.
● Evaluate the performance of the model on the testing set using the evaluate() function. This will
provide metrics such as accuracy and loss on the test set.
● Use the trained model to make predictions on new images, if desired, using the predict()
function.
Source Code with Output-
import tensorflow as tf
import matplotlib.pyplot as plt
from tensorflow import keras
import numpy as np

(x_train, y_train), (x_test, y_test) = keras.datasets.fashion_mnist.load_data()

# There are 10 image classes in this dataset and each class has a mapping corresponding to the following
labels:
#0 T-shirt/top
#1 Trouser
#2 pullover
#3 Dress
#4 Coat
#5 sandals
#6 shirt
#7 sneaker
#8 bag
#9 ankle boot
plt.imshow(x_train[1])

plt.imshow(x_train[0])

# Next, we will preprocess the data by scaling the pixel values to be between 0 and 1, and then reshaping
the images to be 28x28 pixels.

x_train = x_train.astype('float32') / 255.0

x_train = x_train.reshape(-1, 28, 28, 1)

# 28, 28 comes from width, height, 1 comes from the number of channels
# -1 means that the length in that dimension is inferred.
# This is done based on the constraint that the number of elements in an ndarray or Tensor when
reshaped must remain the same.
# each image is a row vector (784 elements) and there are lots of such rows (let it be n, so there are 784n
elements). So TensorFlow can infer that -1 is n.
# converting the training_images array to 4 dimensional array with sizes 60000, 28, 28, 1 for 0th to 3rd
dimension.
x_train.shape
(60000, 28, 28)
x_test.shape
(10000, 28, 28, 1)
y_train.shape
(60000,)
y_test.shape
(10000,)
# We will use a convolutional neural network (CNN) to classify the fashion items.
# The CNN will consist of multiple convolutional layers followed by max pooling,
# dropout, and dense layers. Here is the code for the model:
model = keras.Sequential([
keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(28,28,1)),
# 32 filters (default), randomly initialized
# 3*3 is Size of Filter
 # 28,28,1 size of Input Image
# No zero-padding: every output 2 pixels less in every dimension
# in Paramter shwon 320 is value of weights: (3x3 filter weights + 32 bias) * 32 filters
# 32*3*3=288(Total)+32(bias)= 320
keras.layers.MaxPooling2D((2,2)),
# It shown 13 * 13 size image with 32 channel or filter or depth.
keras.layers.Dropout(0.25),
# Reduce Overfitting of Training sample drop out 25% Neuron
keras.layers.Conv2D(64, (3,3), activation='relu'),
# Deeper layers use 64 filters
# 3*3 is Size of Filter
# Observe how the input image on 28x28x1 is transformed to a 3x3x64 feature map
# 13(Size)-3(Filter Size )+1(bias)=11 Size for Width and Height with 64 Depth or filtter or channel
# in Paramter shwon 18496 is value of weights: (3x3 filter weights + 64 bias) * 64 filters
# 64*3*3=576+1=577*32 + 32(bias)=18496
keras.layers.MaxPooling2D((2,2)),
# It shown 5 * 5 size image with 64 channel or filter or depth.
keras.layers.Dropout(0.25),
keras.layers.Conv2D(128, (3,3), activation='relu'),
# Deeper layers use 128 filters
# 3*3 is Size of Filter
# Observe how the input image on 28x28x1 is transformed to a 3x3x128 feature map
# It show 5(Size)-3(Filter Size )+1(bias)=3 Size for Width and Height with 64 Depth or filtter or
channel
# 128*3*3=1152+1=1153*64 + 64(bias)= 73856
# To classify the images, we still need a Dense and Softmax layer.
# We need to flatten the 3x3x128 feature map to a vector of size 1152
keras.layers.Flatten(),
keras.layers.Dense(128, activation='relu'),
# 128 Size of Node in Dense Layer
# 1152*128 = 147584
 keras.layers.Dropout(0.25),
keras.layers.Dense(10, activation='softmax')
# 10 Size of Node another Dense Layer
# 128*10+10 bias= 1290
])
model.summary()
Model: "sequential"

Layer (type) Output Shape Param #

max_pooling2d (MaxPooling2D (None, 13, 13, 32) 0

dropout (Dropout) (None, 13, 13, 32) 0

conv2d_1 (Conv2D) (None, 11, 11, 64) 18496

max_pooling2d_1 (MaxPooling (None, 5, 5, 64) 0

dropout_1 (Dropout) (None, 5, 5, 64) 0

flatten (Flatten) (None, 1152) 0

dense (Dense) (None, 128) 147584

dropout_2 (Dropout) (None, 128) 0

dense_1 (Dense) (None, 10) 1290

=================================================================
Total params: 241,546
Trainable params: 241,546
Non-trainable params: 0
# Compile and Train the Model
# After defining the model, we will compile it and train it on the training data.
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
history = model.fit(x_train, y_train, epochs=10, validation_data=(x_test, y_test))
# 1875 is a number of batches. By default batches contain 32 samles.60000 / 32 = 1875
# Finally, we will evaluate the performance of the model on the test data.
test_loss, test_acc = model.evaluate(x_test, y_test)
print('Test accuracy:', test_acc)
313/313 [==============================] - 3s 10ms/step - loss: 0.2606 - accuracy: 0.9031
Test accuracy: 0.9031000137329102

Conclusion- In this way we can Classify fashion clothing into categories using CNN.

Assignment Question

1. What is Binary Classification?

2. What is binary Cross Entropy?

4. What is the Epoch Cycle?

5. What is Adam Optimizer?