Building a Convolutional Neural Network (CNN) for CIFAR-10

Classification
Under the Guidance of Sir Abdul Rafay
Submitted by Muhammad Hamza

1. Importing Necessary Modules

We start by importing the required modules:

In [1]: from keras.datasets import cifar10

import matplotlib.pyplot as plt
from tensorflow.keras.utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense, Conv2D, MaxPool2D, Flatten
from sklearn.metrics import classification_report
import numpy as np

cifar10.load_data() loads the CIFAR-10 dataset, splitting it into training and test sets.
We'll use Matplotlib for visualization, Keras for building the CNN, and scikit-learn for evaluating the model.

2. Loading the Dataset

We load the CIFAR-10 data:

(x_train, y_train), (x_test, y_test) = cifar10.load_data()

x_train and y_train contain the training images and labels, respectively.
x_test and y_test contain the test images and labels.

In [2]: (x_train, y_train), (x_test, y_test) = cifar10.load_data()

3. Dataset Shape
Let's check the shape of the data arrays:

x_train.shape
# Output: (50000, 32, 32, 3)

x_test.shape
# Output: (10000, 32, 32, 3)

The shape (50000, 32, 32, 3) indicates 50,000 training images, each with dimensions 32x32 pixels and 3 color channels
(RGB).
Similarly, the shape (10000, 32, 32, 3) corresponds to the test set.

In [3]: x_train.shape

Out[3]: (50000, 32, 32, 3)

In [4]: x_test.shape

Out[4]: (10000, 32, 32, 3)

4. Visualizing a Single Image

Finally, let's visualize a single image from the training set:

single_img = x_train[34]
plt.imshow(single_img)
 In [5]: single_img = x_train[34]

In [6]: #single_img

In [7]: plt.imshow(single_img)

Out[7]: <matplotlib.image.AxesImage at 0x21b3c816480>

Preprocessing Labels and Normalizing Pixel Values

In this section, we'll cover the preprocessing steps for the CIFAR-10 dataset. Let's break down each step:

In [8]: y_train

Out[8]: array([[6],
[9],
[9],
...,
[9],
[1],
[1]], dtype=uint8)

In [9]: y_test

Out[9]: array([[3],
[8],
[8],
...,
[5],
[1],
[7]], dtype=uint8)

In [10]: y_train.shape

Out[10]: (50000, 1)

5. Label Encoding
One-Hot Encoding
The labels in the CIFAR-10 dataset represent different classes (e.g., airplane, car, bird, etc.). To prepare them for training a neural
network, we perform one-hot encoding. This means converting the class labels into binary vectors.

# One-hot encode the labels

y_cat_train = to_categorical(y_train, 10) # 10 classes in CIFAR-10
 y_cat_test = to_categorical(y_test, 10)

y_cat_train and y_cat_test now contain one-hot encoded labels.

In [11]: y_cat_test = to_categorical(y_test, 10)

In [12]: y_cat_train = to_categorical(y_train, 10)

6. Normalizing Pixel Values

Scaling Pixel Intensities
To ensure consistent training, we normalize the pixel values of the images. Normalization scales the pixel intensities to a range between
0 and 1.

# Normalize pixel values (0 to 255) to (0 to 1)

x_train = x_train / 255
x_test = x_test / 255

x_train and x_test now have pixel values in the range [0, 1].

In [13]: x_train= x_train/255

In [14]: x_test= x_test/255

Now let's dive into the details of building a Convolutional Neural Network (CNN) for classifying images from the CIFAR-10 dataset.
Here's a breakdown of each step:

7. Creating the Model Architecture

We're using a sequential model, which allows us to stack layers sequentially. Here's how we set up the layers:

7.1. First Convolutional Layer

We add a convolutional layer with 32 filters (also known as kernels) of size 3x3.
The activation function is ReLU (Rectified Linear Unit).

model.add(Conv2D(filters=32, kernel_size=(3,3), input_shape=(32, 32, 3), activation='relu'))

7.2. Max Pooling Layer

We add a max-pooling layer with a pool size of 2x2.

model.add(MaxPool2D(pool_size=(2,2)))

7.3. Flattening Layer

We flatten the extracted feature maps into a 1D array.

model.add(Flatten())

7.4. Standard Neural Network Layer

We add a dense layer with 128 neurons and ReLU activation.

model.add(Dense(128, activation='relu'))

7.5. Output Layer

The final layer consists of 10 neurons (one for each class in CIFAR-10).
We use the softmax activation function for multiclass classification.

model.add(Dense(10, activation='softmax'))

In [15]: model = Sequential()

# 1st Convolutional Layer

# no of filters= 32 kernel_size 3x3
 # filter and kernel are the same things
model.add(Conv2D(filters=32, kernel_size=(3,3), input_shape=(32, 32, 3), activation= 'relu'))

#Max Pooling Layerabs

model.add(MaxPool2D(pool_size=(2,2)))

#Flatten the extracted pixels into a 1D array

model.add(Flatten())

#Standard Nueral Network Layer

model.add(Dense(128, activation= 'relu'))

#Output layer
model.add(Dense(10, activation='softmax'))

C:\Users\PMLS\miniconda3\envs\ML_Lab\Lib\site-packages\keras\src\layers\convolutional\base_conv.py:99: UserWarning: Do no
t pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` obj
ect as the first layer in the model instead.
super().__init__(

8. Compiling the Model

We compile the model by specifying the loss function, optimizer, and evaluation metric:

model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

In [16]: model.compile(loss='categorical_crossentropy',
optimizer= 'adam',
metrics= ['accuracy'])

9. Model Summary
Here's an overview of the model architecture:

model.summary()

In [17]: model.summary()

Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type) ┃ Output Shape ┃ Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D) │ (None, 30, 30, 32) │ 896 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d (MaxPooling2D) │ (None, 15, 15, 32) │ 0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ flatten (Flatten) │ (None, 7200) │ 0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense (Dense) │ (None, 128) │ 921,728 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_1 (Dense) │ (None, 10) │ 1,290 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 923,914 (3.52 MB)
Trainable params: 923,914 (3.52 MB)
Non-trainable params: 0 (0.00 B)

9. Training the Model

We train the model using the training data ( x_train and y_cat_train ) for 10 epochs:

model.fit(x_train, y_cat_train, epochs=10)

In [18]: model.fit(x_train, y_cat_train, epochs=10)

 Epoch 1/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 26s 15ms/step - accuracy: 0.4173 - loss: 1.6184
Epoch 2/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 22s 14ms/step - accuracy: 0.5973 - loss: 1.1498
Epoch 3/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 22s 14ms/step - accuracy: 0.6468 - loss: 1.0103
Epoch 4/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 22s 14ms/step - accuracy: 0.6836 - loss: 0.9091
Epoch 5/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 22s 14ms/step - accuracy: 0.7085 - loss: 0.8221
Epoch 6/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 22s 14ms/step - accuracy: 0.7427 - loss: 0.7329
Epoch 7/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 22s 14ms/step - accuracy: 0.7688 - loss: 0.6620
Epoch 8/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 23s 15ms/step - accuracy: 0.7973 - loss: 0.5844
Epoch 9/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 23s 15ms/step - accuracy: 0.8208 - loss: 0.5183
Epoch 10/10
1563/1563 ━━━━━━━━━━━━━━━━━━━━ 24s 15ms/step - accuracy: 0.8425 - loss: 0.4610
Out[18]: <keras.src.callbacks.history.History at 0x21b45e27bf0>

Evaluating the Model and Generating Classification Report

Now let's evaluate the performance of the trained model on the test data and generate a classification report. Here are the steps:

10. Model Metrics

First, let's retrieve the available metrics names:

model.metrics_names
# Output: ['loss', 'compile_metrics']

In [19]: model.metrics_names

Out[19]: ['loss', 'compile_metrics']

11. Model Evaluation

Next, we evaluate the model using the test data:

model.evaluate(x_test, y_cat_test)
# Output: [test_loss, test_accuracy]

The test_loss represents the loss on the test set, and test_accuracy is the accuracy achieved.

In [20]: model.evaluate(x_test, y_cat_test)

313/313 ━━━━━━━━━━━━━━━━━━━━ 2s 4ms/step - accuracy: 0.6510 - loss: 1.1666

Out[20]: [1.1772527694702148, 0.6520000100135803]

12. Making Predictions

We predict class labels for the test data:

predictions = model.predict(x_test)

In [21]: predictions = model.predict(x_test)

313/313 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step

13. Generating Classification Report

Finally, we create a classification report to assess the model's performance:

predicted_classes = np.argmax(predictions, axis=1)

report = classification_report(y_test, predicted_classes)
print(report)
 The classification report includes precision, recall, F1-score, and support for each class.

In [22]: predicted_classes = np.argmax(predictions, axis = 1)

In [23]: report = classification_report(y_test, predicted_classes)

In [24]: print(report)

precision recall f1-score support

0 0.70 0.67 0.68 1000

1 0.81 0.68 0.74 1000
2 0.54 0.51 0.52 1000
3 0.47 0.51 0.49 1000
4 0.57 0.61 0.59 1000
5 0.62 0.50 0.55 1000
6 0.72 0.72 0.72 1000
7 0.72 0.74 0.73 1000
8 0.76 0.78 0.77 1000
9 0.64 0.79 0.71 1000

accuracy 0.65 10000

macro avg 0.66 0.65 0.65 10000
weighted avg 0.66 0.65 0.65 10000

Conclusion
In this Class activity, we explored various aspects of building a Convolutional Neural Network (CNN) for classifying images from the
CIFAR-10 dataset. Let's summarize our findings:

1. Dataset Loading and Preprocessing:

We loaded the CIFAR-10 dataset using Keras, obtaining separate training and test sets.
Labels were one-hot encoded, and pixel values were normalized to the range [0, 1].
2. Model Architecture:

Our CNN architecture consisted of:

A 32-filter convolutional layer with ReLU activation.
A max-pooling layer.
Flattening to convert feature maps into a 1D array.
A dense layer with 128 neurons (ReLU activation).
An output layer with 10 neurons (softmax activation).
3. Model Training and Evaluation:

We compiled the model with categorical cross-entropy loss and the Adam optimizer.
Training was performed for 10 epochs.
The model achieved an accuracy of approximately 65% on the test set.
4. Classification Report:

We generated a classification report, including precision, recall, F1-score, and support for each class.

Further fine-tuning, hyperparameter optimization, and data augmentation can enhance model performance. Feel free to explore
additional techniques and continue experimenting with CNNs!