Traffic Signs Recognition using CNN and Keras in Python

We always come across incidents of accidents where drivers' Overspeed or lack of vision leads to major accidents. In winter, the risk of road accidents has a 40-50% increase because of the traffic signs' lack of visibility. So here in this article, we will be implementing Traffic Sign recognition using a Convolutional Neural Network. It will be very useful in Automatic Driving Vehicles.

Convolutional Neural Networks 

A convolutional Neural Network is a Deep Learning network used to pick up features from the image. Initially, they take the input images and then find out the lines, gradients, shapes and borders from the image. 

Then further, Convolutional Layers help in processing the outputs to capture eyes, faces, etc. CNN contains many convolutional layers assembled on top of each other, each one competent of recognizing more sophisticated shapes. With two or three convolutional layers it is viable to recognize handwritten digits and with 25 layers it is possible to differentiate human faces.

Traffic Signs Recognition using CNN and Keras in Python

Here we will be using this concept for the recognition of traffic signs.

Importing Libraries

• Pandas – Use to load the data frame in a 2D array format.
• NumPy – NumPy arrays are very fast and can perform large computations in a very short time.
• Matplotlib – This library is used to draw visualizations.
• Sklearn – It contains multiple libraries having pre-implemented functions of model development and evaluation.
• OpenCV – This library mainly focused on image processing and handling.
• Tensorflow – It provides a range of functions to achieve complex functionalities with single lines of code.

import matplotlib.image as mpimg
import os

from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.preprocessing import image_dataset_from_directory
from tensorflow.keras.preprocessing.image import ImageDataGenerator, load_img
from keras.utils.np_utils import to_categorical
from tensorflow.keras.utils import image_dataset_from_directory
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.layers import Conv2D, MaxPooling2D
from tensorflow.keras.layers import Activation, Dropout, Flatten, Dense
from tensorflow.keras.models import Sequential
from keras import layers
from tensorflow import keras
from tensorflow.keras.layers.experimental.preprocessing import Rescaling
from sklearn.model_selection import train_test_split

import matplotlib.pyplot as plt
import tensorflow as tf
import pandas as pd
import numpy as np
from glob import glob
import cv2

import warnings
warnings.filterwarnings('ignore')

Loading and Extracting the Dataset

The dataset has 58 classes of Traffic Signs and a label.csv file. The folder is in zip format. To unzip the dataset, we will run the code below.

# Extracting the compressed dataset.
from zipfile import ZipFile data_path
   = '/content/traffic-sign-dataset-classification.zip' with
       ZipFile(data_path, 'r') as zip
    : zip.extractall()

Data Visualization

Data visualization means the visualization of data to understand the key features of the dataset.

# path to the folder containing our dataset
dataset = '../content/traffic_Data/DATA'

# path of label file
labelfile = pd.read_csv('labels.csv')

Once we load the dataset, now let's visualize some random images from the different folders. For that, we will use plt.imshow() command.

# Visualize some images from the dataset
img = cv2.imread("/content/traffic_Data/DATA/10/010_0011.png")
plt.imshow(img)

Output : 

Let's see one more image. 

img = cv2.imread("/content/traffic_Data/DATA/23/023_0001.png")
plt.imshow(img)

Output : 

Label File - This file includes the 58 rows and 2 columns. The columns contains the class id and the name of the symbol. And the rows depicts the 58 different classes id and names.

Now, Let's see the label file by printing the top 5 rows.

labelfile.head()

Output:

Let's see the last 5 classes from the label file

labelfile.tail()

Output:

Data Preparation for Training

In this section, we will split the dataset into train and val set. Train set will be used to train the model and val set will be use to evaluate the performance of our model.

import tensorflow as tf

dataset = "/content/DATA" 

train_ds = tf.keras.preprocessing.image_dataset_from_directory(
    dataset,
    validation_split=0.2,
    subset='training',
    seed=123,
    image_size=(224, 224),
    batch_size=32
)

val_ds = tf.keras.preprocessing.image_dataset_from_directory(
    dataset,
    validation_split=0.2,
    subset='validation',
    seed=123,
    image_size=(224, 224),
    batch_size=32
)

Output:

Found 4170 files belonging to 58 classes.
Using 3336 files for training.
Found 4170 files belonging to 58 classes.
Using 834 files for validation.

Once we split the dataset, Let's create the list of the class names and print few images along with their class names.

class_numbers = train_ds.class_names
class_names = []
for i in class_numbers:
    class_names.append(labelfile['Name'][int(i)])

Let's visualize the train dataset and print 25 images from the dataset.

plt.figure(figsize=(10, 10))
for images, labels in train_ds.take(1):
    for i in range(25):
        ax = plt.subplot(5, 5, i + 1)
        plt.imshow(images[i].numpy().astype("uint8"))
        plt.title(class_names[labels[i]])
        plt.axis("off")

plt.show()

Output:

Data Augmentation

Sometimes the data is limited and we the model is not performing well with limited data. For this method, we use Data Augmentation. It is the method to increase the amount and diversity in data. We do not collect new data, rather we transform the already present data.

import tensorflow as tf

data_augmentation = tf.keras.Sequential([
    tf.keras.layers.RandomFlip("horizontal", input_shape=(224, 224, 3)),
    tf.keras.layers.RandomRotation(0.1),
    tf.keras.layers.RandomZoom(0.2),
    tf.keras.layers.RandomFlip("horizontal_and_vertical")  # No 'mode' argument needed
])

Model Architecture

The model will contain the following Layers:

• Four Convolutional Layers followed by MaxPooling Layers.
• The Flatten layer to flatten the output of the convolutional layer.
• Then we will have three fully connected Dense layers followed by the output of the of Softmax activation function.

model = Sequential()
model.add(data_augmentation)
model.add(Rescaling(1./255))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(128, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Conv2D(256, (3, 3), activation='relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(128, activation='relu'))
model.add(Dense(len(labelfile), activation='softmax'))

Let's print the summary of the model.

model.summary()

Output : 

To understand the huge number of parameters and complexity of the model which helps us to achieve a high-performance model let's see the plot_model.

keras.utils.plot_model(
    model,
    show_shapes=True,
    show_dtype=True,
    show_layer_activations=True
)

Output : 

model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(),
              optimizer='adam',
              metrics=['accuracy'])

Model Training 

Now we will train our model, the model is working fine on epochs = 50, but you can perform hyperparameter tuning for better results.

We can also use callback function for early stopping the model training.

# Set callback functions to early stop training
mycallbacks = [EarlyStopping(monitor='val_loss', patience=5)]
history = model.fit(train_ds,
                 validation_data=val_ds,
                 epochs=50,
                 callbacks=mycallbacks)

Output:

The above image shows the last epochs of the model.

Model Evaluation

Let’s visualize the training and validation accuracy with each epoch.

# Loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.legend(['loss', 'val_loss'], loc='upper right')

# Accuracy
plt.plot(hist.history['accuracy'])
plt.plot(hist.history['val_accuracy'])
plt.legend(['accuracy', 'val_accuracy'], loc='upper right')

Output : 

CNN model is performing very well with these layers. Further we can include more traffic and danger sign to warn the drivers.

Colab Link: click here