CS 611: ARTIFICIAL

INTELLIGENCE

B. I. Ya’u, August 26, 2021

Deep learning
 What is Deep Learning?
• Deep learning (DL) is a branch of machine learning that is concerned with teaching computers to do what is
natural to humans, that is, learning by example.

• Deep learning has empowered great projects like the driverless car, making it possible for the car to distinguish
between lampposts and pedestrians, recognize stop signs, etc.

• Deep learning is inspired by the structure of the human brain and how it operates.

• With deep learning, machines are able to perform tasks that require human intelligence to be performed.

• The machines learn from experience and acquire skills with the intervention by humans.

• Deep learning involves the use of artificial neural networks to learn patterns, trends and relations from large
datasets.

• Deep learning algorithms perform tasks repeatedly, while tweaking it in a bid to improve the outcome.

• With deep learning, computer models are able to perform classification tasks from text, images or sound.

• The deep learning models can provide a high level of accuracy, sometimes outperforming humans.

• Training of models is done using large sets of labeled data and neural networks with several layers.
 DL Cont.
• The word deep refers to the number of hidden layers that a neural network has.

• Most neural networks have 2-3 hidden layers, but a deep neural network may have over 100 hidden layers. The training of
such models involves the use of large sets of data.

• Neural network architectures with no need for manual intervention are then used to extract patterns from the data.

• A good example of a deep neural network is the convolutional neural network (CNN). CNN uses 2D convolutional layers,
which makes it good for the processing of 2D data like images.

• With CNN, there is no need for extracting features manually as you are not required to identify the features to be used for
classifying images.

• CNNs directly extract features from images. The ability of deep learning models to extract features automatically makes
them suitable for use in computer vision problems like object classification.

• CNNs rely on numerous layers to detect the features of an image. The complexity of the image features increases at every
layer.

• In the first hidden layer for example, the edges of the mages may be detected while the last layer may learn complex
features in the image.

• A good example of the application of deep learning is in fraud detection systems. Once it learns the normal procedures,
any anomaly will be easily detected and classified as a potential for fraud.
 Artificial Neural Networks (ANN)
• Neural networks form the core of deep learning. A neural network is a kind of
network in which the nodes are seen as artificial neurons.

• The concept of neural networks began in the 1980s. The neural network of the
human being is made up of a network of interconnected neurons for maintaining a
high level of coordination to receive and then transmit messages to the spinal cord
and the brain.

• In machine learning, such types of networks are referred to as Artificial Neural

Networks (ANNs).

• Artificial Neural Networks are made up of neurons that have been created artificially.
These are then taught so that they can adapt to the cognitive skills of human beings.

• Some of the applications of ANNs are image recognition, soft sensors, voice
recognition, time series predictions, and anomaly detection.
 ANN Cont.
• Neural networks are represented in the form of a mathematical model and they are mostly applied in machine
learning. They are made up of neurons which are connected to each other, sending each other signals.

• A neuron receives signals until they exceed its threshold, and this is the time it fires, meaning that it forwards
the signal to the next connected neuron in the network.

• The connections between the neurons can be done in the way we want, even to the same neuron, but the
problem comes in when we need to train the network. This explains why restrictions have to be imposed on
the creation of neural networks.

• For the case of a multi-layer perceptron, the neurons have been arranged into layers, and each neuron is
allowed to pass signals only to the next neuron in the layer. The first layer in this is the input layer, while the last
one is the output layer, and this will have the predicted values.

• In artificial neural networks, learning refers to the process of modifying the bias and the weights that are fed to
the network. Learning in neural networks is facilitated by training it, whereby a certain set of inputs are fed to
the network while expecting a particular output, which is the target output.

• The process of learning in artificial neural networks is similar to the concept of learning in human beings. The
training is done repeatedly until we get the value of weights that give us the targeted outputs. After each
training, normally referred to as an epoch, the weights are adjusted so that the error of the neural network can
be reduced. This is done until the error is minimized completely.
A Single Neuron Neural Network
in Python
 A Single NN in Python Cont.

• A single neuron neural network will transform a given input into a particular
output.

• Based on the given inputs and the assigned weights, we can determine
whether the neuron fired or not.

• The neuron will have 3 input connections and 1 output. We will be using the
tanh activation function.
 Why Activation Function?
• In Neural Network every neuron will do two computations:

a) Linear summation of inputs, if we see the above diagram , it has

two inputs

x1,x2 and bias (b).We have weights w1,w2

sum = (w1*x1+w2*x2)+ b

b) Activation computation.This computation decides, whether a

neuron should be activated or not, by calculating weighted sum
and further adding bias with it. The purpose of the activation
function is to introduce non-linearity into the output of a neuron.

Why do we need Non-linear activation functions :-

• A neural network without an activation function is essentially

just a linear regression model.

• The activation function does the non-linear transformation to

the input making it capable to learn and perform more
complex tasks.
 Tanh or Hyperbolic
• The tanh function is function that can be used as a nonlinear
activation function between layers of a neural network.

• Tanh will map values to be between -1 and 1.

• One of the interesting properties of the tanh function is that

the derivative can be expressed in terms of the function
itself.

• Below is the actual formula for the tanh function along with
the formula for calculating its derivative.

(i) Now it’s output is zero centered because its range in

between -1 to 1 i.e -1 < output < 1 .

(ii) Hence optimization is easier in this method

• The tanh function is usually used in hidden layers of a neural

network as it’s values lies between-1 to 1 hence the mean
for the hidden layer comes out be 0 or very close to it, hence
helps in centering the data by bringing mean close to 0. This
makes learning for the next layer much easier.
 A Single NN in Python Cont.
• We began by importing all the necessary functions from the Numpy library.

• We then created the NeuralNetwork class that will help us implement a neural network.

• To ensure that similar weights are generated after every run, we created a seed.

• The weights to be fed into the network have been stored in a 3 x 1 matrix.

• As we stated earlier, we have used tanh as the activation function of the network.

• The network has been trained by calculating the error in the output and adjusting the weights so as to
reduce this error.

• Before making any adjustments, it is good for us to know the amount of adjustment that needs to be
done.

• This has been calculated by multiplying the error with the input and then with the gradient of the tanh
function.

• It is after this that we adjusted the weight matrix.

 A Single NN in Python Cont.
• The main goal is to find the optimal set of weights for the neuron that will give
the correct results.

• Different training examples will be used to train the neuron.

• At every step, the error and backpropagate the gradients will be calculated.

• The step for calculating the output of the neuron is known as forwarding
propagation

• While the step of calculating the gradients is known as back propagation.

 A Single NN in Python Code
1 2 3
A Single NN in Python Cont.
The code should run as follows:
Convolutional Networks
What are Convolutional Neural Networks?
• A convolutional neural network is a deep
learning algorithm that takes in an input
image and assigns it importance in
terms of learnable weights and biases to
the various objects/aspects of the image
in order to be able to differentiate them
from each other.

• The algorithm requires less pre-

processing compared to other types of
classification algorithms.

• When using the primitive methods, the

filters are hand-engineered, but in
convolutional networks, these can be
learned with enough training.
 Why Convolutional Neural Networks?
• When doing image processing tasks, a lot of work is needed. This means that the network should
have more layers.

• However, a number of problems arise when we try to add more layers to a neural network.

• First, we risk facing the problem of vanishing gradient.

• However, this problem can be solved to some extent using some sensible activation functions,
such as the ReLU family of activations.

• Another problem associated with a deep fully connected network is that the number of parameters
that are trainable in the network, that is, the weights, can grow rapidly.

• This is an indication that the training may become practically impossible or slow down.

• The model will also be exposed to overfitting.

• Convolutional neural networks can help us solve the second problem by relying on the correlations
between the adjacent inputs in images or the time series.
 CNN Cont.
• Convolutional neural networks are used for the classification of images,
clustering them based on similarity and in object recognition by scenes.

• These types of networks can identify faces, street signs, individuals, platypuses,
eggplants, and other aspects regarding visual data.

• They are used together with text analysis through the Optical Character
Recognition (OCR) in which the images are seen as symbols that are to be
transcribed and sound can be applied once they have been represented visually.

• The use of neural networks in image recognition marks one of the reasons as to
why deep learning has become so popular in the world.

• They are widely applied in fields such as machine visions which are highly used
in robotics, self-driving cars, and treatments for visually impaired.
 Example
• Consider an example where we are dealing with images of cats and dogs.

• The pixels that are close to the eyes of the cat are more likely to be the same
as the ones that are close to the cat’s nose rather than those close to the
dog’s nose.

• This means that not every node in a layer needs to be connected to all other
nodes in the next layer.

• The number of weight parameters that need to be trained in the model will be
cut.

• Convolutional neural networks also provide us with a number of tricks that

make it easy for us to train the network.
 Convolutional Neural Network with Keras
• Keras is a powerful and easy-to-use free open source Python library for developing and evaluating
deep learning models.

• It wraps the efficient numerical computation libraries Theano and TensorFlow and allows you to
define and train neural network models in just a few lines of code.

• Keras contains numerous implementations of commonly used neural-network building blocks such
as layers, objectives, activation functions, optimizers, and a host of tools to make working with
image and text data easier to simplify the coding necessary for writing deep neural network code.

• The Keras library will be used to build a convolutional neural network that will recognize hand-
written digits.

• The MNIST which comes with 70,000 handwritten digits from 0 to 9 will be used.

• The MNIST database (Modified National Institute of Standards and Technology database) is a large
database of handwritten digits that is commonly used for training various image processing
systems. The database is also widely used for training and testing in the field of machine learning.
 Loading the Dataset

• We will load the dataset into two variables, X_train and X_test, and y_train and
y_test will be used to hold the matching digits. Here is the code for this:

from keras.datasets import mnist


(X_train, y_train), (X_test, y_test) = mnist.load_data()

• The data has now been loaded.

 Data-Preprocessing
• We now need to reshape our
data so that it can be fed into our
CNN model.

• This will be accomplished using

the Reshape function provided
by Keras.

• The function takes arguments

which are the number of images,
the shape of every image and
the number of color channels
 Build the Model
• We can now use the
Sequential object of
Keras to build a CNN
model.

• First, we import the

Sequential object from
Keras and a number of
other functions:
 Compile the Model
• To compile the model, we will pass three parameters to it. These include the following:

✴Optimizer- we will use the adam optimizer which will adjust the learning rate
throughout the period of training the model.

✴Loss function- we will use categorical_crossentropy as the loss function which

is a popular choice for classification tasks. A lower score will be an indication
that the model is performing better.

✴Metrics- we will use the accuracy metric to return an accuracy score once the
model has been run on the validation data set.

• The code should be as follows:

model.compile(optimizer='adam', loss='categorical_crossentropy',
metrics=['accuracy'])
 Train the Model
• To train the model, we will use the fit() function provided by Keras.

• We will provide the function with the training data, the target data and the
number of epochs for which we need to run the model.

• Here is the code for this:

model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=3)

 Making Predictions
• We will use the predict() function to make predictions.

• The function will return an array of 10 numbers which are probabilities that an image
has every possible digit from 0 to 9.

• We will run a prediction for the first four images in our test set, and display the first
four values in y_test to compare to the actual results:

model.predict(X_test[:4])

y_test[:4]

• The model is corrected since it has predicted 7, 2, 1 and 0 for the first four images,
the correct values for the y_test.

 Code
from keras.datasets import mnist
model = Sequential()

from keras.models import Sequential

model.add(Conv2D(64, kernel_size=3,
activation='relu', input_shape=(28,28,1)))

from keras.layers import Dense, Conv2D,

Flatten
model.add(Conv2D(32, kernel_size=3,
activation='relu')) model.add(Flatten())

(X_train, y_train), (X_test, y_test) = model.add(Dense(10, activation='softmax'))

mnist.load_data()

model.compile(optimizer='adam',
X_train = X_train.reshape(60000,28,28,1)
loss='categorical_crossentropy',
metrics=['accuracy'])

X_test = X_test.reshape(10000,28,28,1)

model.fit(X_train, y_train,
y_train = to_categorical(y_train)
validation_data=(X_test, y_test), epochs=3)

y_test = to_categorical(y_test)
model.predict(X_test[:4]) y_test[:4]

y_train[0]

End of the Slides

Thank You