Deep Learning

Dr. Irfan Yousuf

Institute of Data Science
UET, Lahore
(Week 2; January 26, 2025)
Outline
• Fundamentals of Deep Learning
Instructor
• Dr. Irfan Yousuf
• Ph.D. and Post-Doc from Korea Institute of Science and
Technology, Seoul, South Korea
• Assistant Professor, Department of Computer Science, New
Campus, UET, Lahore
• [email protected]
• Research Interest: Graph Data Science, Social Network
Analysis, Graph Mining
Deep Learning
• AI is intelligence demonstrated by machines, as opposed to
natural intelligence displayed by animals including humans.

• ML is the study of computer algorithms that can improve

automatically through experience and using data. It is seen as
a part of artificial intelligence.

• DL is part of a broader family of machine learning methods

based on artificial neural networks.
Deep Learning
• AI is anything about man-made intelligence exhibited by
machines.
• ML is an approach to achieve AI.
• DL is one technique to implement ML.
Deep Learning
• Deep Learning is a subset of Machine Learning that uses
mathematical functions to map the input to the output.

• These functions can extract non-redundant information or

patterns from the data, which enables them to form a
relationship between the input and the output.

• This is known as learning, and the process of learning is

called training.
Deep Learning
• Modern deep learning models use artificial neural networks
or simply neural networks to extract information.
• These neural networks are made up of a simple mathematical
function that can be stacked on top of each other and
arranged in the form of layers, giving them a sense of depth,
hence the term Deep Learning.

• Deep Learning was first theorized in the 1980s, but it has

only become useful recently because:
• It requires large amounts of labeled data.
• It requires significant computational power (high performing
GPUs)
Neural Networks
• A Neural Network (NN) is a network of neurons that are used
to process information.
• A neural network is a group of interconnected units called
neurons that send signals to one another. Neurons can be
either biological cells or mathematical models.

• The most advanced data processing machine at the time —

the brain.
Types of Neural Networks
• In neuroscience, a Biological Neural Network (BNN) is a
physical structure found in brains and complex nervous
systems – a population of nerve cells connected by synapses.

• In machine learning, an Artificial Neural Network (ANN)

is a mathematical model used to approximate nonlinear
functions. Artificial neural networks are used to solve
artificial intelligence problems.
Biological Neural Networks
• In the context of biology, a neural network is a population of
biological neurons chemically connected to each other by
synapses.
• A given neuron can be connected to hundreds of thousands of
synapses.
• Each neuron sends and receives electrochemical signals
called action potentials to its connected neighbors.
• A neuron can serve an excitatory role, amplifying and
propagating signals it receives, or an inhibitory role,
suppressing signals instead.
Biological Neural Networks
Biological Neural Networks
Biological Neural Networks
Biological Neural Networks
• The brain is principally composed of about 10 billion
neurons, each connected to about 10,000 other neurons.
• Each neuron receives electrochemical inputs from other
neurons at the dendrites.
• If the sum of these electrical inputs is sufficiently powerful to
activate the neuron, it transmits an electrochemical signal
along the axon and passes this signal to the other neurons
whose dendrites are attached at any of the axon terminals.
These attached neurons may then fire.
• It is important to note that a neuron fires only if the total
signal received at the cell body exceeds a certain level. The
neuron either fires or it doesn't, there aren't different grades
of firing.
Types of Neural Networks
• In neuroscience, a Biological Neural Network (BNN) is a
physical structure found in brains and complex nervous
systems – a population of nerve cells connected by synapses.

• In machine learning, an Artificial Neural Network (ANN)

is a mathematical model used to approximate nonlinear
functions. Artificial neural networks are used to solve
artificial intelligence problems.
Neural Networks
Artificial Neural Networks
Artificial Neural Networks
• An artificial neural network (ANN) is composed of artificial
neurons or nodes.
• The connections of the biological neuron are modeled in
artificial neural networks as weights between nodes.
• A positive weight reflects an excitatory connection, while
negative values mean inhibitory connections.
Artificial Neural Networks
• An ANN consists of connected units or nodes called artificial
neurons, which loosely model the neurons in the brain.
• These are connected by edges, which model the synapses in
the brain.
• Each artificial neuron receives signals from connected
neurons, then processes them and sends a signal to other
connected neurons.
• The "signal" is a real number, and the output of each neuron
is computed by some non-linear function of the sum of its
inputs, called the activation function.
• The strength of the signal at each connection is determined
by a weight, which adjusts during the learning process.
Architecture of Simple Neural Networks
• An Artificial Neural Network is made up of 3 components:
Architecture of Simple Neural Networks
• Input Layer: This is literally the layer that inputs
information for the neural network to process. Each circle
represents 1 feature (a piece of information).

• Hidden Layer: These layers do all the processing for neural

networks. Each layer consists of nodes that mimic our brains’
neurons. These nodes receive information from the previous
layer’s nodes, multiply it by weight and then add a bias to it.

• Output Layer: The output layer is the output/conclusions of

the model derived from all the computations performed.
There can be single or multiple nodes in the output layer.
Artificial Neural Networks
• A neural network is characterized by

• Its pattern of connections between the neurons (called its

architecture).
• Its method of determining the weights on the connections (called
its training, or learning, algorithm).
• Its activation function.
Artificial Neural Networks
Artificial Neuron
• An artificial neuron is a mathematical function based on a
model of biological neurons, where each neuron takes
inputs, weighs them separately, sums them up and passes
this sum through a function (linear or nonlinear) to produce
output.
Perceptron
• A Perceptron is an Artificial Neuron. It is the simplest
possible Neural Network.
• A Perceptron is an algorithm for supervised learning of binary
classifiers. This algorithm enables neurons to learn and processes
elements in the training set one at a time
Perceptron
• The original Perceptron was designed to take a number of
binary inputs and produce one binary output (0 or 1).
• The idea was to use different weights to represent the
importance of each input, and that the sum of the values
should be greater than a threshold value before making a
decision like yes or no (true or false) (0 or 1).
Perceptron
• Input Layer: The input layer consists of one or more input neurons,
which receive input signals from the external world or from other
layers of the neural network.

• Weights: Each input neuron is associated with a weight, which

represents the strength of the connection between the input neuron
and the output neuron.

• Bias: A bias term is added to the input layer to provide the

perceptron with additional flexibility in modeling complex patterns
in the input data.
Perceptron
• Activation Function: The activation function determines the output
of the perceptron based on the weighted sum of the inputs and the
bias term.

• Output: The output of the perceptron is a single binary value, either

0 or 1, which indicates the class or category to which the input data
belongs.

• Training Algorithm: The perceptron is typically trained using a

supervised learning algorithm such as the perceptron learning
algorithm or backpropagation. During training, the weights and
biases of the perceptron are adjusted to minimize the error between
the predicted output and the true output for a given set of training
examples.
Perceptron Example
• Imagine a perceptron (in your brain).
• The perceptron tries to decide if you should go to a concert.

• Is the artist good? Is the weather good?

• What weights should these facts have?
Perceptron Example

Criteria Input Weight

Artists is Good x1 = 0 or 1 w1 = 0.7

Weather is Good x2 = 0 or 1 w1 = 0.6

Friends will Come x3 = 0 or 1 w1 = 0.5

Food is Served x4 = 0 or 1 w1 = 0.3

Free Wifi is Available x5 = 0 or 1 w1 = 0.4

Perceptron Example
•Perceptron Algorithm:
•Set a threshold value
•Multiply all inputs with its weights
•Sum all the results
•Activate the output
Perceptron Example
Threshold value = 1.5

x1 * w1 = 1 * 0.7 = 0.7
x2 * w2 = 0 * 0.6 = 0
x3 * w3 = 1 * 0.5 = 0.5
x4 * w4 = 0 * 0.3 = 0
x5 * w5 = 1 * 0.4 = 0.4

0.7 + 0 + 0.5 + 0 + 0.4 = 1.6 (The Weighted Sum)

Return true if the sum > 1.5 ("Yes I will go to the

Concert")
Single Layer Perceptron
• A Single Layer Perceptron (SLP) is the very basic form of
Artificial Neural Network (ANN).
• Taking all the input values and multiplying them by their weights.
• Then, all of these multiplied values are added together to create
the weighted sum.
• The weighted sum is then applied to the activation function,
producing the perceptron's output.
• It is important to note that the weight of an input is
indicative of the strength of a node. Similarly, an input's
bias value gives the ability to shift the activation function
curve up or down. Bias is similar to Intercept in Linear
Regression.
Single Layer Perceptron: Example
Single Layer Perceptron: Exercise

Implement the following function in a Single Layer

Perceptron.
Single Layer Perceptron: Exercise
Multilayer Perceptron
• It is a neural network where the mapping between inputs and
output is non-linear.
• A Multilayer Perceptron has input and output layers, and one
or more hidden layers with many neurons stacked together.
Artificial Neuron
• An artificial neuron is a mathematical function based on a
model of biological neurons, where each neuron takes
inputs, weighs them separately, sums them up and passes
this sum through a function (linear or nonlinear) to produce
output.
Artificial Neuron
• A neuron is a mathematical function modeled on the working
of biological neurons.
• It is an elementary unit in an artificial neural network.
• One or more inputs are separately weighted.
• Inputs are summed and passed through a nonlinear function
to produce output.
• Every neuron holds an internal state called activation signal.
• Each connection link carries information about the input
signal.
• Every neuron is connected to another neuron via connection
link.
Perceptron vs. Neuron
• Neuron employs non-linear activation function and
perceptron employs only a threshold (linear) activation
function.

• The output of a neuron is not necessarily a binary number

and the output of a perceptron is always a binary number.

• A perceptron can be thought of as a standalone model while

the artificial neuron is the smallest computational unit of a
neural network.
Types of Artificial Neural Network
• Shallow neural networks usually have only one hidden layer.

• Deep neural networks have multiple hidden layers.

ML vs. DL
ML vs. DL
• Generally, machine learning is alternatively termed shallow
learning because it is very effective for smaller datasets.

• Deep learning, on the other hand, is extremely powerful

when the dataset is large.

• It can learn any complex patterns from the data and can draw
accurate conclusions on its own. In fact, deep learning is so
powerful that it can even process unstructured data - data that
is not adequately arranged like text corpus, social media
activity, etc.
ML vs. DL
Types of Artificial Neural Network
Types of Artificial Neural Network
Summary
• Fundamentals of Deep Learning