Scribd
Upload
3 views

Lesson 4 - Deep Learning

Deep learning materials

Uploaded by

star
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
3 views

Lesson 4 - Deep Learning

Deep learning materials

Uploaded by

star
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Deep

Learning
What is Deep Learning?

Natural Language
Computer Vision Processing Speech Analysis

● Image Classification ● Language Translation ● Voice Activation


● Facial Recognition ● Sentiment Analysis ● Speech-to-Text
● Object Detection ● ChatGPT ● Text-to-Speech
● Object Tracking ● Text Summarization
● Object Segmentation
● Image Generation
How is it achieved?
Neural Networks
● The backbone of deep learning
● Deep models → add layers after layers
● Neurons are the basic building blocks of a neural
network:
○ Depends on what you are trying to achieve
○ Very flexible, structure them accordingly
● Deep Learning models comprises of multiple
different layers with different purposes
● TL;DR: It’s all MATH
Key Components
Input Layer
● The input layer receives the raw data that will be
processed by the network.
○ Tabular Data
○ 2D images
○ Text
○ Audio waves
● The shape and size of the input layer depend on
the type of data being fed into the model
Key Components
Hidden Layer
● Hidden layers perform the bulk of the
computations through a series of
transformations and feature extractions
● ‘Hidden Representations’ of the data
○ When predicting whether an image is a
dog or a cat → look at the eyes, ears, nose,
facial features
Key Components
Output Layer
● The output layer is the final layer in a neural
network
● It formats the processed information to be
suitable for the task at hand, such as
classification or regression
○ E.g., predicting between dogs, cats and
birds → need 3 neurons at the end
Forward Propagation
Getting the Output Values
● For the input, just multiply with the weights to
Weights
get to the next layer
● Apply the activation function
○ A non-linear transformation
● Repeat steps 1-2 for each layer in the hidden
layer, up till the output to get a value

Activation
Function
Forward Propagation
Activation Functions
● Apply non-linear transformations to better learn
the ‘hidden features’
● A linear function if just a linear mapping, and
you cannot understand the intricacies of the
data
● Examples of activation functions:
○ Sigmoid Function → map to (0, 1)
○ Tanh Function → map to (-1, 1)
○ ReLu Function → map to (0, infinity)
○ ReLu variants (Leaky ReLu)
● Different activation functions for different uses
Training Neural Networks
End Goal
● Train and update the weights
Weights
● Weights should be robust enough to learn the
data distributions and give an accurate output
● Done through a key process: backpropagation
● Basically a bunch of math, mainly in calculus
and linear algebra

Activation
Function
Backpropagation
Loss Function
● Calculate the ‘loss’, which is the difference
Weights
between the expected value and output value
● Regression:
○ Mean-Squared Error
○ Mean Absolute error
● Classification:
○ Log-loss
○ Category Cross-entropy

Target: 5

Activation
Function
Backpropagation
Updating weights
● With the loss calculated, we can ‘backpropagate’
Weights
the loss in reverse
● Start with the last hidden layer
● Calculate the gradient
● Update the weight with this formula:

Gradient: -0.3
New Value: 0.57
Learning Rate
Training Neural Networks
End Result
● Train the model over an appropriate number of
epochs
● Take note: Overfitting occurs in deep learning
models too
● Train the model over a number of epochs, use a
validation and test set to check for overfitting,
stop training the model when a good enough
result or an ideal number of epochs is reached
Applications
Computer Vision
Understanding Images
● Images can be represented by a 2D matrix for
black and white images, or 3D matrix (R,G,B
colours)
● Each pixel value is represented by a number
between 0-255 depending on its intensity
● Convolutional Neural Networks are used to
understand these 2D inputs
Computer Vision
Convolutional Neural Network
● Hidden Layers are convolutional layers
● Use 2D filters (3x3, 5x5, 7x7) and apply it across
the image
● These filters can detect edges and changes
across the whole image
● Pooling layers can be added to ‘aggregate’ the
results and reduce the output sizes of these
filters, preventing overfitting
Computer Vision
Convolutional Neural Network
● After the convolutional and pooling layers, flatten the output into a fully connected layer,
just like a regular neural network
● Use it for prediction or regression tasks accordingly
Natural Language Processing
Attention Mechanism
● Given these 2 sentences:
○ My favourite fruit is apple
○ My favourite brand is apple
● The model must be able to differentiate these 2 sentences
● Use transformers, another deep learning architecture
Natural Language Processing
Understanding Texts
● How to represent texts as values in a neuron?
○ Use tokenization and convert it into a
corpus, a vector representing the words in
the sentence
● Positional Embeddings to understand its
position
● Calculate the attention scores for each word
embeddings
○ E.g., is the word apple related to the word
brand or fruit more?
Natural Language Processing
Transformers
● Depending on the task, use encoders and
decoders, ‘hidden layers’ of the neural network
accordingly
● Basically just ‘specialized’ neurons in a neural
network
● Example of an end goal:
○ Translating English to Spanish
○ Input vector: Good morning
○ Output vector: Buenos días
Exercise: CNNs

You might also like