What is Deep Learning?

Discuss the key characteristics, working and applications of

Deep Learning

Deep Learning is a subset of machine learning that is characterized by the use of deep
neural networks, with multiple layers (hence the term “deep” learning) to perform tasks that
typically require human intelligence. It is inspired by the structure and function of the human
brain, where each layer of neurons processes and transforms the input data to progressively
extract higher-level features.
Deep neural networks (DNNs), consist of interconnected layers of artificial neurons
called nodes. Each node receives input from the previous layer, applies a mathematical
transformation to it, and passes the transformed output to the next layer. The layers closer to
the input are responsible for learning low-level features, while the deeper layers learn more
abstract and complex representations.
 How neural networks progressively extract higher-level features from the raw input

This phenomenon of automatically learning meaningful and informative features (or

representations) from raw data is also referred to as representation learning, which stands as
one of the key strengths of DNNs.

Key characteristics and working of Deep Neural Network

Deep learning works by using artificial neural networks, which are composed of layers of
interconnected nodes (neurons) that process and transform the data through neural network
training.

Key characteristics and working of deep learning include the following:

(1) The Perceptron,
(2) Deep architecture,
(3) Neural Networks, and
(4) Training
 this is how a Deep Learning Algorithm works
• The Perceptron (Neuron)
A perceptron is the structural building block of a deep learning model. It refers to a
simple type of artificial neuron or node in a neural network. It operates by calculating
a weighted sum of its inputs, adding a bias term, and then applying an activation
function to this sum.

Perceptron (or neuron), the structural building block in a neural network

• Deep architecture
The term “deep” refers to the depth of the network, which means it has more than one
hidden layer. Deep architectures enable DNNs to learn and represent intricate features
from data.
• Neural Networks
Deep learning is built upon the concept of artificial neural networks (ANNs), which
 comprises of interconnected nodes, called neurons or units, organized into layers: an
input layer, hidden layers, and an output layer.
• Training
Deep Neural networks are trained using large datasets. The training process comprises
of five main steps: (a) Sampling mini-batch of data and weights initialization, (b)
Forward propagation and Loss calculation, (c) Backpropagation and Optimization, (d)
Repeat the training loop, and (e) Infer

Deep Learning Models

Deep learning encompasses several key architectures, each designed for specific types of data
and tasks. These architectures serve as building blocks for solving a wide range of tasks in
artificial intelligence and machine learning. Here are some of the key deep learning
architectures:

S.No. Deep Learning Models Key characteristics Common applications

1 Feedforward Neural Networks - Also known as multi-layer Classification, Regression

(FNN) perceptrons (MLPs) tasks
- Consist of an input layer,
one or more hidden layers,
and an output layer

2 Convolutional Neural Networks - Designed for structured Image classification,

(CNN) grid data, such as images and object detection, and
videos image generation
- Utilize convolutional layers
to automatically learn image
features

3 Recurrent Neural Networks (RNN), - Suited for sequential data Language modeling,
Long Short-Term Memory (LSTM), like time series, text, and speech recognition, and
Gated Recurrent Unit (GRU) speech machine translation
Networks - Incorporate recurrent
connections to maintain
memory of previous inputs

4 Transformer Networks - Introduced in the context of Machine translation, Text

natural language processing generation (GPT)
(NLP)
- Utilize self-attention
mechanisms to process
sequences
S.No. Deep Learning Models Key characteristics Common applications

5 Generative Adversarial Networks - Comprise a generator and a Image generation, style

(GANs) discriminator network transfer, and data
- Used for generating new augmentation
data samples that resemble a
given dataset

6 Autoencoders - Designed for unsupervised Data denoising, anomaly

learning and dimensionality detection, and feature
reduction. learning
- Consist of an encoder and
decoder to reconstruct input
data

Applications of Deep Learning

Deep learning model used
S.No. Industry Example companies
for

1 Healthcare medical image analysis, Butterfly network for ultrasound devices,

disease diagnosis, drug Arterys for heart image analysis, Deepmind
discovery, and personalized by Google for medical diagnosis, Medtronic
medicine endoscopy device, Sanofi for disease
diagnosis

2 E-commerce personalized product Amazon online, eBay

recommendations, customer
segmentation, and fraud
detection

3 Finance financial fraud detection, American Express, Mastercard for fraud

credit scoring, and detection, Signifyd for anomaly detection,
algorithmic trading Riskified for consumer finance, JP Morgan

4 Automotive self-driving cars for object Tesla, Rivian, Waymo for self-driving
detection, lane detection, and capabilities
path planning

5 Natural Language sentiment analysis, language ChatGPT, a chatbot by OpenAI, Google

Processing (NLP) translation, chatbots, text translate, Microsoft Bing Search, Google
analysis search, Apple News, Meta and X for
sentiment analysis, Grammarly

6 Entertainment content recommendation Netflix, Spotify, YouTube, Amazon Prime

systems, video analysis
 Deep learning model used
S.No. Industry Example companies
for

7 Manufacturing quality control, predictive Toyota in logistics for material handling, ZF

maintenance, and supply Group for quality control, Tesla in supply
chain optimization chain

8 Marketing customer segmentation, Mailchimp for campaigns, IBM Watson for

campaign optimization, and customer segmentation and advertisements,
social media analysis GumGum for ads

9 Customer customer purchase trends, Salesforce Einstein for customer insights,

Relationship buying pattern, predicting Zoho Zia for business insights
Management customer support needs

10 Virtual Assistants speech recognition, speech to Apple Siri, Amazon Alexa, Google
text, transcribing services Assistant

Evolution of Deep Learning: A brief history and Resurgence

A brief history:
Deep Learning might appear as a novel discovery in the field of machine learning, given its
recent name and fame. However, the history of Deep Learning spans several decades, dating
back to 1940s as presented below:
History of Deep Learning

Evolution of Deep Learning from 1940-2010

1950s –> Alan Turing, a British mathematician, first presented the idea that computers would
achieve human-level intelligence
1957 –> Frank Rosenblatt, an American psychologist, introduced the perceptron, a single-
layer neural network
1965 –> Alexey Ivakhnenko, a Soviet mathematician, created a small functional neural
network
1970s –> Limited progress, referred to as the AI winter
1980s –> Backpropagation, a method for training neural networks, was rediscovered by Dr.
Geoffrey Hinton, a British-Canadian psychologist and computer scientist
1989 –> Yann LeCun’s invents machine that can read handwritten digits
1990s –> Multi-layer perceptrons, the inception of CNNs, and LSTM
1999 –> GPUs (Graphics Processing Units) were developed
2000s –> Limited progress in the field of Deep Learning
2012 –> Deep neural network, AlexNet, outperformed other methods for image recognition
and led to the resurgence of Neural Network. Several notable neural network models and
frameworks followed
2017 –> Introduction of Transformer architecture, a game-changer in the field of Deep
Learning models for solving Natural Language Processing tasks
2018 onwards –> Revolution in the AI space took place with the introduction of BERT, GPT-
3, Stable Diffusion models, and systems such as ChatGPT, Bard, Perplexity etc.

Resurgence of Neural Networks:

The resurgence was catalyzed by three key factors:
• Big data
The digital age brought about an unprecedented amount of data. Deep learning
models thrive on vast datasets, and having access to such data allowed for more
effective training of deep neural networks.
• Hardware (GPU):

Neural networks are commonly trained on massive datasets, and often

comprises of millions to billions of parameters. The introduction of Graphics
Processing Units (GPUs) has been instrumental in facilitating this complex
 computation by offering accelerated processing power and parallel computing
capabilities. Unlike Central Processing Units (CPUs), which have a limited number of
cores capable of handling a few software threads at a time, GPUs consist of hundreds
of cores capable of simultaneously managing thousands of threads. The increased
availability of high-performance GPUs at affordable prices has played a pivotal role in
the popularity and success of deep learning.
• Software
o Breakthrough in Deep Learning architectures such as Transformers for
Language Modeling, CNNs for Computer vision, made it possible to handle
complexities in deep neural network and train them effectively
o Development of Deep Learning frameworks such as Pytorch, Tensorflow
made it easier for developers to work with deep networks