Unit 1: Introduction to Deep

Learning

Prof . Sachin S. Patil

D . Y. Patil University Ambi Pune
Prof.Sachin Sambhaji Patil 1
 Need of Deep Learning

• Deep learning has aided image classification, language translation,

speech recognition.

• It can be used to solve any pattern recognition problem and without

human intervention.

Prof.Sachin Sambhaji Patil 2

 Need of Deep Learning

• Deep learning is a subset of machine learning, which is essentially a

neural network with three or more layers. These neural networks
attempt to simulate the behavior of the human brain—albeit far
from matching its ability—allowing it to “learn” from large amounts
of data. While a neural network with a single layer can still make
approximate predictions, additional hidden layers can help to
optimize and refine for accuracy.
Prof.Sachin Sambhaji Patil 3
 Need of Deep Learning

• Deep learning is a branch of machine learning which is based on

artificial neural networks. It is capable of learning complex patterns
and relationships within data. In deep learning, we don’t need to
explicitly program everything.

Prof.Sachin Sambhaji Patil 4

 Need of Deep Learning

1. Deep Learning is a subfield of Machine Learning that involves the

use of neural networks to model and solve complex problems. Neural
networks are modeled after the structure and function of the human
brain and consist of layers of interconnected nodes that process and
transform data.

Prof.Sachin Sambhaji Patil 5

 Need of Deep Learning

2. The key characteristic of Deep Learning is the use of deep neural

networks, which have multiple layers of interconnected nodes. These
networks can learn complex representations of data by discovering
hierarchical patterns and features in the data. Deep Learning
algorithms can automatically learn and improve from data without the
need for manual feature engineering.

Prof.Sachin Sambhaji Patil 6

 Need of Deep Learning

3. Deep Learning has achieved significant success in various fields,

including image recognition, natural language processing, speech
recognition, and recommendation systems. Some of the popular Deep
Learning architectures include Convolutional Neural Networks (CNNs),
Recurrent Neural Networks (RNNs), and Deep Belief Networks (DBNs).

Prof.Sachin Sambhaji Patil 7

 Need of Deep Learning

4. Training deep neural networks typically requires a large amount of

data and computational resources. However, the availability of cloud
computing and the development of specialized hardware, such as
Graphics Processing Units (GPUs), has made it easier to train deep
neural networks.

Prof.Sachin Sambhaji Patil 8

 Fundamentals of Deep Learning

• Deep learning is the branch of machine learning which is based on

artificial neural network architecture. An artificial neural network or
ANN uses layers of interconnected nodes called neurons that work
together to process and learn from the input data.

Prof.Sachin Sambhaji Patil 9

Deep neural network

• The Input Layer

• The Hidden Layer

• The Output Layer

Prof.Sachin Sambhaji Patil 10

 Deep neural network
• input layer, that takes the input data. The hidden layer, which
performs various computation on the input data and the output layer,
which in the above visualization is binary. It is to be noted that a
neural network can have multiple hidden layers.

• These neural networks are used to predict the output as well as

perform classification on the data. The standard notion is that the
neural network learns the pattern of data, then performs predictions
that fall in the same line as the pre-specified pattern.

Prof.Sachin Sambhaji Patil 11

 Fundamentals of Deep Learning
• In a fully connected Deep neural network, there is an input layer and
one or more hidden layers connected one after the other.

• Each neuron receives input from the previous layer neurons or the
input layer. The output of one neuron becomes the input to other
neurons in the next layer of the network, and this process continues
until the final layer produces the output of the network. The layers of
the neural network transform the input data through a series of
nonlinear transformations, allowing the network to learn complex
representations of the input data.
Prof.Sachin Sambhaji Patil 12
Deep Learning

Prof.Sachin Sambhaji Patil 13

Difference between Machine Learning ML and Deep Learning DL

Prof.Sachin Sambhaji Patil 14

Machine Learning Vs. Deep Learning

Prof.Sachin Sambhaji Patil 15

 Bias Variance Tradeoff
• Bias in machine learning refers to the difference between a model’s
predictions and the actual distribution of the value it tries to predict.

• Variance stands in contrast to bias; it measures how much a

distribution on several sets of data values differs from each other.

Prof.Sachin Sambhaji Patil 16

 Different combinations of bias and variance
• Different combinations of bias and variance in machine learning models and
the results they provide.
• Low bias, low variance: ideal model
• A machine learning model with low bias and low variance is considered ideal
but is not often the case in the machine learning practice, so we can speak of
“reasonable bias” and “reasonable variance.”
• Low bias, high variance: results in overfitting
• This combination results in inconsistent predictions that are accurate on
average. It occurs when a model has too many parameters and fits too closely
to the training data.
• High bias, low variance: results in underfitting
• Predictions are consistent but inaccurate on average in this scenario. This
happens when the model doesn’t learn well from the training data or has too
few parameters, leading to underfitting issues.
• High bias, high variance: results in inaccurate predictions
• With both high bias and high variance, the predictions are both inconsistent
and inaccurate on average.
Prof.Sachin Sambhaji Patil 17
bias and variance

Prof.Sachin Sambhaji Patil 18

How to achieve a bias-variance tradeoff?

Prof.Sachin Sambhaji Patil 19

 a bias-variance tradeoff

• bias and variance are interdependent. In other words, lowering a

model’s bias leads to an increase in its variance and vice versa.

• This relationship between bias and variance is known as the bias-

variance tradeoff.

Prof.Sachin Sambhaji Patil 20

 a bias-variance tradeoff

Underfitting Compromise Overfitting

Model
Low Medium High
Complexity

Bias High Low Low

Variance Low Low High

Prof.Sachin Sambhaji Patil 21
 Parametric & Non Parametric Model
• A parametric model can predict future values using only the
parameters.

• While nonparametric machine learning algorithms are often slower

and require large amounts of data, they are rather flexible as they
minimize the assumptions they make about the data.

Prof.Sachin Sambhaji Patil 22

 Parametric vs. Nonparametric modeling

• Parametric models deal with discrete values, and nonparametric

models use continuous values.
• Parametric models are able to infer the traditional measurements
associated with normal distributions including mean, median, and
mode. While some nonparametric distributions are normally
oriented, often one cannot assume the data comes from a normal
distribution.

Prof.Sachin Sambhaji Patil 23

 What are non-parametric methods?

• Non-parametric methods do not make any assumptions

about the underlying distribution of the data. Instead, they
rely on the data itself to determine the relationship between
variables. These methods are more flexible than parametric
methods but can be less powerful.

Prof.Sachin Sambhaji Patil 24

 What are parametric methods?

• Parametric methods are statistical techniques that make

assumptions about the underlying distribution of the data. These
methods typically use a pre-defined functional form for the
relationship between variables, such as a linear or exponential model.

Prof.Sachin Sambhaji Patil 25

• https://www.geeksforgeeks.org/difference-between-parametric-and-
non-parametric-methods/

Prof.Sachin Sambhaji Patil 26

 Parametric Methods Non-Parametric Methods
Parametric Methods uses a fixed number of parameters Non-Parametric Methods use the flexible number of
to build the model. parameters to build the model.

Parametric analysis is to test group means. A non-parametric analysis is to test medians.

It is applicable only for variables. It is applicable for both – Variable and Attribute.

It always considers strong assumptions about data. It generally fewer assumptions about data.

Parametric Methods require lesser data than Non- Non-Parametric Methods requires much more data than
Parametric Methods. Parametric Methods.

Parametric methods assumed to be a normal There is no assumed distribution in non-parametric

distribution. methods.

Parametric data handles – Intervals data or ratio data. But non-parametric methods handle original data.

When we use non-parametric methods then the result or

Here when we use parametric methods then the result
outputs generated cannot be seriously affected by
or outputs generated can be easily affected by outliers.
outliers.

Parametric Methods can perform well in many situations Similarly, Non-Parametric Methods can perform well in
but its performance is at peak (top) when the spread of many situations but its performance is at peak (top)
each group is different. when the spread of each group is the same.

Parametric methods have more statistical power than Non-parametric methods have less statistical power than
Non-Parametric methods. Parametric methods.

As far as the computation is considered these methods As far as the computation is considered these methods
are computationally faster than the Non-Parametric are computationally slower than the Parametric
methods. methods.

Examples: Logistic Regression, Naïve Bayes Prof.Sachin

Model, etc.Sambhaji Patil
Examples: KNN, Decision Tree Model, etc. 27
• https://www.youtube.com/watch?v=9sEiMp8bCnY

Prof.Sachin Sambhaji Patil 28

 Applications of Deep Learning
• What are the different applications of Deep Learning?
• 1. Healthcare
• The healthcare sector has long been one of the prominent
adopters of modern technology to overhaul itself. As such, it is
not surprising to see Deep Learning finding uses in interpreting
medical data for the diagnosis, prognosis & treatment of
diseases , drug prescription .
analyzing MRIs, CT scans, ECG, X-Rays, etc., to detect and notify
about medical anomalies , personalizing treatment , monitoring
the health of patients and more

Prof.Sachin Sambhaji Patil 29

 Applications of Deep Learning
• 2. Personalized Marketing

• Personalized marketing is a concept that has seen much action in the

recent few years. Marketers are now aiming their advertising campaigns
at the pain points of individual consumers, offering them exactly what
they need. And Deep Learning is playing a significant role in this.

Prof.Sachin Sambhaji Patil 30

 Applications of Deep Learning
• 3. Financial Fraud Detection
• Virtually no sector is exempt from the evil called “fraudulent transactions” or
“financial fraud”. However, it is the financial corporations (banks, insurance
firms, etc.) that have to bear the brunt of this menace the most.
• identity theft

• insurance fraud

• investment fraud

• fund misappropriation
Prof.Sachin Sambhaji Patil 31
 Applications of Deep Learning
• 4. Natural Language Processing
• NLP or Natural Language Processing is another prominent area
where Deep Learning is showing promising results.

• Natural Language Processing, as the name suggests, is all about

enabling machines to analyze and understand human language.

Prof.Sachin Sambhaji Patil 32

 Applications of Deep Learning
• 5. Autonomous Vehicles
• Deep learning is playing a huge role in realizing the perceived motives
of self-driving vehicles of reducing road accidents, helping the
disabled drive, eliminating traffic jams, etc.

Prof.Sachin Sambhaji Patil 33

 Applications of Deep Learning
• 6. Fake News Detection
• Deep Learning proposes a way to deal with the menace of fake news by
using complex language detection techniques to classify fraudulent
news sources. This method essentially works by gathering information
from trusty sources and juxtaposing them against a piece of news to
verify its validity.

Prof.Sachin Sambhaji Patil 34

 Applications of Deep Learning
• 7. Facial Recognition
• Facial Recognition is the technological method of identifying
individuals from images and videos by documenting their faces. It uses
advanced biometric technology to record a person’s face and match it
against a database to extract their identity.

• Deep Learning enforced Facial Recognition works by recording face

embedding's and using a trained model to map them against a huge
database of millions of images.

Prof.Sachin Sambhaji Patil 35

 Applications of Deep Learning
• 8. Recommendation Systems
• Deep Learning models process user data acquired from different sources
and compile them to extract consumer info. This information then goes
into deep learning-based recommender systems to generate
appropriate suggestions for the users.

Prof.Sachin Sambhaji Patil 36

 Applications of Deep Learning
• 9. Smart Agriculture
• Deep Learning algorithms capture and analyse agriculture data from the
above sources to improve crop health and soil health, predict the
weather, detect diseases, etc.
• Deep learning also finds uses in the field of crop genomics. Experts use
neural networks to determine the genetic makeup of different crop plants
and use it for purposes like increasing resilience to natural phenomena
and diseases.

• increase crop yield per unit area

• breeding high-quality hybrids

Prof.Sachin Sambhaji Patil 37
 Applications of Deep Learning
• 10. Space Travel
• Deep Learning algorithms capture and analyse agriculture data from the
above sources to improve crop health and soil health, predict the weather,
detect diseases, etc.

• Automating the landing of rockets

• Building space flight systems that can make intelligent decisions without
human intervention

• Also, deep learning will play an active role in helping future rovers on Mars
to navigate and deduce their surroundings better and more independently.
Prof.Sachin Sambhaji Patil 38
 Job Roles in Deep Learning
• 1. Deep Learning Engineer
• Deep Learning Engineers are experts in Deep Learning platforms. Their
primary responsibility is to develop advanced neural models that can
mimic brain functions. Also, they have to run ML tests and experiments
and implement appropriate Algorithms for the optimal functioning of
neural networks. Deep Learning Engineers must have experience with
best practices in software engineering, including unit testing, continuous
integration, and source control.

Prof.Sachin Sambhaji Patil 39

 Job Roles in Deep Learning
• 2. Deep Learning Scientist
• Deep Learning Scientists are essentially researchers in the domain who
are tasked with the job of continuously identifying and finding ways to
optimize the performance of ML/DL models. Much like the Deep
Learning Engineer, Deep Learning Scientists have to design, develop,
configure, and optimize deep neural networks for performing various
tasks like detection, classification etc.

Prof.Sachin Sambhaji Patil 40

 Job Roles in Deep Learning

• 3. NLP Scientist
• NLP Scientists design and developing intuitive DL models and
applications that can both learn the patterns of speech of natural
human languages and also translate words spoken in one language
into other languages. Their primary focus is to create systems that
can understand human languages just like a human would.

Prof.Sachin Sambhaji Patil 41

 Job Roles in Deep Learning
• 4. Image Processing Engineer
• Image Processing Engineers are DL experts who develop and test image
processing and data classification algorithms. They analyze data and
develop methods by using ML techniques for feature extraction and
classification. Visualizing the findings and communicating the results is one
of the most crucial aspects of an Image Processing Engineer’s job. Hence,
they must know how to create visual interpretations of data, write detailed
reports, and create presentations.

Prof.Sachin Sambhaji Patil 42

 Job Roles in Deep Learning
• 5. Data Scientist.
• AI careers: Data scientists collect raw data, analyze it, and glean insights
for a wide range of purposes. They use various technology tools .

• 6. Data Engineer.
• Data engineers are responsible for building and maintaining systems that
capture data from various sources, organize it effectively,

Prof.Sachin Sambhaji Patil 43

 Job Roles in Deep Learning
• 7. Big Data Engineer/Architect

• Big data engineering professionals and architects develop ecosystems

that enable various business verticals and technologies.

• 8. An artificial intelligence engineer works with traditional machine

learning techniques like neural networks and natural language
processing.

Prof.Sachin Sambhaji Patil 44

 Job Roles in Deep Learning

• 9. Research Scientist:
• A research scientist focuses on researching and developing new
machine learning algorithms and technologies.

Prof.Sachin Sambhaji Patil 45

 Job Roles in Deep Learning
• Software Engineer
• Research Analyst
• Data Analyst
• Data Scientist
• Data Engineer
• Neuroinformatics
• Bioinformatician
• Image Recognition
• Software Developer
• Research Scientist
• Research Fellow
• Instructor for Deep Learning
• Applied Scientist
• Full Stack Web Developer for Deep Learning
• Lead Manager – Deep Learning
• Natural Language Process EngineerProf.Sachin Sambhaji Patil 46
 Normalization and Regularization
• What is normalization in deep learning?
• Normalization is a scaling technique in Machine Learning applied
during data preparation to change the values of numeric
columns in the dataset to use a common scale. It is not necessary
for all datasets in a model. It is required only when features of
machine learning models have different ranges.

Prof.Sachin Sambhaji Patil 47

 Normalization and Regularization

• What is regularization in deep learning?

• Regularization is a technique used in machine learning and

deep learning to prevent overfitting and improve the
generalization performance of a model. It involves adding a
penalty term to the loss function during training.

Prof.Sachin Sambhaji Patil 48

 Normalization and Regularization
• Why is normalization required before regularization?
• The reason to normalise your variables beforehand is to
ensure that the regularisation term λ regularises/affects the
variable involved in a (somewhat) similar manner.

Prof.Sachin Sambhaji Patil 49

 Normalization and Regularization

• What is the difference between normalization and

regularization?
• Normalization helps to prepare the data for the learning
algorithm, while regularization helps to prevent overfitting and
improve generalization. The choice of which technique to use
depends on the specific problem and algorithm being used.

Prof.Sachin Sambhaji Patil 50

 Normalization and Regularization

• Standard methods to normalize data in deep learning include Min-

Max Scaling, Z-score Standardization, and Batch Normalization. In
Min-Max Scaling, which rescales data to a fixed range, typically 0 to
1, is particularly useful for models where input values must be
bounded

Prof.Sachin Sambhaji Patil 51

 Tools and Framework used in Deep Learning
• 1. TORCH/PyTorch
• Torch is a scientific computing framework that offers broad support for
machine learning algorithms. It is a Lua based deep learning framework
and is used widely amongst industry giants such as Facebook, Twitter, and
Google.
• It employs CUDA along with C/C++ libraries for the processing and was
made to scale the production of building models and overall flexibility. As
opposed to Torch, PyTorch runs on Python, which means that anyone with
a basic understanding of python can get started on building their deep
learning models.
• In recent years, PyTorch has seen a high level of adoption within the deep
learning framework community and is considered to be quite the
competitor to TensorFlow. PyTorch is basically a port to Torch deep
learning framework used for constructing deep neural networks and
executing tensor computations that are high in terms of complexity.
Prof.Sachin Sambhaji Patil 52
 Tools and Framework used in Deep Learning
• Highlights of PyTorch
• Excellent at rapid prototyping

• Strong support for GPUs as parallel programs can be implemented on

multiple GPUs

• Provides cleaner interface and is easier to use

• Facilitates the exchange of data with external libraries

Prof.Sachin Sambhaji Patil 53

 Tools and Framework used in Deep Learning
• 2. Jupyter notebook
• Jupyter notebook is an open-source tool that is a critical part of data
science. It provides an interactive environment and facilitates interactive
data exploration, promoting reproducible research, and fostering
collaboration and knowledge sharing.

• It offers a single document that contains mathematical equations,

narrative text, visualizations, and any other rich media. This enables users
to make their work easily understandable and shareable by developing and
visualizing the results and adding information, charts, and formulas.
Prof.Sachin Sambhaji Patil 54
 Tools and Framework used in Deep Learning
• 3. PyCharm is an integrated development environment (IDE) used for
programming in Python. It provides code analysis, a graphical debugger,
an integrated unit tester, integration with version control systems, and
supports web development with Django. PyCharm is developed by
the Czech company JetBrains.

• PyCharm was released to the market of the Python-focused IDEs to

compete with PyDev (for Eclipse) or the more broadly focused Komodo
IDE by ActiveState.
Prof.Sachin Sambhaji Patil 55
 Tools and Framework used in Deep Learning
• 3. PyCharm

• It is cross-platform, working on Microsoft Windows, macOS and Linux.

PyCharm has a Professional Edition,

• It is released under a proprietary license and a Community Edition

released under the Apache License.

• PyCharm Community Edition is less extensive than the Professional

Edition.
Prof.Sachin Sambhaji Patil 56
 Tools and Framework used in Deep Learning

• PyCharm Professional Edition is free for open-source projects and for

some educational uses. There is also an Academic license, which is
discounted for other educational use.

• PyCharm Community Edition is distributed under Apache 2 license.

The source code is available on GitHub.

Prof.Sachin Sambhaji Patil 57

 Tools and Framework used in Deep Learning
• 4. TensorFlow
• TensorFlow is inarguably one of the most popular deep learning
frameworks. Developed by the Google Brain team, TensorFlow supports
languages such as Python, C++, and R to create deep learning models along
with wrapper libraries. It is available on both desktop and mobile.
• The most well-known use case of TensorFlow has got to be Google
Translate coupled with capabilities such as natural language processing,
text classification, summarization, speech/image/handwriting
recognition, forecasting, and tagging.
• TensorFlow’s visualization toolkit, TensorBoard, provides effective data
visualization of network modeling and performance.

Prof.Sachin Sambhaji Patil 58

 Tools and Framework used in Deep Learning
• 4. TensorFlow
• TensorFlow is one of the most preferred deep learning frameworks as it is
Python-based, supported by Google, and comes loaded with top-notch
documentation and walkthroughs to guide you.
• Highlights of TensorFlow
• Robust multiple GPU support
• Graph visualization and queues using TensorBoard
• Known to be complex and has a steep learning curve
• Excellent documentation and community support
Prof.Sachin Sambhaji Patil 59
 Tools and Framework used in Deep Learning
• 5. KERAS

• Keras library was developed, keeping quick experimentation as its USP.

Written in Python, the Keras neural networks library supports both
convolutional and recurrent networks that are capable of running on
either TensorFlow or Theano.

• As the TensorFlow interface is tad challenging and can be intricate for

new users, Keras deep learning framework was built to provide a
simplistic interface for quick prototyping by constructing active neural
networks that can work with TensorFlow.
Prof.Sachin Sambhaji Patil 60
 Tools and Framework used in Deep Learning
• 5. KERAS

• In a nutshell, Keras is lightweight, easy-to-use, and has a minimalist

approach. These are the very reasons as to why Keras is a part of
TensorFlow’s core API.

• The primary usage of Keras is in classification, text generation, and

summarization, tagging, translation along with speech recognition, and
others. If you happen to be a developer with some experience in Python
and wish to delve into deep learning, Keras is something you should
definitely check out.
Prof.Sachin Sambhaji Patil 61
 Tools and Framework used in Deep Learning
• Highlights of Keras
• Easy-to-understand and consistent APIs

• Seamlessly integrates with TensorFlow workflow.

• Supports multiple deep learning backends

• Built-in support for distributed training and multi-GPU parallelism

Prof.Sachin Sambhaji Patil 62