0% found this document useful (0 votes)
48 views40 pages

Lesson 1 - Introduction To Deep Learning

Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
48 views40 pages

Lesson 1 - Introduction To Deep Learning

Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 40

Deep Learning

Lesson 1—Introduction to Deep Learning

© Simplilearn. All rights reserved.


Learning Objectives

Understand the evolution of Deep Learning from Artificial


Intelligence and Machine Learning

Describe the meaning and definition of Deep Learning with


the help of a case study

Explore the meaning, process, and types of neural networks with a


comparison to human neurons

Identify the platforms and programming stacks used in Deep


Learning
Introduction to Deep Learning
Topic 1: Evolution of Deep Learning
Artificial Intelligence and Machine Learning

• Traditional analytics relies on hard-coded rules.


• Artificial Intelligence (AI) systems learn by extracting patterns
from input and output data.
• Machine Learning (ML) relies on learning patterns based on
sample data.
• Programs learn from labeled data (supervised learning),
unlabelled data (unsupervised learning), or a combination of
both (semi-supervised learning).
Evolution of Deep Learning

Artificial Intelligence (AI) came around in the middle of 1900s when scientists tried to envision
intelligent machines. Machine Learning evolved in late 1900s. This allowed scientists to train
machines for AI. In early 2000s, certain breakthroughs in multi-layered neural networks facilitated
the advent of Deep Learning. It aids in developing more complex applications, like image processing
or language modeling.
Image Source: Nvidia
Relationship between Artificial Intelligence and Deep Learning

Machine Learning is an approach or subset of Artificial Intelligence that is based on


the idea that machines can be given access to data along with the ability to learn
from it. Deep Learning takes Machine Learning to the next level.

Artificial Intelligence

Machine Learning

Deep Learning

Image Source : Nvidia


Traditional to Deep Learning at a Glance

As you go from the rule-based systems to the deep


learning ones, more complex features and input-
output relationships become learnable.

Source: “Deep learning” by Ian Goodfellow


Use Cases of Artificial Intelligence

Self driving cars Credit card fraud detection Call center agent match

Chatbots Amazon recommendations Image tagging

Amazon Echo and Alexa Concierge robot Connie Baidu AI medical assistant

Amazon GO retail store Email spam Face recognition

Speech recognition Google AlphaGo Apple’s Siri


Introduction to Deep Learning
Topic 2: What is Deep Learning?
What Is Deep Learning?

Deep Learning is a specialized form of Machine Learning that uses supervised,


unsupervised, or semi-supervised learning to learn from data representations.
It is similar to the structure and function of the human nervous system, where
a complex network of inter-connected computation units work in a
coordinated fashion to process complex information.
Deep Learning at a Glance

There are many aspects of Deep Learning as shown below. These will be covered in detail
in subsequent chapters of this course.

Multi-layered neural
Training of large neural
networks 2 3 networks

Multiple levels of
hierarchical representations 4
Multiple non-linear
1
transformations

High-level data abstractions 7 5 Pattern recognition


model
6

Feature extraction
Drivers of Deep Learning
Availability of multi-
layered learning network

Ability of GPUs to perform Ability to leverage Big Data


parallel computing

Drivers of Deep
Learning Expanded use of high-
Availability of large
performing graphics
amount of labeled data
processing units (GPU)

Improved performance Improved scale of data and


of Neural Networks size of neural networks
Case Study: Sowing App

What?
Microsoft India collaborated
with ICRISAT (International
Crop Research Institute for the
Semi-Arid Tropics) to develop a
“Sowing App.”

Image Source: ICRISAT


Case Study: Sowing App

What?

• The timing of sowing is the biggest differentiator between a good and a


failed crop, especially for rainfed crops.
• The app guides farmers on soil conditions and weather and provides
rainfall predictions.
• It helps them get a higher crop yield and have better price control.
• The app calculates crop yield using data from geostationary satellite
images.
Case Study: Sowing App

Notifications are sent to farmers on their phones in their native language:


• Short term weather prediction, especially rainfall
• Soil quality data
• Previous crop history
What?
• When to sow
• When not to sow
• When soil moisture is sufficient for seed gemination
• Pest threats to the crops
• Price forecasts for the crops

Farmers who followed the


prediction benefited by
reaping a 30% higher yield.
Case Study: Sowing App

• The pilot was implemented in Devanakonda Mandal in Kurnool district


of Andhra Pradesh for the groundnut crop.
What?
• The Sowing advisory was developed in 2016 and used by 175 farmers in
a pilot phase. The app is now being scaled in all 13 districts.
• Farmers in a few villages in the following states are now using this app:
Where? • Karnataka
• Maharashtra
• Andhra Pradesh
• Madhya Pradesh
• Telangana
Case Study: Sowing App

• Azure cloud platform was used to deploy this app, along with Artificial
What? Intelligence, Machine Learning, Big Data, and Analytics.
• The technologies that are used to power the app are:
• Cloud Machine Learning
Why? • Satellite imagery
• Advanced analytics
• Microsoft Cortana Intelligence Suite
How? • Machine Learning
• Power BI
• Climate data for the Devanakonda area of Andhra Pradesh from 1986 to
2015 was collected to predict the crop sowing period.
Deep Learning Path
The path to master Deep Learning can be divided into four phases:

Applied Math and


Deep Networks: Deep Learning
Introduction Machine Learning
Modern Practices Research
Basics

You begin by an introduction to the


idea of Deep Learning.
Deep Learning Path

Applied Math and


Deep Networks: Deep Learning
Introduction Machine Learning
Modern Practices Research
Basics

Next, review basics of Math and


core Machine Learning algorithms
used later in Deep Learning.
Deep Learning Path

Applied Math and


Deep Networks: Deep Learning
Introduction Machine Learning
Modern Practices Research
Basics

After that, learn the most


popular forms of Deep
Learning neural networks
prevalent currently.
Deep Learning Path

Applied Math and


Deep Networks: Deep Learning
Introduction Machine Learning
Modern Practices Research
Basics

Finally, review some of the


more recent advances in
Deep Learning.
Introduction to Deep Learning
Topic 3: Artificial Neural Networks
A Peek into the Human Brain
Biological Neuron

• A mammalian brain has billions of neurons.


• Neurons are interconnected nerve cells in the human brain that are involved
in processing and transmitting chemical and electrical signals.
• They take input and pass along outputs.
• A human brain can learn how to identify objects from photos.
• For example, it can learn to identify the characteristics of chairs and thereby
increase its probability of identifying them over time.
A Peek into the Human Brain
Biological Neuron

Synapse

• Dendrites are branches that receive information from other neurons.


• Cell nucleus or Soma processes the information received from dendrites.
• Axon is a cable that is used by neurons to send information.
• Synapse is the connection between an axon and other neuron dendrites.
Human Brain Vs. Artificial Neural Networks

• The computational models in Deep Learning are loosely inspired by the human brain.
• The multiple layers of training are called Artificial Neural Networks (ANN).
• ANNs are processing devices (algorithms or actual hardware) that are modelled on the
neuronal structure of the mammalian cerebral cortex but on a much smaller scale.

Note: In second and third lessons, you will learn more about Artificial Neural Network, Neurons and Perceptron.
Artificial Neural Network: Definition

“Artificial Neural Network is a computing system made up of a number of


simple, highly interconnected processing elements which process information
by their dynamic state response to external inputs.”
- Robert Hecht-Nielsen
Artificial Neural Network: Process

• Artificial Neural Networks contain layers of neurons.

Neurons • A neuron is a computational unit that calculates a piece of


information based on weighted input parameters.

Inputs • Inputs accepted by neuron are separately weighted.


• Inputs are summed and passed through a non-linear function
to produce output.
Outputs
• Each layer of neurons detects some additional information,
such as edges of things in a picture or tumors in a human
Neurons
body.
• Multiple layers of neurons can be used to detect additional
information about input parameters.
Artificial Neural Network: Process

Neurons
• Artificial Neural Network is an interconnected group of
nodes akin to the vast network of layers of neurons in
Nodes
a brain.

Outputs
• Each circular node represents an artificial neuron and an
arrow represents a connection from the output of one
neuron to the input of another.
Neurons
Artificial Neural Network: Process

Neurons

• Inputs are passed into the first layer.


Nodes
• Individual neurons receive the inputs, with each of them
receiving a specific value.
Inputs
• After this, an output is produced based on these values.

Neurons
Artificial Neural Network: Process

• The outputs from the first layer are then passed into the
Neurons second layer to be processed.
• This continues until the final output is produced.
Nodes • The assumption is that the correct output is predefined.
• Each time data is passed through the network, the end
Inputs result is compared with the correct one, and tweaks are
made to their values until the network creates the
Outputs correct final output each time.
Types of Neural Networks

Some of the commonly used neural networks are as follows:

ANN
(Artificial Neural Network)
CNN DNN
(Convolutional Neural Network) (Deep Neural Network)

RNN DBN
(Recurrent Neural Network) (Deep Belief Network)

Types of Neural Networks


Types of Neural Networks
Use Cases

ANN
Computational DNN
Neuroscience
CNN Acoustic
Modeling
Image
Processing

DBN
RNN
Drug
Speech
Discovery
Recognition

Use cases of different Neural


Networks
Case Study: DeepFace
What?

• DeepFace is Facebook’s facial recognition system created using


Deep Learning.
• It uses a nine layer neural network with more than 120 million
connection weights.
• The researchers used four million images uploaded by
Facebook users.
• Facebook has been using this technology since 2015.

Source: hbr.org
Case Study: DeepFace
How?

• Layers of virtual neurons are trained to identify edges, features, a


face, and so on.

• In the example given on the left, the first layer recognizes edges.

• The second layer recognizes facial features like a nose or an ear.

• The third layer recognizes faces. The full face is eventually


recognized in the fourth layer.

• The layers of a neural network work in coordination with each


other to progressively add more insight as the data passes from
input layers toward the output layers.

Source: hbr.org
Introduction to Deep Learning
Topic 4: Deep Learning Platforms
CPU and GPU Specifications

The table below shows that GPUs (right column) are cheaper than
modern CPUs (middle column) for deep learning tasks. In
addition, they support a lot more cores and calculations.
Python Specifications

• Python is limited to execution on one core due to the Global Interpreter Lock (GIL).
• Multiprocessing library in Python allows distributed computation over several cores, but
most advanced desktop hardware comes only with a maximum of 8 to 16 cores.
GPU: Special Packages

• In image processing, there can be an explosion in number of parameters.


• Single processor units cannot handle these easily, but GPUs can. Each GPU is akin to a small
computer cluster.
• However, one has to use special packages like CUDA or OpenCL to write code for GPUs.
• Deep Learning library like TensorFlow makes it easy to write code for either OpenCL or
CUDA-enabled GPUs.
• Deep Learning platforms include: Tensorflow (Python Based), Keras (Python), Torch( C/C++),
and Deeplearning4j(JAVA).
Key Takeaways

Deep Learning is a subset of Machine Learning and Artificial Intelligence and makes
complex features and input-output relationships learnable.
New breakthroughs in neural networks, availability of Big Data, and low-cost high-
performance GPU chips are driving the Deep Learning revolution.
Deep Learning is useful for complex intelligence tasks like face recognition, speech
recognition, machine translation etc.
Artificial Neural Network is a computing system made up of a number of simple, highly
interconnected processing elements that process information by their dynamic state
response to external inputs. It is modeled on the inter-connected neurons in the human
brain.
Each circular node represents an artificial neuron, and an arrow represents a connection
from the output of one neuron to the input of another.
Some of the commonly used neural networks are RNN, CNN, ANN, DNN, and DBN.
GPUs are cheaper than modern CPUs now, in addition to supporting a lot more cores
and calculations.
This concludes “Introduction to Deep Learning”
The next lesson is “Perceptron.”

©Simplilearn. All rights reserved

You might also like