Deep Learning/Deep Artificial Neural Networks

With the reinvigoration of neural networks in the 2000s, deep learning has become an extremely active area of research
that is paving the way for modern machine learning. Deep Learning is most famous for its neural networks such as
Recurrent Neural Networks, Convolutional Neural Networks, and Deep Belief Networks. It’s all about neural networks.
Deep learning algorithms extract layered high-level representations of data in a way that maximizes performance on a
given task. For example, asked to recognize faces, a deep neural network may learn to represent image pixels first with
edges, followed by larger shapes, then parts of the face like eyes and ears, and, finally, individual face identities. In deep
learning, interconnected layers of software-based calculators known as “neurons” form a neural network. The idea is to
replicate an abstracted understanding of how we believe the human brain might process similar information and learn
from its surroundings and sensory input.

The word “deep” comes from the fact that DL algorithms are trained /run on deep neural networks. These are just
neural networks with (usually) three or more “hidden” layers.

Deep Learning is a hierarchical learning methodology based on artificial neural networks which are algorithms inspired
by the structure and function of the brain.

They are modeled after the structure and functioning of the human brain. In order to understand deep learning, we
have to understand how the nervous system in the human body works. As we all know that our nervous system is built
up of neurons. These neurons are able to grasp information that is transmitted to our body. These neurons have the
ability to learn information over time. This principle of ‘learning’ is also utilized by artificial neural networks.

Like ML, DL is not particularly new. Indeed, the principal component of DL systems – artificial neural networks – began
to take shape in the 1940s, seeing significant breakthroughs in the 1960s and each decade thereafter.

DL’s use has accelerated in recent decades. This is due to:

1. the availability of cheapening but increasingly powerful computer hardware; and

Deep learning is a subset of a more general field of artificial intelligence called machine learning. DL is a subtype of ML
and each is a subtype of A.I.
DL can process a wider range of data resources, requires less data preprocessing by humans (e.g., feature labelling), and
can sometimes produce more accurate results than traditional ML approaches (although it requires a larger amount of
data to do so).

Machine Learning vs Machine Learning

Parameter Machine learning Deep Learning

Data Volume Better than DL on small datasets. Better than ML on large datasets.
Computational Cheaper in time to execute and hardware vs. DL. More expensive in time to execute and
Cost/Execution hardware vs. ML.
Time ML takes much less time to train, ranging from a
few seconds to a few hours. DL takes more time as compared to
machine learning to train. The main
reason behind its long time is that so
many parameters in deep learning
algorithm.

Adaptability Both domain specific and application specific ML DL techniques can be adapted to different
techniques and feature engineering are required domains and applications far more easily
to build high-performance ML models. than ML algorithms.
Therefore, the resulting models are less
adaptable, even within similar domains. For example, once one understands the
underlying deep learning theory for the
domain of speech recognition, then
learning how to apply deep networks to
natural language processing isn’t too
challenging since the baseline knowledge
is quite similar.

Feature ML often requires complex feature engineering, DL techniques can eliminate or reduce the
Engineering which is costly in terms of time and hiring or need for complex feature extraction,
contracting domain expertise. thereby reducing the time and cost of
that step, albeit potentially in exchange
for more expense in time and money
regarding hardware and execution time of
DL approaches.

Interpretability Due to the feature engineering and simpler Less capable of interpretation.
models, ML systems are generally easier to
interpret than DL. Often seen as “blackbox” systems that
researchers struggle to unwind and
In other words, it’s easier to understand how explain how and why the system reached
and why the ML algorithm arrived at an a particular outcome.
outcome.
That said, there continue to be significant
This can be incredibly helpful, not to mention developments in this area that are
necessary (e.g., in regulated environments), to
 unwind and correct a system that produces opening up the blackbox so this
incorrect results in unexpected circumstances. distinction may reduce over time.

Hyperparameter Limited tuning capabilities Can be tuned in various different ways.

Hardware depends on low-end machines deep learning depends on high-end

Applications Machine learning is already in use in your email Deep learning technology enables more
inbox, bank, and doctor’s office. complex and autonomous programs, like
self-driving cars or robots that perform
advanced surgery.