Scribd
Upload
55 views13 pages

Deep Vs Shallow Neural Networks

comparison

Uploaded by

sethuramanr1976
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
55 views13 pages

Deep Vs Shallow Neural Networks

comparison

Uploaded by

sethuramanr1976
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 13

deep vs

shallow
neural networks
Presented by:
P.Narmatha shree
Final year CSE
Shallow neural
networkS
Shallow neural networks are comparatively simpler and
easier to train. They are suitable for straightforward tasks
such as data fitting, classification, and pattern
recognition. Examples include logistic regression,
support vector machine, random forest, and cluster
analysis. ​Shallow networks can be trained quickly with
fewer parameters, making them faster to train and
requiring less computational resources.
deep neural
network

Deep neural networks, by comparison, have complex architectures


containing multiple intervening layers, allowing them to learn more
elaborate relationships between inputs and outputs.
Examples of deep networks include convolutional neural networks (CNNs),
generative adversarial networks (GANs), transformer networks, and others.
architecture
Shallow Neural Networks (SNNs): Deep Neural Networks (DNNs):
Shallow neural networks are
characterized by their relatively Deep neural networks, as the name
simple architecture. An SNN suggests, have a more complex
typically consists of three types of architecture with multiple hidden
layers: layers between the input and output
Input Layer: Receives the raw layers. These additional layers allow
data. DNNs to learn more abstract and
Hidden Layer: Contains a one intricate features from the data. The
or two hidden layers where the depth of a DNN refers to the number of
computation and feature hidden layers it contains, which can
extraction occur. range from just a few to hundreds or
Output Layer: Produces the even thousands.
final output or prediction.
complexity
Shallow Neural Networks (SNNs): Deep Neural Networks (DNNs):

The complexity of SNNs is DNNs are inherently more complex due


relatively low due to their simpler to their multiple hidden layers. Each
architecture. With only a single additional layer introduces more
hidden layer, the network can parameters and increases the
model basic patterns and network’s capacity to capture intricate
relationships in the data. This patterns and relationships. While this
added complexity can lead to improved
simplicity makes SNNs easier to
performance on complex tasks, it also
train and less prone to issues like
makes training more challenging.
vanishing gradients.
Learning Capacity
Shallow Neural Networks (SNNs): Deep Neural Networks (DNNs):

SNNs have a limited learning DNNs have a much higher learning


capacity. They are well-suited for capacity. The multiple hidden layers
tasks where the relationships in enable them to learn hierarchical
the data are relatively simple or representations of data, making them
linear. For instance, they perform effective for tasks that require
adequately on problems like understanding complex and abstract
binary classification with well- features. This capability is especially
separated classes. useful for applications such as image
recognition, speech processing, and
natural language understanding.
Risk of Overfitting
Shallow Neural Networks (SNNs): Deep Neural Networks (DNNs):

Due to their fewer parameters and DNNs, with their large number of
simpler architecture, SNNs have a parameters and multiple layers, are
lower risk of overfitting. more prone to overfitting. The high
Overfitting occurs when a model capacity of DNNs allows them to fit the
learns the noise in the training training data very closely, which can
data rather than the underlying lead to overfitting if not managed
properly. Techniques such as
pattern, leading to poor
regularization, dropout, and early
generalization to new data. SNNs
stopping are often used to mitigate
are less likely to overfit as they
overfitting in DNNs.
have limited capacity to memorize
the training data.
Data Requirements
Shallow Neural Networks (SNNs): Deep Neural Networks (DNNs):

SNNs generally require less data DNNs require large amounts of data to
to train effectively. Their simpler train effectively. The multiple layers
architecture means they need and vast number of parameters mean
fewer examples to learn the that DNNs need extensive datasets to
patterns and relationships in the learn and generalize well. In many
data. However, this also limits cases, the performance of a DNN
their ability to handle complex improves as the size of the training
tasks that require a deeper data increases.
understanding of the data.
Parameter Count
Shallow Neural Networks (SNNs): Deep Neural Networks (DNNs):

The number of parameters in DNNs have a significantly higher


SNNs is relatively small due to the number of parameters due to the
limited number of hidden layers. multiple hidden layers and connections
This smaller parameter count between neurons. This increased
translates to lower computational parameter count requires more
and memory requirements, making computational resources for training
and inference. As a result, DNNs often
SNNs more efficient for simpler
necessitate the use of GPUs or other
tasks.
specialized hardware for efficient
training.
Computational Resources
Shallow Neural Networks (SNNs): Deep Neural Networks (DNNs):

SNNs require fewer computational Training DNNs is computationally


resources compared to DNNs. intensive due to the large number of
Their simpler structure allows parameters and the complexity of the
them to be trained and deployed model. GPUs, TPUs, or other
on standard CPUs, making them specialized hardware are often used to
more accessible for tasks with accelerate the training process. The
limited computational resources. high computational demands also
imply that deploying DNNs for
inference can be resource-intensive.
Interpretability
Shallow Neural Networks (SNNs): Deep Neural Networks (DNNs):

SNNs are generally easier to DNNs are often described as “black


interpret due to their simpler boxes” because their complex
structure. With only a single architecture makes them difficult to
hidden layer, it is relatively interpret. The multiple layers and
straightforward to understand nonlinear activations contribute to the
how the network processes input challenge of understanding how the
network arrives at its decisions.
data and generates predictions.
Techniques such as visualization of
This interpretability makes SNNs
activation maps and layer-wise
suitable for applications where
relevance propagation are used to gain
understanding the decision-
insights into DNNs, but interpretability
making process is important.
remains a significant challenge.
SHALLOW NEURAL NETWORK DEEP NEURAL NETWORK

Shallow Neural network with few Deep Neural network with many layers
layers (usually 1 hidden layer). (multiple hidden layers).

Complexity is low Complexity is high.

Limited learning capacity. Higher learning capacity.

Requires less data. Requires more data for effective training.

Fewer parameters counts in the shallow Many more parameters counts in the deep neural
neural networks. networks.

Requires less computational resources. Requires more computational resources (e.g., GPUs).

Easier to interpret. More difficult to interpret.

Example: Single-layer Perceptron, Logistic Example: Convolutional Neural Networks


Regression (CNNs), Recurrent Neural Network(RNNs).
THANK
YOU

You might also like