Scribd
Upload
124 views19 pages

2.neural Network

The document discusses artificial neural networks and their components. It provides: 1) A brief history of artificial neural networks from their origins in the 1800s to recent developments like GANs in 2014. 2) An overview of key concepts in artificial neural networks including neurons, perceptrons, activation functions, and backpropagation. 3) Explanations of why activation functions are important for introducing non-linearity and allowing neural networks to learn complex patterns from data.

Uploaded by

rajthakre81
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
124 views19 pages

2.neural Network

The document discusses artificial neural networks and their components. It provides: 1) A brief history of artificial neural networks from their origins in the 1800s to recent developments like GANs in 2014. 2) An overview of key concepts in artificial neural networks including neurons, perceptrons, activation functions, and backpropagation. 3) Explanations of why activation functions are important for introducing non-linearity and allowing neural networks to learn complex patterns from data.

Uploaded by

rajthakre81
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 19

Meanwhile in the world of AI....

ROAD MAP
 Why Artificial Neural networks
 History of ANN
 Why now?
 Artificial Neural Networks
 Preceptron
 Loss Function
 Activation functions
 Binary Cross Entropy
 Gradient decent
 Back Propagation
WHY NEURAL NETWORKS
HISTORY

1800's 1960's and 70's 1986 1986: MLP, RNN 2012: Dropout 2014: GANs
late-1800's - Neural Simple neural A multilayer Dropout to A generative
Networks appear as Backpropagation reducing overfitting in adversarial network is
networks appear perceptron is a
an analogy to algorithm appears neural networks by a class of machine
• Fall out of favor class of
biological systems • Neural Networks preventing complex learning frameworks
because the feedforward co-adaptations designed by Ian
perceptron is not have a resurgence artificial neural
in popularity on training data. Goodfellow and his
effective by itself, network. colleagues in 2014.
and there were • More
computationally • RNN Recurrent
no good neural network
algorithms for expensive
multilayer nets
WHY NOW?
DAVID HUBEL AND TORSTEIN
WIESEL EXPERIMENT OF VISUAL CORTEX
NEURON
 Computational models inspired by the human
brain:
 Algorithms that try to mimic the brain.
 Massively parallel, distributed system, made up
of simple processing units (neurons)
 Synaptic connection strengths among neurons
are used to store the acquired knowledge.
 Knowledge is acquired by the network from its
environment through a learning process
 Dendrite: Receives signals from other neurons
 Soma: Processes the information
 Axon: Transmits the output of this neuron
 Synapse: Point of connection to other neurons
ARTIFICIAL NEURON :
PERCEPTRON
A Perceptron is an algorithm
used for supervised learning of
binary classifiers. Binary classifiers
decide whether an input, usually
represented by a series of vectors,
belongs to a specific class.
In short, a perceptron is a single-
layer neural network. They consist
of four main parts including input
values, weights and bias, net sum,
and an activation function.
ARTIFICIAL NEURON :
PERCEPTRON

The artificial neuron has the following characteristics:


A neuron is a mathematical function modeled on the
working of biological neurons
It is an elementary unit in an artificial neural network
One or more inputs are separately weighted
Inputs are summed and passed through a nonlinear function
to produce output
Every neuron holds an internal state called activation signal
Each connection link carries information about the input
signal
Every neuron is connected to another neuron via
connection link
ACTIVATION FUNCTION

"A neural network without an activation function


is essentially just a linear regression model."
 An activation function determines the output
behavior of each node, or “neuron” in an
artificial neural network.
 Activation functions are crucial basic components
of artificial neural networks (ANN), since they
introduce non-linear characteristics to the
network.
 This allows the ANN to learn complicated, non-
linear mappings between inputs and response
variables.
PERCEPTRON :AN EXAMPLE
PERCEPTRON :AN EXAMPLE
Without activation function, weight and bias
would only have a linear transformation, or neural
network is just a linear regression model, a linear
equation is polynomial of one degree only which is

WHY
simple to solve but limited in terms of ability to
solve complex problems or higher degree
polynomials.

ACTIVATION But opposite to that, the addition of activation


function to neural network executes the non-

FUNCTION linear transformation to input and make it capable


to solve complex problems such as language
translations and image classifications.

In addition to that, Activation functions are


differentiable due to which they can easily
implement back propagations, optimized strategy
while performing backpropagations to measure
gradient loss functions in the neural networks.
WHY DO WE NEED ACTIVATION FUNCTION
LINEAR VS NON LINEAR ACTIVATION FUNCTION
COMMON ACTIVATION
FUNCTIONS

Linear
ReLU
LeakyReLU
Sigmoid
Tanh
Soft max
RELU
•Equation : f(x) = max(0,x)
•Range : (0 to infinity)
•Pros:
•The function and its derivative both
are monotonic.
•Due to its functionality it does not activate
all the neuron at the same time
•It is efficient and easy for computation.
•Cons:
•The outputs are not zero centered similar
to the sigmoid activation function
•When the gradient hits zero for the
negative values, it does not converge
towards the minima which will result in a
dead neuron while back propagation.
SIGMOID OR LOGISTIC ACTIVATION
FUNCTION
The Sigmoid Function curve looks like a S-shape.
 Sigmoid Function Equation : f(x) = 1 / 1 + exp(-x)
Range : (0 to 1)
Pros:
The function is differentiable. That means, we can find
the slope of the sigmoid curve at any two points
The function is monotonic but function’s derivative is
not
Cons:
It gives rise to a problem of “vanishing gradients”, since
the Y values tend to respond very less to changes in X
Secondly , its output isn’t zero centered. It makes the
gradient updates go too far in different directions. 0 <
output < 1, and it makes optimization harder.
 Sigmoid saturate and kill gradients.
 Sigmoid have slow convergence.

You might also like