Scribd
Upload
34 views45 pages

Neural Networks

The document provides an overview of neural networks including when they should be considered, examples of their use, descriptions of their components and how they function. It discusses the different types of neural networks including feedforward and recurrent networks. It also covers topics like training neural networks, activation functions, convolutional neural networks and using neural networks for classification.

Uploaded by

Diễm Quỳnh Trần
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
34 views45 pages

Neural Networks

The document provides an overview of neural networks including when they should be considered, examples of their use, descriptions of their components and how they function. It discusses the different types of neural networks including feedforward and recurrent networks. It also covers topics like training neural networks, activation functions, convolutional neural networks and using neural networks for classification.

Uploaded by

Diễm Quỳnh Trần
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 45

Neural Networks

Tran Thi Oanh


When to consider neural networks
ØInput is high-dimensional discrete or raw-valued
ØOutput is discrete or real-valued
ØOutput is a vector of values
ØPossibly noisy data
ØForm of target function is unknown
ØHuman readability of the result is not important

Examples:
ØSpeech phoneme recognition
ØImage classification
ØFinancial prediction
Neural Networks
Ø Our brains, in average, contain 86 billion
neurons (nerve cells).
Ø A single neuron may receive one to multiple
inputs from other neurons and sending the
bio current to the corresponding output(s)
when the certain criteria are satisfied.
Ø Nowadays, the actual mechanism of how
human brain works is still a mystery.
Ø However, artificial neural network somehow
achieves learning by imitating the human brain
neuron behavior.
Descriptions
ØUse network of connected nodes (in layers)
ØNetwork connects input, output (categorical)
o inputs like independent variable values in regression
o outputs: {buy, don’t} {paid, didn’t}
{red, green, blue, purple}
{character recognition - alphabetic characters}
Artificial Neural Networks (ANN)
Input Layer Hidden Layers Output Layer

Good

Bad
Types of connectivity
ØFeedforward networks output units
o These compute a series of transformations
o Typically, the first layer is the input and the hidden units
last layer is the output.
ØRecurrent networks
input units
o These have directed cycles in their
connection graph. They can have
complicated dynamics.
o More biologically realistic.
Modelling a Neuron

:Activation value of unit j


ini = å j Wj, iaj Øaj
Øwj,I :Weight on the link from unit j to unit i
ØinI :Weighted sum of inputs to unit i
ØaI :Activation value of unit i
Øg :Activation function
Algorithm for learning ANN
ØInitialize the weights (w0, w1, …, wk)

ØAdjust the weights in such a way that the output of ANN is consistent
with class labels of training examples
E = å [Yi - f ( wi , X i )]
2
o Error function:
i

o Find the weights wi’s that minimize the above error function
• e.g., gradient descent, backpropagation algorithm
Optimizing concave/convex function
ØMaximum of a concave function = minimum of a convex
function
Gradient ascent (concave) / Gradient descent (convex)

Gradient ascent rule


Training the Model
ØEstimate the weights wij that lead to the best predictive results.
o Using the errors iteratively to update the estimated weights
o Errors is distributed across all the hidden nodes

è Back Propagation of Error: errors are computed from the last layer back to
the hidden layers
Perceptron
Problem Definition
Parameters (Coefficients)
Get back to the example
Get back to the example (2)
Non-linearly separable cases
Activation Functions

ØStept(x) = 1 if x >= t, else 0


ØSign(x) = +1 if x >= 0, else –1
ØSigmoid(x) = 1/(1+e-x)
ØIdentity Function
Sigmoid function
Hyperbolic tangent function
Feed forward neural network
ØMulti-layer feed-forward networks
o One or more hidden layers. Input projects only from previous layers onto a
layer.

2-layer or
1-hidden layer
fully connected
network
Input Hidden Output
layer layer layer
ØMulti-layer feed-forward networks

Input Hidden Output


layer layers layer
How A Multi-Layer Neural Network Works?
ØThe inputs to the network correspond to the attributes measured for each training
tuple
ØInputs are fed simultaneously into the units making up the input layer
ØThey are then weighted and fed simultaneously to a hidden layer
ØThe number of hidden layers is arbitrary, although usually only one
ØThe weighted outputs of the last hidden layer are input to units making up the
output layer, which emits the network's prediction
ØThe network is feed-forward in that none of the weights cycles back to an input
unit or to an output unit of a previous layer
ØFrom a statistical point of view, networks perform nonlinear regression: Given
enough hidden units and enough training samples, they can closely approximate
any function

Data Mining: Concepts and Techniques 27


Deep learning – Convolutional
Neural Network
Cited from Pei-Wei Tsai
Background knowledge
Background knowledge
CNN-convolution
CNN-convolution
CNN-convolution
CNN-convolution
CNN – max pooling
CNN – max pooling
CNN – flatten and fully connected network
Video recap
Neural Network as a Classifier
ØWeakness
o Long training time
o Require a number of parameters typically best determined empirically, e.g., the
network topology or “structure.”
o Poor interpretability: Difficult to interpret the symbolic meaning behind the learned
weights and of “hidden units” in the network
ØStrength
o High tolerance to noisy data
o Ability to classify untrained patterns
o Well-suited for continuous-valued inputs and outputs
o Successful on a wide array of real-world data
o Algorithms are inherently parallel
o Techniques have recently been developed for the extraction of rules from trained
neural networks

Data Mining: Concepts and Techniques 42


Recurrent Neural Network (RNN)
43
RNN – Recurrent Neural Network
Thank you!
Q&A

You might also like