Artificial Neural Networks
Multilayer Perceptron (MLP) is a type of feedforward artificial neural network that maps sets of input data to appropriate outputs. It has multiple layers of nodes that are fully connected between layers but not within layers. Additional hidden layers allow MLPs to perform complex nonlinear mapping that single layer perceptrons cannot, such as solving the XOR problem. MLPs are widely used universal approximators with applications in classification, regression, and other tasks due to their adaptability, robustness, and ability to model complex functions.
Copyright:
Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPTX, PDF, TXT or read online from Scribd
Artificial Neural Networks
Multilayer Perceptron (MLP) is a type of feedforward artificial neural network that maps sets of input data to appropriate outputs. It has multiple layers of nodes that are fully connected between layers but not within layers. Additional hidden layers allow MLPs to perform complex nonlinear mapping that single layer perceptrons cannot, such as solving the XOR problem. MLPs are widely used universal approximators with applications in classification, regression, and other tasks due to their adaptability, robustness, and ability to model complex functions.
Artificial Neural Networks
Multilayer Perceptron (MLP) is a type of feedforward artificial neural network that maps sets of input data to appropriate outputs. It has multiple layers of nodes that are fully connected between layers but not within layers. Additional hidden layers allow MLPs to perform complex nonlinear mapping that single layer perceptrons cannot, such as solving the XOR problem. MLPs are widely used universal approximators with applications in classification, regression, and other tasks due to their adaptability, robustness, and ability to model complex functions.
Copyright:
Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPTX, PDF, TXT or read online from Scribd
Artificial Neural Networks
Multilayer Perceptron (MLP) is a type of feedforward artificial neural network that maps sets of input data to appropriate outputs. It has multiple layers of nodes that are fully connected between layers but not within layers. Additional hidden layers allow MLPs to perform complex nonlinear mapping that single layer perceptrons cannot, such as solving the XOR problem. MLPs are widely used universal approximators with applications in classification, regression, and other tasks due to their adaptability, robustness, and ability to model complex functions.
Copyright:
Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPTX, PDF, TXT or read online from Scribd
Download as pptx, pdf, or txt
You are on page 1/ 24
Artificial Neural Networks
Multilayer Perceptron (MLP)
Nicolas Galoppo von Borries COMP290-058 Motion Planning
A new sort of computer
What are (everyday) computer systems good at... and not so good at? Good at Not so good at
Rule-based systems: Dealing with noisy data doing what the programmer Dealing with unknown wants them to do environment data
Massive parallelism Fault tolerance
Adapting to circumstances
Where can neural network systems help
when we can't formulate an algorithmic solution. when we can get lots of examples of the behavior we require. learning from experience when we need to pick out the structure from existing data.
Neural networks to the rescue
Neural network: information processing
paradigm inspired by biological nervous systems, such as our brain.
Structure: large number of highly interconnected processing elements (neurons) working together to process the data
Neural networks to the rescue
Neural networks are configured for a specific application, such as pattern recognition or data classification, through a learning process In a biological system, learning involves adjustments to the synaptic connections between neurons
same for artificial neural networks (ANNs)
Inspiration from Neurobiology
A neuron: many-inputs / one-output unit output can be excited or not excited incoming signals from other neurons determine if the neuron shall excite ("fire") Output subject to attenuation in the synapses, which are junction parts of the neuron
Dendrites: nerve fibres carrying electrical signals to the cell
Cell body: computes a non-linear function of its input
Axon: single long fibre that carries the electrical signal from the cell body to other neurons
Synapse: the point of contact between the axon of one cell and the dendrite of another, regulating a chemical connection whose strength affects the input to the cell.
Mathematical representation The neuron calculates a weighted sum of inputs and compares it to a threshold. If the sum is higher than the threshold, the output is set to 1, otherwise to -1. Non-linearity
A simple perceptron
Its a single-unit network Change the weight by an amount proportional to the difference between the desired output and the actual output. Input Wi = * (D-Y).Ii Learning rate Actual output Desired output
Perceptron Learning Rule
Example: A simple single unit adaptive network
The network has 2 inputs, and one output. All are binary. The output is
1 if W0I0 + W1I1 + Wb > 0 0 if W0I0 + W1I1 + Wb 0
We want it to learn simple OR: output a 1 if either I0 or I1 is 1.
Multilayer Perceptron (MLP)
Feedforward Artificial Neural Network that maps sets of
input data onto a set of appropriate output.
Multiple layers of nodes.
Each layer fully connected to the next layer.
Supervised learning technique
Three Layer MLP
x1
xn
what do the extra layers gain you? Start with looking at what a single layer cant do.
XOR Problem
XOR (exclusive OR) problem 0+0=0 1+1=2=0 mod 2 1+0=1 0+1=1
Perceptron does not work here
Minsky & Papert (1969) offered solution to XOR problem by combining perceptron unit responses using a second layer of units +1
1 3 2
+1
(1,-1) (1,1)
(-1,-1)
(-1,1)
This is a linearly separable problem!
Since for 4 points { (-1,1), (-1,-1), (1,1),(1,-1) } it is always linearly separable if we want to have three points in a class
Properties of architecture No connections within a layer
yi f ( wij x j bi ) j 1
Properties of architecture No connections within a layer No direct connections between input and output layers
yi f ( wij x j bi ) j 1
Properties of architecture No
connections within a layer No direct connections between input and output layers Fully connected between layers
yi f ( wij x j bi ) j 1
Properties of architecture No
connections within a layer No direct connections between input and output layers Fully connected between layers Often more than 3 layers Number of output units need not equal number of input units Number of hidden units per layer can be more or less than input or output units
yi f ( wij x j bi ) j 1
Often include bias as an extra weight
What do each of the layers do?
1st layer draws linear boundaries
2nd layer combines the boundaries
3rd layer can generate arbitrarily complex boundaries
Why MLP?
Popularity - the most used type of NN Universal Approximators - general-purpose models, with a huge number of applications Nonlinearity - capable of modelling complex functions; Robustness - good at ignoring irrelevant inputs and noise Adaptability - can adapt its weights and/or topology in response to environment changes
Who is interested in MLP?
Computer Scientists : study properties of nonsymbolic information processing
Statisticians: perform flexible data analysis Engineers: exploit MLP capabilities in several areas Cognitive Scientists: describe models of thinking Biologists: interpret DNA sequences
Applications
Classification (discrete outputs) Regression (numeric outputs) Reinforcement Learning (Output is not perfectly known)
