Scribd
Upload
7 views16 pages

Multi Layer NN

The document discusses supervised learning in multilayer networks, specifically focusing on backpropagation networks and their architecture, which includes input, hidden, and output neurons. It explains how these networks can perform regression and classification tasks and introduces Gaussian processes as a method for modeling functions without parameterization. Additionally, it touches on parametric approaches to regression using fixed basis functions, with an example related to COVID-19 modeling.

Uploaded by

ayushjain6548
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
7 views16 pages

Multi Layer NN

The document discusses supervised learning in multilayer networks, specifically focusing on backpropagation networks and their architecture, which includes input, hidden, and output neurons. It explains how these networks can perform regression and classification tasks and introduces Gaussian processes as a method for modeling functions without parameterization. Additionally, it touches on parametric approaches to regression using fixed basis functions, with an example related to COVID-19 modeling.

Uploaded by

ayushjain6548
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 16

Supervised Learning in Multilayer

Networks
Multi layer NNs

• Supervised multilayer networks are also known as


backpropagation networks
• The multilayer perceptron is a feedforward network
• It has input neurons, hidden neurons and output neurons
• The hidden neurons may be arranged in a sequence of layers
• The most common multilayer perceptrons have a single hidden layer,
and are known as ‘two-layer’ networks
• The number ‘two’ counting the number of layers of neurons not
including the inputs
Two Layer Network
• Such a feedforward network defines a nonlinear parameterized
mapping from an input x to an output y = y(x; w; A)
• The output is a continuous function of the input and of the
parameters w
• The architecture of the net, i.e., the functional form of the mapping,
is denoted by A
• Feedforward networks can be ‘trained’ to perform regression and
classification tasks
Regression networks
• In the case of a regression problem, the mapping for a network with
one hidden layer may have the form:
Regression networks
• Here l runs over the inputs j runs over the hidden units,
and i runs over the outputs

• The ‘weights’ w and ‘biases’ θ together make up the parameter vector


w.
• The nonlinear sigmoid function at the hidden layer gives the
neural network greater computational flexibility than a standard
linear regression model
What sorts of functions can these
networks implement?
• Let us explore the weight space of a multilayer network.
• In figures 44.2 and 44.3, a network with one input and one output and a large
number H of hidden units, set the biases, and weights
to random values, and plot the resulting function y(x).
• The hidden units’ biases are set to random values from a Gaussian with
zero mean and standard deviation σbias; the input-to-hidden
weights to random values with standard deviation σin; and the bias and
output weights and to random values with standard deviation σout
• The sort of functions that we obtain depend on the values of σbias,
σin and σout. As the weights and biases are made bigger, we obtain
more complex functions with more features and a greater sensitivity
to the input variable.
• The vertical scale of a typical function produced by the network with
random weights is of order ; the horizontal range in which the
function varies significantly is of order ; and the shortest
horizontal length scale is of order
Gaussian Processes
• The idea of Gaussian process modelling is to place a prior P(y(x)) directly on
the space of functions, without parameterizing y(x).
• The simplest type of prior over functions is called a Gaussian process.
• It can be thought of as the generalization of a Gaussian distribution over a
finite vector space to a function space of infinite dimension
• Just as a Gaussian distribution is fully specified by its mean and covariance
matrix, a Gaussian process is specified by a mean and a covariance function.
• Here, the mean is a function of x (which we will often take to be the zero
function), and the covariance is a function C(x; x0) that expresses the
expected covariance between the values of the function y at the points x
and x0.
Standard methods for nonlinear
regression
• We are given N data points
• The inputs x are vectors of some fixed input dimension I.
• The targets t are either real numbers, in which case the task will be a
regression or interpolation task, or they are categorical variables
• For example in which case the task is a classification task.
We will concentrate on the case of regression for the time being
• Assuming that a function y(x) underlies the observed data, the task is
to infer the function from the given data, and predict the function’s
value-- or the value of the observation
Parametric approaches to the
problem
• In a parametric approach to regression we express the unknown
function y(x) in terms of a nonlinear function
• Example: Fixed basis functions--Using a set of basis functions
• we have

• If the basis functions are nonlinear functions of x such as radial basis


functions with fixed mean
• Other possible sets of fixed basis functions include polynomials such
as where p and q are integer powers that depend on h.

• Example: "Mathematical Modeling of COVID-19 and Prediction of


Upcoming Wave," in IEEE Journal of Selected Topics in Signal
Processing, vol. 16, no. 2, pp. 300-306, Feb. 2022, doi:
10.1109/JSTSP.2022.3152674
MATHEMATICAL MODELING OF
COVID-19

You might also like