Scribd
Upload
3 views

Lecture-1

The document provides an overview of neural networks, explaining their structure and functionality based on biological neurons. It details the components of a neuron model, types of activation functions, and various network architectures, including single-layer, multilayer, and recurrent networks. The learning process and adaptation of weights in neural networks are also discussed, emphasizing their ability to model complex tasks similar to human brain functions.

Uploaded by

haidarrahmed9
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
3 views

Lecture-1

The document provides an overview of neural networks, explaining their structure and functionality based on biological neurons. It details the components of a neuron model, types of activation functions, and various network architectures, including single-layer, multilayer, and recurrent networks. The learning process and adaptation of weights in neural networks are also discussed, emphasizing their ability to model complex tasks similar to human brain functions.

Uploaded by

haidarrahmed9
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 26

Neural Networks: An

Overview
By: Dr. Inas Abuqaddom
Email: [email protected]
Creator: Dr. Ola Surakhi
What is a neural network?
• A neural network is an interconnected assembly of simple processing
elements, units or nodes, whose functionality is loosely based on the
natural neuron.
•
• The processing ability of the network is stored in the interunit
connection strengths, or weights, obtained by a process of
adaptation to, or learning from, a set of training patterns.
Human neuron
• The human brain consists of an estimated (100 billion) nerve cells or neuron

• Neurons communicate via electrical signals that are short-lived impulses or


"spikes“

• The interneuron connections are mediated by electrochemical junctions called


synapses

• Each neuron typically receives many thousands of connections from other


neurons and is therefore constantly receiving a multitude of incoming signals,
which eventually reach the cell body.
Human neuron
• Here, they are integrated or summed together in some way and,
roughly speaking, if the resulting signal exceeds some threshold, then
the neuron will "fire" or generate a voltage impulse in response. This
is then transmitted to other neurons via a branching fiber known as
the axon.

• It is this architecture and style of processing that we hope to


incorporate in neural networks
Neural Networks
• The artificial equivalents of biological neurons are the nodes or units
in our neural networks

• Synapses are modelled by a single number or weight so that each


input is multiplied by a weight before being sent to the equivalent of
the cell body

• Here, the weighted signals are summed together by simple arithmetic


addition to supply a node activation
Neural Networks
• The activation is then compared with a threshold; if the activation
exceeds the threshold, the unit produces a high-valued output
(conventionally "1"), otherwise it outputs zero
Neural Networks
• A neural network is a machine that is designed to model the way in
which the brain performs a particular task or function of interest

• The network is usually implemented by using electronic components


or is simulated in software on a digital computer

• The procedure used to perform the learning process is called a


learning algorithm, the function of which is to modify the synaptic
weights of the network in an orderly fashion to attain a desired design
objective.
Models of a neuron
• A neuron is an information-processing unit that is fundamental to the
operation of a neural network
Models of a neuron
• There are three basic elements of the neural model:
1. A set of synapses, or connecting links, each of which is
characterized by a weight or strength of its own
• Specifically, a signal xj at the input of synapse j connected to neuron k is
multiplied by the synaptic weight wkj.
• The first subscript in wkj refers to the neuron in question, and the second
subscript refers to the input end of the synapse to which the weight refers
2. An adder for summing the input signals, weighted by the respective
synaptic strengths of the neuron; the operations described here
constitute a linear combiner
Models of a neuron
3. An activation function for limiting the amplitude of the output of a
neuron. The activation function is also referred to as a squashing
function, in that it squashes (limits) the permissible amplitude range
of the output signal to some finite value

• The bias bk has the effect of increasing or lowering the net input of
the activation function, depending on whether it is positive or
negative, respectively.
Models of a neuron
• In mathematical terms, we may describe the neuron k depicted in
previous figure by writing the pair of equations:

• where x1, x2, ..., xm are the input signals; wk1, wk2, ..., wkm are the
respective synaptic weights of neuron k; uk is the linear combiner output
due to the input signals; bk is the bias; σ is the activation function; and yk
is the output signal of the neuron
Models of a neuron
• The use of bias bK has the effect of applying an affine transformation
to the output uk of the linear combiner in the model of the previous
figure, as shown by
Models of a neuron
Types of Activation Function
• The activation function defines the output of a neuron
1. Threshold Function. For this type of activation function, we have

where vk is the induced local field of the neuron; that is,


Types of Activation Function
2. Sigmoid Function: is by far the most common form of activation
function used in the construction of neural network

• where a is the slope parameter of the sigmoid function


• By varying the parameter a, we obtain sigmoid functions of different slopes
Types of Activation Function
• The slope parameter approaches infinity, the sigmoid function
becomes simply a threshold function.
• Whereas a threshold function assumes the value of 0 or 1, a sigmoid
function assumes a continuous range of values from 0 to 1.
• Note also that the sigmoid function is differentiable, whereas the
threshold function is not.
Types of Activation Function
• The activation functions defined previously range from 0 to 1.
• It is sometimes desirable to have the activation function range from
-1 to 1, in which case, the activation function is an odd function of the
induced local field
Types of Activation Function
• A corresponding form of a sigmoid function, we may use the
hyperbolic tangent function, defined by

• Allowing an activation function of the sigmoid type to assume


negative values
Neural Networks as a Directed
Graph
• A neural network is a directed graph consisting of nodes with
interconnecting synaptic and activation links and is characterized by
four properties:
1. Each neuron is represented by a set of linear synaptic links, an
externally applied bias, and a possibly nonlinear activation link. The
bias is represented by a synaptic link connected to an input fixed at
1
2. The synaptic links of a neuron weight their respective input signals.
Neural Networks as a Directed
Graph

3. The weighted sum of the input signals defines the induced local
field of the neuron in question.
4. The activation link squashes the induced local field of the neuron to
produce an output.
Feedback
• The system is assumed to be linear, consisting of a forward path and a
feedback path as a closed-loop operator of the system

• The analysis of the dynamic behavior of neural networks involving the


application of feedback is unfortunately complicated by the fact that
the processing units used for the construction of the network are
usually nonlinear
Network Architecture
• Single-Layer Feedforward
Networks
• In a layered neural network, the
neurons are organized in the
form of layers
• we have an input layer of source
nodes that projects directly onto
an output layer of neurons
• this network is strictly of a
feedforward type
Network Architecture
• It is illustrated in the figure for
the case of four nodes in both
the input and output layers.

• Such a network is called a single-


layer network, with the
designation “single-layer”
referring to the output layer of
computation nodes (neurons).
Network Architecture
• Multilayer Feedforward
Networks

• the presence of one or more


hidden layers, whose
computation nodes are
correspondingly called hidden
neurons or hidden units
Network Architecture
• The function of hidden neurons is to intervene between the external input
and the network output in some useful manner

• By adding one or more hidden layers, the network is enabled to extract


higher-order statistics from its input

• The source nodes in the input layer of the network supply respective
elements of the activation pattern (input vector), which constitute the input
signals applied to the neurons in the second layer (i.e., the first hidden layer).

• The output signals of the second layer are used as inputs to the third layer,
and so on for the rest of the network.
Network Architecture
• Recurrent Networks

• A recurrent neural network


distinguishes itself from a
feedforward neural network in
that it has at least one feedback
loop

You might also like