Advanced Information Retreival: Chapter 02: Modeling - Neural Network Model
Advanced Information Retreival: Chapter 02: Modeling - Neural Network Model
RETREIVAL
Chapter 02: Modeling Neural Network Model
Neural Networks
Neural Networks
Complex learning systems recognized in animal brains
Single neuron has simple structure
Interconnected sets of neurons perform complex learning tasks
Human brain has 1015 synaptic connections
Artificial Neural Networks attempt to replicate non-linear learning
found in nature
Dendrites
Axon
Cell Body
information
Then generate non-linear response when threshold reached
Signal sent to other neurons via axon
x1
x2
xn
response (y)
Response channeled downstream to other neurons
Numeric
Apply Min-max Normalization to continuous variables
X min( X )
X min( X )
range( X )
max( X ) min( X )
Works well when Min and Max known
Also assumes new data values occur within Min-Max range
Values outside range may be rejected or mapped to Min or Max
X*
Node
1
Node
2
Node
3
W1A
W1B
W2A
W2B
W3A
W3B
Node
A
Node
B
WAZ
Node
Z
WBZ
W0Z
W1A
W1B
W2A
W2B
W3A
W3B
WAZ
Node
B
WBZ
W0B
Node
Z
W0Z
x0 = 1.0
W0A = 0.5
W0B = 0.7
W0Z = 0.5
x1 = 0.4
W1A = 0.6
W1B = 0.9
WAZ = 0.9
x2 = 0.2
W2A = 0.8
W2B = 0.8
WBZ = 0.9
0.7
W = 0.6
= 0.4
The scalarx =value
computed
forW hidden
layer Node A equals
3
3A
3B
For
function
Neurons fire in biological organisms
Signals sent between neurons when combination of inputs
cross threshold
1
1 ex
In Node A, sigmoid function takes netA = 1.32 as input and
produces output
y
1
y
0.7892
1.32
1 e
f (net B )
1
0.8176
1.5
1 e
1
0.8750
1 e 1.9461
Back-Propagation
SSE
2
(
actual
output
)
Re cords OutputNodes
Back-Propagation (contd)
Back-Propagation Rules
j xij
x ij
learning rate
signifies ith input to node j
DOWNSTREAM
where,
W jk j refers to weighted sum of error responsibilities for nodes downstream
DOWNSTREAM
Example of Back-Propagation
Input Layer
Node
1
Node
2
Node
3
W1A
W1B
W2A
W2B
W3A
W3B
WAZ
Node
B
WBZ
W0B
Node
Z
W0Z
A output A (1 output A )
W jk j
DOWNSTREAM
B output B (1 output B )
W jk j
DOWNSTREAM
Summary
Termination Criteria
And
Learning Rate
Document
Terms
k1
Documents
d1
ka
ka
kb
kb
kc
dj
dj+1
kc
kt
dN
wiq
2
sqrt ( i wiq )
wij
2
sqrt ( i wij )
i Wiq Wij =
i wiq wij
Conclusions