‫ﺷﺒﮑﻪ ﻫﺎي ﻋﺼﺒﯽ‬

‫)‪(Neural Networks‬‬

‫‪Perceptron Learning Rule‬‬

‫داﻧﺸﮕﺎه آزاد اﺳﻼﻣﯽ واﺣﺪ ﻋﻠﻮم و ﺗﺤﻘﯿﻘﺎت‬

 ‫ﺷﺒﮑﻪ ﻋﺼﺒﯽ ﭼﯿﺴﺖ؟‬
‫‪ ‬روﺷﯽ ﺑﺮاي ﻣﺤﺎﺳﺒﻪ اﺳﺖ ﮐﻪ ﺑﺮ‬
‫‪Input 0‬‬ ‫‪Input 1‬‬ ‫‪...‬‬ ‫‪Input n‬‬ ‫ﭘﺎﯾﻪ اﺗﺼﺎل و ﺑﻪ ﻫﻢ ﭘﯿﻮﺳﺘﻪ ﭼﻨﺪﯾﻦ‬
‫واﺣﺪ ﭘﺮدازﺷﯽ ﺳﺎﺧﺘﻪ ﻣﯽﺷﻮد‪.‬‬

‫‪ ‬ﺷﺒﮑﻪ از ﺗﻌﺪاد دﻟﺨﻮاﻫﯽ ﺳﻠﻮل‬

‫‪H0‬‬ ‫‪H1‬‬ ‫‪Hm‬‬
‫‪...‬‬ ‫‪Hidden Layer‬‬

‫‪O0‬‬ ‫‪O1‬‬ ‫‪Oo‬‬

‫)ﮔﺮه‪ ،‬واﺣﺪ ﯾﺎ ﻧﻮرون( ﺗﺸﮑﯿﻞ‬
‫‪...‬‬
‫ﻣﯽﺷﻮد ﮐﻪ ﻣﺠﻤﻮﻋﻪ ورودي را ﺑﻪ‬
‫‪...‬‬
‫ﺧﺮوﺟﯽ رﺑﻂ ﻣﯽدﻫﻨﺪ‪.‬‬
‫‪Output 0‬‬ ‫‪Output 1‬‬ ‫‪Output o‬‬

‫‪2‬‬
 ‫ﺷﺒﮑﻪ ﻋﺼﺒﯽ ﭼﯿﺴﺖ؟‬
‫ﺷﺒﮑﻪ ﻋﺼﺒﯽ ﻣﺼﻨﻮﻋﯽ روﺷﯽ ﻋﻤﻠﯽ ﺑﺮاي ﯾﺎدﮔﯿﺮي ﺗﻮاﺑﻊ ﮔﻮﻧﺎﮔﻮن ﻧﻈﯿﺮ‬ ‫‪‬‬

‫ﺗﻮاﺑﻊ ﺑﺎ ﻣﻘﺎدﯾﺮ ﺣﻘﯿﻘﯽ‪ ،‬ﺗﻮاﺑﻊ ﺑﺎ ﻣﻘﺎدﯾﺮ ﮔﺴﺴﺘﻪ و ‪ ...‬ﻣﯽﺑﺎﺷﺪ‪.‬‬

‫ﯾﺎدﮔﯿﺮي ﺷﺒﮑﻪ ﻋﺼﺒﯽ ﺗﺎ ﺣﺪ ﺑﺎﻻﯾﯽ در ﺑﺮاﺑﺮ ﺧﻄﺎﻫﺎي دادهﻫﺎي آﻣﻮزﺷﯽ‬ ‫‪‬‬

‫ﻣﺼﻮن ﺑﻮده و اﯾﻨﮕﻮﻧﻪ ﺷﺒﮑﻪﻫﺎ ﺑﺎ ﻣﻮﻓﻘﯿﺖ در ﺣﻞ ﻣﺴﺎﺋﻠﯽ ﻧﻈﯿﺮ ﺷﻨﺎﺳﺎﯾﯽ‬

‫ﮔﻔﺘﺎر‪ ،‬ﺷﻨﺎﺳﺎﯾﯽ و ﺗﻌﺒﯿﺮ ﺗﺼﺎوﯾﺮ‪ ،‬و ﯾﺎدﮔﯿﺮي رﺑﺎت ﻣﻮرد اﺳﺘﻔﺎده ﻗﺮار‬
‫ﮔﺮﻓﺘﻪاﻧﺪ‪.‬‬

‫‪3‬‬
 ‫ﺷﺒﮑﻪ ﻋﺼﺒﯽ ﭼﻪ ﻗﺎﺑﻠﯿﺖﻫﺎﯾﯽ دارد؟‬
‫‪ ‬ﻣﺤﺎﺳﺒﻪ ﯾﮏ ﺗﺎﺑﻊ ﻣﻌﻠﻮم‬

‫‪ ‬ﺗﻘﺮﯾﺐ ﯾﮏ ﺗﺎﺑﻊ ﻧﺎﺷﻨﺎﺧﺘﻪ‬

‫‪ ‬ﺷﻨﺎﺳﺎﯾﯽ اﻟﮕﻮ‬

‫‪ ‬ﭘﺮدازش ﺳﯿﮕﻨﺎل‬

‫‪ ‬ﯾﺎدﮔﯿﺮي‬

‫‪4‬‬
 ‫ﻣﺴﺎﺋﻞ ﻣﻨﺎﺳﺐ ﺑﺮاي ﯾﺎدﮔﯿﺮي ﺷﺒﮑﻪﻫﺎي ﻋﺼﺒﯽ‬
‫ﺧﻄﺎ در دادهﻫﺎي آﻣﻮزﺷﯽ وﺟﻮد داﺷﺘﻪ ﺑﺎﺷﺪ‪ ،‬ﻣﺜﻞ ﻣﺴﺎﺋﻠﯽ ﮐﻪ دادهﻫﺎي‬ ‫‪‬‬

‫آﻣﻮزﺷﯽ داراي ﻧﻮﯾﺰ ﺣﺎﺻﻞ از دادﻫﺎي ﺳﻨﺴﻮرﻫﺎ ﻧﻈﯿﺮ دورﺑﯿﻦ و ﻣﯿﮑﺮوﻓﻦﻫﺎ‬

‫ﻫﺴﺘﻨﺪ‪.‬‬

‫ﻣﻮاردي ﮐﻪ ﻧﻤﻮﻧﻪﻫﺎ ﺗﻮﺳﻂ ﻣﻘﺎدﯾﺮ زﯾﺎدي زوج )وﯾﮋﮔﯽ ‪ ،‬ﻣﻘﺪار( ﻧﺸﺎن داده‬ ‫‪‬‬

‫ﺷﺪه ﺑﺎﺷﻨﺪ‪ ،‬ﻧﻈﯿﺮ دادهﻫﺎي ﺣﺎﺻﻞ از ﯾﮏ آزﻣﺎﯾﺶ ﭘﺰﺷﮑﯽ‪.‬‬

‫ﺗﺎﺑﻊ ﻫﺪف داراي ﻣﻘﺎدﯾﺮ ﭘﯿﻮﺳﺘﻪ ﺑﺎﺷﺪ‪.‬‬ ‫‪‬‬

‫‪5‬‬
 ‫ﻣﺴﺎﺋﻞ ﻣﻨﺎﺳﺐ ﺑﺮاي ﯾﺎدﮔﯿﺮي ﺷﺒﮑﻪﻫﺎي ﻋﺼﺒﯽ‬
‫زﻣﺎن ﮐﺎﻓﯽ ﺑﺮاي ﯾﺎدﮔﯿﺮي وﺟﻮد داﺷﺘﻪ ﺑﺎﺷﺪ‪ .‬اﯾﻦ روش در ﻣﻘﺎﯾﺴﻪ ﺑﺎ روشﻫﺎي‬ ‫‪‬‬

‫دﯾﮕﺮ ﻧﻈﯿﺮ درﺧﺖ ﺗﺼﻤﯿﻢ ﻧﯿﺎز ﺑﻪ زﻣﺎن ﺑﯿﺸﺘﺮي ﺑﺮاي ﯾﺎدﮔﯿﺮي دارد‪.‬‬

‫ﻧﯿﺎزي ﺑﻪ ﺗﻌﺒﯿﺮ ﺗﺎﺑﻊ ﻫﺪف ﻧﺒﺎﺷﺪ‪ ،‬زﯾﺮا ﺑﻪ ﺳﺨﺘﯽ ﻣﯽﺗﻮان وزنﻫﺎي ﯾﺎدﮔﺮﻓﺘﻪ ﺷﺪه‬ ‫‪‬‬

‫ﺗﻮﺳﻂ ﺷﺒﮑﻪ را ﺗﻌﺒﯿﺮ ﻧﻤﻮد‪.‬‬

‫‪6‬‬
 Early learning algorithms
• Designed for single layer neural networks
• Generally more limited in their applicability
• Some of them are
• Perceptron learning
• LMS or Widrow- Hoff learning
• Grossberg learning

7
 ‫ﭘﺮﺳﭙﺘﺮون‬
‫ﻧﻮﻋﯽ از ﺷﺒﮑﻪ ﻋﺼﺒﯽ ﺑﺮﻣﺒﻨﺎي ﯾﮏ واﺣﺪ ﻣﺤﺎﺳﺒﺎﺗﯽ ﺑﻪ ﻧﺎم ﭘﺮﺳﭙﺘﺮون ﺳﺎﺧﺘﻪ‬
‫ﻣﯽﺷﻮد‪ .‬ﯾﮏ ﭘﺮﺳﭙﺘﺮون ﺑﺮداري از وروديﻫﺎي ﺑﺎ ﻣﻘﺎدﯾﺮ ﺣﻘﯿﻘﯽ را ﮔﺮﻓﺘﻪ و ﯾﮏ‬
‫ﺗﺮﮐﯿﺐ ﺧﻄﯽ از اﯾﻦ وروديﻫﺎ را ﻣﺤﺎﺳﺒﻪ ﻣﯽﮐﻨﺪ‪.‬‬

‫اﮔﺮ ﻧﺘﯿﺠﻪ ﺣﺎﺻﻞ از ﯾﮏ ﻣﻘﺪار آﺳﺘﺎﻧﻪ ﺑﯿﺸﺘﺮ ﺑﻮد ﺧﺮوﺟﯽ ﭘﺮﺳﭙﺘﺮون ﺑﺮاﺑﺮ ﺑﺎ ‪ 1‬و در‬
‫ﻏﯿﺮ اﯾﻦﺻﻮرت ﻣﻌﺎدل ‪) -1‬ﯾﺎ ﺻﻔﺮ( ﺧﻮاﻫﺪ ﺑﻮد‪.‬‬

‫‪8‬‬
 ‫ﭘﺮﺳﭙﺘﺮون‬

‫‪9‬‬
 ‫اﺿﺎﻓﻪ ﮐﺮدن ﺑﺎﯾﺎس‬
‫‪ ‬اﻓﺰودن ﺑﺎﯾﺎس ﻣﻮﺟﺐ ﻣﯽﺷﻮد ﺗﺎ اﺳﺘﻔﺎده از ﺷﺒﮑﻪ ﭘﺮﺳﭙﺘﺮون ﺑﺎ ﺳﻬﻮﻟﺖ ﺑﯿﺸﺘﺮي‬
‫ﻣﻘﺪور ﺑﺎﺷﺪ‪.‬‬

‫‪ ‬ﺑﺮاي اﯾﻨﮑﻪ ﺑﺮاي ﯾﺎدﮔﯿﺮي ﺑﺎﯾﺎس ﻧﯿﺎزي ﺑﻪ اﺳﺘﻔﺎده از ﻗﺎﻧﻮن دﯾﮕﺮي ﻧﺪاﺷﺘﻪ‬
‫ﺑﺎﺷﯿﻢ‪ ،‬ﺑﺎﯾﺎس را ﺑﻪ ﺻﻮرت ﯾﮏ ورودي ﺑﺎ ﻣﻘﺪار ﺛﺎﺑﺖ ‪ 1‬در ﻧﻈﺮ ﮔﺮﻓﺘﻪ و وزن ‪b‬‬
‫را ﺑﻪ آن اﺧﺘﺼﺎص ﻣﯽدﻫﯿﻢ‪.‬‬

‫‪10‬‬
 Perceptron Geometric View
The equation below describes a (hyper-)plane in the input space
consisting of real valued m-dimensional vectors. The plane splits the
input space into two regions, each of them describing one class.
decision
region for C1
p2
w1p1 + w2p2 + b > 0

decision
boundary C1
p1
C2
w1p1 + w2p2 + b < 0
w1p1 + w2p2 + b = 0
11
 Two-Input Case

12
Apple/Banana Sorter

13
 Class Representation

Shape : 1 = Circular -1 = Elliptical

Texture : 1= Smooth -1 = Rough
Weight : 1 > 1 lbs. -1 < 1 lbs.

Perceptron Network

Hamming Network

Hopfield Network
14
 McCulloch-Pitts Perceptron

1
1
1

15
 Apple/Banana Example

16
 Testing the Network

17
 XOR problem
A typical example of non-linearly separable function is the
XOR. This function takes two input arguments with values in
{-1,1} and returns one output in {-1,1}, as specified in the
following table:
x1 x2 x1 ⊗ x2
-1 -1 -1
-1 1 1
1 -1 1
1 1 -1

If we think at -1 and 1 as encoding of the truth values False

and True, respectively, then XOR computes the logical
Exclusive or, which yields True if and only if the two inputs
18
have different truth values.
 XOR problem
In this graph of the XOR, input pairs giving output equal to 1 and -1 are
depicted with green and red circles, respectively. These two classes
(green and red) cannot be separated using a line. We have to use two
lines, like those depicted in blue. The following NN with two hidden
nodes realizes this non-linear separation, where each hidden node
describes one of the two blue lines.

x2
1

-1 1
x1

-1
19
 Supervised Learning
Network is provided with a set of examples of proper network
behavior (inputs/targets)

{ p1, t 1} , { p2, t 2} , … , {pQ,tQ }

20
 Perceptron Architecture AGAIN

21
 Single-Neuron Perceptron

22
 Decision Boundary

23
 Example - OR

24
 OR Solution

0.5

0.5

25
 Multiple-Neuron Perceptron
Each neuron will have its own decision boundary.

T
iw p + bi = 0

A single neuron can classify input vectors

into two categories.

A multi-neuron perceptron can classify

input vectors into 2S categories.
26
 Learning Rule Test Problem

27
 Starting Point

28
 Tentative Learning Rule

29
 Second Input Vector

30
 Third Input Vector

31
 Unified Learning Rule

32
 Multiple-Neuron Perceptrons

33
 Apple/Banana Example

34
 Second Iteration

35
 Check

36
 Perceptron Rule Capability

The Perceptron rule will always

converge to weights which accomplish
the desired classification, assuming that
such weights exist.

37
 Rosenblatt’s single layer perceptron is trained as
follow:

1. Randomly initialize all the networks weights.

2. Apply inputs and find outputs (Feedforward).
3. Compute the errors.
4. Update each weight as

wij (k + 1) = wij (k ) + η pi (k ) e j (k )

5. Repeat steps 2 to 4 until the errors reach the satisfactory

level.
38
 What is η?
 Name : Learning rate.

 Where is living: usually between 0 and 1.

 It can change it’s value during learning.

 Can define separately for each parameters.

39
 Example 1

40
 Example 1

41
 Example 1

42
 Example 1

43
 Example 1

44
 Example 1

45
 Example 1

46
 Example 2

47
 Example 2

48
 Example 2

49
 Example 2

50
 Example 2

51
 Example 2

52
 Perceptron Limitations

53
 Hamming Network

54
 Feedforward Layer

55
 Recurrent Layer

56
 Hamming Operation

57
 Hamming Operation

58
 Hopfield Network

59
 Apple/Banana Problem

60
 Summary
 Perceptron
 Feedforward Network
 Linear Decision Boundary
 One Neuron for Each Decision

 Hamming Network
 Competitive Network
 First Layer – Pattern Matching (Inner Product)
 Second Layer – Competition (Winner-Take-All)
 # Neurons = # Prototype Patterns

 Hopfield Network
 Dynamic Associative Memory Network
 Network Output Converges to a Prototype Pattern
 # Neurons = # Elements in each Prototype Pattern
61