|

p
p  








 




 





















Computation
Visualization
Programming







 
 


x 

|

|
|
|
| 
|

|
| | | | |   
   
|

 
          |


|


|

|
| | | | | | | |  
  
   
| | | | | | | | | 
 |
 |
 
   
   
| | | | | | | | | 
|
 

 |


|   
   
| | | | | | | | | | | | | | | |

|

  
   
|



|

|

 |

 
 
   
|
! 
| "|

|

"|

|
  
 
   
|
#
"| "||

|

|

|
| | | | | | | $%&'())%%%|



|

| | | | | | | $%&'())%%*|

+,|

| | | | | | | 
| 
"|-|
.|/
| | 
|
  "|/|%*)'%0%1&|

|

|
+
|
  | 
  
|
|
 
|
 
"|

|
| 
 |
| 
|

|
p
  p   
  


* |!23- 4 |*110||0%%0||
|  
 "|-|

|
 
|
 
||  |

 ||  
| 
|| 

| 


 | |
|
 
||
| 
|

|

|
|
|
|
|
|
| 

|


|
|  |
|  | ||
|



|
|

|

|||
 | 
 | 
|   
|

 |
|
| 
"|- ^|
+553/6|/78- #- - !| | |

|
|
||9 
|
|
|
  ||

 
||

|
|
|

 |


|
|
|8 
|# 
| | |
 |
 
|
|
|
 "|
|


 |


|

| | |
 

|9
||:

:|

|
 
|  |
|
|
|+/3|
  |*00*0"|+/3#|  |00))0%0*"|+/3#|  |00))0%0."|+/3#|  |0$000))%*."||+/3#|  |
0$000))%*(| | 
|
 ||
 
|
|
| 
 "|-|#
 
|6

|/


 | |
  |

|
|


|
||

|
|
|
 ||

 
"|| |


||

|
  |
 |
|
  
| | -| |

||
|

 |
|


 ||

|
|
|
 
|||

 | |

 |


| "|
|


|

|
|
  |
|
 |
|

 
"|
"|
|  
 |
/6 /"|#  "|# 

"| 
|4  "||3

|

|
|


| 
 "||
 
 
,||| 
  |
|
| 
"|-|
!
|
|
| |
|
| 
 |
|


| 
 |
|
 |

 
|
 
|

   |  
| | ; 

|*110|
+    |   |
/  |*11.|
#

|   |
; |*11)|
  |   |
;  |*11)|
+
  |   |
; |*11&|
+  |   |
|3

|
|<

|.|=3


|**>|
#
 
 
|0%%%| | | #, |     | |3
 
|
|<

 |(|=3


|*0>|
; 
|0%%*|
#

  |   | |   
|

|=3


|*0*>|;  |
0%%0|
! 
|
|
  
|

|=3


|*.>|

3 




Preface




Neu ral Networks   xviii
Basic Chapters   xx
M athematical Notation for Equations and Figures   xxi
 3     xxi

     xxi
       xxi
 p   xxi

 !"
!#      xxii
M athematics and Code Equ ivalents   xxiii
Neu ral Netw ork Design Book    xxiv
Acknowledgments   xxv





In tr oduction

$

Getting Started   1-2
 3   1-2
% &       1-2



Whats New in Ver si on 4.0    1-3
3 '  #     1-3
 & (      1-3
p 

    1-3
)  (    p      1-4
&#* ! '      1-4






iii

  + '  #     1-4

) #('# 
'     1-4



Neu ral Netw ork Applications    1-5

     1-5

      1-5
      1-5

# *    1-5
      1-5
3  3* 3      1-5
) (      1-6
!     1-6
! #      1-6
   1-6
& 
     1-6
& 
     1-6

( 
    1-6
     1-7
,   1-7
-    1-7
'    1-7
' 
     1-7

 ##
     1-7

      1-7

'
##     1-7





Neuron Model and Netw ork Ar chitectures

.

Neuron Model    2-2
'# p
    2-2

( 
  2-3

p
 / &
     2-5



Netw ork Architectures    2-8
 (p
    2-8

  (p
  2-11






iv

3 



Data Structu res   2-13

'#
3
 &
'  p     2-13
'#
' "
  &
 ) # p     2-14
'#
3
  &
) # p       2-16



T raining Styles    2-18
&  # 
0( * ,  p  1   2-18

    2-20



Summar y   2-24

 !"
   2-25





Perceptrons

2

In troduction   3-2
& #3   
   3-3



Neuron Model    3-4



Perceptron Architecture   3-6



Cr eatin g a Perceptr on (newp)   3-7
'#
0 #1  3-8
&  +01    3-9



Lear ning Rules    3-12



Perceptr on Learning Rule (learn p)    3-13



T raining (train)   3-16



L im itations and Cautions   3-21
,
   p#+ 3   -
    3-21









v

Graphical User Interface    3-23

&  
 &    3-23
3  3   p  0 1    3-23

 3       3-27

!3   - 
      3-29
3 p  4)     3-30
& # (# 3##      3-30
'* /  &    3-31



Summar y   3-33

 !"
   3-33
p 
      3-36





L in ear F ilters

5

In troduction   4-2



Neuron Model    4-3



Netw ork Architecture    4-4
3  p
0   1  4-4



Mean Squar e Erro r   4-8



Linear System Design (newli nd)   4-9



Linear Networks with Delays    4-10

 )      4-10

     4-10



L M S Algorithm (lear nwh)   4-13



Linear Classifi cation (train)









vi

3 



   4-15

L im itations and Cautions   4-18

,*   #  '  #  4-18
   # '  #    4-18
  )    /     4-18

  -     4-19




Summar y   4-20

 !"
   4-21
p 
      4-25





Backpropagation

6

Overview   5-2



Fundamentals    5-4
 
     5-4
'#
0 #1  5-8

    5-8




Faster T raining   5-14
/  - 0 71   5-14
-    0 1   5-16
38
   #    5-17
 ' -
    5-23
9
:p  #   5-26
 *   :"
0 #1    5-28
- 
  #  *   :"
0 #1    5-30



Speed and Memory Comparison   5-32
'
##     5-49



Impr oving Generalization    5-51
- 
+     5-52
! '      5-55
'
## ) 
     5-57







v ii

Prepr ocessing and Postprocessi ng   5-61

0 ##7 ##7##1    5-61
 ' ) *0  771   5-62
33#  0 71  5-63
3:
   0 1  5-64



Sample Training Session   5-66



L im itations and Cautions   5-71



Summar y   5-73





Control Systems

;

In troduction   6-2



NN P redictive Contr ol   6-4
'  #& (  6-4
3 * 3     6-5
 pp3 * 3      6-6



NARMA-L2 (F eedback Linearization) Control    6-14
&  (( p-:.      6-14
p-: .3   6-16
 p-:.3      6-18



M odel Reference Contr ol    6-23
  - (   3       6-25



Importing and Exporting    6-31
& # !p      6-31
& # !
)     6-35



Summar y   6-38








v iii

3 



Radial Basis Networks

<

In troduction   7-2
& #- 
    7-2



Radial Basi s Functions    7-3
p
     7-3
p     
     7-4
!) 0  1   7-5
 !((  )  0 1     7-7
) #    7-8



Generali zed Regr ession Networks   7-9
p     
     7-9
)  0  1   7-10



Probabilistic Neu ral Networks    7-12
p     
     7-12
)  0 1     7-13



Summar y   7-15

     7-16
p 
      7-18





Self-Or ganizing and



=

Learn. Vector Quant. Nets



In troduction   8-2
& #' ( :,+/9
   8-2



Competitive Learning    8-3
 
     8-3
3 3#   * p
p   0 1    8-4
>  -
 0  1  8-5
 -
 0 1  8-5






ix


    8-6
 !#     8-7



Self-Organizing Maps   8-9

 0 7  71    8-10

)  
07 7#71    8-14
 
     8-17
3   ' (,+3p
p  0 #1    8-18

0 #1   8-19

!#     8-23



Lear ning Vector Quanti zation Networks    8-31
 
     8-31
3 /9p  0 * "1   8-32
/9$ -
 0 * $1    8-35

    8-36

'
 # /9.$ -
 0 * .1    8-38



Summary and Conclusions   8-40
' ( :,+    8-40
 / 9
+p     8-40

     8-41
p 
      8-42





Recurrent Networks

?

In troduction   9-2
& #- 
 p   
  9-2



Elman Netw orks    9-3
 
     9-3
3 ! #p  0  #1  9-4

!#p      9-5









x

3 




|
|
| |
 |

| |
|
|
|
| |
|
 |
||
|
|
| |
||
|
| |

| |

|
 |

| |
 ||
||
|

















|