Scribd
Upload
132 views4 pages

Lecture 1 - Neural Network Definitions and Concepts 1

Neural networks are simplified models of the brain that consist of connected processing units like neurons. They can learn from experience, generalize, and are fault tolerant. The history of neural networks began in the 1940s and saw major developments like the perceptron in the 1950s and backpropagation in the 1970s. Neural networks are now used widely in applications such as pattern recognition, control systems, speech recognition and more. Biologically, neurons are information processing cells that receive and transmit electrical signals at connections called synapses.

Uploaded by

Ammar Alkindy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
132 views4 pages

Lecture 1 - Neural Network Definitions and Concepts 1

Neural networks are simplified models of the brain that consist of connected processing units like neurons. They can learn from experience, generalize, and are fault tolerant. The history of neural networks began in the 1940s and saw major developments like the perceptron in the 1950s and backpropagation in the 1970s. Neural networks are now used widely in applications such as pattern recognition, control systems, speech recognition and more. Biologically, neurons are information processing cells that receive and transmit electrical signals at connections called synapses.

Uploaded by

Ammar Alkindy
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Dr.

Qadri Hamarsheh

Neural Network Definitions and Concepts


Outline
Introduction.
 Definition of neural network.
 A brief history of neural networks.
 Neural Network Applications.
 Biological neural networks.
Introduction
Definition of neural network
 Artificial Neural Networks (ANN) are computers whose architecture is a
simplified model of the brain, they typically consist of many hundreds of
simple processing units which are wired together in a complex
communication network.
 Each unit or node is a simplified model of a real neuron which fires (sends
off a new signal) if it receives a sufficiently strong input signal from the
other nodes to which it is connected.
 Study of artificial neural networks is motivated by their similarity to
successfully working biological systems.

Comparison between the brain and the computer


Brain Computer
Working continuously. Passive data storage.
Parallel. Serial.
Can reorganize itself and therefore Static.
is able to learn (capability to learn)
Generalize and associate data
Fault tolerance against noisy input Fault non-tolerance against noisy
data. input data.

1
Dr. Qadri Hamarsheh

 So let us summarize the main characteristics we try to adapt from biology:


Self-organization and learning capability.
Generalization capability.
Fault tolerance.

A brief history of neural networks


The beginning
 As soon as 1943: Warren McCulloch and Walter Pitts introduced
models of neurological networks, calculate nearly any logic or arithmetic
function (electronic brains).
 1947: Walter Pitts and Warren McCulloch indicated a practical field of
application, namely the recognition of spacial patterns by neural networks.
 1949: Donald O. Hebb formulated the classical Hebbian rule which
represents in its more generalized form the basis of nearly all neural
learning procedures. The rule implies that the connection between two
neurons is strengthened when both neurons are active at the same time.
 1950: The Karl Lashley defended the thesis that brain information storage
is realized as a distributed system.
Golden age
 1951: For his dissertation Marvin Minsky developed the neurocomputer
Snark.
 1957-1958: Frank Rosenblatt developed the first successful
neurocomputer, Mark I perceptron, which was capable to recognize simple
20 × 20 pixel image
 1959: Frank Rosenblatt described different versions of the perceptron,
formulated and verified his perceptron convergence theorem. He
described a learning rule adjusting the connecting weights.
 1960: Bernard Widrow and Marcian E. Hoff introduced the ADALINE
(ADAptive LInear NEuron), a fast and precise adaptive learning system
being the first widely commercially used neural network (analog telephone
for real-time adaptive echo filtering) and was trained by Widrow-Hoff rule
or delta rule.
 1961: Karl Steinbuch introduced technical realizations of associative
memory.
 1969: Marvin Minsky and Seymour Papert published a precise
mathematical analysis of the perceptron to show that the perceptron model
was not capable of representing many important problems (XOR problem
and linear separability).
Two of the reasons for the "quiet years" of the 1970s were the failure of
single layer perceptrons to be able to/solve such simple problems
(mappings) as the XOR function and the lack of a general method of
training a multilayer net.
 1974: Paul Werbos developed a learning procedure called
backpropagation of error.

2
Dr. Qadri Hamarsheh

 1982:
o Teuvo Kohonen described the self-organizing feature maps (SOM)–
also known as Kohonen maps. He was looking for the mechanisms
involving self-organization in the brain.
o John Hopfield also invented the so-called Hopfield networks
(Hopfield has developed a number of neural networks based on fixed
weights and adaptive activations).
 1983: Fukushima, Miyake and Ito introduced the neural model of the
Neocognitron which could recognize handwritten characters.
 1986: The backpropagation of error learning procedure as a generalization
of the delta rule was separately developed and widely published by the
Parallel Distributed Processing Group: Non-linearly-separable
problems could be solved by multilayer perceptrons.

Neural Network Applications (Where are neural nets being used?)


 Signal Processing.
o Electronics
 Control.
o Automotive
o Defense
 Pattern Recognition.
 Medicine.
 Speech Production and Recognition.
 Industrial
o Robotics
o Transportation
o Manufacturing
o Oil & Gas
 Business.
o Banking
o Credit Card Activity
o Checking
o Financial
 Entertainment
 Insurance
 Securities
 Telecommunications

Biological neural networks


 Neurons are information processing cells.
 A neuron is nothing more than a switch with information input and output.
The switch will be activated if there are enough stimuli of other neurons
hitting the information input. Then, at the information output, a pulse is sent
to, for example, other neurons.
3
Dr. Qadri Hamarsheh

Components of biological neuron

Components of a neuron
 The dendrites of a neuron receive the information by special connections,
the synapses (Incoming signals from other neurons or cells are transferred
to a neuron by synapses).
 The signals are electric impulses that are transmitted across a synaptic gap
by means of a chemical process. The action of the chemical transmitter
modifies the incoming signal (typically, by scaling the frequency of the
signals that are received) in a manner similar to the action of the weights
in an artificial neural network.
 Some synapses transfer a strongly stimulating signal, some only weakly
stimulating ones.
 Dendrites branch like trees and receive electrical signals from many
different sources, which are then transferred into the nucleus of the cell.
 In the soma the weighted information is accumulated.
 After the cell nucleus (soma) has received a plenty of activating
(=stimulating) and inhibiting (=diminishing) signals by synapses, the soma
accumulates these signals. If the accumulated signal exceeds a certain
value (called threshold value), the soma of the neuron activates an
electrical pulse which then is transmitted to the neurons connected to the
current one.
 The axon transfers outgoing pulses.

You might also like