THE HUMAN BRAIN

The human nervous system may be viewed as a three-stage system, as depicted in the block
diagram of Fig. 1 (Arbib, 1987). Central to the system is the brain, represented by the neural
(nerve) net, which continually receives information, perceives it, and makes appropriate decisions.
Two sets of arrows are shown in the figure. Those pointing from left to right indicate the forward
transmission of information-bearing signals through the system. The arrows pointing from right to
left (shown in red) signify the presence of feedback in the system.

The receptors convert stimuli from the human body or the external environment into electrical
impulses that convey information to the neural net (brain). The effectors convert electrical
impulses generated by the neural net into discernible responses as system outputs. The struggle to
understand the brain has been made easier because of the pioneering work of Ramon y Cajal
(1911), who introduced the idea of neurons as structural constituents of the brain. Typically,
neurons are five to six orders of magnitude slower than silicon logic gates; events in a silicon chip
happen in the nanosecond range, whereas neural events happen in the millisecond range. However,
the brain makes up for the relatively slow rate of operation of a neuron by having a truly staggering
number of neurons (nerve cells) with massive interconnections between them. It is estimated that
there are approximately 10 billion neurons in the human cortex, and 60 trillion synapses or
connections (Shepherd and Koch, 1990). The net result is that the brain is an enormously efficient
structure. Specifically, the energetic efficiency of the brain is approximately10-16 joules (J) per
operation per second, whereas the corresponding value for the best computers is orders of
magnitude larger.
Synapses, or nerve endings, are elementary structural and functional units that mediate the
interactions between neurons. The most common kind of synapse is a chemical synapse, which
operates as follows: A presynaptic process liberates a transmitter substance that diffuses across
the synaptic junction between neurons and then acts on a postsynaptic process. Thus a synapse
converts a presynaptic electrical signal into a chemical signal and then back into a postsynaptic
electrical signal (Shepherd and Koch, 1990). In electrical terminology, such an element is said to
be a nonreciprocal two-port device. In traditional descriptions of neural organization, it is assumed
that a synapse is a simple connection that can impose excitation or inhibition, but not both on the
receptive neuron.
Earlier we mentioned that plasticity permits the developing nervous system to adapt to its
surrounding environment (Eggermont, 1990; Churchland and Sejnowski, 1992). In an adult brain,
plasticity may be accounted for by two mechanisms: the creation of new synaptic connections
between neurons, and the modification of existing synapses. Axons, the transmission lines, and
dendrites, the receptive zones, constitute two types of cell filaments that are distinguished on
morphological grounds; an axon has a smoother surface, fewer branches, and greater length,
whereas a dendrite (so called because of its resemblance to a tree) has an irregular surface and
more branches (Freeman, 1975). Neurons come in a wide variety of shapes and sizes in different
parts of the brain. Figure 2 illustrates the shape of a pyramidal cell, which is one of the most
common types of cortical neurons. Like many other types of neurons, it receives most of its inputs
through dendritic spines; see the segment of dendrite in the insert in Fig. 2 for detail. The pyramidal
cell can receive 10,000 or more synaptic contacts, and it can project onto thousands of target cells.
The majority of neurons encode their outputs as a series of brief voltage pulses.

These pulses, commonly known as action potentials, or spikes,3 originate at or close to the cell
body of neurons and then propagate across the individual neurons at constant velocity and
amplitude. The reasons for the use of action potentials for communication among neurons are
based on the physics of axons. The axon of a neuron is very long and thin and is characterized by
high electrical resistance and very large capacitance. Both of these elements are distributed across
the axon. The axon may therefore be modeled as resistance-capacitance (RC) transmission line,
hence the common use of “cable equation” as the terminology for describing signal propagation
along an axon. Analysis of this propagation mechanism reveals that when a voltage is applied at
one end of the axon, it decays exponentially with distance, dropping to an insignificant level by
the time it reaches the other end. The action potentials provide a way to circumvent this
transmission problem (Anderson, 1995).

EQUIVALENT MODELS OF A NEURON:

A neuron is an information-processing unit that is fundamental to the operation of a neural

network. The block diagram of Fig. 3 shows the model of a neuron, which forms the basis for
designing a large family of neural networks studied in later chapters. Here, we identify three basic
elements of the neural model:
1. A set of synapses, or connecting links, each of which is characterized by a weight or
strength of its own. Specifically, a signal xj at the input of synapse j connected to neuron k
is multiplied by the synaptic weight wkj. It is important to make a note of the manner in
which the subscripts of the synaptic weight wkj are written. The first subscript in wkj refers
to the neuron in question, and the second subscript refers to the input end of the synapse to
which the weight refers. Unlike the weight of a synapse in the brain, the synaptic weight
of an artificial neuron may lie in a range that includes negative as well as positive values.
2. An adder for summing the input signals, weighted by the respective synaptic strengths of
the neuron; the operations described here constitute a linear combiner.
3. An activation function for limiting the amplitude of the output of a neuron. The activation
function is also referred to as a squashing function, in that it squashes (limits) the
permissible amplitude range of the output signal to some finite value. Typically, the
normalized amplitude range of the output of a neuron is written as the closed unit interval
[0,1], or, alternatively, [-1,1].

Figure 3: Nonlinear model of a neuron, labeled k.

The neural model of Fig. 3 also includes an externally applied bias, denoted by bk. The bias bk
has the effect of increasing or lowering the net input of the activation function, depending on
whether it is positive or negative, respectively.

In mathematical terms, we may describe the neuron k depicted in Fig. 3 by writing the pair of
equations:
where x1, x2, ..., xm are the input signals; wk1, wk2, ..., wkm are the respective synaptic weights of
neuron k; uk (not shown in Fig. 3) is the linear combiner output due to the input signals; bk is the
bias; φ(·) is the activation function; and yk is the output signal of the neuron.