LIST OF FIGURES

FIGURE No. NAME OF FIGURE PAGE No.

2.1 Components of neuron 12

2.2 The synapse 12

2.3 Neuron model 13

2.4 A simple neuron 14

2.5 MCP neuron 16

2.6 Network layers 17

2.7 Feed forward networks 18

2.8 Feed back networks 19

2.9 The Mcculloh Pitts model 19

2.10 Transfer functions 22

2.11 Back Propagation network 24

2.12 The principle layout for a reinforcement 26

learning agent

2.13 Flowchart for implementation of applications 28

3.1 History of VHDL 34

4.1 Back Propagation network 45

5.1 Circuit architecture of prototype chip 51

5.2 Block diagram of neuron 52

5.3 Block diagram of synapse 53

5.4 Block diagram of error generator 54

FIGURE No. NAME OF FIGURE PAGE No.

5.5 Block diagram of Back Propagation 55

6.1 Simulation results for synapse 61

6.2 Simulation results for neuron 63

6.3 Simulation results for synapse 65

6.4 Simulation results for neuron 67

6.5 Simulation results for error generator at output 69

6.6 Simulation results for weight update unit 71

6.7 Simulation results for weight transfer unit 73

6.8 Simulation results for error generator at input 75

6.9 Simulation results for weight update unit 77

6.10 Simulation results for weight transfer unit 79

6.11 Simulation results for final entity 82-86

LIST OF TABLES
TABLE No. NAME OF TABLE PAGE No.

2.1 Truth Table before applying the firing rules 15

2.2 Truth Table after applying the firing rules 15

5.1 Table of signals for Back Propagation 56

algorithm

Chapter 3 VHSIC Hardware Description Language (VHDL)

3.1Introduction………… ................................................................................................... 32
3.2 History of VHDL………………………………………………………….…33
3.3 Advantages of VHDL………………………………………………...………34
3.4 Entities and Architecture……………………………………………………..35
3.5 Data Types …………………………………………………………………...35
3.6 Design Units………………………………………………………………….36
3.6.1 Entities……………………………………………………....………….37
3.6.2 Architectures…………………………………………………………...37
3.6.3 Packages and Package Bodies………………………………………….38
3.6.4 Configurations…………………….……………………………………39
3.7 Levels of abstraction………………………………………………………...39
3.7.1 Behavior…………………………………………………………...…..40
3.7.2 Dataflow……………………………………………………………….41
3.7.3 Structure……………………………………………………………….41
3.8Objects, Data Types and Operators…………………………………………..42
3.8.1 Using Signals………………………………………………………...…42
3.8.2 Using Variables………………………………………………………...42
Chapter 4 Back propagation Algorithm
4.1 Introduction…………………………………………………………………..43
4.2 History of algorithm………………………………………………………….43
4.3 Leaning with the back propagation algorithm………………………………..44
4.4 Implementation of back propagation algorithm……………………………...46
4.5 Drawbacks…………………………………………………………………....48
Chapter 5 Hardware Implementation
5.1 Introduction…………………………………………………………….…….50
5.2 Chip architecture……………………………………………………….…….51
5.3 Detailed analysis of the block diagram and hardware implementation………54
5.4 Brief Description of the Entities…………………………………………...…56
5.4.1 Synapse…………………………………………………………………56
5.4.2 Neuron……………………………………………………………….…57
5.4.3 Error Generator at the output…………………………………………...57
5.4.4 Weight update…………………………………………………………..58
5.4.5 Weight transfer unit…………………………………………………….58
5.4.6 Error Generator at the Input……………………………………………58
Chapter 6 Simulation Results
6.1Synapse…………………………………………………………………….…60
6.1.1 Inputs and outputs for the entity Synapse……………………………...60
6.1.2 Simulation Results for the entity Synapse…………………….………..61
6.2Neuron………………………………………………………………………..62
6.2.1 Inputs and outputs for the entity Neuron……………………………….62
6.2.2 Simulation Results for the entity Neuron………………………………63
6.3Synapse……………………………………………………………………….64
6.3.1 Inputs and outputs for the entity Synapse……………………………...64
6.3.2 Simulation Results For the entity Synapse…………………….……….65
6.4Neuron……………………………………………………………………..…66
6.4.1 Inputs and outputs for the entity Neuron……………………………….66
6.4.2 Simulation Results for the entity Neuron………………………………67
6.5Error Generator at Output…………………………………………………….68
6.5.1 Inputs and outputs for the entity Error Generator at the output………..68
6.5.2 Simulation Results for the entity Error Generator at the output……..…69
6.6Weight Update……………………………………………………………..…70
6.6.1 Inputs and outputs for the entity Weight Update ...................... ……...70
6.6.2 Simulation Results for the entity Weight Update………………………71
6.7Weight Transfer………………………………………………………………72
6.7.1 Inputs and outputs for the entity Weight Transfer .................... ……...72
6.7.2 Simulation Results for the entity Weight Transfer.………………….…73
6.8Error Generator at Input……………………………………………………...74
6.8.1 Inputs and outputs for the entity Error Generator at Input…………….74
6.8.2 Simulation Results for the entity Error Generator at Input……………75
6.9Weight Update…………………………………………………………….…76
6.9.1 Inputs and outputs for the entity Weight Update ................. …………76
6.9.2 Simulation Results for the entity Weight Update ……………………...77
6.10Weight Transfer……………………………………………………………..78
6.10.1 Inputs and outputs for the entity Weight Transfer .............................................. ……..78
6.10.2 Simulation Results for the entity Weight Transfer …………………...79 6.11 Final
Entity………………………………………………………………….80
6.11.1 Inputs and Outputs for the Final Entity…………………………….…80
6.11.2 Simulation Results for the Final Entity……………………………….82
Chapter 7 Conclusion and Future Scope
7.1 Conclusion…………………………………………………………………..…88
7.2 Future scope…………………………………………………………….……..89
References ….………………………………………………………………………………...90
List of
Publications 93
CHAPTER 1
INTRODUCTION
1.1 INTRODUCTION

CHAPTER 2
INTRODUCTION TO NEURAL NETWORKS
2.1 INTRODUCTION
CHAPTER 3 VHSIC HARDWARE DESCRIPTION LANGUAGE

3.1 INTRODUCTION
Hardware description languages are especially useful to gain more control of parallel processes
as well as to circumvent some of the idiosyncrasies of the higher level programming languages.
For example, the compiler will often add latency to loops during compilation for implementation.
This can be difficult to fix in the higher-level languages, though the solution may be quite
obvious at the hardware description level. One particularly frustrating peculiarity is the
implementation of multipliers. For all multiply commands, the complier requires three multipliers
to be used, though typically one is sufficient. The compiler’s multipliers also are intended for
integers. For a fixed-point design, the decimal point must be moved after every multiply. This is
much easier to implement at the hardware description level. [11]
VHDL is a programming language that has been designed and optimized for describing the
behavior of digital systems. VHDL has many features appropriate for describing the behavior of
electronic components ranging from simple logic gates to complete microprocessors and
custom chips. Features of VHDL allow electrical aspects of circuit behavior (such as rise and fall
times of signals, delays through gates, and functional operation) to be precisely described. The
resulting VHDL simulation models can then be used as building blocks in larger circuits (using
schematics, block diagrams or system-level VHDL descriptions) for the purpose of
simulation.
VHDL is also a general-purpose programming language: just as high-level programming
languages allow complex design concepts to be expressed as computer programs, VHDL allows
the behavior of complex electronic circuits to be captured into a design system for automatic
circuit synthesis or for system simulation. Like Pascal, C and C++, VHDL includes features
useful for structured design techniques, and offers a rich set of control and data representation
features. Unlike these other programming languages, VHDL provides features allowing concurrent
events to be described. This is important because the hardware described using VHDL is
inherently concurrent in its operation.
One of the most important applications of VHDL is to capture the performance
specification for a circuit, in the form of what is commonly referred to as a test bench. Test
benches are VHDL descriptions of circuit stimuli and corresponding expected outputs that
verify the behavior of a circuit over time. Test benches should be an integral part of any
VHDL project and should be created in tandem with other descriptions of the circuit.
One of the most compelling reasons for you to become experienced with and knowledgeable
in VHDL is its adoption as a standard in the electronic design community. Using a standard
language such as VHDL virtually guarantees that you will not have to throw away and recapture
design concepts simply because the design entry method you have chosen is not supported in a
newer generation of design tools. Using a standard language also means that you are more likely to
be able to take advantage of the most up-to-date design tools and that you will have access to a
knowledge base of thousands of other engineers, many of whom are solving problems similar to
your own.
3.2 HISTORY OF VHDL

1981 - Initiated by US DoD to address hardware life-cycle crisis

1983-85 - Development of baseline language by Intermetrics, IBM and TI

1986 - All rights transferred to IEEE

1987 - Publication of IEEE Standard

1987 - Mil Std 454 requires comprehensive VHDL descriptions to be delivered with Asics
1994 - Revised standard (named VHDL 1076-1993)
1981

Fig-3.1: History of VHDL

3.3 ADVANTAGES OF VHDL

Why choose to use VHDL for your design efforts? There are many likely reasons. If you ask most
VHDL tool vendors this question, the first answer you will get is, "It will improve your
productivity." But just what does this mean? Can you really expect to get your projects done faster
using VHDL than by using your existing design methods? The answer is yes, but probably not
the first time you use it, and only if you apply VHDL in a structured manner. VHDL (like a
structured software design language) is most beneficial when you use a structured, top-down
approach to design. Real increases in productivity will come later, when you have climbed
higher on the VHDL learning curve and have accumulated a library of reusable VHDL
components.
Productivity increases will also occur when you begin to use VHDL to enhance communication
between team members and when you take advantage of the more powerful tools for simulation
and design verification that are available. In addition, VHDL allows you to design at a more
abstract level. Instead of focusing on a gate-level implementation, you can address the behavioral
function of the design.
How will VHDL increase your productivity? By making it easy to build and use libraries of
commonly used VHDL modules. VHDL makes design reuse feel natural. As you discover the
benefits of reusable code, you will soon find yourself thinking of ways to write your VHDL
statements in ways that make them general purpose. Writing portable code will become an
automatic reflex.
Another important reason to use VHDL is the rapid pace of development in electronic design
automation (EDA) tools and in target technologies. Using a standard language such as VHDL
can greatly improve your chances of moving into more advanced tools (for example, from a basic
low-cost simulator to a more advanced one) without having to re-enter your circuit descriptions.
Your ability to retarget circuits to new types of device targets (for example, ASICs, FPGAs, and
complex PLDs) will also be improved by using a standard design entry method. [12]
3.4 ENTITIES AND ARCHITECTURES
Every VHDL design description consists of at least one entity/architecture pair. An entity
declaration describes the circuit as it appears from the "outside" - from the perspective of its
input and output interfaces. If you are familiar with schematics, you might think of the entity
declaration as being analogous to a block symbol on a schematic.
The second part of a minimal VHDL design description is the architecture declaration. Before
simulation or synthesis can proceed, every referenced entity in a VHDL design description must
be bound with a corresponding architecture. The architecture describes the actual function—or
contents—of the entity to which it is bound. Using the schematic as a metaphor, you can think of
the architecture as being roughly analogous to a lower-level schematic referenced by the higher-
level functional block symbol.
3.5 DATA TYPES
Like a high-level software programming language, VHDL allows data to be represented in terms of
high-level data types. A data type is an abstract representation of stored data, such as you might
encounter in software languages. These data types might represent individual wires in a circuit, or
they might represent collections of wires.
The bit data type has only two possible values: ‘1’ or ‘0’. (A bit vector is simply an array of
bits.) Every data type in VHDL has a defined set of values, and a defined set of valid operations.
Type checking is strict, so it is not possible, for example, to directly assign the value of an
integer data type to a bit vector data type. There are ways to get around this restriction, using
what are called type conversion functions.
3.6 DESIGN UNITS
One concept unique to VHDL (when compared to software programming languages and to its main
rival, Verilog) is the concept of a design unit. Design units in VHDL (which may also be referred
to as library units) are segments of VHDL code that can be compiled separately and stored in a
library. [13]
You have been introduced to two design units already: the entity and the architecture. There are
actually five types of design units in VHDL; entities, architectures, packages, package bodies,
and configurations. Entities and architectures are the only two design units that you must have in
any VHDL design description. Packages and configurations are optional.
3.6.1 ENTITIES
A VHDL entity is a statement (indicated by the entity keyword) that defines the external
specification of a circuit or sub-circuit. The minimum VHDL design description must include at
least one entity and one corresponding architecture.
When you write an entity declaration, you must provide a unique name for that entity and a port
list defining the input and output ports of the circuit. Each port in the port list must be given a
name, direction (or mode, in VHDL jargon) and a type. Optionally, you may also include a special
type of parameter list (called a generic list) that allows you to pass additional information into an
entity.
3.6.2 ARCHITECTURES
A VHDL architecture declaration is a statement (beginning with the architecture keyword) that
describes the underlying function and/or structure of a circuit. Each architecture in your design
must be associated (or bound) by name with one entity in the design.
VHDL allows you to create more than one alternate architecture for each entity. This feature is
particularly useful for simulation and for project team environments in which the design of the
system interfaces (expressed as entities) is performed by a different engineer than the lower-
level architectural description of each component circuit, or when you simply want to experiment
with different methods of description.
An architecture declaration consists of zero or more declarations (of items such as intermediate
signals, components that will be referenced in the architecture, local functions and procedures,
and constants) followed by a begin statement, a series of concurrent statements, and an end
statement.
3.6.3 PACKAGES AND PACKAGE BODIES
A VHDL package declaration is identified by the package keyword, and is used to collect
commonly used declarations for use globally among different design units. You can think of a
package as a common storage area, one used to store such things as type declarations,
constants, and global subprograms. Items defined within a package can be
made visible to any other design unit in the complete VHDL design, and they can be compiled
into libraries for later re-use.
A package can consist of two basic parts: a package declaration and an optional package body.
Package declarations can contain the following types of statements:
• Type and subtype declarations
• Constant declarations
• Global signal declarations
• Function and procedure declarations
• Attribute specifications
• File declarations
• Component declarations
• Alias declarations
• Disconnect specifications
• Use clauses
Items appearing within a package declaration can be made visible to other design units through
the use of a use statement.
If the package contains declarations of subprograms (functions or procedures) or defines one or
more deferred constants (constants whose value is not immediately given), then a package body is
required in addition to the package declaration. A package body (which is specified using the
package body keyword combination) must have the same name as its corresponding package
declaration, but it can be located anywhere in the design, in the same or a different source file.
3.6.4 CONFIGURATIONS
The final type of design unit available in VHDL is called a configuration declaration. You can
think of a configuration declaration as being roughly analogous to a parts list for your design. A
configuration declaration (identified with the configuration keyword) specifies which
architectures are to be bound to which entities, and it allows you to change how components are
connected in your design description at the time of simulation. Configuration declarations are
always optional, no matter how complex a design description you create. In the absence of a
configuration declaration, the VHDL standard specifies a set of rules that provide you with a
default configuration. For example, in the case where you have provided more than one
architecture for an entity, the last architecture compiled will take precedence and will be bound to
the entity.
3.7 LEVELS OF ABSTRACTION
VHDL supports many possible styles of design description. These styles differ primarily in how
closely they relate to the underlying hardware. When we speak of the different styles of VHDL,
we are really talking about the differing levels of abstraction possible using the language—
behavior, dataflow, and structure. [14]
Suppose the performance specifications for a given project are: "the compressed data coming out
of the DSP chip needs to be analyzed and stored within 70 nanoseconds of the strobe signal
being asserted..." This human language specification must be refined into a description that can
actually be simulated. A test bench written in combination with a sequential description is one such
expression of the design. These are all points in the Behavior level of abstraction.
After this initial simulation, the design must be further refined until the description is something
a VHDL synthesis tool can digest. Synthesis is a process of translating an abstract concept into a
less-abstract form. The highest level of abstraction accepted by today’s synthesis tools is the
Dataflow level.
The Structure level of abstraction comes into play when little chunks of circuitry are to be
connected together to form bigger circuits. (If the little chunks being connected are actually quite
large chunks, then the result is what we commonly call a block diagram.) Physical
information is the most basic level of all and is outside the scope of VHDL. This level involves
actually specifying the interconnects of transistors on a chip, placing and routing macro cells
within a gate array or FPGA, etc.
3.7.1 BEHAVIOR
The highest level of abstraction supported in VHDL is called the behavioral level of abstraction.
When creating a behavioral description of a circuit, you will describe your circuit in terms of its
operation over time. The concept of time is the critical distinction between behavioral
descriptions of circuits and lower-level descriptions (specifically descriptions created at the
dataflow level of abstraction).
In a behavioral description, the concept of time may be expressed precisely, with actual delays
between related events (such as the propagation delays within gates and on wires), or it may simply
be an ordering of operations that are expressed sequentially (such as in a functional description of
a flip-flop). When you are writing VHDL for input to synthesis tools, you may use behavioral
statements in VHDL to imply that there are registers in your circuit. It is unlikely, however, that
your synthesis tool will be capable of creating precisely the same behavior in actual circuitry as
you have defined in the language. (Synthesis tools today ignore detailed timing specifications,
leaving the actual timing results at the mercy of the target device technology.) It is also unlikely that
your synthesis tool will be capable of accepting and processing a very wide range of behavioral
description styles.
3.7.2 DATAFLOW
In the dataflow level of abstraction, you describe your circuit in terms of how data moves through
the system. At the heart of most digital systems today are registers, so in the dataflow level of
abstraction you describe how information is passed between registers in the circuit. You will
probably describe the combinational logic portion of your circuit at a relatively high level (and let
a synthesis tool figure out the detailed implementation in logic gates), but you will likely be quite
specific about the placement and operation of registers in the complete circuit. The dataflow level
of abstraction is often called register transfer logic, or RTL.
There are some drawbacks to using a dataflow method of design in VHDL. First, there are no
built-in registers in VHDL; the language was designed to be general-purpose, and VHDL’s
designers on its behavioral aspects placed the emphasis. If you are going to write VHDL at the
dataflow level of abstraction, you must first create (or obtain) behavioral descriptions of the
register elements you will be using in your design. These elements must be provided in the form of
components (using VHDL’s hierarchy features) or in the form of subprograms (functions or
procedures).
3.7.3 STRUCTURE
The third level of abstraction, structure, is used to describe a circuit in terms of its components.
Structure can be used to create a very low-level description of a circuit (such as a transistor-level
description) or a very high-level description (such as a block diagram).
In a gate-level description of a circuit, for example, components such as basic logic gates and flip-
flops might be connected in some logical structure to create the circuit. This is what is often
called a net list. For a higher-level circuit—one in which the components being connected are
larger functional blocks—structure might simply be used to segment the design description into
manageable parts.
Structure-level VHDL features, such as components and configurations, are very useful for
managing complexity. The use of components can dramatically improve your ability to re-use
elements of your designs, and they can make it possible to work using a top-down design
approach.
3.8 OBJECTS, DATA TYPES AND OPERATORS
VHDL includes a number of language elements, collectively called objects that can be used to
represent and store data in the system being described. The three basic types of objects that you
will use when entering a design description for synthesis or creating functional tests (in the form
of a test bench) are signals, variables and constants. Each object that you declare has a specific
data type (such as bit or integer) and a unique set of possible values. [15]
3.8.1 USING SIGNALS
Signals are objects that are used to connect concurrent elements (such as components, processes
and concurrent assignments), similar to the way that wires are used to connect components on a
circuit board or in a schematic. Signals can be declared globally in an external package or locally
within architecture, block or other declarative region.
3.8.2 USING VARIABLES
Variables are objects used to store intermediate values between sequential VHDL statements.
Variables are only allowed in processes, procedures and functions, and they are always local to
those functions. The 1076-1993 language standard adds a new type of global variable that has
visibility between different processes and subprograms. Variables in VHDL are much like variables
in a conventional software programming language. They immediately take on and store the value
assigned to them and they can be used to simplify a complex calculation or sequence of
logical operations.
LIST OF PUBLICATIONS
> Submitted a paper on BACK PROPAGATION ALGORITHM IMPLEMENTATION IN
VHDL” at CHITKARA INSTITUTE OF ENGG & TECH, CHANDIGARH in June 2006.
> Presented a paper on “CHARACTER RECOGNITION USING NEURAL NETWORKS” in
conference on ELECTRONIC CIRCUITS AND COMMUNICATION SYSTEMS at THAPAR
INSTITUTE OF ENGG. & TECH.,
PATIALA in February 2006.
> Presented a paper on “FREE SPACE OPTICS” in conference on ELECTRONIC
CIRCUITS AND COMMUNICATION SYSTEMS at THAPAR INSTITUTE OF
ENGG. & TECH., PATIALA in February 2006.