Overview of

2 Language Processors

Syllabus
Programming languages and language processors, Language processing activities, Programn
execution, Fundamental of language processing, Symbol tables, Data structures for language
processing: Search data structures, Allocation data structures.

Contents
2.1 Programming Languages and Language Processors
Marks 7
Winter-17, 19,
2.2 Language Processing Activities
2.3 Fundamental of Language Processing
2.4 Fundamentals of Language Specification Winter-18, 19, Marks 3
2.5 Language Processor Development Tools Winter-14, 17· Marks 3
2.6 Symbol Tables Data Structures for Language Processing
2.7 Multiple Choice Questions
2.8 Short Questions and Answers

(2- 1)
 2-2 Overview of Language Processors
System Software

2.1 Programming Languages and Language Processors

:
GTU Winter-15, 17, 19
Language processor is a software and it links a specification gap or execution
gap.

Object C programs
oriented C++
programs preprocessor
(C++)

Errors
generated
by preprocessor

(a

Object
oriented Programs
language Translator for
programs C++ machine
(C++) language

Errors
generated
by translator.

(b)

Fig. 2.1.1 Example of language processors

Source program is input to a language processor and target program is output of

language processor. Language used for writing a source program and target
program is called source language and target language respectively.
Following are the practical requirement met by language processors :
Language processor provide facility to write, store and manage code. It checks the
code line by line to find syntax error.
1. Language translator 2. Detranslator
3. Preprocessor 4. Language migrator
• In a computer system, language translator bricges an execution gap to the
machine language.
-
TECHNICAL PUBLICATIONS An up thrust for knowledge
System Software 2-3 Overview of Language Prooessors

Detranslator bridges an execution gap from the machine language of a system.

Preprocessor also bridges an execution gap, but it is not a language trarnslator.
Language migrator bridges the specification gap between two programming
languages.
Fig. 2.1.1shows an example of language processors. Each language processor uses
its own set of commands.
Input source program is same but output target program is different in the
diagram. So one is preprocessor arnd other is translator.

2.1.1 Semantic Gap

Semantic : It represent the rules of meaning of a domain.
Semantic gap : It represent the difference between the semantics of two domains.
• Fig. 2.1.2 shows the semantic gap
between two domains. The
programmer job is to overcome
the semantic gap by translating
human language into
programming language. Domain 2
Domain 1 execution/
One side of application domain application
Computer
includes a user files to organize,
maintain database file and real Semantic gap
time applications.
Fig. 2.1.2 Semantic gap of two domain
Problems of semantic gap
1. Poor quality of software
2. Large development efforts

3. Development time is large.

Software engineering concept is used to tackled the above problems. SE uses
programming language and methodologies to solve these problem.
Software engineering steps are grouped into two types :
1. Specification, design and coding steps.
2. Programming language implementation steps.
New domain is introduce while implementing software. It is programming
language domain. There are three domains : application domain, PL domain and
execution domain.
Specification gap is between application domain and PL domain. Execution gap is
between PL domain and execution domain.
TECHNICAL PUBLICATIONS - Ân up thrust for knowledge
 2-4 Overview of Language
System Sofvare Processors

Fig. 2.1.3 shows specification and execution gaps.

Domain2
Domain1 programming
application Domain 3
language
execution
Specification
Execution
gap
gap
Fia. 2.1.3 Specification
and execution gaps
The specification gap is
bridged by the software
development team
gap by the designer of
the PL processor using and execution
PL domain helps
translator or an interpreter.
in improving the quality of
the software.
212 Interpreters
An interpreter is a
language processor. It
links an execution gap High level
language Inter preter Program
without generating program execution
machine language result
program. Fig. Input
2.1.4 Output
shows the interpreter.
Fig. 2.1.4 Interpreter
An interpreter is
language translator.
The language processing activities
of an interpreter cannot be separated
program execution from its
activities. This is because of the
There is also an absence of an output absence of target program.
interface of interpreter.
Fig. 2.1.5 shows the interpreter
domain.

Interpreter Domain

Programming Programn
language execution
domain domain

Fig. 2.1.5 Interpreter domain

TECHNICAL PUBLICATIONS - An up thrust for knowledge

 Syetem oftwane 2-6 Gvervievw of Languaga ProcessoIs

Problem Orlented Language (POL)

ApplicatiGn
.Software aystem quality is normally poor., They also dornalr
vequlre lage amount of time and cffort to develop
the gystem and for bridging the epeclflcation gap. Spscificaticn
. Fig. 2.1.6 showe problem orlented language dormain.
Solution to this problem to develop the PL
la Prohiem
domait. PL domain is very close or identical to the orient
languag
application domain.
Speclfication gap is small because of PL domain. So Evecuticn
gap
PL can ony be used for specillc applications. Because
of this, they are called problem oriented languages.
Execution
Procecduro Orlontod Languago domain
• Procedure orlented language provides general
Fig. 2.1.8 POL dormaln
purpose facilitles required in most application
domains. This type of language is not dependent of specific application domains.
Execution gap is large becausc of problem oriented language and specification gap
is larger because of procedure oriented language. An application designer. solves a
problem of large specification gap.
• It contains a systematic order of statemernts functions and comnands to complete
a computational program.

Advantagos
1. Itrequires less memory space.
2. It is suitable for general purpose programming

3. Support for portable source code.

4. Dala can casily flow from one functíon to another.

Dlsadvantagos
1. It does not support automatic memory management.
2. Itdocs not provide security for data.
3. Larger codes are difficult to maintain.

Rovlow Quostlons

I. Which activity reduce speciflcation gap? Explain with suitable example. Who will reduce execution
gap ? GTU : Winter-17, Marks 7
2. Give eNmples of language processors. GTU Winter-19, Marks 3

TECHNICAL PUBLICATIONg An up thrust for knowledge

System Sofwere 2-6 Overvlew of Language Processors

2,2 Language Processing Activities

Language processing activities are related to the specification gap and execution
gap. It is divided into two
types.
1. Program generation 2. Programn
execution
Aim of programn generation activity is to generate automatic program. In this
activity, a specification language of an application domain is the source language.
• A procedure oriented program language is
the target language. Source language is
a problem oriented
language.
1) Program generation activity
Program generator is a software system. Program specification
is input to this
system. It generates output in target program
larnguage. ig. Z2.1 Shows a
program generation activity.

User
program
Specification Software
system

Program Errors
generator
any

Required
PLfor
programs

Fig. 2.2.1 Program generation

Program generator domain is introduced by this program generator. Because

of
this new domain, the specification gap is the gap between the application
domain
and the program generator domain.
Fig. 2.2.2 shows program generator domain.

User Prograrm Target

generator programming Program
application execution
domain domain
language
domain domain

Specification
gap
Fig. 2.2.2 Program generator domain
ITECHNICAL PUBLICATIONS- An up thrust
for knowledae
 System Software 2-7 Overviow of Languago ProcOssors

It reduces the specification gap. Reliability of the generated programn is

increases.
It also helps the programmer for easily writing the specification of the program.
Software testing reduces the gap to the PL domain which is performed by the
generator.
Compiler is used to bridge gap between the target PL domain and the execution
domain.
2) Program execution
Following are the methods used for program execution.
1. Program translation 2. Program interpretation

1. Program translation : It bridges the execution gap by translating source program

into the target program. Source program is written in a programming language and a
target program is a assembly language. Fig. 2.2.3 shows the program translation.

User programs

if
Language Generates any
translator

Secondary
storage
Database device
Program
for
machine
language

Reguired
target program

Fig. 2.2.3 Program translation

Characteristics
1. Beforeexecution of program, it must be translated.
2. Translated program may be saved in a file.
3. Program must be retranslated with modifications.

TECHNICAL PUBLICATIONS - An up thrust for knowledge

System Softwere 2-8 Overview of Language Processors

2. Program interpretation :
It reads the source program and stores it in its main memory. During program
interpretation, it takes a source statement and determines its meanings. Then it
perform the actions which to be implement on that statement.
The action may be computational and input-output. Fig. 2.2.4 shows the
interpretation.
Main
Interpreter Memory

Program Source
COunter program Disk
data

Errors
(a) Interpretation
Main
Processor Memory

Program Machine
cOunter language
program Disk

data

(b) Program execution

Fig. 2.2.4, Interpretation and-program execution

Program counter increments the memory address for next instruction. CPU uses
the program counter for next instruction.
Instruction execution cycle consists of following steps :
1. Fetch the instruction

2. Decode the instruction.

3. Execute the instruction.
This cycle is repeated for all instructions. The instruction address in the program
counter is updated at the end of the cycle CPU select the next instruction for
execution.
The above process is called an interpretation cycle. Interpretation cycle consists of
following steps :

1. Fetch the statement' 2. Analyse the statement 3. Execute the statement.

TECHNICAL PUBLICATIONS - An up thrust for knowledge

System Sofware 2-9 Overview of Language Processors

Characteristics of Interpretation
1. Source program
is retained in the source fornm itself.
2. Statement is analysed
during its interpretation.

23 Fundamental of Language Processing

• Language processing is the combination of analysis
of SP and synthesis of TP
specification of the source program consists of three components. These are
1. Lexical rule 2. Syntax rule 3. Semantic rule
Lexical rule specify the information and formation of valid lexical units in the
sOurce language. Syntax rule specify the information and formation of valid
statements in the source language. Semantic rule is associated with the valid
statements of the language.
Analysis of source statement consists of lexical, syntax and semantic analysis.
Synthesis phase consists of two main activities.
1. Creation of data structures
in the target program.
2. Generation of target code.

Language processor phases and passes

Fig. 2.3.1 shows the phases of language processor. Language processor consists
of
two phases.
1.Analysis phase
2.Synthesis phase
Language processing can be performed on a each statement, i.e. analysis of a
source statement followed by synthesis of equivalent target statements.
The above process is not feasible because of 1) Forward reference and 2) Memory
requirements and organization of a language processor.
Forward reference
Forward reference of a program entity is a reference to the entity which precedes
its definition in the program. Using a variable before value is assigned to it is
called a forward reference.
Forward referenced varlables have the scope of the location where they are first
referenced. They can not be read before a value has been assigned to them, but
they can be used for further forward reference.
Language processor pass
It is the processing of every statement in a source program or its equivalent
representation to perform a language processing function.

PUBLICATIONS
TECHNICAL -An up thrust for knowledge
System Software Overview of Language Processors
2-10

Source program

Input

Language processor

Analysis Errors
phase may get generated

Synthesis
phase Errors
may get
generated

Output Target
program

Fig. 2.3.1 Language processor phase

Intermediate representation
It is a data structure that is constructed from input data to a program and from
turn.
which part or all of the output data of the program is constructed in

Propertles
1. Must be convenient for semantic analysis phase to produce.

2. Memory efficiency.
3. Must be convenient to translate into real assembly code for all desired target
machines.
4. Ease of use.

- An up thrust for knowledoe

TECHNICAL PUBLICATIONS
 System Sofiware
2-11 Overview of Language Processors
Fig. 2.3.2 shows the block diagram of IR.
Source program

Front end processing

Scanning) (Parsing Semantic
analysis

Intermediate
representation

Back end processing

Required
target
program

Fig. 2.3.2 IR

First pass performs analysis of the source program and reflects its results in the
intermediate representation. The second pas reads and analysis the IR.

2.4 Fundamentals of Language Specification GTU : Winter-18, 19

The compiler takes a source program written in high level language as an input
and converts it into a target assembly language. The assembler then takes this
assembly code as input and produces a relocatable machine code as output. Then
a program called loader is called for performing the task of loading and link
editing.
The task of loader is to perform the relocation of an object code. Relocation of an
object code means allocation of load time addresses which exist in the memory
and placement of load time addresses and data in memory at proper locations.
The link editor links the object modules and prepares a single module from
several files of relocatable object mnodules to resolve the mutual references. These
files may be library files and these library files may be referred by any program.
TECHNICAL PUBLICATIONS -
An up thrust for knowledge
System Software 2-12 Overview of Language Processors

Fig. 2.4.1 shows language processing

Source Program

Compiler

Target
assembly

Assembler

Relocatable machine
Code

Loader/Link-editor

Library, relocatable
object file

Executable machine
COde

Output
Executable Language
COde processing

Data
Fig: 2.4.1 Language processing
Example 24 Which type of gap makes the software buggy or unreliable ? Which nethods
Can be used to Overcome this situation ? GTUWinter-18, Marks 3

Solution : Specification and execution gap makes the software buggy or unreliable :
Specification Gap : It is the semantic gap between the two specifications of the
same task.
Execution Gap : The gap between the semantics of programs written in the
different programming languages.
Methods used to overcome this situation is as follows
:

a) Use of intermediate domain : Semantic domain is split into series of smaller

semantic gap between successive domains.
across domain.
b) System program Conversion to specification
:

TECHNICAL PUBLICATIONS -An up thrust for knowledge

System Software 2- 13 Overview of Language Processors

Review Question

1 Define following terms : 1) Semantic gap

2) Specification gap
3) Execution gap
GTU : Winter-19, Marks 3

25 Language Processor Development Tools GTUWinter-14, 17

When our program is compiled, initially the compiler takes
the input source
program and breaks it into stream of tokens. The
tokens can be keywords,
identifiers, relational operators and so on. These tokens are
grouped together by
the syntax analysis phase of compiler in order to detect the syntax of
programming Constructs. If from a group of tokens the syntax can not be
identified then a syntax error may get generated.
Creating a compiler is a critical task. Hence we can automate the process of
building compiler by using automated tools for generating various phases of
compiler. LEX and YACC are such tools. The LEX is used for generating the
lexical analyzer program for the compiler and YACC is used to generate the
syntax analyzer program automatically.
Basically LEX takes a set of description of tokens in the form of regular
expressions. If then produces the C program which is actually a lexical analyzer
program. The set of description in the form of regular expression is called LEX
specification file. Similarly for YACC the specification ile contains the context free
grammar which is used to analyse the syntax of tokens. The specification file
which when submitted to LEX or YACC generates a lexical analyzer or syntax
analyzer respectively.
The usage of LEX and YACCis as shown below.

Lex specification Lexical

Lex Executable
file analyzer compile program

Input Executable Sequence of

program file tokens

(a) Usage of LEX

TECHNICAL PUBLICATIONS- An up thrust for knowledge

System Softwaro 2-14 Overvie of Language Processorg

YACC specification Syntax C Executable

file YACC analyzer compile program

Input Executable Syntax tree

program file.

(b) Usage of YACC

Fig. 2.5.1 Usage of LEX and YCC

Thus the use of LEX and YACC tools help in generating the lexical and svmtay
analyzers which ultimately enhances efficiency of compiler building.

2.6 Symbol Tables Data Structures for Language Processing

Data structures are classified on the basis of the following criteria :
1. Nature of a data structure - Linear or nonlinear data structure.
2. Purpose of a data structure - Search or an allocation of data structure.
3. Lifetime of a data structure - Used during language processing or during target
program execution.
A linear data structure consists of a linear arrangement of elements in the
memory. It requires a contiguous area of memory for its elements. This leads to
wastage of memory.
The elements of a nonlinear data structure are accessed using pointers. The
elements need not occupy contiguous areas of memory. There is no wastage of
memory. But it leads to lower search efficiency.
Fig. 2.6.1 shows the linear and nonlinear data structures.

(a) Linear (b) Nonlinear

Fig. 2.6.1 Linear and nonlinear data structure
TECHNICAL PUBLICATIONS - An up thrust for
knowedge
 System Software 2-15 Overview of Lenguege Procassors

Search data structures are used during language processing to maintain

attríbute
information concerning different entities in the source program.
This type of data structures are characterized by using entry for an
entity is
created only once but may be searched for a large number of times. The important
point is the search efficiency.
Allocation data structures are characterized by the address of the memory area
allocated to an entity is known to the user of that entity. In this method, search
operations are not conducted. The important points are allocation or deallocation
speed and efficiency of memory utilization for this type of data structures.

2.6.1 Search Data Structures

• It is a set of entries, each entry accommodating
the information concerming one
entity. Each entry contains the key field and thís field is used for searching.
Entry formats
Set of fields are used in a search structure for each entry. Search structure consists
of two parts :
Fixed parts
b. Variant parts
Each part consists of a set of fields.
Compiler's symbol table have the following entries.
a. Fixed part : Fields symbol and class
b. Variant parts

Sr. No. Tag value Variant part fields

Variable Type, length, dimension information.

2. Label Statement number..

Procedure name Address of parameter ist, number of

parameters.
4. Type
Function name
of
returned value, length of returmed
value, address of parameter list.

Entry format
Entry format are of two types :
a. Fixed length
b. Variable length

TECHNICAL PUBLICATIONS- An up thrust for knowladge

 System Sofiware 2- 16 Overview of Language Processora

Entry is declared as a record or structures of the language.

Fixed length entry format
:
In this format, each record consists of the following fields
1. Fields in the fixed part of the entry.

2.Uvi sFre (set of fields in all variant part of entry).

All records in the search structures is an identical format. This enables the use of
homogeneous linear data structures like arrays.
• Benifit of using linear organizations enables the use of efficient search procedures.
Disadvantages of this type of organization is inefficient. use of memory. Most of
the records may contain redundant fields.
Variable length entry format
Record consists of following fields :

1. Fields in the fixed part of the entry, including tag field.

2. {f |
fe SF if tag = vj)
• Nomemory wastage in this organization.
Hybrid entry formats
Hybrid entry format is used to combine the access efficiency of the fixed entry
format and memory efficiency of the variable entry format.
In this format each entry is split into two halyes : fixed part and variable part.
A pointer field is used in fixed parts and it points to the variable part of the entry.
Fig. 2.6.2 shows the hybrid entry format.

Fixed part Pointer Length Variable part

Fig. 2.6.2 Hybrid entry format

Search structure operations

Operation to be performed on search data structures are
:

1. add : Add the entry of a symbol.

2. search : Search and lbcate the entry of a symbol.

3. delete : Delete the entry of a symbol.

Table organizations
A table is a linear data structure. The entries of a table occupy adjoining areas of
the memory.

TECHNICAL PUBLICATIONS - An up
thrust for knowledge
System Sofware 2-17 Overviow of Languago ProOco3SOrs

Tables using the fixed length entry

#1
organization posses the property of
#2
positional determinacy.
#3
Tables using variable length entries do Occupied
not possess the property of positional entries
determinacy. Variable length entries
are generally avoided in linear data
structures. #f
Fixed length entries are used in linear
data structures. Free entries
Fig. 2.6.3 shows the sequential search #n
organization.
Fig. 2.6.3 Sequential search
Symbols used in this search are :
n = Number of entries in the table
f = Number of occupied entries
Operations
1. Add a symbol: Symbol is added to the first free entry in the table. The value of 'f
is updated accordingly.
2. Delete a synbol : Deletion can be done two ways :
a. Physical deletion
b. Logical deletion
• In physical deletion, an entry is deleted by erasing or by overwriting..
Logical deletion of an entry is performed by adding some information to the entry
to indicate its deletion.

2.6.2 Allocation Data Structures

We study the stack and heap in this section.

2.6.2.1 Stacks
Stack is a linear data structure.
Properties
1. Stack is an unbounded array that is treated in a last-in, first-out (LIFO) manner.
The last element stored is the first one removed.
2.Only the last entry is accessible at any time.
• A Stack Pointer (SP) indicates the position or frame at the top of the stack.
TECHNICAL PUBLICATIONS- An up thrust for knowledge
 2- 18 Overview of Language
System Sofiware PrOcessors

- area.
Stack Base (SB) It points to the first word of the stack
Top of Stack (TOS) - It points to the last entry allocated in the stack.

Fig. 2.6.4 shows the stack model.

TOS –

SB 100 SB 100 SB 100 SB

200 200 200

300 300 300

400 400 400

500 500 TOS 500
TOS 600 600

TOS 700

Fig. 2.6.4 Stack model

When an entry is pushed on the stack, TOS is incremented
by 1. When an entry is
popped, it is decremented by 1.
Extended stack model
Apart from SB and TOS, record base pointer (RB)
and reserved pointer are used in
extended stack model.
Record base pointer pointing to the first word of the last record
in stack.
Reserved pointer is the first word of each record.
It is used for housekeeping
Purposes.
Fig. 2.6.5 shows the allocation and deallocation time
actions in the extended stack
model. (See Fig. 2.6.5 on next page)
2.6.2.2 Heap

Heap allows the allocation and deallocation of entries in a

random order. The
heap data structure does not provide any' specific means to access an allocated
entity.

TECHNICAL PUBLICATIONS- An up thrust for knowledge

 Softwere
System 2- 19 Overview of Language Processors

SB

SB

RB3

Record RB

TOS

TOS

(a) Extended stack model (b)

SB, RB

TOS

(c)
Fig. 2.6.5 Allocation and deallocation time actions

Review Question

1. Explain memory allocation in block structured language. GTU': Winter-17, Marks 3

27 Multiple Choice Questions

Q.1 A islanguage processor which bridges an execution gap but is
a not a
language translator.
a
detranslator b language migratory
c preprocessor fd interpreter

Q.2 An interpreter is a language processor which bridges gap without generating

a machine language program.
b specification
execution
cexecution and specification ja specification and design
source language.
Q.3
rules which govern the formation of valid statements in the
a b Syntax
Semantic

TECHNICAL PUBLICATIONS- An up thrust for knowledge

System Software 2-21 Overview of Language Processors

2.81 Short Questions and Answers

Q.1 Define language processor GTU: Summer17, Mark I
Ans. : Language processor is a software and it links a specification gap or execution gap.

Q.2 Define semantic and semantic gap. GTU Summer-17, Mark I

Ans. :
Semantic : It represent the rules of meaning of a domain.
Semantic gap : It represent the difference between the semantics of two domains.
Q.3 Define specification gap.

Ans. Specification gap is between application domain and PL domain. The specification
:
cap is bridged by the software development team and execution gap by the designer of
the PL processor using translator or an interpreter.

Q,4 Define execution gap.

Ans. : Execution gap is between PL domain and execution domain. The execution gap
is bridged by the designer of the PL processor using translator or an interpreter.

Q.5 What is language ?

processing
Ans. : Language processing is the combination of analysis of SP and synthesis of TP
Specification of the source program consists of three components. These are lexical rule,
syntax rule and semantic rule
Q,6 Define forward reference.
Ans. : Forward reference of a program entity is a reference to the entity which precedes
its definition in the program. Using a variable before a value is assigned to it is called
a forward reference.
a.7 What is language processor pass, ?
Ans. : It is the processing of every statement in a source program or its equivalent
representation to perform a language processing function.

TECHNICAL PUBLICATIONS An up thrust for knowledge

-