2/18/2008

1
10.1 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Objective:
Learn how to parse and evaluate an expression.
In other words:
How does the computer evaluate
the following expression?
5*2+3*2+50/10
10.2 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
To parse a sentence in a formal language is to break it down
into its syntactic components.
Parsing is one of the most basic functions every compiler
carries out.
Production of a correct machine language
representation
2/18/2008
2
10.3 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Computer languages are represented by formal grammars.
These grammars are usually represented by another formally
defined language, such as Backus-Naur Form (or BNF), or
syntax diagrams.
Heres an example of BNF. It defines what a product is.
<product> ::= <number> | <product> * <number>
This says that a product is either a number or an another
product followed by * followed by a number. Note that its a
recursive definition.
The | symbol is part of BNF, and it means or. (The *
symbol is not part of BNF, but is part of the language being
defined.)
10.4 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
<expression> ::= <term> |
<expression> + <term> |
<expression> <term>
<term> ::= <factor> |
<term> * <factor> |
<term> / <factor>
<factor> ::= <number> | ( <expression> )
The following BNF definitions describe the syntax of an
arithmetic expression.
2/18/2008
3
10.5 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
From the preceding definitions, it should be clear that
the following are all syntactically correct expressions:
5
5 + 8
(5 + 8)
5 * 2 + 1
5 * (2 + 1)
while the following are not:
+
5 +
((5 + 8)
5 * 2 + + 1
5 * 2 + 1
expression
5 * 2 + 1
expression term
5 * 2 + 1 term
factor -> number
5 * 2 + 1 term
factor
5 * 2 + 1
factor
factor
10.6 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Heres the BNF for an expression again.
<expression> ::= <term> |
<expression> + <term> |
<expression> <term>
This definition is left-recursive the recursive part is the leftmost
item on the right-hand-side on the definition.
Heres a right-recursive version which defines the same thing.
<expression> ::= <term> |
<term> + <expression> |
<term> <expression>
Do they evaluate an expression in the same way?
5+3-2+1
8
6
7
5+3-2+1
3
0
5
We will use iterative processing.
5+3-2+1
2/18/2008
4
10.7 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Before we parse a sentence in a formal language, its
convenient to break it down into a list of lexemes.
A lexeme (sometimes also called token) is the smallest
syntactic unit in a language. The following are typical kinds of
lexemes:
Numbers (i.e. numeric strings)
Names (i.e. alphanumeric strings)
Operators (including brackets) (e.g. *, +, ^, etc)
For example, the sentence (64 + 7) * 128 is broken down
into a list of seven lexemes: (, 64, +, 7, ) , *, and
128.
10.8 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
In what follows, well assume that the expression is a string
pointed to by a pointer expression.
Well also assume that we have available the following
access functions:
Dont worry about the data type string and how these access
functions are written for now. We will deal with them later.
2/18/2008
5
10.9 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Grammar
definition
using BNF
Evaluate an
Expression
Evaluate a
term
Evaluate a
factor
Access
routines
Example
10.10 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Heres the BNF again.
<expression> ::= <term> |
<expression> + <term> |
<expression> <term>
To evaluate an expression E iteratively, this is what you do.
Evaluate the next term in E, and call the result X.
While the next character in E is an operator (+ or ),
Read past the operator.
Evaluate the next term in E, calling the result Y.
Let X be X + Y or X Y, depending on the operator.
The value of E is the final value of X.
2/18/2008
6
10.11 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
E1 + E2 E3 + E4
Y X +
X Y -
X Y +
X
To evaluate an expression E iteratively, this is what you do.
Evaluate the next term in E, and call the result X.
While the next character in E is an operator (+ or ),
Read past the operator.
Evaluate the next term in E, calling the result Y.
Let X be X + Y or X Y, depending on the operator.
The value of E is the final value of X.
10.12 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Heres the C++ code. It should be self-explanatory.
2/18/2008
7
10.13 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Evaluating a term is almost identical. Heres the BNF.
<term> ::= <factor> |
<term> * <factor> |
<term> / <factor>
To evaluate a term T, this is what you do.
Evaluate the next factor in T, and call the result X.
While the next character in T is an operator (* or /),
Read past the operator.
Evaluate the next factor in T, calling the result Y.
Let X be X * Y or X / Y, depending on the operator.
The value of T is the final value of X.
10.14 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
And heres the C++ code.
2/18/2008
8
10.15 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Evaluating a factor is a little different, and involves a
recursive call. Heres the BNF again.
<factor> ::= <number> | ( <expression> )
To evaluate a factor F, this is what you do.
If the next character in F isnt ( then let X be the first
number in F.
Otherwise evaluate the next expression in F, and call the
result X. Then read past the ).
The value of F is X.
10.16 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Heres the C++ code.
2/18/2008
9
10.17 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Note that evalExpression(), evalTerm() and evalFactor() are
mutually recursive. That is:
evalExpression()---calls---evalTerm()---calls---evalFactor()
calls
We need to use function prototype to specify functions
arguments before they are used. Here are the function
prototypes for all three functions:
10.18 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Some C++ compilers understand a data type called string.
Here are some examples of access functions for string data
type.
2/18/2008
10
10.19 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Now we can write our access routines for this program:
10.20 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
2/18/2008
11
10.21 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
10.22 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
7 * ( 5 + 4 )
expression
2/18/2008
12
10.23 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
7 * ( 5 + 4 )
expression
10.24 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
7 * ( 5 + 4 )
Returns result = 7
expression string = *(5+4) expression
2/18/2008
13
10.25 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
( 5 + 4 )
result = 7
op = *
expression
10.26 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
5 + 4 )
expression nextChar() = (
2/18/2008
14
10.27 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
5 + 4 )
expression nextChar() = (
10.28 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
5 + 4 )
expression
Result = 5
2/18/2008
15
10.29 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
4 )
expression Returns temp = 4
op = +
result = 9
10.30 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
)
expression result = 9
2/18/2008
16
10.31 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
expression temp = 9
result = 7
10.32 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
expression
final result = 63
2/18/2008
17
10.33 Lecture 10 Parsing
Feb 2008 EE2/ISE1 Algorithms & Data Structures
Grammar
definition
using BNF
Evaluate an
Expression
Evaluate a
term
Evaluate a
factor
Access
routines
Example