Syntactic Analysis

Introduction
I Second phase of the compiler.
I Main task:
I Analyze syntactic structure of program and its components
I to check these for errors.
I Role of parser:
token
source Lexical parse IR
Parser Rest of
Analyzer Front end
tree
req

Symbol
Table

I Approach to constructing parser: similar to lexical analyzer

I Represent source language by a meta-language, Context Free Grammar
I Use algorithms to construct a recognizer that recognizes strings generated by the
grammar.
This step can be automated for certain classes of grammars. One such tool:
YACC.
I Parse strings of language using the recognizer.

1/1
Context Free Grammar (CFG)
I Syntax analysis based on theory of automata and formal languages, specifically
the equivalence of two mechanisms of context free grammars and pushdown
automata.
I Context free grammars used to describe the syntactic structures of programs of
a programming language. Describe what elementary constructs there are and
how composite constructs can be built from other constructs.
Stmt → if (Expr) Stmt else Stmt
Note recursive nature of definition.
I Formally, a CFG has four components:
a) a set of tokens Vt , called terminal symbols, (token set produced by the scanner)
examples: if, then, identifier, etc.
b) a set of different intermediate symbols, called non-terminals, syntactic categories,
syntactic variables, Vn
c) a start symbol, S ∈ Vn , and
d) a set of productions P of the form
A → X1 · · · Xn
where A ∈ Vn , Xi ∈ (Vn ∪ Vt ), 1 ≤ i ≤ m, m ≥ 0.
I Sentences generated by starting with S and applying productions until left with
nothing but terminals.
I Set of strings derivable from a CFG G comprises the context free language,
denoted L(G ).

2/1
CFG - example.
I Nonterminal start with uppercase letters. rest are non-terminals.
I If-then-else:
Stmt → IfStmt | other
IfStmt → if ( Exp ) Stmt ElseStmt
ElseStmt → else Stmt | 
Exp → 0 | 1
Example strings:
other
if (0) other
if (1) other else if (0) other else other
Derivation of if (1) other else if (0) other else other:
Stmt ⇒ IfStmt ⇒ if (Exp) Stmt ElseStmt
⇒ if (1) Stmt ElseStmt
...
I Grammar for sequence of statements:
StmtSeq → Stmt; StmtSeq | Stmt
Stmt → s
L(G) = { s, s;s, s;s;s, ... }
I What if statment sequence is empty?
StmtSeq → Stmt; StmtSeq | 
Stmt → s
L(G) = { , s;, s;s;, s;s;s;, ... }
Note: Here ’;’ is not a statement separator, but a terminator.
What if we want a statement separator?
StmtSeq → NonEmpStmtSeq | 
NonEmpStmtSeq → Stmt; NonEmpStmtSeq | Stmt
Stmt → s

3/1
Context Free Grammar (CFG) - cont’d.
I Notations:
1. Nonterminals: Uppercase letters such as A, B, C
2. Terminals: lower case letters such as a,b, c, operators +,−, etc,
punctuation, digits, and boldface strings such as id.
3. Nonterminals or terminals: Upper-case letters late in alphabet, such
as X , Y , Z .
4. Strings of terminals: lower-case letters late in alphabet, such as x, y ,
z.
5. Strings of grammar symbols: lower-case greek letters α, β, etc.
6. Write A → α1 , A → α2 , etc as
A → α1 |α2 | · · ·
I Example:
E → E A E | ( E ) | − E | id
A → +| − | ∗ |/| ↑
I Derivation of strings: a production can be thought of as a rewrite
rule in which nonterminal on left is replaced by string on right side.
Notation: Write such a replacement as E ⇒ (E).
Example:
E ⇒ −E ⇒ −(E ) ⇒ −(id)

4/1
CFG - cont’d.
I Notation: Write αAβ ⇒ αγβ if A → γ.
∗
I Notation: Write α ⇒ β to denote that β can be derived from α in zero or
more steps.
∗
L(G ) = {α| S ⇒ α}
∗
I Sentential form: α is a sentential form, if S ⇒ α and α contains
non-terminals.
Example: E + E
I Leftmost derivation: Derivation α ⇒ β is leftmost if the leftmost terminal in
α is replaced.
Example:
∗ ∗ ∗ ∗
E ⇒ EAE ⇒ idAE ⇒ id + E ⇒ id + id
Production sequence discovered by a large class of parsers (the top-down
parsers) is a leftmost derivation; hence, these parsers are said to produce
leftmost parse.
I Rightmost derivation: Derivation α ⇒ β is left most if the rightmost terminal
in α is replaced.
Example:
∗ ∗ ∗ ∗
E ⇒ EAE ⇒ EAid ⇒ E + id ⇒ id + id
Also, called canonical derivation. Corresponds well to an important class of
parsers (the bottom-up parsers). In particular, as a bottom up parser discovers
the productions used to derive a token sequence, it discovers a rightmost
derivation, but in reverse order : last production applied is discovered first,
while the first production is the last to be discovered.
5/1
Representations of derivations
I Derivations represented graphically by a derivation of parse tree:
I Root: start symbol, leaves: grammar symbols or 
I Interior nodes: nonterminals; Offsprings of a nonterminal represent application of
a rule.
I Example: Parse tree for leftmost and rightmost derivations of string id + id ∗ id:
E E

E + E E * E

id E * E E + E id

id id id id
I Abstract syntax tree: A more abstract representation of the input string.
Stmt if
if ( exp ) Stmt else Stmt
0 other other

0 Other Other
I Parse tree may contain information that may not be needed in later phases of
compiler. AST does not include intermediate nodes primary used for derivation
purposes.
I In general, during the semantic analysis phase, the parse tree of a string may
be converted into an abstract syntax tree.

6/1
Parse Tree - Examples
I Parse tree for string: if (o) other else other
if
Stmt

0 other other

IfStmt

if ( exp ) Stmt ElseStmt

0 other else Stmt

other

I Parse tree for string: s;s;s

StmtSeq seq

Stmt ; StmtSeq s s s

Stmt ; StmtSeq

s
s
s

7/1
Properties of Context Free Grammars
I Context free grammars that are limited to productions of the form A → a
B and C →  form the class of regular grammars. Languages defined by
regular grammars are a proper subset of the context-free languages.
I Why not use lexical analysis during parsing?
I Lexical rules are in general simple.
I RE are more concise and easier to understand.
I Domain specific language so that efficient lexical analyzer can be
constructed.
I Separate into two manageable parts. Useful for multi-lingual programming.
I Non-reduced CFGs: A CFG containing nonterminals that are unreachable
or derive no terminal string.
Example:
S → A|B
A → a
B → B b
C → c
Nonterminal C cannot be reached from S. B does not derive any strings.
Useless terminals can be safely removed from a CFG without affecting the
language. Reduced grammar:
S → A
A → a
Algorithms exist that check for useless nonterminals.

8/1
Properties of Context Free Grammars - Ambiguity
I Ambiguity : A context free grammar is ambiguous if it allows different
derivation trees for a single tree.
E E
E − E E − E
id E − E E − E id
id id id id

Each tree defines a different semantics for −

I No algorithm exists for automatically checking if a grammar is ambiguous
(impossibility result). However, for certain grammar classes (including
those that generate parsers), one can prove that grammars are
unambiguous.
I How to eliminate ambiguity: one way is to rewrite the grammar: Example:
S → if E then S | if E then S else S

S → M|U
M → if E then M else M
U → if E then S | if E then M else U
Represents semantics:Match each else with the closet previous unmatched
then. The above transformation makes the grammar unnecessarily
complex.
I Another approach: Disambiguate by defining additional tokens end.
S → if E then S end | if E then S else S end
I Provide information to the parser so that it can handle it in a certain way.
9/1
Properties of Context Free Grammars - cont’d.
I Left recursion: G is left recursive if for a nonterminal A, there is a
+
derivation A ⇒ Aα
Top-down parsing methods cannot handle left-recursive grammars. So
eliminate left recursion.
I Left factoring : Factor out the common left prefixes of grammars: Replace
grammar A → αβ1 |αβ2 by the rule:
A → αA0
A0 → β1 |β2
I Context free grammars are not powerful enough to represent all constructs
of programming languages.
Cannot distinguish the following:
I L1 = {wcw |w ∈ (a|b)∗ }: Conceptually represents problem of verifying that
an identifier is declared before used. Such checkings are done during the
semantic analysis phase.
I L2 = {an b m c n c m |n ≥ 1 ∧ m ≥ 1}. Abstracts the problem of checking that
number of formal parameters agrees with the number of actual parameters.
I L3 = {an b n c n |n ≥ 0}.
CFG’s can keep count of two items but not three.

10 / 1
Properties of Context Free Grammars - cont’d.
I Context free grammar can capture some of language semantics as
well.
I Example grammar:
<exp> ::=<exp> + <term> | <term>
<term> ::=<term> * <term>
| ‘(’<exp>‘)’
| <number>
<number> ::= 0 | 1 | · · · | 9
I Precedence of * over +: by deriving * lower in the parse tree.
I Left recursion
<exp> ::= <exp> + <term>
left associativity of +
I Right recursion:
<exp> ::= <term> + <exp>
right associativity of +

11 / 1
Backus-Naur Form(BNF)
I BNF: a kind of CFG.
I First used in Algol60 report. Many extensions since, but all similar and most
give power of context-free grammar.
I Has four parts: (i) terminals (atomic symbols), (ii) non-terminals (representing
constructs), called syntactic categories, iii) productions and iv) a starting
nonterminal.
I Each nonterminal denotes a set of strings. Set of strings associated with
starting nonterminal represents language.
I BNF uses following notations:
(i) Non-terminals enclosed in < and >.
(ii) Rules written as
X ::= Y
(a) X is LHS of rule and can only be a NT.
(b) Y can be a string, which is a terminal, nonterminal, or concatenation of terminal
and nonterminals, or a set of strings separated by alternation symbol |.
I Example: Terminals: A, B, · · · Z; 0, 1, · · · 9
Nonterminals: <id>, <rest>, <alpha>, <alphanum>, <digit>
Starting NT: <id>
Productions/rules:
<id> ::= <alpha> | <alpha><rest>
<rest> ::= <rest><alphanum> | <alphanum>
<alphanum> ::= <alpha> | <digit>
<alpha> ::= A | B | ··· | Z
<digit> ::= 0 | 1 | ··· | 9
12 / 1
Extended BNF (EBNF)
I Extend BNF by adding more meta-notation =⇒ shorter productions
I Nonterminals begin with uppercase letters (discard <>)
I Terminals that are grammar symbols (’[’ for instance) are enclosed in ‘’.
I Repetitions (zero or more) are enclosed in {}
I Options are enclosed in []:
I Use () to group items together:
Exp ::= Item {+ Item} | Item {- Item}
=⇒
Exp ::= Item {(+|-) Item}
Conversion from EBNF to BNF and Vice Versa
I BNF to EBNF:
i) Look for recursion in grammar:
A ::= a A | B =⇒ { a } B
ii) Look for common string that can be factored out with grouping and options.
A ::= a B | a =⇒ A := a [B]
I EBNF to BNF:
i) Options []:
A ::= a [B] C =⇒
A’ ::= a N C
N ::= B | 
ii) Repetition {}:
A ::= a B1 B2 ... Bn C =⇒
A’ ::= a N C
N ::= B1 B2 ... Bn N | 
13 / 1