Introduction To Shift Reduce Parsing
Introduction To Shift Reduce Parsing
Orientation
Specify Syntax Using Context-Free Grammar Nonterminals Terminals Productions Given a grammar, Parser Generator produces a parser Starts with input string Produces parse tree
Expr p Expr Op Expr Expr p (Expr) Expr p
- Expr
Expr p num Op p + Op p Op p *
Today s Lecture
How generated parser works How parser generator produces parser Central mechanism Pushdown automaton, which implements Shift-reduce parser
Pushdown Automata
Consists of Pushdown stack (can have terminals and nonterminals) Finite state automaton control Can do one of three actions (based on state and input): Shift: Shift current input symbol from input onto stack Reduce: If symbols on top of stack match right hand side of some grammar production NT p F Pop symbols (F) off of the stack Push left hand side nonterminal (NT) onto stack Accept the input string
Input String
num * ( num + num )
num
num
num
SHIFT
num
num
num
num
SHIFT
num
num
num
REDUCE
num
num
Expr
REDUCE
Expr
SHIFT
* Expr
SHIFT
Op Expr
REDUCE
num
* ( num + num )
Op Expr
SHIFT
num
* ( num + num )
( Op Expr
SHIFT
num
* num + num )
( Op Expr
SHIFT
num
* num + num )
SHIFT
num
* + num )
REDUCE SHIFT
num
num + num )
SHIFT
num
num + num )
SHIFT
num
num num )
REDUCE SHIFT
num
num
+ num )
SHIFT
num
num
+ num )
SHIFT
num
num
+ )
REDUCE SHIFT
num
num
num )
REDUCE SHIFT
num
num
num )
SHIFT
num
num
num )
SHIFT
num
num
num
REDUCE
num
num
num
) Expr (
REDUCE
Op Expr
num
num
num
) Expr (
ACCEPT!
Op Expr
num
num
num
Basic Idea
Goal: reconstruct parse tree for input string Read input from left to right Build tree in a bottom-up fashion Use stack to hold pending sequences of terminals and nonterminals
Potential Conflicts
Reduce/Reduce Conflict Top of the stack may match RHS of multiple productions Which production to use in the reduction? Shift/Reduce Conflict Stack may match RHS of production But that may not be the right match May need to shift an input and later find a different reduction
Conflicts
Original Grammar
Expr p Expr Op Expr Expr p (Expr) Expr p
New Grammar
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
- Expr
Expr p num Op p + Op p Op p *
Conflicts
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
num
num
Conflicts
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
SHIFT
num
num
Conflicts
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
num
SHIFT
num
Conflicts
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
REDUCE SHIFT
num num
Conflicts
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
SHIFT
num num
Conflicts
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
SHIFT
num num
Shift/Reduce/Reduce Conflict
Options:
Reduce Reduce Shift
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
num num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
REDUCE
num num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr Expr
SHIFT
num
num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
SHIFT
num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr Expr
REDUCE
Expr
num
num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
FAILS!
num
num
Shift/Reduce/Reduce Conflict
Both of These Actions Work
Expr
Reduce Shift
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
num num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
num num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Op Expr
REDUCE
num
num
Shift/Reduce/Reduce Conflict
What Happens if Choose
num Op Expr
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
SHIFT
num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Expr Op Expr
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
REDUCE
num
num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
REDUCE
num
num
Shift/Reduce/Reduce Conflict
What Happens if Choose
Reduce
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
ACCEPT
num
num
Conflicts
What Happens if Choose
Shift
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
SHIFT
num num
Conflicts
What Happens if Choose
num Expr
Shift
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
SHIFT
num
Conflicts
What Happens if Choose
Expr Expr
Shift
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
REDUCE
num
num
Conflicts
What Happens if Choose
Shift
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr Expr
REDUCE
Conflicts
What Happens if Choose
Shift
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr Expr
ACCEPT
Shift/Reduce/Reduce Conflict
This Shift/Reduce Conflict Reflects Ambiguity in Grammar
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
num num
Shift/Reduce/Reduce Conflict
This Shift/Reduce Conflict Reflects Ambiguity in Grammar
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
num num
Shift/Reduce/Reduce Conflict
This Shift/Reduce Conflict Can Be Eliminated By Lookahead of One Symbol
Expr p Expr Op Expr Expr p Expr - Expr Expr p (Expr) Expr p Expr Expr p num Op p + Op p Op p *
Expr
num num
Constructing a Parser
We will construct version with no lookahead Key Decisions Shift or Reduce Which Production to Reduce Basic Idea Build a DFA to control shift and reduce actions In effect, convert grammar to pushdown automaton Encode finite state control in parse table
Parser State
Input Token Sequence ($ for end of input) Current State from Finite State Automaton Two Stacks State Stack (implements finite state automaton) Symbol Stack (terminals from input and nonterminals from reductions)
Parse Tables
State s0 s1 s2 s3 s4 s5 ( shift to s2 error shift to s2 error reduce (2) reduce (3) ACTION ) error error shift to s5 shift to s4 reduce (2) reduce (3) Goto $ error accept error error reduce (2) reduce (3) X goto s1 goto s3
Implements finite state control At each step, look up Table[top of state stack] [ input symbol] Then carry out the action
Input (())
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s0
Parser Tables
State s0 s1 s2 s3 s4 s5 ( shift to s2 error shift to s2 error reduce (2) reduce (3) ACTION ) error error shift to s5 shift to s4 reduce (2) reduce (3) Goto $ error accept error error reduce (2) reduce (3) X goto s1 goto s3
Shift to sn Push input token into the symbol stack Push sn into state stack Advance to next input symbol
Parser Tables
State s0 s1 s2 s3 s4 s5 ( shift to s2 error shift to s2 error reduce (2) reduce (3) ACTION ) error error shift to s5 shift to s4 reduce (2) reduce (3) Goto $ error accept error error reduce (2) reduce (3) X goto s1 goto s3
Reduce (n) Pop both stacks as many times as the number of symbols on the RHS of rule n Push LHS of rule n into symbol stack
Parser Tables
State s0 s1 s2 s3 s4 s5 ( shift to s2 error shift to s2 error reduce (2) reduce (3) ACTION ) error error shift to s5 shift to s4 reduce (2) reduce (3) Goto $ error accept error error reduce (2) reduce (3) X goto s1 goto s3
Reduce (n) (continued) Look up Table[top of the state stack][top of symbol stack] Push that state (in goto part of table) onto state stack
Parser Tables
State s0 s1 s2 s3 s4 s5 ( shift to s2 error shift to s2 error reduce (2) reduce (3) ACTION ) error error shift to s5 shift to s4 reduce (2) reduce (3) Goto $ error accept error error reduce (2) reduce (3) X goto s1 goto s3
Input (())$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s0
Input (())$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s0
Input ())$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s2 s0
Input ())$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s2 s0
Input ))$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
( (
Input ))$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
( (
Input )$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
) ( (
Input )$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
) ( (
Input )$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
) ( (
Input )$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s2 s0
Input )$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s2 s0
Input )$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s2 s0
X (
Input )$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s2 s0
X (
Input )$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
X (
Input )$
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
X (
Input $
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
) X (
Input $
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
) X (
Input $
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
) X (
Input $
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s0
Input $
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s0
Input $
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s0
Input $
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s0
Input $
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s1 s0
Input $
Grammar
S p X $ (1) X p (X ) (2) X p ( ) (3)
s1 s0
Key Concepts
Pushdown automaton for parsing Stack, Finite state control Parse actions: shift, reduce, accept Parse table for controlling parser actions Indexed by parser state and input symbol Entries specify action and next state Use state stack to help control Parse tree construction Reads input from left to right Bottom-up construction of parse tree