Bottom-up Parsing Part II

Handles and Viable Prefixes

Arles Rodríguez
[email protected]

Facultad de Ciencias
Departamento de Matemáticas
Universidad Nacional de Colombia
 Handles
• Bottom-up parsing performs two actions:

Shift

Reduce
 Handles
• How do we decide when to shift or reduce?
• Given the grammar:

• And the step: int|*int+int

– We could decide to reduce by
T|*int+int
– This is a big mistake, why?
 Handles
• How do we decide when to shift or reduce?
• Given the grammar:

• And the step: int|*int+int

– We could decide to reduce by
T|*int+int
– This is a big mistake, because there is no
production that begins with T *, we got stuck.
 Handle intuition
• Want to reduce only if the result can still be
reduced to the start symbol.
• Assume a rightmost derivation:

Reduction goes that way

• We can say that is a handle of
– We can reduce to and get back to S.
 Handles formalize the intuition

• A handle is a reduction that also allows

further reductions back to the start symbol.
• We only want to reduce at handles.
• We need to know how to find handles.
Exercise
 Lessons Learned
• A valid handle is a full right side of a
production at the top of the stack, never
inside.
• A handle is in a context that can be further
back reduced back to the start symbol.
• Only one reduction can be done at a time.
 Handles
• Informal proof that handles
appear at the top of the stack:
– True initially, stack is empty
– Immediately after reduce a handle:
• Right-most non-terminal is on the top
of the stack. X|
• Next handle must be on the right side
of right-most non-terminal, because
Next handle
this is a right-most derivation.
• Sequence of shift moves reaches next
handle.
 Handles
• In shift-reduce parsing, handles appear at the
top of the stack.
• Handles are never to the left of the rightmost
terminal.
– Therefore, shift reduce moves are sufficient so
the | never need to move left.
• Bottom-up parsing algorithms are based on
recognizing handles.
 Recognizing handles
• Bad news:
– There are no known efficient algorithms for
handle detection.
• Good news:
– There are good heuristics for guessing handles.
– For some CFGs, the heuristics always guess
correctly.
Recognizing handles
All CFGs

Unambiguous CFGs

LR(k) CFGs

LALR(k) CFGs
SLR(k)
CFGs
 Recognizing handles: viable prefixes
• It is not obvious how to detect handles.
• At each step the parser sees only the stack,
not the entire input
– We start with information we can get from the
stack.
• Definition: is a viable prefix if there is an
such that | is a state of a shift-reduce parser.
 Recognizing handles: viable prefix
• Definition: is a viable prefix if there is an such
that | is a state of a shift-reduce parser.
– A viable prefix does not extend past the right end
of the handle.
– It is a viable prefix because is a prefix of the
handle.
– If the parser has viable prefixes on the stack no
parsing error has been detected.
 Recognizing handles: viable prefixes

For any grammar, the set of viable prefixes is a

regular language.

• We can define an automata that accept viable

prefixes.
• We need the item definition to compute
automata that accept viable prefixes.
 Recognizing handles: item
• Definition: an item is a production with a “.”
somewhere on the rhs.
• The items for are:

• The only item from is

– Those are known as LR(0) items.
 Recognizing handles: prefixes
• The problem in recognizing viable prefixes is
that the stack has only bits and pieces of the
rhs of productions:
– If it had a complete rhs, we could reduce
• These bits and pieces are always prefixes of
rhs of productions.
 Recognizing handles: example
• Consider the grammar:

• And the input: (int)

 Recognizing handles: example
• Consider the grammar:

• And the input: (int)

• Then is a state of shift-reduce parse
• is a prefix of the rhs of
– Will be reduced after the next shift
• Item , says that so far we have seen (E of this
production and hope to see )
 Structure of the stack
• The stack may have many prefixes of rhs’s:

• Let be a prefix of rhs of

– will eventually reduce to
– The missing suffix of starts with
• there is a for some
• Recursively, eventually reduces to the missing
part of
 Structure of the stack
• Consider the grammar:

• And the string (int*int) where (int*|int) is a state

of a shift reduce parse

– “” is a prefix of the rhs of

• Starting from the top of stack:

– “” is a prefix of the rhs of

– “(” is a prefix of the rhs of
Recognizing viable prefixes: stack of items

• And the string (int*int) where (int*|int) is a

– “” is a prefix of the rhs of

state of a shift reduce parse:

– “” is a prefix of the rhs of

– “(” is a prefix of the rhs of
• Corresponds to the “stack of items”:
 Recognizing viable prefixes: idea

To recognize viable prefixes, we must recognize

a sequence of partial rhs’s of productions where
each partial rhs can eventually reduce to part of
the missing suffix of its predecesor.
 Recognizing viable prefixes: algorithm

1. Add a dummy production to grammar G

2. We are going to build a NFA that recognize
the viable prefixes:
• The NFA states are the items of the grammar G
• Input of NFA is the stack
• NFA(stack) = yes or not
• We read the stack from bottom to top
 Recognizing viable prefixes: algorithm

3. For item and input X (X can be terminal or

non-terminal) add transition to extend a prefix:

4. For item and production (X is non-terminal)

add:

(try to discover where end of prefixes are)

 Recognizing viable prefixes: algorithm

5. Every state in the NFA is going to be an

accepting state if the NFA consume the entire
stack.
– If gets stuck reject.
6. Start state is
 Recognizing viable prefixes: example
• Given the grammar:

1. Add a dummy production to grammar G

Recognizing viable prefixes: example
Recognizing viable prefixes: example

′
𝑆 →.𝐸
Recognizing viable prefixes: example

This is probably the final

′
𝑆 →𝐸. production we reach at the end of
bottom-up parsing
E
′
𝑆 →.𝐸
Recognizing viable prefixes: example

′
𝑆 →𝐸.
E
′
𝑆 →.𝐸
𝜀
𝐸 →.𝑇
Recognizing viable prefixes: example
• As a NFA we cannot worry about
which production to use.

• We can compile NFA into DFA

′
𝑆 →𝐸.
E 𝜀 𝐸→.𝑇+𝐸
′
𝑆 →.𝐸
𝜀
𝐸 →.𝑇
Recognizing viable prefixes: example

′
𝑆 →𝐸.
E 𝜀 𝐸→.𝑇+𝐸
′
𝑆 →.𝐸
𝜀
• When we reach a state where the
𝐸 →.𝑇 dot is all the way to the right
hand side this could be a handle
𝑇 that you might want to reduce.
𝐸 →𝑇 .
Recognizing viable prefixes: example

′
𝑆 →𝐸.
E 𝜀 𝐸→.𝑇+𝐸
′
𝑆 →.𝐸
𝜀
𝜀 𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example

𝑇 →.(𝐸)
′
𝑆 →𝐸.
𝜀 E 𝜀 𝐸→.𝑇+𝐸
′
𝑆 →.𝐸 𝑇 → . 𝑖𝑛𝑡
𝜀 𝜀
𝐸 →.𝑇
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example

𝑇 →.(𝐸)
′
𝑆 →𝐸.
𝜀 E 𝜀 𝐸→.𝑇+𝐸
′
𝑆 →.𝐸 𝑇 → . 𝑖𝑛𝑡
𝜀 𝜀
𝐸 →.𝑇 𝜀 𝑇 →.𝑖𝑛𝑡∗𝑇
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example

𝑇 →.(𝐸)
′
𝑆 →𝐸. 𝑇
𝜀 E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸
𝜀
′
𝑆 →.𝐸
𝜀 𝜀 𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇 • We can have T as input
𝜀
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example

𝑇 →.(𝐸)
𝜀
′
𝑆 →𝐸. 𝑇
𝜀 E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸
𝜀
′
𝜀
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇 • We can have all derivations of T
𝜀
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸)
𝜀 𝑇 →(. 𝐸)
′
𝑆 →𝐸. 𝑇
𝜀 E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸
𝜀
′
𝜀
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇
𝜀
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .)
′
𝑆 →𝐸. 𝑇
𝜀 E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸
𝜀
′
𝜀
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇
𝜀 • We can have E as input
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .)
′ 𝜀
𝑆 →𝐸. 𝜀
𝑇
𝜀 E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸
𝜀
′
𝜀
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇
𝜀 • We can have all derivations of E
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
′ 𝜀
𝑆 →𝐸. 𝜀
𝑇
𝜀 E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸
𝜀
′
𝜀
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇
𝜀
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
′ 𝜀
𝑆 →𝐸. 𝜀
𝑇 +¿
𝜀 E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E
𝜀
′
𝜀
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇
𝜀
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
′ 𝜀
𝑆 →𝐸. 𝜀
𝑇 +¿
E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E→𝑇 +.𝐸 𝐸
𝜀 𝜀
′
𝑆 →.𝐸 𝜀
𝜀
E→𝑇 + 𝐸.
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇
𝜀 • We can have E as input
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
𝜀 𝜀
′ 𝜀
𝑆 →𝐸. 𝑇 +¿
E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E→𝑇 +.𝐸 𝐸
𝜀 𝜀
′
𝑆 →.𝐸 𝜀
𝜀
𝜀
E→𝑇 + 𝐸.
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇
𝜀 • We can have all derivations of E
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
𝜀 𝜀
′ 𝜀
𝑆 →𝐸. 𝑇 +¿
E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E→𝑇 +.𝐸 𝐸
𝜀 𝜀
𝜀
′
𝜀 𝑇 → 𝑖𝑛𝑡 . E→𝑇 + 𝐸.
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡 𝑖𝑛𝑡

𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇
𝜀
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
𝜀 𝜀
′ 𝜀
𝑆 →𝐸. 𝑇 +¿
E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E→𝑇 +.𝐸 𝐸
𝜀 𝜀
𝜀
′
𝜀 𝑇 → 𝑖𝑛𝑡 . E→𝑇 + 𝐸.
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡 𝑖𝑛𝑡

𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇 𝑇 →𝑖𝑛𝑡.∗𝑇
𝜀
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
𝜀 𝜀
′ 𝜀
𝑆 →𝐸. 𝑇 +¿
E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E→𝑇 +.𝐸 𝐸
𝜀 𝜀
𝜀
′
𝜀 𝑇 → 𝑖𝑛𝑡 . E→𝑇 + 𝐸.
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡 𝑖𝑛𝑡

𝑖𝑛𝑡 ∗ 𝑇 →𝑖𝑛𝑡∗.𝑇
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇 𝑇 →𝑖𝑛𝑡.∗𝑇
𝜀
𝑇
𝐸 →𝑇 .
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
𝜀 𝜀
′ 𝜀
𝑆 →𝐸. 𝑇 +¿
E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E→𝑇 +.𝐸 𝐸
𝜀 𝜀
𝜀
′
𝜀 𝑇 → 𝑖𝑛𝑡 . E→𝑇 + 𝐸.
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡 𝑖𝑛𝑡

𝑖𝑛𝑡 ∗ 𝑇 →𝑖𝑛𝑡∗.𝑇
𝐸 →.𝑇 𝑇 →.𝑖𝑛𝑡∗𝑇 𝑇 →𝑖𝑛𝑡.∗𝑇 𝑇
𝜀
𝑇 𝑇 →𝑖𝑛𝑡∗𝑇 .
𝐸 →𝑇 .
• We can have T as input
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
𝜀 𝜀
′ 𝜀
𝑆 →𝐸. 𝑇 +¿
E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E→𝑇 +.𝐸 𝐸
𝜀 𝜀
𝜀
′
𝜀 𝑇 → 𝑖𝑛𝑡 . E→𝑇 + 𝐸.
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡 𝑖𝑛𝑡

𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →𝑖𝑛𝑡.∗𝑇
𝜀 𝑇 →.𝑖𝑛𝑡∗𝑇 ∗
𝜀 𝜀
𝑇 𝐸 →𝑇 . 𝑇 → 𝑖𝑛𝑡 ∗ . 𝑇 𝑇

𝜀 𝑇 →𝑖𝑛𝑡∗𝑇 .
• Or derivations of T
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
𝜀 𝜀
′ 𝜀
𝑆 →𝐸. 𝑇 +¿
E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E→𝑇 +.𝐸 𝐸
𝜀 𝜀
𝜀
′
𝜀 𝑇 → 𝑖𝑛𝑡 . E→𝑇 + 𝐸.
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡 𝑖𝑛𝑡

𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →𝑖𝑛𝑡.∗𝑇
𝜀 𝑇 →.𝑖𝑛𝑡∗𝑇 ∗
𝜀 𝜀
𝑇 𝐸 →𝑇 . 𝑇 → 𝑖𝑛𝑡 ∗ . 𝑇 𝑇

𝜀 𝑇 →𝑖𝑛𝑡∗𝑇 .
Exercise
Valid items
 Valid items: motivation
• Until now, we have the complete NFA for
recognizing viable prefixes of a grammar.
• Using the standard subset of states
construction we can build a DFA automaton
equivalent to the NFA.
 Recognizing viable prefixes: example
¿
𝑇 →.(𝐸) 𝐸 ¿
𝜀 𝑇 →(. 𝐸) 𝑇 →( 𝐸 .) 𝑇 → ( 𝐸) .
𝜀 𝜀
′ 𝜀
𝑆 →𝐸. 𝑇 +¿
E 𝐸→.𝑇+𝐸 𝐸→𝑇 .+𝐸 E→𝑇 +.𝐸 𝐸
𝜀 𝜀
𝜀
′
𝜀 𝑇 → 𝑖𝑛𝑡 . E→𝑇 + 𝐸.
𝑆 →.𝐸 𝜀
𝜀 𝜀
𝑇 → . 𝑖𝑛𝑡 𝑖𝑛𝑡

𝑖𝑛𝑡
𝐸 →.𝑇 𝑇 →𝑖𝑛𝑡.∗𝑇
𝜀 𝑇 →.𝑖𝑛𝑡∗𝑇 ∗
𝜀 𝜀
𝑇 𝐸 →𝑇 . 𝑇 → 𝑖𝑛𝑡 ∗ . 𝑇 𝑇

𝜀 𝑇 →𝑖𝑛𝑡∗𝑇 .
 Valid items: Deterministic Automata
E 𝐸 → 𝑇 +𝐸 .

T (
T + 𝑇 → ( 𝐸 .)
(
E )
T
(
𝑇 → ( 𝐸) .
′
𝑆 →𝐸.
(
E int
int
This is int * T
the . 𝑇 → 𝑖𝑛𝑡 ∗ 𝑇 .
start
state int

• Each state is a set of items

 Valid Items

• The states of the DFA are called:

– Canonical collection of items.
– Canonical collections of LR(0) items.
 Valid items
• Definition: A given item is valid for a viable
prefix if:

by a right-most derivation.
• After parsing , the valid items are the possible
tops of the stack of items.
 Valid items intuition
• An item I is valid for a viable prefix α, if the

input α in a state s containing I

DFA recognizing viable prefixes terminates on

item stack might be after reading input α

• The items in s describe what the top of the
 Valid items
• An item is often valid for many prefixes
• Example: The item is valid for prefixes:
 Valid items: Deterministic Automata
E 𝐸 → 𝑇 +𝐸 .

T (
T + 𝑇 → ( 𝐸 .)
(
E )
T
(
𝑇 → ( 𝐸) .
′
𝑆 →𝐸.
(
E int
int
int * T
. 𝑇 → 𝑖𝑛𝑡 ∗ 𝑇 .
int

Given start and look ( and see that sequences are possible
 Valid items: Deterministic Automata
E 𝐸 → 𝑇 +𝐸 .

T (
T + 𝑇 → ( 𝐸 .)
(
E )
T
(
𝑇 → ( 𝐸) .
′
𝑆 →𝐸.
(
E int
int
int * T
. 𝑇 → 𝑖𝑛𝑡 ∗ 𝑇 .
int

Given start and look ( and see that sequences are possible
 Valid items: Deterministic Automata
E 𝐸 → 𝑇 +𝐸 .

T (
T + 𝑇 → ( 𝐸 .)
(
E )
T
(
𝑇 → ( 𝐸) .
′
𝑆 →𝐸.
(
E int
int
int * T
. 𝑇 → 𝑖𝑛𝑡 ∗ 𝑇 .
int

Given start and look ( and see that sequences are possible
 Valid items: Deterministic Automata
E 𝐸 → 𝑇 +𝐸 .

T (
T + 𝑇 → ( 𝐸 .)
(
E )
T
(
𝑇 → ( 𝐸) .
′
𝑆 →𝐸.
(
E int
int
int * T
. 𝑇 → 𝑖𝑛𝑡 ∗ 𝑇 .
int

Given start and look ( and see that sequences are possible
Exercise: valid items
 Observation

• If a valid item has symbols to the left of the “.”,

these symbols appear at the top of the stack.
¡Thank you!
 References
• Aho et al. Compilers: principles,
techniques, and tools. Torczon et al. (2014)
(Section 4.5.-4.6)
• Slides are based on the design of Aiken Alex. CS 143.
https://web.stanford.edu/class/cs143/