Outline More about handles Bottom-Up Parsing Algorithms Viable prefixes: A building block for recognizing handles Computing

ing viable prefixes Simple LR parsing (SLR)

1 CS780(Prasad) L134ALG 2

Lecture Notes by Profs. Alex Aiken and George Necula (UCB)

CS780(Prasad) L134ALG

Review
is a handle of in a rightmost derivation if S =>* X Handles always appear at the top of the stack
To the right of the rightmost non-terminal. Thus shift and reduce moves are sufficient.

Example: Reduction w/o a Handle Recall our favorite grammar:

ET+E|T T int * T | int | (E)

To implement shift-reduce parsing, we must detect handles. But even if there are symbols on the stack that can be reduced, that doesnt mean there is a handle.
CS780(Prasad) L134ALG 3

is not part of any rightmost derivation.

Now the step T * int + int

int * int + int

Thus, int is not a handle of int * int + int.

CS780(Prasad) L134ALG 4

Notes on Handles Every handle has some production rhs on top of the stack. But a production rhs on top of the stack is not necessarily a handle. In other words, whether a given stack has handle can depend on the contents of the entire stack. Unique Handles: If a grammar is unambiguous, then every step of a rightmost derivation has a unique handle.
CS780(Prasad) L134ALG 5

Viable Prefixes It is not obvious how to detect handles. At each step the parser sees only the stack, not the entire input. is a viable prefix if there is an such that | is a state of a shift-reduce parser.

CS780(Prasad)

L134ALG

Huh? What does this mean? A few things:

A viable prefix does not extend past the right end of the handle. Its a viable prefix because it is a prefix of the handle. A viable prefix can be extended to a sentential form by adding terminals to the right. As long as a parser has viable prefixes on the stack, no parsing error has been detected.
CS780(Prasad) L134ALG 7

Important Fact #3 Important Fact #3 about bottom-up parsing: For any grammar, the set of viable prefixes is a regular language. This fact is non-obvious. We show how to compute automata that accept viable prefixes.
CS780(Prasad) L134ALG 8

Items An item is a production with a . somewhere on the rhs. The items for T (E) are
T T T T .(E) (.E) (E.) (E).

Intuition The problem in recognizing viable prefixes is that the stack has only bits and pieces of the rhs of productions.
If it had a complete rhs, we could reduce.

These bits and pieces are always prefixes of rhs of productions.

The only item for X is X . Items are often called LR(0) items.
CS780(Prasad) L134ALG 9 CS780(Prasad) L134ALG 10

Example Consider the input (int)

Then (E|) is a state of a shift-reduce parse. (E is a prefix of the rhs of T (E)
Will be reduced after the next shift.

Generalization The stack may have many prefixes of rhss Prefix1 Prefix2 . . . Prefixn-1 Prefixn Let Prefixi be a prefix of rhs of Xi i
Now Prefixn is a prefix of n Which eventually reduces to Xn Which should be a prefix of the missing part of n-1

Item T (E.) says that so far we have seen (E of this production and hope to see )
CS780(Prasad) L134ALG 11

Recursively, Prefixk+1Prefixn eventually reduces to the missing part of k

CS780(Prasad) L134ALG 12

An Example
Consider the string (int * int): (int *|int) is a state of a shift-reduce parse. ( is a prefix of the rhs of T (E) is a prefix of the rhs of E T int * is a prefix of the rhs of T int * T

An Example (Cont.) The stack of items T (.E) E .T T int * .T Says Weve seen ( of T (E) Weve seen of E T Weve seen int * of T int * T
13 CS780(Prasad) L134ALG 14

CS780(Prasad)

L134ALG

Recognizing Viable Prefixes Idea: To recognize viable prefixes, we must

Recognize a sequence of partial rhss of productions, where Each sequence can eventually reduce to part of the missing suffix of its predecessor.

An NFA Recognizing Viable Prefixes 1. Add a dummy production S S to G 2. The NFA states are the items of G 3. For item E .X add transition E .X X E X. 4. For item E .X and production X add E .X X . 5. Every state is an accepting state 6. Start state is S .S
CS780(Prasad) L134ALG 16

Including the extra production

CS780(Prasad)

L134ALG

15

E T+.E

NFA for Viable Prefixes of the Example

T . (E) S E.
E (

Translation to the DFA

T S E . E S . E E . T E .T + E T .(E) T .int * T T .int T E T. E T. + E T int. * T int T int. int * + int

E .T E .T + E T .(E) T .int * T T .int int (

E T + E.

T (.E)

E T

T (E.) E T.+E
int +

T (E). E T+.E
E

( T (. E) E .T E .T + E E T .(E) T .int * T T .int

E . T+E

S . E

T int.

T .int

E T+E. T int.* T

T int * T. T T (E.) )

E . T
T

T int * .T T .(E) T .int * T T .int

T .int * T E T.

int

* T

T int *.T

T int * T.
17

T (E).
L134ALG

(
31

CS780(Prasad)

Valid Items Item X . is valid for a viable prefix if S =>* X by a right-most derivation. Intuition: The valid items are the prefixes of productions we might see after Items Valid for a Prefix An item I is valid for a viable prefix if the DFA recognizing viable prefixes terminates on input in a state containing I.
CS780(Prasad) L134ALG 32

Valid Items Example An item is often valid for many prefixes. Example: The item T (.E) is valid for prefixes ( (( ((( (((( ...
CS780(Prasad) L134ALG 33

E T+.E

Valid Items for (((

T S E . E S . E E . T E .T + E T .(E) T .int * T T .int T E T. E T. + E T int. * T int T int. int * + int

E .T E .T + E T .(E) T .int * T T .int int T int * T. T T (E.) ) ( (

E T + E.

Lingo
(

T (. E) E .T E .T + E E T .(E) T .int * T T .int

The states of the DFA are canonical collections of items or canonical collections of LR(0) items

T int * .T T .(E) T .int * T T .int (

T (E).

(There are other ways of constructing LR(0) items.)

CS780(Prasad) L134ALG 35

34

SLR Parsing LR = Left-to-right scan SLR = Simple LR Idea: Assume

SLR Moves
Reduce by X if Shift if
s contains item X . t Follow(X) s has a transition labeled t

stack contains next input is t DFA on input terminates in state s

If there are conflicts under these rules, the grammar is not SLR. The rules amount to a heuristic for detecting handles.
The SLR grammars are those where the heuristics detect exactly the handles.

CS780(Prasad)

L134ALG

36

CS780(Prasad)

L134ALG

37

Nave SLR Parsing Algorithm 1. Let M be DFA for viable prefixes of G. 2. Let |x1xn$ be the initial configuration. 3. Repeat until configuration is S|$
Let | be current configuration. Run M on current stack If M rejects , report parsing error
Stack is not a viable prefix Shift if X . a I Reduce if X . I and a Follow() Report parsing error if neither applies
L134ALG

Notes If there is a conflict in the last step, grammar is not SLR(k). k is the amount of lookahead
In practice, k = 1.

If M accepts with items I, let a be next input

CS780(Prasad)

38

CS780(Prasad)

L134ALG

39

E T+.E

7
E E T + E.

SLR Example Configuration DFA Halt State |int * int$ 1 Action shift
2

Configuration |int * int$

T S E . E S . E E . T E .T + E T .(E) T .int * T T .int T E T. E T. + E

E .T E .T + E T .(E) T .int int (

+ int

6
( T (. E) E .T E .T + E E T .(E) T .int * T T .int

T .int * T

T int. * T

int T int. int *

4
T

T int * T.

T int * .T T .(E) T .int

11

T (E.) )

T .int * T

T (E).

9
(
41

10

CS780(Prasad)

L134ALG

40

E T+.E

7
E E T + E.

SLR Example Configuration DFA Halt State Action |int * int$ 1 shift int | * int$ 3 * not in Follow(T) shift
2

Configuration int | * int$ E .T

T S E . E S . E E . T E .T + E T .(E) T .int * T T .int T E T. E T. + E

E .T + E T .(E) T .int int

+ int

6
( T (. E) ( E .T E .T + E E T .(E) T .int * T T .int

T .int * T

T int. * T

int T int. int *

4
T

T int * T.

T int * .T T .(E) T .int

11

T (E.) )

T .int * T

T (E).

9
(
44

10

CS780(Prasad)

L134ALG

42

E T+.E

7
E E T + E.

SLR Example Configuration |int * int$ int | * int$ int * | int$ DFA Halt State 1 3 * not in Follow(T) 11 Action shift shift shift
2

Configuration int * | int$ E .T

T S E . E S . E E . T E .T + E T .(E) T .int * T T .int T E T. E T. + E

E .T + E T .(E) T .int int

+ int

6
( T (. E) ( E .T E .T + E E T .(E) T .int * T T .int

T .int * T

T int. * T

int T int. int *

4
T

T int * T.

T int * .T T .(E) T .int

11

T (E.) )

T .int * T

T (E).

9
(
48

10

CS780(Prasad)

L134ALG

45

E T+.E

7
E E T + E.

SLR Example Configuration |int * int$ int | * int$ int * | int$ int * int |$ DFA Halt State 1 3 * not in Follow(T) 11 3 $ Follow(T) Action shift shift shift red. T int
2

Configuration int * int |$

T S E . E S . E E . T E .T + E T .(E) T .int * T T .int T E T. E T. + E

E .T E .T + E T .(E) T .int int (

+ int

6
( T (. E) E .T E .T + E E T .(E) T .int * T T .int

T .int * T

T int. * T

int T int. int *

4
T

T int * T.

T int * .T T .(E) T .int

11

T (E.) )

T .int * T

T (E).

9
(
53

10

CS780(Prasad)

L134ALG

49

SLR Example Configuration |int * int$ int | * int$ int * | int$ int * int |$ int * T |$ DFA Halt State 1 3 * not in Follow(T) 11 3 $ Follow(T) 4 $ Follow(T) Action shift shift shift red. T int red. T int*T

SLR Example Configuration |int * int$ int | * int$ int * | int$ int * int |$ int * T |$ T |$ DFA Halt State 1 3 * not in Follow(T) 11 3 $ Follow(T) 4 $ Follow(T) 5 $ Follow(T) Action shift shift shift red. T int red. T int*T red. ET

CS780(Prasad)

L134ALG

54

CS780(Prasad)

L134ALG

59

SLR Example Configuration |int * int$ int | * int$ int * | int$ int * int |$ int * T |$ T |$ E |$ DFA Halt State 1 3 * not in Follow(T) 11 3 $ Follow(T) 4 $ Follow(T) 5 $ Follow(T) Action shift shift shift red. T int red. T int*T red. ET accept
62

Notes
Skipped using extra start state S in this example to save space on slides. Rerunning the automaton at each step is wasteful
Most of the work is repeated.

An Improvement Remember the state of the automaton on each prefix of the stack. Change stack to contain pairs Symbol, DFA State
CS780(Prasad) L134ALG 63

An Improvement (Cont.) For a stack sym1 , state1 . . . symn, staten

Goto Table Define Goto[i,A] = j if statei A statej Goto is just the transition function of the DFA
One of two parsing tables.

staten is the final state of the DFA on sym1 symn

Detail: The bottom of the stack is any,start where

any is any dummy symbol start is the start state of the DFA

CS780(Prasad)

L134ALG

64

CS780(Prasad)

L134ALG

65

Refined Parser Moves Shift x

Action Table For each state si and terminal a

Push a, x on the stack a is current input x is a DFA state

If si has item X .a and Goto[i,a] = j then action[i,a] = shift j If si has item X . and a Follow(X) and X S then action[i,a] = reduce X If si has item S S. then action[i,$] = accept Otherwise, action[i,a] = error

Reduce X
As before

Accept Error
CS780(Prasad) L134ALG 66

CS780(Prasad)

L134ALG

67

SLR Parsing Algorithm

Let I = w$ be initial input Let j = 0 Let DFA state 1 have item S .S Let stack = dummy, 1 repeat case action[top_state(stack),I[j]] of shift k: push I[j++], k reduce X A: pop |A| pairs, push X, Goto[X,top_state(stack)] accept: halt normally error: halt and report error
CS780(Prasad) L134ALG 68

Notes on SLR Parsing Algorithm Note that the algorithm uses only the DFA states and the input
The stack symbols are never used!

However, we still need the symbols for semantic actions.

CS780(Prasad)

L134ALG

69