Shift-Reduce Parsing

Natural Language Processing: Jordan

Boyd-Graber
University of Colorado Boulder
6. OCTOBER 2014
Adapted from material by Jimmy Lin and Jason Eisner

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

1 of 17

Shift-Reduce Parsing

Alternative to arc-factored models

Cognitively plausible

Better at short-range dependencies

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

2 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Example

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

3 of 17

Components

Process a sentence word by word from a buer

You can temporarily place store words on a stack

As you process you can either:

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

4 of 17

Components

Process a sentence word by word from a buer

You can temporarily place store words on a stack

As you process you can either:
Shift: Move a word from the buer to the stack

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

4 of 17

Components

Process a sentence word by word from a buer

You can temporarily place store words on a stack

As you process you can either:
Shift: Move a word from the buer to the stack
Left: The top of the stack is the child of the buers next word

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

4 of 17

Components

Process a sentence word by word from a buer

You can temporarily place store words on a stack

As you process you can either:
Shift: Move a word from the buer to the stack
Left: The top of the stack is the child of the buers next word
Right: The buers next word is the child of the top of the stack

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

4 of 17

Initial and Final Conditions

Initially the stack has root, the buer has the sentences words,

and there are no edges

At the end, the buer must be empty

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

5 of 17

Action: Left

Add an edge (wj , wi )

wi is the top of the stack

wj is the first word of the buer

Pop the stack

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

6 of 17

Action: Left

Add an edge (wj , wi )

wi is the top of the stack

wj is the first word of the buer

Pop the stack

Stack and buer must be non-empty; wi cannot be the root

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

6 of 17

Action: Right

Add an edge (wi , wj )

wi is the top of the stack

wj is the first word in the buer
Pop the stack

Replace wj by wi at the head of buer

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

7 of 17

Action: Right

Add an edge (wi , wj )

wi is the top of the stack

wj is the first word in the buer
Pop the stack

Replace wj by wi at the head of buer

Stack and buer must be non-empty

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

7 of 17

Shift

Removes wi from the buer

Places it on the stack

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

8 of 17

Shift

Removes wi from the buer

Places it on the stack

Buer must be non-empty

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

8 of 17

Shift Reduce Example

Buer

Stack
[root

[economic, news, had, little, eect,

on, financial, markets, .]

Next transition: 1. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Buer

Stack
[root , economic

[news, had, little, eect, on,

financial, markets, .]

Next transition: 2. Left

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Buer

Stack
[root

[news, had, little, eect, on,

financial, markets, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

3. Shift

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Buer

Stack
[root , news

[had, little, eect, on, financial,

markets, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

4. Left

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Buer

Stack
[root

[had, little, eect, on, financial,

markets, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

5. Shift

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Buer

Stack
[root

, had

[little, eect, on, financial,

markets, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

6. Shift

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
, had , little

[eect, on, financial, markets, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

7. Left

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
, had

[eect, on, financial, markets, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

8. Shift

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
, had , eect

[on, financial, markets, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

9. Shift

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
, had , eect , on ]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

[financial, markets, .]

10. Shift

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root , had , eect , on ,
financial ]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

Buer
[markets, .]

11. Left

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
, had , eect , on ]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

[markets, .]

12. Right

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
, had , eect

[on, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

13. Right

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
, had

[eect, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

14. Right

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
]

[had, .]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

15. Shift

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
, had]

[.]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

16. Right

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
]

[had]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

17. Right

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[

Buer
]

[root]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

18. Shift

Shift-Reduce Parsing

9 of 17

Shift Reduce Example

Stack
[root

Buer
]

[]

Next transition:

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

9 of 17

Transition Sequence Algorithm

Start with root on stack, buer with whole sentence

If theres nothing on the stack, you must shift

If the top of the stack is the child of the top of the buer, then

make a left edge

If the top of the buer is is a child of the top of the stack and the

top of the buer has no children that have yet to be added to the
tree, then make a right

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

10 of 17

Transition Sequence Algorithm

Start with root on stack, buer with whole sentence

If theres nothing on the stack, you must shift

If the top of the stack is the child of the top of the buer, then

make a left edge

If the top of the buer is is a child of the top of the stack and the

top of the buer has no children that have yet to be added to the
tree, then make a right

Part of Homework 6

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

10 of 17

How to apply to data

Create oracle for all sentences

Create three-way classifier for each possible actions

Features
The top of the stack
Top two words on buer
The parts of speech of the words

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

11 of 17

Complexity

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

12 of 17

Complexity

A word can only enter the stack once

So complexity is O(2N)

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

12 of 17

Comparison

Shift-reduce parsers are faster

Shift-reduce parsers do better at local (deeper) connections

Arc-factored models do better at long-distance dependencies (e.g.,

verbs)

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

13 of 17

In Class

Transition Sequence to Parse

Parse to Transition Sequence

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

14 of 17

Buer

Stack
[root

[I, am, the, very, model, of, a,

modern, major, general]

Edges
Next move: 1. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Buer

Stack
[root , I

[am, the, very, model, of, a,

modern, major, general]

Edges
Next move: 2. Left

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Buer

Stack
[root

[am, the, very, model, of, a,

modern, major, general]

Edges
,I

am
Next move: 3. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Buer

Stack
[root , am

[the, very, model, of, a, modern,

major, general]

Edges
,I

am
Next move: 4. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Buer

Stack
[root , am , the

[very, model, of, a, modern, major,

general]

Edges
,I

am
Next move: 5. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Buer

Stack
[root , am , the , very

[model, of, a, modern, major,

general]

Edges
,I

am
Next move: 6. Left

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Buer

Stack
[root , am , the

[model, of, a, modern, major,

general]

Edges
,I
am
, very
model
Next move: 7. Left

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Buer

Stack
[root , am

[model, of, a, modern, major,

general]

Edges
,I
am
, very
model
, the
model
Next move: 8. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am

Buer
, model

[of, a, modern, major, general]

Edges
,I
am
, very
model
, the
model
Next move: 9. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am

Buer
, model , of

[a, modern, major, general]

Edges
,I
am
, very
model
, the
model
Next move: 10. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am

Buer
, model , of , a ]

[modern, major, general]

Edges
,I
am
, very
model
, the
model
Next move: 11. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am
modern ]

, model , of , a ,

Buer
[major, general]

Edges
,I
am
, very
model
, the
model
Next move: 12. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am , model , of , a ,
modern , major]

Buer
[general]

Edges
,I
am
, very
model
, the
model
Next move: 13. Left

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am
modern ]

, model , of , a ,

Buer
[general]

Edges
,
,
,
,

I
am
very
model
the
model
major
general
Next move: 14. Left

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am

Buer
, model , of , a ]

[general]

Edges
,
,
,
,
,

I
am
very
model
the
model
major
general
modern
general
Next move: 15. Left

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am

Buer
, model , of

[general]

Edges
,
,
,
,
,
,

I
am
very
model
the
model
major
general
modern
general
a
general
Next move: 16. Right

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am

Buer
, model

[of, ]

Edges
,
,
,
,
,
,
,

I
am
very
model
the
model
major
general
modern
general
a
general
of ! general
Next move: 17. Right

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root , am

Buer
]

[model, ]

Edges
,
,
,
,
,
,
,
,

I
am
very
model
the
model
major
general
modern
general
a
general
of ! general
model ! of
Next move: 18. Right

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root

Buer
]

[am]

Edges
,
,
,
,
,
,
,
,
,

I
am
very
model
the
model
major
general
modern
general
a
general
of ! general
model ! of
am ! model
Next move: 19. Right

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[

Buer
]

[root]

Edges
,
,
,
,
,
,
,
,
,
,

I
am
very
model
the
model
major
general
modern
general
a
general
of ! general
model ! of
am ! model
root ! am
Next move: 20. Shift

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Stack
[root

Buer
]

[]

Edges
,
,
,
,
,
,
,
,
,
,

I
am
very
model
the
model
major
general
modern
general
a
general
of ! general
model ! of
am ! model
root ! am

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

15 of 17

Transition Sequence Algorithm

Start with root on stack, buer with whole sentence

If theres nothing on the stack, you must shift

If the top of the stack is the child of the top of the buer, then

make a left edge

If the top of the buer is is a child of the top of the stack and the

top of the buer has no children that have yet to be added to the
tree, then make a right

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

16 of 17

Parse to Transition Sequence

Action
s

Head Index

Natural Language Processing: Jordan Boyd-Graber

Head Word

Boulder

Dep Index

Dep Word

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s

Head Index

Natural Language Processing: Jordan Boyd-Graber

Head Word

Boulder

Dep Index

Dep Word

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l

Head Index

Head Word

Dep Index

Dep Word

cat

fat

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s
l

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

sat

cat

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s
l
s

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

sat

cat

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s
l
s
s

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

sat

cat

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s
l
s
s
s

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

sat

cat

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s
l
s
s
s
l

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

sat

cat

mat

the

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s
l
s
s
s
l
r

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

sat

cat

7
5

mat
on

6
7

the
mat

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s
l
s
s
s
l
r
r

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

sat

cat

7
5
4

mat
on
sat

6
7
5

the
mat
on

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s
l
s
s
s
l
r
r
r

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

sat

cat

7
5
4
0

mat
on
sat
None

6
7
5
4

the
mat
on
sat

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17

Parse to Transition Sequence

Action
s
s
l
l
s
l
s
s
s
l
r
r
r
s

Head Index

Head Word

Dep Index

Dep Word

3
3

cat
cat

2
1

fat
the

sat

cat

7
5
4
0

mat
on
sat
None

6
7
5
4

the
mat
on
sat

Natural Language Processing: Jordan Boyd-Graber

Boulder

Shift-Reduce Parsing

17 of 17