Grammar and Parsing

CSE

Ho Chi Minh City University of Technology

2023.01
Outline

1 Context-Free Grammars (CFG)

Grammar and Sentences Structure
What makes a Good Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

2 Probabilistic Context-Free Grammars (PCFG)

3 Dependency Parsing
Dependency relations
Dependency formalisms
Transition–Based Dependency parsing
MaltParser

4 Relation Extraction with Stanford Dependencies

 Grammar and Sentences Structure

Context-Free The most common method to study the structure of

Grammars (CFG)
Grammar and Sentences
sentence is how sentence is broken into its major subparts
Structure
What makes a Good
and how those subparts are broken up in turn, is as a tree.
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies Hình: (3.1) Syntactic structure of sentence “John ate the pizza”

CSE Grammar and Parsing 2023.01 1/53

 Grammar and Sentences Structure

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing
S consists initial noun phrase NP and verb phrase VP. initial
NP is made of NAME John. Initial VP is composed of verb
Probabilistic
Context-Free
Grammars
(PCFG) ate and NP, which consists ART the and N pizza
Dependency The structure of sentence may be represented in another way:
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 2/53

 Context Free grammar (CFG)

Context-Free
Grammars (CFG)
Grammar and Sentences

G = (S, P, N, T )
Structure
What makes a Good

S: start symbol S,
Grammar
Top-down Parser

P: production rules, which have the form: A → α;

A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic N, T : set of lexical symbols (word categories).

Context-Free
Grammars N is set of non- terminal symbols; T is set of terminal
(PCFG)
symbols.
Dependency
Parsing There are two important process based on derivations:
sentence generation and sentence parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing There are two methods of search (structure of sentence):
Top-down and Bottom-up.
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 3/53

 What makes a Good Grammar

Context-Free
Grammars (CFG)

To construct a grammar for language, we are interested in

Grammar and Sentences
Structure

generality, the range of sentences the grammar analyzes

What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser correctly; selectivity, the range of non-sentences it identifies
as problematic; and understandability, the simplicity of
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
grammar itself
(PCFG) Beginning with small grammar, such as those that describe
Dependency
Parsing
only few types of sentences, one structural analysis of a
Dependency relations sentence may appear as understandable as another.
Then we attempt to extend a grammar to cover a wide range
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser of sentences, however, we often find one analysis is easily
Relation
Extraction with
extendable, while the other requires complex modification.
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 4/53

 What makes a Good Grammar (cont.)

Context-Free
Grammars (CFG)
This analysis retains its simplicity and generality as it is
Grammar and Sentences
Structure
extended is more desirable.
To pay close attention to the way the sentence is divided
What makes a Good
Grammar

into its subparts, called constituents.

Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing
By using our intuition we can apply specific tests, as
Probabilistic
Context-Free follows.
Grammars
(PCFG) To decide that a group of words forms a particular constituent;
Dependency
Try to construct a new sentence that involves that group of
Parsing words in conjunction with another group of words classified as
the same type of constituent.
Dependency relations
Dependency formalisms

Example:
Transition–Based
Dependency parsing
MaltParser

Relation I ate a hamburger and a hot dog (NP-NP).

Extraction with
Stanford
Dependencies
I will eat the hamburger and throw away the hot dog
(VP-VP)

CSE Grammar and Parsing 2023.01 5/53

 Top-down Parser

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser Parsing algorithm is a procedure that searches through
A bottom-Up Chart Parser
Top-Down Chart Parsing various ways of grammar rules to find a combination that
Probabilistic
Context-Free
generates a tree that could be the structure of the input
Grammars
(PCFG)
sentence.
Dependency A top-down parser starts with the S symbol and attempts to
Parsing
Dependency relations
rewrite it into a sequence of terminal symbols that matches
Dependency formalisms
Transition–Based
the classes (categories) of the words in the input sentence.
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 6/53

 A Simple Top-Down Parsing Algorithm
The algorithm starts with the initial state ((S) 1) and no
Context-Free
Grammars (CFG) backup
Grammar and Sentences
Structure
What makes a Good
1 Select the current state: take the first state of the
Grammar
Top-down Parser
possibilities list and call it C. If the possibilities list is empty,
A bottom-Up Chart Parser
Top-Down Chart Parsing
then the algorithm fails ( no successful parse is possible)
Probabilistic 2 If C consists of an empty list and the position of word is at
Context-Free
Grammars the end of sentence, then the algorithm is succeed
(PCFG)

Dependency
3 Otherwise, generate the next possible states.
Parsing
Dependency relations
3.1 If the first symbol on the symbol list C is terminal (lexical
Dependency formalisms symbol), and the next word in the sentence can be in that
Transition–Based
Dependency parsing class, then create a new state by removing the first symbol
MaltParser
from the symbol list C, update the word position and add it to
Relation
Extraction with the possibilities list
Stanford
Dependencies
3.2 If the first symbol of C is non-terminal, generate a new state of
each rule in the grammar that can rewrite that non-terminal
symbol and add them all to the possibilities list.
CSE Grammar and Parsing 2023.01 7/53
 A Simple Top-Down Parsing Algorithm

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Example: Parse the sentence: 1 the 2 dogs 3 cried4
Top-Down Chart Parsing
Grammar:
Probabilistic
Context-Free
1. S → NP VP
Grammars
(PCFG)
2. NP → ART N
Dependency 3. NP→ ART ADJ N
Parsing
Dependency relations
4. VP → V
Dependency formalisms
Transition–Based
5. VP → V NP
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 8/53

 A Simple Top-Down Parsing Algorithm (continue)

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies Hình: Top-Down Depth first parser of “The dog cried”

CSE Grammar and Parsing 2023.01 9/53

 Parsing as a Search Procedure

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
The top-down parser is described as a search procedure that
What makes a Good
Grammar
implements as follows.
Top-down Parser
A bottom-Up Chart Parser
The possibilities list is initially set to the start state S of the
Top-Down Chart Parsing
parser.
Probabilistic
Context-Free 1 Select the first state from possibilities list and remove it
Grammars
(PCFG) from the list.
Dependency
Parsing
2 Generate new states from a current state by trying every
Dependency relations
Dependency formalisms
possible option from the selected (there may be none if we
Transition–Based
Dependency parsing
on a bad path).
MaltParser
3 Add the states generated in step 2 to the possibilities list
Relation
Extraction with and repeat step 1.
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 10/53

 Parsing as a Search Procedure

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

Hình: A Top Down Parse of “The old man cried”

CSE Grammar and Parsing 2023.01 11/53
 Parsing as a Search Procedure

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford Hình: Depth-first strategy and breadth-first strategy
Dependencies

CSE Grammar and Parsing 2023.01 12/53

 A bottom-Up Chart Parser

Context-Free
Grammars (CFG)

The main difference between top-down and bottom-up

Grammar and Sentences
Structure

parsers is the way the grammar rules are used.

What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser The extension algorithm Add a constituent C from position
p1 to p2 :
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars 1 Insert C into chart from p1 to p2 ;
(PCFG)

Dependency
2 For any active arc of the form: X → X1 ... • C ...Xn from p0
Parsing to p1 , add a new active arc X → X1 ...C • ...Xn from p0 to p2
Dependency relations
Dependency formalisms
Transition–Based
3 For any active arc of the form: X → X1 ... • C from p0 to p1 ,
Dependency parsing
MaltParser
add a new constituent of type X from p0 to p2 to the
Relation agenda.
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 13/53

 A Bottom-up Chart Parsing Algorithm

Context-Free
Grammars (CFG)
Grammar and Sentences
Do until there is no input left.
If the agenda is empty, then look up the interpretations
Structure
What makes a Good 1
Grammar
Top-down Parser (categories) of the new word in the input and add them to
the agenda.
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
2 Select a constituent from the agenda (let’s call it
Grammars
(PCFG)
constituent C from p1 to p2 ).
Dependency 3 For any grammar rule X → CX1 ...Xn , add active arc of the
Parsing
Dependency relations form X → •CX1 ...Xn from p1 to p2 .
Dependency formalisms
Transition–Based
Dependency parsing
4 Add C to the chart by the extension algorithm.
MaltParser
—————————————————————————
Relation
Extraction with Example: To parse a sentence “The large can can hold the
Stanford
Dependencies water”

CSE Grammar and Parsing 2023.01 14/53

 A bottom-Up Chart Parser

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies Hình: After parsing the large can

CSE Grammar and Parsing 2023.01 15/53

 A bottom-Up Chart Parser

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies
Hình: The chart after adding hold, omitting arcs generated for the
first NP

CSE Grammar and Parsing 2023.01 16/53

 A bottom-Up Chart Parser

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with Hình: The chart after the NPs are found, omitting all but the
Stanford
Dependencies crucial active arcs

CSE Grammar and Parsing 2023.01 17/53

 A bottom-Up Chart Parser

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Hình: The final chart

Grammar and Parsing 2023.01 18/53
 Efficiency Considerations

Context-Free
Grammars (CFG)
Grammar and Sentences
Chart–based parser can be considerably more efficient than
parsers that rely only on a search because the same
Structure
What makes a Good

constituent never constructed more than once.

Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing The complexity of the pure top-down or bottom-up parser
Probabilistic
Context-Free
could require up to C n operations to parse a sentence of the
Grammars
(PCFG)
length n, C is a constant that depends on the specific
Dependency
algorithm we use.
The complexity of the chart-based parser is Kxn3 . K is a
Parsing
Dependency relations
Dependency formalisms
Transition–Based
constant that depends on the algorithm, and n is the
sentence length.
Dependency parsing
MaltParser

Relation
Extraction with
The chart-based parser would be up to many times faster
Stanford
Dependencies
than a pure parser.

CSE Grammar and Parsing 2023.01 19/53

 Top-Down Chart Parsing

Context-Free
Grammars (CFG)
Grammar and Sentences

So far we have seen a simple top-down method and a

Structure
What makes a Good

bottom-up chart-based method for parsing context-free

Grammar
Top-down Parser

grammars.
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic Now a new method is presented actually captures the

Context-Free
Grammars advantages of both, that is top-down chart parser.
(PCFG)
Top-down arc Introduction Algorithm
Dependency
Parsing To add an arc S → C1 ... • Ci ...Cn ending at position j, do the
following.
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing For any rule in the grammar of the form Ci → X1 ...Xk ,
recursively add new arc Ci → •X1 ...Xk from position j to j.
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 20/53

 Top-Down Chart Parsing Algorithm

Context-Free
Grammars (CFG) Initialization: for every rule in the grammar of the form
Grammar and Sentences
Structure
What makes a Good
S → X1 ...Xk add an arc; labeled S → •X1 ...Xk using the arc
Grammar
Top-down Parser
introduction algorithm.
A bottom-Up Chart Parser
Top-Down Chart Parsing
Parsing: Do until there is no input left
Probabilistic 1 If agenda is empty, look up the interpretation of the next
Context-Free
Grammars word and add them to the agenda.
(PCFG)

Dependency
2 Select constituent from the agenda (call it constituent C).
Using the arc extension algorithm, combine C with every
Parsing
Dependency relations 3
Dependency formalisms
Transition–Based
active arc on the chart. Any new constituents are added to
the agenda.
Dependency parsing
MaltParser

Relation
Extraction with
4 For any active arcs created in step 3, add them to the chart
Stanford
Dependencies
using the top-down arc introduction algorithm.

CSE Grammar and Parsing 2023.01 21/53

 Top-Down Chart Parsing

Context-Free Example: 1 the 2 large 3 can 4 can 5 hold 6 the 7 water8

Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with Hình: The chart after adding hold, omitting arcs generated for the
Stanford
Dependencies
first NP

CSE Grammar and Parsing 2023.01 22/53

 Top-Down Chart Parsing

Context-Free
Grammars (CFG) Example: 1 the 2 large 3 can 4 can 5 hold 6 the 7 water8
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Hình: The final chart for top-down filtering algorithm
Dependencies

CSE Grammar and Parsing 2023.01 23/53

 Probabilistic – or stochastic – context-free
grammars (PCFGs)
Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
G = (T, N, S, R, P)
Grammar
Top-down Parser T is a set of terminal symbols
A bottom-Up Chart Parser
Top-Down Chart Parsing
N is a set of nonterminal symbols
Probabilistic
Context-Free S is the start symbol (S ∈ N)
Grammars
(PCFG) R is a set of rules/productions of the form X → γ
Dependency
Parsing P is a probability function
Dependency relations
Dependency formalisms
P: R → [0, 1]
Transition–Based
Dependency parsing
A grammar G generates a language model L.
=1
MaltParser
P
γ∈T ∗ P(γ)
Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 24/53

 PCFGs

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 25/53

 The probability of trees and strings

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing
P(t) – The probability of a tree t is the product of the
Probabilistic probabilities of the rules used to generate it.
Context-Free
Grammars P(s) – The probability of the string s is the sum of the
(PCFG)

Dependency
probabilities of the trees which have that string as their yield
Parsing
P(s) = where t is a parse of s =
P P
Dependency relations j P(s, t) j P(t)
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 26/53

 Frame Title

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 27/53

 The probability of trees and strings

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 28/53

 The probability of trees and strings

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 29/53

 The probability of trees and strings

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 30/53

 Dependency relation

Context-Free
Dependency syntax postulates that syntactic structure
Grammars (CFG) consists of lexical items linked by binary asymmetric relations
(“arrows”) called dependencies
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford The arrows are commonly typed with the name of
grammatical relations (subject, prepositional object,
Dependencies

apposition, etc.)
CSE Grammar and Parsing 2023.01 31/53
 Dependency relation
Dependency syntax postulates that syntactic structure
consists of lexical items linked by binary asymmetric relations
Context-Free
Grammars (CFG)
Grammar and Sentences
Structure (“arrows”) called dependencies
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
The arrow connects a head (governor, superior, regent) with
Stanford
Dependencies
a dependent (modifier, inferior, subordinate)
Usually, dependencies form a tree (connected, acyclic,
single-head)
CSE Grammar and Parsing 2023.01 31/53
 Dependency relation

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies
Selected dependency relations from the Universal
Dependency set [4]
CSE Grammar and Parsing 2023.01 32/53
 Dependency Grammar and Dependency Structure

Context-Free
Grammars (CFG) Example of dependency structure with universal dependency
relations:
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms

Clausal relations NSUBJ and DOBJ identify the subject and

Transition–Based
Dependency parsing

direct object of the predicate cancel, while NMOD, DET,

MaltParser

Relation
Extraction with
Stanford
and CASE relations denote modifiers of the nouns flights and
Dependencies Houston

CSE Grammar and Parsing 2023.01 33/53

 Dependency formalisms
Dependency structure is directed graph:
G = (V, A) V: set of vertices A: set of ordered pairs of
Context-Free
Grammars (CFG)
Grammar and Sentences
Structure vertices, We will refer A as arcs.
V corresponds exactly to the set of words in the given
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser sentence.
A captures the head-dependent and grammatical function
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
relationship between the elements in V
(PCFG) Dependency Tree
Dependency
Parsing
Dependency tree is directed graph that satisfies the following
Dependency relations
Dependency formalisms
constraints:
Transition–Based
Dependency parsing
1 There is single designated root node that has no incoming
MaltParser
arcs.
2 With the exception of root node, each vertex has exactly
Relation
Extraction with
Stanford
Dependencies one incoming arc.
3 There is a unique path from the root node to each vertex in
V.
CSE Grammar and Parsing 2023.01 34/53
 Projectivity

Context-Free An arc from a head to a dependent is said to be projective if

Grammars (CFG)
Grammar and Sentences there is a path from the head to every word that lies
between the head and the dependent in the sentence.
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
A dependency tree is then said to be projective if all the
Top-Down Chart Parsing
arcs that make it up are projective.
Probabilistic
Context-Free Consider the following example:
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies
Arc from flight to its modifier was is non-projective, since there is
no path from flight to intervening words this and morning.
CSE Grammar and Parsing 2023.01 35/53
 Projectivity

Context-Free Dependencies from a CFG tree using heads must be

Grammars (CFG)
Grammar and Sentences
projective
Structure
What makes a Good
There must not be any crossing dependency arcs when the
words are laid out in their linear order, with all arcs above
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing
the words.
Probabilistic
Context-Free
But dependency theory normally does allow non-projective
Grammars
(PCFG)
structures to account for displaced constituents
Dependency
You can’t easily get the semantics of certain constructions
Parsing right without these non-projective dependencies
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 36/53

 Relation between phrase structure and
dependency structure

A dependency grammar has a notion of a head. Officially,

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure CFGs don’t.
What makes a Good
Grammar
Top-down Parser
But modern linguistic theory and all modern statistical
A bottom-Up Chart Parser
Top-Down Chart Parsing
parsers (Charniak, Collins, Stanford, ... ) do, via
Probabilistic
hand-written phrasal “head rules”:
Context-Free The head of a Noun Phrase is a noun/number/adj/...
Grammars
(PCFG) The head of a Verb Phrase is a verb/modal/...
Dependency
Parsing The head rules can be used to extract a dependency parse
Dependency relations
Dependency formalisms
from a CFG parse
The closure of dependencies gives constituency from a
Transition–Based
Dependency parsing

dependency tree
MaltParser

Relation
Extraction with
Stanford But the dependents of a word must be at the same level
Dependencies
(i.e., “flat”) – there can be no VP!

CSE Grammar and Parsing 2023.01 37/53

 Relation between phrase structure and
dependency structure (cont.)
Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 38/53

 Relation between phrase structure and
dependency structure (cont.)
Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies a) Example of a phrase structure "Vinken will join the board
as a nonexecutive director Nov 29"
CSE Grammar and Parsing 2023.01 39/53
 Relation between phrase structure and
dependency structure
Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies b) To translate a) structure to dependency structure

CSE Grammar and Parsing 2023.01 40/53

 Relation between phrase structure and
dependency structure
Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
c) Dependency structure of sentence
Dependencies

CSE Grammar and Parsing 2023.01 41/53

 Transition – Based Dependency parsing

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser Dependency parsing is motivated by a stack-based approach
called shift-reduce parsing.
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free Configuration consists of a stack, an input buffer of words,
Grammars
(PCFG)
or tokens, and a set of relations representing a dependency
Dependency tree.
Parsing
Dependency relations Parsing process consists of a sequence of transitions through
the space of possible configurations.
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 42/53

 Basic transition- based dependency parser

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

Basic transition-based dependency parser

CSE Grammar and Parsing 2023.01 43/53
 MaltParser [Nivre et al. 2008]

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
A simple form of greedy discriminative dependency parser
The parser does a sequence of bottom-up actions
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser Roughly like “shift” or “reduce” in a shift-reduce parser, but
the “reduce” actions are specialized to create dependencies
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
with head on left or right
(PCFG) The parser has:
Dependency
Parsing a stack δ, written with top to the right. which starts with the
Dependency relations ROOT symbol
a buffer β, written with top to the left. which starts with the
Dependency formalisms
Transition–Based

input sentence
Dependency parsing
MaltParser

Relation a set of dependency arcs A. which starts off empty

Extraction with
Stanford
a set of actions
Dependencies

CSE Grammar and Parsing 2023.01 44/53

 Basic transition-based dependency parser

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing

Notes: Unlike the regular presentation of the CFG reduction

Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing step, dependencies combine one thing from each stack and
buffer
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 45/53

 MaltParser (cont.)

Context-Free
Grammars (CFG)
Grammar and Sentences
Actions (“arc-eager” dependency parser)
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser
This is the common “arc-eager” variant: a head can
Relation
Extraction with immediately take a right dependent before its dependents are
Stanford
Dependencies found

CSE Grammar and Parsing 2023.01 46/53

 MaltParser (cont.)

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 47/53

 Example

Context-Free
Grammars (CFG)
Grammar and Sentences

Happy children like to play with their friends.

Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 48/53

 Example

Context-Free
Grammars (CFG)

Example: Happy children like to play with their friends

Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 49/53

 Evaluation of Dependency Parsing

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 50/53

 Stanford Dependencies [de Marneffe et al. LREC
2006]
Context-Free
Grammars (CFG) The basic dependency representation is projective
It can be generated by postprocessing headed phrase
Grammar and Sentences
Structure

structure parses (Penn Treebank syntax)

What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing
It can also be generated directly by dependency parsers,
Probabilistic such as MaltParser, or the Easy-First Parser
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 51/53

 Graph modification to facilitate semantic analysis

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
Bell, based in LA, makes and distributes electronic and
What makes a Good
Grammar
computer products.
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 52/53

 Graph modification to facilitate semantic analysis

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
Bell, based in LA, makes and distributes electronic and
What makes a Good
Grammar
computer products.
Top-down Parser
A bottom-Up Chart Parser
Top-Down Chart Parsing

Probabilistic
Context-Free
Grammars
(PCFG)

Dependency
Parsing
Dependency relations
Dependency formalisms
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 52/53

 REFERENCE OF CHAPTER 3

Context-Free
Grammars (CFG)
Grammar and Sentences
Structure
What makes a Good
Grammar
Top-down Parser
A bottom-Up Chart Parser
1 http://www.sfs.uni-tuebingen.de/∼dm/10/ss/dep/dg-slides-
Top-Down Chart Parsing 2x2.pdf
Probabilistic
Context-Free 2 https://web.stanford.edu/∼jurafsky/NLPCourseraSlides.html
Grammars
(PCFG)
3 Speech and Language Processing. Daniel Jurafsky & James H.
Dependency
Parsing
Martin. Copyright 2018.
Dependency relations
Dependency formalisms 4 https://universaldependencies.org/u/dep. Access: Jan 2023.
Transition–Based
Dependency parsing
MaltParser

Relation
Extraction with
Stanford
Dependencies

CSE Grammar and Parsing 2023.01 53/53

Thank you!