Scribd
Upload
8 views8 pages

Lin 13

The document discusses parsing, which involves determining if a string of symbols can be generated by a context-free grammar. It also discusses treebanks, which are corpora with parsed text used for testing parsers, and provides an example of evaluating parser performance metrics on a treebank.

Uploaded by

Brock Ternov
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
8 views8 pages

Lin 13

The document discusses parsing, which involves determining if a string of symbols can be generated by a context-free grammar. It also discusses treebanks, which are corpora with parsed text used for testing parsers, and provides an example of evaluating parser performance metrics on a treebank.

Uploaded by

Brock Ternov
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Spurious Ambiguity

• Most parse trees of most NL sentences make no


sense.

19
Parsing
• Given a string of non-terminals and a CFG,
determine if the string can be generated by the
CFG.
– Also return a parse tree for the string
– Also return all possible parse trees for the string
• Must search space of derivations for one that
derives the given string.
– Top-Down Parsing: Start searching space of
derivations for the start symbol.
– Bottom-up Parsing: Start search space of reverse
deivations from the terminal symbols in the string.
Parsing Example

VP

Verb NP
book that flight
book Det Nominal

that Noun

flight
Treebanks
• English Penn Treebank: Standard corpus for
testing syntactic parsing consists of 1.2 M words
of text from the Wall Street Journal (WSJ).
• Typical to train on about 40,000 parsed sentences
and test on an additional standard disjoint test set
of 2,416 sentences.
• Chinese Penn Treebank: 100K words from the
Xinhua news service.
• Other corpora existing in many languages, see the
Wikipedia article “Treebank”

85
First WSJ Sentence

( (S
(NP-SBJ
(NP (NNP Pierre) (NNP Vinken) )
(, ,)
(ADJP
(NP (CD 61) (NNS years) )
(JJ old) )
(, ,) )
(VP (MD will)
(VP (VB join)
(NP (DT the) (NN board) )
(PP-CLR (IN as)
(NP (DT a) (JJ nonexecutive) (NN director) ))
(NP-TMP (NNP Nov.) (CD 29) )))
(. .) ))
86
Parsing Evaluation Metrics
• PARSEVAL metrics measure the fraction of the
constituents that match between the computed and
human parse trees. If P is the system’s parse tree and T
is the human parse tree (the “gold standard”):
– Recall = (# correct constituents in P) / (# constituents in T)
– Precision = (# correct constituents in P) / (# constituents in P)
• Labeled Precision and labeled recall require getting the
non-terminal label on the constituent node correct to
count as correct.
• F1 is the harmonic mean of precision and recall.

87
Computing Evaluation Metrics

Correct Tree T Computed Tree P


S S
VP
VP
Verb NP VP
book Det Nominal Verb NP
the Nominal PP book Det Nominal PP
Noun Prep NP Noun Prep NP
the
flight through Proper-Noun flight through Proper-Noun
Houston Houston
# Constituents: 12 # Constituents: 12
# Correct Constituents: 10
Recall = 10/12= 83.3% Precision = 10/12=83.3% F1 = 83.3%
Treebank Results
• Results of current state-of-the-art systems on the
English Penn WSJ treebank are slightly greater than
90% labeled precision and recall.

89

You might also like