CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
Lab 6: CFG & Parse Tree
References:
1. Natural Language Processing with Python, by Steven Bird, Ewan Klein and Edward Loper,
2014.
QUICK REVIEW
CFG has been the most influential grammar formalism for describing language syntax. This is not
because CFG has been generally adopted as such for linguistic description, but rather because most
grammar formalisms are derived from or can somehow be related to CFG. For this reason, CFG is
often used as a base formalism when parsing algorithms are described.
The standard way to represent the syntactic structure of a grammatical sentence is as a syntax tree,
or a parse tree, which is a representation of all the steps in the derivation of the sentence from the
root node. This means that each internal node in the tree represents an application of a grammar
rule.
PRACTICES
Parse Tree 01
import nltk
text2 = nltk.CFG.fromstring("""
S -> NP VP
PP -> P NP
NP -> Det N | PP NP | Det N PP | 'I'
VP -> V NP | VP PP | V
Det -> 'a'
N -> 'book'
V -> 'write'
""")
text1 = nltk.tokenize.word_tokenize("I write a book")
print(text1)
parser = nltk.ChartParser(text2)
for tree in parser.parse(text1):
print(tree)
tree.draw()
Output
Level 3 Asia Pacific University (APU) Page 1 of 10
CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
['I', 'write', 'a', 'book']
(S (NP I) (VP (V write) (NP (Det a) (N book))))
Parse Tree 02
import nltk
groucho_grammar = nltk.CFG.fromstring("""
S -> NP VP
PP -> P NP
NP -> Det N | Det N PP | 'I'
VP -> V NP | VP PP
Det -> 'an' | 'my'
N -> 'elephant' | 'pajamas'
V -> 'shot'
P -> 'in'
""")
sent = ['I', 'shot', 'an', 'elephant', 'in', 'my', 'pajamas']
parser = nltk.ChartParser(groucho_grammar)
for tree in parser.parse(sent):
tree.draw()
print(tree)
Parse Tree 03
import nltk
Level 3 Asia Pacific University (APU) Page 2 of 10
CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
text2 = nltk.CFG.fromstring("""
S -> NP VP
PP -> P NP
NP -> Det N | PP NP | Det N PP
VP -> V NP | VP PP | V
N -> 'Alice' | 'Bob'
V -> 'loves'
Det ->
P ->
""")
text1 = nltk.tokenize.word_tokenize("Alice loves Bob")
print(text1)
print()
parser = nltk.ChartParser(text2)
for tree in parser.parse(text1):
print(tree)
tree.draw()
Parse Tree 04 – Adjective Phrase
The little bear saw the fine fat trout in the brook
Clue:
Level 3 Asia Pacific University (APU) Page 3 of 10
CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
NP DT Nom
Nom Adj N | Adj Adj N
import nltk
text2 = nltk.CFG.fromstring("""
S -> NP VP
PP -> P NP
NP -> Det N | Det N PP | Det Nom | 'the'
VP -> V NP | VP PP
Nom -> Adj N | Adj Adj N
Det -> 'the'
N -> 'bear' | 'trout' | 'brook'
V -> 'saw'
P -> 'in'
Adj -> 'little' | 'fine' | 'fat'
""")
text1 = nltk.tokenize.word_tokenize("the little bear saw the fine
fat trout in the brook")
print(text1)
print()
parser = nltk.ChartParser(text2)
for tree1 in parser.parse(text1):
tree1.draw()
print(tree1)
Level 3 Asia Pacific University (APU) Page 4 of 10
CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
Parse Tree 05 – Adjective Phrase
import nltk
grammar2 = nltk.CFG.fromstring("""
S -> NP VP
NP -> Det Nom | Det N | PropN
Nom -> Adj Nom | N
VP -> V Adj | V NP | V S | V NP PP
PP -> P NP
PropN -> 'Buster' | 'Chatterer' | 'Joe'
Det -> 'the' | 'a'
N -> 'bear' | 'squirrel' | 'tree' | 'fish' | 'log'
Adj -> 'angry' | 'frightened' | 'little' | 'tall'
V -> 'chased' | 'saw' | 'said' | 'thought' | 'was' | 'put'
P -> 'on'
""")
sent = ['the', 'angry', 'bear', 'chased', 'the', 'frightened',
'little', 'squirrel']
parser = nltk.ChartParser(grammar2)
for tree in parser.parse(sent):
tree.draw()
print(tree)
Level 3 Asia Pacific University (APU) Page 5 of 10
CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
Parse Tree 06 – Adverb Phrases (AdvP)
E.g.: Ken snores very loudly
import nltk
sentence = "Ken snores very loudly"
gram = nltk.CFG.fromstring("""
S -> NP VP
NP -> N
VP -> V ADV
N -> 'Ken'
V -> 'snores'
DEG -> 'very'
ADV -> DEG ADV | 'loudly'
""")
Level 3 Asia Pacific University (APU) Page 6 of 10
CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
token = nltk.tokenize.word_tokenize(sentence)
print(token)
parser = nltk.ChartParser(gram)
for tree in parser.parse(token):
print(tree)
tree.draw()
import nltk
from nltk.tokenize import word_tokenize
sents = [
"unfortunately the cat killed the mouse",
"the cat unfortunately killed the mouse",
"the cat killed the mouse unfortunately"
]
grammar = nltk.CFG.fromstring("""
S -> ADV NP VP | NP VP
NP -> DT N
VP -> ADV VP | VP ADV | V NP
DT -> 'the'
N -> 'cat' | 'mouse'
Level 3 Asia Pacific University (APU) Page 7 of 10
CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
V -> 'killed'
ADV -> 'unfortunately'
""")
parser = nltk.ChartParser(grammar)
for sent in sents:
print(sent)
for tree in parser.parse(word_tokenize(sent)):
tree.draw()
print(tree)
Unfortunately the cat killed the mouse
The cat unfortunately killed the mouse
Level 3 Asia Pacific University (APU) Page 8 of 10
CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
The cat killed the mouse unfortunately
Draw Parse Tree using COLAB
Run the following set of codes to set the COLAB platform
import nltk
nltk.download('punkt')
### CREATE VIRTUAL DISPLAY ###
!apt-get install -y xvfb # Install X Virtual Frame Buffer
import os
os.system('Xvfb :1 -screen 0 1600x1200x16 &') # create virtual display w
ith size 1600x1200 and 16 bit color. Color can be changed to 24 or 8
os.environ['DISPLAY']=':1.0' # tell X clients to use our virtual DISPLAY
:1.0.
%matplotlib inline
### INSTALL GHOSTSCRIPT (Required to display NLTK trees) ###
!apt install ghostscript python3-tk
Example Program ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
import nltk
from IPython.display import display
text2 = nltk.CFG.fromstring("""
S -> NP VP
PP -> P NP
NP -> Det N | PP NP | Det N PP | 'I'
VP -> V NP | VP PP | V
Det -> 'a'
N -> 'book'
V -> 'write'
Level 3 Asia Pacific University (APU) Page 9 of 10
CT107-3-3-TXSA - Text Analytics and Sentiment Analysis Parse Tree
""")
text1 = nltk.tokenize.word_tokenize("I write a book")
print(text1)
parser = nltk.ChartParser(text2)
for tree in parser.parse(text1):
display(tree) # tree.draw()
# print(tree)
Level 3 Asia Pacific University (APU) Page 10 of
10
