Unit-3 R20-Regulations NLP

UNIT–III Grammars for Natural Language

Grammars for Natural Language, Movement Phenomenon in Language, Handling questions in

Context Free Grammars, Hold Mechanisms in ATNs, Gap Threading, Human Preferences in Parsing,
Shift Reduce Parsers, Deterministic Parsers.

Movement Phenomena in Language

Many sentence structures appear to be simple variants of other sentence structures. In some cases,
simple words or phrases appear to be locally reordered; sentences are identical except that a phrase
apparently is moved from its expected position in a basic sentence. This section explores techniques
for exploiting these generalities to cover questions in English.

As a starting example, consider the structure of yes/no questions and how they relate to their
assertional counterpart. In particular, consider the following examples:

Jack is giving Sue a back rub.

He will run in the marathon next year.
Is Jack giving Sue a back rub?
Will he run in the marathon next year?
As you can readily see, yes/no questions appear identical in structure to their assertional
counterparts except that the subject NPs and first auxiliaries have swapped positions. If there is no
auxiliary in the assertional sentence, an auxiliary of root "do", in the appropriate tense, is used:

John went to the store. Henry goes to school every day.

Did John go to the store? Does Henry go to school every day?

SEAGI-NB 1
Unit-3 R20-Regulations NLP

Taking a term from linguistics, this rearranging of the subject and the auxiliary is called subject-aux
inversion.

On the other hand, if you are interested in how it is done, you might ask one of the following
questions:

How will the fat man put the book in the corner?
In what way will the fat man put the book in the corner?

If you are interested in other aspects, you might ask one of these questions:
What will the fat man angrily put in the corner?
Where will the fat man angrily put the book?
In what corner will the fat man angrily put the book?
What will the fat man angrily put the book in?

What will the fat man angrily put the book in? Each question has the same form as the original
assertion, except that the part being questioned is removed and replaced by a wh-phrase at the
beginning of the sentence. In addition, except when the part being queried is the subject NP, the
subject and the auxiliary are apparently inverted, as in yes/no questions. This similarity with yes/no
questions even holds for sentences without auxiliaries. In both cases, a "do" auxiliary is inserted:
I found a bookcase.
Did I find a bookcase?
What did I find?

The term movement arose in transformational grammar (TG). TG posited two distinct levels of
structural representation: surface structure, which corresponds to the actual sentence structure,
and deep structure. A CFG generates the deep structure, and a set of transformations map the deep
structure to the surface structure.
For example, the deep structure of "Will the cat scratch John?" would be:

SEAGI-NB 2
Unit-3 R20-Regulations NLP

The yes/no question is then generated from this deep structure by a transformation expressed
schematically as follows

Handling Questions in Context-Free Grammars

Definition:
A context-free grammar consists of a set of rules or productions, each of which expresses the
ways that symbols of the language can be grouped and ordered together, and a lexicon of
words and symbols.

Context Free Grammar (CFG) - Formal Definition

Context-free grammar G is a 4-tuple.
G = (V, T, S, P)
These parameters are as follows;
 V – Set of variables (also called as Non-terminal symbols)
 T – Set of terminal symbols (lexicon)
o The symbols that refer to words in a language are called terminal symbols.
o Lexicon is a set of rules that introduce these symbols.
 S – Designated start symbol (one of the non-terminals, S ∈ V)
 P – Set of productions (also called as rules).
o Each rule in P is of the form A → s, where
o A is a non-terminal (variable) symbol.
o Each rule can have only one non-terminal symbol on the left hand side of the
rule.
o s is a sequence of terminals and non-terminals. It is from (T U V)*, infinite set
of strings.

SEAGI-NB 3
Unit-3 R20-Regulations NLP

A grammar G generates a language L.

Example context-free grammar
G = (V, T, S, P)
V = {S, NP, VP, PP, Det, Noun, Verb, Aux, Pre}
T = {‘a’, ‘ate’, ‘cake’, ‘child’, ‘fork’, ‘the’, ‘with’}
S=S
P = { S → NP VP
NP → Det Noun | NP PP
PP → Pre NP
VP → Verb NP
Det → ‘a’ | ‘the’
Noun → ‘cake’ | ‘child’ | ‘fork’
Pre → ‘with’
Verb → ‘ate’}

Some notes:

Note 1: In P, pipe symbol (|) is used to combine productions into single representation for
productions that have same LHS. For example, Det → ‘a’ | ‘the’ derived from two rules Det
→ ‘a’ and Det → ‘the’. Yet it denotes two rules not one.
Note 2: The production highlighted in red are referred as grammar, and green are referred
as lexicon.
Note 3: NP – Noun Phrase, VP – Verb Phrase, PP – Prepositional Phrase, Det – Determiner,
Aux – Auxiliary verb
Sample derivation:
S → NP VP
→ Det Noun VP
→ the Noun VP
→ the child VP
→ the child Verb NP
→ the child ate NP
→ the child ate Det Noun
→ the child ate a Noun
→ the child ate a cake

From this derivation, we can understand that the sentence ‘the child ate a cake’ is a valid
and accepted sentence by the grammar G. in other words, the sentence is part of the
language produced by G.

SEAGI-NB 4
Unit-3 R20-Regulations NLP

The main goal is to extend a context-free grammar minimally so that it can handle questions. We
want to reuse as much of the original grammar as possible. For yes/no questions, this is easily done.
We can extend following Grammar with one rule that allows an auxiliary before the first NP and
handles most examples:

Grammar: A simple Grammar in abbreviated form

S [+inv] -> (AUXAGR ?a SUBCAT ?v) (NP AGR ?a) (VP VFORM ?v)
This enforces subject-verb agreement between the AUX and the subject NP, and ensures that the VP
has the right VFORM to follow the AUX. This one rule is all that is needed to handle yes/no
questions, and all of the original grammar tor assertions can be used directly for yes/no questions.

An algorithm for automatically adding GAP features to a grammar. Note that it does not modify any
rule that explicitly sets the GAP feature already, allowing the grammar designer to introduce rules
that do not follow the conventions encoded in the algorithm. In particular, the rule for subject-aux
inversion cannot allow the gap to propagate to the subject NP.

An algorithm for adding GAP features to grammar

SEAGI-NB 5
Unit-3 R20-Regulations NLP

Using this procedure, a new grammar can be created that handles gaps. All that is left to do is
analyze where the fillers for the gaps come from. In wh questions, the fillers are typically
NPs or PPs at the start of the sentence and are identified by a new feature WH that identifies
a class of phrases that can introduce questions. The WH feature is signaled by words such as
who, what, when, where, why, and how (as in how many and how carefully). These words
fall into several different grammatical categories, as can be seen by considering what type of
phrases they replace. In particular, who, whom, and what can appear as pronouns and can be
used to specify simple NPs:

Who ate the pizza?

What did you put the book in?
The words "what" and "which" may appear as determiners in noun phrases, as in
What book did he steal?
Words such as "where" and "when" appear as prepositional phrases:
Where did you put the book?
The word "how" acts as an adverbial modifier to adjective and adverbial phrases:
How quickly did he run?
Finally, the word "whose" acts as a possessive pronoun:
Whose book did you find?
A simple NP and PP grammar handling wh-words

A simple NP and PP grammar handling wh-word

SEAGI-NB 6
Unit-3 R20-Regulations NLP

Hold Mechanisms in ATNs

What are Augmented Transition Networks?
Augmented Transition Networks (ATNs) are a type of transition network used for parsing
sentences in natural language processing. They extend finite state machines by
incorporating recursive procedures and registers, allowing for more sophisticated parsing
capabilities. ATNs can capture hierarchical structures in language, making them capable of
representing complex syntactic constructs.

Need of Augment Transition Networks

Natural languages pose complexities that cannot be handled by traditional transition
networks designed for modeling regular languages only. In this regard, ATNs introduce
augmented features which can store and manipulate extra information as well as permitting
recursive transitions into these networks thereby making them capable of dealing with some
context sensitive as well as context free aspects of natural language.

Features can also be added to a recursive transition network to produce an aug mented
transition network (ATN). Features in an ATN are traditionally called registers. Constituent
structures are created by allowing each network to have a set of registers. Each time a new
network is pushed, a new set of registers is created. As the network is traversed, these
registers are set to values by actions associated with each arc. When the network is popped,
the registers are assem bled to form a constituent structure, with the CAT slot being the
network name.

Below Grammar is a simple NP network. The actions are listed in the table below the
network. ATNs use a special mechanism to extract the result of following an arc. When a
lexical arc, such as arc 1, is followed, the constituent built from the word in the input is put
into a special variable named "*". The action

DET := *

then assigns this constituent to the DET register. The second action on this arc,

AGR := AGR*

SEAGI-NB 7
Unit-3 R20-Regulations NLP

assigns the AGR register of the network to the value of the AGR register of the new word
(the constituent in "*").

Agreement checks are specified in the tests. A test is an expression that succeeds if it returns
a nonempty value and fails if it returns the empty set or nil.

A simple NP network

A simple S network

SEAGI-NB 8
Unit-3 R20-Regulations NLP

If a test fails, its arc is not traversed. The test on arc 2 indicates that the arc can be followed
only if the AGR feature of the network has a non-null intersection with the AGR register of
the new word (the noun constituent in "*").

Features on push arcs are treated similarly. The constituent built by traversing the NP
network is returned as the value "*". Thus in Grammar 4.12, the action on the arc from S to
S1,

SUBJ := *

would assign the constituent returned by the NP network to the register SUBJ. The test on arc
2 will succeed only if the AGR register of the constituent in the SUBJ register has a non-null
intersection with the AGR register of the new constituent (the verb). This test enforces
subject-verb agreement.

SEAGI-NB 9
Unit-3 R20-Regulations NLP

With the lexicon in Section, the ATN accepts the following sentences:
The dog cried.
The dogs saw Jack.
Jack saw the dogs.
Consider an example. A trace of a parse of the sentence "The dog saw Jack" is shown in
Figure 4.13. It indicates the current node in the network, the current

word position, the arc that is followed from the node, and the register manipulations that are
performed for the successful parse. It starts in the S network but moves immediately to the
NP network from the call on arc 4. The NP network checks for number agreement as it
accepts the word sequence The dog. It constructs a noun phrase with the AGR feature plural.
When the pop arc is followed, it completes arc 4 in the S network. The NP is assigned to the
SUBJ register and then checked for agreement with the verb when arc 3 is followed. The NP
"Jack" is accepted in another call to the NP network.

SEAGI-NB 10
Unit-3 R20-Regulations NLP

Gap Threading
A third method for handling gaps combines aspects of both the GAP feature approach and the
hold list approach. This technique is usually called gap threading. It is often used in logic
grammars, where two extra argument.

positions are added to each predicate—one argument for a list of fillers that might be used in
the current constituent, and one for the resulting list of fillers that were not used after the
constituent is parsed. Thus the predicate
s (position-in, position-out, fillers-in, fillers-out)
is true only if there is a legal S constituent between position-in and position-out of the input.
If a gap was used to build the 5, its filler will be present in fillers-in, but not in fillers-out. For
example, an S constituent with an NP gap would correspond to the predicate s(ln, Out, (NP],
nil). In cases where there are no gaps in a constituent, the fillers-in and fillers-out will be
identical.
Consider an example dealing with relative clauses. The rules required are shown in Grammar
5.16. The various feature restrictions that would be needed to enforce agreement and
subcategorization are not shown so as to keep the example simple.
To see these rules in use, consider the parse of the sentence The man who we saw cried in
Figure 5.17. The relative clause is analyzed starting with step 7. Using rule 9, the word who
is recognized as a relative pronoun, and the variable Filler is bound to the list [NP]. This filler
is then passed into the embedded S (step 9), to the NP (step 10), and then on to the VP (step
12), since it is not used in the NP. From there it is passed to the NP predicate in step 14,
SEAGI-NB 11
Unit-3 R20-Regulations NLP

which uses the filler according to rule 10. Note that no other NP rule could have applied at
this point, because the filler must be used since the FillersOut variable is nil. Only rules that
consume the filler can apply. Once this gap is used, the entire NP from positions 1 to 6 has
been found and the rest of the parse is straightforward.

Just as a convenient notation was designed for definite clause grammars, which then could be
simply translated into a PROLOG program, a notation has been designed to facilitate the
specification of grammars that handle gaps. In particular, there is a formalism called
extraposition grammar, which, besides allowing normal context free rules, allows rules of the
form
REL-MARK ... TRACE -> REL-PRO
which essentially says that the constituent REL-MARK, plus the constituent TRACE later in
the sentence, can be rewritten as a REL-PRO. Such a rule violates the tree structure of
syntactic forms and allows the analysis shown in Figure 5.18 of the NP the mouse that the cat
ate. Such rules can be compiled into a grammar using the gap threading technique.

SEAGI-NB 12
Unit-3 R20-Regulations NLP

Human Preferences in Parsing

Generally the people will know simple rules to parse sentences. The Psycholinguists have conducted
many investigations into parsing using a variety of techniques. These studies have revealed some
general principles concerning how people resolve ambiguity.

The most basic result from these studies is that people do not give equal weight to all possible
syntactic interpretations. This can be illustrated by sentences that are temporarily ambiguous,
which cause a conscious feeling of having pursued the wrong analysis, as in the sentence
"The raft floated down the river sank". When you read the word "sank" you realize that the
interpretation you have constructed so far for the sentence is not correct. In the literature,
such sentences are often called "garden-path" sentences, based on the expression about
leading someone down a garden path. Here are a few of the general principles that appear to
predict when garden paths will arise.
Minimal Attachment
The most general principle is called the minimal attachment principle, which states that there
is a preference for the syntactic analysis that creates the least number of nodes in the parse
tree. Thus, given Grammar 6.1, the sentence "The man kept the dog in the house" would be
interpreted with the PP "in the house" modifying the verb rather than the NP "the dog". These
SEAGI-NB 13
Unit-3 R20-Regulations NLP

two interpretations are shown in Figure 6.2. The interpretation with the PP attached to the VP
is derived using rules 1.1, 1.2, and 1.6 and three applications of rule 1.4 for the NPs. The
parse tree has a total of 14 nodes. The interpretation with the PP attached to the NP is derived
using rules 1.1, 1.3, 1.5, and 1.6 and three applications of rule 1.4, producing a total of 15
nodes in the parse tree. Thus this principle predicts that the first interpretation is preferred,
which probably agrees with your intuition.

This principle appears to be so strong that it can cause certain sentences to be almost
impossible to parse correctly. One example is the sentence
We painted all the walls with cracks.
which, against all common sense, is often read as meaning that cracks were painted onto the
walls, or that cracks were somehow used as an instrument to paint the walls. Both these
anomalous readings arise from the PP being attached to the VP (paint) rather than the NP (the
walls). Another classic example is the sentence
The horse raced past the barn fell.
which has a reasonable interpretation corresponding to the meaning of the sentence "The
horse that was raced past the barn fell". In the initial sentence, however, creating a reduced
relative clause when the word "raced" is encountered introduces many more nodes than the
simple analysis where "raced" is the main verb of the sentence. Of course, this second
interpretation renders the sentence unanalyzable when the wordfell is encountered.
Right Association
The second principle is called right association or late closure. This principle states that, all
other things being equal, new constituents tend to be interpreted as being part of the current
constituent under construction (rather than part of some constituent higher in the parse tree).
Thus, given the sentence
George said that Henry left in his car.
The preferred interpretation is that Henry left in the car rather than that George spoke in the
car. Both interpretations are, of course, syntactically acceptable analyses. The two
interpretations are shown in Figure 6.3. The former attaches the PP to the VP immediately
preceding it, whereas the latter attaches the PP to the VP higher in the tree. Thus the right
association principle prefers the former. Similarly, the preferred interpretation for the
sentence "I thought it would rain yesterday" is that yesterday was when it was thought to rain,
rather than the time of the thinking.

Lexical Preferences
In certain cases the two preceding principles seem to conflict with each other. In the sentence
"The man kept the dog in the house", the principle of right association appears to favor the
interpretation in which the PP modifies the dog, while the minimal attachment principle
appears to favor the PP modifying the VP. You might suggest that minimal attachment takes
priority over right association in such cases; however, the relationship appears to be more
complex than that. Consider the sentences
SEAGI-NB 14
Unit-3 R20-Regulations NLP

1. I wanted the dog in the house.

2. I kept the dog in the house.
3. I put the dog in the house.

The PP "in the house" in sentence I seems most likely to be modifying "dog" (al though the
other interpretation is possible, as in the sense "I wanted the dog to be in the house"). In
sentence 2, the PP seems most likely to be modifying the VP (although modifying the NP is
possible, as in "I kept the dog that was in the house"). Finally, in sentence 3, the PP is
definitely attached to the VP, and no alternative reading is possible.

These examples demonstrate that lexical items, in this case the verb used, can influence
parsing preferences. In many cases, the lexical preferences will override the preferences
based on the general principles. For example, if a verb subcategorizes for a prepositional
phrase, then some PP must be attached to the VP. Other PP5 might also be identified as
having a strong preference for attachment within the VP. If neither of these cases holds, the
PP will be attached according to the general principles.

Thus, for the preceding verbs, "want" has no preference for any PPs, whereas "keep" might
prefer PPs with prepositions "in", "on", or "by" to be attached to the VP. Finally, the verb
"put" requires (subcategorizes for) a PP beginning with "in", "on", "by", and so on, which
must be attached to the VP.

SEAGI-NB 15
Unit-3 R20-Regulations NLP

Shift Reduce Parsers

The Shift Reduce parsers are used to improve the efficiency of parsers in NLP. The
uncertainty is passed forward through the parse to the point where the input eliminates all but
one of the possibilities. The efficiency of the technique described in this section arises from
the fact that all the possibilities are considered in advance, and the information is stored in a
table that controls the parser, resulting in parsing algorithms that can be much faster than
described thus far.

These techniques were developed for use with unambiguous context-free grammars -
grammars for which there is at most one interpretation for any given sentence. While this
constraint is reasonable for programming languages, it is clear that there is no unambiguous
grammar for natural language. But these techniques can be extended in various ways to make
them applicable to natural language parsing

Specifying the Parser State

Consider using this approach on the small grammar in Grammar 6.4. The technique involves
predetermining all possible parser states and determining the transitions from one state to
another. A parser state is defined as the complete set of dotted rules (that is, the labels on the
active arcs in a chart parser) applicable at that position in the parse. It is complete in the sense
that if a state contains a rule of the form Y -> ... o X ..., where X is a nonterminal, then all
rules for X are also contained in the state. For instance, the initial state of the parser would
include the rule

S -> o NP VP
as well as all the rules for NP, which in Grammar 6.4 is only
NP -> o ART N

Thus the initial state, S0, could be summarized as follows:

Initial State S0: S -> o NP VP
NP -> o ART N

SEAGI-NB 16
Unit-3 R20-Regulations NLP

In other words, the parser starts in a state where it is looking for an NP to start building an S
and looking for an ART to build the NP. What states could follow this initial state? To
calculate this, consider advancing the dot over a terminal or a nonterminal and deriving a new
state. If you pick the symbol
ART, the resulting state is
State S1: NP -> ART o N
If you pick the symbol NP, the rule is

S -> NP o VP
in the new state. Now if you expand out the VP to find all its possible starting symbols, you
get the following:
State S2: S -> NP o VP
VP -> o AUX V NP
VP -> o V NP
Now, expanding SI, if you have the input N, you get a state consisting of a completed rule:
State S1': NP -> ART N o
Expanding S2, a V would result in the state
State S3: VP -> V o NP
NP -> o ART N
An AUX from S2 would result in the state
State S4: VP -> AUX o V NP
and a VP from 52 would result in the state
State S2': S -> NP VP o
Continuing from state S3 with an ART, you find yourself in state S1 again, as you would also
if you expand from S0 with an ART. Continuing from S3 with an NP, on the other hand,
yields the new state
State S3': VP -> V NP o
Continuing from S4 with a V yields
State S5: VP -> AUX V o NP
NP -> o ART N

SEAGI-NB 17
Unit-3 R20-Regulations NLP

and continuing from S5 with an ART would produce state S1 again. Finally, continuing from
S5 with an NP would produce the state
State S5': VP -> AUX V NP o
Now that this process is completed, you can derive a transition graph that can be used to
control the parsing of sentences, as is shown in Figure 6.5.
A Shift-Reduce Parser
These states can be used to control a parser that maintains two stacks: the parse stack, which
contains parse states (that is, the nodes in Figure 6.5) and grammar symbols; and the input
stack, which contains the input and some grammar symbols. At any time the parser operates
using the information specified for the top state on the parse stack. The states are interpreted
as follows. The states that consist of a single rule with the dot at the far right-hand side, such
as S2',
S -> NP VP o indicate that the parser should rewrite the top symbols on the parse stack
according to this rule. This is called a reduce action. The newly derived symbol (S in this
case) is pushed onto the top of the input stack.
Any other state not containing any completed rules is interpreted by the transition diagram. If
the top input symbol matches an arc, then it and the new state (at the end of the arc) are
pushed onto the parse stack. This is called the shift action. Using this interpretation of states
you can construct a table, called the oracle, that tells the parser what to do in every situation.
The oracle for Grammar 6.4 is shown in Figure 6.6. For each state and possible input, it
specifies the action and the next state. Reduce actions can be applied regardless of the next
input, and the accept action only is possible when the input stack is empty (that is, the next
symbol is the empty symbol e). The parsing algorithm for using an oracle is specified in
Figure 6.7.
Consider parsing "The man ate the carrot". The initial state of the parser is

SEAGI-NB 18
Unit-3 R20-Regulations NLP

SEAGI-NB 19
Unit-3 R20-Regulations NLP

Deterministic Parsers.
A deterministic parser can be built that depends entirely on matching parse states to direct its
operation. Instead of allowing only shift and reduce actions, however, a richer set of actions
is allowed that operates on an input stack called the buffer. (The cat ate the fish).

Rather than shifting constituents onto the parse stack to be later consumed by a reduce action,
the parser builds constituents incrementally by attaching buffer elements into their parent
constituent, an operation similar to feature assignment. Rather than shifting an NP onto the
stack to be used later in a reduction S -> NP VP, an S constituent is created on the parse stack
and the NP is attached to it. Specifically, this parser has the following operations:
o Create a new node on the parse stack (to push the symbol onto the stack)
o Attach an input constituent to the top node on the parse stack
o Drop the top node in the parse stack into the buffer
The drop action allows a completed constituent to be reexamined by the parser, which will
then assign it a role in a higher constituent still on the parse stack. This technique makes the
limited lookahead technique surprisingly powerful.
To get a feeling for these operations, consider the situation in Figure 6.8, which might occur
in parsing the sentence "The cat ate the fish". Assume that the first NP has been parsed and
assigned to the SUBJ feature of the S constituent on the parse stack. The operations
introduced earlier can be used to complete the analysis. Note that the actual mechanism for
deciding what operations to do has not yet been discussed, but the effect of the operations is
shown here to provide intuition about the data structure. The operation
Attach to MAIN-V

would remove the lexical entry for ate from the buffer and assign it to the MAIN-V feature in
the S on the parse stack. Next the operation
Create NP

would push an empty NP constituent onto the parse stack, creating the situation in Figure 6.9.
Next the two operations
Attach to DET

Attach to HEAD

SEAGI-NB 20
Unit-3 R20-Regulations NLP

would successfully build the NP from the lexical entries for "the" and "fish". The input buffer would
now be empty.

The operation Drop pops the NP from the parse stack and pushes it back onto the buffer, creating
the situation in Figure 6.10.

SEAGI-NB 21