Chapter 1

Natural language Processing in Prolog

Natural language systems in Prolog Natural language processing in Prolog started with Prolog itself, when Colmerauer in the early 1970s developed a Prolog-based natural language system called metamorphosis grammar (Colmerauer,1978). The ideas developed here have since been applied in various logic grammar formalisms and systems, e.g. by Dahl (1977), who made a natural language system for Spanish. The most inuential formalism, Denite Clause Grammar, was developed by Warren and Pereira and applied to make a knowledge based system CHAT-80 (Pereira,1980) for geography with a natural language interface. CHAT80 can answer the following question in extensio: " Which country bordering the Mediterranean borders a country that is bordered by a country whose population exceeds the population of India " The solution, Turkey, was found within 405 milliseconds (1980).

Also the ORBI expert system for environmental resources (Pereira,L.,1982) is a Prolog based system with very advanced language capabilities:

> Which descriptors of the aptitude intensive agric at point 56,78 that are greater than 2, are equal factor F1 of point 12,95? ....... > And at point 65,78 ? ........ The system will understand that this is an ellipsis, whose meaning is found by replacing point 56,78 in the question above with point 65,78 . Prolog is superbly adapted to the task of natural language processing in all its main aspects: - as a grammar formalism - as a formalism for meaning representation - as a knowledge base query language

Dictionary | | | V -------|Lexical | --|analysis|--> : -------- : : : : : : : Characters :

Clause Semantic Knowledge grammar net base | --<-------| | | | | | V V V V --------------------------|Syntax | |Semantic| | Query | |analysis| --> |analysis|-->|Processing|-> --------- : ---------- : ------------ : : : : : : : : : Answers : : Syntax trees Query expression

Interpreted words FIGURE

The principal model of the translation

The reference model In TUC Systems, there are 4 distinct phases. They were chosen so, because it is considered good engineering practice to separate subproblems into submodules in order to get each module manageable. The architecture is reected in the main Prolog program: tuc_system:lexical(user,Wordlist), syntax(Wordlist,Syntaxtree), semantics(Syntaxtree,Queryexpression), query(Queryexpression). The information in the semantic net is available as global information.

Denite Clause Grammars We shall briey introduce the reader to Denite Clause Grammars. The sentence to anlyse is John loves Mary We can let the text John loves Mary be represented as facts concerning numberd points in the text, and which words that connect them txt(john,0,1). txt(loves,1,2). txt(mary,2,3). A context free grammar in DCG format could look like s --> np,vp. np --> proper_noun. vp --> verb,proper_noun. verb --> text(loves). proper_noun --> txt(john). proper_noun --> txt(mary). This text is automatically translated into a set of Prolog clauses s(X0,X2) :- np(X0,X1),vp(X1,X2). np(X0,X1) :- proper_noun(X0,X1). vp(X0,X1,2) :- verb(X0,X1),np(X1,X2) proper_noun(X0,X1) :-txt(john,X0,X1). proper_noun(X0,X1) :-txt(mary,X0,X1). verb(X0,X1) :-txt(loves,X0,X1).

s(X0,X2) :- np(X0,X1),vp(X1,X2). np(X0,X1) :- proper_noun(X0,X1). vp(X0,X1,2) :- verb(X0,X1),np(X1,X2) proper_noun(X0,X1) :-txt(john,X0,X1). proper_noun(X0,X1) :-txt(mary,X0,X1). verb(X0,X1) :-txt(loves,X0,X1). Now, the analysis is left to Prolog as a combinational puzzle: ?-s(0,3). Yes, because

np(0,1) because proper_noun(0,1) because txt(john,0,1) vp(1,3) because verb(1,2) because txt(loves,1,2) np(2,3) because proper_noun(2,3) because txt(mary,2,3)

The real Denite Clause Grammar of Prolog uses pair of lists in stead of pair of points, and one generic txt predicate. txt(Word,[Word|Rest],Rest). Instead of txt(Word), we can use [Word] The empty phrase is accepted by [], which corresponds to txt(_,X,X) and the txt(W,X,Y) predicate is usually called as C(X,W,Y)

A proper understanding of the DCG formalism involves list handling in Prolog, but that is not necessary for the understanding at the moment. DCG grammars is translated into executable Prolog code, and as such is executed by a top down, left to rigt manner with unication and backtracking. As such, left recursive rules spells trouble and is strongly disfavoured.

Attributes The mechanism for expanding nonterminals in the grammar into prolog goals extends to terms with parameters in general. Thus, if the DCG grammar was changed to s(k(U,V)) --> np(U),vp(V). np(pling) --> proper_noun. vp(plong) --> verb,proper_noun. proper_noun --> txt(john). proper_noun --> txt(mary). this would be expanded to s(k(U,V),X0,X2) np(pling,X0,X1) vp(plong,X0,X2) :- np(U,X0,X1),vp(V,X1,X2). :- proper_noun(X0,X1). :- verb(X0,X1),proper_noun(X1,X2)

proper_noun(X0,X1) :-txt(john,X0,X1). proper_noun(X0,X1) :-txt(mary,X0,X1). verb(X0,X1) :-txt(loves,X0,X1).

Attribute Grammar The arguments in the nonterminals of the DCG grammar follows the translation to Prolog. The unication mechanism of Prolog applies also for the codes that are expanded by Prolog. Such arguments can then be used in a manner that resembles and supercedes conventional attribute grammars as known in compiler techniques. In natural language analysis, attributes are used for at least 3 different purposes: to parametrize the syntax analysis for agreement checks to create a parse tre or a syntaxtree to create the result in terms of a semantics Let us make an example, where we implement an agreement check.

Agreement Agreement in English is concerned with the three features: Case: Nominative: he,she Accusative: him,her Genitive: his,her Number: Singular: A man | Plura: They,Two men Subcategorization of verbs Some verbs take an object, like kiss,kill John kissed Mary. Other verbs, like sing and die, dont take any objects. John sings, Mary dies. These features have to agree in correct English. For example, the starred (*) sentences are not correct John They They Mary sings. die. kill Mary. denied it He met. * They dies * They die them. * Her loves John * They kiss they. *

Some verb forms must agree in number with the subject. The subject of a phrase is usually in nominative case, while the object is in accusative phase. Furthermore, in English, only pronouns are modied in the accusative case, he-him, I - me). In order to keep a record of these features, we let there be parameters to the categories of the grammars: s --> np(Num,nom),vp(Num). np(sin,_) --> proper_noun. np(Num,Case) --> pronoun(Num,Case). vp(Num) --> verb(intrans,Num). vp(Num) --> verb(trans,Num),np(Num,acc). proper_noun --> [john]. proper_noun --> [mary]. pronoun(sin,nom) --> [he]. pronoun(sin,acc) --> [him]. pronoun(sin,nom) --> [she]. pronoun(sin,nom) --> [her]. verb(intrans,sin) --> [dies]. verb(intrans,plu) --> [die]. verb(trans,sin) --> [kisses]. verb(trans,plu) --> [love].

10

Generating syntax trees The attributes of clauses can be made to make Syntax trees. Let us suppose we want to make a syntax tree of the sentence John loves Mary s(s(NP,VP)) --> np(NP),vp(VP). np(np(N)) --> proper_noun(N). vp(vp(V,NP)) --> verb(V),np(NP). proper_noun(proper_noun(john)) --> [john]. proper_noun(proper_noun(mary)) --> [mary].

?- s(SyntS,[john,loves,mary],[]). SyntS = s(np(proper_noun(john), vp(verb(loves),np(proper_noun(mary))))) which can be rendered as a proper parse tree. s np john vp verb loves np proper_noun mary
11

Curly Brackets in DCG Intersparsed between the productions, we can have Prolog predications surrounded by curly brackets. These predications are not expanded into Prolog by augmentation of twin text variables, but are just copied as they are. These can be used to perform tests and other operations on the attributes, which are not already implicit by the unication. One example, written explicitly out is listed below to show the principle, but it stressed that this example is strongly equivalent with the example above using attributes. s(SyntS) --> np(NP),vp(VP),{SyntS=s(NP,VP)}. np(SyntN) --> proper_noun(N),{SyntN=np(N)}. vp(SyntVP) --> verb(V),np(NP),{SyntVP=vp(V,NP)}. proper_noun(SyntN) --> [john], {SyntN=proper_noun(john)}. proper_noun(SyntN) --> [mary], {SyntN=proper_noun(mary)}.

12

Time ies like an arrow As another amusing example, consider the sentence Time flies like an arrow . This innocuous little example is sometimes used to show the futility of trying to analyse natural language sentences, because it has so many interpretations: Time passes very quickly The time is ying with the same speed as an arrow. The time is ying in a way similar to arrows. Some time-related ies are fond of arrows. (as fruit ies like a banana ) Some ies called Time ies are fond of arrows. Take the time of ies quickly ! Take the time of ies in a way similar way as an arrow would do it! Take the time of ies with arrow form ! Take the time of ies in the same way as you would take the time of an arrow ! etc.

13

A grammar for this may not necessarily be unambiguous: Consider the grammar, which we represent in DCG. sentence sentence noun_phrase noun_phrase noun_phrase noun_phrase verb_phrase verb_phrase complement comparison composite_noun --> --> --> --> --> --> --> --> --> --> --> noun_phrase,verb_phrase. verb,noun_phrase,comparison. noun. proper_noun. article,noun. composite_noun. verb,complement. verb,comparison. noun_phrase. comparator,noun_phrase. proper_noun,noun.

verb --> [time]. verb --> [flies]. verb --> [like]. comparator --> [like]. noun --> [time]. noun --> [flies]. noun --> [arrow]. proper_noun --> [time]. article --> [an].

14

?-sentence(Parsetree,[time,flies,like,an,arrow],[]). sentence noun_phrase noun time verb_phrase verb flies comparision comparator like noun_phrase article an noun arrow sentence noun_phrase proper_noun time verb_phrase verb flies comparision comparator like noun_phrase article an noun arrow
15

sentence noun_phrase composite_noun proper_noun time noun flies verb_phrase verb like complement article an noun_phrase noun arrow sentence verb time noun_phrase noun flies comparison comparator like noun_phrase article an noun arrow

16

While ambiguity in compiler languages is considered an unacceptable fault of the language itself, we have to live with it in natural language, because it is inherently ambiguous at all levels. A later semantic analysis will know the ambiguity of like, and can check in a semantic net what attributes are meaningful, i.e Time can y (in a certain sense) A y has no attribute which can have a value Time. It is not relevant to take the time of ies. etc. It is important to nd the right sense of the word. This is often possible by taking into account the grammar, and also the semantic classes of the nouns involved. In general, nding the right sense also gives the right semantic code, i.e. we can safely forward the problem to the pragmatic processing.

17

Extending Denite Clause Grammars So far, the context free grammars that we started with has been extended with attributes and unication to cope for both syntax trees an logical semantics. However, there are syntactic constructions that fall outside context free grammars, even when attributes are involved. One such phenomenon is movement, which has been mentioned earlier. We have seen one example of relative clause which would cover the sentence

A dog chases a cat that catches a mouse that squeak However, it would not cover the sentence, which has an equivalent meaning

A mouse that a cat that a dog chases catches squeak There are numerous variations of phrases that would demand a separate grammar rule, however the principle is very simple if we allow certain phrases to be moved into places where they are missing, gaps. To understand what is going on, we observe that the sentence contains certain places where a noun phrase is missing, because it has been moved to another place. A mouse(1) that a cat catches (1) squeaks

Here, we imagine a gap (1) is where the ller (mouse(1)) should logically t in, i.e. A mouse(1) squeaks and a cat(2) cathes (1) and a dog chases (2).
18

The rules for these movements can be formulated by an extension of the grammar. statement --> noun_phrase,verb_phrase. noun_phrase --> determiner,noun,rel_clause. rel_clause --> [that], statement - noun_phrase. rel_clause --> [] determiner --> [a]. noun --> [mouse] | [cat] | [dog] verb_phrase --> intrans_verb . verb_phrase --> trans_verb, noun_phrase.

19

The meaning of the phrase statement-np (1). is any statement where a noun phrase is missing. Since (A) statement --> np(2) verb_phrase

the np(1) may be missing in one of the right sides of statement. That means that (A) implicitlitly denes two new grammar rules: (A1) (A2) statement --> np(2)-np(1) verb_phrase statement --> np(2) verb_phrase- np(1)

The same principle applies recursively to the newly introduced rules. For instance, np(2)-np(1) may be matched by an empty word, or a complex noun phrase2 with a missing simpler noun phrase(1)

20

To see that the above statement is matched by this grammar is shown by the sketched parse tree. statement np(1) determiner [a] noun [mouse] rel_clause [that] statement-np(1) np(2) determiner [a] noun [cat] rel_clause [] verb_phrase-np(1) trans_verb [catches] np-np(1) [] verb_phrase intrans_verb [squeaks] The actual implementation is done by an extension of the DCG mechanism. In short, it involves the introduction of an extraposition stack, that holds the llers, ready to be lled in whenever the grammar expects a noun phrase, but only nds a gap. Details of this scheme, with implementations will be found in an appendix.

21

Verb-free language "To be or to have, that is the question" i.e. a language where only the verbs be and have were taken as semantically primitive, while all other verbs had to be dened in terms of these. Thus, questions may be reformulated as:

not but not but

"When were you born" "What is your date of birth" "Where do you live ?" "What is your address ?"

not but

"I walked to the course today " the explicit formulation of the conclusion of the state of matter you want the listener to draw: "I am tired" "My house is close to the lecture room" "Today my car is broken" "My leg is healed".

or or or

22

Disambiguation As we have demonstrated, the interpretation of the words are ambiguous, and in TUC, each word is actually presented with the list of possible interpretations. The disambiguation can be done as a separate process, or, as in TUCs case, the disambiguation is done by heavily relying on the semantic net. Also because Prolog parses using backtracking, it will try the various possiblities, but will accept one interpretation as soon as it ts sematically.

23

Semantic analysis with semantic nets

thing / | ako / | / ako / | course | person / | ako / | / | ako hero | | participant | | ako | researcher FIGURE

--has_a--> address

A semantic net for researchers

" What is the address of the course ?" The sentence is syntactically correct, and should be recognised as a question which deserves an answer. However, the question is semantically wrong, because an address is not defined for a course, so the answer
24

should be an indication of the error. " What is the address of the participants ?"

address is defined as an attribute of participant, so there is no problem

" What is the address of the researchers ?" address is not directly defined for researchers, but researchers are a kind of participants, and as such inherit the address attribute

"

What is the address

of the persons ?"

persons have no address attribute, and do not inherit any from a super class. So, the query implicitly contains a qualification that can be formulated as: " What is the address of the kind of persons that have an address ?"

These kinds of persons are the participants (and maybe others).

25

Verbed Language When we introduce verbs into the language, we also need to check that the verbs t together with the nouns. This is especially important for two reasons: we dont want to allow meaningless questions to be answered, even if they are grammatical Example: which yellow nations pass midnight on the city. we need to nd the right interpretation among several. This is called disambiguation Examples: 1 The man saw the dog in the park with a telescope 2 The dog saw the man in the park with a telescope. y It is obvious to us that the phrase with a telescope is a verb complement in [1] but a noun modier in [2]. We can say this, because dogs do not have telescopes, and dogs can not see with a telescope. Such information is to be stored in a semantic net together with the class ans attribute hierarchy. In TUC, these declarations are stored in a tabular form which can be given by some examples:

26

object ako thing person ako animate. animal ako animate. park ako place. man ako person, dog ako animal. telescope ako object.

tv_templ(see,animate,thing). v_compl(see,person,with,telescope). n_compl(with,person,telescope).

27

EXERCISES

1.

Make a Prolog dialogue:

program to understand the follow

All boys are something. Some boys are good. Some boys are bad. Magnus is a boy. Magnus is good. Is Magnus good ? yes Who is good ? Magnus

2. Make a Prolog program that will recognize name and address from a name list, with the same competence as yourself regarding background information, misspelling, truncation and other distortions. 3. Make an attribute

grammar to capture the sentenc

Which country bordering the Mediterranean borders a country that is bordered by a country whose population exceeds the population of Indi 5. Make a natural language system to analyse sentences in Naturally Readable Logic, an translate them to Prolog. Example:
28

a man is the grandfather of a person if this man is the father of another person and this other person is the parent of the person. a man is the parent of a person if this man is the father of the person. a woman is the parent of a person if this woman is the mother of the person. Olav is the father of Harald. Harald is the father of Haakon Harald is the father of Martha

29