(b) There are different types of mapping factors may arise like one-to-one, one-to-many and

many to many.
(c) Some noise or disturbing factors are also there.
(d) The level of interaction of the source components may be complex one.
(e) The problem solver might be unknown about some more complex problems.
(f) The intermediary actions may also be unavailable.

Consider an example of an English sentence which is being used for communication with a
keyword based data retrieval system. Suppose I want to know all about the temples in India. So I
would need to be translated into a representation such as The above sentence is a simple sentence
for which the corresponding representation may be easy to implement. But what for the complex
queries?

Consider the following query.

“Ram told Sita he would not eat apple with her. He has to go to the office”.

This type of complex queries can be modeled with the conceptual dependency representation
which is more complex than that of simple representation. Constructing these queries is very
difficult since more informationare to be extracted. Extracting more information will require
some more knowledge. Also the type of mapping process is not quite easy to the problem solver.
Understanding is the process of mapping an input from its original form to a more useful one.
The simplest kind of mapping is “one-toone”.
In one-to-one mapping each different problems would lead to only one solution. But there are
very few inputs which are one-to-one. Other mappings are quite difficult to implement. Many-to-
one mappings are frequent is that free variation is often allowed, either because of the physical
limitations of that produces the inputs or because such variation simply makes the task of
generating the inputs.
Many to one mapping require that the understanding system know about all the ways that a target
representation can be expressed in the source language. One-to-many mapping requires a great
deal of domain knowledge in order to make the correct choice among the available target
representation.
The mapping process is simplest if each component can be mapped without concern for the other
components of the statement. If the number of interactions increases, then the complexity of the
problem will increase. In many understanding situations the input to which meaning should be
assigned is not always the input that is presented to the under stander.
Because of the complex environment in which understanding usually occurs, other things often
interfere with the basic input before it reaches the under stander. Hence the understanding will be
more complex if there will be some sort of noise on the inputs.

Natural Language Processing

Introduction to Natural Language Processing
103
  Language meant for communicating with the world.
 Also, By studying language, we can come to understand more about the world.
 If we can succeed at building computational mode of language, we will have a powerful
tool for communicating with the world.
 Also, We look at how we can exploit knowledge about the world, in combination with
linguistic facts, to build computational natural language systems.
Natural Language Processing (NLP) problem can divide into two tasks:
1. Processing written text, using lexical, syntactic and semantic knowledge of the language
as well as the required real-world information.
2. Processing spoken language, using all the information needed above plus additional
knowledge about phonology as well as enough added information to handle the further
ambiguities that arise in speech.
Steps in Natural Language Processing
Morphological Analysis
 Individual words analyzed into their components and non-word tokens such as
punctuation separated from the words.
Syntactic Analysis
 Linear sequences of words transformed into structures that show how the words relate to
each other.
 Moreover, Some word sequences may reject if they violate the language’s rule for how
words may combine.
Semantic Analysis
 The structures created by the syntactic analyzer assigned meanings.
 Also, A mapping made between the syntactic structures and objects in the task domain.
 Moreover, Structures for which no such mapping possible may reject.
Discourse integration
 The meaning of an individual sentence may depend on the sentences that precede it. And
also, may influence the meanings of the sentences that follow it.
Pragmatic Analysis
 Moreover, The structure representing what said reinterpreted to determine what was
actually meant.
Summary
 Results of each of the main processes combine to form a natural language system.
 All of the processes are important in a complete natural language understanding system.
 Not all programs are written with exactly these components.
 Sometimes two or more of them collapsed.
 Doing that usually results in a system that is easier to build for restricted subsets of
English but one that is harder to extend to wider coverage.

Steps Natural Language Processing

Morphological Analysis
 Suppose we have an English interface to an operating system and the following sentence
typed: I want to print Bill’s .init file.
 The morphological analysis must do the following things:

104
  Pull apart the word “Bill’s” into proper noun “Bill” and the possessive suffix “’s”
 Recognize the sequence “.init” as a file extension that is functioning as an adjective in the
sentence.
 This process will usually assign syntactic categories to all the words in the sentence.
Syntactic Analysis
 A syntactic analysis must exploit the results of the morphological analysis to build a
structural description of the sentence.
 The goal of this process, called parsing, is to convert the flat list of words that form the
sentence into a structure that defines the units that represented by that flat list.
 The important thing here is that a flat sentence has been converted into a hierarchical
structure. And that the structure corresponds to meaning units when a semantic analysis
performed.
 Reference markers (set of entities) shown in the parenthesis in the parse tree.
 Each one corresponds to some entity that has mentioned in the sentence.
 These reference markers are useful later since they provide a place in which to
accumulate information about the entities as we get it.

Semantic Analysis
 The semantic analysis must do two important things:
1. It must map individual words into appropriate objects in the knowledge base or
database.
2. It must create the correct structures to correspond to the way the meanings of the
individual words combine with each other.
Discourse Integration
 Specifically, we do not know whom the pronoun “I” or the proper noun “Bill” refers to.
 To pin down these references requires an appeal to a model of the current discourse
context, from which we can learn that the current user is USER068 and that the only
person named “Bill” about whom we could be talking is USER073.
 Once the correct referent for Bill known, we can also determine exactly which file
referred to.
Pragmatic Analysis
 The final step toward effective understanding is to decide what to do as a result.

105
  One possible thing to do to record what was said as a fact and done with it.
 For some sentences, a whose intended effect is clearly declarative, that is the precisely
correct thing to do.
 But for other sentences, including this one, the intended effect is different.
 We can discover this intended effect by applying a set of rules that characterize
cooperative dialogues.
 The final step in pragmatic processing to translate, from the knowledge-based
representation to a command to be executed by the system.

Syntactic Processing
 Syntactic Processing is the step in which a flat input sentence converted into a
hierarchical structure that corresponds to the units of meaning in the sentence. This
process called parsing.
 It plays an important role in natural language understanding systems for two reasons:
1. Semantic processing must operate on sentence constituents. If there is no syntactic
parsing step, then the semantics system must decide on its own constituents. If
parsing is done, on the other hand, it constrains the number of constituents that
semantics can consider.
2. Syntactic parsing is computationally less expensive than is semantic processing.
Thus it can play a significant role in reducing overall system complexity.
 Although it is often possible to extract the meaning of a sentence without using
grammatical facts, it is not always possible to do so.
 Almost all the systems that are actually used have two main components:
1. A declarative representation, called a grammar, of the syntactic facts about the
language.
2. A procedure, called parser that compares the grammar against input sentences to
produce parsed structures.
Grammars and Parsers
 The most common way to represent grammars is a set of production rules.
 The first rule can read as “A sentence composed of a noun phrase followed by Verb
Phrase”; the Vertical bar is OR; ε represents the empty string.
 Symbols that further expanded by rules called non-terminal symbols.
 Symbols that correspond directly to strings that must found in an input sentence called
terminal symbols.
 Grammar formalism such as this one underlies many linguistic theories, which in turn
provide the basis for many natural language understanding systems.
 Pure context-free grammars are not effective for describing natural languages.
 NLPs have less in common with computer language processing systems such as
compilers.
 Parsing process takes the rules of the grammar and compares them against the input
sentence.
 The simplest structure to build is a Parse Tree, which simply records the rules and how
they matched.
 Every node of the parse tree corresponds either to an input word or to a non-terminal in
our grammar.
 Each level in the parse tree corresponds to the application of one grammar rule.

106
Example for Syntactic Processing – Augmented Transition
Network
Syntactic Processing is the step in which a flat input sentence is converted into a hierarchical
structure that corresponds to the units of meaning in the sentence. This process called parsing.
It plays an important role in natural language understanding systems for two reasons:
1. Semantic processing must operate on sentence constituents. If there is no syntactic
parsing step, then the semantics system must decide on its own constituents. If parsing is
done, on the other hand, it constrains the number of constituents that semantics can
consider.
2. Syntactic parsing is computationally less expensive than is semantic processing. Thus it
can play a significant role in reducing overall system complexity.
Example: A Parse tree for a sentence: Bill Printed the file

The grammar specifies two things about a language:

1. Its weak generative capacity, by which we mean the set of sentences that contained
within the language. This set made up of precisely those sentences that can completely
match by a series of rules in the grammar.
2. Its strong generative capacity, by which we mean the structure to assign to each
grammatical sentence of the language.
Augmented Transition Network (ATN)
 An augmented transition network is a top-down parsing procedure that allows various
kinds of knowledge to incorporated into the parsing system so it can operate efficiently.
 ATNs build on the idea of using finite state machines (Markov model) to parse sentences.
 Instead of building an automaton for a particular sentence, a collection of transition
graphs built.
 A grammatically correct sentence parsed by reaching a final state in any state graph.
 Transitions between these graphs simply subroutine calls from one state to any initial
state on any graph in the network.
 A sentence determined to be grammatically correct if a final state reached by the last
word in the sentence.
 The ATN is similar to a finite state machine in which the class of labels that can attach to
the arcs that define the transition between states has augmented.
Arcs may label with:
 Specific words such as “in’.
 Word categories such as noun.

107
  Procedures that build structures that will form part of the final parse.
 Procedures that perform arbitrary tests on current input and sentence components that
have identified.

Semantic Analysis
 The structures created by the syntactic analyzer assigned meanings.
 A mapping made between the syntactic structures and objects in the task domain.
 Structures for which no such mapping is possible may rejected.
 The semantic analysis must do two important things:
 It must map individual words into appropriate objects in the knowledge base or
database.
 It must create the correct structures to correspond to the way the meanings of the
individual words combine with each other. Semantic Analysis AI
 Producing a syntactic parse of a sentence is only the first step toward understanding it.
 We must produce a representation of the meaning of the sentence.
 Because understanding is a mapping process, we must first define the language into
which we are trying to map.
 There is no single definitive language in which all sentence meaning can describe.
 The choice of a target language for any particular natural language understanding
program must depend on what is to do with the meanings once they constructed.
 Choice of the target language in Semantic Analysis AI
 There are two broad families of target languages that used in NL systems,
depending on the role that the natural language system playing in a larger system:
 When natural language considered as a phenomenon on its own, as for example
when one builds a program whose goal is to read the text and then answer
questions about it. A target language can design specifically to support language
processing.
 When natural language used as an interface language to another program (such as
a db query system or an expert system), then the target language must legal input
to that other program. Thus the design of the target language driven by the
backend program.

Discourse and Pragmatic Processing

To understand a single sentence, it is necessary to consider the discourse and pragmatic context
in which the sentence was uttered.
There are a number of important relationships that may hold between phrases and parts of their
discourse contexts, including:
1. Identical entities. Consider the text:
 Bill had a red balloon. o John wanted it.
 The word “it” should identify as referring to the red balloon. These types of
references called anaphora.
2. Parts of entities. Consider the text:
 Sue opened the book she just bought.
 The title page was torn.
 The phrase “title page” should be recognized as part of the book that was just
bought.

108
 3. Parts of actions. Consider the text:
 John went on a business trip to New York.
 He left on an early morning flight.
 Taking a flight should recognize as part of going on a trip.
4. Entities involved in actions. Consider the text:
 My house was broken into last week.
 Moreover, They took the TV and the stereo.
 The pronoun “they” should recognize as referring to the burglars who broke into
the house.
5. Elements of sets. Consider the text:
 The decals we have in stock are stars, the moon, item and a flag.
 I’ll take two moons.
 Moons mean moon decals.
6. Names of individuals:
 Dev went to the movies.
7. Causal chains
 There was a big snow storm yesterday.
 So, The schools closed today.
8. Planning sequences:
 Sally wanted a new car
 She decided to get a job.
9. Implicit presuppositions:
 Did Joe fail CS101?
The major focus is on using following kinds of knowledge:
 The current focus of the dialogue.
 Also, A model of each participant’s current beliefs.
 Moreover, The goal-driven character of dialogue.
 The rules of conversation shared by all participants.

Statistical Natural Language Processing

Formerly, many language-processing tasks typically involved the direct hand coding of
rules, which is not in general robust to natural-language variation. The machine-learning
paradigm calls instead for using statistical inference to automatically learn such rules through the
analysis of large corpora of typical real-world examples (a corpus (plural, "corpora") is a set of
documents, possibly with human or computer annotations).
Many different classes of machine learning algorithms have been applied to natural-language
processing tasks. These algorithms take as input a large set of "features" that are generated from
the input data. Some of the earliest-used algorithms, such as decision trees, produced systems of
hard if-then rules similar to the systems of hand-written rules that were then common.
Increasingly, however, research has focused on statistical models, which make
soft, probabilistic decisions based on attaching real-valued weights to each input feature. Such
models have the advantage that they can express the relative certainty of many different possible
answers rather than only one, producing more reliable results when such a model is included as a
component of a larger system.
Systems based on machine-learning algorithms have many advantages over hand-produced rules:

109
  The learning procedures used during machine learning automatically focus on the most
common cases, whereas when writing rules by hand it is often not at all obvious where
the effort should be directed.
 Automatic learning procedures can make use of statistical inference algorithms to
produce models that are robust to unfamiliar input (e.g. containing words or structures
that have not been seen before) and to erroneous input (e.g. with misspelled words or
words accidentally omitted). Generally, handling such input gracefully with hand-written
rules—or more generally, creating systems of hand-written rules that make soft
decisions—is extremely difficult, error-prone and time-consuming.
 Systems based on automatically learning the rules can be made more accurate simply by
supplying more input data. However, systems based on hand-written rules can only be
made more accurate by increasing the complexity of the rules, which is a much more
difficult task. In particular, there is a limit to the complexity of systems based on hand-
crafted rules, beyond which the systems become more and more unmanageable.
However, creating more data to input to machine-learning systems simply requires a
corresponding increase in the number of man-hours worked, generally without significant
increases in the complexity of the annotation process.
Spell Checking
Spell checking is one of the applications of natural language processing that impacts billions of
users daily. A good introduction to spell checking can be found on Peter Norvig’s webpage. The
article introduces a simple 21-line spell checker implementation in Python combining simple
language and error models to predict the word a user intended to type. The language model
estimates how likely a given word `c` is in the language for which the spell checker is
designed, this can be written as `P(C)`. The error model estimates the probability `P(w|c)`
of typing the misspelled version `w` conditionally to the intention of typing the correctly
spelled word `c`.The spell checker then returns word `c` corresponding to the highest value of
`P(w|c)P(c)` among all possible words in the language.

Module 3
LEARNING
Learning is the improvement of performance with experience over time.
Learning element is the portion of a learning AI system that decides how to modify the
performance element and implements those modifications.
We all learn new knowledge through different methods, depending on the type of material to be
learned, the amount of relevant knowledge we already possess, and the environment in which the
learning takes place. There are five methods of learning . They are,
1. Memorization (rote learning)
2. Direct instruction (by being told)
3. Analogy
4. Induction

110