Unit-III

Semantic Parsing: Introduction

• In language understanding is the identification of a meaning representation that is
detailed enough to allow reasoning system to make deduction(the process of reaching a decision
or answer by thinking about the known facts).
• But at the same time, is general enough that it can be used across many domains with little
to no adaptation(not capable of adjusting to new conditions or situations).
• It is not clear whether a final, low-level, detailed semantics representation covering
various applications that use some form of language interface can be achieved or
• An ontology(a branch of metaphysics concerned with the nature and relations of being) can be created that can be
created that can capture the various granularities and aspects of meaning that are
embodied in such that a variety of applications.
• None of these approaches are not created, So two compromise approaches have emerged in
the NLP for language understanding.
• In the first approach, a specific, rich meaning representation is created for a limited
domain for use by application that are restricted to that domain, such as travel
reservations, football game simulations, or querying a geographic database.
• In the second approach, a related set of intermediate-specific meaning representation is
created, going from low-level analysis to a middle analysis, and the bigger understanding task
is divided into multiple, smaller pieces that are more manageable, such as word sense
disambiguation followed by predicate-argument structure recognition.
• Here two types of meaning representations: a domain-dependent, deeper representation
and a set of relatively shallow but general-purpose, low-level, and intermediate
representation.
• The task of producing the output of the first type is often called deep semantic parsing, and
the task of producing the output of the second type is often called shallow semantic parsing.
• The first approach is so specific that porting to every new domain can require anywhere from
a few modifications to almost reworking the solution from scratch.
• In other words, the reusability of the representation across domains is very limited.
• The problem with second approach is that it is extremely difficult to construct a general
purpose ontology and create symbols that are shallow enough to be learnable but detailed
enough to be useful for all possible applications.
• Therefore, an application specific translation layer between the more general representation
and the more specific representation becomes necessary.
 2.Semantic Interpretation
• Semantic parsing can be considered as part of Semantic interpretation, which involves various
components that together define a representation of text that can be fed into a computer to
allow further computations manipulations and search, which are prerequisite for any
language understanding system or application. Here we start with discus with structure of
semantic theory.
• A Semantic theory should be able to:
1. Explain sentence having ambiguous meaning: The bill is large is ambiguous in the sense that
is could represent money or the beak of a bird.
2. Resolve the ambiguities of words in context. The bill is large but need not be paid, the
theory should be able to disambiguate the monetary meaning of bill.
3. Identify meaningless but syntactically well-formed sentence: Colorless green ideas sleep
furiously.
4. Identify syntactically or transformationally unrelated paraphrasers of concept having the
same semantic content.
• Here we look at some requirements for achieving a semantic representation
Structural ambiguity
• Structure means syntactic structure of sentences.
• The syntactic structure means transforming the a sentence into its underlying syntactic
representation and in theory of semantic interpretation refer to underlying syntactic
representation.
 Word Sense
• In any given language, the same word type is used in different contexts and with different
morphological variants to represent different entities or concepts in the world.
• For example, we use the word nail to represent a part of the human anatomy and also to
represent the generally metallic object used to secure other objects.

Entity and Event Relation

• The next important component of semantic interpretation is the identification of various
entities that are sparkled across the discourse using the same or different phrases.
• The predominant tasks have become popular over the years: named entity recognition and
coreference resolution.
 Predicate-Argument Structure
• Once we have the word-sense, entities and events identified, another level of semantics structure
comes into play: identifying the participants of the entities in these events.
• Resolving the argument structure of predicate in the sentence is where we identify which entities
play what part in which event.
• A word which functions as the verb does here, we call a predicate and words which function as
the nouns do are called arguments. Here are some other predicates and arguments:
• Selena slept
argument predicate
Tom is tall
argument predicate
Percy placed the penguin on the podium
argument predicate argument argument
• “Sanchon serves vegetarian food” can be described in FOPC as: Server (Sanchon,
VegetarianFood)
• Generally, this process can be defined as the identification of who did what
to whom,
where, why and how.
 Meaning Representation
• The final process of the semantic interpretation is to build a semantic representation or
meaning representation that can then be manipulated by algorithms to various application ends.
• This process is sometimes called the deep representation.
• The following two examples
 3.System Paradigms
• It is important to get a perspective on the various primary dimensions on which the problem of
semantic interpretation has been tackled.
• The approaches generally fall into the following three categories: 1.System architecture 2.Scope 3.
Coverage.
• System Architectures
a.Knowledge based: These systems use a predefined set of rules or a knowledge base to obtain
a solution to a new problem.
b.Unsupervised: These systems tend to require minimal human intervention to be functional by
using existing resources that can be bootstrapped for a particular application or problem domain.
c.Supervised: these systems involve the manual annotation of some phenomena that appear in a
sufficient quantity of data so that machine learning algorithms can be applied.
d.Semi-Supervised: manual annotation is usually very expensive and does not yield enough data to
completely capture a phenomenon. In such instances, researches can automatically expand the
data set on which their models are trained either by employing machine-generated output directly
or by bootsrapping off an existing model by having humans correct its output.
 2.Scope:
a.Domain Dependent: These systems are specific to certain domains, such as air travel
reservations or simulated football coaching.
b.Domain Independent: These systems are general enough that the techniques can be applicable
to multiple domains without little or no change.
3.Coverage:
a.Shallow: These systems tend to produce an intermediate representation that can then be
converted to one that a machine can base its action on.
b. Deep: These systems usually create a terminal representation that is directly consumed by a
machine or application.
 4.Word Sense
• Word Sense Disambiguation is an important method of NLP by which the meaning of a word
is determined, which is used in a particular context.
• In a compositional approach to semantics, where the meaning of the whole is composed on the
meaning of parts, the smallest parts under consideration in textual discourse are typically the
words themselves: either tokens as they appear in the text or their lemmatized forms.
• Words sense has been examined and studied for a very long time.
• Attempts to solve this problem range from rule based and knowledge based to completely
unsupervised, supervised, and semi-supervised learning methods.
• Very early systems were predominantly rule based or knowledge based and used dictionary
definitions of senses of words.
• Unsupervised word sense induction or disambiguation techniques try to induce the senses or
word as it appears in various corpora.
• These systems perform either a hard or soft clustering of words and tend to allow the tuning of
these clusters to suit a particular application.
• Most recent supervised approaches to word sense disambiguation, usually application-
independent-level of granularity (including small details). Although the output of supervised
approaches can still be amendable to generating a ranking, or distribution, of membership
sense.
• Word sense ambiguities can be of three principal types: i.homonymy ii.polysemy iii.categorial
ambiguity.
• Homonymy defined as the words having same spelling or same form but having different and
unrelated meaning. For example, the word “Bat” is a homonymy word because bat can be an
implement to hit a ball or bat is a nocturnal flying mammal also
• Polysemy is a Greek word, which means “many signs”. polysemy has the same spelling but
different and related meaning.
• Both polysemy and homonymy words have the same syntax or spelling. The main difference
between them is that in polysemy, the meanings of the words are related but in homonymy, the
meanings of the words are not related.
• For example: Bank Homonymy: financial bank and river bank
Polysemy: financial bank, bank of clouds and book bank: indicate collection of
things.
• Categorial ambiguity: the word book can mean a book which contain the chapters
or police register which is used to enter the charges against some one.
text book and note book
• In the above note book belongs to the grammatical category of noun, and text book is verb.
• Distinguishing between these two categories effectively helps disambiguate these two senses.
• Therefore categorical ambiguity can be resolved with syntactic information (part of speech)
alone, but polyseme and homonymy need more than syntax.
• Traditionally, in English, word senses have been annotated for each part of speech separately,
whereas in Chinese, the sense annotation has been done per lemma.
Resources
• As with any language understanding task, the availability of resources is key factor in the
disambiguation of the wors senses in corpora.
• Early work on wors sense disambiguation used machine readable dictionaries or thesaususes
as knowledge sources.
• Two prominent sources were the Longman dictionary of contemporary English (LDOCE) and
Roget’s Thesaurus.
• The biggest sense annotation corpus OntoNotes released through Lissuistic Data Consortium
(LDC).
• The Chinese annotation corpus is HowNet.
 Systems
• Researchers have explored various system architectures to address the sense disambiguation
problem.
• We can classify these systems into four main categories: i. rules based or knowledge ii.
Supervised iii.unsupervised iv. Semisupervised
Rule Based:
• The first generation pf word sense disambiguation systems was primarily based on dictionary
sense definitions.
• Much of this information is historical and cannot readily be translated and made available for
building systems today. But some of techniques and algorithms are still available.
• The simplest and oldest dictionary based sense disambiguation algorithm was introduced by
leak.
• The core of the algorithm is that the dictionary sense whose terms most closely overlap with
the terms in the context.
• Another dictionary based algorithm was suggested Yarowsky.
• This study used Roget’s Thesaurus categories and classified unseen words into one of these 1042
categories based on a statistical analysis of 100 word concordances for each member of each category.
• The method consists of three steps, as shown in Fig.
• The first step is a collection of contexts.
• The second step computes weights for each of the salient words.
• P(w|Rcat) is the probability of a word w occurring in the context of a Roget’s Thesaurus category Rcat.
• P(w|Rcat) |Pr(w) , the probability of a word (w) appearing in the context of a Roget category divided
by its overall probability in the corpus.
• Finally, in third step, the unseen words in the test set are classified into the classified into the
category that has the maximum weight.
 Supervised
• The simpler form of word sense disambiguating systems the supervised approach, which
tends to transfer all the complexity to the machine learning machinery while still requiring
hand annotation tends to be superior to unsupervised and performs best when tested on
annotated data.
• These systems typically consist of a machine learning classifier trained on various features
extracted for words that have been manually disambiguated in a given corpus and the
application of the resulting models to disambiguating words in the unseen test sets.
• A good feature of these systems is that the user can incorporate rules and knowledge in the
form of features.
Classifier:
• Probably the most common and high performing classifiers are support vector machine
(SVMs) and maximum entropy classifiers.
Features: Here we discuss a more commonly found subset of features that have been useful in
supervised learning of word sense.
• Lexical context: The feature comprises the words and lemma of words occurring in the entire
paragraph or a smaller window of usually five wors.
• Parts of speech : the feature comprises the POS information for words in the window
surrounding the word that is being sense tagged.
• Bag of words context: this feature comprises using an unordered set of words in the context
window.
• Local Collections : Local collections are an ordered sequence of phrases near the target word
that provide semantic context for disambiguation. Usually, a very small window of about
three tokens on each side of the target word, most often in contiguous pairs or triplets, are
added as a list of features.
• Syntactic relations: if the parse of the sentence containing the target word is available, then
we can use syntactic features.
• Topic features: The board topic, or domain, of the article that word belongs to is also a good
indicator of what sense of the word might be most frequent.
• Semantic analysis starts with lexical semantics,which studies individual
words’ meanings (i.e., dictionary definitions).
• Semantic analysis then examines relationships between individual words
and analyzes the meaning of words that come together to form a sentence.
• This analysis provides a clear understanding of words in context. For
Example: “The boy ate the apple” defines an apple as a fruit.
“The boy went to Apple” defines Apple as a brand or store.
https://www.youtube.com/watch?v=eEjU8oY_7DE https://www.youtube.com/watch?
v=W7QdqCrX_mY https://www.youtube.com/watch?v=XLvv_5meRNM
https://www.geeksforgeeks.org/understanding-semantic-analysis- nlp/#:~:text=Semantic
%20Analysis%20is%20a%20subfield,process%20to%20us%20as%20humans.
• What is deep parsing in NLP?
In deep parsing, the search strategy will give a complete syntactic structure to a sentence. It is the task of
parsing a limited part of the syntactic information from the given task. It is suitable for complex NLP
applications. It can be used for less complex NLP applications.

• How can semantics be represented in natural language processing systems?

The semantics, or meaning, of an expression in natural language can be abstractly represented as a logical
form. Once an expression has been fully parsed and its syntactic ambiguities resolved, its meaning
should be uniquely represented in logical form.
• What is meant by semantic translation?
Semantic translation is the process of using semantic information to aid in the translation of data in one
representation or data model to another representation or data model.
• Semantic meaning can be studied at several different levels within linguistics. The three major types of
semantics
are formal, lexical, and conceptual semantics
Categories of Semantics
Nick Rimer, author of Introducing Semantics, goes into detail about the two categories of semantics.
"Based on the distinction between the meanings of words and the meanings of sentences, we can
recognize two main divisions in the study of semantics: lexical semantics and phrasal semantics.
Lexical semantics is the study of word meaning, whereas phrasal semantics is the study of the
principles which govern the construction of the meaning of phrases and of sentence meaning out of
compositional combinations of individual lexemes.
A bird's bill, also called a beak, A bird's horny projecting jaws; a bill