Introduction to Semantic Analysis

Semantic Analysis is a subfield of Natural Language Processing (NLP)

that attempts to understand the meaning of Natural Language. Understanding
Natural Language might seem a straightforward process to us as humans.
However, due to the vast complexity
and subjectivity involved
in human
language, interpreting it is quite a complicated task for machines. Semantic
Analysis of Natural Language captures the meaning of the given text while
taking into account context, logical structuring of sentences and grammar roles.

Parts of Semantic Analysis

Semantic Analysis of Natural Language can be classified into two broad parts:

1. Lexical Semantic Amnalysis: Lexical Semantic Amnalysis involves

understanding the meaning of each word of the text individually. It basically
refers to fetching the dictionary meaning that a word in the text is deputed to
carry.
2. Compositional Semantics Analysis: Although knowing the meaning of each
word of the text is essential, it is not sufficient to completely understand the
meaning
of the text.

For example, consider the following

two sentences:

« Sentence 1: Students love GeeksforGeeks.

+ Sentence 2: GeeksforGeeks loves Students.

Although
both these sentences 1 and 2 use the same set of root words {student,

love, geeksforgeeks}, they convey entirely different meanings.

Hence, under Compositional Semantics Analysis, we try to understand how
combinations of individual words form the meaning of the text.

Tasks involved in Semantic Analysis

In order to understand the meaning of a sentence, the following are the major
processes involved in Semantic Analysis:

1. Word Sense Disambiguation

2. Relationship Extraction

Word Sense Disambiguation:

In Namwral Language, the meaning of a word may vary as per its usagein
sentences and the context of the text. Word Sense Disambipguation involves
interpreting the meaning of a word based upon the context of its occurrence in
a text.

For example,
the word 'Bark'
may mean 'the sound made by a dog'
or 'the
outermost
layer of a tree.’

Likewise,
the word 'rock’ may mean ' a stone'
or ' a genre of music' - hence,
the accurate meaning of the word is highly dependent upon its context and usage
in the text.
Thus, the ability of a machine to overcome the ambiguity invelved in identifying
the meaning of a word based on its usage and context is called Word Sense
Disambipuation.

Relll:mnslup Extraction:

Another important task involved in Semantic Analysis is Relationship

Extracting. It involves firstly identifying various entities present in the sentence
and then extracting the relationships between those entities.
For example, consider the following sentence:

Semantic Analysis is a topic of NLP which is explained on the GeeksforGeeks

blog. The entities involved in this text, along with their relationships, are shown
below.

{ LN %

Semantic Analysis GeeksforGeeks Blog

Elements of Semantic Analysis

Some of the critical elements of Semantic Analysis that must be scrutinized

and taken into account while processing Natural I anguage are:

« Hyponymy: Hyponymys refers to a term that is an instance of a

generic term. Thev can be understood by taking class-object as an
analogy. For example: ' Color' 1s a hypernymy while ' gray’. ' dlug,
‘red', etc, are 1tz hyponyms.
« Homonymy: Homonymy refers to two or more lexical terms with
the same spellings but completely distinct in meaning. For example:
'Rose' might mean ' the past form of rise’ or ' a flower', - same
spelling but different meanings; hence, ' rose' is a homonymy.
« Synonymy: When two or more lexical terms that might be spelt
distinctly have the same or similar meaning, they are called
Synonymy. For example: (Job, Occupation), (Large, Big), (Stop,
Halt).
 Antonymy: Antonymy refers to a pair of lexical terms that have
contrasting meanings - they are symmetric to a semantic axis. For
example: (Day, Night), (Hot, Cold), {Large, Small).

Polvsemy: Polysemy refers to lexical terms that have the same

spelling but multiple closely related meanings. It differs from
homonymy becavse the meanings of the terms need not be closely
related in the case of homonymy. For example: 'man' may mean ' the
human species’ or ' a male human' or ' an adult male human' - since
all these different meanings bear a close association, the lexical term
"mar 15 a polvsemy.

Meronomy: Meronomy refers to a relationship wherein one lexical

term is a constituent of some larger entity. For example: ' Whee! 1s a
meronym of ' dutomobile

Meaning Representation
‘While, as humans, it is pretty simple for us to understand the meaning of
textual information, it is not so in the case of machines. Thus, machines tend
1o represent the text in specific formats in order to interpret its meaning. This
formal structure that is used to understand the meaning of a text is called
meaning representation.

Basic Units of Semantic System:

In order to accomplish Meaning Representation in Semantic Analysis, it is

vital to understand the building units of such representations. The basic units

of semantic systems are explained below:

1. Entity: An entity refers to a particular unit or individual in specific

such as a person or a location. For example GeeksforGeeks, Dethi,
etc.
 2. Concept: A Concept may be understood as a generalization of
entities. It refers to a broad class of individual units. For example
Learning Pertals, City, Students.
3. Relations: Relations help establish relationships between various
entities and concepts. For example: 'GeeksforGeeks 1s a Learning
Portal', Delhi 1s a City.". etc.
4. Predicate: Predicates represent the verb structures of the sentences.
In Meaning Representation, we employ these basic units to represent textual
information.

Approaches to Meaning Representations:

Now that we are familiar with the basic understanding of Meaning

Representations, here are some of the most popular approaches to meaning
representation:

First-order predicate logic (FOPL)

Semantic Nets
BN

Frames
Conceptual dependency (CD)
Rule-based architecture
=@

Case Grammar

Conceptual Graphs
Semantic Analysis Techniques
Based upon the end goal one is trying to accomplish, Semantic Analysis can
be used in various ways. Two of the most common Semantic Analysis
techniques are:

Text Classification
In-Text Classification, our aim is to label the text according to the insights we
intend to gain from the textual data.
For example:

In Sentiment Analysis, we try to label the text with the prominent

emotion they convey. It is highly beneficial when analyzing customer
reviews for improvement.
In Topic Classification. we try to categories our text into some
predefined categories. For example: Identifying whether a research
paper 1z of Physics, Chemistry or Maths
In Intent Classification, we try to determine the intent behind a text
message. For example: Identifying whether an e-mail received at
customer care service is a query, complaint or request.

Text Extraction

In-Text Extraction, we aim at obtaining specific information from our text.

For Example,

In Keyword Extraction, we try to obtain the essential words that

define the entire document.
In Entity Extraction, we try to obtain all the entities involved in a
document.

Significance of Semantics Analysis

Semantics Analysis is a crucial part of Natural Language Processing (NLP). In

the ever-expanding era of textual information, it is important for organizations
to draw insights from such data to fuel businesses. Semantic Analysis helps
machines interpret the meaning of texts and extract useful information, thus
providing invaluable data while reducing manual efforts.
 Besides, Semantics Analysis is also widely employed to facilitate the
processes of automated answering systems such as chatbots - that answer user
queries without any human interventions.

e Ambiguity
Ambiguity in computational linguistics is a situation where a word or a sentence
may have more than one meaning. That is, a sentence may be interpreted in more
than one way. This leads to uncertainty in choosing the right meaning of a
sentence especially while processing natural langnages by computer.

« Ambiguity is a challenging task in natural language understanding (NLU).

+ The process of handling the ambiguity is called as disambignation.
. Ambiguity presents in almost all the steps of natural language processing.
{Steps of NLP - lexical analysis, syntactic analysis, semantic analysis,
discourse analysis, and pragmatic analysis).

Consider the following sentence for an example;

"Raj tried to reach his friend on the mobile, but he didn't attend”
In this sentence, we have the presence of lexical, syntactic, and anaphoric
ambiguities.
1. Lexical ambiguity - The word " tried"” means "attempted’ not "judged"”
or "tested” . Also, the word "reach " means "establish communication”

not "gain” or "pass” or " sirive’.

2. Syntactic ambiguity - The phrase "on the mobile" attached to "reach"
and thus means "using the mobile”. It is not attached to "friend".
3. Anaphoric ambiguity - The anaphor " he" refers the "friend"” not "Raj".
The following are the types of ambiguity with respect to natural language
processing task;

1. Lexical ambiguity
It is class of ambiguity caused by a word and its multiple senses especially when
the word is part of sentence or phrase. A word can have multiple meanings under
different part of speech categories. Also, under each POS category they may have
multiple different senses. Lexical ambiguity is about choosing which sense of a
particular word under a particular POS category.
In a sentence, the lexical ambiguity is caused while choosing the right sense of a
word under a correct POS category.

For example, let us take the sentence "I saw a ship". Here, the words " saw” and

"ship” would mean multiple things as follows;

Saw = present tense of the verb saw (cut with a saw) OR past tense of the verb
see (perceive by sight) OR a noun saw (blade for cutting) etc. According to
WordNet, the word "saw" 15 defined under 3 different senses in NOUN category
and under 25 different senses in VERB category.
Ship = present tense of the verb ship (transport commercially) OR. present tense
of the verb ship (travel by ship) OR a noun ship (a vessel that carries passengers)
etc. As per WordNet, the word "ship” is defined with 1 sense under NOUN
category and 5 senses under VERB category.
Due to multiple meanings, there arises an ambiguity in choosing the right sense
of "saw" and "ship".

Handling lexical ambiguit

Lexical ambiguity can be handled using the tasks like POS tagping and Word
Sense Disambiguation.
 Word Sense Disambiguation

» We understand that words have different meanings based on the context of its usage in
the sentence. If we talk about human langnages, then they are ambiguous too because
many words can be imterpreted in multiple ways depending upon the context of their
oCCurTence.

» Word sense disambiguation, in namiral language processing (NLF), may be defined as

the ability to determine which meaning of word is activated by the use of word in a
particular context. Lexical ambiguity, syntactic or semantic, is one of the very first
problem that any NLP system faces. Part-of speech (POS) taggers with high level of
acouracy can solve Word's syntactic ambigmity. On the other hand, the problem of
resolving semantic ambiguity is called W5D (word sense disambiguation). Resolving
semantic ambiguity is harder than resolving syntactic ambiguity.

» For example, consider the two examples of the distinct sense that exist for the
word "hass” —

1. 1can hear bass sound.

2. He likes to eat grilled bass.

» The ocourrence of the word bass clearly denotes the distinct meaning. In first sentence,
it means frequency and in second, it means fish Hence, if it would be disambiguated
by WSD then the correct meaning to the above sentences can be assigned as follows —

1. Icanhear bass/frequency sound.

2. He likes to eat grilled bass/fish.

<+ Evaluation of WSD

The evaluation of W3D requires the following two inputs —

A Dictionary

The very first input for evaluation of W5D is dictionary, which is vsed to specify the senses to
be disambigunated.
 Test Corpus

Another input required

by WSD is the high-annotated test corpus that has the target or comrect-
senses. The test corpora can be of two types &minsu;

» Lexical sample — This kind of corpora is used in the system_ where it is required
to disambiguate a small sample of words.
» All-words— This kind of corpora is used in the system, where it is expected to
disambignate all the words in a piece of running text.

% Approaches and Methods to Word Sense Disambiguation

(WsD)

Approaches and methods to WSD are classified according to the source of knowledge used in
word disambiguation.

Let uz now see the four conventional methods to WSD —

Dictionarv-based or Knowledge-based Methods

As the name suggests, for disambiguation, these methods primarily rely on dictionaries,
treasures and lexical knowledge base. They do not use corpora evidences for disambigpation.
The Lesk methodis the seminal dictionary-based method introduced by Michael Leskin 1956.
The Lesk definition, onwhich
the Lesk algorithm is based is "measure overlap between sense
definitions for all words in context”. However, in 2000, Kilgarmiff and Rosensweig gave the
simplified Lesk definition as "measure overlap between sense definitions of word and
current context”, which further means identify the correct sense for one word at 2 time. Here
the current context is the set of words in surrounding sentence or paragraph.

Supervised Methods

For disambiguation, machine learning methods make use of sense-annotated corpora to train.
These methods assume that the context can provide enough evidence on its own to
disambiguate the sense. In these methods, the words knowledge and reasoning are deemed
unnecessary. The context is represented as a set of "features” of the words. It includes the
information about the surrounding words also. Support vector machine and memory-based
 learning are the most successful supervised learning approaches to WSD. These methods rely
on substantial amount of manually sense-tagged corpora, which is very expensive to create.

Semi-supervised Methods

Due to the lack of training corpus, most of the word sense disambiguation algerithms use semi-
supervised learning methods. It is because semi-supervised methods use both labelled as well
as unlabeled data. These methods require very small amount of annotated text and large amount
of plain unannotated text. The technique that is used by semisupervised methods is
bootstrapping from seed data.

Unsupervised Methods

These methods assume that similar senses occur in similar context. That is why the senses can
be induced from text by clustering word occurrences by using some measure of similarity of
the context. This task is called word sense induction or discrimination. Unsupervised methods
hawve great potentialto overcome the knowledge acquisition bottleneck due to non-dependency
on manual efforts.

<+ Applications of Word Sense Disambiguation (WSD)

Word sense disambiguation (WSD) is applied in almost every applicaton of language

technology.

Let us now see the scope of WSD —

Machine Translation

Machine translation or MT is the most obvious application of WSD. In MT, Lexical choice for
the words that have distinct translations for different senses, is done by WS5D. The senses in
MT are represented as words in the target langnage. Most of the machine translation systems
do not use explicit W5D module.

Information Retrieval (IR)

Information retrieval (IR) may be defined as a software program that deals with the
organization, storage, retrieval and evaluation of information from document repositories
particularly textual information. The system basically assists users in finding the information
they required but it does not explicitly return the answers of the questions. WSD is used to
 resolve the ambiguities of the queries provided to IR system. As like MT, current IR systems
do not explicitly use WS5D module and they rely on the concept that user would type enough
context in the query to only retrieve relevant documents.

Text Mining and Information Extraction (TE)

In most of the applications, WSD is necessary to do accurate analysis of text. For example,
‘WSD helps intelligent gathering system to do flagging of the comect words. For example,
medical intelligent system might need flagging of "illegal drugs" rather than "medical drugs”

Lexicography

WSD and lexicography can work together in loop because modern lexicography is
corpusbased. With lexicography, WS5D provides rough empirical sense groupings as well as
statistically significant contextual indicators of sense.

% Difficulties in Word Sense Disambiguation (WSD)

Followings are some difficulties faced by word sense disambiguation (W3D) —

Differences between dictionaries

The major problem of W5D is to decide the sense of the word because different senses can be
very closely related. Even different dictionaries and thesauruses can provide different divisions
of words into senses.

Different algorithms for different applications

Another problem of WS5D is that completely different algorithm might be needed for different
applications. For example, in machine translation, it takes the form of target word selection;
and in information retrieval, a sense inventory is not required.

Inter-judge variance

Another problem of W5D is that W3D systems are generally tested by having their results on
a task compared against the task of human beings. This is called the problem of interjudge
variance.
 Word-sense discreteness

Another difficulty in W5D is that words cannot be easily divided into discrete submeanings.

Discourse Processing

The most difficult problem of Al is to process the namral language by computers or in other
words natural language processing is the most difficult problem of artificial intelligence. If we
talk about the major problems in INLP, then one of the major problems in NLP is discourse
processing — building theories and models of how utterances stick together to form coherent
discourse. Actually, the language always consists of collocated, structured and coherent groups
of sentences rather than isclated and unrelated sentences like movies. These coherent groups
of sentences are referred to as discourse.

«»» Concept of Coherence

Coherence and discourse structure are interconnected in many ways. Coherence, along with
property of good text, is used to evaluate the output quality of natural language generation
system. The question that arises here is what does it mean for a textto be coherem? Suppose
we collected one sentence from every page of the newspaper, then will it be a discourse? Of-
course, not. It is because these sentences do not exhibit coherence. The coherent discourse must
possess the following properties —

Coherence relation between utterances

The discourse would be coberent if it has meaningful connections between its utterances. This
property is called coherence relation. For example, some sort of explanation must be there to
justify the connection between utterances.

Relationship between entities

Another property that makes a discourse coherent is that there must be a certain kind of
relationship with the entities. Such kind of coherence is called entity-based coherence.

4+ Discourse structure
 An important question regarding discourse is what kind of structure the discourse must have.
The answer to this question depends upon the segmentation we applied on discourse. Discourse
segmentations may be defined as determining the types of structures for large discourse. It is
quite difficult to implement discourse segmentation, but it is very important for information
retrieval, text summarization and information extractionkind of applications.

%+ Algorithms for Discourse Segmentation

In this section, we will learn about the algorithms for discourse segmentation. The algorithms
are described below—

Unsupervised Discourse Segmentation

The class of unsupervised discourse segmentation is often represented as linear segmentation.

‘We can understand the task of linear segmentation
with the help of an example. In the example,
there is a task of segmenting the text into multi-paragraph units; the units represent the passage
of the original text. These algorithms are dependent on cohesion that maybe defined as the use
of certain linguistic devices to tie the textual units together. On the other hand, lexicon cohesion
is the cohesion that is indicated by the relationship between two or more words in two units
like the use of synonyms.

Supervised Discourse Segmentation

The earlier method does not have any hand-labeled segment boundaries. On the other hand,
supervised discourse segmentation needs to have boundary-labeled training data. It is very easy
to acquire the same. In supervised discourse segmentation, discourse marker or cue words play
an important role. Discourse marker or cue word is a word or phrase that functions to signal
discourse structure. These discourse markers are domain-specific.

«*. Text Coherence

Lexical repetition is a way to find the structure in a discourse, but it does not satisfy the
requirement of being coherent discourse. To achieve the coherent discourse, we must focus on
coherence relations in specific. As we know that coherence relation defines the possible
connection between utterances in a discourse. Hebb has proposed such kind of relations as
follows —
We are taking two terms 5, and 5, to reprezent the meaning of the two related sentences —

Result

It infers that the state asserted by termS, conld canse the state asserted by 5, . For example, two
statements show the relationship result: Ram was canght in the fire. His skin burned.

Explanation

It infers that the state asserted by S. could cause the state asserted by S.. For example, two
statements show the relationship — Bam fought with Shyam's friend. He was drumlk

Parallel

It mfers p(al,a2....) £ rom assertion of 5, and p(bl.b2....) from assertion 5. Here ai and bi are

similar for all i. For example, two statements are parallel — Ram wanted car. Shyam wanted
money.

Elaboration

It infers the same proposition P from both the assertions — 5, and § For example, two
statements show the relation elaboration: Ram was from Chandigarh. Shyam was from Kerala.

Occasion

It happens when a change of state can be inferred from the assertion of S, final state of which
can be inferred from% and vice-versa. For example, the two statements show the relation
occasion: Ram picked up the book. He gave it to Shyam.

The coherence of entire discourse can also be considered by hierarchical stmcture between
coherence relations. For example, the following passage can be represented as hierarchical
structure—

« 5 —Ram went to the bank to deposit money.

« 5 —He then took a train to Shyvam's cloth shop.
s 5,—He wanted to buy some clothes.
« 5,—He do not have new clothes for party.
« 5:—He alzo wanted to talk to Shyam regarding his health
 Occasion (e;e3)

S () Explanation (e:)

52 (&) Parallel (es;es) ‘

Explanation (e3)

/
53 (&) Se (&4)

Reference Resolution

Interpretation of the seatences from any discourse is another important task and to achieve this
we need to know who or what entityis being talked about. Here, interpretation reference is the
key element Reference
may be defined as the linguistic expression to denote an entity or
individual. For example, in the passage, Ram, the managerof ABC bank, saw his friend Shyam
at a shop. He went to meet him, the linguistic expressions like Ram, His, He are reference.

On the same note, reference resolution maybe defined as the task of determining what entities
are referred to by which linguistic expression.

<+ Terminology Used in Reference Resolution

 We use the following terminologies in reference resolution —

« Referring expression — The natural language expression

that is usedto perform
reference is called a referring expression. For example, the passage used abowve
is a referring expression.
» Referent — It is the entity that 15 referred. For example, in the last given example
Ram is a referent.
+ Corefer— When two expressions are used to refer to the same entity, they are
called corefers. For example,
Ram and he are corefers.
+» Antecedent— The term has the licenseto use another term. For example, Ram is
the antecedent of the reference he.
+ Anaphora & Anaphoric — It may be defined 25 the reference to an entity that
has been previously introduced into the sentence. And, the referring expression
is called anaphoric.
» Discourse model— The model that contains the representations of the entities
that have been referred to in the discourse and the relationship they are engaged
in.

«+* Types of Referring Expressions

Let us now see the different types of referring expressions. The five types of referring
expressions are described below —

Indefinite Noun Phrases

Such kind of reference represents the entities that are new to the hearer into the discourse
context. For example — in the sentence Ram had gone around ene day to bring im some food
— zome i an indefinite reference.

Definite Noun Phrases

Opposite to above, such kind of reference represents the entities that are not new or identifiable
to the hearer into the discourse context. For example, in the sentence - I used to read The Times
of India - The Times of India is a definite reference.
 Pronouns

It is a form of definite reference. For example, Ram laughed as loud as he could. The
word he represents pronoun referring expression.

Demonstratives

These demonstrate and behave differently

than simple definite pronouns. For example, this and
that are demonstrative pronouns.

Names

It is the simplest type of referring expression. It can be the name of a person, organization and
location also. For example, in the above examples, Ram is the name-refereeing expression.

++ Reference Resolution Tasks

The two reference resolution tazks are described below.

Coreference Resolution

It is the task of finding referring expressions in a text that refer to the same entity. In simple
words, it is the task of finding corefer expressions. A set of coreferring expressions are called
coreference chain. For example - He, Chief Manager and His - these are referring expressions
in the first passage given as example.

Constraint on Coreference Resolution

In English, the main problem for coreference resolution is the pronoun it. The reason behind
this is that the pronoun it has many uses. For example, it can refer much like he and she. The
pronounit also refers to the things that do not refer to specific things. For example, It's raining.
It is really good.

Pronominal Anaphora Resolution

Unlike the coreference resolution, pronominal anaphora resclution may be defined as the task
of finding the antecedent for a single pronoun. For example, the pronoun is his and the task of
pronominal anaphora resolution is to find the word Ram because Ram is the antecedent.