Unit V

Semantic Analysis
 process of understanding the meaning of text at various levels, including word,
sentence, and document levels.
 It involves techniques and methods to extract and represent the semantic content of
text data.
 Understanding meaning of language for human is easy, but it is difficult to
understand by machine.
 So, the goal of semantic analysis is to extract exact meaning, or dictionary meaning,
from the text.
 Semantic Analysis of Natural Language can be classified into two parts-
 1. Lexical Semantic Analysis
 involves understanding the meaning of each word of the text individually.
 Itbasically refers to fetching the dictionary meaning that a word in the text is assigned to
carry.
 2. Compositional Semantics Analysis
 Although knowing the meaning of each word of the text is essential, it is not sufficient to
completely understand the meaning of the text.
 understand how combinations of individual words form the meaning of the text.
 Lexical Semantics

 The semantic analysis process begins by studying and analyzing the dictionary
definitions and meanings of individual words also referred to as lexical
semantics.
 analyzing the set of words in the text to understand their meanings.
 fetching the dictionary definition for the words in the text.
 E.g.the word ‘Blackberry’ could refer to a fruit, a company, or its products,
along with several other meanings.
 context is vital in semantic analysis and requires additional information to
assign a correct meaning to the whole sentence or language.
 Application of Semantic Analysis

 Uber’s social listening

 Uber uses semantic analysis to analyze users’ satisfaction or dissatisfaction levels

via social listening.
 when a new change is introduced on the Uber app, the semantic analysis
algorithms start listening to social network feeds to understand whether users are
happy about the update or if it needs further refinement.
 Google’s semantic algorithm – Hummingbird

 Google incorporated ‘semantic analysis’ into its framework by developing its

tool to understand and improve user searches.
 Hummingbird, results are shortlisted based on the ‘semantic’ relevance of the
keywords.
 Cdiscount’s semantic analysis of customer reviews

 onlineretailer of goods and services, uses semantic analysis to analyze and

understand online customer reviews.
 When a user purchases an item on the ecommerce site, they can potentially give
post-purchase feedback for their activity.
 This allows Cdiscount to focus on improving by studying consumer reviews and
detecting their satisfaction or dissatisfaction with the company’s products.
 Uber’s customer support platform to improve maps

 Uberuses semantic analysis to analyze and address customer support tickets

submitted by riders on the Uber platform.
 The analysis can segregate tickets based on their content, such as map data-related
issues, and deliver them to the respective teams to handle.
 The platform allows Uber to streamline and optimize the map data triggering the
ticket.
 IBM’s Watson conversation service

 Used to derive meaning from unstructured data.

 Itanalyzes text to reveal the type of sentiment, emotion, data category, and the
relation between words based on the semantic role of the keywords used in the
text.
 According to IBM, semantic analysis has saved 50% of the company’s time on the
information gathering process.
 Conversational chatbots
 These chatbots act as semantic analysis tools that are enabled with keyword
recognition and conversational capabilities.
 These tools help resolve customer problems in minimal time, thereby increasing
customer satisfaction.
 Chatbots help customers immensely as they facilitate shipping, answer queries, and
also offer personalized guidance and input on how to proceed further.
 some chatbots are equipped with emotional intelligence that recognizes the tone of
the language and hidden sentiments, framing emotionally-relevant responses to
them.
 Automated ticketing support
 Itunderstands the text within each ticket, filters it based on the context, and
directs the tickets to the right person or department (IT help desk, legal or sales
department, etc.).
 Sentiment analysis
 understand the voice of their customers, extract sentiments and emotions from
text, and, in turn, derive actionable data from them.
 Ithelps capture the tone of customers when they post reviews and opinions on
social media posts or company websites.
 Search engine results
 Search engines use semantic analysis to understand better and analyze user
intent as they search for information on the web.
 with the ability to capture the context of user searches, the engine can provide
accurate and relevant results.
 Language translation
 semantic analysis methods are extensively used by language translators
 translatortools can determine a user’s intent and the meaning of input words,
sentences, and context.
WordNet Statistics
 Attachment for fragment of
English

 Itis easy to miss sentence fragments because all a series of words needs is a
capital letter at the beginning and ending punctuation. It looks like a sentence.
 Yet,for a sentence to be truly complete, it must contain an independent clause,
which tells the whole story even when isolated from its context.
 What is a sentence fragment?
 A sentence fragment is an incomplete sentence;
 it’s a partial sentence that’s missing another necessary part to make it complete.
 Put simply, a sentence fragment is a clause that falls short of a complete because
it is missing one of three critical components:
a subject,
a verb, and
a complete thought.
 Fixing sentence fragments
 Fixing a sentence fragment involves one of two things: giving it the components it
lacks or fastening it onto an independent clause.
 E.g. There are many ways to frighten little brothers; for example, you could hide
under their beds and wait for dark.
Notice that in order to properly connect these two clauses with a semicolon, you
need to do some rewriting in order to ensure both can function as independent
clauses. In other words, you need to fortify the fragment with a subject and a
verb to turn it into a sentence. Notice in the example above you’ll need to edit
other parts of your fragment to turn it into a grammatically correct independent
clause.
 If a semicolon seems too formal for your purposes, you could write your text as two
sentences—but don’t forget to make sure the second one has a subject and a verb:
 E.g. There are many ways to frighten little brothers.
 Such as, you could hide under their beds and wait for dark.
 Sentence fragment examples

 E.g. Because of the rain.

 On its own, because of the rain doesn’t form a complete thought. It leaves us wondering
what happened because of the rain. To complete it, we need further explanation:

 Because of the rain, the party was canceled.

 Now the fragment has become a dependent clause attached to a sentence that has a
subject (the party) and a verb (was canceled). Our thought is complete.
 E.g. I ran.
I ran may be a short thought, but it has a subject (I) and a verb (ran). Nothing in
the sentence demands further explanation.
 Avoiding sentence fragments not only makes your writing easier to read, but it can
also make you sound more polished in polite correspondence. We’ve all had emails
ending with:
 E.g. Looking forward to seeing you.
 That sentence lacks a subject. Adding the subject will build a stronger, more confident-
sounding sentence:
 I’m looking forward to seeing you.
 Recast the sentence
 Sometimes it’s better to rethink the sentence so that it’s naturally more complete. This could mean
scrapping the entire fragment and starting over from scratch, or it could involve something minor
such as adding a punctuation mark.
 Use a grammar tool
 Some sentence fragments are easy to identify but not all of them.
 Ifyou’re confused about whether you have a sentence fragment or a complete
sentence, try using grammar checker tool, which will identify mistakes and
suggest corrections.
 Relations among lexemes & their
senses

 Terminology
 Word forms:-runs, ran, running. Any, possibly inflected, form of a word.
(Morphology)
 Lemma:-A basic word form, used to represent all forms of the same word.
 Lexeme:-RUN(V), GOOD(A)
 An abstract representation of a word (and all its forms), with a part-of-speech and
a set of related word senses.
 Lexicon:-A (finite) list of lexemes.
 Relations between words /
senses

 Homonymy
 Polysemy
 Synonymy
 Antonymy
 Hypernymy
 Hyponymy
 Meronymy
 Homonymy

 Homonyms: lexemes that share a form, but unrelated meanings (same spelling and
same pronunciation)
 E.g. bat (wooden stick thing) vs bat (flying scary mammal)
 bank (financial institution) vs bank (riverside)
 It Can be homophones, homographs, or both:
 Homophones: write and right, piece and peace(sound the same but are different in
meaning or spelling)
 Homographs: bass and bass (sound same/same spelling but diff. meaning)
 Homonymy causes problems for NLP applications:-

 Word Sense Disambiguation (WSD)

 Information Retrieval (IR)
 Machine Translation
 Named Entity Recognition (NER)
 Text Summarization
 Polysemy

 when a single word has multiple related meanings

 E.g.bank the building, bank the financial institution, bank the biological
repository
 different related senses
 Most non-rare words have multiple meanings
 Synonyms

 Word that have the same meaning in some or all contexts.

 Two lemmas with the same sense
 E.g. big / large , automobile / car , water / H20 .
 Antonyms

 Senses that are opposites with respect to one feature of their meaning
 Two lemmas with the opposite sense
 E.g. dark / light, short / long, hot / cold, up / down.
 Hyponyms and Hypernyms
 Hyponym: - the sense is a subclass of another sense
 car is a hyponym of vehicle
 dog is a hyponym of animal
 mango is a hyponym of fruit
 Hypernym: - the sense is a superclass
 vehicle is a hypernym of car
 animal is a hypernym of dog
 fruit is a hypernym of mango
Meronymy
 relationship that exists between two
words when one word is a part or a
member of the other.
 In this relationship, the part is called
the meronym, and the whole is
called the holonym.
 E.g. “wheel” is a meronym of “car,”
and “car” is the holonym of “wheel.”
 Semantic Ambiguity

 occurswhen a word or phrase has more than one possible meaning or

suggestion in a single sentence.
 E.g. He is a star
 To resolve semantic ambiguity, search engines need to understand the query
and the underlying concepts and relations that it expresses.
 One common method for doing this is using semantic parsing, which is a
process of converting a natural language query into a logical form that can be
processed by a knowledge base or a reasoning system.
 Word Sense Disambiguation
(WSD)

 Itis the process of identifying the correct sense of a word from a set of possible
senses, based on the context in which the word appears.
 WSD aims to disambiguate the correct sense of a word in a particular context.
 E.g. the word “bank” can have different meanings in the sentences –
 “I deposited money in the bank”
 “The boat went down the river bank”.
 Some common approaches to WSD include using WordNet, supervised
machine learning, and unsupervised methods such as clustering.
 Difficulties in Word Sense Disambiguation

 Different Text-Corpus or Dictionary:

 One issue with word sense disambiguation is determining what the senses are because
different dictionaries and thesauruses divide words into distinct senses.
 Some academics have proposed employing a specific lexicon and its set of senses to address
this problem. In general, however, research findings based on broad sense distinctions have
outperformed those based on limited ones.
 The majority of researchers are still working on fine-grained WSD.
 PoS Tagging:

 Part-of-speech tagging and sense tagging have been shown to be very tightly coupled in any
real test, with each potentially constraining the other.
 Both disambiguating and tagging with words are involved in WSM part-of-speech tagging.
 Sometimes, algorithms designed for one do not always work well for the other, owing to the
fact that a word’s part of speech is mostly decided by the one to three words immediately
adjacent to it, whereas a word’s sense can be determined by words further away.
 Sense Inventories for Word Sense Disambiguation
 Sense Inventories are the collection of abbreviations and acronyms with their possible
senses. Some of the examples used in Word Sense Disambiguation are:
 Princeton WordNet: is a vast lexicographic database of English and other languages that is
manually curated. For WSD, this is the de facto standard inventory. Its well-organized Synsets, or
clusters of contextual synonyms, are nodes in a network.
 BabelNet: is a multilingual dictionary that covers both lexicographic and encyclopedic
terminology. It was created by semi-automatically mapping numerous resources, including
WordNet, multilingual versions of WordNet, and Wikipedia.
 Wiktionary: a collaborative project aimed at creating a dictionary for each language separately,
is another inventory that has recently gained popularity.
 WSD Methods  3. Unsupervised and Semi-
 1.Knowledge-Based Methods Supervised Methods
 Lesk’s  Clustering Approaches
Algorithm
 Dictionary  Co-occurrence Graphs
and Thesaurus-Based
Methods  Semi-Supervised Learning
 Graph-Based Algorithms
 2. Supervised Learning Methods
 Feature-Based Models
 Neural Network Models
 Sequence Models
 1.Knowledge-Based Methods (Lesk‘s
Algorithm)
 the correct sense of a word in a given context can be identified by selecting the sense
whose dictionary definition has the greatest overlap with the words in the word's
actual context.
 This method utilizes a lexical database such as WordNet, which provides definitions
and example sentences for words in each of their senses.
 Necessary Configuration (i/p)
 LexicalResource: The algorithm requires a dictionary or a lexical database like
WordNet that provides definitions for each sense of potentially ambiguous words.
 Ambiguous Word: Choose the word whose sense needs to be disambiguated based on
the surrounding context.
 Steps:-
 Word Definition Retrieval: For each sense of the ambiguous word, retrieve the
corresponding dictionary definition.
 Contextual Words Extraction: Extract the set of words from the surrounding
context of the ambiguous word in the text.
 Overlap Calculation: For each sense, calculate the overlap between the words in
the definition and the words in the context. Overlap is typically the count of
common words, but it can be enhanced to include synonyms, hypernyms, and
other semantic relations.
 Sense Selection: The sense with the highest overlap is chosen as the correct sense
for the instance in question.
 E.g. Consider the word "bank" in the sentence: "He sat on the river bank.“
 Dictionary Definitions (simplified):
 Bank (n1): "The land alongside or sloping down to a river or lake."
 Bank (n2): "A financial institution that accepts deposits and channels the money into lending activities."
 Context: "He sat on the river bank."
 Overlap Calculation:
 n1: Overlap words might include "river", "land".
 n2: No overlap.
 The sense "n1" (the land alongside or sloping down to a river or lake) will be selected because it has a
higher overlap with the context words.
 Advantages of Lesk Algorithm

 Simplicity: Easy to understand and implement.

 No Training Required: Unlike supervised methods, it does not require a
labeled dataset, which can be costly to create.
 Interpretability: The decision-making process is transparent (based on lexical
matching).
 Disadvantages of Lesk Algorithm

 Dependence on Quality of Definitions: The effectiveness is highly dependent

on the quality and completeness of the dictionary or lexical database.
 Context Sensitivity: The algorithm may struggle with polysemy where
different senses have overlapping keywords in their definitions.
 Limitedby Literal Overlap: The basic version only considers literal overlap,
which might not capture deeper semantic relationships.
 2. Supervised (Naïve Bayes)

 Naïve Bayes is a probabilistic classifier based on applying Bayes' theorem with

strong (naïve) independence assumptions between the features.
 In the context of WSD, each sense of a word is treated as a class, and the
features typically involve the words and possibly other contextual cues
surrounding the ambiguous word.
 Steps:-
 1. Feature Extraction:
 Contextual features around the ambiguous word are extracted. These can include:
Bag of Words (BoW): Nearby words are used as features.
Part of Speech (POS): POS tags of nearby words.
Collocations: Specific word combinations or patterns near the target word.
Syntactic Dependencies: Relationships with other words in a sentence.
 Text preprocessing might be involved here, such as stemming, lemmatization, and stop-
word removal.
 2. Training the Model:
 A training dataset where each instance of the ambiguous word is annotated with the
correct sense is required.
 The Naïve Bayes classifier learns the likelihood of each feature given a particular sense
(P(feature | sense)) and the prior probability of each sense (P(sense)).
 3. Sense Prediction:
 For a new instance of the ambiguous word, compute the posterior probability of each
sense given the observed features in the context using Bayes' theorem:
P(sense∣features)∝P(features∣sense)×P(sense)
 Predict the sense with the highest posterior probability.
 E.g. disambiguating the word "bank" in the sentence "She deposited the check at the bank.“
 Features might include: "deposit," "check," proximity indicators.
 Classes (senses):
 Bank1: Financial institution
 Bank2: Riverbank
 Assuming from training data:
 P(Bank1) is high when "deposit," "check" are nearby.
 P(Bank2) might be high with words like "river," "water."
 For the given context, the classifier calculates probabilities:
 P(Bank1 | features) would likely be higher than P(Bank2 | features).
 Hence, the classifier predicts "Bank1."
 Advantages of Naïve Bayes for
WSD
 Efficient: Requires a small amount of training data to estimate the necessary
probabilities and is computationally simple.
 Performs Well Under Independence Assumption: Although features (words)
in text data are not truly independent, Naïve Bayes often performs surprisingly
well even under this unrealistic assumption.
 Scalable: Handles large vocabularies and data sets efficiently.
 Disadvantages of Naïve Bayes
for WSD
 Independence Assumption: The assumption that all features are independent
given the class is rarely true in language, which can limit performance in some
contexts.
 Feature Representation: Relies heavily on good feature engineering; poor
choice of features can significantly degrade performance.
 Handling of Context: More complex syntactic and semantic relationships may
not be captured well.
 3. Supervised (Decision List)

 A decision list is a sequence of if-then rules arranged in a priority order, where each rule
tests a condition based on the context of the target word.
 Working
 1. Feature Extraction:
 The first step in using a decision list for WSD is to extract features from the training data. These
features typically include:
 The words immediately surrounding the ambiguous word.
 Part-of-speech tags of the surrounding words.
 Collocations or common phrases associated with particular senses of the word.
 Possibly syntactic dependencies or semantic patterns.
 2. Rule Generation:
 Based on the training data, where each instance of the ambiguous word is
labeled with the correct sense, the system generates a list of if-then rules. Each
rule associates a particular pattern or feature with a specific sense of the word.
 Rules are typically derived by identifying features that are highly predictive of
a particular sense. For example, the presence of "money" near the word "bank"
might strongly suggest the financial sense of "bank.“
 3. Rule Scoring and Ordering:
 Each rule is assigned a score based on its predictive power, which might be
calculated using metrics such as pointwise mutual information, likelihood
ratios, or simply frequency counts of the correct classification in the training
data.
 The rules are then ordered in a list from the most to the least powerful (i.e., the
rules that are most reliable in predicting a sense come first).
 4. Classification:
 To disambiguate a new instance of an ambiguous word, the decision list is
processed from top to bottom. The context of the instance is checked against
each rule until a rule that matches the context is found.
 The sense associated with the first matching rule is assigned to the word.
 E.g. Consider the word "bark." A decision list for "bark" might look like this:
 Ifthe word "dog" or "howl" appears near "bark," then "bark" means the sound
a dog makes.
 Ifthe word "tree" or "wood" appears near "bark," then "bark" means the outer
covering of a tree.
 Default to the sound a dog makes if no other rules apply.
 Advantages of Decision Lists for
WSD
 Interpretability: Each rule in the list has a clear rationale and is easy to
understand.
 Efficiency: Classification involves sequentially checking conditions until a
match is found, which can be very fast.
 Flexibility:Rules can be easily added, removed, or modified based on
additional data or insights.
 Disadvantages of Decision Lists for
WSD
 Overfitting: Highly specific rules might perform well on training data but
poorly on unseen examples.
 Data Dependence: The effectiveness of the rules depends heavily on the
quality and representativeness of the training data.
 Scalability: As
the number of potential senses and contextual features grows,
managing and prioritizing the rules can become challenging.