What are Regular Expressions?

o Definition: Regex defines search patterns for text processing.

o Importance: Used for validation, extraction, and
manipulation of text data.
 Real-World Example:

o Validation: Ensuring a user enters a valid phone number

format (e.g., 123-456-7890).

Basic Syntax of Regular Expressions

o Literals: Matching exact characters.

o Metacharacters: . (matches any character), * (zero or more),
+ (one or more).
o Character Classes: [A-Za-z] matches any letter, \d matches
digits.

 Real-World Example:

o Matching URLs: https?://[^\s/$.?#].[^\s]*

 Matches HTTP and HTTPS URLs.

Regex Examples

o Email Addresses: Regex to match standard email formats.

 Example: \b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-
Z|a-z]{2,}\b

o Phone Numbers: Regex for validating phone number

formats.

 Example: ^\d{3}-\d{3}-\d{4}$

o Extracting Dates: Regex for extracting dates in

dd/mm/yyyy format.

 Example: \b\d{2}/\d{2}/\d{4}\b
  Detailed breakdown of how each example matches the
target text.

Applications of Regular Expressions

o Data Validation: Ensuring valid formats for email addresses

and phone numbers.
o Text Parsing: Extracting specific data from logs and
documents.
o Web Scraping: Extracting data from web pages.

 Real-World Example:

o Log File Analysis: Extracting error messages from system

logs.

Understanding Words

 : "What are Words in NLP?"

 :

o Definition: Sequences of characters separated by spaces or

punctuation.
o Importance: Basic units of meaning in text.

 Real-World Example:

o Tweet Analysis: Tokenizing tweets to analyze sentiments

and trends.

Word Tokenization Techniques

o Whitespace Tokenization: Splitting text based on spaces.

o Punctuation-Based Tokenization: Handling punctuation
marks as delimiters.
 o Libraries: Using nltk and spaCy.

 Real-World Example:

o News Article Tokenization: Breaking down articles into

words for topic modeling.

Introduction to Corpora

o Definition: A collection of texts used for analysis and model

training.
o Importance: Provides data for building language models and
performing linguistic research.

Real-World Example:

o Sentiment Analysis: Using a corpus of movie reviews to

train sentiment analysis models.

Types of Corpora

: "Types of Corpora"

o Monolingual Corpora: Texts in one language.

 Example: The Brown Corpus (English texts).

o Parallel Corpora: Texts in multiple languages.

 Example: Europarl Corpus (European Parliament

proceedings).

o Tagged Corpora: Annotated with linguistic information.

  Example: Penn Treebank (annotated with part-of-
speech tags and syntactic trees).

 Detailed use cases and applications for each type.

Working with Corpora in Python

o Libraries: Using nltk and TextBlob.

o Accessing Corpora: Loading and exploring texts.

Real-World Example:

o Frequency Analysis: Analyzing word frequency in the

Brown Corpus using Python.

Text Normalization

Introduction to Text Normalization

 Definition: Text normalization transforms text into a uniform format
to improve text processing.

o Purpose: To minimize discrepancies and variations,

ensuring consistency in data analysis and processing.
o Examples: Standardizing text inputs in user interfaces,
cleaning text data for NLP models.

Lowercasing

o Definition: Converting all characters to lowercase.

o Purpose: To treat words like "Apple" and "apple" as the
same word.
o Real-World Example: Search engines and text
classification.

 Example: "Machine Learning" → "machine learning"

Removing Punctuation

"Removing Punctuation"

o Definition: Eliminating punctuation marks from text.

o Purpose: To focus on the of the text without
interference from punctuation.
o Real-World Example: Preparing text for sentiment analysis.

 Example: "Good job, well done!" → "Good job well

done"

Removing Stop Words

"Removing Stop Words"

 o Definition: Filtering out common words that may not carry
significant meaning (e.g., "is," "the").
o Purpose: To focus on important words in text processing.
o Real-World Example: Text summarization and search
engines.

 Example: "The quick brown fox jumps over the lazy

dog" → "quick brown fox jumps lazy dog"

Stemming

"Stemming"

o Definition: Reducing words to their root form.

o Purpose: To consolidate different forms of a word into a
single term.
o Real-World Example: Information retrieval and search
engines.

 Example: "running," "runner," and "runs" → "run"

o Note: Stemming might result in non-dictionary words.

Lemmatization

: "Lemmatization"

o Definition: Converting words to their base or dictionary

form.
o Purpose: To standardize words with contextually accurate
base forms.
o Real-World Example: Text analysis and machine learning
models.

 Example: "better" → "good"

 Note: Lemmatization requires a dictionary and part-
of-speech tagging.
 Examples of Text Normalization

: "Text Normalization: Real-World Examples"

o Customer Reviews: Standardizing text for analysis.

 Example: "Great service!!!" → "great service"

o Social Media Analysis: Handling variations and

abbreviations.
 Example: "u" → "you"
o Medical Records: Normalizing terminology for consistency.

 Example: "Hypertension" → "high blood pressure"

Applications of Text Normalization

"Applications of Text Normalization"

Search Engines: Improved query matching and retrieval.

o Text Classification: Better accuracy in categorizing

documents.
o Machine Translation: Consistent text for accurate
translations.

Minimum Edit Distance

Introduction to Minimum Edit Distance

 o Definition: Minimum edit distance measures the smallest
number of operations needed to change one string into
another.
o Operations: Insertions, deletions, substitutions.
o Purpose: Used in applications like spell-checking,
plagiarism detection, and DNA sequence alignment.

Calculating Minimum Edit Distance

o Dynamic Programming Approach: A matrix is used to

compute the distance efficiently.
o Steps:

1. Create a matrix where rows represent characters of the

first string and columns represent characters of the
second string.
2. Initialize the matrix with distances based on insertion
and deletion operations.
3. Compute the minimum distance for each cell based on
operations (substitution, insertion, deletion).
4. Extract the final distance from the matrix.

Example of Minimum Edit Distance Calculation

"Minimum Edit Distance Example"

o Example: Transforming "flaw" to "lawn".

  Operations:

1. Substitute 'f' with 'l' → "law"

2. Insert 'n' at the end → "lawn"

 Minimum Edit Distance: 2

o A visual representation of the matrix can be

shown to detail each step.

Applications of Minimum Edit Distance

o Spell Checking: Identifying closest correct spelling for

misspelled words.

 Example: "recieve" corrected to "receive"

o Plagiarism Detection: Comparing documents for copied

or similarity.
 Example: Comparing academic papers to detect
similar phrases.
o Bioinformatics: Comparing genetic sequences to find
similarities and differences.

 Example: Aligning DNA sequences to detect

mutations.

Additional Examples of Minimum Edit Distance

 o Name Matching: Matching different spellings of names.

 Example: "Johnathan" vs. "Jonathan" → Minimum

Edit Distance: 1

o Product Listings: Matching similar product names in e-

commerce.

 Example: "Bluetooth Speaker" vs. "bluetooth

speakers" → Minimum Edit Distance: 1

Words and Vectors

Introduction to Word Representations

 o Definition: Words can be represented in numerical formats
(vectors) to capture semantic meaning and relationships.
o Purpose: To enable machines to process and analyze text by
converting words into a form they can understand.
o Examples: Word embeddings like Word2Vec, GloVe,
FastText.

One-Hot Encoding

o Definition: A method of representing words as binary

vectors with a single '1' and the rest '0'.
o Purpose: Simple representation but inefficient for large
vocabularies.
o Example:

 Vocabulary: [cat, dog, mouse]

 "cat" → [1, 0, 0]
 "dog" → [0, 1, 0]
 "mouse" → [0, 0, 1]

o
 In this example, each word is represented by a vector
that is as long as the number of unique words in the
vocabulary. The position of '1' in each vector
corresponds to the index of the word in the vocabulary
list. Although this method is straightforward, it does
not capture any semantic relationships between the
words; all words are equally distant from each other.

Distributed Representations (Word Embeddings)

: "Word Embeddings"
 o Definition: Distributed representations of words in a
continuous vector space where semantically similar words
are closer together.
o Purpose: To capture the context and meaning of words,
allowing for more sophisticated analysis.
o Example:
 Words like "king," "queen," "man," and "woman"
might be positioned such that the vector difference
between "king" and "queen" is similar to "man" and
"woman."

 In this example, word embeddings can capture

relationships like gender by positioning words in a
vector space such that similar relationships have
similar vector differences. This allows for operations
like "king - man + woman = queen" to be meaningful.

Example: Word2Vec

Definition: Word2Vec is a model that represents words as vectors

based on their surrounding context.

o Example:

 Sentence: "The cat sat on the mat."

 Context: The word "cat" would be represented based
on its neighbors "the," "sat," "on," "the," and "mat."

 Word2Vec uses a context window around each word

to learn its representation. In this example, the model
would learn that "cat" is likely to appear near words
like "sat" and "mat," helping it understand that "cat"
has some semantic similarity to those words.

Example: GloVe
Definition: GloVe stands for Global Vectors and is trained on global
word-word co-occurrence statistics.

o Example:

 Sentence: "Paris is to France as Tokyo is to Japan."

 Vector Relationship: The relationship "Paris -
France" is similar to "Tokyo - Japan" in the vector
space.

 GloVe captures global statistical information about

words. In this example, it can learn relationships like
capital cities to their countries by analyzing the overall
co-occurrence patterns across a large corpus. The
model learns that "Paris" is related to "France" in the
same way that "Tokyo" is related to "Japan," and this
relationship is reflected in the vector space.

Challenges with Word Embeddings

o Out-of-Vocabulary Words: Words not seen during training

may not have reliable embeddings.

 Example: New slang or product names may not be

well-represented.

o Polysemy: Words with multiple meanings may not be

accurately captured.

 Example: The word "bank" could mean a financial

institution or the side of a river.

o
  Word embeddings are trained on a fixed vocabulary,
so words not in the training set are problematic.
Additionally, words with multiple meanings might
have a single vector that doesn't capture all their uses,
leading to errors in understanding context.

Cosine Similarity

Introduction to Cosine Similarity

o Definition: Cosine similarity measures the cosine of the

angle between two non-zero vectors in a multi-dimensional
space.
o Purpose: To determine the similarity between two word
vectors regardless of their magnitude.
o Application: Used in text analysis to find similar documents,
phrases, or words.

Calculating Cosine Similarity

o Formula: Cosine Similarity = (A · B) / (||A|| ||B||)

o

 A · B: Dot product of vectors A and B.

 ||A||: Magnitude of vector A.
 ||B||: Magnitude of vector B.

o Range: Values range from -1 (completely dissimilar) to 1

(completely similar).

o Example:

 Vectors:
  A = [1, 2, 3]
 B = [4, 5, 6]

 Dot Product: 1*4 + 2*5 + 3*6 = 32

 Magnitude: ||A|| = sqrt(1^2 + 2^2 + 3^2) = 3.74, ||B||
= sqrt(4^2 + 5^2 + 6^2) = 8.77
 Cosine Similarity: 32 / (3.74 * 8.77) = 0.974

 Cosine similarity measures how similar two vectors

are by looking at the angle between them rather than
their lengths. In this example, the vectors A and B are
almost parallel, resulting in a cosine similarity close to
1, indicating high similarity.

Example: Document Similarity

"Cosine Similarity: Document Similarity Example"

o Example:

 Document 1: "I love machine learning."

 Document 2: "Machine learning is fascinating."
 Vector Representation:

 Document 1: [1, 1, 0, 0, 1, 0]
 Document 2: [0, 1, 1, 1, 1, 0]

 Cosine Similarity Calculation: High similarity due to

common words "machine" and "learning."

 Here, we represent documents as vectors where each

position corresponds to a word in the vocabulary. The
words "machine" and "learning" appear in both
 documents, making their vectors similar. The cosine
similarity between these vectors will be high,
indicating that the documents are similar in .

Example: Word Similarity

 Word Pairs: "king" and "queen," "man" and

"woman."
 Vector Representation:

 "king": [0.7, 0.1, 0.2]

 "queen": [0.65, 0.1, 0.25]

 Cosine Similarity: High similarity due to small angle

between vectors.

 In this example, the vectors for "king" and "queen" are

similar because they share similar contexts in the
training data, resulting in a small angle between them.
The cosine similarity will be close to 1, showing that
these words are closely related in meaning.

Limitations of Cosine Similarity

o Magnitude Independence: Cosine similarity does not

account for the magnitude of vectors, only direction.

 Example: Two vectors with different lengths but the

same direction will have a similarity of 1.

o Context Dependence: Cosine similarity does not consider

the semantic meaning of words in different contexts.

 Example: Words like "bank" (financial institution)

and "bank" (riverbank) may have similar vectors in
some models despite different meanings.

 Cosine similarity is useful for determining the

directional similarity between vectors, but it might
 miss subtleties like word magnitude or polysemy. In
the case of words with multiple meanings or different
contexts, cosine similarity might not always reflect
true semantic similarity.

Introduction to Bag of Words

o Definition: A simple model that represents text as an

unordered collection (or "bag") of words, ignoring grammar
and word order.
o Purpose: To convert text data into numerical vectors for
analysis.
 o Example:
 Sentence: "The cat sat on the mat."
 Bag of Words Representation: {"The": 2, "cat": 1,
"sat": 1, "on": 1, "mat": 1}
o

 This model counts the occurrence of each word in a

document. In the example, the word "The" appears
twice, while the other words appear once. The result is
a frequency-based vector representing the sentence.

Bag of Words: Term Frequency (TF)

"Term Frequency (TF) in Bag of Words"

o Definition: Term Frequency measures the frequency of a

word in a document.
o Purpose: To gauge the importance of a word within a single
document.
o Example:

 Document: "The quick brown fox jumps over the lazy

dog."
 TF for "the": 2/9 (since "the" appears twice in a nine-
word document).

 The TF value is calculated by dividing the number of

times a word appears in the document by the total
number of words in the document. This helps in
normalizing the word count, making it comparable
across documents of different lengths.

Bag of Words: Term Frequency-Inverse Document Frequency

(TF-IDF)

"TF-IDF in Bag of Words"

 Definition: TF-IDF is a statistic that reflects how important a
word is to a document in a collection or corpus.

o Purpose: To balance word frequency within a document

against the word's overall frequency in the corpus,
highlighting unique words.
o Example:

 Corpus: Three documents -

 Doc1: "The cat sat on the mat."

 Doc2: "The dog sat on the log."
 Doc3: "The cat and the dog played."

 TF-IDF for "cat" in Doc1:

 TF ("cat", Doc1): 1/6 (since "cat" appears once

in a six-word document).
 IDF ("cat"): log(3/2) = 0.18 (since "cat" appears
in 2 out of 3 documents).
 TF-IDF ("cat", Doc1) = 1/6 * 0.18 = 0.03.

 TF-IDF decreases the weight of words that appear

frequently across all documents and increases the
weight of words that are more unique to a particular
document. In this case, "cat" has a relatively low TF-
IDF score in Doc1 because it appears in multiple
documents, meaning it's not particularly unique to
Doc1.

Limitations of Bag of Words

o Word Order Ignorance: The model ignores the order of

words, which can lead to loss of meaning.

 Example: "Dog bites man" vs. "Man bites dog" – both

would have the same vector representation.

o Sparse Vectors: High-dimensional and sparse

representations when dealing with large vocabularies.
  Example: A document with a vocabulary of 10,000
words would have a 10,000-dimensional vector, most
of which would be zeros.

 Bag of Words is simplistic and doesn't capture word

context or semantic relationships. It leads to high-
dimensional vectors that are inefficient for large
corpora and fails to distinguish between sentences
with different meanings but similar word
compositions.

Word2Vec

Introduction to Word2Vec

o Definition: A neural network model that learns word

associations from large text datasets and represents words as
vectors in a continuous vector space.
o Purpose: To capture semantic meanings and relationships
between words.
o Example:

 Words like "king," "queen," "man," and "woman"

might have vectors such that the relationship "king -
man + woman" is similar to "queen."

 Word2Vec trains on large corpora to position words in

a vector space where semantically similar words are
close together. This allows for meaningful vector
arithmetic, such as "king - man + woman = queen,"
illustrating how the model understands relationships
between words.

Word2Vec: Skip-gram Model

Definition: The Skip-gram model predicts the context words

given a target word.
 o Purpose: To learn word embeddings by predicting
surrounding words for a given word in a sentence.
o Example:

 Sentence: "The cat sat on the mat."

 Target Word: "cat"
 Predicted Context: "The," "sat," "on," "the," "mat."

 The Skip-gram model uses each word as input and

predicts the context words within a certain window
size. For the word "cat," the model would try to
predict its surrounding words "The," "sat," "on," "the,"
and "mat." This approach helps the model learn word
associations based on their contexts.

Word2Vec: Continuous Bag of Words (CBOW) Model

o Definition: The CBOW model predicts a target word based

on its context words.
o Purpose: To learn word embeddings by predicting the target
word from its surrounding words.
o Example:

 Sentence: "The cat sat on the mat."

 Context: "The," "sat," "on," "the," "mat"
 Predicted Word: "cat"

 CBOW uses context words to predict a target word. In

this example, the model uses the context words "The,"
"sat," "on," "the," and "mat" to predict the word "cat."
This approach helps the model understand how words
are typically used together in language.

Real-World Application: Word2Vec

o Example: Google's Search Engine

 Word2Vec can enhance search engines by

improving the understanding of synonyms and related
terms. For example, searching for "affordable laptop"
 might also return results for "cheap notebook" because
Word2Vec understands that "affordable" and "cheap"
have similar meanings.

o Example: Recommendation Systems

 E-commerce sites use Word2Vec to

recommend products by understanding the
relationships between products. For instance, if a user
buys "shampoo," the system might recommend
"conditioner" because they often appear together in
purchasing patterns.

Advantages of Word2Vec

 : "Advantages of Word2Vec"
 :
o Semantic Relationships: Captures relationships between
words that go beyond simple co-occurrence.

 Example: Understanding analogies like "king is to

queen as man is to woman."

o Efficient: Generates dense vectors that are lower-

dimensional and computationally efficient.

 Example: Word2Vec can represent a word with just a

100-dimensional vector, capturing rich semantic
information.

o Context-Aware: Understands words based on their context

within a sentence.

 Example: Differentiates between "bank" (financial

institution) and "bank" (riverbank) based on
surrounding words.

 Word2Vec excels in creating meaningful word

representations that capture complex relationships,
making it superior to simpler models like Bag of
Words. Its dense vector representations are efficient
and enable advanced language understanding.
 Limitations of Word2Vec

o Requires Large Datasets: Needs extensive training data to

produce accurate embeddings.

 Example: Training Word2Vec on a small corpus may

not capture the full range of word meanings.

o Context Limitation: Single embedding for each word,

ignoring polysemy.

 Example: "Apple" (fruit) vs. "Apple" (company)

might be conflated in a single vector.

 Word2Vec's performance depends heavily on the size

and quality of the training data. Moreover, it struggles
with polysemy since it assigns

Introduction to Word Sense Disambiguation (WSD)

 Definition: WSD is the process of identifying the correct meaning

of a word based on its context when the word has multiple
meanings.
 Purpose: To resolve ambiguity in language processing, enabling
more accurate understanding and translation.
 Example:
o Word: "Bank"
 Meaning 1: A financial institution.
 Meaning 2: The side of a river.
o Sentence: "She deposited money in the bank."
o

 In this sentence, the word "bank" refers to a financial

institution, not the side of a river. WSD helps in
determining this correct sense based on the context.
Importance of WSD

 Natural Language Processing (NLP): Essential for improving the

accuracy of various NLP tasks like machine translation,
information retrieval, and text mining.
 Example:
o Machine Translation: Correctly translating ambiguous words
like "bark" (of a tree vs. a dog sound) to avoid
mistranslations.


o WSD is crucial in applications where understanding the

exact meaning of words in context affects the overall
performance, such as translating text into another language
or retrieving relevant documents in a search engine.

Approaches to Word Sense Disambiguation

 Knowledge-based Methods: Use dictionaries, thesauri, and lexical

databases like WordNet to match words with their senses.

o Example:

 WordNet may list multiple senses for "bass" (fish vs.

musical instrument), and the method uses context to
select the correct one.
 Supervised Methods: Train machine learning models on annotated
corpora where the correct sense is labeled.

o Example:

 A model trained on sentences where "bass" is tagged

as either "fish" or "instrument" learns to predict the
sense in new sentences.

 Unsupervised Methods: Cluster similar contexts together and

assign senses based on patterns that emerge.

o Example:

 Sentences with "bass" in the context of fishing might

cluster together, while those in a musical context form
another cluster.
 

o Different approaches to WSD offer various benefits and

drawbacks, with knowledge-based methods relying on
existing resources, while supervised methods require labeled
data, and unsupervised methods attempt to infer meaning
from patterns without labeled data.

Knowledge-based WSD Example

 Example:

o Sentence: "The bass was too low."

o Possible Senses:

 "Bass" as a type of fish.

 "Bass" as a low-frequency sound in music.

o Context: Related words like "sound" or "music" in the

surrounding text.

o Using a lexical database like WordNet, the system identifies

"bass" in the context of "low" and "sound," leading to the
correct interpretation as a musical term.

Supervised WSD Example

 Example:

o Training Data: Sentences like "He played the bass guitar"

(labeled as musical instrument) vs. "He caught a bass in the
lake" (labeled as fish).
o Model Prediction: "He adjusted the bass to improve the
sound quality."

o The model, having learned from labeled examples, identifies

"bass" in the context of "adjusted" and "sound quality" as
referring to the musical sense rather than the fish.

Unsupervised WSD Example

  Example:

o Context Clustering: Grouping sentences like "The fisherman

caught a bass" with others mentioning "lake" or "fishing"
forms one cluster, while "The bass line was too low" clusters
with sentences mentioning "music" or "guitar."

o By clustering similar contexts, the unsupervised method can

infer that "bass" in the context of "fisherman" and "lake"
likely refers to the fish, while "bass" near "line" and "music"
refers to the musical sense.

Evaluation of WSD Techniques

 Precision and Recall: Measure the accuracy of WSD models in

correctly identifying the intended sense.

o Example: If 100 instances of "bank" are evaluated and 90

are correctly disambiguated, the precision is 90%.

 Baseline Comparison: Compare WSD methods against a simple

baseline like random guessing.

o Example: If random guessing achieves 50% accuracy, a

WSD method with 80% accuracy significantly outperforms
it.

o Evaluation metrics like precision and recall are crucial for

assessing the effectiveness of different WSD approaches,
helping to determine which method performs best in a given
context.

Challenges in Word Sense Disambiguation

 Ambiguity and Polysemy: Words with multiple meanings and

subtle differences pose challenges.

o Example: "Run" can mean to jog, to manage, or to operate,

depending on the context.
  Lack of Training Data: Supervised methods require large
annotated datasets, which can be scarce.
 Contextual Understanding: Some sentences provide limited
context, making disambiguation difficult.

o Example: "She walked to the bank." (without additional

context, it's unclear whether it's a financial institution or
riverbank).

o WSD faces significant challenges due to the inherent

complexity of language, with polysemy, context-dependence,
and limited data resources making accurate disambiguation a
difficult task.

Real-World Applications of WSD

 Machine Translation: Enhances the quality of translation by

accurately identifying word meanings.

o Example: Correctly translating "bank" as "banco" (financial)

vs. "riva" (riverbank) in Italian.

 Information Retrieval: Improves search engine results by

focusing on the intended meaning of search queries.

o Example: A search for "apple" returns technology-related

results rather than fruit when the query context is about
computers.

 Speech Recognition: Helps in correctly interpreting spoken words

with multiple meanings based on the context.

o Example: Recognizing "bass" in a sentence about fishing vs.

music, depending on the conversation topic.

o WSD is integral to many applications in NLP, where

resolving ambiguity directly impacts the accuracy and
relevance of the results, such as in translation, search engines,
and voice assistants.

Future Directions in WSD

 Contextual Embeddings: Advances in deep learning, like BERT
and GPT, provide context-aware representations that improve
WSD.

o Example: BERT can differentiate "bank" based on entire

sentence context rather than just adjacent words.

 Cross-lingual WSD: Applying WSD across multiple languages to

improve multilingual NLP tasks.

o Example: Disambiguating words in multilingual corpora for

better translation and cross-lingual search.

 Hybrid Approaches: Combining knowledge-based, supervised,

and unsupervised methods to leverage the strengths of each.

o Example: Using WordNet alongside deep learning models

for more accurate disambiguation.

o The field of WSD is evolving with new technologies that

promise more accurate and context-sensitive disambiguation,
leading to better performance in a wide range of NLP
applications.