Scribd
Upload
15 views

Logistic Regression Classification in Natural Language Processing (NLP) Final

The document discusses logistic regression as a key classification tool in Natural Language Processing (NLP), highlighting its relationship with neural networks and its application in tasks such as spam detection. It outlines the components of logistic regression, including feature representation, classification functions, loss functions, and optimization algorithms, while also explaining how to convert categorical features into numerical values using one-hot encoding. The conclusion emphasizes the effectiveness and interpretability of logistic regression, while acknowledging its limitations with complex data.

Uploaded by

maan younis
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
15 views

Logistic Regression Classification in Natural Language Processing (NLP) Final

The document discusses logistic regression as a key classification tool in Natural Language Processing (NLP), highlighting its relationship with neural networks and its application in tasks such as spam detection. It outlines the components of logistic regression, including feature representation, classification functions, loss functions, and optimization algorithms, while also explaining how to convert categorical features into numerical values using one-hot encoding. The conclusion emphasizes the effectiveness and interpretability of logistic regression, while acknowledging its limitations with complex data.

Uploaded by

maan younis
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 14

AKRE UNIVERSITY FOR APPLIED SCIENCES TECHNICAL

COLLEGE OF INFORMATICS-AKRE

Logistic Regression Classification


in Natural Language Processing
(NLP)

Academic Year: 2024-2025


name
INTRODUCTION

 Logistic regression is one of the most important analytic


tools for classification in NLP. As the textbook states:
 "Logistic regression is the baseline supervised machine
learning algorithm for classification, and has a very close
relationship with neural networks. A neural network can
be viewed as a series of logistic regression classifiers
stacked on top of each other."
IDEA OF LOGISTIC REGRESSION

We’ll compute w∙x+b


And then we’ll pass it through the sigmoid
function:
 σ(w∙x+b)
And we'll just treat it as a probability
THE SIGMOID FUNCTION

Equation
THE SIGMOID FUNCTION

This is the Sigmoid Function Plot, showing how logistic regression maps input
values to probabilities between 0 and 1. The red dashed line represents the decision
threshold at 0.5.
TURNING A PROBABILITY INTO A CLASSIFIER

if w∙x+b > 0
if w∙x+b ≤ 0
COMPONENTS OF LOGISTIC REGRESSION

 Four Key Components:


1.Feature Representation:
Convert text to numerical vectors (e.g., count of positive words).
2.Classification Function (Sigmoid/Softmax):
Transforms output into probabilities.
3.Loss Function (Cross-Entropy Loss):
Measures error between predictions and true labels.
4.Optimization Algorithm (Gradient Descent):
Updates weights to minimize error.
CLASSIFICATION WITH LOGISTIC REGRESSION (USING
TEXT FEATURES)

 In this example, we classify emails as Spam (1) or Not


Spam (0) based on three textual features:
• Keyword Presence (e.g., "Discount," "Free," "Meeting")
• Sender Type (e.g., "Company," "Individual," "Unknown")
• Number of Links in the email (e.g., "Few," "Moderate," "Many")
 Since logistic regression requires numerical input, we use
one-hot encoding to convert categorical values into
numerical features.
8
Define the Dataset

Keyword Sender Type Link Count Spam (1) / Not Spam (0)
Discount Company Many 1
Free Unknown Moderate 1
Meeting Individual Few 0
Offer Company Moderate 1
Hello Individual Few 0

9
Apply One-Hot Encoding

Each categorical feature is converted into separate binary columns:

Spam (1) / Not Spam


Individual
Company

Moderate
Unknown
Discount

Meeting

Many
Hello
Offer
Free

Few

(0)
1 0 0 0 0 1 0 0 1 0 0 1
0 1 0 0 0 0 0 1 0 1 0 1
Now,
0 our0 categorical
1 features
0 are
0 transformed
0 into numerical
1 0 0binary0values,1 0
10

which
0 can
0 be used0 in logistic
1 regression.
0 1 0 0 0 1 0 1
LOGISTIC REGRESSION MODEL

Assuming the model has been trained, we define the learned weights as:

11
COMPUTE MODEL OUTPUT

Features (One-Hot
g(z) Predicted y
Encoded)

(1,0,0,0,0,1,0,0,1,0,0) −2(1)+1.1(1)+1.4(1)+1-2(1) + 1.1(1) + 1.4(1) + 1= 1.5 0.223 1.223 0.818 1

(0,1,0,0,0,0,0,1,0,1,0) 1.5(1)−0.9(1)+0.8(1)+11.5(1) - 0.9(1) + 0.8(1) + 1 = 2.4 0.091 1.091 0.916 1

(0,0,1,0,0,0,1,0,0,0,1) 1.2(1)+0.7(1)−0.5(1)+11.2(1) + 0.7(1) - 0.5(1) + 1 = 2.4 0.091 1.091 0.916 0

(0,0,0,1,0,1,0,0,0,1,0) 0.5(1)+1.1(1)+0.8(1)+10.5(1) + 1.1(1) + 0.8(1) + 1= 3.4 0.033 1.033 0.968 1

(0,0,0,0,1,0,1,0,0,0,1) −1.3(1)+0.7(1)−0.5(1)+1-1.3(1) + 0.7(1) - 0.5(1) + 1= -0.1 1.105 2.105 0.476 0


12
DECISION RULE

•If g(z)≥0.5, classify as Spam (1).


•If g(z)<0.5, classify as Not Spam (0).

•Emails with strong spam keywords (like "Discount," "Free," or "Offer") are classified as Spam (1).
•Emails with neutral words (like "Meeting" or "Hello") are more likely to be Not Spam (0).
•The sender type and the number of links also influence classification.
Conclusions

•Effectiveness in Text Classification: Logistic regression is a simple yet powerful


algorithm for binary classification problems like spam detection, sentiment analysis,
and topic classification.
•Feature Representation Matters: Proper text representation techniques like One-Hot
Encoding, TF-IDF, or Word Embeddings significantly impact model performance.
•Interpretability: Unlike deep learning models, logistic regression provides clear
decision boundaries and is easy to interpret, making it useful for explainable AI.
•Limitations: While effective for linearly separable data, logistic regression struggles
with complex, high-dimensional text data, where advanced models like SVMs, Random
Forests, or Deep Learning perform better.

You might also like