ESE -3

TITLE: Hate Speech Detection using Machine Learning

Submitted by:

Ashish Parasar (23225006)

Submission date 22nd September 2024

Under the Guidance of:

MANJULA SHANBHOG (SCHOOL OF SCIENCES NCR)

1) Introduction to Hate Speech

Hate speech is defined as any form of communication, whether in written, spoken, or behavioral
form, that belittles, attacks, or discriminates against individuals or groups based on specific
characteristics, including but not limited to race, ethnicity, religion, gender, sexual orientation,
disability, or nationality. It is an increasingly pressing issue in today’s digital age, where social
media and other online platforms provide a global stage for people to express their opinions—
both positive and negative. While free expression is fundamental to democratic societies, hate
speech can incite violence, deepen societal divisions, and cause lasting harm to marginalized
communities.

The rise of the internet has exacerbated the problem, as individuals can now hide behind the
anonymity of online platforms, making it easier for harmful rhetoric to spread quickly and
virally. Social networks, comment sections, and online forums have become breeding grounds
for offensive content, often directed at vulnerable groups. In addition, the vast scale of online
communication makes it difficult for human moderators alone to identify and respond to hate
speech in a timely manner. As a result, there has been growing demand for automated tools that
can detect and prevent the spread of such harmful content.

One solution lies in the field of artificial intelligence (AI), particularly in machine learning (ML).
Machine learning algorithms have proven to be effective in recognizing patterns in large
datasets, which makes them ideal for automating the detection of hate speech. By training
models on labeled data, machine learning systems can learn to differentiate between offensive
language and neutral or positive communication. This enables platforms to automatically flag or
remove hateful content, thus maintaining safer, more inclusive online environments.

In this assignment, we explore how machine learning techniques can be applied to the problem
of detecting hate speech in online comments. By using a labeled dataset of online comments, we
will preprocess the data, build a neural network model, and train it to classify comments as toxic
or non-toxic. The goal is to demonstrate how automated systems can help in the fight against the
spread of hate speech and provide a foundation for further research and development in this area.

2. Dataset and Preprocessing

In this assignment, we use the dataset from the Jigsaw Toxic Comment Classification
Challenge, which was released as part of a competition to advance research in identifying toxic
behavior in online forums. This dataset is an extensive collection of comments sourced from
various online platforms, and it has been labeled based on the presence of toxicity. Each
comment in the dataset is annotated under several categories that represent different forms of
harmful speech, including:

 Toxic: Comments containing abusive language or personal attacks.

 Severe Toxic: A more extreme form of toxicity, where the language is highly offensive or
intended to severely demean others.
 Obscene: Comments containing vulgar or inappropriate language.
 Threat: Comments that include threats of violence or harm.
 Insult: Language aimed at insulting or demeaning individuals or groups.
 Identity Hate: Comments that attack a person or group based on their identity, including
race, gender, ethnicity, religion, or sexual orientation.

Given the multilabel nature of this dataset, each comment can belong to one or more of these
categories. This makes it a complex, real-world problem where multiple toxic traits often overlap
within the same comment.

Preprocessing Steps
Before we can use the dataset to train a machine learning model, the data must be preprocessed.
Raw text data cannot be directly fed into a machine learning algorithm, so several steps are
necessary to clean and transform the text into a suitable format for the model. The main steps in
the preprocessing pipeline include:

1. Tokenization: Tokenization is the process of breaking down the text data into individual
words or tokens. Each comment in the dataset is converted into a sequence of words, which
enables the model to process the input at a granular level. This is an essential step in natural
language processing (NLP) tasks, as it allows the model to recognize patterns, relationships, and
occurrences of specific words in the text. For example, the comment "This is terrible!" would be
tokenized into three tokens: ["This", "is", "terrible!"].

In our case, we use TensorFlow's Text Vectorization layer, which simplifies the tokenization
process by automatically breaking down each comment into individual tokens. Additionally, it
allows us to standardize and normalize the text by converting all words to lowercase and
removing punctuation and other unnecessary characters.

2. Vectorization: Once the comments are tokenized, we need to convert these sequences of
words into a numerical format that can be understood by the machine learning model.
Vectorization is the process of mapping each token (word) to a unique integer, representing the
word in a fixed-size vocabulary. Each comment is then represented as a sequence of these
integers, allowing the model to learn from patterns in the text.

In this assignment, we used TensorFlow’s Text Vectorization layer to accomplish vectorization.

This layer converts each tokenized comment into a fixed-length sequence of integers, where each
integer corresponds to a specific word in the dataset's vocabulary. To ensure consistency, we set
a fixed sequence length for all comments (e.g., 1,800 tokens). Comments shorter than this length
are padded with zeros, while longer comments are truncated.

By using this vectorization technique, we transform the raw text data into structured numerical
data that can be processed by the model. This step is crucial for training the machine learning
algorithm, as the numerical representations preserve the semantic relationships between words
while enabling efficient computations during training.

This code demonstrates how the TextVectorization layer is used to process the comments. The
adapt() method learns the vocabulary from the dataset, while the vectorizer() function converts
each comment into a numerical sequence.
3. Model Building
To detect hate speech in online comments, we employed a neural network model. Neural
networks have proven to be highly effective in natural language processing (NLP) tasks, such as
text classification, due to their ability to learn complex patterns in data. The architecture of our
model is specifically designed to process sequential data, such as text, and is structured to
capture the relationships between words and phrases within the comments.

Neural Network Architecture

Our neural network model consists of several key layers, each designed to perform a specific
function in the overall process of learning and classification:

1. Embedding Layer: The embedding layer is the first layer of the network. Its purpose is to
transform each word in the input text into a dense vector representation, known as an
embedding. Embeddings are low-dimensional representations of words that capture semantic
information about the relationships between words. For example, words with similar meanings
(e.g., "good" and "great") will have embeddings that are close to each other in the vector space.
In this model, the embedding layer takes each word in the tokenized and vectorized comment
and converts it into a 32-dimensional vector.

The embedding layer allows the model to learn and generalize across similar words, improving
its ability to understand and classify diverse comments. These learned embeddings are crucial for
the model to capture the nuances of hate speech, which may be expressed in many different
ways.

1. Bidirectional LSTM Layer: After the embedding layer, we introduce a Bidirectional Long
Short-Term Memory (LSTM) layer. LSTM is a type of recurrent neural network (RNN) that is
well-suited for processing sequences of data, such as sentences or paragraphs. LSTMs are
capable of learning long-term dependencies in sequential data, meaning they can retain
information from earlier words in the sentence when predicting later words.

A bidirectional LSTM processes the input data in both forward and backward directions. This
allows the model to capture context from both past and future words in the sequence, improving
its ability to understand the meaning of the comment. For instance, the word "great" might have
a different meaning depending on whether it is followed by a positive or negative word, and
bidirectional LSTMs help the model capture this context.

In our model, the LSTM layer has 32 units, which control the size of the hidden states and output
representations generated by the LSTM. The tanh activation function is used within the LSTM
cells, which allows the network to model non-linear relationships in the data.
1. Fully Connected (Dense) Layers: Following the LSTM layer, we add a series of fully
connected layers (also known as dense layers). These layers serve as feature extractors and
classifiers, taking the output from the LSTM and learning to make predictions about the
classification of each comment. Each dense layer contains a certain number of neurons that apply
learned weights to the input, helping the model to focus on the most important features for
classification.

In our model, we use three dense layers:

 The first dense layer contains 128 neurons, using the relu activation function. This
function introduces non-linearity into the model, allowing it to learn complex patterns in
the data.
 The second dense layer increases the number of neurons to 256, enhancing the model's
ability to capture intricate relationships in the data.
 The third dense layer returns to 128 neurons to further refine the learned features.

1. Output Layer: The final layer in the model is the output layer, which is responsible for
making predictions about whether the comment contains hate speech. Since this is a multi-label
classification problem (i.e., each comment can belong to multiple categories such as toxic,
obscene, or insulting), the output layer consists of six neurons, each corresponding to one of the
target labels (toxic, severe toxic, obscene, threat, insult, identity hate).

We apply the sigmoid activation function in the output layer. The sigmoid function outputs a
probability between 0 and 1 for each label, indicating the likelihood that the comment belongs to
that category. A threshold (typically 0.5) is then used to classify each comment as belonging to a
particular class if the probability exceeds the threshold.

Model Compilation
After defining the architecture, the model is compiled using the binary cross-entropy loss
function, which is appropriate for multi-label classification tasks. The optimizer used is Adam, a
popular choice for optimizing neural networks, as it adapts the learning rate during training and
converges quickly.

Summary of the Model

 Input Layer: Processes tokenized comments.

 Embedding Layer: Maps words to a 32-dimensional vector space.

 Bidirectional LSTM Layer: Captures sequential dependencies in both forward and

backward directions.

 Dense Layers: Extracts features and refines the model’s understanding of the data.
  Output Layer: Uses sigmoid activation to predict the probability of each category (toxic,
obscene, etc.).

The code for building and compiling the model is as follows:

4. Model Training
Training the neural network model is crucial for ensuring it can learn to detect hate speech
effectively. The model was trained for three epochs, using the Adam optimizer and binary
cross-entropy loss function. The dataset was split into training, validation, and testing sets to
monitor performance and prevent overfitting.

Loss and Validation Loss

During training, the model’s loss and validation loss were monitored. The graph below
illustrates the reduction in both metrics over two epochs. The blue line represents the loss during
training, while the orange line represents the validation loss, calculated from the validation set
after each epoch.

 The training loss decreases as the model learns from the dataset, indicating that it is
minimizing the error in predictions.

 The validation loss also decreases over time, suggesting that the model generalizes well
to unseen data, although it is slightly lower than the training loss. This indicates that the
model was not overfitting during the training phase.

This graph is a good indicator of how well the model learned and how robust it is for real-world
data.
5. Results
After training the model, we evaluated its performance using standard metrics such as precision,
recall, and accuracy on the test dataset. These metrics are critical in understanding how well the
model is able to detect hate speech and how it performs on unseen data.

 Precision: Precision measures how many of the positive predictions made by the model
were actually correct. For hate speech detection, this means how many of the comments
that were classified as toxic were truly toxic. A high precision indicates that the model
makes fewer false-positive errors.

 Recall: Recall measures how many of the actual positive instances were correctly
identified by the model. In other words, it tells us how many of the toxic comments were
correctly detected by the model. A high recall indicates that the model is good at
identifying hate speech, even at the risk of generating some false positives.

 Accuracy: Accuracy represents the overall correctness of the model’s predictions,

calculated as the ratio of correct predictions (both positive and negative) to the total
number of predictions. However, accuracy can be misleading in imbalanced datasets
where one class significantly outnumbers the others, which is often the case with hate
speech detection.

The following code snippet demonstrates how these metrics were calculated:
Evaluation Results
The model produced the following results based on the test dataset:

 Precision: 0.79 (79%)

 Recall: 0.72 (72%)
 Accuracy: 0.88 (88%)

These results demonstrate that the model was fairly good at identifying hate speech while
maintaining a balance between precision and recall. The accuracy is also relatively high,
indicating that the model performed well overall in both detecting hate speech and correctly
identifying non-toxic comments.

6. Challenges and Limitations

Despite the promising results of the model, there are several challenges and limitations to
consider when building and deploying hate speech detection systems:

1. Subtlety of Offensive Language: Hate speech is often subtle, involving sarcasm, irony, or
coded language, which makes it difficult for models to detect. People may use euphemisms,
abbreviations, or slightly altered spelling to evade detection, and a purely data-driven model may
struggle to capture these nuances without extensive retraining.

2. Context Dependency: Hate speech can depend heavily on the context in which it is used.
Words that are considered offensive in one context might not be in another. For example, certain
words used among friends in a colloquial manner may be harmless but could be considered
offensive in a different setting. Capturing the context of conversations is a significant challenge
for current models.

3. Bias in Training Data: The quality and diversity of the training data play a crucial role in
determining the model’s fairness and effectiveness. If the dataset is biased or lacks representation
of certain groups, the model may inadvertently learn biased patterns, leading to unfair treatment
of specific communities. For example, some models have been known to overflag comments
from marginalized communities because of biases in the training data.

4. Generalization: Hate speech is an evolving problem. New terms, slang, and ways of
expressing hate emerge regularly, requiring constant updates to the model. A model trained
today may not be as effective in identifying hate speech in the future unless it is continually
retrained with fresh data.
8. Conclusion
In this assignment, we have demonstrated the use of machine learning techniques to detect hate
speech in online comments. By utilizing a neural network model with embedding, LSTM, and
dense layers, we successfully built and trained a model capable of classifying toxic comments
with reasonable accuracy. The model was evaluated using key metrics such as precision, recall,
and accuracy, highlighting its strengths and areas for improvement.

Despite the challenges—such as the complexity of language, biases in training data, and ethical
considerations—machine learning remains a powerful tool in the fight against hate speech. As
advancements in natural language processing continue and ethical AI practices evolve, we can
expect models to become more accurate and fair, paving the way for safer and more inclusive
online spaces.

9. References
1 https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/data

2 https://www.youtube.com/watch?v=ZUqB-luawZg