Related work

Some recent works have focused on analyzing current challenges in language recognition
based on existing works. Jurgens et al. [1] presented a position paper outlining the current
challenges in combating cyberbullying and proposing several strategies to address them. They
argued that most existing research focuses only on a narrow definition of abuse and that the
scope of the problem needs to be broadened to address more subtle but serious forms of abuse
such as microaggressions. Second, they found that we need to develop proactive techniques to
prevent abuse in the future, rather than focusing only on automatic detection. Ultimately, they
argued, the community should play its role by contextualizing its efforts within a broader justice
framework, including expressive powers, restorative justice, and procedural justice, to support
and promote a healthier community. Another study by Vidge et al. [2] presented challenges and
limitations in detecting offensive content. They identified several challenges of the offensive
content detection task from three different perspectives. From a research perspective, there are
three challenges: difficulties in classifying offensive content, identifying offensive content, and
considering context. From the community's point of view, the biggest challenges are data
generation and dissemination and ethical issues. They also highlighted challenges based on the
limits of research, including several unexplored questions related to multimedia content,
implementation of justice and explainability, and transdisciplinary applications. McAveney et al.
[3] outlined and explored current challenges in expressing hate speech in text. To understand
the problem, they proposed a simple multi-view SVM approach that provides better
interpretability than more complex neural models. Based on the experiment, they found two
remaining problems in detecting hate speech in a text: (i) change of subject or subject's
perspective over time; (ii) Hate speech detection is a closed-loop system that focuses only on
the momentary characteristics of the phenomenon, while hate speech propagation is always
looking for ways to outsmart the system.

The scientific study of abuse language, especially in the field of NLP, has grown incredibly
rapidly over the last five years. The work of Schimdt and Wiegand [4] was the first study to
provide a brief, comprehensive and systematic overview of hate speech detection tasks. This
work presented what has been done so far in the hostile speech detection task, focusing on the
feature extraction approach. However, they also have several separate sections describing
bullying, classification methods, available data with labeling procedures, and general challenges
of hate speech detection tasks. The work of Fortuna and Nunes [5] complements the
aforementioned work by providing a more comprehensive critical review of the field. First, they
provided a more detailed discussion of the definition of hate speech, based on several previous
suggestions from other studies. They also reviewed the feature extraction approach,
categorizing it into general text-mining functions and specific hate speech detection functions. A
full description of the available data, including methods of collection and annotation, was also
provided. Finally, they described the challenges and opportunities arising from their research to
provide a better understanding of future research developments. Mishra et al. [6] also aimed to
provide a comprehensive picture of Internet usage detection tasks. This study presents existing
data and explores approaches to address this problem, including an analysis of their strengths
and weaknesses. In their conclusions, they highlighted remaining challenges in the field and
provided an overview of future developments: (i) abuse detection research still focuses only on
specific languages ​and also on specific abuse phenomena; (ii) most current approaches are
susceptible to word confusion; (iii) hardship in case of indirect abuse; (iv) the constant change in
abuse phenomena makes it difficult to identify new emerging phenomena.

Focusing specifically on the issue of resources to identify offensive phenomena, there are two
very recent studies that provide a critical overview of available resources, datasets, and
reference corpora to identify language use. Vidgen and Derczynski [7] presented a critical
analysis of existing offensive language resources, discussing the goals behind their
development, the taxonomies applied and the annotation procedure. They also discussed
different ways to share datasets, including the launch of https://hatespeechdata.com/, a website
for a constantly updated list of datasets flagged for hate speech, cyberbullying and offensive
language.
Finally, they presented best practices for creating offensive language data based on research
findings. Similarly, Poletto et al. [8] provided a systematic review of resources and benchmarks
for abuse speech detection tasks. They described different strategies for developing data
sections on hate speech based on five benchmarks, including type, topic specification, data
source, annotation procedure, and language. They also provided an overview of all the
resources available for hostile speech detection tasks by type, including corpora, resources
available for common tasks, as well as vocabularies.
Finally, they presented an overview of the impact of the keywords used for data collection on
the creation of hate speech corpora. Taken together, these recent studies on language
resources span and highlight the high availability of benchmark datasets for evaluating offensive
language and hate speech detection systems in multiple languages ​and across many topical
foci. The message is that such availability lays the groundwork for the urgent challenge of
exploring stable and beneficial architectures across languages ​and domains of abuse. However,
none of these works specifically and comprehensively address the multilingual and
multi-regional perspective and its associated challenges, although this is a key issue that we
address in this work, which aims to develop an agenda and compass for researchers in the
region. field for future work.

Resources

1. Jurgens D, Chandrasekharan E, Hemphill L (2019) A just and comprehensive strategy for

using NLP to address online abuse. In: Proceedings of the 57th annual meeting of the
association for computational linguistics. Association for Computational Linguistics (ACL), pp
3658–3666

2. Vidgen B, Harris A, Nguyen D, Tromble R, Hale S, Margetts H (2019) Challenges and

frontiers in abusive content detection. In: Proceedings of the third workshop on abusive
language online. Association for Computational Linguistics, Florence, pp 80–93.
https://doi.org/10.18653/v1/W19-3509, https://www.aclweb.org/ anthology/W19-3509

3. MacAvaney S, Yao HR, Yang E, Russell K, Goharian N, Frieder O (2019) Hate speech
detection: Challenges and solutions. Plos One 14(8):e0221152

4. Schmidt A, Wiegand M (2017) A survey on hate speech detection using natural language
processing. In: Ku L, Li C (eds) Proceedings of the fifth international workshop on natural
language processing for social media, SocialNLP@EACL 2017, Valencia, Spain, April 3, 2017.
Association for Computational Linguistics, pp 1–10. https://doi.org/10.18653/v1/w17-1101

5. Fortuna P, Nunes S (2018) A survey on automatic detection of hate speech in text. ACM
Comput Surv 51(4):85:1–85:30. https://doi.org/10.1145/3232676

6. Mishra P, Del Tredici M, Yannakoudakis H, Shutova E (2019) Author profiling for hate speech
detection. arXiv:1902.06734
7. Vidgen B, Derczynski L (2020) Directions in abusive language training data: Garbage in,
garbage out. arXiv:2004.01670

8. Poletto F, Basile V, Sanguinetti M, Bosco C, Patti V (2020) Resources and benchmark

corpora for hate speech detection: A systematic review. Language resources and evaluation.
https:// link.springer.com/article/10.1007/s10579-020-09502-8

Methodology:

NLP Techniques for the Detection of Abusive Language

In this section, we discuss highlights of papers focusing on computing for derogatory discourse
detection, as well as various studies focusing on related ideas. Finding the right highlights for a
clustering problem can be one of the most demanding tasks when using artificial intelligence.
Next, we will divide this particular section into sections to describe highlights officially used by
other authors. We divide highlights into two classifications: general highlights used in content
mining, which are regular in other areas of content mining; and specific obnoxious chat
detection highlights what we found in obnoxious chat search reports commonly identified as
symptoms of this problem. We present our survey in this area.

Features used in Text Analysis:

Most of the found writings try to adapt the certainly known methods of content mining to the
programmed site of hate speech. We characterized common highlights from content mining into
regularly used highlights. Let's start with the simplest methods, using word references and
vocabularies.

Lexicons:
One system of content mining is the use of vocabulary. This method includes a summary of the
speech being viewed and included in the content (dictionary). Defined numbers can be
legitimately used as highlights or to process results. Because of its recognition of disgusting
discourse, it is targeted for exploitation.

Distance Metric:

Some studies have pointed to the fact that in instant messages it is conceivable that hateful
words are covered by an intentional typo, usually a single character substitution. These terms
include "@ss", "shlt" or gay haters. , such as "joo". The Lowenstein difference, that is, the basic
number of changes that are important to change a word into another that can be used for a
specific reason. Different metrics can also be used to complement word-based terms/methods.

Bag of Words:

There is such a model that lexicons are word packages (consisting of several sets of words). In
this case, the corpus is configured to depend on the word in the preparation data, and not on a
given set of words, as in dictionaries. After collecting each word, the repetition of each word is
used as an element to prepare the separator. The disadvantages of such methodologies are
the neglect of word order and, in addition, syntactic and semantic substance. This can lead to
false distinctions when words are used in different settings. N-grams can be accepted to
override this limitation.

N-grams:

N-grams are one of the most used methods of hate speech in the transformed area and related
issues. The most widely observed methods of N-grams are to combine progressive words into
entries where N represents the size. In such a situation, the goal is to sum all word forms of size
N and calculate allocations. This license improves the implementation of classifiers because it
somewhat unifies the configuration of each word.
This method is not so powerless against spelling variations in word usage. For evidence
discriminating against harmful language, we find that character-based N-gram features are
more cautious than character-based N-gram features. One obstacle is that related words can
have a large gap in the sentence, and a response to this problem, such as expanding the N
ratio, violates the concern for speed. Also, thinkers note that higher N values ​work better than
any lower value.
The researchers report in the review that an explanation for the properties of N-grams is that
they are remarkably insightful on the modified obnoxious speech recognition problem, but
perform better when combined with others.

Profanity Spaces:

Untapped modes are a mix of the word reference method and the aforementioned method
called N-grams. Its purpose is to check if any other singular pronouns are followed by a
space-sized rude word, and then form a true or false segment with those messages.

Term Frequency Inverse Document Frequency (TF-IDF):

TF-IDF has been used for such game plan problems. TF-IDF is the critical range of a word of a
record in the collection and the complements of events with which the word appears in the
record. In any case, from the bag of words or N-grams, it is indisputable, considering how the
amount of repetition of one word made the repeated term more consistent, which is a kind of
compensation for the fact that a couple of words seem to be spoken. even more consistently.

Part of speech Feature:

The part of speech (POS) makes it possible to improvise the meaning of a given point and to
recognize the function of a word in relation to a sentence. These methods include word class
recognition such as singular pronoun, verb, non-third definite present singular construction,
adjectives, determiners, verb stem constructions. Part of the discourse was also used to identify
hate speech. In each case, POS was shown to cause a violation of category recognition rate at
the time it was used as salient cues.

Lexicon Based Features:

A standard analysis language proposed by the Stanford Natural Language Processing Group
was used to obtain the syntactic conditions of the sentence. The resulting features are sets of
words in the structure "(Congressman, subject)", where that of the department is the agent
position (e.g. "You, by any technology, doll." suggests that "non-hedgehog", the department, is
the pronoun "you", representative) variant). These functions are also used in the despise talk ID.

Rule-based Approaches:

Some standards-based systems are used to mine the substance. A method based on the law of
class association, which is more than numbers, is updated with phonetic information. Learning
is not excluded and they are based on a pre-collected summary using certain conventional
methods or again some verbal references to bits of subjectivity. For example, twitter used
rule-based strategies to manage antagonistic and exciting content using related words as
features. In addition, they added causative and related words that focus only on one or a
couple of individuals, chasing a socially dangerous event as features, whose real purpose is the
setting of terms.

Word Sense Disambiguation Techniques:

In this particular question, the senses of a word are identified in relation to the relevant phrase
or set of words that occurs in addition to that word. The study assumed the stereotypical mood
of the conversations to assess whether the topic was Semitic or not.

Point Classification:

The purpose of these highlights is to find the conceptual theme present in the report. In a
special report, a subject presenting phonetic prominence was used to distinguish gifts that have
a place in the element being characterized (race or religion).

Sentiment and Opinion:

Offensive language has a negative extreme; authors used opinion as an element to find hate
discourse. Various methods have been considered (eg multi-stage, single-stage). Researchers
usually use this component in combination with others that have been shown to improve results.
Word Embeddings:

Some researchers use the paragraph2vec method to treat customer comments as harmful or
clean, and in addition to predict the key word of the message. FastText is also used. The
problem cited in Dislike Discourse is that phrases should be grouped together instead of words.
A solution may be to average the vectors of all words in a sentence, but this strategy has limited
suitability. On the other hand, several content creators recommend adding annotations to solve
this problem.

Classification results for Logistic Regression on the complete dataset

 Classification results for Logistic Regression on high agreement dataset

Deep Learning:

In addition, deep learning methods have recently been used very precisely in the organization of
content and concept research: CNN, different variants of RNN (LSTM, Bidirectional-LSTM,
Stacked Bidirectional-LSTM), BERT and other architectures. We tested two options for the
embedding layer of our neural networks: (1) pretrained 300-dimensional GloVe embeddings and
(2) deeply contextualized ELMo embeddings. Adding a manually drawn (dense) feature vector
also improves several architectures. The most effective model in both the All-Data and
High-Agreement-Data scenarios is a neural architecture that includes a CNN and a dense
hand-drawn feature vector. It achieves an F1 score of 53.2% when evaluated on the entire
dataset and an F1 score of 68.6% for samples with high agreement. As expected, deep learning
architectures using the ELMo embedding model are more efficient than those initialized with
GloVe embedding.
This is because ELMo computes embedding dynamically using a pre-trained two-way
language model and can take linguistic context into account. Similar to traditional machine
learning results, the All-Data best architecture uses the entire response text as input, and the
High-Agreement-Data best model uses only the query text as input. Therefore, we find that
adding additional interaction context did not improve modeling efficiency. This also applies to
NLI-CNN, where we used more complex heuristics than simple concatenation to add contextual
information.

Other Features:

Different highlights used in this clustering task were in methods such as named object
recognition (NER), theme extraction, semantic refinement methods to check polarity,
frequencies of individual pronouns in the first and second person, emoticons, and case
proximity. letters In addition, wiring and drain pipes were used in the evaluations before the
component removal process. Post characteristics were similarly seen as hashtags, mentions,
retweets, URLs, number of names, terms used in tags, number of notes (again checking blogs
and the like), and association with visual and audio content. for example a picture, video or
sound attached to a message.

What has been done so far in abusive language detection study?

This section presents research on language abuse detection and focuses on building reliable
models in different domains. We collect all publications found in Google Scholar using four
main keywords, namely “cross-domain offensive language detection”, “cross-domain hate
detection”, “cross-platform offensive speech detection” and “cross-platform hate detection”.
These keywords are selected after several observations using different keywords. We limit our
query to the first five pages of each keyword and sort the results by relevance without a time
filter. In addition, we also check the works cited and references of the first five pages of each
article to obtain relevant publications. To avoid the loss of the latest works, we use the same
keywords in the publications of the most important NLP conferences of the last three years on
the platforms of ACL Anthology. Finally, we exclude some works that simply experiment with
different datasets without the goals and insights of domain agnostic models. The figure below
summarizes the document collection methodology for this survey.

We carefully read each article to get several important pieces of information that will be
discussed in this study. The table below summarizes the work done in this direction. Most
studies focused only on English, and we found only two studies on Italian [27] and Arabic [24].
Most of the chosen studies conducted a cross-domain experiment, where the domain can refer
to topical focuses or platforms. We also noticed that this research[4] focus is still relatively new,
with the earliest works initiated in 2018 [64, 145, 147]. All studies adopted a supervised
approach by training a model on a training set and predicting instances on the test set.
Following, we provide a deeper discussion to compare each work based on the models
(traditional machine learning based, neural based, or transformer based), features (a very wide
variant of features), and approaches adopted to deal with domainshift specifically.

Models

A wide variety of models was adopted to deal with this task. Some studies exploited traditional
machine learning approaches such as linear support vector machine classifiers (LSVC) [64, 92,
94], logistic regression (LR) [121], and support vector machine (SVM) [24, 147]. Their argument
for adopting the traditional approach was to provide better explainability of the knowledge
transfer between domains. Some other studies adopted several neural-based models, including
convolutional neural networks (CNN) [75, 141], long short-term memory (LSTM) [8, 75, 92, 94,
145], bidirectional LSTM (Bi-LSTM) [115], and gated recurrent unit (GRU) [27]. The most recent
works focus more on investigating transferability or generalizability of stateof-the-art
transformer-based models such as Bidirectional Encoder Representations from Transformers
(BERT) [19, 48, 66, 79, 83, 90, 92, 134] and its variant like RoBERTa [48] in the cross-domain
abusive language detection task. In the early phases of cross-domain abusive language
detection, specific models which adopt joint-learning [115] and multi-task [145] architectures
achieved the best performance. These architectures were proven to be effective for transferring
knowledge between domains. However, in the latest studies, transformer-based models
succeed in achieving state-of-the-art results. The most recent study by Glavas et al. [48] shows
that ROBERTa outperformed other models such as BERT in the crossdomain setting of the hate
speech detection task. This result confirms a recent finding on other natural language
processing tasks [18], i.e., that a pre-training language