Adult content in Social Live Streaming Services:

Characterizing deviant users and relationships

Nikolaos Lykousas∗ , Vicenç Gómez† , Constantinos Patsakis∗

∗ University of Piraeus, Greece [email protected],[email protected]
† Universitat Pompeu Fabra. Barcelona, Spain [email protected]

Abstract—Social Live Stream Services (SLSS) exploit a new successful, especially in young users2 , and LM has been ranked
level of social interaction. One of the main challenges in these as the top grossing social app in the U.S. on Google Play since
services is how to detect and prevent deviant behaviors that August 2016 and one of the top five social apps on Apple App
violate community guidelines. In this work, we focus on adult Store.
content production and consumption in two widely used SLSS,
namely Live.me and Loops Live, which have millions of users Both these apps share many similarities regarding com-
producing massive amounts of video content on a daily basis. We munity policies, e.g., they explicitly forbid broadcasters from
use a pre-trained deep learning model to identify broadcasters engaging in, or broadcasting any sex-related content that
of adult content. Our results indicate that moderation systems in
arXiv:1806.10577v1 [cs.SI] 27 Jun 2018

promotes sexual activity, exploitation and/or assault. Moreover,

place are highly ineffective in suspending the accounts of such both apps prohibit violence and/or self-harm, bullying, harass-
users. We create two large datasets by crawling the social graphs
of these platforms, which we analyze to identify characterizing
ment, hate speech, on-screen substance use, posting of private
traits of adult content producers and consumers, and discover contact information, prank calls to emergency authorities or
interesting patterns of relationships among them, evident in both hotlines and solicitation or encouragement of rule-breaking.
networks. There is a variation on the user’s age, as in LM users have
to be at least 18 while in LL the users have to be at least 13
I. I NTRODUCTION years old.
The wide adoption of mobile technologies have completely To counter possible violations of the aforementioned poli-
redesigned the way we consume and produce information as cies, both services have implemented reporting mechanisms,
well as the way we interact with people. This shift and cultural so that users can easily report a channel once they identify an
change has lead to the emergence of many new Social Media underage user or detect suspicious behavior, or violations of
platforms that focus on features and topics that the traditional the service policies. On top of that, LM employs a team of
ones like Facebook and Twitter are lacking, with typical human moderators around the world, working 24/7 to respond
examples including Snapchat, Periscope, and musical.ly. Many to users’ reports. Violators are subject to immediate suspension
of these platforms operate solely on mobile devices. or ban from the app. Those safeguards are in place to protect
Social Live Streaming Services (SLSS) are examples of this young people, since live streaming apps and sites can expose
new type of platforms in which users can actually live stream them to graphic and distressing content and can leave them
parts of their daily lives. These services provide a new level vulnerable to bullying and online harassment [1].
of interaction and hook their subscribers as the users become However, these mechanisms do not seem to be working as
part of the daily life of others. Practically, users decide when intended. Many users report, for example in the app reviews,
to open up their cameras and share snapshots of what they do, that they are constantly witnessing many violations of the
what they think or live at the moment with others and interact aforementioned policies. It is therefore a challenge to design
with them via chat messages. detection mechanisms of deviant behavior that scale up to the
In this work, we analyze a gray area of these services: massive amounts of streamed video data produced in these
adult content production and consumption. Clearly, most SLSS services.
have a clear policy against adult content and facilitate some Main Contributions: In this work we perform an in depth
mechanisms to detect and ban the misbehaving users, either analysis of two SLSS to understand and characterize deviant
in the form of filters from the service provider or by peer behaviors involving the production and consumption of adult
reporting. content in these platforms. To the best of our knowledge, this is
To this end we consider two SLSS, namely, Live.me (www. the first quantitative study of deviant behaviors in SLSS. First,
liveme.com) and Loops Live (www.loopslive.com), from now we collect two large datasets with user profile information and
on LM and LL, respectively. Both operate as video chat apps directed friendship links for LM and LL, following a sampling
solely in mobile phones and have millions of users that produce scheme that enables us to sufficiently cover the relevant part
massive amounts of video content on a daily basis. To quantify of social graphs. Next, we use a deep learning classifier to
the latter, Cheetah Mobile’s CEO; the company which owns automatically identify producers of adult content from the
Live.me, reported that more than 200,000 hours of live video available broadcast replays, and compare our findings with
are broadcast daily on Live.me1 . Both platforms are very the moderation (banned users) of each platform. While our
1 https://seekingalpha.com/article/4075406-cheetah-mobiles-cmcm-ceo-fu- 2 https://seekingalpha.com/article/4025223-cheetah-mobiles-cmcm-ceo-fu-
sheng-q1-2017-results-earnings-call-transcript sheng-q3-2016-results-earnings-call-transcript
results are consistent with the moderation of LL, we observe [11] analyzed traffic patterns and user characteristics of
many cases of undetected deviant behavior in LM. Moreover, YouNow. [12] crawled Inke and identifed that the main reasons
we characterize adult content producers and consumers based that users are hooked in these services are the follower-
on their profile attributes, and analyze their relationships to followee model, the awards incentivisation, and the multi-
discover interesting patterns. dimensional interaction between broadcasters and viewers.
Similar results, but with real users, were also reported by [13]
A. Ethical considerations for the case of Facebook Live, Periscope, and Snapchat.

Clearly our methodology has the capacity to collect large Legal and ethical questions about SLSS were raised
bodies of data, including streams, messages and metadata by [14]. Recently, [15] performed an empirical study on law
exchanged between individuals around the world. There are infringements in several SLSS. While the focus was not on
therefore certain privacy considerations that must be taken adult content, the researchers found that around 17.9% of
into account. To anonymize users, we allocated a new unique their sample, consisting of more than 7, 500 streams, somehow
random identifier for every user whose data we collected, violated a law, e.g., copyright, road traffic, insult, etc. Different
obfuscating her platform-wide identity (user ID). We highlight information behaviors of users, focusing on the assessment of
that the terms of both services underlines that all data (and streamers’ behavior with emphasis on produced content and
metadata for LM) and activity are by default public. Despite motivations, as well as demographics, were studied in [3], [4].
their “public” nature, we follow Zimmer’s approach [2]. In The copyright aspect is also studied in [16], but in terms of
this regard, the data remains anonymized during all the steps broadcasting sport events.
of our analysis, and we report only aggregated information.
The collected datasets are publicly available online 3 . B. Adult content in Social Media
In the computer science literature, adult content consump-
II. R ELATED W ORK tion has mostly been studied in the context of adult websites,
several of which incorporate social networking functionality
As the adult content problem on SLSS has not been studied and features. Examples include the work by [17] that provides
in the literature, we loosely categorize prior work into two an overview of behavioral aspects of users in the PornHub
main categories, reflecting the fundamental concepts present social network, a recent paper [18] on the detection of fake
in this study. Finally, we provide a functional overview of the user profiles in the same network, and various studies on the
two platforms that we study. categorization of content, frequency of use, and analysis of
user behavior in such platforms [19], [20]. To the best of our
A. Social Live Streaming Services knowledge, the only other work that studies the production and
consumption of adult content in general-purpose online social
In SLSS users are able to stream their own live shows in networks is a recent article by [21]. The authors perform a
real time as broadcasters, and to join the live shows of other large-scale analysis of the adult content diffusion dynamics in
users as viewers/audience. The audience is able to interact Tumblr and in Flickr, while also examining and comparing
with the streamers through a chat and reward them with the demographics of adult content producers and consumers
virtual rewards, e.g., points, gifts, badges (some of which are across these platforms. A wider corpus of research has been
purchasable), or money. Also, various SLSS give broadcasters produced by social and behavioral scientists, mostly based on
the opportunity to monetize part of the virtual gifts they receive surveys of relatively small numbers of individuals.
from the audience during their brodcasts. Users of SLSS
employ their own mobile devices (e.g. smartphones, tablets) or C. Live.me & Loops Live functional overview
their PCs and webcams for broadcasting. In contrast to other
social media, SLSS are mostly synchronous [3], [4], but they This study uses data collected from LM and LL platforms,
can also support asynchronous interactions between users, like introduced previously. Most of the features and functionality
direct messages and comments on broadcast video replays. offered by those platforms are mobile-only, in that users
wishing to actively participate in their communities need to
We differentiate between two kinds of SLSS: General own mobile devices such as smartphones and tablets running
live streaming services (without any thematic limitation), on Android or iOS.
e.g. YouNow, Twitters Periscope, Cheetah Mobile’s Live.me,
(now-defunct) Meerkat Streams, YouTube live or IBM’s Us- The dynamics of both communities are based mostly on
tream, and Topic-specific live streaming services, e.g. Twitch three possible actions performed by the users: (a) create
(games), or Picarto (art). real-time broadcasts and optionally associate hashtags repre-
senting thematic categories/user interests with them; (b) join
Since SLSS are quite new, the literature in the field is rather broadcasts created by other users and interact with them as
limited. Some of these studies investigate the performance of well as with the other viewers. Those interactions include
such services, e.g., Meerkat and Periscope [5], [6], Periscope exchanging chat messages with other viewers, and rewarding
[7] and Twitch [8]. Human factors and user experience were the broadcasters with “likes” and purchasable virtual gifts; and
studied in [9]. Having access to a large dataset of Inke, a (c) follow other users and receive notifications when they are
Chinese SLSS, [10] identified several patterns in the users, broadcasting.
e.g., fast interest shifts, user dedication to broadcasters as well
as the locality bonds between users. Contrary to other popular SLSS like Periscope [5], all the
broadcasts in LM and LL are public. All active broadcasts are
3 https://github.com/nlykousas/asonam2018. visible on a global public list. In both platforms, the concept of
re-sharing/re-posting broadcasted content across different users 1

is not present. Nevertheless, users are able to get shareable 2

links to live shows that can be used for promoting broadcasters 4 Proxy
on other social media.
5 3

As already discussed, both platforms enable users to report

community policy violators and underage users, who con-
sequently get their accounts banned after their activity has 6
been reviewed by moderators. Additionally, LM offers safety 7 Social Live
Streaming
features to proactively protect its users, like the “Admin” 8 Seed Services
feature, which enables broadcasters to allow other trusted users
to be administrators for their broadcasts to block commenters 9
Adult
on their behalf in real time. keywords list
Fig. 1: Data collection methodology. A proxy intercepts the
Both platforms are equipped with more advanced features.
messages from the smartphone to the SLSS. To decrypt the
Some significant examples are the ability to view currently
traffic and derive the API, a root certificate is installed in the
popular/trending or “featured” broadcasts, either globally (both
smartphone (steps 1-6). Then, we use an adult keyword list to
services), or by geographical region (LM), or by hashtag (both
get an initial set of seed users, which are then queried to collect
LM and LL) and the ability to find users or hashtags matching
even more users. Based on their properties (e.g. banned, gifts)
a search term. The mechanics of the broadcast featuring system
we build our dataset (steps 7-9).
are different for each platform, but in both cases factors such
as the number of viewers, the amount of user interaction within
the broadcast including likes, gifts and messages, and the
duration of the live show are taken into account. Moreover, of our knowledge, no open-source clients for these services
the popularity and experience of a user is reflected by their exist at the time of writing, hence, we follow a similar method
“level”, which is determined by their participation in activities as in [6]. For each platform, we analyze the network traffic
such as broadcasting, joining broadcasts of others, sending between the app and the service. More precisely, we set up
and receiving gifts, chatting, etc. Leveling up enables users to a so-called SSL-capable man-in-the-middle proxy between a
receive various privileges such as discounts for buying virtual mobile device with the specific apps installed and the LM
currency and access to premium gifts. and LL services that acts as a transparent proxy. The proxy
intercepts the HTTPS requests sent by the mobile device and
Broadcasters have the incentive to get their live shows pretends to be the server to the client and the client to the
featured, since this leads to a better visibility within the app, server, enabling us to examine and log the exchange of requests
thus attracting a higher number of viewers who in turn can and responses between the client apps and the servers.
potentially reward them with virtual gifts. Once a broadcaster
has received a certain amount of virtual gifts, they are able cash We select a set of APIs that allow us to crawl the social
them out for real money. Finally, both platforms offer a range graph edges, extract user profile and broadcast information,
of synchronous interaction features traditionally provided from and use the search capabilities offered by the services. Content-
the majority of OSNs like direct messaging between users and wise, both services use the HTTP Live Streaming (HLS)
the ability to “block” users. protocol [22] for hosting and delivering broadcast video re-
plays, similar to other well known live streaming services such
Follow edges in LM and LL social graphs are directed; as YouNow, Periscope and Twitch. Figure 1 illustrates this
users can follow other users who do not follow them back. In architecture (steps 1-5).
addition, following someone does not require their permission.
In the context of this study we focus specifically on the We first identify a set of seed nodes likely to be involved
user-specific attributes and following-follower social graphs of with the production of adult content, in order to bootstrap a
those platforms. subsequent crawling procedure for sampling the social graphs.
To accomplish this, we take advantage of the aforementioned
search APIs. A seed node is defined as a user that satisfies the
III. DATA COLLECTION METHODOLOGY three following conditions: (i) having a username that contains
In this section, we first describe our methodology for a pornographic term, (ii) having broadcasted activity, and (iii)
collecting and labeling the data. The objective of our data being banned by the system4 . For the first condition, we use the
collection methodology is twofold: to identify adult content list of adult keywords provided by [21] in the context of their
producers by analyzing the available broadcast replays and proposed deviant graph extraction procedure. This list contains
to sufficiently sample the portion of the social graphs where 5, 283 search keywords from professional adult websites.
adult content production and consumption phenomena are Using these three criteria, we were able to identify 390 and
predominant. To accomplish this, we develop a novel data col- 47 seed nodes for LM and LL, respectively. Figure 1 (steps
lection and labeling approach which we detail in the following 6-7) illustrates the seed identification step. Note that this step
subsections. does not consider the network structure. The next step (denoted
as 8-9 in Figure 1) consists in traversing and collecting profile
A. Sampling the social graphs
4 In both services, although the accounts of banned users are deactivated,
Both LM and LL applications communicate with their their past activity in the platform is still retained, thus enabling us to perform
servers using an API with SSL-protected access. To the best the described analysis.
TABLE I: Network statistics of the crawled graphs: number 1.0
of nodes |N |, number of edges |E|, number of banned nodes
|B|, average degree hki, density D, and reciprocity ρ.
|N | |E| |B| hki D ρ
4.32 ×10−6 0.5

CDF
LM 2,942,407 37,440,992 142,345 25.4 0.14
LL 273,177 1,193,780 114 8.73 1.59 ×10−5 0.08

LM seeds
LM all
LL seeds
information as well as broadcast video replays from each user, 0.0 LL all
following the friendship links. We follow a Breadth-First (BF) 0.0 0.5 1.0
Adult content production score
traversal limited to two hops away (undirected distance) from
the seed nodes. Thus, our network consists of the union of Fig. 2: Cumulative distribution function of the adult content
the 2-hop ego-networks of all seed nodes. This union resulted score values.
in one single connected component in both platforms. For
computational reasons, we discard those nodes in the boundary
that appear as neighbors of a node with degree higher than users: adult content producers, or simply producers (based on
10K. These nodes correspond to only 718 and 267 profiles their broadcast activity), consumers (based on their relation
for LM and LL, respectively, a very small proportion of the with producers), and normal users that are not included in any
complete 2-hop ego-networks. of the two other categories.

We emphasize that our interest is not in capturing the entire One could argue that these consumers are lurkers. The
network of users, but a tractable subset of tightly connected lurking phenomenon in social networks has been studied in
groups of users in which adult content is predominant. BF great depth [25], [26]. In general, lurkers are passive users
search covers satisfactorily small regions of a graph [23] and who do not contribute to the community. While consumers
has been used in many analyses. in our scenario could also be lurkers, we argue that, despite
the obvious resemblance, they are not. Indeed, their behavior
During the data collection period, which lasted from Jan. seems passive as they do not create content. Nevertheless,
to Nov. of 2017, we repeated this crawling procedure once they have actual interactions by, e.g., providing praise and
per week on average. Based on the number of installations currencies to producers, or by publicly chatting with the
reported by LM on Google Play (20M −50M installations), we producers, that promote specific behaviors and content.
managed to crawl roughly 5.8% − 14.5% and 5.46% − 27.3%
of the entire LM and LL networks, respectively. Given the network, our criterion to establish whether a user
is a producer is exclusively based on the images of the user’s
Table I summarizes the obtained networks for both plat- broadcast activity. This choice disregards indirect sources of
forms. As expected, the LM network is much larger than the information and does not require manual inspection, allowing
LL network, containing approximately 10 times more users us to scale up the method efficiently. Alternative approaches
and 30 times more edges. The LL network is, however, one are based on manual inspection of metadata only [21], which
order of magnitude more dense than the LM one. may not been sufficient for our purposes, or using crowdsourc-
The approach we followed has three main limitations. ing approaches for categorizing broadcasts [4], which would
Firstly, the set of replays that we captured includes only the require a pool of crowd workers to be potentially exposed to
available replays of past broadcasts at crawling time. Replays offensive material.
that were deleted in-between our crawls as well as all live To this end, we use OpenNSFW5 , a deep neural network
broadcasts streamed during our crawls were not included. This model pre-trained to detect pornographic images. Convolu-
is not a fundamental limitation, and can be fixed by using tional Neural Networks are the state of the art in image
more sophisticated approaches [5]. Moreover, while we can classification problems [27], [28]. OpenNSFW takes an image
determine whether an account is banned (suspended) or active, as input and provides a value representing confidence in an
none of the platforms provides metadata to determine the image’s resemblance to pornography. We feed the network
reason behind the account suspension. This means that our with frames sampled from the broadcasts at 1/3 Hz, and keep
dataset includes false positives that were banned because of the highest confidence score for every broadcast replay. This
other unrelated policy violations. This limitation is addressed value represents the maximum probability a replay contains
in the next subsection, in which we consider the replay’ content pornographic content. Then, at the end of our data collection
to determine whether a user is deviant or not. Finally, there period, we can associate every user with the highest value
is a small probability of false negatives, a portion of deviant provided by OpenNSFW over all of their replays in the dataset.
users that are not retrieved by our method. This can happen The aforementioned value can be considered as a user’s adult
because moderators can only identify a limited number of users content production score. For the users we were unable to
engaging in inappropriate behavior [24] and those may lie collect any broadcast data, we set this value to zero.
isolated (more than two hops away) from the seed nodes.
Figure 2 shows the cumulative distribution function (CDF)
B. Labeling the users of the adult content production score for both LM and LL
networks differentiating between seed users and all the users
Having described our procedure to identify an adult content
related network, we now describe how we label the users 5 https://yahooeng.tumblr.com/post/151148689421/open-sourcing-a-deep-
within this network. We differentiate between three types of learning-solution-for.
 TABLE II: Distribution of users according to their class. users are distributed in each class. In the case of LM, we
Class Live.me Loops Live observe that only 43.5% of the labeled producers have been
Producers 7,135 (228 seeds) 92 (33 seeds) banned. Since it is unlikely that the frames extracted from the
Consumers 30,872 (0 seeds) 1,243 (0 seeds) broadcasts contained adversarial perturbations [29] against the
Normal 2,904,400 (162 seeds) 271,842 (14 seeds)
OpenNSFW model, we can safely assume that moderation is
highly ineffective in detecting such cases.
in our sample. We observe that a very small proportion of all On the contrary, moderation of LL is consistent with
users (only around 0.4%) scored above 0.5, indicating that the our labeling outcome, with 96.7% of users placed in the
vast majority of users do not broadcast adult related material. producers class being banned. This consistency provides fur-
On the other hand, the seed users have been assigned high ther confirmation of our decision to use a pre-trained deep
scores (all starting at 0.5 for LM and around 0.8 for LL). This learning classifier for detecting adult content. Finally, the high
confirms that our choice of seed nodes and the outcome of the number of banned users placed in the normal class suggest the
classifier agree to a large extent. existence of a significant proportion of policy violators outside
the context of our study.
We establish whether a user is a producer using a thresh-
olding approach. In particular, we consider the probability
distribution of the scores for both banned and non-banned IV. P ROFILING DEVIANT USERS
users and choose as a threshold the Bayesian decision rule that
In this section, we present our efforts to characterize adult
separates the two classes. This results in a threshold of 0.82
content producers, consumers and their relationships in the
and 0.93 for LM and LL, respectively. Although the described
sampled networks. We first consider a set of features directly
approach does not require any human supervision, in order to
accessible from each user and analyze their relevance for
further evaluate the “goodness” of the threshold, we manually
distinguishing between classes: normal users, producers, and
inspected the frames for 100 LM and 50 LL random producers.
consumers. We then look at the network structure to gain
All of them contained either nudity or semi-nudity, suggesting
understanding about the relations between consumers and
the validity of our thresholding method.
producers.
We establish whether a user is a consumer based on the set
of producers and the network structure. In particular, we label a A. Features
user as a consumer if the user follows at least two adult content
producers. While following a single user (producer or not) can Based on the available profile information we collected
be expected by random chance, following two users of the from the two platforms, we define a set of features that can
producer class (given they only make up for a minor fraction be grouped as follows:
of the total users), is much less likely to be by chance. Our
definition of consumer is stricter than the one of [21], which • Network features: , Number of followers. , number of
defines as a passive consumer a user that follows at least one followings. , number of bidirectional friends.
single producer. In our analysis, for those users that fall in both • User-based features
categories, i.e., producers that also followed at least two other ◦ Pornographic username (binary): whether the username
producers, the producer label is considered more relevant. In contains a pornographic term.
practice, only 9 users of LM and 2 users of LL should have ◦ Suspended/Banned (binary): whether the account has
been labeled as both producers and consumers. Finally, users been suspended by platform moderators.
who do not fall into the above classes are labeled as normal ◦ Replay count: Number of past broadcasts available for
users. replaying.
◦ Level: An integer value reflecting the participation level
Table II summarizes the resulting labeling according to our of a user in various SLSS-specific activities.
proposed procedure. As expected, we observe that only a small ◦ Praise (only LM): Total number of likes received in all
proportion of the crawled networks are not labeled as normal user’s broadcasts.
users. We also show in parenthesis how the seed nodes are ◦ Income (only LM): Total virtual currency value of gifts
distributed in the three categories. Recall that seed users are received in all of user’s broadcasts.
banned users with broadcast activity and with a adult-related
username. Although most of them are labeled as producers, We assessed the relative power of these features in discrim-
there is a significant proportion labeled as normal users. This inating the three user classes by using the Mean Decrease
can be explained by the fact that those users may exhibit other Impurity (MDI) metric, where a higher score implies a more
(non adult-related) deviant behaviors and thus not relevant for important feature. Table IV reports the ranking of the top five
our analysis, or because their score did not reach our threshold, most important features in differentiating the three user classes
as reported from the OpenNSFW classifier. Remarkably, none for each platform.
of them are labeled as consumers, which already suggests that
producers are not well connected between them. TABLE III: Proportion (total in parenthesis) of banned ac-
counts in each class.
C. Effectiveness of SLSS moderation systems Class Live.me Loops Live
Having identified the aforementioned user classes, we Producers 43.5% (3,109) 96.7% (89)
proceed to examine how our labeling approach compares to Consumers 9.6% (2,970) 0.08% (1)
Normal 4.6% (136,266) 0.008% (24)
the moderation of each platform. Table III shows how banned
 1.0 1.0 1.0 1.0 1.0

0.8 0.8 0.8 0.8 0.8

0.6 0.6 0.6 0.6 0.6

CDF

CDF
CDF

CDF

CDF
0.4 0.4 0.4 0.4 0.4
LM Producers LM Producers LM Producers
LM Consumers LM Consumers LM Consumers
0.2 LM Normal 0.2 LM Normal 0.2 LM Normal 0.2 0.2
LL Producers LL Producers LL Producers LM Producers LM Producers
LL Consumers LL Consumers LL Consumers LM Consumers LM Consumers
0.0 LL Normal 0.0 LL Normal 0.0 LL Normal 0.0 LM Normal 0.0 LM Normal
100 101 102 103 104 105 100 101 102 103 104 105 100 101 102 103 104 100 101 102 103 104 105 106 107 100 101 102 103 104 105 106 107
Number of followers Number of followings Number of friends Number of likes - praise Value of gifts received - income

(a) # Followers (b) # Followings (c) # Friends (d) Praise (e) Income
Fig. 3: Cumulative distribution functions (CDFs) of the different user profile attributes.

TABLE IV: Top 5 features for differentiating the three classes. approximately 65% of consumers and the 33% of normal users
LM LL in our dataset fall in this “unpopular” category, see Figure 3d.
Rank Feature MDI Feature MDI
This either means that they have not received any likes during
1 #Followings 0.31 #Followings 0.37 their shows, or they have never broadcasted anything. A similar
2 #Followers 0.25 #Friends 0.26
3 Praise 0.15 #Followers 0.19 trend is observed for the total value of the virtual gifts received,
4 #Friends 0.12 Banned 0.10 represented by the income attribute and illustrated in Figure 3e.
5 Income 0.06 Porn nickname 0.05 Only 21% of producers have not received gifts, while the same
holds for the 88% of consumers and the 65% of normal users.

The number of followings, followers and friends are among

B. Deviant relationships
the highest ranked features for both networks, which suggests
relevance of social relationships for characterizing the given To determine the community structure of the these net-
classes. Also, for LM we observe that the amount of likes works, existing variants of the Louvain method [30] do not
(praise) and virtual gifts (income) are highly important as find well identifiable clusters of users. In both networks, we
well. To get a deeper insight on how these features are observe that producers and consumers are distributed nearly
distributed across the different classes, we plot their cumulative uniformly across the clusters. Further, the results vary sig-
distribution functions (CDFs) in Figure 3. Information about nificantly between different runs. We thus adopt a different
praise and income was not available for LL, preventing us from approach in order to better understand the network structure.
performing an 1-to-1 comparison between the two datasets.
Instead, we observe a high importance for the banned and In particular, we examine who in the sampled networks
pornographic username attributes. This is due to the fact that, is significant with regards to their social relationships. We
as shown in Subsection III-C, the banned LL users are almost use the ranking HITS algorithm [31] to identify the hubs and
exclusively adult content producers, and also a significant part authorities in the social graphs. The basic principle behind
of them have pornographic usernames (see Subsection III-A). HITS algorithm is the following mutually reinforcing relation-
ship between hubs and authorities: good hubs point to many
From Figure 3a, we observe that adult content producers good authorities and vice-versa. Interestingly, it appears that
tend to have many more followers than the other classes, and adult content consumers have the highest hub scores among all
there exists a lower bound to the follower number of producers, users in both networks, as shown in Figure 4b. Moreover, the
approximately 10 and 250 for LL and LM, respectively. In highest authority scores in LM belong almost exclusively to
contrast, consumers in LM are found to have the least amount producers. The latter could be correlated with the significance
of followers among the three classes. For the number of of the number of followers and followings to discriminate
followings (Figure 3b), however, the situation is reversed. producers and consumers, from normal users, as previously
Consumers dominate over the other classes by following shown. For LL, we observe that most authoritative users do
significantly higher amounts of users, while the producers not belong to the producers class, and exhibit characteristics
come last in this aspect with around 41% (LL) and 10% expected of prominent users in a social community such as
(LM) of them not following any other users. The distribution the number of followers in the order of hundred of thousands.
of the friend number reveals that consumers are much more The reason behind this difference between LM and LL could
likely to form reciprocal relationships, while it appears to be be attributed to the very limited extent of the adult content
almost identical for producers and normal users, as Figure 3c production behavior in the later. Therefore legitimate popular
indicates. Furthermore, we found that adult content producers users dominate the authority scores in the sampled graph by
tend to receive the highest amount of praise and income among being followed by consumers. Additionally, we notice that the
the three classes. We note that, although the higher (undirected) hub scores of the highly authoritative users are particularly
degree of consumers and producers is explained by the criteria low in both networks. This finding contradicts other studies
used in the seed selection, the edge directionality can not be on different social networks such as Twitter [32], [33], where
fully attributed to our sampling method, which is blind with researchers observed many well-connected users that have high
respect to it. scores as both authorities and hubs.
Additionally, consumers receive much less recognition for An interesting finding was that in LM, the ratio of banned
their broadcasting activities compared to both normal users and to non-banned users increases along the increase of authority
producers. In fact, while no producers with zero praise exist, score. To better demonstrate this, we bin the users based on
 100
Banned 1.0 orders of magnitude more densely connected to each other
Non-Banned
80 0.8
in the sampled networks. On the contrary, the subgraphs of
producers and normal users are much more sparse compared
% of users in bin

60 0.6 to consumers, with the producers being only slightly more

CDF
40 0.4
dense connected than random for LM. This finding comes
in contrast with the behavior of adult content producers in
20 0.2 LL Consumers Tumblr and Flickr, where they are observed to form densely
LL Producers & Normal
LM Consumers interconnected communities [21]. In LL the producers appear
0.0 LM Producers & Normal
0
[0, 0.2) [0.2, 0.4) [0.4, 0.6) [0.6, 0.8) [0.8, 1] 0.0 0.2 0.4 0.6
Hub score
0.8 1.0 to have a density comparable to those of consumers, but given
Authority score intervals
their limited number, this is due to the existence of producers
(a) authority score bins (LM) (b) CDFs of hub scores who also exhibit consumer behavior.
Fig. 4: User relationship insights, provided by HITS.
In summary, we can conclude that the closely knit groups
TABLE V: Comparison in terms of link density D between of consumers act as a “bridge” between the otherwise isolated
the crawled networks of producers, consumers normal users producer nodes. Concretely, from a network perspective, the
and a correspoding random network. most effective way to reach adult content in the studied
networks is by traversing the social links of consumer nodes
Class Crawled graph D Null model D that point to both producers and other consumers, enabling
Producers 1.90 × 10 −5
4.55 × 10−6 the reach of even more deviant nodes belonging in those two
LM Consumers 1.20 × 10−3 4.46 × 10−6
categories. Since adult content producers are isolated in the
network, we speculate that some of the consumers are actively
Normal 2.18 × 10−7 4.32 × 10−6
monitoring the list of active broadcasts (see Subsection II-C),
Producers 1.91 × 10−3 0 and proceed to follow users who broadcast adult content, while
LL Consumers 3.28 × 10−3 1.42 × 10−5 also possibly sharing links to such live streams with other
Normal 1 × 10−5 1.59 × 10−5 consumers. These “consumer leaders” are likely to become
popular among their kin by being followed by many other
consumers, thus serving as a means for diffusion of informa-
tion about producers, effectively compensating for the absence
their authority score and we calculate the fractions of banned of the content reposting functionality in SLSS.
and non-banned users in each bin, as shown in Figure 4a. We
observe that 99.5% of users in our sample fall into the first
bin, having authority score less than 0.2. We can thus conclude V. D ISCUSSION
that banned users are more densely concentrated towards the
higher end of the authority score spectrum, and the reason With the continuous growth of SLSS, an increase in deviant
behind their suspension was likely the production of adult behaviors on social media is expected. In our work, we
content, since they are followed by the consumers/hubs. A show that current moderation mechanisms may have important
similar phenomenon is observed for LL, but with banned users limitations when addressing the detection of adult content con-
mostly concentrated in the 0.02 − 0.35 authority score range, sumption and production. Our approach overcomes scalability
while the 97.7% of users have authority scores below 0.02. issues that appear when a large number of humans are needed
to categorize the content, at the cost of relying on the ac-
Based on the arguments above, we expect that consumers curacy of automatic image classification. Image classification
will follow multiple producers, a considerable portion of which is the primary application domain for machine learning [36],
will be banned. To quantify the relationship between the reaching human-level performance in many tasks. Our results
fractions of banned users and producers followed by con- could be further improved by replacing or accommodating the
sumers, we calculate their correlation using Spearman’s rank OpenNSWF classifier with more effective models.
correlation coefficient ρ. Indeed, there exists a nearly perfect
correlation for LL with ρ = 0.96, meaning that consumers The inefficiency of moderation can be partially attributed
do not follow almost any banned users outside the producers to a voyeur phenomenon. Many adult content producers are not
class, and a moderately strong correlation in LM (ρ = 0.63). reported to moderators as the consumers like the content, so
their accounts are not suspended, allowing them to continue
Another dimension to examine is the connectivity within broadcasting inappropriate content. Moreover, although con-
each class in the context of the sampled graphs. For this we suming any kind of content, including adult, is not explicitly
measure the edge density, computed as the ratio of edges prohibited by the community guidelines of these platforms,
between the users belonging in each class over the total number suspending the accounts of the users who intentionally seek
of possible edges between them. To account for the differences adult content would be meaningful, due to the law of supply
in sizes of the subnetworks [34], we resort to a comparison of and demand. It is therefore necessary to incorporate effective,
the connectivity of the sampled graphs with a null model that real-time detection mechanisms of deviant behaviors in the
randomly rewires the edges while keeping the degree of each existing moderation systems, in order to maintain the SLSS
node unchanged, as described in [35]. communities safe, especially for the younger audience.
Table V contains the link density comparison between the In future work, we will investigate quantitatively the iden-
subgraphs induced by producers, consumers and normal users tification between consumers and lurkers. Moreover, we plan
in each sampled network and the null model. We observe to develop graph-based features for the detection and classifi-
that consumers, when compared to the null model, are several cation of adult content producers and consumers in SLSS by
exploiting the characteristics of deviant behavior presented in [15] F. Zimmer, K. J. Fietkiewicz, and W. G. Stock, “Law infringements in
this paper, as well as study other available data from broadcast- social live streaming services,” in International Conference on Human
related user interactions in SLSS (chat messages, likes, gift Aspects of Information Security, Privacy, and Trust, 2017, pp. 567–585.
exchange), to further analyze the nature of deviant behaviors [16] M. Edelman, “From Meerkat to Periscope: Does intellectual prop-
erty law prohibit the live streaming of commercial sporting events?”
in such platforms. Columbia Journal of Law & the Arts, vol. 39, no. 4, 2016.
[17] G. Tyson, Y. Elkhatib, N. Sastry, and S. Uhlig, “Are people really social
ACKNOWLEDGMENTS in porn 2.0?” in 9th International Conference on Web and Social Media,
ICWSM. AAAI Press, 2015, pp. 236–444.
We thank Andreas Kaltenbrunner for his helpful comments [18] W. Magdy, Y. El-khatib, G. Tyson, S. Joglekar, and N. R. Sastry, “Fake
and suggestions. This work was supported by the European it till you make it: Fishing for catfishes,” in ASONAM. ACM, 2017,
Commission under the Horizon 2020 Programme (H2020), as pp. 497–504.
part of the Practicies project (Grant Agreement no. 740072), [19] M. Schuhmacher, C. Zirn, and J. Völker, “Exploring youporn categories,
and by the Spanish Ministry of Economy and Competitiveness tags, and nicknames for pleasant recommendations,” in Search and
under the Marı́a de Maeztu Units of Excellence Programme Exploration of X-rated Information : WSDM’13 Workshop Proceedings.
ACM, 2013, pp. 27–28.
(MDM-2015-0502).
[20] G. Tyson, Y. Elkhatib, N. Sastry, and S. Uhlig, “Demystifying porn
2.0,” in Proceedings of the 2013 conference on Internet measurement
R EFERENCES conference. ACM Press, 2013, pp. 417–426.
[1] S. Bearne, “Is live streaming your life good business or dangerous?” [21] M. Coletto, L. M. Aiello, C. Lucchese, and F. Silvestri, “On the
http://www.bbc.com/news/business-39778550, 2017. behaviour of deviant communities in online social networks.” in 10th
International Conference on Web and Social Media, 2016, pp. 72–81.
[2] M. Zimmer, ““But the data is already public”: on the ethics of research
in Facebook,” Ethics and information technology, vol. 12, no. 4, pp. [22] R. Pantos and W. May, “HTTP live streaming,” https://tools.ietf.org/
313–325, 2010. html/rfc8216, Tech. Rep., 2017.
[3] K. Scheibe, K. J. Fietkiewicz, and W. G. Stock, “Information behavior [23] M. Kurant, A. Markopoulou, and P. Thiran, “Towards unbiased BFS
on social live streaming services,” Journal of Information Science sampling,” IEEE Journal on Selected Areas in Communications, vol. 29,
Theory and Practice, vol. 4, no. 2, pp. 6–20, 2016. no. 9, pp. 1799–1809, 2011.
[4] M. B. Friedländer, “And Action! Live in front of the Camera: An [24] J. Cheng, C. Danescu-Niculescu-Mizil, and J. Leskovec, “Antisocial
Evaluation of the Social Live Streaming Service YouNow,” Interna- Behavior in Online Discussion Communities,” in 9th International
tional Journal of Information Communication Technologies and Human Conference on Web and Social Media, ICWSM, 2015, pp. 61–70.
Development, vol. 9, no. 1, pp. 15–33, 2017. [25] A. Tagarelli and R. Interdonato, “Lurking in social networks: topology-
[5] B. Wang, X. Zhang, G. Wang, H. Zheng, and B. Y. Zhao, “Anatomy of based analysis and ranking methods,” Social Network Analysis and
a personalized livestreaming system,” in Proceedings of the 2016 ACM Mining, vol. 4, no. 1, p. 230, 2014.
on Internet Measurement Conference. ACM, 2016, pp. 485–498. [26] D. Perna, R. Interdonato, and A. Tagarelli, “Identifying users with
[6] M. Siekkinen, E. Masala, and T. Kämäräinen, “A First Look at Quality alternate behaviors of lurking and active participation in multilayer
of Mobile Live Streaming Experience,” in Proceedings of the 2016 ACM social networks,” IEEE Transactions on Computational Social Systems,
on Internet Measurement Conference. ACM, 2016, pp. 477–483. vol. 5, no. 1, pp. 46–63, 2018.
[7] L. Favario, M. Siekkinen, and E. Masala, “Mobile live streaming: [27] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification
Insights from the periscope service,” in Multimedia Signal Processing with deep convolutional neural networks,” in Advances in neural
(MMSP), 2016 IEEE 18th International Workshop on, 2016, pp. 1–6. information processing systems, 2012, pp. 1097–1105.
[8] J. Deng, G. Tyson, F. Cuadrado, and S. Uhlig, “Internet scale user- [28] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image
generated live video streaming: The Twitch case,” in International recognition,” in Proceedings of the IEEE conference on computer vision
Conference on Passive and Active Network Measurement. Springer, and pattern recognition, 2016, pp. 770–778.
2017, pp. 60–71. [29] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfel-
[9] J. C. Tang, G. Venolia, and K. M. Inkpen, “Meerkat and periscope: I low, and R. Fergus, “Intriguing properties of neural networks,” arXiv
stream, you stream, apps stream for live streams,” in Proceedings of preprint arXiv:1312.6199, dec 2013.
the 2016 CHI Conference on Human Factors in Computing Systems. [30] V. D. Blondel, J. L. Guillaume, R. Lambiotte, and E. Lefebvre, “Fast
ACM, 2016, pp. 4770–4780. unfolding of communities in large networks,” Journal of Statistical
[10] M. Ma, L. Zhang, J. Liu, Z. Wang, W. Li, G. Hou, and L. Sun, “Char- Mechanics: Theory and Experiment, vol. 2008, no. 10, mar 2008.
acterizing user behaviors in mobile personal livecast,” in Proceedings [31] J. M. Kleinberg, “Authoritative sources in a hyperlinked environment,”
of the 27th Workshop on Network and Operating Systems Support for Journal of the ACM, vol. 46, no. 5, pp. 604–632, sep 1999.
Digital Audio and Video, ser. NOSSDAV’17. ACM, 2017, pp. 43–48. [32] K. Lee, P. Tamilarasan, and J. Caverlee, “Crowdturfers, campaigns,
[11] D. Stohr, T. Li, S. Wilk, S. Santini, and W. Effelsberg, “An analysis and social media: tracking and revealing crowdsourced manipulation of
of the YouNow live streaming platform,” in 2015 IEEE 40th Local social media,” in Seventh International AAAI Conference on Weblogs
Computer Networks Conference Workshops (LCN Workshops). IEEE, and Social Media, 2013, pp. 331–340.
oct 2015, pp. 673–679. [33] A. Java, X. Song, T. Finin, and B. Tseng, “Why we twitter: Under-
[12] J. Zhao, M. Ma, W. Gong, L. Zhang, Y. Zhu, and J. Liu, “Social standing microblogging usage and communities,” in Proceedings of the
media stickiness in mobile personal livestreaming service,” in Quality 9th WebKDD and 1st SNA-KDD 2007 Workshop on Web Mining and
of Service (IWQoS), 2017 IEEE/ACM 25th International Symposium on. Social Network Analysis, ser. WebKDD/SNA-KDD ’07. New York,
IEEE, 2017, pp. 1–2. NY, USA: ACM, 2007, pp. 56–65.
[13] O. L. Haimson and J. C. Tang, “What makes live events engaging on [34] J. Leskovec, J. Kleinberg, and C. Faloutsos, “Graphs over time:
Facebook Live, Periscope, and Snapchat,” in Proceedings of the 2017 densification laws, shrinking diameters and possible explanations,” in
CHI Conference on Human Factors in Computing Systems. ACM, Proceedings of the eleventh ACM SIGKDD international conference on
2017, pp. 48–60. Knowledge discovery in data mining. ACM, 2005, pp. 177–187.
[14] C. Faklaris, F. Cafaro, S. A. Hook, A. Blevins, M. O’Haver, and [35] S. Xiao, G. Xiao, T. H. Cheng, S. Ma, X. Fu, and H. Soh, “Robustness
N. Singhal, “Legal and ethical implications of mobile live-streaming of scale-free networks under rewiring operations,” EPL (Europhysics
video apps,” in Proceedings of the 18th International Conference Letters), vol. 89, no. 3, p. 38002, feb 2010.
on Human-Computer Interaction with Mobile Devices and Services [36] M. I. Jordan and T. M. Mitchell, “Machine learning: Trends, perspec-
Adjunct. ACM, 2016, pp. 722–729. tives, and prospects,” Science, vol. 349, no. 6245, pp. 255–260, 2015.