Impact of Counter Narratives - ONLINE - 1 PDF

THE IMPACT OF
COUNTER-NARRATIVES
Insights from a year-long cross-platform pilot study of
counter-narrative curation, targeting, evaluation and impact
Tanya Silverman
Christopher J. Stewart
Zahed Amanullah
Jonathan Birdwell
www.againstviolentextremism.org
About
The Institute for Strategic Dialogue (ISD) is a London-based ‘think and do tank’ that has pioneered policy and
operational responses to the rising challenges of violent extremism and inter-communal conflict.
Combining research and analysis with government advisory work and delivery programmes, ISD has been at the
forefront of forging real-world, evidence-based responses to the challenges of integration, extremism and terrorism.
The Against Violent Extremism (AVE) network is a unique and powerful new global force in the ongoing struggle
to tackle violent extremism. Former violent extremists ('formers') and survivors of violent extremism (‘survivors’) are
empowered to work together to push back extremist narratives and prevent the recruitment of 'at risk' youths.
AVE uses technology to connect, exchange, disseminate and influence all forms of violent extremism (from far right and
far left to AQ-linked and inspired and gangs). It leverages the lessons, experiences and networks of individuals who
have dealt first-hand with extremism. Through the website and YouTube channel, members can stay in touch, share
ideas, collaborate, find investment and partners, and project their messages to wider audiences.
About the authors
Tanya Silverman coordinates the Against Violent Extremism (AVE) network at the Institute for Strategic Dialogue. AVE
is a partnership between ISD, Jigsaw (formerly Google Ideas) and Gen Next which comprises of former extremists and
survivors of extremism. Previously, she was involved in all areas of the organisation’s security and counter-extremism
work, particularly counter-narratives.
Christopher J. Stewart is a programe associate at the Institute for Strategic Dialogue, focussing mainly on ISD's Online
Civil Courage Initiative (OCCI). The OCCI was launched by Facebook in Berlin, January 2016, and is designed to
counter hate speech and extremism online. Previously, Christopher helped to deliver innovative counter extremist
campaigns in projects run with private partners Google, Facebook and Twitter, and government partners such as, the
United States State Department.
Zahed Amanullah heads the counternarrative projects at ISD, with extensive experience in creating and managing on
and offline resources and technology to combat radicalisation, hate speech, and to promote dialogue between Muslim
communities and wider society. He is the Chair of the Concordia Forum, a global network of leaders from Muslim
backgrounds that holds conferences and exclusive retreats annually. He is a founding board member of CEDAR, a
pan-European network of Muslim professionals and a founding board member of AMILA (American Muslims Intent
on Learning and Activism). Zahed has a BS from the University of California, Berkeley and a graduate diploma in
management from the University of Bath.
Jonathan Birdwell is Head of Policy and Research at the Institute for Strategic Dialogue. Jonathan leads ISD’s research
and policy strategy, policy reports and policy maker networks, including the Policy Planner’s Network, the Strong Cities
Network, and the educational film series, Extreme Dialogue. Prior to joining ISD, Jonathan was based at the think tank
Demos, where he authored reports on the relationship between non violent radicals and violent extremists, and online
supporters of far-right xenophobic political parties and street-based movements.
Acknowledgements
The authors would like to thank Yasmin Green at Jigsaw (formerly Google Ideas) for her deep involvement with this pilot project, as well as Caitlin
MacDonald for her editorial input in this publication.
All errors and omissions are the sole responsibility of the authors.
Copyright © 2016 Institute for Strategic Dialogue, All rights reserved. Any copying, reproduction or exploitation of the whole or any part of this document
without prior written approval from the Institute for Strategic Dialogue is prohibited.
This material is offered free of charge for personal and non-commercial use, provided the source is acknowledged. For commercial or any other use,
prior written permission must be obtained from the Institute for Strategic Dialogue. In no case may material be sold, altered, or rented. The Institute
does not generally take positions on policy issues. This views expressed in this publication are those of the authors and do not necessarily reflect the
views of the Institute for Strategic Dialogue.
Table of Contents
SUMMARY 5
INTRODUCTION 10
• Project aims 11
• Selecting partner organisations 12
• Methodology 13
CAMPAIGN OVERVIEWS 14
• Targeting 15
• The ExitUSA Campaign 16
• The Average Mohamed Campaign 18
• The Harakat-ut-Taleem Campaign 20
CAMPAIGN FINDINGS AND ANALYSIS 22
• Awareness 23
• Reach, gender and age 25

• Video views and viewer retention 25
• Viewer retention versus engagement metrics 26
• Engagement 27
• Likes, shares and comments 27

• Breakdown of engagements 29
• Engagements: Increasing the online presence of the campaign 30
organisations
• Comments 31
• Impact 36
• Sustained engagement 36
• Sustained constructive engagement 37
• Sustained antagonistic engagement 42
• Summary of impact 44
RECOMMENDATIONS 46
• Strengths and weaknesses of platforms 48

• Recommendations for counter-narrative campaigners 48
• Recommendations for social media platforms 49
CONCLUSION 50
GLOSSARY 52
SUMMARY
Executive Summary
The concept of creating counter-narratives1 in order to push back against extremist

recruitment and propaganda has become well established in recent years. In practice,
however, it has proven difficult to curate this content in a systematic way, target it toward
at risk audiences, and - most importantly - measure constructive impact on their behaviour.
Over the past few years, the Institute for Strategic Dialogue (ISD) has developed research
and pilot studies conducted with the Against Violent Extremism (AVE) network of former
extremists and survivors of extremism that it manages. This research has been conducted
in order to test and improve methodologies that help optimise the impact of counter-
narrative campaigns.
This report presents the development, deployment, and evaluation of three counter-
narrative campaigns orchestrated by AVE and Jigsaw (an incubator within Alphabet
that uses technology to address geopolitical issues) with additional in-kind and financial
support from Facebook and Twitter. This project builds on the findings of a previous
counter-narrative pilot project conducted in 2014 and attempts to find ways to achieve
greater scale, authenticity, efficiency, reach, and impact.
The key objectives of this project were to:
• Assist a wide geographic, ideological, and cultural variety of small non-profit

organisations to develop and disseminate counter-narrative content to target
audiences by utilising a variety of social media advertising tools;
• Analyse the strengths, weaknesses, and overall effectiveness of different social media
platforms to facilitate reach and engagement among target audiences; and
• Provide guidance to build the capacity of similar non-profit organisations to produce

effective counter-narrative content in the future by creating ‘how-to’ videos and a
counter-narrative ‘toolkit’.2
To achieve these objectives, AVE identified two pre-existing organisations and assisted
them in the creation of counter-narrative content along with the development and
execution of a target audience strategy. In the third example, AVE created a fronting
organisation from scratch, building a brand through multiple accounts across multiple
platforms in response to security concerns from the third party organisation.
The three campaigns included:
• Average Mohamed. A non-profit organisation that uses animation to encourage
1 A counter-narrative is a message that offers a positive alternative to extremist propaganda, or alternatively aims to deconstruct or delegitimise extremist
narratives. For further definitions of terms refer to the glossary at the end of this publication.
2 www.counternarratives.org
The Impact of Counter-Narratives |5

critical thinking among Somali youth (in Somali and English) about extremist
ideologies.
• ExitUSA. A project of the US-based non-profit organisation ‘Life After Hate’, which
aims to discourage individuals from joining white power movements and encourage
defection by offering a ‘way out’.
• Harakat-ut-Taleem. A front organisation created by AVE for a third party Pakistani

communications company with sound experience in creating counter-narrative
documentaries and content. They aim to counter Taliban recruitment narratives in
Pakistan.
The campaigns were deployed in October 2015 with a total of 15 videos spread across
the organisations. The methodologies used in the campaigns were informed by AVE’s
previous experience using social media advertising tools to reach target audiences.
While the three counter-narrative campaigns are different in terms of content, approach
and target audiences, a number of common findings emerged from the campaigns. The
evaluation criteria used for evaluating the campaigns were ‘awareness’, ‘engagement’
and ‘impact’.
Key findings
Our hypothesis was that a small amount of funding and guidance for counter-narrative
campaigners, in terms of deploying social media advertising tools to reach ‘target
audiences’, could dramatically improve the awareness, engagement and impact of
counter-narratives and NGOs working in this space. The findings presented in this report
support this hypothesis in a highly compelling way.
The three campaigns received over 378,000 videos views and over 20,000 total
engagements, including likes, shares, replies, retweets and comments. Over 480
comments were made in response to the content.
Average Mohamed’s page likes on Facebook increased by sevenfold, while the campaign
doubled its number of Twitter followers and subscribers on YouTube. ExitUSA tripled
its Twitter followers, doubled its YouTube subscribers and increased its Facebook page
likes by 48 per cent. Harakat-ut-Taleem went from no presence on social media to 116
Facebook likes, 6 YouTube subscribers, and 62 Twitter followers.
Our qualitative analysis of comments, and in particular ‘sustained engagements’, provide

a persuasive indication of content inspiring the consideration of different viewpoints,
critical thinking and sowing the seeds of doubt. Average Mohamed’s videos inspired
young Muslims to debate the role of gender in Islam and the struggle of having multiple
identities. ExitUSA’s videos led to constructive and antagonistic exchanges with users
who clearly held neo-Nazi views. Harakat-ut-Taleem’s videos performed better in Urdu
in Pakistan than they did with English sub-titles in the UK, demonstrating the ability for
6| Institute for Strategic Dialogue

counter-narratives coordinated in the UK to have a global impact.
Most importantly, the campaigns demonstrated that people going through the process of
personal deradicalisation are willing to reach out and contact an organisation on social
media in response to a counter-narrative campaign. In the clearest examples of impact,
eight individuals reached out to ExitUSA asking for assistance ‘getting away from hate’ in
response to their campaign.
Recommendations
This project demonstrates that a coordinated effort between content creators, social
media companies, and private sector partners can substantially boost the awareness,
engagement and impact of counter-narrative campaigns and NGOs.
One of the key aims of this project is to assist small and medium size NGOs in producing,
disseminating and evaluating counter-narratives and counter speech. Based on these
campaigns, we make the following recommendations to campaigners:
• Allocate modest amounts of funding in order to dramatically increase the reach

and impact of counter-narrative campaigns. A Facebook-based counter-narrative
campaign, for example, used a £2,500 ($3,750) budget to achieve a reach of +670,000
individual users.
• Conduct or utilise in-depth research on target audience interests and behaviours in

order to maximise the potential of social media advertising tools and thus the impact
of a campaign.
• Test a range of topics and message tones with similar target audiences – as a form
of A/B or ‘split testing’ – in order to determine which resonate, and then apply the
findings to the campaign.
• Train at least one member of staff to become an expert at using social media and
marketing tools. Ensure that social media and marketing analytics are properly applied
and monitored during the campaign.
• Supplement counter-narrative campaigns with user engagement strategies. This may

require labour-intensive activities, such as engaging in conversation with audience
members.
• Use alternative media channels in geographic areas where access is limited or

censored.
• Coordinate online counter-narrative campaigns with offline campaigns or events.
• Be aware of the presence of online ‘bots’ used to scrape websites and social media.
These may skew results and give a misleading picture of impact and reach.

• Spread online campaigning budgets over as long a period of time as possible. This
can also help ensure that campaigns maximise reach with finite resources, and allow
them to be responsive.
The findings from this project also demonstrate that each social media platform
provides different strengths and weaknesses for disseminating advertised content to
particular audiences across the radicalisation spectrum, from upstream (few or no signs
of radicalisation) to downstream (more signs of radicalisation). Facebook produced the
greatest reach, video views and engagement for each campaign. YouTube provided the
lowest cost-to-views ratio and the highest rates of viewer retention. Twitter provided
the second largest number of video views across platforms as well as the highest
engagement-to-impressions ratio.
In addition to recommendations for counter-narrative campaigners, this pilot project also

suggests recommendations for social media platforms to support and streamline this
process. The most important recommendation is the need to provide detailed training
and updated ‘how to’ guides and toolkits for NGOs following changes made to advertising
interfaces.
Next Steps: Recommendations

This project shows the potential to significantly augment and amplify the vital work that
many organisations are doing to tackle violent extremist narratives online. More can be
done to scale this work to a greater extent by generating more data to help understand
impact.
The science of measuring the impact of online counter-narratives is still in its early days.
In this project, we have begun to build what is likely the first ever evaluation framework
for online counter-narratives that goes beyond ‘reach’ and ‘engagement’ figures. With
the beginnings of a framework in place, what’s needed now is to test and compare a wide
range of counter-narratives to add to and begin to construct a true, scientific evidence
base for understanding impact. This would revolutionise our understanding of counter-
narrative impact. In particular, this should include:
• Sourcing a wider range of content that allows for testing of different counter-
narrative components. This would include: 1) intervention ‘type’ (i.e. prevention
versus deradicalisation content; 2) ideology (i.e. far-right versus Islamist messages);
3) different message tones (i.e. thoughtful, personal story, cartoon).
• Explore the possibility of identifying and redirecting pre-existing counter-narrative

content with greater facility. This project demonstrated that the process of creating
counter-narrative content can be slow and arduous. One possible method of getting
around this bottleneck is to use social platforms to categorise and map the landscape
of counter-narrative content by analysing a broad range of ‘pre-existing narratives’
from campaigners across the social media spectrum. This could then be used to

explore the feasibility of redirecting and targeting pre-existing, ‘natural world’ content
(as opposed to curated) to key audiences.
• Use of offline market research techniques to better understand user journeys,

experience and relationship between online content and offline opinion or
behaviour change. This should include focus groups and user experience testing with
target or proxy audiences to help understand how people react to and engage with
content. This should also include in-depth interviews with formers and mentors and
intervention providers who work with young people in the context of prevention and
deradicalisation in order to gain a better understanding of how to analyse comments
and sustained engagements.

INTRODUCTION
Introduction
The concept of creating counter-narratives3 in order to push back against extremist

recruitment and propaganda has become well established in recent years. In practice,
however, it has proven difficult to curate this content in a systematic way, target it toward
at risk audiences, and - most importantly - measure constructive impact on their behaviour.
Over the past few years, the Institute for Strategic Dialogue (ISD) has developed research
and pilot studies conducted with the Against Violent Extremism (AVE) network of former
extremists and survivors of extremism that it manages. This research has been conducted
in order to test and improve methodologies that help optimise the impact of counter-
narrative campaigns.
This report outlines the key findings and results of three counter-narrative campaign
pilot projects curated through the AVE network in coordination with Jigsaw and with
additional in-kind assistance or financial support from Facebook and Twitter. Although
some details and figures have been omitted for security purposes, the results of the
study are presented in full.
Project aims:
• Review a selection of counter-narrative campaign concepts in collaboration with
three NGOs covering a variety of geographies and ideologies
• Curate campaign content with these organisations whilst developing their

understanding of best practices from our own research
• Set target audiences, disseminate the campaigns, and monitor their impact over a set
duration
• Analyse campaign success metrics and produce recommendations for NGOs and
social media platforms based on this data

The insights derived from these campaigns will help to further build knowledge and an
evidence base of effectiveness for the growing field of counter-narratives. In particular,
we highlight how advertising tools can be utilised, and evaluations undertaken, in a cost
effective manner with the aim of encouraging small and medium-sized charities to create,
disseminate and evaluate counter-narratives themselves.
Both quantitative and qualitative measures were used in order to effectively assess each
of the counter narrative campaigns. These measures included:
3 A counter-narrative is a message that offers a positive alternative to extremist propaganda, or alternatively aims to deconstruct or delegitimise extremist
narratives. For further definitions of terms refer to the glossary at the end of this publication.

• Awareness metrics: indications of whether target audiences were reached
(including impressions, reach, video views and viewer retention).
• Engagement metrics: how much users interacted with each respective campaign’s
content and social media accounts and shared content (including video retention
rates, numbers of comments, shares, likes).
• Impact metrics: whether campaigns’ content appeared to encourage critical

thinking and spark discussions around violent extremism by target audiences through
sustained engagements, or encourage a target audience to reach out directly for
support.
Selecting partner organisations
In order to identify organisations to produce counter-narrative content

for this project, the AVE network put out a call for proposals amongst
AVE members and to wider networks. AVE offered to provide partner
NGOs with expertise and additional in-kind support from Facebook, and
in-kind advertisements from Twitter, to scale and target their counter-
narratives.
The organisations chosen by AVE were as follows:
ExitUSA is an “exit”/outreach programme run by Life After

Hate (LAH). It is intended to help individuals who want
to leave white supremacist groups, as well as provide
support for former members of these groups. Their focus
is on the far right in the USA. They have not engaged with
targeted counter-narrative campaigns previously.
Average Mohamed is a non-profit organisation that uses

the medium of animation to counter the ideology of
Islamist extremist groups. Created by a Somali-American,
it uses democratic principles as an alternative to channel
frustrations both online and offline. Their focus is on ISIS
and other Islamist extremist groups. Although Average
Mohamed was featured at the White House CVE Summit
as an example of a counter-narrative, previouslty it did
not employ an online targeting element to their video
campaigns in order to reach the right audience.

Harakat ut Taleem is a front organisation created by AVE
for the counter-narrative content campaigners that chose
to remain anonymous for the campaign, and this report,
due to security concerns around disseminating anti-
Taliban content in Pakistan. The organisation has worked
with the development sector for the past seven years and
created numerous awareness-raising campaigns. Their
current focus is on Islamist extremism and, specifically,
the Taliban in Pakistan. They have not developed targeted
counter-narrative campaigns previously.
Methodology
The aim of this project was to increase the capacity of organisations to undertake counter-
narrative campaigns designed and delivered for a target audience. In addition to providing
guidance on the creation of counter-narrative content, a variety of online platforms were
employed and the campaigns assessed through each platform’s respective advertising
and targeting capabilities. The methodology was chosen to be replicable and utilised by
small-medium organisations with limited technical knowledge and small budgets, and
was informed by AVE’s Counter-Narrative Toolkit.
With its lower cost than traditional advertising methods, social media platforms provide
the third sector with an opportunity to promote their work to key target audiences. The
platforms chosen for this project were Facebook, Twitter and YouTube. These were
chosen because of the sophistication of their advertising platforms as well as the billions
of users these platforms reach on a regular basis. AVE had previously used Facebook
advertising and Google AdWords (which is used on YouTube). This was the first time AVE
utilised Twitter’s advertising function, and received in-kind support from Twitter to do so.
Other niche social media platforms, including Ask.fm, Telegram, and Instagram, were
initially considered but could not be included in the study because they lacked an
advertising function (Ask.fm), employed encryption in their messaging streams (Telegraph),
or because their advertising function was still in development (Instagram).

CAMPAIGN OVERVIEWS
Campaign overviews
Each organisations’ goals and campaign objectives helped to plot their online campaign,
understand what video content to produce and reach their target audience with that
content. Gaining a brief overview of their social media experience helped to recognise
their overall expertise in social media, not limited to social media marketing, and provide
recommendations to improve their online presence. As social media marketing is a useful
tool for reaching target audiences, AVE worked with organisations to produce targeting
criteria to use on each platform that would enable their content to be seen by the right
people.
Each counter-narrative video can be categorised into a number of ‘tones’ with different
message ‘content’. This has been informed by ISD’s previous work with One to One
online interventions4. The different tones that each video can be categorised into are:
scholarly, reflective, sentimental, and casual. Equally important, the content of the
videos can be categorised as presenting: consequences of negative actions, personal
questions, ideological challenges, personal stories, and offers of assistance. It is useful to
categorise the tone and content of videos to understand what counter-narrative content
might resonate with each organisation’s respective target audience.
Targeting
The following profiles on each campaign provide an overview of the targeting criteria
used to try to reach the intended audiences. AVE advised each organisation to draft a
case study of a mock target audience which, alongside previous campaigns and research
conducted by ISD and the AVE network, informed the targeting criteria. The full lists
of locations and ‘interests’ were expansive for each campaign, and these profiles only
present a sample. They give an indication of both broad and specific interests that were
used, and can act as an example for future campaigns about the depth and breadth of
research required to create successful targeting criteria.
The report does not provide the targeting criteria for each individual video within each
campaign, although AVE did tailor each of the targeting criteria depending on the
message and content of the video. For example, within Average Mohamed’s campaign,
the Be Like Aisha video used more female-focused targeting due its message of Muslim
female empowerment. In Harakat-ut-Taleem’s campaign, the targeted locations chosen
(regions and cities within Pakistan and the UK) reflected whether or not the video had
English subtitles or not.
4 http://www.strategicdialogue.org/wp-content/uploads/2016/04/One2One_Web_v9.pdf

The ExitUSA Campaign
ExitUSA’s counter-narrative campaign was made up of four videos, designed to discredit
far-right extremist groups, ‘sow the seeds of doubt’ in far-right extremist individuals, and
promote their exit program among to disaffected ‘formers’ looking for a way out, and
their concerned families and friends.
Organisation Goals and Campaign Objectives
GOALS OBJECTIVES
• Encourage disaffected far-right members to • Increase online awareness over the issue of
disengage from their groups and seek the far-right extremism in the US
help of ExitUSA • Engage directly with far-right extremists in
• Provide a supportive community for former an online setting
far-right extremists • Build awareness of the organisation's
• Sow the 'seeds of doubt' in the minds of far- project among family and friends of would-
right members be extremists
The primary focus of the campaigning period was to garner video views. However, the
campaign period also placed ads for engagements to understand if these types of ads
would encourage more comments than video view ads.
Video Content and Campaign Targeting
ExitUSA’s video development used the stories of both ExitUSA staff and AVE’s ‘formers’
to highlight personal stories and the "false truths" often told by extremist groups. The
organisation was keen to target the videos at a broad geographic audience, as their
research indicated violent white-supremacists were sparsely located across all areas of
the US.
AGE, GENDER AND INTERESTS AND TWITTER

LOCATION KEYWORDS FOLLOWERS
13-35 & 35-60+ Stormfront, Aryan Brotherhood, #WhiteResister, WHITE PRIDE,
Male and Female Justice for Germans, Aryan Southern Belle, BossMan, Van
Locations in the US Nations, Cultural Marxism, Von Faust, WHITE WATCH
White Voice Network, National 14/88, Awakened Saxon, UNSB,
Front, Stop White Genocide, fin_lander, SEIG, Nazi Follower,
White American, Michigan A. White, light4rights #WR,
Militia, Alabama Crimson Tide, NSM Nationalist Talk, White
Confederate States Army, Oak Resistance, WhiteFront, Storm
Creek, Wisconsin, Sodom and Fog, Valiant 87, DOUG AKA
Gomorrah, American History X, THOR, WhiteWizard, Grand
Nazi Germany, Aryan, Right Wing Wizard, Aryanpower18, SS
News, Hardcore Skinhead or Command (NNP)
Apartheid, We are at War, White
Power, Confederate Flag, WR,
WPWW, KKK, Ku Klux Klan, 88,
Affirmative Action, 14 Words, Jim
Crow Laws

ExitUSA VIDEO TITLES AND DESCRIPTION
NO JUDGEMENT JUST HELP
The video explains that since 9/11 home-grown

white supremacists have killed more Americans
on US soil than ISIS, Al-Qaeda and the Taliban
combined and continues to offer a way out to
those involved in white supremacist movements.
This video has a reflective tone and presents an
ideological challenge and an offer of assistance.
THERE IS LIFE AFTER HATE
Tim Zaal spent half of his life thinking that he had

killed a man one night when he was out looking for
a fight. It wasn’t until he came face to face with his
victim many years later that he realised the power
of forgiveness. This video has a reflective and
sentimental tone, and presents a personal story.
OAK CREEK
This video tackles the same issues as No Judgement

Just Help but is presented in a different format.
THE FORMERS
The Formers in this video begin by provocatively

telling the viewer the ‘truths’ they believed whilst
still in their violent extremist movements. At the
end of the video they reveal that these were ‘lies’
that they fell for, highlighting the realities of hateful
ideologies. This video has a reflective tone and
presents an ideological challenge and personal
story.

The Average Mohamed Campaign
Average Mohamed’s campaign consisted of five videos, aimed at reaching young Somali-
Muslims living in the United States as part of a preventative educational campaign. The
organisation hoped to instil the critical thinking needed to build resilience among youth
and prevent violent-radicalisation in future.
GOALS OBJECTIVES
• Create messages using Islamic principles • Promote five videos to young Somali-
that promote peace and encourage Muslims living in the US, each with a distinct
democracy theme, in a bid to educate youth and
• Empower young Muslims through counter discredit extremist thinking
ideology messages and discourage them • Established Average Mohamed as an
from joining Islamist-extremist groups online voice of counter-extremism and pro-
democracy
The Average Mohamed campaign involved five videos that each addressed a distinct
theme: identity, gender equality, democracy, being a Muslim in Western culture, and
slavery. The videos revolved around the central character, Average Mohamed, who
would discuss the themes with other characters. The organisation used an animation
style designed to resonate with a young audience.
The target audience for the Average Mohamed campaigns was primarily young
Somali-Americans aged 14-25 in communities with high Somali Muslim populations in
Minneapolis, San Diego, Seattle and Washington, but also including a broader US reach.
The organisation also wanted to go beyond a US audience, which led to experimental
targeting in the UK. The counter-narrative content is educational and preventative, which
allowed for broader targeting.

14-25 Niqāb, Hijab, Quran, Extremism, AbuTurki, Meshal, Alshammari,
Male and Female Prophets and Messengers in Islam, Meshari, Althomali, Saif_alhasan,
Locations in the US and UK Muhammad, Islamic State, Mujahideen, FAISAL, Hafsa haji elmi, Barkhad,
Burqa, Human Rights, Womens Rights, Mohammed Omer, Harbi Jama,
Sunnah, Fard, Imam, Islamism, Fiqh, Isse Islmail Yasin, Adbibarry Yassin,
Hadith, Aisha, Sharia, Allah, Women Kali_duul, Adan Suheyfa, Abdifatah
in Islam, Sheikh or Prophets in Islam,
Jama Mire, Ibrahim Gedow Awale,
Almuharjirah, Baqiyah, Bagiyahfamily,
Abdi Gudcur, Sa'eed Mahmoud,
digitaldawah, IAmAhmed, Haram,
Ummah, Muslimummah, Muslimah,
Abdirahman, Junyadi, Abdikadir
AverageMohamed, Slavery, Identity, aden, Salman Sheikh, InaAhmed,
Abd Allah, Sunna, Womens Rights Islam, Shadija 11/15, Nurr, Ridwan, Abas
Aisha, Jijab, Niqaab, Kuffar, Shirk, Hijara, Kareem Nor, Somali Pictures,
Wear Burka, Muslims West, Islam Airfizzo's Wife, Everything_Somalia

AVERAGE MOHAMED VIDEO TITLES AND DESCRIPTION
BE LIKE AISHA
Whilst many Islamist extremist groups say that

women shouldn’t be heard, this video explains
that Aisha was a pillar of Islam and a mother to all
Muslims. She was an important theologian and the
source of over 2000 hadiths. The video is scholarly
and reflective and offers an ideological challenge.
A MUSLIM IN THE WEST
This video tackles the key themes of an identity

crisis, duality in identity, and understanding other
cultures. It questions whether the viewer lives in the
‘old’ or ‘new’ world, and encourages them that they
can be a part of both Islam and the West. The video
has a sentimental and casual tone and presents a
personal question.
IDENTITY IN ISLAM
Tackling the issue of identity a Muslim youth in the

West might face, this video discredits the need to
label oneself into these categories. It unpacks the
realities of a multi-cultural world and explains that an
individual has many identities. It has a sentimental
tone and asks a personal question.
ISLAM AGAINST SLAVERY
The video explores the idea of slavery in Islam,

and explains that Muslims have a duty of care to
the needy, especially women. It discredits slavery
using Quranic verses to explain that extremists’
interpretation that, slavery is okay, is wrong. It
utilises a scholarly and reflective tone and presents
an ideological challenge.
THE BULLET OF THE BALLOT
This video tackles a key theme of democracy using

music to create a slick and catchy animation. It
also tackles the issues addressed in the previous
4 videos. The tone is casual and the content asks a
personal question.

The Harakat-ut-Taleem Campaign
The Harakat-ut-Taleem was a front organisation created to protect the anonymity of the
Pakistan-based organisation behind the campaign, which consisted of six videos (three
videos in Urdu and three with English subtitles) designed to dissuade young people from
joining the Taliban in Pakistan and show Western young people the destructive nature
of violent extremist groups through personal stories. The videos combined testimonial
narrations, with dramatisation of the narrator’s story. This campaign has now been
removed from the internet.
GOALS OBJECTIVES
• Discourage young Pakistanis in the UK • Discourage young Pakistanis in rural areas

and Pakistan from joining an Islamist- of Pakistan from joining the Taliban by using
extremist group by highlighting the negative the personal stories of those with first-hand
consequences experience of the violent extremist group
• Establish an anonymous online CVE
organisation to disseminate counter-
narratives in an area with a high-security risk
For this campaign, the counter-narrative campaigners had pre-existing videos that were
ready to be used with some modifications to make them more ‘social media friendly.’
As the content was in Urdu we reached out to an Urdu-speaking AVE former to ensure
the content was emotive and true to the situation in Pakistan. We helped to enlist the
assistance of one of YouTube’s resident creatives who also spoke Urdu and had a good
understanding of the content. From this, AVE recommended the third party organisation
shorten the length of the videos to retain interest in the target audience, re-film some
video-content to add to the emotive impact of the films, and include English subtitles on
the video to broaden the scope of the campaign to English-speaking Pakistani’s in the
UK.

14-25 E.g: Authentic Taweed, Yusha Evans, Masoodzai, M Rafi, Stanikzai, Irshad,
Male and Female Prisoners of Faith, Anwar al-Awlaki, Raghand, Faridon Noor, Amir Afghan,
Locations in Pakistan and UK Women in West for Khifah, Aniem Zahidullah, Zaland, Mansoor Shah,
Choudry, Islamic State, Green Birds, Abdul Wajid, Shinwary Ali, Gul
Ummah News, Jihad, Syria, Religion Afghan, Parmakhtag.com, RT Pashto
Truth, Tafseer, Al-Qadr, Khyber, Hadith,
Allah, Baqiyah, Taliban, Baqiyafamily,
Tehrik-i-Taliban, Dieinyourrage, Taliban,
GenerationKhilafah, FreeAbuBaraa

HARAKUT-UT-TALEEM VIDEO TITLES AND DESCRIPTION
IMPLICATIONS OF OUR ACTIONS

ENGLISH SUBTITLES/URDU
This video highlights the negativity of the Taliban’s
actions by showing how involvement ensnares
families and creates divisions. It also explores the
importance of education by discussing polio.
ISLAM IS PEACE
In this video the narrator tells his story, following his

father’s death and his new responsibilities as man
of the house. He reflects back on when he began
to be indoctrinated at the mosque, and the imam’s
influence on young people. A key theme is the
importance of repentance and redemption.
PROTECT YOUR LOVED ONES

The protagonist in this video details the crimes he

witnessed as a member of the Taliban, including the
kidnap of his uncle and the sexual assault of women,
which ultimately led him to leave the movement and
seek redemption.

CAMPAIGN FINDINGS
AND ANALYSIS
Campaign Findings and Analysis
There are a number of metrics which can be useful to measure how many people viewed
and engaged with video content. In addition, the social media platforms chosen for
disseminating the campaigns allow the aggregation of vast amounts of data based on
these metrics. However, understanding the impact counter-narrative content has on an
individual’s opinions and their behaviour remains a huge challenge.
The different metrics that we used to analyse how the counter-narrative campaigns
performed included the following:
• Awareness: The number of people who the videos were advertised to, measured
primarily by ‘impressions’ data across platforms, and ‘reach’ on Facebook;
• Engagement: The number of people who engaged with the videos, including ‘likes’,
comments, shares, link clicks and page likes accrued by the promoted videos, as well
as video retention rates; and
• Impact: The amount of sustained engagement achieved by users, particularly in the

‘target audience’, and qualitative analysis of comments and discussions.
Quantitative measures of ‘awareness’, ‘reach’ and ‘engagements’ provide a general

indication of the number of people engaging with the campaigns. However, qualitative
analysis comments and sustained engagements provide a better window into the impact
that a campaign is having.
One objective of this study was to help NGOs understand which measures and platforms
are best suited to them and their aims and roughly what costs they can expect are needed
to achieve them (see Table 1 for costs of reach and engagement for the campaign).
Ultimately, improving our understanding of how to measure the impact of counter-
narrative campaigns requires further testing on a much larger scale.
Awareness: Impressions, reach, video views, and retention

Our metrics of ‘awareness’ capture how many people were exposed to the counter-
narrative campaigns and how many people subsequently viewed that content. These
metrics include ‘impressions’, ‘reach’, ‘video views’ and ‘viewer retention’.
Facebook, Twitter and YouTube each offer a different definition of what an ‘impression’
is. For Facebook an ‘impression’ includes the number of times the content appeared in
a newsfeed or side-bar; for YouTube, how many times the content or ad appeared as in-
stream advertisement; and for Twitter, the number of times a promoted tweet appeared
in a user’s feed. Facebook also provides an additional ‘reach’ metric, which refers to

the number of individual users who had the content or advert appear on their personal
newsfeed.
In general, the more impressions a counter-narrative video receives, the greater number
of people who will potentially watch and engage with the video. Across the three
campaigns, there were over 1.6 million impressions achieved on Facebook, YouTube and
Twitter. Table 1 below illustrates the cost of so many people being shown the content for
this project.
TABLE 1: COSTS FOR REACH AND ENGAGEMENT BY CAMPAIGN
CAMPAIGN TITLE ExitUSA AVERAGE HARAKAT-UT-

MOHAMED TALEEM
Overall cost £839.72 £930.47 £816.36
Reach 212,051 456,113 88,351

FACEBOOK
Cost/1000 £3.96 £2.04 £9.24
Reach
Cost/ £0.39 £0.29 £1.46
Engagement
Overall cost £392.84 £454.50 £12,002
Reach N/A N/A N/A

YOUTUBE
Cost/1000
£10.37 £7.12 £12.71
Reach
Cost/
Engagement £0.04 £0.02 £0.06
Overall cost £927.32 £1,754.00 £1,754.00
Reach N/A N/A N/A

TWITTER
Cost/1000
£7.35 £17.01 £17.39
Reach
Cost/
Engagement £0.55 £0.24 £0.36
With the smallest potential audience, Harakat-ut-Taleem received the highest impressions-
to-reach ratio. This meant that a small group of people were seeing the videos appear on
their newsfeed multiple times. A counter-narrative campaign looking to produce an online
‘flash-mob’ to a very specific audience could find this useful, but for Harakat-ut-Taleem it
was not a specific aim and left it as the least cost efficient of the three campaigns.
Further testing would be needed to determine if a higher impressions-to-reach rate

leads to higher levels of engagement with counter-narrative content, or whether it would
be better to spread campaigns over a longer time frame if they are targeting smaller
audiences. It may be that users need to have content advertised to them multiple times
before they engage with it.
Reach, gender and age
Advertising tools can also provide an indication of the demographic breakdown of the
target audience being served content. Figure 1 below illustrates that, across campaigns,
there is a notably greater percentage of males being ‘reached’ by the respective
campaigns on Facebook. These types of insights can help an organisation understand
who is responding to their content, and to consider designing different content for more
specific demographic groups (e.g. female specific videos targeting only females through
advertising tools).
100%
MALE
PERCENTAGE OF VIDEO WATCHED
90%
80% FEMALE
70% UNKNOWN
60%
50%
40%
30%
20%
10%
0%
AVERAGE MOHAMED ExitUSA HARAKUT-UT-TALEEM
Figure 1: Gender breakdown of Facebook ‘reach’ metric across three campaigns
Video views and viewer retention
Overall, across the three campaigns – including all twelve videos – there were 378,694
video views during the campaign period. Our analysis showed that higher numbers of
video views were achieved on Facebook and Twitter, respectively, across the campaigns.
‘Viewer retention’ rates –the duration a viewer keeps watching a video – provide a more
valuable indication of whether the video content is actually being watched by users
online.
For example, Average Mohamed’s campaign on Twitter achieved more video views than
both Harakat-ut-Taleem and ExitUSA combined, but it was ExitUSA’s videos that had
a higher retention rate on average across all videos. This suggests that getting lots of
views do not necessarily translate into sustained views.
Looking at viewer retention rates can also reveal at what point in a video a user’s attention
is lost. This can inform future content design: by highlighting uninteresting parts of a

video that can be changed, or by encouraging the production of a shorter video, which
may be more likely to put the right message across to the audience.
While the different statistics available for advertisers makes it difficult to provide
comparisons between viewer retention rates across platforms, the data from the
campaigns suggests that – while there are higher numbers of video views of Facebook
and Twitter – viewer retention rates appear to be significantly longer on YouTube.
This could be because YouTube is primarily a video-based platform, meaning users might
be more willing to watch a video ad for longer. It could also suggest that YouTube’s
targeting tools – based on users’ behaviour, rather than self-reported demographic and
interest information – are more effective at reaching interested target audiences.
As a point of intrigue, we found that the Harakat-ut-Taleem videos targeted in Pakistan

(without English sub-titles) received higher viewer retention rates compared to the
English subtitled versions targeted in the UK (see Figure 2). This could suggest that the
content was resonating more with a Pakistani audience given the content of the videos
and messages.
That said, the English subtitled versions did not underperform and the difference is
marginal. This could further suggest that counter-narratives tackling issues outside of the
location they are targeted in can perform well. This would need to be tested further, but
it implies that counter-narrative content production could be replicated across different
localities, saving organisations time and money.
100%
ENGLISH (SUB)
PERCENTAGE OF VIDEO WATCHED
90%
80% URDU (ORIGINAL)
70%
60%
50%
40%
30%
20%
10%
0%
IMPLICATIONS OF OUR ACTIONS PROTECT YOUR LOVED ONES ISLAM IS PEACE
Figure 2: Average % of video viewed for Harakat-ut-Taleem’s Urdu and English-subtitled versions.
Viewer retention versus engagement metrics
Looking for an indication of the relationship between viewer retention rates and user
engagement – including likes, shares, and comments, the data suggests that there is less
of a correlation between the average length of a video being watched and the number of
engagements, than there is with overall views or impressions. This was revealed when
analysing the Average Mohamed videos in particular.

A Muslim in the West had the highest average percentage of video viewed on Facebook,
but the lowest number of total engagements. On the other hand, Be like Aisha, had the
highest number of engagements on Facebook but not the highest average percentage
of video viewed. It did, however, receive the highest number of impressions and video
views on Facebook. This may mean that the message in the Be like Aisha video resonated
quickly with the viewership and they were compelled to engage with the content. The
Slavery video had low engagement combined with the lowest average viewer retention
rate, and suggests that the video did not resonate with the target audience as well as
the other videos. This could be because the message was lost with the slow start to the
video, or that the issue of slavery was less compelling to the target audience compared
to gender and identity issues. Indeed, the ‘A Muslim in the West’ video had the highest
average percentage of video viewed, suggesting that the topic of dual identities is one
that resonated with Average Mohamed’s target audience.
This is not conclusive and would require more testing. Conducting focus groups, market
research or user experience research with target audiences could provide further insights
into message resonance and how viewer retention relates to engagement and ultimately
attitude shifts. However, the feasibility of this would depend on the target audience. For
example, ExitUSA’s target audience are individuals with far right propensities, or those
who are currently in far right groups. Of course, finding individuals such as this to take
part in focus groups and research might not be practical or safe.
Engagement: Likes, shares and comments

Engagement metrics outline how many people liked, disliked, shared, retweeted, or
commented as a result of the campaign videos. Whether this quantitative engagement
data can, on their own, be taken as a measure of a counter-narrative’s success or impact
depends on a campaign’s objectives. For some campaigns which are targeted further
‘upstream’ at prevention or education (such as Average Mohamed), simply reaching more
people, and more people engaging with the content could be used as a measure for
success or impact. However, for organisations like ExitUSA – which are more focused on
disengagement or deradicalisation – sheer quantitative measures are more ambiguous.
Nonetheless, ‘engagement’ metrics across all types of campaigns are an important
component of overall evaluation as they do give some indication of how counter-narrative
content – including message types and tone – resonate with target audiences.
As is the case with reach, what constitutes an ‘engagement’ differs across the three
platforms: likes, shares, comments, link clicks and page likes from promoted posts,
adverts and videos on Facebook; comments, likes and dislikes on the videos on YouTube;
retweets, favourites, media clicks and replies to, and of, promoted tweets on Twitter.
Table 2 displays the total engagement figures for each campaign, across each platform.
This includes ‘engagement’ with ‘video view’ adverts, as well as ‘engagement’ adverts. In
total, the campaigns received 20,694 ‘engagements’ during the campaign period.

TABLE 2: TOTAL ENGAGEMENTS FOR BOTH ‘VIDEO VIEW’ AND
‘ENGAGEMENT’ ADVERTISEMENTS
FACEBOOK YOUTUBE TWITTER CAMPAIGN

TOTALS
AVERAGE
MOHAMED
3,241 210 7,359 10,810
HARAKUT-UT-
TALEEM
646 3 4,814 5,463
ExitUSA 2,127 42 1,692 4,421
PLATFORM
TOTAL 6,014 255 14,425 20,694
For all campaigns, total ‘engagements’ were significantly higher on Facebook and Twitter
compared to YouTube. This is due to a number of factors, including how people generally
use different social media platforms and what social media platforms offer and label
as engagements. For example, on YouTube users can comment, like or dislike videos,
compared to a wider range of ‘engagements’ that users can take on Facebook. Moreover,
it may be that users in general are more likely to like or comment on content on Facebook
or Twitter, than they are on YouTube. This highlights the limitations of simply presenting
a quantitative number of engagements as a sign of ‘success’ in terms of measuring the
impact of counter-narrative content on different platforms.
With this caveat in mind, engagement-to-impressions ratios give a snapshot of campaigns’

performance. As Table 3 illustrates, Twitter provided the highest engagement-to-
impressions ratios, followed by Facebook. Again, this is in part driven by how Twitter
functions as a platform and defines engagements. Ultimately, this data needs to be
collected for a much greater number of counter-narrative campaigns to help get a better
indication of what represents a good engagement-to-impressions rate for different
platforms and different target audiences.
TABLE 3: ENGAGEMENT-TO-IMPRESSION RATIO ACROSS SOCIAL

MEDIA PLATFORMS
FACEBOOK YOUTUBE TWITTER
AVERAGE
MOHAMED
1 : 141 1 : 304 1 : 14
HARAKUT-UT-
TALEEM
1 : 582 1 : 19,293 1 : 21
ExitUSA 1 : 142 1 : 889 1 : 123
TOTAL 1 : 288 1 : 6,829 1 : 72


Breakdown of engagements
As we break down different engagement types across the different platforms, we begin
to understand cross-platform dynamics more clearly.
The first series of tables below present Facebook engagements broken down by
campaign videos. On Facebook there appeared to be a correlation between impressions
and engagement metrics. This was confirmed by the fact that Average Mohamed had the
highest levels of engagement across all of their videos, as well as the highest number of
impressions and the broadest target audience. For ExitUSA and Average Mohamed we
ran adverts for ‘video views’, while for Harakat-ut-Taleem we ran adverts for ‘engagements’
to test out these different options. There appeared to be a clear indication that video
view ads are better at fuelling engagement than engagement ads.
Twitter garnered the highest number of total engagement for Average Mohamed (7,354),
followed by Harakat-ut-Taleem (4,814) and ExitUSA (1,692). Overall, there was less of a
correlation between impressions and engagements on Twitter compared to Facebook.
All three campaigns received just over 100,000 impressions (with ExitUSA receiving
slightly more at 126,625). Yet, Average Mohamed received over four times as many
engagements as ExitUSA on Twitter.
RETWEETS ON
40 HARAKAT-UT-TALEEM’S CONTENT
RETWEETS ON
275 AVERAGE MOHAMED’S CONTENT
RETWEETS ON
141 EXITUSA’S CONTENT

Average Mohamed received the highest number of retweets overall (278) compared to
Harakat-ut-Taleem (141) and ExitUSA (40) as well as ‘likes’ (352, 286 and 32, respectively).
However, all three campaigns received a similar number of ‘replies’. Interestingly, Harakat-
ut-Taleem received the highest number of new followers (69), suggesting the value of
targeted Twitter campaigns to build the online presence of new organisations.
While there was some indication that the two videos that performed best on Facebook –
ExitUSA’s Formers video and Average Mohamed’s Be Like Aisha video – also performed
best on Twitter (Formers had the most total engagements among the ExitUSA videos,
while Be Like Aisha had the most retweets among Average Mohamed’s videos), the
performance metrics across all of the campaign’s videos was more evenly spread on
Twitter.
YouTube garnered a considerably lower number of engagements across campaigns

compared to Facebook and Twitter. Moreover, the videos that received the most
engagements on Facebook or Twitter were not necessarily the videos that received the
most engagements on YouTube. In ExitUSA’s YouTube campaign, Oak Creek received

the most attention from the target audience, receiving 21 total engagements, including 6
likes, 2 dislikes, 13 comments. The Formers video, which performed best on Facebook,
received the second highest number of engagements with 11 likes, 1 dislike, and 4
comments. The remaining two videos produced negligible results. This is not to say that
the video content was not interesting (as we can gauge from the average percentage of
video viewed on YouTube, discussed later). It could simply be because the content of the
videos did not incite a reaction, as Oak Creek and Formers is more provocative in nature.
Similar to ExitUSA, Average Mohamed’s videos that performed best on YouTube were
not necessarily videos that performed best on Facebook and Twitter. On YouTube, it was
the Identity video that accumulated the most number of engagements, including 35 likes,
18 dislikes, and 22 comments. The low number of engagements on Harakat-ut-Taleem’s
videos on YouTube were the result of difficulty of targeting in Urdu, and because YouTube
is banned in Pakistan. The English subtitled versions of the videos also did not do very
well in terms of inspiring engagements.
Engagements: Increasing the online presence of the campaign

organisations
In addition to users’ engagements with the campaign videos, we were also able to
measure ‘engagements’ with the organisations overall and measure the extent to which
the campaigns helped to boost their online profile. Table 4 provides the before and after
figures for Facebook page likes, organic research per day, Twitter followers and YouTube
subscribers.
The figures show that support for a one month targeted campaign can have a significant
impact on increasing the online presence of organisations doing important prevention
and disengagement work. They also provide quantifiable evidence that the campaigns
were effective at reaching their target audiences.

TABLE 4: BEFORE AND AFTER ENGAGEMENT STATISTICS
FOR ALL ORGANISATIONS
FACEBOOK TWITTER YOUTUBE
CAMPAIGN CAMPAIGN PAGE ORGANIC

FOLLOWERS SUBSCRIBERS
PERIOD LIKES REACH/DAY
Before 126 0 to 3 90 137
AVERAGE
733 256 to 93 152
MOHAMED Gained
15,474
Total at end 859 0 to 1,339 183 274
Before 192 1 to 34 50 16
ExitUSA 94 956 to 2,779 105 16

Gained
Total at end 286 0 to 32 155 32
Before 0 0 0 0
HARAKAT-UT-
TALEEM Gained 116 0 62 6
Total at end 116 0 62 6
At the completion of the project, Average Mohamed’s page likes on Facebook increased
by sevenfold, while the campaign also doubled its number of Twitter followers and
subscribers on YouTube. ExitUSA tripled its number of Twitter followers, doubled its
YouTube subscribers and increased its Facebook page likes by 48 per cent. Finally,
Harakat ut-Taleem, created as a new organisation for this project, gained 116 page likes
on Facebook, 62 followers on Twitter, and six subscribers to their YouTube channel, (See
Table 4)
Comments
Commenting on content suggests a deeper form of engagement (or being moved by

the content) than liking or sharing, whilst at the same time providing researchers with a
potentially clearer indication of how users are responding to counter-narrative content.
In terms of our metrics, ‘comments’ are counted in the comment section of promoted
posts and videos on Facebook and YouTube, and replies to promoted tweets on Twitter.
In total, the campaigns generated 484 comments during the campaign period. This
included 340 comments made on Facebook, 68 on YouTube, and 76 on Twitter across all

three campaigns.5 In terms of the breakdown of comments by campaign:
COMMENTS WERE MADE ON

40 HARAKAT-UT-TALEEM’S CONTENT

305 AVERAGE MOHAMED’S CONTENT

139 EXITUSA’S CONTENT

Whilst Facebook garnered the most comments for ExitUSA and Average Mohamed,
Harakat-ut-Taleem received the bulk of theirs on Twitter. Our analysis focused primarily
on comments on Facebook, with comments categorised into four subsections:
• Supportive comments praising the counter-narrative campaign, organisation, video

or subjects addressed in the video.
• Negative comments expressing distaste towards the counter-narrative campaign,

organisation, video or subjects addressed in the video.
• Misunderstood comments, likely expressing negativity (above) but due to a

misunderstanding of the content.
• Unrelated comments that do not relate to the content, campaign, or organisation, or

are spam.
As an overview of all the campaigns, Average Mohamed received the most comments.
Analysis shows that 66% of these were supportive6:
[Facebook User 1] Please keep up your work... it is very important
[Facebook User 2] Average mohamed is cool, check it
Extract 1: A sample of supportive comments received by Average Mohamed on their

Facebook visitors wall.7
5 On Twitter ‘mentions’ were categorised as comments for the purpose of this analysis.
6 Comments were also categorised as supportive if users ‘tagged’ friends but did not expressly make a supportive sentiment of about the content. Although we
cannot tell for sure whether a user was tagging a friend for negative reasons, we included these as supportive on the rationale that they didn’t include a negative
sentiment and that tagging increases awareness of the campaign.
7 These quotes have been extracted from social media users and as such typos have been intentionally kept in the text body

[YouTube User 1]: So much respect for you brother, taking time to address
these issue, believe me when I say this, your message will get far not only to
non Muslim but Muslim who have forgotten their religion.
[YouTube User 2]: You’re doing an awesome job of highlighting our religion’s
standpoint on very important topics. People need to be educated about the
real Islam so they are not fooled by the media or extremists. May Allah bless
your work and accept your efforts.
Extract 2: A sample of supportive comments received by Average Mohamed on YouTube.
The remaining comments were divided evenly between ‘negative’ and ‘unrelated’.
[Facebook User 3]: These people are crazy
[Facebook User 4]: not true, read your Quran well
[YouTube User 3]: The only “right” Islam has the right to be wiped off the face
of the earth because of their insanity and demon worship.
[Twitter User 1]: @AverageMohamed why do so many Muslims shout “Allah

Snack Bar” right before/after they kill an infidel? #InfidelQuestions
Extract 3: A sample of negative and unrelated comments received by Average Mohamed on their
Facebook and YouTube promoted videos and promoted Tweets.
ExitUSA also received high comment traffic, with 32% of their comments being supportive
of the content, campaign, or organisation:
[Facebook User 5]: This is a great video, thank you guys for making it
[Facebook User 6]: I respect what you guys are doing. It’s infinitely harder to
realise that the will consume you from the inside out and to make a change
towards love and peace. Well done, I commend you.
Extract 4: A sample of the supportive comments received by ExitUSA on Facebook promoted posts.

[Facebook User 7]: So as a devoted WN who has been involved in the
digital wing of the ideology sicne [sic] 2012-ish, what’s your sales pitch for
me leaving? What do I personally stand to gain by leaving a movement that
I’ve been a mover and shaker in, a movement where I’m making a positive
difference for my people? What’s your case in favour of me giving up that?
ExitUSA: Well, if you are like I was after 2-3 years in the movement you are
probably still feeling the high, the rush and the excitement at the significance,
acceptance and brotherhood you are getting and there is very little chance
of persuading you to leave. I was in the movement for 15 years and I know
that feeling wears off. When you find how empty, destructive and lonely it is
behind the illusion, we will still be here for you.
Extract 5: An example of a current white national extremist posting on ExitUSA’s visitor wall.
A persistent issue with the ExitUSA was people misunderstanding the objective of
the organisation, and numerous people failing to finish the videos before choosing to
comment. In total, 14% of ExitUSA’s comments could be categorised as ‘misunderstood’,
which often then led to greater engagement from people who felt the need to correct
other users’ confusion:
[Facebook User 8]: Am I the only person who understood this video?
Extract 6: A typical Facebook user response when another user has misunderstood the
content of the video.
Negative comments made up for 18%. An example of a negative video comment is “Once
a racist always a racist.” Comments like this are unsurprising given the hesitation towards
formers, and the disbelief that they can de-radicalise. Here, negative comments also
include those which bring up wider societal issues regarding ethnicity. Interestingly, this
includes individuals that express, to a greater or lesser extent, sympathies towards to
‘white power’. Examples of this include:
[Twitter User 2]: @ExitUSATeam you’re #Insane its not about hate its about
survival #WhitePride #WaronWhites #WhiteGenocide by #Multicultralism
[Twitter User 3]: I am a recent reluctant convert to white nationalism from

basic bitch conservative. I would LOVE to believe I am wrong.
[Twitter User 4]: @ExitUSATeam Will you please tell our psychotic leaders to
stop flooding white counties with millions of non-whites that’s pretty hateful.

[Twitter User 5]: @ExitUSATeam just wanted to let you know I think this page
is bullshit. It’s not hate to be anti-Islam, it’s common fucking sense!
[Facebook User 9]: 14/88! SIEG HEIL!!!
[Facebook User 10]: I’m proud to be white damnit! So I’m a racist, yeah blacks
got BET, no we don’t… if I’m a racist for being proud of my race and not
wanting non American workers to steal out jobs then so fucking be it, I’ll wear
that proudly too.
[Facebook User 11]: How can you fully trust someone who completely turned
their back to something they allegedly were committed too death… their
tattoos meant nothing, what they wore on the outside portrayed nothing
they claimed to hold on the inside.
Extract 7: A sample of four tweets and three Facebook comments that present the kind of negative
language and tone used by some Twitter users against ExitUSA.
While it’s clearly difficult to describe these comments as ‘positive’, they do demonstrate
that ExitUSA’s campaign was effective at reaching its target audience.
Three individuals specifically expressed distaste for ExitUSA as an organisation on Twitter.

When looking further into the individuals’ backgrounds, they appeared part of the target
audience based on open-source information.
Harakat-ut-Taleem garnered the most comments on Twitter.8 Both supportive and

negative comments were equally divided, with one comment asking “what is the msg
in the story?”. This comment was categorised as “misunderstood.” Examples of positive
comments are:
[Twitter User 6]: A great initative (short vids) against religious terrorism by
this twitter account. Please follow them guys.
[Twitter User 7]: I can understand what you are trying to portray my lesson for
every Muslim – gain knowledge over religion you will earn janah
Extract 8: Two examples of positive commentary on Twitter
Negative comments included asking about the organisation’s mission statement, with
the authenticity of the organisations messaging being brought into question by one
user who asked: “y r u destroying our beloved Pakistan???? U r Indian agents.” These
8 This can be down to a number of variables: that their target audience is most active on Twitter; targeting on twitter was more efficient to reaching them; Twitter’s
interface might cater for ‘newbies’ better by deemphasising the lack of pre-existing social presence. This remains inconclusive.

suspicions likely arise as a result of the anonymity of the campaign and campaigners. It
may be likely this anonymity also affected engagement. Further research into the role of
authorship and engagement should be carried out.
Overall, these comments show that Facebook performed best at encouraging people
to comment. Twitter proved the best platform for comments for Harakat-ut-Taleem. It is
not conclusive why. The campaigns all received a majority of “supportive” comments.
There were still a substantial number of negative comments, but the content within these
comments help not only to understand target audiences’ attitudes towards the campaigns
and content, but also to understand whether the target audience is being reached. This
is shown best with ExitUSA’s commenters that displayed sympathetic attitudes towards
white power.
The next section on impact provides further qualitative analysis of the comments and
discussions that were sparked by the three campaigns. The content of the discussions
that took place across the campaigns provide some insights as to the general reaction
of the target audience of the counter-narrative content and overall campaigns. This also
helps us to better understand the kinds of individuals that commented on the content,
and to understand whether or not the target audience were a part of these discussions.
Impact: Sustained engagement

Assessing the impact that online counter-narratives have on attitudes and behaviour is,
as mentioned earlier, extremely difficult. The anonymity that the Internet provides makes it
impossible to know exactly who users are and how they are reacting to content. Moreover,
further research is needed into the process of disengagement and deradicalisation in
general, which could take place over a long period of time. It could be that an individual
with extreme views reacts with anger and aggression after watching a powerful counter-
narrative video but then, over time, begins to question their extreme beliefs.
It is extremely unlikely that a user who clearly has violent views will proclaim a change of
heart after watching a counter-narrative video. Nonetheless, sustained engagements –
or individuals reaching out to counter-narrative campaigners for assistance – provide the
clearest indication of impact for online counter-narrative campaigns.
Counter-narratives are designed to provoke a reaction in the intended audience:

comments and sustained engagements can provide us with valuable information on the
topics users are most likely to engage with constructively and antagonistically.
Sustained engagements can be understood as:
• An individual user making more than one ‘comment’ on a Facebook post or YouTube
video.
• An individual user replying more than once to a tweet, or starting a conversation on
Twitter with other users or the organisation.
• An individual user sending a direct message to the organisation on Facebook or

Twitter.
Sustained engagements can be divided into two distinct categories:
• Sustained Constructive Engagement: a conversation about the content or campaign

where individual users comment more than once in a positive manner. This can
encourage critical thinking.
• Sustained Antagonistic Engagement: a user who negatively disputes or dismisses
the content, or campaign, to other users or the organisation. This can potentially sow
the seeds of doubt.
ExitUSA’s counter-narrative ads outperformed the other two campaigns with regards to
fostering both constructive and antagonistic discussions. This is attributed to ExitUSA’s
capacity and willingness to engage with comments, and the ‘authentic voices’ of its
members. On average, ExitUSA replied to comments within less than one day, a response
rate that was evident across all the platforms.9 Many of the discussions that took place in
the comment sections of their posts involved members of the organisation.
In comparison, fewer conversations took place on Average Mohamed and Harakat-

ut-Taleem posts, and there was minimal constructive engagement with either of the
campaigns’ YouTube pages. This may be because of the less provocative nature of
their videos, or perhaps just the manner in which their target audiences act and present
themselves online. Arguably, as both Average Mohamed and Harakat-ut-Taleem were
more educational, and their aims geared more towards prevention, they could not spark
conversation as effectively as the more provocative content presented by ExitUSA and
targeted at an audience that already held extremist beliefs.
Average Mohamed did manage to generate large quantities of engagement on specific

posts, which was mainly categorised as sustained antagonistic engagement, with users
outside of the target audience arguing about theological, cultural, social and political
issues. The position taken with Harakat-ut-Taleem was to provide minimal responses
to comments and messages which, predictably, undermined its potential to achieve
sustained engagement. However, without a natural Urdu speaker at ISD, and with
no support from the content providers, this was considered as the most appropriate
approach.
Sustained constructive engagement
Both ExitUSA and Average Mohamed offer compelling evidence of sustained engagement,
with the comment section beneath their videos often acting as an interface from which
users could discuss politically, socially and culturally divisive topics. Comments on racism
and racial equality were particularly good at stimulating sustained engagement, especially
when users were confronted by challenges to their perceptions and ideologies. These
kinds of conversations presented different world views that offer the opportunity for
9 A response is either a direct message to their profiles, or when a user questions the organisation in a comment or Tweet.

critical thinking, which could potentially build resilience to extremist narratives.
Both ExitUSA and Average Mohamed’s campaigns spurred these kinds of discussions.
Below is an example from ExitUSA.
[EXIT User 1]: No, people of color can’t be racist to white folks. Prejudiced and
discriminatory? Absolutely, but that isn’t racism.
[EXIT User 2]: [User 1] look up the actual definition of racism. Then you can
talk about who can and can’t be racist.
[EXIT User 1]: Race isn’t real. It’s a social construct.
Extract 9: A conversation between two users in a Facebook comment thread beneath the
ExitUSA video titled ‘Formers’.
For ExitUSA, the clearest examples of dialogue involving far-right extremists or

sympathisers came from Facebook. This may be due to Facebook’s comment interface,
which makes viewing and responding to other users comments a relatively simple
process without the interaction being posted on a commenters’ own personal feed. It
is interesting to note that the platform which requires the least amount of anonymity
from its users, Facebook, would appear to encourage the greatest quantity of impactful
engagements.10
This is given further proof when we examine the direct messages received by ExitUSA.
There were eight cases of people in the process of personal de-radicalisation reaching
out to ExitUSA, via the messaging function, and asking for information and support.
[EXIT Former 1]: Hey guys I am a former white supremacist
[ExitUSA]: Hi [former]. Thank you for reaching out. How can we help you?
[EXIT Former 1]: I have been away from it for a few months but I still get old
feelings an thoughts
Extract 10: The opening lines of dialogue between a former who directly Facebook messaged
ExitUSA asking for support11
10 A possible reason for this is that the public nature of Tweets, and the abundance of trolls and advertising comment bots on YouTube, may work to deter legitimate
conversation when held up against Facebook. Further research will help us understand this with greater certainty.
11 These are the opening lines of dialogue from a conversation ExitUSA had with a Facebook user who was going through the first steps of personal de-
radicalisation. The ease with which the two parties could communicate with each other allowed the conversation to move quickly over various aspects of the de-
radicalisation process, and address the support that the user was looking for.

[EXIT Former 2]: What is this about? […] I was just wondering how you help
whould [sic] love to speak to someone about this. I’d like help to get away
from hate
[ExitUSA]: Hi [Anonymous User 1]. I’m glad you reached out. My name is
Christian, I was one of the founding Northern Hammerskins in 1987. I left the
movement in 1995. The first major hurdle is reaching out.
Extract 11: the opening lines of dialogue between a former who directly Facebook messaged
ExitUSA asking for support
These messages are perhaps the most direct evidence possible of counter-narrative
campaigns having impact. The people who directly messaged ExitUSA asking for support
appeared to be encouraged to maintain a dialogue with the ExitUSA team because of
the authenticity of its members as formers. This may have also worked to breakdown
some of the hesitancy the users may have felt when engaging with an organisation via a
platform that does not protect a user with much anonymity.12
The capacity of the organisation, and the experience of its members, meant that these
messages were given the necessary attention to sustain a constructive engagement.
One individual (former) wrote that they were grateful to have seen the video:
[EXIT Former 2]: What is this about? […] I was just wondering how you help
[ExitUSA]: Ok, I promise we will work through this.
[EXIT Former 2]: It means alot thank you
[ExitUSA]: I’m out of town right now, but might be able to get some time next
week after xmas and before new years. No, thank you! This is a huge first
step.
Extract 12: The concluding lines of dialogue of a message between a former and ExitUSA on Facebook
ExitUSA aims to help far-right extremists to exit from their respective movements and
actively offers help to those who want to leave, which could be the reason they received
direct messages. The targeting criteria for their campaigns were informed by this. As
such, it is more likely that individuals would reach out in a bid to get help to do this.
Average Mohamed’s counter-narrative content aims to be purely educational and

preventative; by equipping youth with knowledge to be resilient to extremist narratives.
For them, the targeting criteria was broader and thus the success of the campaign should
12 Further research can help support this campaigns findings that an organisation willing to engage with users produces a more impactful counter-narrative.

not necessarily be judged in terms of encouraging direct messages from individuals
‘reaching out’ for help.
Responding to messages can be an issue of capacity. ExitUSA has a dedicated team of

professional intervention providers that deal with sensitive individuals and information.
Responding to users, especially potentially volatile or sensitive ones, requires training. An
organisation disseminating counter-narratives should be prepared – in terms of training,
personnel and protocols – to deal with the possibility of individuals reaching out to them
directly for help as a result of seeing a campaign.
A number of discussions took place in the comments sections across campaigns that
give a good indication that the audience were engaging in sustained discussions on the
subjects that Average Mohamed aim to unpack, including democracy and Islam, Islam
and the West, women’s role in Islam, slavery in Islam, and identity issues. One example of
an exchange on gender equality issues is provided below:
[AM User 1]: Qur’an (4:3) – (Wife-to-husband ratio) "Marry women of your
choice, Two or three or four."
[AM User 1]: Qu’ran (2:282) – (Court Testimony) "And call to witness, from
among your men, two witnesses. And if two men be not found then a man
and two women." Muslim apologists offer creative explanations to explain
why All felt that a man’s testimony in court was [worth more than a womans]
[AM User 1]: Qur’an (4:11) – (Inheritance) “The male shall have the equal
of the portion of two females” (see also verse 4:176). In Islam, sexism is
mathematically established.
Extract 13: A series of comments posted in the comments section underneath the ‘Be Like Aisha’ video.
Using passages from the Qur’an was a common theme of users looking to argue that women and men
are not equal in Islam.
In one instance a conversation between two users (totalling 25 comments between

them) that covered gender equality, religious belief, politics, scripture, and war ended
with a positive outcome on the merits of debate:
[AM User 3]: So if women are equal to men in the Islamic faith, then why is
she still wearing a hijab?
[AM User 4]: Sorry the bible also tells women to wear hijab, they just don’t.
I’m sure you wouldn’t call the bible “oppression” right?
[AM User 3]: Well you all can have your beliefs and I can have mine but at the
end of the day I don’t see Christians beheading people or stoning people or
pushing gays off a roof top. I don’t see any Islamic countries that are not torn

by war and strife.
[AM User 4]: Actually [AM User 3] maybe not Christians in America but the
Christians in Africa kill anyone who is not a Christian and refuses to convert :)
[
AM User 3]: I love how you all assume I am Christian lol
[AM User 4]: […] you believe men treat women bad in Islam to my belief it’s
you American men who treat women badly and are disrespectful to women,
you guys make music videos with naked women and have women expose
skin […]
[AM User 4]: Anyway I have to go now and I’ll try to come by as much as I can
:)
[AM User 3]: Debating like this is always good for the mind
[EXIT Former 2]: Yea when ever is convenient for you thanks for posting that
video I’m glad I watched it
[ExitUSA]: Hey [former], that’s pretty normal. While most of us have been out
for years, it takes hard work and a support system to pull yourself out.
Extract 14: A portion of the 25 comment conversation between two users.
As evidence of a campaign having impact, these are encouraging signs. Average

Mohamed and Harakat-ut-Taleem campaigns led to fewer examples of sustained
constructive engagement compared to ExitUSA, but that is not to say they achieved no
impactful results. Average Mohamed’s Be Like Aisha video, received more comments,
likes and shares than any other Facebook post of all the campaigns. Additionally, whilst
the other videos received substantially less comments than Be Like Aisha, there were
still examples of sustained dialogue between users within these:
[AM User 1]: They were not Christians. They were extremists. And are you
bringing slavery into this?
[AM User 2]: You asked why am I here and then you answered.. No one
brought me here .. im here and paying for school.. its all business if I don’t
have the money I wouldn’t be welcomed here.. you said all those bad acts
came from christains but its not ur [sic] business…
[AM User 1]: For being a self policing force of the single most populated
religion on the planet, as a whole, there hasn’t been much condemnation or
correction. Just more terror and tyranny.

[AM User 2]: Ok … can you tell me the cause of those terrors? Or is it just
because for being a muslim you have to be violent and do harm or others...?
[AM User 3]: At the end of the day we are all the same, human beings. I’m not
going to continue with this foolishness because it really saddens me that we
all can’t just get along.
[AM User 1]: The cause? Following the laws in your Quran.
[AM User 4]: Lol so they are “Christian extremsits” yet ISIS are your average
muslim? Bahahahaha … another thing you brought 9/11 and I brought you
slavery which Christians in the south loved and KKK loved who are also
Christians :)
Extract 15: A conversation between users in the comment section of the ‘Muslim in the West’
Facebook advertisement.
It’s clearly difficult to interpret these conversations and whether they are a good or bad
thing, without being able to conduct in-depth interviews with the individuals taking part
in the discussion.
Sustained antagonistic engagement

Engagements were labelled as antagonistic when an individual user disputed or dismissed
the content or organisation in an unconstructive way, or disputed other users’ comments.
Examples of this can be seen from Twitter users who responded to the ExitUSA promoted
tweets with accusations of the organisation having a bias towards non-whites.
As to be expected, scepticism towards the motivations of the organisations was a theme

that was apparent across all three campaigns, and produced numerous examples of
antagonistic commentary or responses from users. The importance of presenting the
messengers of a counter-narrative as authentic appears to be a key aspect in the
ability of the campaign to produce impactful results, and underlines the importance
of organisations using credible messengers. It also suggests that making public the
role of AVE within or during the campaigns could have undermined this credibility by
exacerbating such scepticism.13
There were, overall, five conversations (counted as repeated comments by more than
one individual) in response to ExitUSA’s campaign that included posts with racial hatred
in the form of texts or images. Eleven individuals were, at a minimum, sympathetic to
white supremacist ideology. Again, this gives a good indication that the campaigns were
reaching their target audience.
The below discussion is a continuation of the discussion that was shown in ‘Comments’
13 This highlights the need for organisations to understand how the branding of their organisation can influence the message of their counter-narrative.

section, and an example that members of far-right groups were also viewing and engaging
with the content:
[Exit User 5]: How can you fully trust someone who completely turned their
back to something they allegedly were committed too until death… their
tattoos meant nothing, what they wore on the outside portrayed nothing
they claimed to hold on the Inside […]
[ExitUSA]: We turned our backed on fear, misinformation, insecurity, and

being disconnected from the world around us. When you begin to question
the lies and drama, when you feel the weight of the disconnect between
heart and mind, we’ll be here for you with no judgment.
[Exit User 5]: when you begin to question the lies and drama” You should
ALWAYS question everything which is brought before you and not take it for
word of mouth, or face value.
“when you feel the weight of the disconnect between heart and mind”
Obviously you people didn’t work either..
You walked away from fear? More like embraced it and walked with it. You
didn’t have the heart, determination, will, love, honor, and endurance to
follow through the hardships.
You blame your short comings on the movement just as you did when you
got into the “SCENE” and blamed your short comings on society. You guys
are cowards and deserters, you had no concept of Loyalty, Honor, Pride, Will,
Courage, Truth and Love for you people… you were in it.
Extract 16: Discussion between user and ExitUSA
Other examples include: one ExitUSA commenter wrote: “There is nothing wrong with
having pride in your race WHITE PRIDE WORLD WIDE”. In a similar vein, another user
exclaimed “Now it is hate to defend your face, your culture, and your country with it is
jeopardised by vicious, unruly, unjustified groups that are led by un-American and un-
Godly beliefs and agendas?????” whilst another explained “sorry, proud I’m white.”
Again, it’s difficult to know how to interpret these discussions based on the limited
information available about the users, and the complexity of the process of disengagement
in general. Nonetheless, in our framework of impact – this type of discussion should
be seen as a positive thing in the context of assessing the effect of counter-narratives.
The simple fact that this individual engaged in a conversation with ExitUSA suggests a
potential ‘cognitive opening’ for having impact.
The topic that led to the most sustained discussion concerned the Black Panthers and
their comparison to white hate groups, such as the KKK. Eight individuals contributed
to this discussion, with half stating that the Black Panthers were not a hate group, and

the other half stating that both the KKK and the Black Panthers are one in the same.14
Again, this highlights that sustained antagnoistic engagement can encourage exposure
to differing points of view that may lead to critical thinking.
As noted above, ExitUSA were very active in responding to negative comments and they
noticed that they also got a lot more comments when they engaged. Examples of their
responses to antagonistic discussions included:
• “There is nothing wrong with being proud of who you are or where you come from.
The problem arises when that pride is used to dehumanize others.”
• “Every community has drivers of racial tension. We are talking about people in
our communities today that act out the violence we once promoted. Having been
"drivers" ourselves we want to assist people who followed us down the same dead
end to escape that place of intolerance.”
• “There is nothing wrong being proud of your heritage or ethnicity but when it happens
to the exclusion of all others and brings with it an implied violence it has gone beyond
simple pride and is about anger and hatred. We all found that carrying around that
anger and hatred was draining and exhausting while not accomplishing anything. I
challenge you to look in your own heart and ask yourself what is it really about, is it
about pride or anger? If it is anger, we are here to help you.”
Summary of impact: towards a future framework
With the campaigns complete, we are now able to offer the beginnings of an evaluation
framework for measuring the impact of counter-narratives that goes beyond impressions,
reach and engagement metrics. Comments – and in particular, sustained engagements
or conversations – provide a better indication for researchers about how individual users
and target audiences overall may be reacting to the messages in counter-narrative videos.
To be sure, the majority of these comments and discussions will be hostile, aggressive or
difficult to interpret. Further testing and evaluation with a wide range of counter-narrative
content aimed at different target audiences is needed to further our understanding of
impact.
In general, it would seem that fostering conversations online, even if they are antagonistic,
is a positive thing. Exposure to alternative viewpoints can potentially foster critical thinking
or plant a ‘seed of doubt’ that later matures into a change in attitudes and behaviours.
Understanding this better requires further testing and research: trialling more and varied
counter-narrative campaigns, but also other research – for example, with focus groups
(for those aiming at preventative target audiences) with ‘formers’ who have gone through
the process themselves, as well as interviews with intervention providers who work with
young people who are disengaging.
14 One individual posted six links to YouTube videos that expressed the violent attitudes of the Black Panthers. The discussion changed to one of positivity, and one
user wrote: “It's honestly nice to see that most of you don't agree with the "New ideology" of The Black Panthers as I think the vast majority of Americans feel
that ALL of us are just Americans, regardless of color”. This user’s own comment is a fair analysis of the discussions.

RECOMMENDATIONS
Recommendations
Overall, this project demonstrates that a coordinated effort between campaigners, social
media companies, and private sector partners can produce impactful counter-narratives.
This methodology - an organised cycle of curation, production, data acquisition and
analysis – can be replicated and applied globally to help scale up counter-narrative
efforts that push back against extremist recruitment.
One of the key aims of this project is to assist small and medium size NGOs in producing,
disseminating and evaluating counter-narratives and counter speech. These pilot
campaigns raised a number of insights about how to make counter-narrative production
and dissemination more efficient and effective. Based on our findings, we make a series
of recommendations for NGOs and counter-narrative campaigners – many of whom will
only be vaguely familiar with social media marketing. We also make recommendations
to social media platforms about how they can best support this process. Social media
marketing tools can be incredibly effective at identifying and delivering content to
key target audiences. However, they can also be complex, confusing, opaque and
unpredictable.
TABLE 5: SUMMARY OF STRENGTHS AND WEAKNESSES OF

PLATFORMS
PLATFORM STRENGTHS WEAKNESSES
• Global reach covers numerous • Facebook interface regularly changes,

territories and demographics making it difficult to maintain up-to-
date best practice guides.
• Language specific versions offer
greater scope for CVE campaigns • In the ‘Manage Ads’ sections, the
column totals are often wrong – this
• Intuitive and easy to use interface makes it problematic when trusting
makes constructing advertisements a the statistical data.
quick process
• Large discrepancies in the successes
• Allows for dialogue between the of some videos compared to others,
users, and between users and the despite similar or identical targeting
organisation and content. It is not always clear why
FACEBOOK • In-depth targeting functionality based
some adverts achieve such higher
results.
on self-reported user demographics
including age, location, education, • Some thumbnails – the picture
employment, as well as interests, that accompanies an advert – were
groups and pages liked, allows for very restricted, while others with similar
specific targeting of advertisements content were not. For example, some
and videos. thumbnails showing men holding
guns were allowed, and some were
• Private messaging, visitor wall posts not.
and user comments provide three
different routes for users to contact the • Some statistical insights are not
organisation. clearly defined i.e. ‘People Talking’.

PLATFORM STRENGTHS WEAKNESSES
• Largest video hosting site, with a • AdWords interface not easy to

global reach. negotiate, and requires more research
and study into how to use it compared
• Easy to use video upload function to the advertising platforms built-in to
makes adding new content to a page Facebook and Twitter. This is an issue
a simple process. for CVE NGO’s without prior experience
in advertising on YouTube.
• Ability to input a detailed and
exhaustive list of keywords to use for • AdWords regularly changes and
targeting adverts additions can undermine the value of
previous best practice guides.
• Allows for dialogue between the
users, and between users and the • Large discrepancies in the successes
YOUTUBE organisation of some videos compared to others,
despite similar or identical targeting
• AdWords function ‘My Client Center’ and content. It is not always clear why
allows for multiple YouTube accounts some adverts achieve such higher
to be managed by one account. This results.
is useful when conducting multiple
targeted campaigns. • Quantity of videos being uploaded, and
advertisements being run, mean that
• Ease with which advert targeting can the achieving impressions and views
be changed during the advertising can be expensive.
period, makes it easy to alter and re-
focus targeting.
• Large amounts of statistical insights

are offered by AdWords, allowing for
a multitude of different analysis.
• Highly-active audience and with • In-kind support does not allow

global reach. campaigns to be set-up over a set
amount of time, which is frustrating if
• Widely used by extremist groups with orchestrating a wider online campaign
far-right or Islamist sympathies, which involving multiple platforms.
are easily identifiable once hashtags
and Twitter handles are researched. • Much less self-reported data or other
targeting criteria e.g. groups liked, or
• Speed with which content moves interests.
around Twitter and people interact
TWITTER with it means a campaign can quickly • Does not generate many replies
learn if their content is engaging with or comments on counter-narrative
the right people. content.
• Allows you to target by similar users, • Promoted tweets do not have much
meaning once you do your research longevity, in terms of engagement or
you can reach people similar to your reach, past the initial promoted push.
target audience.
• Unlike Facebook, you cannot target
users based on keywords AND
interests, only keywords OR interests.
Recommendations for counter-narrative campaigners

Working with counter-narrative campaigners, each with their own interests and
idiosyncrasies was in many ways the most challenging part of this project. However,
maximising the impact that campaigners can have – including producing and disseminating
content at scale – requires closer working, trust building, and collaboration between
campaigners, social media platforms and organisations like ISD. NGOs and campaigners
should have an established social media presence, dedicated personnel, and a
commitment to engage and be responsive to the advice of social media companies.

The key strategic principles are as follows:
• Conduct in-depth research on target audience interests and behaviours before

designing and disseminating content.
• Test a range of topics and message tones with similar target audiences in order to
determine which resonate most strongly.
• Train at least one member of staff to become an expert at using social media and
respective marketing tools.
• Supplement counter-narrative campaigns with user engagement strategies.
• Use alternative media channels in geographic areas where access is limited.
• Online counter-narrative campaigns should operate alongside offline campaigns.
• Be aware of the presence of online ‘bots.’
• Spread online campaigning budgets over a longer period of time.
• Identify testable hypotheses, demonstrate results, and identify new courses of action.
• Use identifiable subjects where possible.
Additionally, we also make a series of recommendations for campaigners that are specific
to each platform. Overall, as our findings elucidate, each social media platform provides
different benefits and challenges for disseminating advertised content. Moreover, content
that does well on one platform will not necessarily perform best on other platforms.
Ultimately, campaigners and disseminators need to adopt a comprehensive approach
that utilises all of the major social media platforms:
Facebook produced the greatest reach, video views and engagement for each of the
organisations. In terms of campaigners using Facebook to disseminate content, we
recommend:
• Utilise Facebook for preventative campaigns that seek to build resilience among
broader demographic groups.
• Upload videos directly to Facebook instead of linking from other sources.
YouTube provided the lowest cost-to-views ratio and the highest rates of viewer
retention. For campaigners using YouTube to disseminate content, we recommend:
• Utilise YouTube for campaigns that aim to dissuade people further downstream in the
radicalisation spectrum.
• Undertake or utilise rigorous research into your target audience’s behaviour online
before designing and disseminating content and campaigns. Use YouTube for longer
video content.
Twitter provided the second largest number of video views across platforms as well
as the highest impressions to engagement ratio:
• Utilise Twitter to generate engagements with users and to build the online profile of
a new brand or counter-narrative campaigner.
• Utilise Twitter to target those further downstream on the radicalisation spectrum by
utilising the user-handle based targeting function.

Recommendations for social media platforms
In addition to recommendations for counter-narrative campaigners, this pilot project also
suggests a number of recommendations for social media platforms to support this process.
As noted throughout this report, social media marketing tools can be highly effective at
identifying key target audiences for organisations working to tackle radicalisation. This
is true whether the aims of the organisation are more preventative in nature (reaching a
broader audience) or whether they are aiming more at persuading individuals who may
already be interested in extremist ideas.
However, social media marketing is very much a new science. As such, it contains a
huge amount of numbers and metrics that can be difficult to interpret – particularly for
small NGOs without this expertise. The two biggest challenges for NGOs creating and
disseminating content using marketing tools are the frequency with which advertising
tools and platforms change, and apparent inconsistency and inaccuracy relating to the
figures reported.
We therefore recommend that social media platforms:
• Provide detailed training – and updated ‘how to’ guides and toolkits – for NGOs to
coincide with changes made to advertising interfaces.
• Support NGOs to undertake ‘split testing’ and ‘user experience’ research to increase
understanding of effective image and message usage as well as user journeys.
• Provide accurate and more in-depth analytics that can take the place of market
research.
Finally, these campaigns point to a number of gaps in our knowledge about how to
measure the impact of counter-narratives online. To help improve our understanding,
we make the following recommendations:
• Initiate further research into the process of disengagement and deradicalisation in

order to gain a better understanding of how to analyse comments and sustained
engagements.
• Evaluate counter-narratives online with a greater use of offline focus groups and user
experience testing with target or proxy audiences.
Conclusion
By implementing a methodology that incorporates partnerships, curation, content
creation, deployment, and evaluation, this study demonstrates that the use of counter-
narrative messaging with measurable impact is replicable and scalable, though not
without difficulty. Working with multiple partners and campaigns, we now have a much
better idea of the interplay between key factors such as geography, language, ideology,
audience, and media platforms.
While resourcing for content creation will continue to be an issue, future campaigns can

benefit from this study and, in applying the same methodology, produce data that will
further optimise and extend it. This virtuous cycle of learning, which takes into account the
most current trends and promises to improve the impact on at-risk individuals, holds the
key to a comprehensive, global strategy that can compete with the targeted, professional
production that extremists will continue to use in the future.
In this regard, ISD has produced two reports that complement the methodology described
here. The first one, entitled The Counter-Narrative Handbook15, was written to help
anyone looking to proactively respond to extremist propaganda with counter-narrative
campaigns and is intended as a beginner’s guide for those with little or no previous
experience of counter-narrative campaigning. It takes readers through the main stages
of creating, launching and evaluating an effective counter-narrative campaign. It can also
be used alongside ISD's freely available online Counter-narrative Toolkit, which can be
found at www.counternarratives.org.
The second report, entitled Youth Innovation Labs: A Model for Preventing and
Countering Violent Extremism, is based on ISD’s experiences in creating, running and
evaluating in-country innovation labs through its Youth Civil Activism Network (YouthCAN).
These labs bring together independent content-creators, from civil society and NGO
campaigners to young activists, and connects them with private sector partners to amplify
their counter-narrative messages through training, networking and campaign support. As
of mid-2016, these labs have been held in Amsterdam, Budapest, Madrid, Tirana, Nairobi,
Kigali, and Berlin, training over 400 individuals who have created counter-narrative
campaigns reaching targeted audiences of over 500,000.
In a world where extremist use of communication technologies continues to exploit

vulnerable people, we can significantly increase the scale of organic, accessible, and
effective counter-narrative content with these replicable and scalable methodologies.
15 http://www.strategicdialogue.org/wp-content/uploads/2016/06/Counter-narrative-Handbook_1.pdf

Glossary
Awareness: a metric that indicates reach and levels of audience engagement,
exposure of content to target audiences and help determine whether your online
(including impressions, reach, video views objectives were met.
and viewer retention).
Metrics: different social media or website
Clicks: the number of times people have analytics services will offer different types
clicked on your ad or a link in your posts. of data. There are a vast range of different
metrics that can help you understand who
Counter-narrative: a message that you reach, how well you engage your
offers a positive alternative to extremist audience, and the impact your campaign is
propaganda, or alternatively aims to having.
deconstruct or delegitimise extremist
narratives. Organic growth/reach: audience reach or
engagement on websites or social media
Downstream: an audience that includes platforms generated from searches and/or
those already engaging with violent as a result of unpaid campaign strategies
extremist material, or actively participating and tactics.
in online extremist communities or
networks. Promoted content: posts or tweets
that have been promoted through paid
Engagements: interactions between advertising to appear in selected audiences’
audience members or with campaigners news feeds on social media platforms.
themselves, which could be positive or
negative and help provide an insight into Reach: the total number of people that
reactions to a campaign. received an impression of your post or ad
on their screens or newsfeeds.
Impact: a measurable change in behaviour,
ideally constructive, that can be attributed
to exposure to or engagement with counter-
narrative content.
Impressions: the number of times content

or adverts appear on a user’s screen online.
Avoid over-relying on impressions as a
metric of impact, people don’t necessarily
take notice of every ad that appears on
their screen.
In-built analytics: analytics services

available on social media platforms that
allow you to monitor your campaign’s
52 | Institute for Strategic Dialogue

The Institute for Strategic Dialogue (ISD) is a London-based ‘think and
do tank’ that has pioneered policy and operational responses to the rising
challenges of violent extremism and inter-communal conflict.
Combining research and analysis with government advisory work and

delivery programmes, ISD has been at the forefront of forging real-world,
evidence-based responses to the challenges of integration, extremism and
terrorism.
The Institute for Strategic Dialogue
P.O. Box 7814, London, United Kingdom

[email protected] • www.strategicdialogue.org
Any copying, reproduction or exploitation of the whole or any part of this document without prior written approval from the Institute for Strategic Dialogue is prohibited. Institute
for Strategic Dialogue is the operating name of the Trialogue Educational Trust (registered charity number 1076660). The Institute for Strategic Dialogue is a company limited by
guarantee, registered in England and Wales, number 06581421, and a registered charity, number 1141069.

Impact of Counter Narratives - ONLINE - 1 PDF

Uploaded by

Copyright:

Available Formats

Impact of Counter Narratives - ONLINE - 1 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Impact of Counter Narratives - ONLINE - 1 PDF

Uploaded by

Copyright:

Available Formats

THE IMPACT OF

About the authors

CAMPAIGN FINDINGS AND ANALYSIS 22

• Reach, gender and age 25

• Likes, shares and comments 27

• Strengths and weaknesses of platforms 48

The concept of creating counter-narratives1 in order to push back against extremist

The key objectives of this project were to:

• Assist a wide geographic, ideological, and cultural variety of small non-profit

• Provide guidance to build the capacity of similar non-profit organisations to produce

The three campaigns included:

• Average Mohamed. A non-profit organisation that uses animation to encourage

The Impact of Counter-Narratives |5

• Harakat-ut-Taleem. A front organisation created by AVE for a third party Pakistani

Our qualitative analysis of comments, and in particular ‘sustained engagements’, provide

6| Institute for Strategic Dialogue

• Allocate modest amounts of funding in order to dramatically increase the reach

• Conduct or utilise in-depth research on target audience interests and behaviours in

• Supplement counter-narrative campaigns with user engagement strategies. This may

• Use alternative media channels in geographic areas where access is limited or

• Coordinate online counter-narrative campaigns with offline campaigns or events.

The Impact of Counter-Narratives |7

In addition to recommendations for counter-narrative campaigners, this pilot project also

Next Steps: Recommendations

• Explore the possibility of identifying and redirecting pre-existing counter-narrative

8| Institute for Strategic Dialogue

• Use of offline market research techniques to better understand user journeys,

The Impact of Counter-Narratives |9

The concept of creating counter-narratives3 in order to push back against extremist

• Curate campaign content with these organisations whilst developing their

The Impact of Counter-Narratives |11

• Impact metrics: whether campaigns’ content appeared to encourage critical

Selecting partner organisations

In order to identify organisations to produce counter-narrative content

The organisations chosen by AVE were as follows:

ExitUSA is an “exit”/outreach programme run by Life After

Average Mohamed is a non-profit organisation that uses

12| Institute for Strategic Dialogue

The Impact of Counter-Narratives |13

The Impact of Counter-Narratives |15

Organisation Goals and Campaign Objectives

Video Content and Campaign Targeting

AGE, GENDER AND INTERESTS AND TWITTER

16| Institute for Strategic Dialogue

NO JUDGEMENT JUST HELP

The video explains that since 9/11 home-grown

THERE IS LIFE AFTER HATE

Tim Zaal spent half of his life thinking that he had

This video tackles the same issues as No Judgement

The Formers in this video begin by provocatively

The Impact of Counter-Narratives |17

Organisation Goals and Campaign Objectives

Video Content and Campaign Targeting

AGE, GENDER AND INTERESTS AND TWITTER

18| Institute for Strategic Dialogue

Whilst many Islamist extremist groups say that

A MUSLIM IN THE WEST

This video tackles the key themes of an identity

Tackling the issue of identity a Muslim youth in the