Journalism Studies

ISSN: 1461-670X (Print) 1469-9699 (Online) Journal homepage: http://www.tandfonline.com/loi/rjos20

Framing a Conflict! How Media Report on

Earthquake Risks Caused by Gas Drilling

Alette Eva Opperhuizen, Kim Schouten & Erik Hans Klijn

To cite this article: Alette Eva Opperhuizen, Kim Schouten & Erik Hans Klijn (2018): Framing a
Conflict! How Media Report on Earthquake Risks Caused by Gas Drilling, Journalism Studies, DOI:
10.1080/1461670X.2017.1418672

To link to this article: https://doi.org/10.1080/1461670X.2017.1418672

© 2018 The Author(s). Published by Informa

UK Limited, trading as Taylor & Francis
Group

Published online: 11 Jan 2018.

Submit your article to this journal

Article views: 125

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=rjos20
 FRAMING A CONFLICT! HOW MEDIA
REPORT ON EARTHQUAKE RISKS CAUSED
BY GAS DRILLING
A Longitudinal Analysis Using Machine
Learning Techniques of Media Reporting on
Gas Drilling from 1990 to 2015

Alette Eva Opperhuizen, Kim Schouten, and Erik Hans Klijn

Using a new analytical tool, supervised machine learning (SML), a large number of newspaper
articles is analysed to answer the question how newspapers frame the news of public risks, in
this case of earthquakes caused by gas drilling in The Netherlands. SML enabled the study of
2265 news articles published over a period of 25 years. Our study shows that there is a dispropor-
tional relation between media reporting and actual risk; and that the use of dramatization bias in
framing the issue about gas drilling increased, but the use of personalization and negativity bias did
not become more dominant after a major media change in 2013. Sensational/tabloid newspapers
make more use of personalization bias, whereas quality newspapers make more use of value con-
flict and political disagreement in the framing about gas drilling.

KEYWORDS framing; machine learning technique; media bias; mediatization; risk; risk amplification

Introduction: Media Attention on Public Risks

Risk events and issues are popular object of news reports. News media seem to be
more interested in the dramatic aspects of the news than in presenting information
about the risk event itself or the background to it (Beattie and Milojevich 2017). This reflects
what several scholars argue about media: they make use of particular reporting frames to
serve and inform their audience and to make the information comprehensible for the
readers (Semetko and Valkenburg 2000; Entman 2007; Baumgartner and Jones 2009;
Bennett 2009; Patterson 2000). According to the mediatization literature, attracting a
large audience has become more and more important because of commercial pressure
on media (Cook 2005; Bennett 2009; Hjarvard 2013; Strömbäck and Esser 2014). The
claim in this literature (but also in other literature, like agenda forming and risk literature)
is that the institutional rules of media are more and more dominated by commercial rules
(reaching a wide audience) and that this has consequences for the frames and biases media
use in news providing (more sensational, dramatized, etc. see Bennett 2009). This literature
Journalism Studies, 2018
https://doi.org/10.1080/1461670X.2017.1418672
© 2018 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group
This is an Open Access article distributed under the terms of the Creative Commons Attribution-
NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/),
which permits non-commercial re-use, distribution, and reproduction in any medium, provided
the original work is properly cited, and is not altered, transformed, or built upon in any way.
2 ALETTE EVA OPPERHUIZEN ET AL.

also emphasize that the logic of the media are penetrating other spheres of society
(especially politics) which they call the “mediatization of society” (Hjarvard 2013; Strömbäck
and Esser 2014). If it is true that (some) media use particular bias in their frames to serve
their readers for commercial reasons, then that is an important observation, because
media are a major source of information for citizens to reach a judgement about public
risk events (Renn 1992). Although the literature often argues that this mediatization
trend is visible in almost every part of the media landscape, the so-called tabloids/sensa-
tional newspapers are supposed to be more susceptible to using bias in there reports
related to mediatization, than quality newspapers (Uribe and Gunter 2007). Public risk
issues are attractive for news media (Slovic 2000), and consequently media coverage of
these risks may be particularly prone to framing the news related to mediatization (Hjarvard
2013; Strömbäck and Esser 2014). Critics state, however, that the empirical support for the
claims in the mediatization literature about the penetration of institutional media rules and
the resulting influence media in other spheres of society is to date not very impressive (for
this criticism, see Vliegenthart, Boomgaarden, and Boumans 2011; Van Aelst et al. 2014).
There is certainly little research in this field about the reporting of public risks. Therefore,
in-depth analysis of risk events and their media coverage is needed, preferably covering
a long time period to look at the use of media frames and biases about emerging public
risks.
The gas drilling case in The Netherlands offers a very good opportunity to study
media attention over a long period and at the same time provides in-depth knowledge
about media attention on a case with public risks. The risk of earthquakes in The Nether-
lands is a consequence of human activities. Since 1960, the Dutch State has allowed gas
drilling in the Northern region, which generates high revenues but also increases public
risk. Media reporting on this risk was very limited for many years. In recent years, the
increasing frequency of earthquakes has led to a broad social and political debate about
the benefits and risks of human actions to drill for gas. In this social and political debate,
news media play a critical role. The way media frame the risk of earthquakes is therefore
important. This leads to our research questions:
RQ1: How do media pay attention over time to the risks of earthquakes as result of gas
drilling in The Netherlands?

RQ2: Which news biases dominate this attention, can differences during the time period
being observed, and does this differ for various newspapers?

In this study, we look at news items in 5 different newspapers over the last 25 years
about gas drilling in The Netherlands. To be able to analyse a large number of news items, a
relatively new research method was applied in this research, i.e. machine learning tech-
niques (language processing). On the basis of handmade codes, the computer “learned”
to recognize odes in documents, and this enabled us to analyse a large dataset of news
items (N = 2265 items).

Media Logic as Institutional Feature and Its Consequences for Media

Reporting on Public Risks
News media fulfil a democratic and a commercial task when distributing information
to the public. In their democratic function, media inform the public and can operate as a
 FRAMING A CONFLICT! 3

“watchdog” in the political system (Bennett 2009). In the case of public risks, this means that
they can raise awareness about the nature of the risk and its consequences; but they can
also alert citizens to stimulate public discussion or to take action to mitigate the risk. A
model for how information about risk and its consequences is communicated in society
has been proposed by Kasperson et al. in (1988). In their Social Amplification of Risk Frame-
work (SARF) they postulate how communications about risk events and issues pass from
sender to message receivers through intermediate “stations”. These stations can be
persons, groups and organizations but especially the media. During transmission of the
messages each station can add biases to reframe the message, which may result in attenu-
ations or amplifications of the perceived risk. Whether or not information about a risk has
serious impact on society is according to SARF to a large extend determined by amplifier
stations which further disseminate and transfer messages about the risk so that it
“ripples” through society. A risk issue can easily be amplified in society when emotional
elements are added such as anger, fear, conflict, trust and (lack of) compassion (Renn
1992; Slovic 2000). Thus, media reporting about risk is not necessarily a reflection of the
actual hazard and its primary consequences. Johnson and Covello (2012) argued that
media may exaggerate some risk and ignore others, because media tend to focus on
drama, wrongdoing and conflicts. Soumerai, Ross-Degnan, and Kahn (1992) concluded
that media tend to concentrate on rare and dramatic hazards, and often fail to report
common serious risk. Recently Stewart and Lewis (2017, 122) argued that in the field of
geoscience communication. “factual information is to be subordinate to values and beliefs
… ”. Wahlberg and Sjoberg (2000), concluded however, that news media are not always as
biased in their reporting as often thought.
Media are not only transmitter stations reporting about risk and other issues of inter-
est for society. They are also commercial entities that have to survive in a competitive
market (Landerer 2013). Several scholars argue that this commercial interest is reflected
as an institutional practice of the media which is called media logic, i.e. set of rules and prac-
tices regulating actors’ behaviour within media as an institution (Cook 2005; Scott 1995; Asp
2014; Klijn and Koppenjan 2016), Esser and Strömbäck (2014). Media logic, especially the
rules that are connected with the need to survive in a commercial competitive market
(see Landerer 2013) may significantly influence the selection (content) and tone of news
coverage, introducing biases that find their origin in pressure for media to reach a large
audience and bring the news in such a way that it is attractive to a large audience.
General application of media logic results in similarities of content and sentiment of
news coverage in various media a number of authors argue (Altheide and Snow 1992; Hjar-
vard 2013; Landerer 2013). In general media logic may result in less factual information and
the news may contain more human interest stories and drama to attract news consumers
(Mazzoleni and Schulz 1999; Bennett 2009; Hjarvard 2013; Strömbäck and Esser 2014).
Media logic manifests itself in the framing of news content. A frame is described by
Entman “as a process of culling a few elements of perceived reality and assembling a nar-
rative that highlights connections among them to promote a particular interpretation”
and “make them more salient in a communication text, in such a way as to promote a
particular problem definition, causal interpretation, moral evaluation and/or treatment
recommendations” (Entman 1993, 52). A frame tells what elements are meaningful and
are uncovered by stories and story lines. In media articles they are mainly represented
by certain (combination of) words, emotional meanings and combinations of these
(see also Entman 2007).
4 ALETTE EVA OPPERHUIZEN ET AL.

Patterson (2000) looked at 10 years of news production in the United States and
noticed a trend towards more negative attention (especially towards politicians) and a
move towards soft news and attention on personalized news. Bartholome, Lecheler, and
de Vreese (2015) reported that journalists commonly use storytelling, adding elements of
conflict to a story to transform events into news commodities. This may result in personal-
ized stories resulting in a situation of news as a sales commodity, rather than news as trans-
ferring information in a meaningful way to the public; like inform and educate citizens, to
publicize actions of the government, to provide a platform for competing or dissenting
opinions, or to serve as an advocate of competing political views (Mcnair 2009; Eberl,
Wagner, and Boomgaarden 2016) Flew and Swift (2015) argue that since media have to
compete for attention in the public spheres journalists have to present more details
about the issue in a more dramatic way (see also Esser 1999; Slovic 2000). This observation
fits with Baumgartner and Jones (2009) conclusions in their agenda forming study about
journalists’ preference for conflict. In an in-depth analysis of six Dutch spatial projects
and their decision processes, Korthagen (2015) shows that conflict and dramatization
biases are clearly present in media attention for these projects. The way different media
outlets use biases may also depend on whether they are more or less sensation-orientated.
More sensation-focused newspapers (the so-called tabloids) attempt to make themselves
more attractive by entertainment-oriented stories, and news items become more sensa-
tional (Blumler and Gurevitch 1995; Grabe, Zhou, and Barnett 2011). Earlier research has
shown that sensationalistic newspapers are more focused on personalization and conflict
than quality newspapers are (Norris and Kennedy 2001). Sensationalistic newspapers also
use more biases in there framing that provoke emotional reactions with readers (Mott
1962).
Entman (2007, 166) defined “ … consistent patterns in the framing of mediated com-
munication …” as a bias. He argued that by introducing biases one particular side of an
issue of interest is highlighted by a media outlet. Since news reports must be saleable
the information of media has certain biases. A bias of media can be seen as “structural unre-
ality of images” (Baudrillard 1995, 46). Based on the work of a variety of authors (like Pat-
terson 2000; Bennett 2009; Burscher et al. 2014 and Korthagen 2015) common
information biases may be identified when media biases make consistent use of narrative
elements referring to the following:

1. Personalization bias: a strong tendency in the news to emphasize the personal aspect
of news and downplay the socioeconomic or political context in which the event
takes place. Emphasizing the personal aspect of news may make the news more
appealing to readers and viewers, but the greater complexity of the issue may be
ignored or relegated.
2. Dramatization bias: a strong tendency towards dramatizing news, emphasizing crisis
and conflict in stories, rather than continuity or harmony.
3. Fragmentation bias: an increasing focus on isolated stories and events, separating
these from the wider context and from one another.
4. Authority–disorder bias, or a preoccupation with order and whether authorities are
capable of maintaining or restoring that order. At the same time, a shift has taken
place from an attitude where the media are favourable to politicians and authorities
towards an attitude where they are suspicious of them.
5. Negativity bias, refers to the tendency of the news to be more negative in general.
 FRAMING A CONFLICT! 5

Baumgartner and Jones (2009) studied the media reporting of various public risk and
benefits in the United States of America. They show that policy systems and the agendas in
these system may be stable for long period but then show rapid changes as result of con-
tinuing pressure from outside. Media attention is very important in building up this
pressure they find. Or as they formulate it
A major source of instability in American politics is the shifting attention of the media.
Media outlets generally base their stories on a limited number of sources and imitate
each other, so ideas and stories often spread quickly once they have become a topic of
interest. (Baumgartner and Jones 2009, 103)
Baumgartner and Jones also observe that media have a fascination with conflict and
competition and media attention seems to generate more media attention (positive feed-
back). This is in line with observations of other authors that journalists have a tendency to
follow each other because they are afraid to miss a “scope” and thus media attention tends
to generate more media attention (Bennett 2009).
These theoretical observations lead us to the following hypotheses about our case
(gas drilling and its media coverage):
H1: Media attention increases disproportionally when actual risk events (number and
intensity of earthquakes) increases;

H2: From the onset media disproportionally increased their reporting about the risk of
earthquakes, a tendency for all newspapers to reframe the news content using more per-
sonalization, dramatization, and negativity biases;

H3: The use of personalization, dramatization and conflict biases to frame the news will be
more prominent in sensational newspapers than in quality newspapers.

The Case: Gas Drilling-Earthquakes 25 Years in the Media

Gas drilling started in the early 1960s in the northern part of The Netherlands after a
large volume of gas was discovered. Production and sales of this gas have resulted in very
large revenues for the Dutch government. The gas drilling activity, at €265 billion, is an
important activity for the Dutch economy. Gas drilling also has negative consequences
however; it causes land subsidence and earthquakes.

Gas Drilling in The Netherlands: Actors and Responsibilities

After the gas fields were found, a public–private collaboration was established, called
the het gashuis (gas building). The natural gas exploration company (NAM) is responsible for
safe extraction of gas and for the external effects of the extraction. It makes a production plan,
which has to be approved by the Minister of Economic Affairs. The minister has authority on
behalf of the State over gas extraction, including licences for exploration and exploitation.
The minster has the power to approve, disapprove, or attach conditions to the production
plan. The controlling agency is the State Supervision of the Mines (SODM). SODM ensures
that all parties comply with the legislative rules, and it monitors the implementation of gas
extraction. The supervision authority focuses on safety, health, milieu and effective extraction.
6 ALETTE EVA OPPERHUIZEN ET AL.

Earthquakes in The Netherlands: Facts and Figures

The first earthquakes started in the late 1980s, and the frequency and magnitude
have increased since then. In 1994, more than 20 were detected; this number decreased
until 2003. From 2003 to 2015, there was a substantially higher risk compared with the
1990s. Peak years for the number and magnitude of earthquakes were 2003, 2006, 2009
and from 2011 (see Figure 3 in results section).

Risk Development and Media

In the first decades of gas drilling, experts evaluated land subsidence as the only
negative side-effect of gas drilling. Although land subsidence was seen as an undesirable
side-effect, experts did not predict great risk concerns. However, the issue of public risk
arose when in 1986 the number of earthquakes increased. The relation between gas extrac-
tion and risk effects has been proved since 1993 (KNMI 1993). The risk of earthquakes was
higher and more complex than was initially thought, but the concerned parties did not see
the earthquakes as a safety problem but rather as a material damage problem, e.g. cracks in
houses (Dutch Safety Board 2015). Although in 2003, 2006, 2009 and from 2011 there was a
significantly higher risk (stronger earthquakes of >3, 3.5 Richter scale), this did not lead to a
perception change by concerned parties, social groups or mass media (Dutch Safety Board
2015). News media did not pay much attention to the risk of land subsidence resulting in
damaging nature and houses. In august of 2012 a larger earthquake (3.6 Richter scale)
occurred, this earthquake led to activity in the policy sphere. The controlling agency is
the SODM started a risk analysis. In the beginning of 2013 SODM published they report,
which concludes that safety cannot be guaranteed for the inhabitants of the Northern
region. This warning may function as trigger event and awoke new interest in the earth-
quake risk. The concerned parties, especially the minister, were requested to take action.
Citizens felt more unsafe and angry towards involved parties. Citizens’ trust level
dropped to an absolute minimum, because the citizens assumed that the concerned
parties had downplayed the seriousness of the risk situation and because of the lack of
transparent information (Dutch Safety Board 2015). The increased risk itself and the
SODM warning gave rise to increased social, media, and political attention. In 2014, the Min-
ister of Economic Affairs presented a package of measures to ensure the safety of civilians.
At the minister’s request, the Dutch Safety Board decided to launch an investigation into
the decision-making process. The Board (2015, 7): conclude that: “the parties concerned
deemed the safety risk to the population to be negligible and thus disregarded the uncer-
tainties surrounding this risk assessment”

Method: Machine Learning Technique

This media analysis is based on a SML by five Dutch newspapers over the period
1990–2015 on gas drillings in The Netherlands. A case study approach was chosen
because of the ability to generate in-depth knowledge of media reporting and news
framing of public risks. The gas drilling case is interesting because of the long period of
time in which the drillings took place, the changing perspective on the public risk of gas
drilling and the change in media coverage over time. This allows a longitudinal study of
frame variation in media coverage.
 FRAMING A CONFLICT! 7

Data Collection
In this article, one local and four national newspapers are selected. The national
newspapers have different political orientations: Dagblad van het Noorden (a locally
oriented newspaper), NRC Handelsblad (a centre-right quality newspaper), de Volkskr-
ant (a centre-left quality newspaper), de Telegraaf (a right-leaning sensational newspa-
per), and Algemeen Dagblad (non-politically orientated sensational newspaper). The
articles were selected from digital archive LexisNexis NL. The search query “gaswinning
OR gasboring AND Groningen AND NOT Waddenzee” was used to select the relevant
articles. Although LexisNexis is a comprehensive newspaper database in the Nether-
lands, the local newspaper Dagblad van het Noorden was only available from 1999
to 2016. The national newspapers were available from 1990 to 2016. This may have
led to some missing information in the sample reports. A total of 4113 articles were
found in the database. Because Dagblad van het Noorden has geographical variants
(i.e. “North”, “East”, “South” and “West” editions), one news article from this newspaper
could appear in each edition, which led to many duplicates in our dataset. After
removing all duplicate news items, a total database of 2265 relevant media reports
constitutes the final sample. Eight hundred and twenty six (36 per cent) of the
reports originate from the national newspapers, and 1439 (64 per cent) from the
local newspaper.

Qualitative Content Analysis

The unit of analysis was a news report. First a subset of 102 media reports was used
for inductively human coding. After a first indicative round of coding, Patterson’s (2000)
coding scheme in combination with Burscher’s et al. (2014) frame indicator questions for
quantitative content analysis was used. Patterson’s code scheme does not provide yes or
no indicator questions for analysis. Hence the indicator questions of Burscher’s et al.
(2014) media frame analysis are used. Only for three of the five information biases men-
tioned above, three could be operationalized. For fragmentation and authority-bias no ade-
quate operationalizations could be developed. This study focus on:
1. Personalization operationalized as: “is the story about the use of the human interest
frame. Human interest stories use a human”. Labelled categories are yes or no.
2. Dramatization is operationalized in two ways: (i) Political disagreement is operationa-
lized as: “Does the item reflect disagreement between parties, individuals, groups or
countries?” (ii) value conflict operationalized as: “Does the item refer to two sides or
more than two sides of the problem?” Labelled categories are yes or no.
3. Negativity is operationalized by Patterson as: is the report was favourable or unfavour-
able towards gas drilling? Labelled categories are positive–neutral–negative. To illus-
trated in more detail three examples are given:
. An example of a news item coded as positive:
The country is full self-confident and proud, they not cared of no one. From the periphery
we have become the centre. The classic image of the needy, indignant and distressing
North no longer exists. This is a performance of format.
. An example of a news item coded as neutral:
8 ALETTE EVA OPPERHUIZEN ET AL.

The gas extraction in Groningen will be limited to 27 billion cubic meters until 30 Septem-
ber 2016. Minister Henk Camp of Economic Affairs follows the judgment of the State
Council, which said last month that no more than 27 billion kuub should be won.
. An example of a news item coded as negative: “The gas operator NAM apologizes
for the earthquake distress in Groningen. But they do not even think about a pro-
duction reduction, it is all about the money and not about the people”.
. Than the human-coded subset of 102 articles was exported from ATLAS.ti to
XML and formed the input for the machine learning component.
SML: Train Model, Predicted Codes and Evaluate Performance
SML was preferred over solely human coding content analysis because it enable us to
code biases in the news without relying on small sample. With this technique, a computer
learns to code from a set of human-coded training documents (Sebastiani 2002). In this
longitudinal study a set of 2265 articles was available and because it is not feasible to anno-
tate such a great number of news items manually, a machine learning approach is taken, in
which an algorithm learns to recognize patterns in the text that correspond to the manually
assigned codes. In this way, only a subset of the news items needs to be human-coded, as
the machine learning algorithm is able to predict the codes for the remaining part of the
dataset. Also, in the work of Burscher et al. (2014) “Teaching the computer to code
frames in News” they conclude that SML is well suited for frame coding, for theory but
also as methodology (Burscher et al. 2014). Lazer et al. (2009) argue that Computational
Social Science can help with comprehensive societal-level communication patterns. More
specifically, and in line with this study, different scholars argue that SML can contribute
to substantial issues in framing reaches, including “looking at frame variation over time”
(Matthes and Schemer 2012). As the machine learning algorithm is a statistical method
that works with numerical values, it cannot work with plain textual documents. Figure 1
shows the three processing steps that are performed to transform the plain text documents
into numerical vectors that can be used for machine learning.
In the process of programming Java was applied. The first step is a pre-processing
step that involves cleaning the document of any formatting and adjusting the text to
prevent mistakes later on in the process. For instance, headlines of news items do not
have a full stop at the end, but, when formatting is removed, a headline is hard to separate
from the first sentence of the actual news item. Adding an extra empty line makes it clear
that this is a separate sentence. The same is true for sentences that end with a quote, as the
full stop is usually put inside the quote. However, with a full stop denoting the end of the
sentence, the end quote is by default incorrectly merged with the next sentence.
The second step consists of running the pre-processed documents through a natural
language pipeline for Dutch, called Frog, which extracts all kinds of linguistic information
from the text (Van den Bosch et al. 2007). On a basic level, it splits the text, which is
simply a long list of characters for a computer, into groups of characters that comprise
words. Then, the list of words is grouped into sentences, followed by determining the
word type of each word within the sentence (e.g. nouns, verbs, adjectives and so forth). Fur-
thermore, the words are morphologically analysed, which means that they are related to
their lemma, or dictionary form. This step is useful, because it allows the algorithm to
know that some words, even though they are different in form (e.g. be, are and is), are prac-
tically the same in meaning. Another task for the morphological analyser is to split com-
pound nouns into their constituent parts. As the Dutch language allows for the creation
 FRAMING A CONFLICT! 9

FIGURE 1
The machine learning process

of new nouns by simply compounding two or more existing nouns, it is informative to know
the constituents. For example, the Dutch word aardgasbeving (i.e. natural gas earthquake) is
split into aard (earth), gas (gas) and beving (quake), relating it to the more regular word
aardbeving (earthquake) because it shares two constituents. For humans this is apparent,
but for computers, when two lists of characters are not exactly the same, they are comple-
tely different—a fact that is often useful, as words and their inverted counterparts can be
quite similar (e.g. (un)informative, hypernym vs. hyponym). The last step in this part of the
process is to combine words that are part of a phrase or chunk, by assigning them a
chunk tag. Chunks are multi-word expressions that together have a different meaning
than when considered separately (e.g. United States of America).
The third step is to select from all this information those bits of information—called fea-
tures—that are expected to be informative with respect to the prediction task. This feature
designing and feature selection phase are repeated multiple times until satisfactory results
are achieved. By training the algorithm using a selection of features and measuring its perform-
ance, useful feedback is retrieved that can help in designing and selecting better features. In
this work, the set of selected features consists of the lemmas of the words in the document,
the chunk tags assigned to the words in the document, and the morphological constituents
of words in the document. All these features are binary, meaning that they are encoded
with a 1 if present in the document and a 0 otherwise. The Support Vector Machine (SVM)
10 ALETTE EVA OPPERHUIZEN ET AL.

FIGURE 2
SVM

implementation is based on SMVLib. Hence, the input vector has a length equal to the number
of different features, with mostly 0s and a relatively small number of 1s, see Figure 2.
Besides these binary features, a sentiment dictionary from the CLiPS Pattern project
(De Smedt and Daelemans 2012) is used to count the number of positive and negative
words in a document, as well as the number of objective and subjective words. Further-
more, as each word has a numeric value in this dictionary for sentiment and subjectivity,
a total sentiment value and a total subjectivity value is also computed. These values are
added as numeric features to the input vector. The machine learning process consists of
two phases: a feature development phase (left side of Figure 1) and an active learning
phase (right side of Figure 1). In each phase, the textual input is processed modelled as
a vector of numeric values, as described above. In the first phase, different sets of features
are experimented with and the main output is the definitive set of features used to predict
the codes. The feature selection is used as input for the second phase, where active learning
is used to check and correct the predictions about which the algorithm is least confident.
After a certain number of rounds of active learning, the generated predictions are final and
the content analysis can commence.

Accuracy and Reliably

To ensure that the given results are accurate and reliable two scores are computed:
the accuracy scores (shows how accurate the algorithm is) and the standard deviation (how
precise the algorithm is in case of repeatability). The accuracy of the algorithm is measured
with F1-score, a measure that is the harmonic mean of precision and recall. Precision
measures how many of the predictions that have been made by the algorithm are correct.

Correctly predicted codes

.
Correctly predicted codes + Incorrectly predicted codes
 FRAMING A CONFLICT! 11

Recall, on the other hand, measures how many of the codes that should have been
predicted, are actually found by the algorithm.

Correctly predicted codes

.
Correctly predicted codes + Missed codes

Precision and recall balance each other, in the sense that it is easy to get high pre-
cision at the expense of having low recall (e.g. predicting only a few instances that are
easy to find) and high recall at the expense of low precision (e.g. predicting a code every-
where). The F1-score represents the balance between these two important measures. Tra-
ditionally, when measuring performance, part of the manually coded data set is used for
training the machine learning algorithm, and part of it is reserved for testing only. This
ensures one is measuring predictive power of the algorithm rather than goodness of fit.
Since the manually coded portion of the dataset is relatively small, the performance of
machine learning will vary based on which news items are in the test set. If the test set con-
sists of news items that are easy to classify the performance will obviously be higher than
when the test set consists of hard to classify news items. To counter this, the algorithm is
run 20 times, where the split between training and test set is randomly performed each
time. The reported F1-scores are therefore the average score over those 20 runs. To give
an impression of the stability of the results, the standard deviation over those 20 scores
has also been computed. The higher this number, the larger the variation among those
20 F1-scores. To achieve a higher F1-score and lower variation, a procedure called Active
Learning is employed. With Active Learning, the machine learning algorithm is used to
output not just the predictions themselves, but also the probability for each possible
code. A low probability indicates that it was hard for the algorithm to assign a code to
that news item. Then, for each code, the news items with the lowest probability are manu-
ally coded and added to the training set (Table 1). In Table 2, an overview of the perform-
ance for each of the codes is presented, before and after performing a round of Active
Learning. The last column denotes the majority baseline, which entails simply predicting
the dominant code for each news item. For example, as about 73 per cent of the news
items has the “No disagreement” label, the baseline, by naively predicting “No disagree-
ment” for all news items, would achieve an F1-score of 73 per cent. Intuitively, to have
any added benefit, an algorithm should exceed this baseline, as is the case for each of
the codes. Note that this baseline is thus an indicator of how difficult it is to predict this
code. It is naturally a lot harder to predict the sentiment code correctly, than for instance
the disagreement code. All the codes are above the baseline.

Results
How Do Media Pay Attention Over Time to the Risks of Earthquakes?
The results show that the risk of earthquakes as a result of gas drilling did not attract
much attention in the national Dutch newspapers until 2012. Annually, for each newspaper,
less than 10 news items covered this risk. NRC Handelsblad started reporting about the risk
of earthquakes in the 1990s with a few (n = 27) articles in the period between 1990 and
2002, but other national newspapers did not follow.
12 ALETTE EVA OPPERHUIZEN ET AL.

TABLE 1
Coding scheme—questions for bias indications (this conceptualization is based on Patterson
2000, 24–26 and Burscher et al. 2014, 197)

Definition Indicator questions Category

Personalization bias
Personalization bias adds a human face to . Does the item provide a human Yes
news coverage. example or human face on the No
issue?
Dramatization bias
Highlights conflict between individuals, groups or institutions.
Political disagreement = the political . Does the item reflect disagreement Yes
disagreement bias highlight conflict between political parties about the No
between political actors. technical activity of gas drilling?
Value conflict = the value conflict bias . Does the item refer to two sides Yes
highlight conflict in groups in society (finical gain vs. earthquake risk) of No
between benefit on the one side (the the gas drilling activity now nor in
winners) and the risk (losers) on the other the future?
side.
Negativity bias
This code is designed to pick up whether . Is the report favourable or Positive
the story is thought on the whole to be in unfavourable towards gas drilling? Neutral
the good news or bad news category. In Negative
some instances, it might be helpful to ask
the following questions: If about a
newsmaker and you were his/her press
secretary, would you consider this a
favourable or an unfavourable story? If
about an institution (e.g. Congress), does
this reflect favourably or unfavourably
on the institution? (Patterson 2000)

In 2002, other newspapers started to become interested, which the exception of

Algemeen Dagblad, which started to report about this topic only in 2008. The national news-
papers increased their reporting from 2002 to 2006, followed by a small decrease in 2007
and 2008; see Figure 3. A slight increase followed in the years 2009 to 2011, again followed
by a small decrease in report numbers in 2012.
The local newspaper (Dagblad van het Noorden) is an exception and started to
increase covering the news about risks a few years earlier than the national newspapers;
see Figure 3. From 1999 onwards, usually 10 or more news items were reported annually

TABLE 2
F1-scores

F1-score before active F1-score after active Majority

learning learning SD baseline
Negativity 0.46 0.53 0.11 0.42
political conflict 0.71 0.74 0.11 0.63
value conflict 0.78 0.81 0.09 0.72
Personalization 0.82 0.82 0.08 0.72
 FRAMING A CONFLICT! 13

FIGURE 3
Amount of news articles (left) and earthquakes (right) per year

until 2012. Both for the local and the 4 national newspapers in 2013, the number of
reported news items increased at least 10 times, and increased further in 2014 and 2015
(see Figure 3). The results show a disproportional increase in media attention after 2013,
revealing that media attention was only partially related to the increased risk itself in this
period.

Which News Biases Are Used in Framing the Risk and Benefits of Gas
Drilling?
To unravel the potential use of information biases in framing the use by the newspa-
pers over the 25 years of news coverage, a separation was made between the period before
2013 and after. As there were a disproportional number of articles before and after 2013,
the media coverage from 1990 to 2012 is combined.
By using the outcomes of SML the potential use information biases have been
studied. The application of biases is shown in Figure 4 panel a; for personalization, panel
b; for political disagreement, panel c. for value conflict and panel d. for negativity.

Personalization
In 20 per cent in 1990–2012 and 40 per cent in 2014 of the news articles there were
strong elements of a human story, human face, or human example (see Figure 4(a)). There
was no proportional rising or falling trend in the use of personalization bias. The slight
drop in the use of personalization bias in 2013 was followed by an increase in 2014.
After the following drop in 2015, again in about 33 per cent of news items a personaliza-
tion bias was applied, comparable to the period 1990–2012, despite the large increase in
the total number of news item. This is interesting since the mediatization literature
suggests that there is a growing trend towards personalization. We thus cannot find
this trend in our data.
14 ALETTE EVA OPPERHUIZEN ET AL.

FIGURE 4
Percentage of media used biases in different time intervals. Panel a for personalization,
Panel b for political disagreement. panel c for value conflict, panel d for negativity

Dramatization: Political Disagreement and Value Conflicts

An increase in the use of a political disagreement bias by all the newspapers each
year is seen after 2012 (see Figure 4(b)), i.e. the use of a political disagreement frame
becomes more prominent in reports from media outlets. Before 2013, almost no news
media addressed political disagreement. There was a linear increase of 5 per cent in all
the news reports each year, meaning that in 2015 about 15 per cent of all the articles
covered this dramatization bias. This can of course be related to the fact that after 2013
the issue was also discussed more in the political arena and the discussion about reducing
the amount of gas drilling started.
Even more prominent in the years 2013–2015 is the reporting about value conflicts,
but even before 2012 approximately 10 per cent of the news items covered value conflict
(see Figure 4(c)). The media and public discourse shifted from a damage issue towards a
safety issue from 2013. The use of a value conflict frame increased in 2013 and stabilized
in 2014, followed by a further increase in 2015. Altogether, there was an increase of
nearly 20 per cent in 3 years (2013, 2014 and 2015), whereas before 2013 almost one in
10 news media reports was dedicated to disagreement between the interests of economic
values and safety values, in 2015 almost 1 in 3 papers used this dramatization bias. This is
 FRAMING A CONFLICT! 15

interesting because, although of course these value conflicts are clear and important in this
case, they basically did not change much over time. However, with the rising attention on
the risks of earthquakes and the position of citizens, this value conflict became more pro-
minent and interesting to report.

Negativity
In the period between 1990 and 2012, the majority of news articles were neutral in
their sentiment, i.e. they did not use a negative media frame. Around 30 per cent of the
total news coverage used elements that indicate that the tone of the article was negative.
Only a few articles were coded as positive (see Figure 4(d)). When the articles dealt with the
topic of gas drillings, there was a lack of positive news reporting. After the year 2013, media
became more negative in their reporting. In 2014, almost one in two of the many articles
made use of negativity bias. In 2015, the effect reduced slightly, and the negative bias was
less used than in 2014.

Difference Between Newspapers

The data show that sensational newspapers used more personalization bias than
quality newspapers did (see Figure 5(a)). Quality newspapers focused more on dramatiza-
tion, in particular value conflicts (see Figure 5(b)).
In 2014, the personalization bias was used to the same extent by all the newspapers.
With Algemeen Dagblad (21 per cent), NRC Handelsblad (22 per cent), Dagblad van het
Noorden (26 per cent), and de Volkskrant (30 per cent), the newspapers are almost equival-
ent in the use of this frame, see Figure 5(a). Only de Telegraaf (44 per cent) used the perso-
nalization bias significantly more. This shows a difference between this sensational
newspaper and the quality newspapers in the use of a personalization bias in framing
the news.
The use of political disagreement is relatively low in comparison to other biases (see
Figure 5(b)). The quality newspaper de Volkskrant had already started using political dis-
agreement before 2013 in almost one in 10 articles. Other newspapers published less on
the disagreement between political parties or individuals from political parties. There
was a small difference (approximately 3 per cent) between the newspapers in their
average use of narratives of the political disagreement from 2013 to 2015: Algemeen
Dagblad (12 per cent), de Volkskrant (10 per cent), NRC Handelsblad (10 per cent),
Dagblad van het Noorden (9 per cent), and de Telegraaf (9 per cent). From 2012 to 2015,
Algemeen Dagblad had the strongest increase in the use of this bias, from less than 5 per
cent to more than 15 per cent (see Figure 5(b)).
As shown in Figure 5(c) all the newspapers shifted from 2013 to 2014 in the use of
value conflicts in framing their news, except for NRC Handelsblad. After 2013, there was a
significant difference between the newspapers and their use of a value conflict frame.
The quality papers used the value conflict frame more often than sensational newspapers.
NRC Handelsblad (42 per cent) and de Volkskrant (36 per cent) used the value conflict more
often than Algemeen Dagblad (27 per cent), de Telegraaf (22 per cent), and Dagblad van het
Noorden (21 per cent).
Negativity bias is most commonly used by all the media in framing the news about
earthquake risk, as illustrated in Figure 5(d)). Only the Algemeen Dagblad (30 per cent)
16 ALETTE EVA OPPERHUIZEN ET AL.

(a) Personalisation (b) Political disagreement

60% 20%

50%
15%
40%

30% 10%

20%
5%
10%

0% 0%
2013 2014 2015 2013 2014 2015
AD DvhN De Telegraaf de Volkskrant NRC AD DvhN De Telegraaf de Volkskrant NRC

(c) Value conflict (d) Negativity

60% 60%

50% 50%

40% 40%

30% 30%

20% 20%

10% 10%

0% 0%
2013 2014 2015 2013 2014 2015

AD DvhN De Telegraaf de Volkskrant NRC AD DvhN De Telegraaf de Volkskrant NRC

FIGURE 5
Percentage of national and local news media used biases in different time intervals. Panel
a for personalization, panel b for political disagreement, panel c for value conflict, panel
d for negativity

compared to other newspapers reported less negatively. The other newspapers de Volkskr-
ant (50 per cent), de Telegraaf (49 per cent), NRC Handelsblad (48 per cent), and Dagblad van
het Noorden (42 per cent) used negativity bias in almost half of their articles, but none of
them increased or decreased the use of negativity over the years.

Conclusion and Discussion

In most framing studies news sentiment and content are coded using content analy-
sis (Matthes 2009). Human coding is, however, a research-intensive process, and therefore
much research focuses on a relatively small selection of news articles. By using SML, we
were able to perform a longitudinal study of frame variation in media coverage over
time. It was possible to answer the question
How do media over time pay attention to the risks of earthquakes as result of gas drilling
in The Netherlands. And which news biases dominate (and does this differ during the time
period) and does this differ for various newspapers?

Media Attention Unevenly Distributed

Earthquakes were reported in The Netherlands around 1990 as result of gas drilling.
Although there have been earthquakes since 1990, and in 1993 researchers reported the
 FRAMING A CONFLICT! 17

relationship between the earthquakes and gas drilling, media coverage on earthquakes has
been very limited. The media analysis of earthquake risks in The Netherlands indicates that
the media played only a minor role in signalling the earthquake risks in an early phase
(democratic function). In the years 2003 and 2009, a slight increase in the number of
news articles can be seen compared with previous years, but these increases did not con-
tinue in subsequent years and are not in relation to the increase of the actual earthquakes
in the first decade of the twenty-first century. If media coverage is mainly a reflection of
hazard, one would expect a major change in media coverage of earthquake risk in 2009,
a year with many and stronger earthquakes. This, however, does not show itself in the
media analysis. The absences of media attention are remarkable because media are seen
as the most prominent information channel related to risk communication for the
general public. However, the lack of reporting and signalling of the slowly emerging earth-
quake risk is in line with the theory of Baumgartner and Jones. The tone and content of the
media reporting had been almost “stable” for a long period (1990–2012). Our conclusion
could be that the media performed their role as the so-called watchdog not very prominent
before 2012 and only became active after 2012. Probably triggered by the publication in
(trigger event) of a report of the SODM in 2013. At the same time 2013 was also a year
with more and more intense earthquakes. Therefore, the result shows that the fluctuations
in media attention can only partially be related to the actual earthquake hazard; the
increased earthquake risk itself does not seem to be decisive in the enormous and rapid
media attention shift in 2013. These data confirm our first hypothesis, i.e. that media report-
ing is disproportional to the actual risk event.

Biases and Patterns in Media Attention

Also in agreement with Baumgartner and Jones is the fast disrupted shift in media
attention for the risk of earthquakes in 2013. Not only the number of media reports
increased dramatically, also the framing of the news shifted. New specific information
biases were consistently introduced by the media to reframe the news, which is also in
agreement with scholars such as Entman (2007) and Baumgartner and Jones (2009). In par-
ticular the dramatization bias was introduced after 2012 to reframe the reporting. This
suggests also that media attention is also partly a result of media logic itself, since in par-
ticular dramatization may be used by media outlets to serve the readers. Our data seem to
point at the conclusion that once an issue has reached a certain critical mass and gained
momentum, newspapers report more and more about it, and positive feedback mechan-
isms can be observed where media attention causes more new media attention (see also
Baumgartner and Jones 2009 for this phenomenon).This seems to be an indication of
the journalist following each other (the pack of journalist) but since we have not inter-
viewed journalist about their choices we cannot prove this.
In that race for attention, visible after 2012, all newspapers made use of personaliza-
tion, dramatization, and negativity biases in their reporting on earthquake risks. This
phenomenon of copying behaviour and the homogenization of content in order to
reach a larger readers population has been reported before by scholars such as Entman
(1993).
Negativity is the most dominant information bias. This is not surprising because,
whereas negativity concerns all kinds of topics, other biases are more forced to focus on
certain topics, e.g. safety vs. money (value conflict), or political debate (political
18 ALETTE EVA OPPERHUIZEN ET AL.

disagreement), or a story about a human (personalization). The second most dominant bias
is the personalization, followed by the value conflict and the political disagreement as dra-
matization bias, which occurs least in all the news media. This is consistent with what
Bennett (2009) proposed when he highlighted the mediatization element. Interestingly,
although negativity and personalization biases are often used by all the newspapers
when they report about the gas drilling risks, our analysis does not show an increase in
the use of these media bias in the period 2013 to 2015. Dramatization bias, in particular
political and value conflicts however, became much more prominent in the news reporting.
Our second hypothesis that from the onset media disproportionally increases rela-
tively more than the physical risk of earthquakes, for all newspapers to use more persona-
lization, dramatization, and negativity biases may be expected, is thus not fully supported
by our data. Actually support for the hypothesis is only found for dramatization bias, but not
for negativity and personalization biases. Whether or not this is unique for the Dutch gas
drilling is unclear, and our second hypothesis deserves further study with other cases of
developing public risk after the introduction of manmade technologies.
According to the literature, the use of biases can be explained by the different char-
acters of the newspapers. From the literature, we expected a difference between sensa-
tional papers (Algemeen Dagblad and de Telegraaf) and quality papers (de Volkskrant
and NRC Handelsblad) in the use of biases. Hypothesis 3, about more use of personalization
and disagreement biases in sensational newspapers than in quality newspapers, is not fully
supported by our data. Actually, for the disagreement biases, the reverse is observed, i.e.
more value conflict framing in the quality newspapers NRC Handelsblad and de Volkskrant.
In the sensational newspapers, we see greater use of personalization bias. The sensational
newspaper de Telegraaf reported almost twice as many personal stories as the other news-
papers. It can thus be concluded that the expectation is partially supported. The use of
negativity and political disagreement does not differ much between sensational and
quality newspapers. They all report a lot of negative news, and all make limited use of
the political frame.

Limitations and Final Reflections

This study has its limitations, one of the obvious being the fact that although we
covered a long period we have only analysed one case. Further research should show
whether the patterns we find also hold for other cases and especially other countries
with different media landscapes. It is also clear that our coding cannot be disconnected
entirely from the events and contextual situation of the case. Thus we find more dramatiza-
tion after 2013 and especially political disagreement, but we would also argue that political
disagreement increases because of the massive media attention which increases pressure
on politicians. And of course machine coding has some disadvantages over human coding.
SML was applied because of it reliably in coding (no human judgement) and of it time
savings whereby an extensive analysis could be provided. Because it is a relative new tech-
nique and not often used in the social science it was challenging in applying the technique.
Therefor this research was still a very time costly. There is much more research needed in
the field of social and communicational science, to make SML an accessible technique for
content analysis. Despite these limitations we however think that looking in the way we did
in this article contributes to our understanding of attention patterns of media and their
effects. And it shows the rapid changes in attention patterns and the way public risks
 FRAMING A CONFLICT! 19

are discussed. This makes decision-making around these issue even more unpredictable
and complex since all actors involved in the issue will have to react on these changing
media attention. In this way media reframing the news contributes significantly to the com-
plexity of decision-making of public risks but also to the challenge for public managers and
public office holders to manage these processes. We may think that we observe biases in
media attention but those biases also generate political and policy attention and thus have
positive effects. This is something we should be looking more into depth as it is a major part
of political and policy processes.

DISCLOSURE STATEMENT
No potential conflict of interest was reported by the authors.

REFERENCES
Altheide, David L, and Robert Snow. 1992. “Media Logic and Culture: Reply to Oakes.” Media 5:
465–472.
Asp, Kent. 2014. “News Media Logic in a New Institutional Perspective.” Journalism Studies 15
(December): 256–270.
Bartholome, Guus, Sophie Lecheler, and Cleas de Vreese. 2015. “Manufacturing Conflict? How
Journalists Intervene in the Conflict Frame Building Process.” The International Journal of
Press/Politics 20 (4): 438–457.
Baudrillard, Jean. 1995. “The Gulf War Did Not Take Place. Sydney: Power.
Baumgartner, Frank R., and Bryan D. Jones. 2009. Agendas and Instability in American Politics.
Chicago: The University of Chicago Press.
Beattie, Peter, and Jovan Milojevich. 2017. “A Test of the ‘News Diversity’ Standard.” The Inter-
national Journal of Press/Politics 22 (1): 3–22.
Bennett, W. Lance. 2009. News. The Politics of Illusion. 8th ed. New York: Pearson Longman.
Blumler, Jay G., and Michael Gurevitch. 1995. The Crisis of Public Communication. London:
Routledge.
Burscher, Björn, Daan Odijk, Rens Vliegenthart, Maarten de Rijke, and Claes H. de Vreese. 2014.
“Teaching the Computer to Code Frames in News: Comparing Two Supervised Machine
Learning Approaches to Frame Analysis.” Communication Methods and Measures 8 (3):
190–206.
Cook, Timothy E. 2005. Governing with the News. The News Media as a Political Institution. Chicago,
IL: University of Chicago Press.
Duch Safety Board. 2015. Earthquake Risks in Groningen. Den Haag. Accessed october, 2015.
https://www.onderzoeksraad.nl/uploads/phase-docs/843/33ef77ab629erapport-
gaswinning-groningen-nl-interactief.pdf.
Eberl, J.-M., M. Wagner, and Hajo. G. Boomgaarden. 2016. “Are Perceptions of Candidate Traits
Shaped by the Media? The Effects of Three Types of Media Bias.” The International
Journal of Press/Politics 22 (1): 111–132.
Entman, Robert M. 1993. “Framing: Toward Clarification of a Fractured Paradigm.” Journal of Com-
munication 43 (4): 51–58.
Entman, Robert M. 2007. “Framing Bias: Media in the Distribution of Power.” Journal of Communi-
cation 57 (1): 163–173.
20 ALETTE EVA OPPERHUIZEN ET AL.

Esser, Frank. 1999. “‘Tabloidization’ of News.” European Journal of Communication 14 (3): 291–324.
Esser, Frank, and Jesper Strömbäck, eds. 2014. Mediatization of Politics: Understanding the Trans-
formation of Western Democracies. Basingstoke: Palgrave Macmillan.
Flew, Terry, and Adam Swift. 2015. “Engaging, Persuading, and Entertaining Citizens.” The Inter-
national Journal of Press/Politics 20 (1): 108–128.
Grabe, Maria Elizabeth, Shuhua Zhou, and Brooke Barnett. 2011. “Explicating Sensationalism in
Television News: Content and the Bells and Whistles of Form.” Journal of Broadcasting &
Electronic Media 45 (4): 635–655.
Hjarvard, Stig. 2013. The Mediatization of Culture and Society. Oxon: Routledge.
Johnson, Branden B., and Vincent T. Covello, eds. 2012. “The Social and Cultural Construction of
Risk: Essays on Risk Selection and Perception.” Science & Business Media 3: 147–175.
Springer.
Kasperson, Roger E, Ortwin Renn, Paul Slovic, Halina S. Brown, Jacque Emel, Robert Goble, Jeanne
X. Kasperson, and Samueal Ratrick. 1988. “The Social Amplification of Risk: A Conceptual
Framework.” Risk Analysis 8 (1988): 177–187.
Klijn, Erik-Hans, and Joop Koppenjan. 2016. Governance Networks in the Public Sector.
KNMI. 1993. Eindrapport multidisciplinair onderzoek naar de relatie tussen gaswinning en aardbe-
vingen. De Bilt: KNMI.
Korthagen, Iris. 2015. Media Logic Versus the Logic of Network Governance.
Landerer, Nino. 2013. “Rethinking the Logics: A Conceptual Framework for the Mediatization of
Politics.” Communication Theory 23 (3): 239–258.
Lazer, David, Alex S. Pentland, Lada Adamic, Sinan Aral, Albert L. Barabasi, Devon Brewer, Nicho-
las Christakis, et al. 2009. “Life in the Network: The Coming Age of Computational Social
Science.” Science 323 (5915): 721–723.
Matthes, Jörg. 2009. “What’s in a Frame? A Content Analysis of Media Framing Studies in the
World’s Leading Communication Journals, 1990–2005.” Journalism & Mass Communication
Quarterly 86 (2): 349–367.
Matthes, Jörg, and Christian Schemer. 2012. “Diachronic Framing Effects in Competitive Opinion
Environments.” Political Communication 29 (3): 319–339.
Mazzoleni, Gianpietro, and Winfried Schulz. 1999. “‘Mediatization’ of Politics: A Challenge for
Democracy?” Political Communication 16 (3): 247–261.
Mcnair, Brian. 2009. An Introduction to Political Communication. New York: Taylor & Francis.
Mott, Frank. 1962. American Journalism : A History, 1690–1960. 3d ed. New York: Macmillan.
Norris, Pippa, and John F Kennedy. 2001. “A Virtuous Circle? The Impact of Political Communi-
cations in Post-Industrial Democracies.” In Challenges to Democracy, 100–117. Houndmills,
Basingstoke: Palgrave Macmillan.
Patterson, Thomas E. 2000. “Doing Well and Doing Good: How Soft News and Critical Journalism
Are Shrinking the News Audience and Wakening Democracy—And What News Outlets
Can Do about It.” (December).
Renn, Ortwin. 1992. “Risk Communication: Towards a Rational Discourse with the Public.” Journal
of Hazardous Materials 29 (3): 465–519.
Scott, W. Richard. 1995. Institutions and Organizations. Ideas, Interests and Identities. London: Sage.
Sebastiani, Fabrizio. 2002. “Machine Learning in Automated Text Categorization.” ACM Comput-
ing Surveys 34 (1): 1–47.
Semetko, Holli a, and Patti M. Valkenburg. 2000. “Framing European Politics: A Content Analysis
of Press and Television News.” Journal of Communication 50 (September 1998): 93–109.
Slovic, Paul. 2000. The Perception of Risk. London: Routledge.
 FRAMING A CONFLICT! 21

De Smedt, Tom, and Walter Daelemans. 2012. “Pattern for Python.” Journal of Machine Learning
Research 13: 2063–2067.
Soumerai, Stephen. B., Dennis. Ross-Degnan, and Jessica Spira Kahn. 1992. “Effects of Professional
and Media Warnings About the Association Between Aspirin Use in Children and Reye’s
Syndrome.” The Milbank Quarterly 70: 155–182.
Stewart, Iain. S., and Deirdre Lewis. 2017. “Communicating Contested Geoscience to the Public:
Moving from ‘Matters of Fact’ to ‘Matters of Concern’.” Earth-Science Reviews 174: 122–133.
Strömbäck, Jesper, and Frank Esser. 2014. “Introduction.” Journalism Studies 15 (3): 243–255.
Uribe, Rodrigo, and Barrie Gunter. 2007. “Are ‘Sensational’ News Stories More Likely to Trigger
Viewers’ Emotions Than Non-Sensational News Stories?” European Journal of Communi-
cation 22 (2): 207–228.
Van Aelst, Peter, Gunnar Thesen, Stefaan Walgrave, and Rens Vliegenthart. 2014. “Mediatization
and Political Agenda-Setting: Changing Issue Priorities?” In Mediatization of Politics, 200–
220. London: Palgrave Macmillan UK.
Van den Bosch, Antal, Bertjan Busser, Sander Canisius, and Walter Daelemans. 2007. “An Efficient
Memory-based Morphosyntactic Tagger and Parser for Dutch.” Selected Papers of CLIN
2007: 99–114.
Vliegenthart, Rens, Hajo G Boomgaarden, and Jelle W Boumans. 2011. “Changes in Political News
Coverage: Personalization, Conflict and Negativity in British and Dutch Newspapers.” In
Political Communication in Postmodern Democracy, 92–110. London: Routledge.
Wahlberg, Anders AF, and Lennart Sjoberg. 2000. “Risk Perception and the Media.” Journal of Risk
Research 3 (1): 31–50.

Alette Eva Opperhuizen (author to whom correspondence should be addressed),

Department of Social and Behavioural Sciences, Erasmus University Rotterdam, The
Netherlands. E-mail: [email protected]
Kim Schouten, Erasmus Studio, Erasmus University Rotterdam, The Netherlands. E-mail:
[email protected]
Erik Hans Klijn, Department of Social and Behavioural Sciences, Erasmus University Rot-
terdam, The Netherlands. E-mail: [email protected]