A Critical Review of Content Analysis
A Critical Review of Content Analysis
A Critical Review of Content Analysis
The complexity and melding of modern day use of language, communication and
technology has generated an urgency in the many fields of research to find ways to sort
out the background noise, from the essence of why and how language or communication
may have changed technology, or how technology may have changed language or
communication. This background noise can be incoherent and loud, and have produced
assumptions about who has power over what. Assumptions taken on face value can be the
instruments of domination and liberation -
1
BROWN, R. M. (1988) Starting from scratch : a different kind of writer's manual, Toronto; New York,
Bantam Books. Rita Mae Brown is an American who alludes to Foucault in her feminist writings and
political support for gay, lesbian and transgender issues.
2
Discourse on rules of engagement in cyberspace, handbooks on online communities for example
“John Coate’s Principles of Cyberspace Innkeeping” AGRE, P. E. (1997) Reinventing technology,
rediscovering community : critical explorations of computing as a social practice, Greenwich, Conn.
[u.a., Ablex Publ., p. 188, Appendix A
1
…language is power, life and the instrument of culture, the instrument of
domination and liberation. Angela Carter in Notes from the Frontline 3
The objective of this paper is twofold. The first is to examine two research methods,
namely content analysis and critical discourse analysis, as representations of distinct and
opposing approaches. The second, and the heart of this paper, is to develop a critique of
content analysis using critical discourse analysis as a point of reference for doing so. To
start with, this paper will provide a brief historical overview of the two research methods.
From the historical review, the aim is to produce useful definitions of the two research
methods. The main thrust of this essay is the critique of content analysis using the
framework known as critical linguistics study (CLS) or critical discourse analysis (CDA).
Finally, we aim to arrive at some broad conclusions on the research methods, preferred
by this paper and the reasons behind these conclusions.
The systematic analysis of text can be traced as far back to the 17th Century, but the actual
use of the term content analysis did not appear in English until 1941 (Krippendorff, 2005,
p. 3). Max Weber proposed a large-scale content analysis of the press in 1911, but this
never took off; that Weber advocated the use of content analysis to understand the mass
media is relatively unknown (Krippendorff, 2005, p. 4). It is no coincidence that the term
content analysis is closely associated with the rise of mass media and communications.
In mass media, the audience was treated as a passive receptor, and the media was seen as
a set of all powerful tools for ‘circulating effective symbols’ akin to a hypodermic needle
with its direct, undifferentiated impact on atomized individuals (Mattelart and Mattelart,
1999, p. 26). This was described by Sigmund Freud as the “tyranny of suggestion”, as
the willingness to give up individuality for the influence of suggestion by others in the
pursuit of “…being in harmony with them rather in opposition to them…” (Mattelart and
Mattelart, 1999, p. 14).
3
Cited in BERTENS, H. & NATOLI, J. (2002) Postmodernism : the key figures, Oxford, Blackwell. Angela
Carter is an English novelist known for feminist, magical realism and science fiction works.
2
The oft-cited definition from Kaplan and Goldsen in 1943 provides an insight as to why
content analysis resonated in an era of mass media:
The choice words of quantitative, data, hypotheses and classification denotes a scientific
and objective-based approach inspired by the natural sciences, an interpretation which is
reinforced by Bernard Berelson:
Language is legislation, speech is its code. We do not see the power which is in speech
because we forget that all speech is a classification, and that all classifications are
oppressive (Barthes and Sontag, 1983).
The use of the term discourse dates back to the 14th Century, to the Middle English
discours and from Medieval Latin discursus4. Its basic meaning “refers to a speech or to a
text, that is to an instance of language use spoken or written” (Danesi and Rocci, 2009, p.
55). Discourse is an area of linguistics not limited by the total sum of its linguistic sub-
units or its sentence5 and it is a focus on “the particular way in which language is used in
a certain social group, community, institution, social class, ethnic group, sub-culture,
ideology, generation, etc” (Danesi and Rocci, 2009, p. 55).
4
This definition of “discourse” was accessed from the Merriam Webster Online Dictionary on 23 Dec
2009 at http://www.merriam-webster.com/dictionary/discourse
5
Merriam Webster Online Dictionary definition - 3 a : formal and orderly and usually extended expression of
thought on a subject b : connected speech or writing c : a linguistic unit (as a conversation or a story) larger than a sentence
3
In its secondary meaning, doing discourse has come to mean the analysis of speech and
text (used in the broadest terms) to “reveal the values, practices and social rules which
account for these particular ways of using language to produce texts and speech” (Danesi
and Rocci, 2009, p. 55). This secondary meaning of discourse, originally limited to the
study of linguistics, has become popular in the social sciences due to Norman Fairclough
and Michel Foucault. In studying television culture, John Fiske’s approach with speech
and non-speech, or textual and non-textual elements “so that one can talk of discourse of
the camera or of lighting” (Fiske, 1987, p. 14) smacks of this secondary discourse.
Because of the fact that any discourse is not the sole creation of an individual, and it is a
product of a social context or construction, there are preexisting meanings and uses of
discourse in practice that is worthy of closer study since “we do not speak our discourse
but our discourse speaks us” (Fiske, 1987, p. 15). Therefore, discourse is the interaction
of text and context (Reynolds, 1996, p. 4) and the aim of discourse analysis is to question
the use of language in a given time and place to construct meaning (Gee and NetLibrary,
1999, p. 93 and p. 110) and to make sense of the context or text, sub-text or non-text.
4
An important part of discourse analysis is to reveal the underlying assumptions, and as a
result, discourse methods are “unavoidably reflexive because the strong social
constructivist epistemology that forms its foundation applies equally to the work of
academic researchers (Phillips and Hardy, 2002)”. Discourse analysts are themselves
part of the reality being constructed using criteria, frameworks and themes can create
certain assumptions which affect meaning and how we make sense of the discourse.
Language is political. That’s why you and me, my Brother and Sister, that’s why we
supposed to choke our natural self into the weird, lying, barbarous, unreal, white speech
and writing habits that the schools lay down like holy law. Because, in other words, the
powerful don’t play; they mean to keep that power, and those who are the powerless (you
and me) better shape up – mimic/ape/suck – in the very image of the powerful, or the
powerful will destroy you – you and our children. (Jordan, 1973)6
June Jordan sums up the essence of critical discourse analysis. It is easy to understand
how critical discourse analysis or CDA, has become a stand-alone research method. It
represents the voice of the unheard, unspoken and the repressed. What is meant by
critical in critical discourse analysis? It is critical because it analyses what is ‘hidden’ as
opposed to the obvious in discourse. By focusing on underlying structural relationships
of dominance, discrimination, power and control (Wodak, 2005, p. 204), CDA is able to
change the status quo by exposing abuse of power, increasing transparency and
empowering the powerless with a voice to remedy social wrongs. In effect, CDA is “an
analysis of power effects, of the outcome of power, of what power does to people,
groups, and societies, and of how this impact comes about” (Blommaert, 2005, p. 1). It is
not a reaction to power alone, but rather the focus is the role of language as part of the
machinery of power and the way this may create inequality in society:
6
June Millicent Jordan (July 9, 1936 - June 14, 2002) was a Caribbean American poet, novelist,
journalist, biographer, dramatist, teacher and committed activist in the construction of race, gender,
sexuality, politics, war, violence, and human rights. She is widely regarded as one of the most
significant and prolific Black, bisexual writers of the twentieth century.
5
Power belongs to those who can determine the use, abuse, the rejection,
definition/re-definition of the words – the messages – we must try and send each
other (Jordan, 1973).
Its most influential voice is Norman Fairclough and his landmark publication “Language
and Power” (Fairclough, 2001). As opposed to social construction, CDA is “socially
constitutive as well as socially conditioned” (Blommaert, 2005, p. 25) due to the fact that
in a modern contemporary society, discourse or the language of use is an increasingly
important instrument of power. Power is central to CDA.
7
Citing Habermas (1967) Erkenninis and Interesse (Knowledge and Interest), p. 259
6
Content analysis is a quantitative method to analyze the frequency, occurrence or
recurrence of selected words, phrases or specified topics in large bodies of text, from
written documents, transcripts of films, videos, speeches and other types of written
communication. It requires an organized and structured approach, applying a coding
system to identify common and repeatable words, phrases or themes. For example in
political science, you can count the frequency of certain words or phrases that appear in a
political speech (e.g. the use of the word “change” in Obama’s election campaign).
McNabb, in his analysis of research methods in political science, argues that this creates
two separate but related issues. The structured approach tends to be inherently inflexible
and lacks creativity within the method, and “…the contextual meaning is often lost or, at
the least, made problematic”. Correspondingly, any implied meanings in a text cannot be
captured since the interpretive (hermeneutic) aspect of the analysis is lost in the
systematic approach (McNabb, 2004, p. 470). McNabb suggests that content analysis
works best where the communication being analyzed is “clear, straightforward, obvious
and simple” and does not work well on a text with “subtle and/or intricate meanings”.
This is because content analysis is helpful to “describe attributes of messages, without
reference to the intentions of the message sender or effect of the message on the receiver”
(McNabb, 2004, p. 470) (Holsti, 1969, p. 27).
There are various assumptions about content analysis - quantifying content somehow
reveals the objective of the communication; the character of the communicator is
somehow shown by objective representation of the content; you can somehow reveal the
effects of the content on the recipients. These are widely recognized problems in the
discipline – firstly, it is rooted in the belief that scientific methods would provide an
objective analysis, secondly, that the simple method of “counting and measuring fact”
ignoring “whether facts indeed had any independent existence apart from the social
situation which structured them and the class perceptions which defined them…”(Harris
and Lee, 1986, p. 202).
7
It is no surprise for social scientists to be critical of content analysis. In adverting where
content analysis is widely used, it has also come under scrutiny for the blind faith in
linking frequency of text and the interest or intent of the producer with the response of
the audience (Dyer, 1988). Thurs, content analysis is limited to “large-scale, objective
and systematic surveys of manifest content using the counting of content as the basis for
later interpretation” (Dyer, 1988, p. 108). Dyer has strong reservations that “the counting
or quantification of isolated elements in a piece of content [can] tell us everything about
how meaning is produced in a text nor how the audience understand…” (Dyer, 1988, p.
111). Advertisements are part of a cultural communication, and “their significance cannot
always be reduced to their manifest, objective content” (Dyer, 1988, p. 111). This
argument is based on the claim that content analysis is not linked to an explanatory
theoretical framework (Strinati, 1995). In cultural studies, this means “…it does not have
an explanation of the relationship between the popular cultural text being analysed and
the social structural context – including the underlying power relations – in which it can
be located” (Strinati, 1995, p. 175). For example, studies of sexist content does not
contribute to the understanding of why and how social structures produce such content
(Strinati, 1995)8. Feminists such as Helen Baehr are critical of content analysis producing
studies of men and women “represented performing particular roles rather than asking
questions about how and why representations occur”(Strinati, 1995, p. 175).
The feminist critique of content analysis has therefore argued that it only serves to
provide a snapshot and static picture of social and gender relations, and this does not
overweigh the advantages of being able to mobilize content analysis to support through
quantifying the prevalence of masculine points of view in popular culture. For Baehr,
content analysis as a method influences the questions asked and the conclusions drawn to
work against the interests of feminism (Strinati, 1995). So much for the objectivity of
content analysis, and the feminist critique supports the argument of this paper that content
analysis can be and has been mired in its own politics and its own research agenda.
8
Citing Baehr H. (1981) “The impact of feminism on media studies – just another commercial break?”
appearing in SPENDER, D. (1981) Men's studies modified : the impact of feminism on the academic
disciplines, Oxford; New York, Pergamon Press.
8
Moving to another discipline, computer scientists rank content analysis as “the least
appealing of the several research methodologies available to the communication
researcher”, because of “the limits of the method itself and the drudgery required to carry
it out” (Stevenson, 2001, West, 2001). In the age of computer-aided content analysis,
these concerns may be somewhat alleviated but it is replaced by the enormous volume of
data retrieval and the fact that the researcher often does not see the original data. This
raises the possibility of invalid data or inaccurate or incomplete data being included in the
samples taken or not taken as the case may be. Computerization may improve efficiency
but it does not overcome the inherent limitations of content analysis as a research
methodology (Stevenson, 2001). West concurs and extrapolates on this sentiment -
“The tendency of many content analytic studies to count, and then to report the
frequency of various constructs with the suggestion that they have some sort of
obvious meaning, has been decried by numerous commentators. There have been
too many studies in which some depiction of some entity is described as
appearing on television so many times, and that number then described as
indicating some sort of bias, or the lack thereof, without any serious effort to
justify the contention through rigorous application of prior theoretical arguments.
Often, the critique of content analysis as mindless empiricism is closer to the truth
than we might like to admit.” (West, 2001, p. 80)
To avoid computer-aided content analysis from being relegated to a glorified form of data
mining highlights the inherent problem of content analysis, where the researcher has to be
careful in laying how assumptions are made, the basis for making these assumptions and
the derivation of meaning from the text(s) as a whole and not the sum of the parts.
Content analysis is popular with organizational researchers and audiences due to the
positivist research framework and computer-aided analysis of large ‘corpus’ has become
more feasible and easy to process as a result. This is because “it need not require re-
theorizing of either discourse data or of discourse analysis”(Grant et al., 2004, p. 225).
9
In public relations, content analysis is widely employed and its practical usefulness is
assumed within and outside the industry. The researchers, analysts and practitioners are
however careful in the use of this method since it can be error prone and not an easy to
employ correctly.
The primary concerns are reliability, validity and inference. For example, it is difficult to
achieve a high degree of reliability with negative political advertising, since “determining
whether an ad is negative or positive often involves making judgment calls” (Austin and
Pinkleton, 2006, p. 189). Validity is another characteristic of content analysis, and it is
usually determined by examining whether the researcher is measuring what is supposed
to be measured, and the degree to which the researcher is successful in doing so. This was
discussed above (Stevenson, 2001, West, 2001). Public relations practitioners are acutely
aware of the concerns regarding inference and the temptation to make use of content
studies to draw conclusions and interpretations of wider application than possible from
the content itself (Austin and Pinkleton, 2006, p. 189). This negates the main aim of
content analysis “to provide a precise description of communication content” and the
problem arises that “when a company equates content analysis with public opinion
analysis…public opinion resides in the perceptions of the public, not in the content of
media messages” (Austin and Pinkleton, 2006, p. 189).
10
Critical discourse analysis takes this approach one step further by placing messages or
texts in the context of changes in a broader discourse of politics, economics or in
contemporary society. Fairclough uses a broad range of methods related to the subject
matter of discourse – using interviews, scrutinizing pamphlets and advertisements,
studying the mass media and reviewing policy documents – putting them in
contemporary society, for example the discourse of New Labour (Fairclough, 2001) and
the language of new capitalism (Fairclough, 2000, Chiapello and Fairclough, 2002).
Data collection is also used in critical discourse analysis, as it is in content analysis, but
the starting point is different since it is based on the linguistic concept of a “corpus”,
being a carefully selected sample of discourse which is usually extensive and subject to
detailed linguistic analysis – but focused on moments of crisis to highlight power
struggles or points of change (Grant et al., 2004). Since content analysis is about
revealing its “manifest content”, basically, the determines the presence of certain words
and contents in text (Grant et al., 2004, p. 225), the approach of critical discourse analysis
is in sharp contrast to content analysis in terms of revealing the complex relationships
between discourse, language and society:
Faircloughs work offers not only a menu of possible methods but an insistence
that the researcher must be aware of the complex relationships between language
and social processes in collecting and analyzing discourse as data (Grant et al.,
2004, p. 227).
The key differentiator between critical discourse analysis and content analysis is the
“context-sensitive” approach, being a focus on language in use for discourse analysis,
which takes into consideration historical and social factors that go beyond the text (Grant
et al., 2004, p. 10). These factors “are adjudged to influence and shape the way a text is
produced, disseminated and consumed. Examples of these context-sensitive approaches
include studies that draw on pragmatics, socio-linguistics, institutional dialogue,
systemics and critical discourse analysis” (Grant et al., 2004).
11
Critical discourse analysis is the most influential context-sensitive approach to the study
of organizational discourse. It reveals the role of language in relation to ideology, power
and socio-cultural change. It is multi-disciplinary, combining several research methods
that examines –the content, structure and meaning of text, the discursive elements used to
communicate meanings and beliefs – and it considers the social content where discursive
events takes place (Grant et al., 2004, p. 11). This three-dimension theoretical framework
of text, discursive and social practice is sorely missing in content analysis research.
Furthermore, content analysis misses the big picture, by its focus on the sum of the parts.
Critical discourse analysis can rely on intertexuality so that text are not treated as separate
and disjointed discursive units, rather it is “a link in a chain of texts, reacting to, drawing
in and transforming other texts” (Dijk, 1997)9. Intertexuality acts as the bridge and the
value add is “it mediates the connection between language and social context, and
facilitates more satisfactory bridging the gap between texts and contexts” (Fairclough,
1994, p. 189). While content analysis is the simple examination of verbal and written
interaction, critical discourse analysis enables us to “appreciate the importance of ‘who
uses language, how, why and when’(Grant et al., 2004, citing Dijk 1997, p. 2)”. Critical
discourse analysis is therefore able to address not only who uses language but how, when
and why, and the particular ends which are “critical” is the inter-relations between
language and power in society. Fairclough argues that in a contemporary society,
ideology and language are used to manufacture consent or at least, it is used to secure the
acquiescence to the exercise of power:
9
Taken from article called “Critical Discourse Analysis” by Norman Fairclough and Ruth Wodak
1997, p. 262 featured in T. Van Dijk book.
12
It is therefore critical to understand how language maintains or changes power relations,
and the way to do this is to analyze the language to reveal the structures and processes
“hidden” by the language, not only to raise our own consciousness but also to be able to
take action through resistance or to enable changes.
Fairclough was also critical of the tendency to focus on observation and presentation of
facts without regard to the social conditions that created the conditions in the first place,
or may in fact change the original conditions. The research results may intentionally or
unintentionally lend legitimacy to the facts asserted and their underlying power relations
(Fairclough, 2001, p. 7).
Conclusion
Language has power to dominate and liberate. This power remains hidden from us,
through the social constructions of modern contemporary life. The rise of mass media and
communications, and the convergence of language, communication and technology make
it more difficult to discern and decipher hidden texts, messages or debates.
13
Content analysis and critical discourse analysis are research methods that started with
different historical origins, and yet serve different ends, producing separate outcomes.
Content analysis is historically rooted in the natural sciences, discourse analysis in
linguistics. Critical discourse analysis started as a secondary limb of discourse analysis,
and taking full advantage of elements such as being interdisciplinary and social
construction epistemology, it has been transformed into stand-alone research
methodology by Norman Fairclough.
The purpose of critical discourse analysis is to make visible the effects of language to
hide the ideological domination, to make transparent any structures that are legitimizing
power relations and to change the status quo through critical observation and conveyance
of knowledge. Content analysis is a quantitative method to analyze the frequency,
occurrence or recurrent of selected words, phrases or topics in large bodies of text.
To summarize, the critique developed in this paper makes the argument that content
analysis - is blind to the implied meaning in any text; is not sensitive to cultural texts not
reducible to manifest content; is not a context-sensitive methodology; is apt to present a
static picture of social and gender relations; is not objective since it is mired in its own
politics and research agenda; is not reliable when analyzing content such as negative
political advertising; is unable to generate a high degree of validity without narrow and
well-defined assumptions; is problematic when it yields to the temptation of drawing
conclusions of a wider application than possible from the content itself.
14
In conclusion, this paper has established that content analysis has limited and specific
application as a research method when compared to the more flexible, creative and
theoretical based framework of critical discourse analysis.
15
KRIPPENDORFF, K. (2005) Content analysis : an introduction to its methodology,
Thousand Oaks [u.a.], SAGE.
MATTELART, A. & MATTELART, M. (1999) Theories of communication : a short
introduction, London [u.a.], Sage Publ.
MCNABB, D. E. (2004) Research methods for political science : quantitative and
qualitative methods, Armonk, NY [u.a.], M.E. Sharpe.
PHILLIPS, N. & HARDY, C. (2002) Discourse analysis : investigating processes of social
construction, Thousand Oaks, Calif. [u.a.], Sage.
REYNOLDS, M. (1996) Guy Cook: The Discourse of Advertising. Applied linguistics.,
17, 127.
SPENDER, D. (1981) Men's studies modified : the impact of feminism on the academic
disciplines, Oxford; New York, Pergamon Press.
STEVENSON, R. L. (2001) In Praise of Dumb Clerks: Computer-Assisted Content
Analysis. Progress in communication sciences., 3-12.
STRINATI, D. (1995) An introduction to theories of popular culture, London; New
York, Routledge.
WEST, M. D. (2001) Theory, method, and practice in computer content analysis,
Westport, CT, Ablex Pub.
WODAK, R. (2005) A new agenda in (critical) discourse analysis theory, methodology
and interdisciplinarity, Amsterdam [u.a., Benjamins.
WODAK, R. & MEYER, M. (2009) Methods of critical discourse analysis, Los Angeles;
London, SAGE.
16