Sage Research Methods

Content Analysis: An Introduction to Its Methodology

For the most optimal reading experience we recommend using our website.
https://methods.sagepub.com/book/mono/content-analysis-4e/toc

Author: Klaus Krippendorff

Pub. Date: 2022
Product: Sage Research Methods
DOI: https://doi.org/10.4135/9781071878781
Methods: Content analysis, Research questions, Sampling
Keywords: software, population, coincidence, newspapers, mass media, attitudes, language
Disciplines: Communication and Media Studies, Marketing
Access Date: April 21, 2025
Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Publisher: SAGE Publications, Inc.

City: Thousand Oaks
Online ISBN: 9781071878781

© 2022 SAGE Publications, Inc. All Rights Reserved.

Page 2 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Front Matter

• Copyright
• Preface to the Fourth Edition

Chapters

• Introduction
• Part I | • CONCEPTUALIZING CONTENT ANALYSIS
◦ • History
◦ • Conceptual Foundation
◦ • Uses and Inferences
• Part II | • COMPONENTS OF CONTENT ANALYSIS
◦ • The Logic of Content Analysis Designs
◦ • Unitizing
◦ • Sampling
◦ • Recording/Coding
◦ • Data Languages
◦ • Analytical Constructs
• Part III | • ANALYTICAL PATHS AND EVALUATIVE TECHNIQUES
◦ • Analytical/Representational Techniques
◦ • Computer Aids
◦ • Reliability
◦ • Validity
◦ • A Practical Guide

Back Matter

• Glossary
• References
• About the Author

Page 3 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Copyright

Page 4 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

FOR INFORMATION:

SAGE Publications, Inc.

2455 Teller Road

Thousand Oaks, California 91320

E-mail: [email protected]

SAGE Publications Ltd.

1 Oliver’s Yard

55 City Road

London EC1Y 1SP

United Kingdom

SAGE Publications India Pvt. Ltd.

B 1/I 1 Mohan Cooperative Industrial Area

Mathura Road, New Delhi 110 044

India

SAGE Publications Asia-Pacific Pte. Ltd.

3 Church Street

#10-04 Samsung Hub

Singapore 049483

Copyright © 2019 by SAGE Publications, Inc.

All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic
or mechanical, including photocopying, recording, or by any information storage and retrieval system, without
permission in writing from the publisher.

Page 5 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Printed in the United States of America

Library of Congress Cataloging-in-Publication Data

Names: Krippendorff, Klaus, author.

Title: Content analysis : an introduction to its methodology / Klaus Krippendorff, University of Pennsylvania.

Description: Fourth Edition. | Los Angeles : SAGE, [2018] | Includes bibliographical references and index.

Identifiers: LCCN 2017050739 | ISBN 9781506395661 (Paperback : acid-free paper)

Subjects: LCSH: Content analysis (Communication)

Classification: LCC P93 .K74 2018 | DDC 302.2—dc23 LC record available at https://lccn.loc.gov/
2017050739

This book is printed on acid-free paper.

Acquisitions Editor: Terri Accomazzo

Editorial Assistant: Erik Helton

Production Editor: Andrew Olson

Copy Editor: Michelle Ponce

Typesetter: C&M Digitals (P) Ltd.

Proofreader: Ellen Howard

Indexer: Jeanne Busemeyer

Cover Designer: Anupama Krishnan

Marketing Manager: Liz Thornton

Page 6 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Preface to the Fourth Edition

Content analysis is one of the most important research techniques in the social sciences. It acknowledges
that society is enacted in talk, texts, and other modalities of communication and that understanding social
phenomena cannot be achieved without understanding how language operates in the social world. Content
analysts inquire into social phenomena by treating data not as physical events but as communications that
are created and disseminated to be seen, read, interpreted, enacted, and reflected upon according to the
meanings they have for their recipients. Interpreting communications as texts in the contexts of their social
uses distinguishes content analysis from other empirical methods of inquiry.

Investigative methods in the natural sciences, by contrast, rarely are concerned with meanings, contents,
intentions, references, communications, and what they do. Natural scientists may have good reasons to
presume that the objects of their attention are not aware of how they are being observed and do not
speak a language to articulate how they want to appear, but they also rarely reflect on how their own
writings shape their conception of nature and what it allows them to do or prevents them from studying.
The conception of nature as preexisting and independent of scientific attention is convenient for it relieves
natural scientists from being held accountability for the realities that their theories construct, but it also
prevents them from contributing to understanding social phenomena that arise in how they are described
and enacted. Where social scientists adopt natural scientific methods of inquiry, it is the epistemology
inscribed in these methods that prevents them from addressing what matters in the social life of people, what
motivated and inspires them, and how they enact what they know. Certainly, content analysis is not the only
research method that takes meanings seriously, but it is a method that has the additional advantage of being
unobtrusive—not affecting the sources or receivers of what is being analyzed—able to process large volumes
of data—providing insights that exceed the comprehension of ordinary readers of textual matter—and its
findings can mediate between individuals and challenge the operation of institutions.

In the first edition of Content Analysis, published in 1980, I suggested that content analysis was at a
crossroads. Content analysts at that time had a choice: They could continue their shallow counting game,
motivated by a journalistic fascination with numbers and a narrow conception of science in which quantitative
measurements provided the only evidence that counted (Lasswell, 1949/1965b), or they could redirect their
attention to social phenomena that are both generated by and constituted in texts and images and, hence,
needed to be understood through their written and pictorial manifestations. The latter motivated the first
edition, and I am pleased to say that the epistemology and logic I developed therein have survived critical
assessments and the challenges posed by the radical transformations that the information revolution and

Page 7 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

social media have introduced into contemporary social practices. The widespread use of electronic, and
hence computer-readable, texts concerning virtually everything that matters to us as social beings has moved
content analysis, particularly computer-aided text analysis, into the center of how society examines itself.

In the 1970s, content analysis was a research method that had entered the psychological and social sciences
but was used mainly by journalists and communication researchers. During that time, the amount of human
effort required to collect, transcribe, and code textual data made content analysis a time-consuming, labor-
intensive, and often costly effort that limited the scope of what it could address. However, it became an
efficient alternative to public opinion research and a method of tracking markets, political influence, and
emerging ideas. It was used to detect biases in reporting of facts, settle legal disputes, understand how
institutions establish themselves in the texts they produce, and explore the mind of individuals through what
they said or wrote.

The first edition of Content Analysis reported numerous analytical innovations but barely met the challenges
of the new era of electronic communication. It taught a generation of social scientists about the methodology
of this research technique. It soon became the leading text on the subject and was translated into Italian,
Persian, Japanese, Spanish, Indonesian, and Hungarian.

Although the outline of the second, 2004 edition of Content Analysis remained essentially unchanged from
that of the first, the book clarified numerous methodological issues and responded to the technique’s latest
challenges. All chapters were substantially rewritten, addressing developments that had taken place since
1980, especially concerning computer-aided text analysis. It added a practical guide that incorporated input
from my students, colleagues, and own practical and consulting experiences in various academic and
commercial research projects. In 2004, it received the International Communication Association’s Fellows
Book Award for its lasting contribution to the advancement of communication studies.

To broaden the range of examples and recommend solutions to practical problems, Mary Angela Bock
and I coedited The Content Analysis Reader (2009). This reader was meant to be a companion to the
methodologically oriented Content Analysis. I have been using the Reader with considerable success, heard
from others about its added benefits, and recommend it to students, teachers, and practitioners of content
analysis.

The third, 2013 edition of Content Analysis continued the careful and critical examination of how texts and
images can support inferences about phenomena not accessible to independent observations and updated
several chapters, notably on computer aids. It also added a convenient glossary. In 2014, it appeared in its
Chinese translation. In 2017, my work on content analysis was honored by receiving the first methodological

Page 8 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

innovation award from the Mass Communication Division of the International Communication Association.

The outline of this fourth edition is kept essentially as it was before. It corrects a previously published error
in the formula for computing the reliability of unitizations. Its discovery by a group of French scholars is
just one example of the many fruitful collaborations that the book encouraged. It includes a section on
crowdsourcing and expands Krippendorff’s Alpha in several dimensions: for assessing the reliability of text
retrievals, crowdcoding, and of multi-valued coding of predefined units as well as of unitized textual continua.
And it points to software among many versions known to be correct.

I thank my students at the University of Pennsylvania’s Annenberg School for Communication for their interest
and for giving me useful feedback and my colleagues for presenting me with the challenging methodological
problems of their content analyses. I would also like to thank numerous readers of the earlier editions—both
students and practicing content analysts—for sharing their comments and recommendations and Sage
Publications for its continuing support of content analysis literature.

During the 38 years of publications of Content Analysis, its three previous editions have reached an enormous
audience. It has been widely adopted as a text in social science, humanities, and business curricula, served
researchers as a guide for the design and execution of large and small content analyses, and provided
epistemologically sound standards for justifying as well as critically evaluating content analysis findings. When
I travel to national and international conferences, I continue to be amazed and pleased to meet researchers
from all over the world who tell me how studying this text has helped them in their current inquiries. I hope
this fourth edition will withstand critical examinations and continue to advance scholarship based on texts,
images, and communications in all fields that acknowledge to operate within language.

—Klaus KrippendorffProfessor of Communication andGregory Bateson Professor Emeritus, forCybernetics,

Language, and CultureThe Annenberg School for Communication,University of Pennsylvania

Page 9 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Introduction

The term content analysis is about 70 years old. Webster’s Dictionary of the English Language included the
term in its 1961 edition, defining it as “analysis of the manifest and latent content of a body of communicated
material (as a book or film) through classification, tabulation, and evaluation of its key symbols and themes in
order to ascertain its meaning and probable effect.” The intellectual roots of content analysis, however, can
be traced far back in human history, to the beginning of the conscious use of symbols and voice, especially
writing. This conscious use, which replaced the magical use of language, has been shaped by the ancient
disciplines of philosophy, rhetoric, and cryptography. It has also spawned religious inquisitions and political
censorship on the part of ruling establishments. Today, symbolic phenomena are institutionalized in art, liter-
ature, education, and the mass media, including the Internet. Theoretical and analytical concerns are found
in such academic disciplines as anthropology, linguistics, social psychology, sociology of knowledge, and the
comparatively younger field of communication studies. Many practical pursuits have grown from these fields:
psychotherapy, advertising, politics, the arts, and so on. Virtually all disciplines within the whole spectrum of
the humanities and the social sciences, including those that seek to improve the political and social conditions
of life, are concerned with the functions and effects of symbols, meanings, and messages. In recent years, the
emergence of the information society has moved the minutiae of communication—texts, contexts, images,
interfaces, and, above all, information—into the very center of researchers’ attempts at self-understanding.

However ancient the roots of analyzing symbolic and textual matter might be, today’s content analysis is sig-
nificantly different, in aim and in method, from that of the past. Contemporary content analysis has three dis-
tinguishing characteristics.

First, content analysis is an empirically grounded method, exploratory in process, and predictive or inferential
in intent. Many of our current concepts relating to language are of Greek origin; for example, the words sign,
significance, symbol, and logic all have Greek roots. However, the ancient Greeks’ interest in language was
largely prescriptive and classificatory, not empirical. Aristotelian logic set the standards for clear expression,
and much of rhetorical theory was directed toward a normative conception of persuasive argumentation. Sci-
ence that explores rather than declares is a relatively recent accomplishment. Only a century ago, George
Boole and his contemporaries believed that the brain works according to (Boolean) logic and that human con-
duct is entirely rational. However, computers built on this logic turned out to be rather disappointing thinking
machines. Empirical research in psychology is replacing Aristotelian categories in favor of a “psycho-logic.”
And we no longer measure human communication against the ideal of transmitting information. Instead, we

Page 10 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

inquire into what happens to the relationships between people who converse with one another.

With new conceptualizations and an empirical orientation, contemporary content analysts join other re-
searchers in seeking valid knowledge or practical support for actions and critique. However, unlike re-
searchers who employ other empirical techniques, content analysts examine data, printed matter, images, or
sounds—texts—in order to understand what they mean to people, what they enable or prevent, and what the
information conveyed by them does. These are questions for which natural scientists have no answers and
for which their methods are generally insensitive.

Second, contemporary content analysis transcends traditional notions of symbols, contents, and intents. This
may be seen in the evolution of the concept of communication, in how the development of media technolo-
gies has shaped our attention to communication, and in the role of culture in assigning significance to what
is being analyzed. I would argue that in recent years our awareness of communication has undergone six
conceptual revolutions, as described below, and it probably is not stopping with these:

• The idea of messages: the early awareness not only that verbal discourse is portable when written
but also that writing has predictable effects. This awareness emerged in ancient Greece when mes-
sengers were used as the carriers of significance; history became documented; laws of the land
were laid down in writing; and written instructions built organizational structures, directed events, and
influenced (and possibly deceived) their receivers or the public. The concept of a message was a
precursor of the rhetorical exploration of language. Tropes, syllogisms, and meanings came to be
thought of as inherent qualities of speeches, letters, or documents. But a message is the metaphor-
ical container of all these, a “container of content,” a vehicle for shipping meanings from one place
to another—for example, when we now leave a message for someone on voice mail or say that a
message was meaningful (full of meanings) or meaningless (void of meanings).
• The idea of channels: the awareness of constraints that every medium imposes on human commu-
nication. This awareness came with the increased reliance on different media of communication and
served to explain their limitations: The alphabet limits what one can say in writing; the telephone con-
fines communication to sound; and a television station can air no more than what is transmittable
without interference from other stations, appealing to large audiences, and deemed profitable by
sponsors. The channel metaphor conjures images of canals and pipes with restricted capacities for
shipping messages (with their contents) of certain forms and volumes.
• The idea of communication: the awareness that it establishes relations between senders and re-
ceivers, enables the negotiation of complementary roles, constitutes social structures, and commu-
nities whose members have a sense of knowing each other. This awareness has ancient roots in
the conception of dialogue but has entered current awareness through the limitations experienced

Page 11 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

in the use of mass communication. By producing and disseminating identical messages—news and
entertainment—to everyone, the mass media promised to be an agent of sharing, of building publics,
ideally beyond borders. Modeled on the success of mass production and ever-expanding markets for
useful goods, the mass media also made us aware of where this one-way model failed: making peo-
ple aware of often amplifying social differences, limiting face-to-face conversations, curtailing public
deliberations, and addressing only commercially viable issues. In U.S. culture, mass-media technol-
ogy has become synonymous with progress, and the lack of communication is often blamed for most
interpersonal problems or national conflicts.
• The idea of systems: the awareness of global, dynamic, and technologically supported interdepen-
dencies. This idea emerged with the growth of communication networks—telephone nets, wire ser-
vices, mass-media systems, and most recently the Internet—transforming commerce, politics, and
interpersonal relationships, creating networks whose properties have largely defied theoretical ac-
counts. Unlike the one-way mass media, systems are marked by the interactivity and simultaneity of
parallel communication on a massive scale and with the potential of near universal participation.
• The idea of computation: the awareness of the increasing replacement of communicators by algo-
rithms. Messages may be generated by robots and addressed to computers, whether it involves the
use of automatic teller machines, online shopping, robo-calls, or computer-generated messages on
the Internet. This changes the conception of messages as understandable by senders and receivers
to what has computable consequences. Letter writing is being replaced by e-mailing, and interper-
sonal relations are shaped by the platforms that mediate them. Computation has expanded access
to textual matter—consider the amazing power of search engines, the individual ability to travel to
distant places virtually—but it precludes computationally unavailable experiences. Computation not
only has made social life more variable but also has introduced complexities that easily escape un-
derstanding.
• The idea that reality is discursively co-constructed: the recognition that we live together in language,
can never think alone (Sloman & Fernbach, 2017), and constitute our institutions as interdependent
realities through what we say to and do with each other. Wittgenstein’s (1958) notion of language
as consisting of numerous language games that coordinate actions in the world comes to mind. The
empirical fact that almost all innovations are the product of collaborations, that even scientific ac-
complishments arise and are vetted in discourse communities renders texts as actively supporting or
challenging social realities.

This rather sketchy history of communication suggests that researchers who are concerned with texts can no
longer treat them as symbolic or representational, nor can they limit themselves to questions about “who says
what, through which channels, to whom, and with which effects” (Lasswell, 1960). The popular and simplistic
notion of “content” has outlived its explanatory value (Krippendorff, 2017): content, the what of a communica-

Page 12 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

tion, something that authors are conceived of as entering into the texts they are producing, something that is
preserved in texts while shipped to remote receivers, who are expected to remove its content as intended and
henceforth share it with others through still other containers. This bizarre notion of message content renders
authors as authorities of what messages contain and to the conception of content analysts as experts, able
to objectively account for what messages really contain or are intended to convey, distinct from ordinary and
usually inaccurate readings by less qualified readers.

The absence of authors, especially historical ones, makes the use of this container metaphor nonverifiable.
Mass media messages tend to replace individual authors by institutional constructs, and their effects are
more important than what someone may have intended. Trust in them has become trust in the technology of
transmission. The mass media coordinate the lives of their audience members. Their practices challenge old
distinctions among channels of communication, obviating physical distances, and pushing capacities of the
human participants to their limits. This erodes the validity of traditional communication theories, all the while
enabling computer systems to thrive in this new environment. It is these computer systems that simulate and
coordinate parts of the very social processes that researchers wish to understand. This is a radically changing
world in which texts play distinctly new roles. Newspaper accounts, public opinion polls, corporate reports,
files in government agencies, credit information, bank transactions, and, above all, textual data archives of
social media—all are now linked into networks that can be analyzed from numerous positions. In effect, the
social systems that we conceived of as explaining society are now holographically retreating into our comput-
ers. This development calls for a redefinition of content analysis, one that aligns content—the presumed tar-
get of the research effort—with how contemporary society operates and understands itself through its texts.

With the container metaphor rendered useless, perhaps the term content analysis no longer fits the realities
of contemporary society. For better or for worse, I continue to use the term in this book, but I also plead with
readers to unflinchingly oppose the naive and misleading entailments of this still pervasive metaphor.

Third, contemporary content analysis has been forced to develop a methodology of its own, one that enables
researchers to plan, execute, communicate, reproduce, and critically evaluate their analyses whatever the
particular results. Content analysts have had to develop such a methodology for three reasons:

• Content analysts now face larger contexts. The shift in interest from small collections of printed mes-
sages to systems and then to electronic texts and images circulating in the environment of content
analysts is tied less to the nature of textual data than to the increasingly complex worlds that produce
and are sustained by these data. This shift calls for theories and conceptions that earlier content an-
alysts did not need. Although content analysts have frequently lamented the lack of general theories
that could justify their work, progress in implementing more specific or microlevel theories is encour-

Page 13 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

aging. This is especially true where content analysis has migrated through disciplines that were not
previously concerned with textual data, such as the cognitive sciences and artificial intelligence.
• Greater numbers of researchers need to collaborate in the pursuit of large-scale content analyses.
This observation is a correlate of the growing sample sizes of relevant texts, the analysis of which
easily exceeds what individual analysts can handle. It implies that content analysts must work to-
gether, in parallel, and as research teams. Teamwork, however, needs to be organized reliably. Both
the social problem of coordinating researchers and the methodological problem of assuring replica-
bility tend to be solved through the adoption of a language whose vocabulary enables researchers to
clarify the analytical procedures they use, negotiate the individual responsibilities of the participants,
assure agreement on the analytical categories, and evaluate the performance of team members. A
novel solution of the human problem of analyzing increasing volumes of textual data is to outsource
the coding of textual matter to crowds who can interpret data reliably. The methods to cope with this
practice are currently at their infancy. Developing them moves content analysis closer to other social
science methods such as survey research. Economy and speed are noticeable benefits of outsourc-
ing the coding of textual data, but reliability and depth are still issues to be improved upon.
• The large volumes of electronically available data call for computer aids. Such aids need to convert
large bodies of electronic text into more manageable representations that preserve the ability to an-
swer the research questions that content analysts seek to derive from available data. Currently, two
competing approaches are pursued to handle overwhelming volumes of data. One is to develop text
analysis software capable of computing complex statistics of the character strings of text or pixelized
images. However, exactly how such software transforms raw texts into abstract representations is of-
ten uncertain and the subject of exaggerated claims, for example, being content extractors or promis-
ing to mine meanings. The other approach is to develop less ambitious computer aids that merely re-
place the labor-intensive and recurrent parts of more complex content analyses. For example, quali-
tative data analysis software tends to allow recursive coding decisions to be automatically extended
to large documents. Software that could serve as aids to parts of content analytical steps requires
some transparency, enabling analysts to assess their appropriateness by tracing and evaluating the
transformations that texts are undergoing in the process.

To be clear, methodology is not a value in itself. The purpose of methodology is to enable researchers to
plan and examine critically the logic, composition, and protocols of research procedures; to evaluate the per-
formance of individual techniques; and to estimate the likelihood of particular research designs to contribute
to knowledge. All researchers must become proficient in defining the terms of their analysis and justify the
analytical steps taken to a skeptical friend or critical colleague. Methodology provides a language for talking
about the process of research, not about subject matters. In the history of scientific pursuits, the development

Page 14 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

of methodology has always been a major accomplishment. For example, for thousands of years, humans pre-
served history by retelling or chanting stories, for example, of the Iliad before Homer put it in writing. It was
only a century ago that the historian Leopold von Ranke gave the “document” the methodological status it
now has in the academic study of history. Similarly, scholars practiced “content analysis” well before Berelson
and Lazarsfeld (1948) undertook the first codification of this method. Although many critics claim that each
content analysis is unique, possibly by focusing mainly on its subject matter, I would argue that all content
analyses share a procedural logic that can be examined and is used to justify their procedures in terms of
socially acceptable criteria. What content analyses have in common form the substance of this book.

I disagree with the frequent contention that content analysis is “nothing more than what everyone does when
reading a newspaper, except on a larger scale.” Content analysis may have been that way, in its early, jour-
nalistic stage, and its methodology does not rule out such readings, but this narrow definition is no longer
sufficient today. As newspaper readers, we are perfectly justified in applying our individual worldviews to texts
and enacting our interest in what those texts mean to us; in fact, as citizens, we can hardly do otherwise. But
as content analysis researchers, we must do our best to explicate what we are doing and describe how we
derive our judgments, so that others—especially our critics—can replicate our results or build on them.

This book, then, introduces readers to ways of analyzing meaningful matter, texts, images, and voices—that
is, data whose physical manifestations are secondary to what they mean to particular populations of people.
The chapters are grouped into three main parts. Part I, “Conceptualizing Content Analysis,” begins with a
brief chapter on the history of content analysis. In Chapter 2, I develop a definition of content analysis that
distinguishes this technique from other methods of inquiry, and in Chapter 3, I present a discussion of some of
the ways in which content analysis has been applied. The chapters in Part II, “Components of Content Analy-
sis,” outline the procedures used in content analyses, beginning with their procedural logic and moving natu-
rally from “unitizing,” “sampling,” “recording/coding” in terms of formal “data languages,” and “analytical con-
structs.” The chapters in Part III, “Analytical Paths and Evaluative Techniques,” trace several paths through
content analysis protocols. In this part of the book, I discuss analytical constructs that enable researchers to
draw inferences from data, the use of computers and computational techniques, and the two principal criteria
used in evaluating content analyses: reliability and validity. In the final chapter, I provide a practical guide that
summarizes the foregoing discussion from a practitioner’s perspective.

Readers who have never undertaken a content analysis may want to begin by reading Chapter 1, on the
history of content analysis, and Chapter 3, on the uses of this technique, to get a sense for whether it suits
their research interests. If it does, they might want to familiarize themselves with the conceptual foundations
of content analysis by reading Chapter 2. Beginners in content analysis are advised to start with a small pilot

Page 15 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

project, to get a feel for what is involved in undertaking a larger project. Methodology without some practice
is empty. The guidelines in Chapter 14, although written as a summary, could also serve as a start. In this
chapter, readers will find many helpful references to pertinent chapters in this volume, which may answer
emerging questions and place these answers within the context of larger methodological issues. Beginning
researchers will soon realize that content analyzing text is not a mechanical task and neither is designing of
a project. Both undertakings require creativity and competence.

Readers who have had some experience with coding will acquire a larger perspective on what they had been
doing. As the table of contents suggests, coding is only a small part of content analysis—despite popular
misconceptions. In fact, only Chapter 7 is devoted to issues of coding and recording content analysis data.At-
tention to their problems is particularly important when data or texts are unwieldy, complex, and deserving
of careful attention. By coding/recording textual matter, one learns to appreciate both the conceptual prob-
lems involved in imposing analytical categories on ordinary readings of text and the ways in which competent
researchers have managed to solve such problems. I like to mention The Content Analysis Reader (Krippen-
dorff & Bock, 2009), which provides many more examples of solutions to practical problems than this book
could discuss in sufficient length. Designing a content analysis is something different, however. I recommend
that readers who have had experience with coding expand on that experience by examining the chapters of-
fered here about all the other components of content analysis, adding these to the conceptual framework they
seek to operationalize enriches an analysis. Such readers might well look into Chapter 11, on computer aids,
to gain an alternative perspective on manual unitizing and coding.

Readers who have already undertaken content analyses or similar text-based research will discover in this
book alternative paths for inquiries of texts and a vocabulary that can be used in deliberating about what is
involved in analyzing texts—unlike observational studies of naturalistic phenomena—as data whose signifi-
cance stems from the meanings that others bring to their readings. Those who think they know what content
analysis is are advised to start with Chapter 2, on the conceptual foundations of content analysis. This chapter
discusses the ways that researchers talk about content and exposes readers to the larger perspective they
will need in order to conceive a content analysis or critically evaluate the content analyses of others. As a
condition for publication, scholarly journals increasingly demand some demonstration of why a content analy-
sis should be taken seriously. In the past, content analysts relied heavily on conceptions of content as “con-
tained” in messages, as discussed above, or “inherent” to texts. This settled the thorny issue of multiple text
interpretations by fiat and consequently disabled explicitness about the researchers’ procedures. Several re-
search traditions—such as interpretive research, discourse analysis, literary scholarship, and rhetoric—tend
to be plagued by similar conceptions. Researchers from these traditions would greatly benefit from explicating

Page 16 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

their approaches, checking their results against the work of others, and evaluating the social consequences
of their findings outside their own schools of thought—as I am suggesting.

For experts in content analysis, this book raises several epistemological questions that practitioners rarely
ask, transforms them into methodological ones, and provides new solutions to practical problems.

Readers who must make decisions concerning whether or not to trust the findings of content analyses and
other text-based research—for instance, judges in courts of law, practitioners in the fields of public relations
and advertising, and reviewers of research submitted for funding or publication in scientific journals—will find
the vocabulary of this book useful as they need to weigh the quality of findings and make informed recom-
mendations for improvements. Such readers will find the discussions in Chapters 2 (on the conceptual foun-
dations), 12, and 13 (on reliability and validity) especially applicable to evaluative endeavors.

While this book may serve as a handbook for various practitioners, it grew out of my experiences in teaching
courses, seminars, and workshops in content analysis. In the context of education, I conceive of it foremost as
a textbook for advanced undergraduate and beginning graduate students. Teachers and their students may
not want to work through all the chapters in their numerical order. For instance, those intending to use com-
puters will find Chapter 11 more important than Chapter 7, on recording/coding, and may omit Chapter 12,
on reliability, which is not a problem for software applications; but they ought to consider the easily ignored
validity of computer uses, discussed in Chapter 13. Students with specific projects in mind may pass over
sections that may not be useful to their projects. However, readers should not rule out chapters as irrelevant
before knowing what they offer.

Finally, for me, the book will have achieved its purpose if it helps to make the newly available wealth of elec-
tronic texts accessible to systematic analysis, if it improves the social significance of research in the human-
ities and the social sciences, and if it encourages developments beyond the methods discussed therein. Un-
derstanding the realities that social actors construct with the textual matter they produce and transmit is an
ongoing challenge.

Page 17 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

https://doi.org/10.4135/9781071878781

Glossary

Abductive inference:

The process of proceeding from true propositions in one logical domain to propositions in another logical
domain, believed to be true on account of presumed empirical relationships between them—that is, from
particulars to particulars without generalizations governing both. (2.4.5)

Accuracy:

One of three kinds of Reliability. Accuracy is a measure of the extent to which a data-making instrument (e.g.,
a Coding instruction, measuring device, observational accounting practice) produces data that are accurate
relative to a trusted standard. Standards may be established by a panel of experts, a proven method, or a
norm. If the standard is considered true, accuracy measures the extent to which the generated data are valid.
(12.2.1)

Alpha (α):

A generalized coefficient of agreement used to assess the Reliability of Unitizing, Coding, and Data. Alpha
measures the agreement among independent Observers, coders, or analysts relative to what can be
expected by chance, that is, the agreement that would be observed when they randomly assign the values
from a population estimate of values to given Units or randomly segment a given continuum into units of
different lengths and kinds. It is applicable to any number of observers, all standard Metrics, missing data,
and small Sampling sizes. Alpha generalizes several special-purpose coefficients but goes far beyond either
of them. (12.2.3)

Analytical construct:

An operationalization (formalization) of the content analyst’s knowledge of how Text is used in a chosen
Context. It is the ground on which Abductive inferences from given text to unobserved features of content
analytical interests are justifiable. An analytical construct is akin to a computable model of the network of
stable correlations between available texts and the analyst’s Research question. (2.4.4; 4.2.1; 9)

Association:

At least two kinds are distinguished: (a) co-occurrences of words, concepts, or symbols from which cognitive

Page 18 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

proximities can be inferred (10.6; 11.4.3), and (b) linguistically expressed connections between attitude
objects considered either associative (e.g., is, has, likes, belongs) or dissociative (e.g., is not, dislikes,
opposes) in degrees. (9.2.3; 11.4.3)

Association structures:

Sets of objects, concepts, or symbols plus their Associations of various strengths between all pairs (11.4.5);
not to be confused with Semantic networks, whose relations have diverse meanings. (10.5; 11.4.4)

Attention:

A common Inference from the amount or relative frequency of a source’s coverage of an issue.

Attribution:

The use of language, including adjectives and stories, in declaring objects, events, or people to be of a certain
kind or to have certain qualities. (10.5)

CATA:

See Computer-aided text analysis.

Categorizing:

The process of grouping ideas, objects, and data into mutually exclusive sets. In Content analysis,
categorizing reduces a diversity of Recording/coding units into convenient kinds—the values of a nominal
Variable.

Cluster sample:

A sample that follows a hierarchical conception of the population, proceeding from general to subordinate
Units; for example, Sampling newspapers from a list of all major newspapers of a country, then sampling
particular issues from the newspapers sampled, then sampling particular articles from the issues sampled,
and so on. At each step, all units available on that level should have the same chance of being included.
(6.2.5)

Coder:

See Observer.

Page 19 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Coding:

The process of Categorizing, describing, evaluating, judging, or measuring descriptively undifferentiated Units
of analysis, thereby rendering them analyzable in well-defined terms. (12.2.3)

Coding instructions:

Instructions given to Observers to standardize processes of Recording, categorizing, or valuing phenomena

of analytical interest. They enable several independent observers to generate reliable (Reliability) data. And
they are instrumental for data analysts and the stakeholders in research results to infer from the data what
observers saw, read, and recorded.

Coefficient of imbalance:

A measure of the degree to which favorable attributes of an issue outweigh unfavorable ones. (3.3.2)

Coincidence matrix:

A tabulation of the pairwise coincidences of categories or numerical values assigned to each of a given set
of units by any number of Observers. Rows and columns of this matrix bear the names of the categories or
values used. The matrix is square and symmetrical. Its diagonal cells contain perfect matches and its off-
diagonal cells the disagreements. Its entries sum to the number of values paired. It visualizes the Reliability
of the Data it tabulates. (12.3.2; 12.4.4) Not to be confused with a Contingency matrix.

Computer-aided text analysis (CATA):

Text analysis in which the analytical task is divided between humans doing what they do best and computers
employed for what they do better: searching through huge bodies of literature and processing character
strings at high speeds but with limited ability to identify meanings. (11.4)

Concordance:

A literary account of words, phrases, and their contexts and locations in the work of an author. Software
designed for creating concordances has been used in Content analysis as well, especially in surveying the
vocabulary of a body of Texts in view of their contextual meanings.

Content:

Something that, according to a popular metaphor, is contained in messages or Texts regardless of who

Page 20 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

reads them. Content analysts avoid the use of this content-objectifying metaphor by acknowledging both the
analysts’ informed attention to available texts as well as the many worlds of readers, writers, and users for
whom the analyzed texts could have all kinds of meanings. (2.2)

Content analysis:

A Research technique for making replicable and valid Inferences from Texts (or other meaningful matter) to
the Contexts of their use. (for contested definitions, 2.1)

Context:

Generally, the world in which a body of Text plays a role or has meanings to those attending to it. In Content
analysis, one distinguishes the many worlds of others in which texts play various roles, from which texts are
sampled (decontextualized) for analysis, and the world of the analyst, in which particular Research questions
are posed and answered with the help of available texts. The context of the analyst need not coincide with the
worlds of others; however, its construction must be justifiable to the content analyst’s peers and users of the
content analysis results. (2.4.3) Most specifically, the linguistic context of a word is its textual environment,
which enables readers to make context-specific sense of that word.

Context units:

Textual matter that is defined to limit what Observers or readers are to consider when Categorizing, or
describing Recording/coding units. The size of context units is important. If allowed to be too large, coding
tends to become unreliable. If defined too small or not at all, recording units may lose the information that
stems from the textual environment of these units. For example, personal pronouns are meaningless without
reference to the contexts in which they occur. (5.2.3)

Contingency analysis:

Analysis of the co-occurrences of words, concepts, or symbols, especially relative to their co-occurring by
chance, in a body of Text to infer the Association structures of authors or readers of that text. (10.6)

Contingency matrix:

A cross-tabulation of observed Units of analysis created in order to analyze statistical dependencies between
two categorical or nominal Variables describing these units. Contingencies tend to be analyzed by association

statistics, such as χ2 or product-moment correlations. When Reliability data are recorded in contingency
matrices, these matrices are square, rows and columns refer to different Observers, perfect matches of

Page 21 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

categories occur in their diagonal cells, and systematic mismatches occur in their off-diagonal cells. However,
analyzing reliability with association statistics would be a mistake, as reliability requires observers to be
interchangeable, and statistical dependencies among the proclivities of individual observers are misleading
indicators of reliability. The entries of contingency matrices sum to the number of units described. (12.3.4.2;
12.3.5) A contingency matrix must not be confused with a Coincidence matrix, whose entries sum to the
values assigned to units and is used in evaluating reliability. (12.3.2; 12.4.4)

Conversation:

Fundamentally open and self-governed human interactions that rely, but not exclusively, on linguistic
utterances or Texts, each responding to preceding utterances or texts. The prototypical conversation is
interpersonal. However, content analysts have extended the notion of conversation when making Inferences
from diplomatic exchanges, deliberations in small groups, and therapeutic sessions regarding how
relationships develop interactively and interactive processes unfold. (3.6)

Crowdcoding:

A version of crowdsourcing that invites anonymous Internet users to code or evaluate texts for subsequent
analysis. Besides savings in cost and time to train observers to generate analyzable data, crowdcoding is
often justified by appealing to the wisdom of crowds, the belief that collective judgments are better than
individual ones (7.4)—tested by the Decisiveness of majorities (12.4.3)

Data (plural of datum):

Semidurable records taken as the unquestioned basis for reasoning, discussion, or calculation. As such, data
are not found but generated at least by declaration more typically by experiments, surveys, or measurements.
Data must show some diversity (convey information—hence the plural of datum), must be comparable with
each other, and, in the case of scientific data, must be obtained by replicable methods or be validatable. The
data of Content analyses are Texts. (4.1; 8)

Data language:

The formal organization of Data—for example, a system of categories and measurements. A data language
has a syntax and a semantics. Its syntax must render the data amenable to a proposed analysis (e.g., be
computable). Its semantics links the data to the phenomena they are thought to represent, often formalized
in Coding instructions that specify the relationship between each datum and what Observers saw or read, or
embodied in measuring instruments. (8)

Page 22 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Data reduction:

Amounts to simplifying typically large volumes of Data—for example, by counting, abstracting, computing
indices, offering statistical accounts of analytical results, or describing a statistical distribution by its
parameters. In more qualitative approaches, data reduction means selecting representative quotes or
prototypical examples from the analyzed Text, summarizing or abstracting an analyst’s reading. All content
analyses start with volumes of texts and reduce them to the answers to Research questions. (4.1.1; 10)

Decisiveness of majorities or aggregates:

One of four Reliability measures that assesses the degree to which individual observer’s judgments,
evaluations, or descriptive accounts of phenomena deviate from their majority or average. It is based on
data obtained by replication of data generating processes, but unlike Replicability, it answers the question of
whether a majority can be trusted more so than individual observers. (12.4.4)

Deductive inference:

The process of proceeding from a general proposition, considered true, to particular propositions that are
logically implied and are therefore considered true as well. (2.4.5)

Dictionary (CATA):

A sometimes user-specifiable computational tagging of single or compound words in a body of Text. Tags
have meanings that are simpler and more general than those occurring in the text, ideally preserving the
meanings that are relevant for answering a given Research question. CATA dictionaries make use of thesauri,
Stemming, Lemmatization, and disambiguation rules that rely on the linguistic environment of tagged Units of
text. They aim at Data reduction by providing frequencies of tags in text. The relationship of tags to what they
tagged demonstrates their Semantic validity. (11.4.1)

Difference functions:

In the calculation of the agreement coefficient Alpha, difference functions weigh the frequencies of coinciding
values by the Metric differences between these values. A difference function can be stated mathematically
or tabularly. In matrix form, a difference function is defined by a list of all differences between two sets of
categories, ranks, values, (12.3.4) or sets of values. (12.7.2)

Direct/indirect indicators:

Page 23 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Analytical constructs in the form of correlations between textual attributes and the phenomena of interest
are called direct indicators (see Index), and those that take the form of a network of correlates, including
intervening Variables, are called indirect indicators. The history of Content analysis shows a move from direct
to indirect indicators of the answers to Research questions. (9.3.3)

Evaluative assertion analysis:

The use of two-valued predicates to infer the valuation of an object from how it is linguistically associated (see
also Association). (9.2.3)

Extrapolations:

Inferences about the unobserved gaps between observations, such as between data points (interpolations),
or beyond a recorded history of observations (predictions). Extrapolations assume the continuity of observed
phenomena, trends, patterns, or differences. (3.2; 9.3.1)

General Inquirer:

The historically first operational dictionary approach to computational Content analysis; accepts several CATA
Dictionaries. (11.4.1)

Go-words:

A list of words to be included in an analysis. (11.3.1)

Index:

A Variable that is computed on textual attributes for its correlation with extratextual phenomena or variables
of interest. An index empirically relates logically distinct kinds and may therefore be used to infer particular
phenomena or compare several Texts in their own terms. (3.4; 9.3.3)

Inductive inference:

The process of proceeding from particular propositions, such as a sample of observations, to general
propositions, such as to statistical generalizations of that sample, accounting for these observations in most
if not all respects. (2.4.5)

Inference:

The process of passing from accepted propositions, statements, or data to other propositions or statements

Page 24 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

whose validity is believed to be preserved in that process. One can distinguish Deductive, Inductive, and
Abductive inferences. In Content analysis inferences are typically abductive, proceeding from available Text
to the analyst’s Research question, which pertains to unavailable features of the analyst’s chosen Context.
(2.1; 2.4.5)

Institutions:

Habitual social practices that are enacted within a community and serve normative functions for how members
organize themselves and construct the realities they live by. Content analysts may infer how the exchange of
textual matter encourages, constructs, or undermines particular institutional realities. (3.7)

KWIC (keyword in context) lists:

The tabulation of character strings of a specified length, within which selected keywords or phrases occur in a
Text. KWIC lists consist of the contexts in which chosen words or phrases occur in analyzed texts suggesting
their meanings or roles they play within these texts. (11.3)

Latent content:

See its opposite, Manifest content. (2.1)

Lemmatization:

The process of Stemming a word to its stem or root form after considering the grammatical function of the
word in a sentence. (11.3; 11.4.1)

Levels of measurement:

See Metric.

LexisNexis:

A company that makes probably the largest English-language electronic text archive available to subscribers.
Lexis provides a searchable database of U.S. laws and legal decisions. Nexis provides access to published
data from more than 20,000 multilingual news sources, journals, biographical and reference materials,
business reports, and regulatory findings.

LIWC (Linguistic Inquiry and Word Count):

A CATA Dictionary approach. (11.4.1)

Page 25 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Manifest content:

Texts that are easy to read, generally understood, unambiguously interpretable, and therefore yield high
agreement, even among untrained coders. Early definitions of Content analysis sought to exclude Latent
content for its methodological difficulties, requiring experts to identify and code. (2.1)

Matrix:

A rectangular array of numbers or symbols, arranged in rows and columns, often summed to its marginal
totals. (See Coincidence matrix, Contingency matrix, and Difference functions.)

Meme:

A unitary idea, message, behavior, or style that spreads throughout a community of minds. Analogous to
genes, which transmit biological information, memes transmit cultural information and are thought to affect
the content and organization of human minds. (11.4.5)

Metric:

An explicit ordering of the values of a Variable, distinguished by the kinds of operations on these values that
preserve their ordering (8.7). Most common are binary, nominal, ordinal, interval, and ratio metrics, also called
levels of measurement (8.6). Analytical methods tend to make specific demands on the metric of the data they
can process. Conversely, the metric of given data limits the choices among analytic/computational techniques
available. In the analysis of reliability, metrics are operationalized in the form of Difference functions.

Networked texts:

Whereas writing is continuous, and many social phenomena, ranging from speech to historical developments,
proceed linearly, hypertext can be read in sequences at their readers’ digressions, and users of social media
generate large volumes of messages and dynamically link them across many. All posts by Internet users
respond to previously received messages and acquire their meanings by what happens after they are posted,
commented on, reproduced, and acted upon. Individual posts are short and quite uninformative of what
happens. Social media texts are networked, and their content analyses cannot ignore the contexts of their
connections. (2.3; 11.3.4)

Observer:

A generic term for a coder, unitizer, transcriber, judge, rater—anyone who has access to undifferentiated

Page 26 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

phenomena and transforms them into analyzable Data. (qualifications, 7.2; training, 7.3; reliability, 12.2)

Plagiarism:

An established resemblance of a work by one author and a previously published work by another, controlled
for common cultural backgrounds or joint exposure to events. (10.5)

Precision:

A measure of the extent to which the items retrieved by a search algorithm are relevant to a research
question. This proportion is extended to a reliability coefficient for text retrieval. (12.5)

Propaganda analysis:

In Content analysis, the term applies to three different conceptions: (a) Using printed matter and speeches
to identify propagandists in pursuit of doctrinaire intents or in the unacknowledged service of a foreign
government. Keys to this identification are the use of devious persuasion techniques. (b) Making Inferences
from an enemy country’s domestic broadcasts about planned military actions, political support of the
governing elite by its population, prediction of leadership succession, and opposition to the government. (c)
Assessing the effectiveness of one’s own propaganda efforts. (1.4)

QDA software:

See Qualitative data analysis software.

Qualitative content analysis:

Conventionally contrasted with Quantitative content analysis—involves a close reading of textual matter,
reorganizing relevant parts of it into analytical categories, and creating interpretations, narratives of scholarly
interest relating to the meanings and uses of the analyzed Text. Any interpretation of a text is an Inference
that should have Semantic validity—that is, it must be plausibly related to the original text. To demonstrate
that connection, qualitative content analysts prefer to support their inferences by quoting from the analyzed
text. Qualitative content analysts are not opposed to counting, but their inferences rarely rely on frequencies.
(1.7)

Qualitative data analysis (QDA) software:

Software that provides a means to efficiently manage bodies of Text as well as graphical, audio, and
video data in the pursuit of text-driven Qualitative content analysis. It accounts for analyst-introduced

Page 27 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

unitizing, coding, annotating, retrieving, sorting, counting, and graphing of various relationships among textual
elements. (11.5)

Quantitative content analysis:

Conventionally contrasted with Qualitative content analysis—relies heavily on enumerating coded textual
matter. However, all Coding of Text involves qualitative judgments or identifications, so that the distinction
between qualitative and quantitative approaches is largely one of emphasis, often falsely identified with being
interpretive versus scientific. (2.1)

Quantitative newspaper analysis:

A precursor to Content analysis developed around the end of the 19th century. It measured the amount of
space in newspapers devoted to various subject matters, largely to objectify and criticize developments in
newspaper publishing at that time. (1.2)

Random sample:

A sample of Units of analysis selected from a known population where each unit has the same probability of
being included. (6.2.1)

Recall:

A measure of the extent to which a search algorithm retrieves all relevant items contained in a database. This
proportion entails an epistemological paradox as the sole motivation for using search algorithms, which is the
human inability to know what a data base contains. (12.5)

Recording:

Creating somewhat durable records from transient or otherwise unanalyzable phenomena. Recording ranges
from the use of mechanical devices (voice recorders) and transcriptions to Unitizing and Coding of Text.
Durability is a prerequisite of all re-search efforts and Data in particular, which moreover require analyzability
by available techniques. (7)

Recording/coding units:

Identified within unanalyzed Text, recording units are independently describable, transcribable, categorizable,
or codable units. Each recording or coding unit becomes represented by an enumerable record, code, or
datum (see Data). To include information from the textual surroundings of such units without abandoning their

Page 28 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

analytically required independence, content analysts often ask observers to describe them as a function of
Context units within which they occur. (5.2.2)

Relational content analysis:

A form of content analysis that seeks to set itself apart from categorical accounts of textual elements, focusing
on relations, contingencies, or semantic connections between them. It underlies Contingency analysis,
Evaluative assertion analysis, Semantic networks, and Webgraphs.

Reliability:

The attribute of Data on which researchers can rely in answering their Research questions. Unreliability
results from two factors: disagreements among independent ways of generating data about the phenomena
in question and lack of information (variation) in the resulting data to answer given research questions about
these phenomena. In Content analysis, the first is assessed by agreement coefficients, such as Alpha, applied
to how several Observers unitize the same continuum and/or record, describe, or code the same set of
Units. The second becomes evident by tracing the information in data throughout an analytical process to
its results. (When variation is absent, data cannot be correlated with anything; and when undetected noise
in the data is not eliminated during an analysis, the answers to Research questions may well be unrelated
to the phenomena of interest.) Four kinds of reliability are distinguished: Stability, Replicability, Accuracy or
Surrogacy, (12.1) and Decisiveness of majorities. (12.4) High reliability is a prerequisite of Validity but does
not guarantee it. (12.1)

Replicability:

One of four kinds of Reliability. Replicability measures the extent to which data-making instruments (Coding
instructions, measuring devices, or observational methods) can be relied upon to generate the same Data
from the same set of phenomena in diverse circumstances, employing different and independently working
Observers, coders, judges, or measuring instruments. It is a function of the degree to which independently
obtained data of the same phenomena agree. (12.2.1)

Re-presentation:

A text-invoked recall of something previously experienced, making that experience present again, as distinct
from representation, which suggests some objective relationship between Text and what it is about. (3.5)

Research or re-search:

Page 29 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

A repeated and replicable search within Data for patterns that explain them. Content analysts move outside
such searches by relying on identifiable properties in Texts to answer Research questions pertaining to the
Contexts of these texts.

Research design:

The network of procedural steps a researcher plans to take to proceed from generating Data to research
results. Research designs should be explicit so that they can be replicated or critically evaluated for the
conclusiveness of the findings. (4; 14.1)

Research questions:

In Content analysis, research questions must satisfy three requirements: (a) They must pertain to currently
unknown phenomena in the Context of analyzed Texts, (b) they must allow several possible answers among
which the analysis selects, and (c) they must provide the possibility—even if only in principle—of alternative
ways to answer the chosen question. The requirement (a) is implicit in the definition of content analysis, (b)
prevents researchers from merely generalizing from or confirming what they already know, and (c) assures
that the answers of research questions are validatable in principle, by means other than a content analysis,
subsequent observations, correlations with related variables, or successful actions based on the results.
(2.4.2; 14.1.2.1)

Sampling:

The process of selecting a representative sample from a larger population of Units so that an analysis of
the sample enables the researcher to draw conclusions about that population. The Inference involved is
inductive—not deductive or abductive. (6)

Sampling units:

Units distinguished for selective inclusion in an analysis. (5.2.1)

Sampling validity:

The assurance that a sample is large enough to represent a population of phenomena in its composition so
that the sample can be studied in place of that population. Because Content analysis makes Inferences from
Texts to nontextual phenomena, two populations are important: (a) the population of texts actually sampled
and analyzed, and (b) the population phenomena to be inferred from the sampled texts. Sampling validity
is assured when texts are sampled so that the phenomena of interest are fairly represented in the sample.

Page 30 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

(13.2.1)

Semantic differential scales:

Seven-point scales between polar-opposite attributes (good-bad, strong-weak, active-passive), usually

labeled from -3 to +3, and widely used in assessments of sources’ evaluations of word meanings or attitude
objects referred to. (7.4.4)

Semantic networks:

Sets of objects, concepts, or symbols connected by linguistically meaningful binary relations, such as two-
place predicates (11.4.4); not to be confused with Association structures, whose binary relations are merely
of varying strengths.

Semantic validity:

One Validity criterion unique to content analysis, measuring the degree to which relevant readings of a Text
are preserved in processes of Unitizing, Coding, and analyzing the text. (13.2.2)

Sentiment analysis:

A computational Content analysis using a dictionary of pleasant–unpleasant, active–passive, and easy to

imagine–hard to imagine attributes of selected attitude objects or issues. It is intended to infer their emotional
value. (11.4.1)

Snowball sample:

A multistage sample obtained by examining an initial sample of Units for leads to other Sampling units and so
on, until some practical closure is reached. (6.2.6)

Stability:

One of four kinds of Reliability. Stability measures the extent to which a researcher or data-making instrument
repeatedly generates the same Data from the same set of phenomena. Stability is not sufficient to establish
the reliability of content analysis data, as consistent biases, prejudices, or misunderstandings of Coding
instructions would not become manifest in repetitions. Replicability, which can detect individual
idiosyncrasies, provides a stronger measure. (12.2.1)

Standards:

Page 31 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Established rules, principles, or measures against which deviations can be measured. Reliability and Validity
are two standards against which the quality of a Content analysis is judged (12.1). Reliability determines the
degree to which data represent the phenomena to be analyzed and validity the extent to which its inferences
are borne out by independent evidence. Content analysis may also use standards in its Research questions,
for example, in establishing Plagiarism, to evaluate biases (Coefficient of imbalance), or to judge an account
as true or false. (3.3; 9.3.2)

Stemming:

The process of reducing grammatical variations of words to their common stems, bases, or root forms, used
to simplify CATA Dictionaries. (11.3)

Stop-words:

A list of words to be excluded from an analysis, largely because they are judged irrelevant to the Research
question. (11.3.1)

Stratified sample:

A sample whose Units are selected (randomly or systematically) from several known subpopulations (strata)
in a population so that each subpopulation is represented in the sample either equally or in proportion to its
size. (6.2.3)

Surrogacy:

The degree to which a measure or model can serve as a surrogate of other phenomena. It is computationally
equivalent to Accuracy, one of four forms of reliability. (12.2.1) (12.4)

Systematic sample:

A sample that consists of every kth Unit from a serial population or list of textual units. (6.2.2)

TAT:

See Thematic Apperception Test.

Text analysis:

Provides accounts of the character strings, compositions, coincidences, syntactical structures, and layouts of
given Text. Word counts and KWIC lists are basic. Text mining and Webgraphs make use of search engines

Page 32 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

and more complicated algorithms for analyzing texts. Computer text analysis has little place for meanings and
leaves the burden of relating its findings to the given Research question to the content analyst. (11.3)

Text mining:

The largely computerized search and identification of usually rare textual elements in a large bodies of Texts.
The textual elements searched and found should not be confused with information that could answer the
Research questions content analysts ask. Although there are occasions where names of individuals, places,
or concepts lead to such answers, the latter always requires human judgments. (11.3.2)

Texts:

Anything variable and textured that has meaning to somebody, including to the analyst, and can be examined
or read, interpreted, and acted upon repeatedly: letters, e-mails, blogs, literature, images, video recordings,
transcripts of conversations, political speeches, historical records, police reports, accounts of legal
proceedings, corporate statements, advertisements, medical documents, photographic images, posters,
cultural artifacts, and so on. (2.4.1)

Thematic Apperception Test (TAT):

A psychological test involving the content analysis of interpretations of ambiguous images by subjects in order
to reveal hidden attitudes, prejudices, motives, and personality variables.

Trends:

See Extrapolations.

Unitizing:

Identifying contiguous and nonoverlapping sections within a descriptively undifferentiated continuum of

Text or other spatially organized matter, typically generating Recording or coding units—words, sentences,
utterances, rhetorical moves, metaphors, themes, images, anything—that may thereafter be coded whole and
subsequently analyzed. (5.3; 12.2.3)

Units:

Decontextualized but information-bearing textual wholes that are distinguished within an otherwise
undifferentiated continuum and thereafter considered separate from their context and independent of each
other (5.1). Content analysts distinguish three kinds of units by the functions they serve within an analysis:

Page 33 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Sampling, Recording or Coding, and Context units (5.2). Units may be distinguished physically (5.3.1),
syntactically (5.3.2), categorially (5.3.3), propositionally (5.3.4), and thematically (5.3.5).

Units of enumeration:

The units that are actually counted or measured. Although Recording/coding units may be described
numerically, for example, in terms of their size, their location, or their statistical properties, enumeration
typically follows Coding, and units of enumeration often coincide with Recording/coding units. (5.2.3)

Unobtrusiveness:

The attribute of a Research method that uses Data about which their sources or object of study have no
knowledge, and the collection of which does not affect these sources or objects. Content analysis is mostly
unobtrusive, as it relies on Text not generated for the purpose of analysis. By contrast, survey research and
psychological experiments are obtrusive, as asking questions or instructing subjects affects those studied.
(2.5)

Unstructuredness:

The attribute of phenomena that appear in a form that available analytical techniques cannot (yet) handle.
Unstructured phenomena may be transient, as is human speech, or possess properties that are variable and
complex, as in most human communications. Much of content analysis involves transforming unstructured
matter into analyzable Text. (2.5)

Validating evidence:

Data that substantiate analysts’ results by means other than those used in the analysis. In Content analysis,
validating evidence is largely ex post facto, as its Research questions are posed in the very absence of direct
evidence for what it seeks to infer from available Text. (2.4.6)

Validity:

The quality of a claim to be as stated, true, or correct. A Content analysis is valid to the extent the Inferences
about the Context of the analyzed Texts withstand the test of independently obtained Validating evidence.
(13)

Variable:

A conceptual unity that embraces alternatives, something that can vary. Variables are essential to any Data

Page 34 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

language. Open variables are placeholders for Units as observed or found in Text. Closed variables have a
predefined range of or consist of a list of conceptually possible values (8.3). The values of a variable may be
unordered (8.4), or ordered variously (8.5), and possess diverse Metrics (8.6).

Varying probability sample:

A sample whose Units are selected according to known probabilities of their relevance to a Research
question. It compensates for known inequalities in the units’ informativeness, importance, or self-sampling
biases. (6.2.4)

Webgraphs:

Graphical depictions of the links between Web pages available on the Internet. Inasmuch as links are
established by website designers for directing readers’ attention to other relevant matter, they represent
potential traffic and networks of meanings within communities of users. (11.3.4)

WordNet:

A lexical database for the English language. It groups English words into sets of synonyms, similar to a
thesaurus, provides short definitions for these sets, and orders them according to the semantic relations
between them. Its organization is based on common uses of language. It supports Text mining and
computational linguistics approaches to information processing and has been used in the development of
content analysis instruments. (11.3.2)

Note: Numbers in parentheses within and following glossary entries refer to sections or whole chapters of this
volume in which the concepts are discussed in depth.

Page 35 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

References

Abell, Peter. (1987). The syntax of social life: The theory and method of comparative narratives. New York,
NY: Oxford University Press.

Abell, Peter. (1993). Some aspects of narrative method. Journal of Mathematical Sociology, 18, 93–134.

Abelson, Robert P. (1963). Computer simulation of hot cognition. In Silvan S. Tomkins & Samuel Messick
(Eds.), Computer simulation of personality (pp. 277–298). New York, NY: Wiley.

Abelson, Robert P. (1968). Simulation of social behavior. In Gardner Lindzey & Elliot Aronson (Eds.), The
handbook of social psychology (pp. 274–356). Reading, MA: Addison-Wesley.

Abelson, Robert P., & Rosenberg, Milton J. (1958). Symbolic psychologic: A model of attitude cognition.
Behavioral Science, 3, 1–13.

Adorno, Theodor W. (1960). Television and the patterns of mass culture. In Bernard Rosenberg & David M.
White (Eds.), Mass culture (pp. 474–488). New York, NY: Free Press.

Ahn, Luis von; & Dabbish, Laura. (2004). Labeling images with a computer game. Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, 319–326.

Albig, William. (1938). The content of radio programs 1925–1935. Social Forces, 16, 338–349.

Albrecht, Milton C. (1956). Does literature reflect common values? American Sociological Review, 21,
722–729.

Alexa, Melina. (1997). Computer-assisted text analysis methodology in the social sciences (Arbeitsbericht 7).
Mannheim, Germany: Zentrum für Umfragen, Methoden und Analysen.

Alexa, Melina, & Züll, Cornelia. (1999). A review of software for text analysis (Nachrichten 5). Mannheim,
Germany: Zentrum für Umfragen, Methoden und Analysen.

Allen, Liska E. (1963). Automation: Substitute and supplement in legal practice. American Behavioral
Scientist, 7, 39–44.

Allport, Gordon W. (1942). The use of personal documents in psychological science. New York, NY: Social
Science Research Council.

Allport, Gordon W. (Ed.). (1965). Letters from Jenny. New York, NY: Harcourt Brace Jovanovich.

Page 36 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Allport, Gordon W., & Faden, Janet M. (1940). The psychology of newspapers: Five tentative laws. Public
Opinion Quarterly, 4, 687–703.

Alpaydin, Ethem (2004). Introduction to machine learning. Cambridge, MA: MIT Press.

Altheide, David L. (1987). Ethnographic content analysis. Qualitative Sociology, 10, 65–77.

Alwin, Duane F. (1974). Approaches to the interpretation of relationships in the multitrait-multimethod matrix.
In Herbert L. Costner (Ed.), Sociological methodology 1973–1974 (pp. 79–105). San Francisco, CA: Jossey-
Bass.

American Educational Research Association, American Psychological Association, and National Council on
Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC:
American Psychological Association.

American Educational Research Association, American Psychological Association, and National Council on
Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC:
American Psychological Association.

American Educational Research Association, American Psychological Association, and National Council on
Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC:
American Psychological Association.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., rev.).
Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed., rev.).
Washington, DC: Author

American Psychological Association. (1954). Technical recommendations for psychological tests and
diagnostic techniques. Psychological Bulletin, 51(Suppl. 2), 200–254.

Andsager, Julie L., & Powers, Angela. (1999). Social or economic concerns: How news and women’s
magazines framed breast cancer in the 1990s. Journalism & Mass Communication Quarterly, 76, 531–550.

Armstrong, Robert P. (1959). Content analysis in folkloristics. In Ithiel de Sola Pool (Ed.), Trends in content
analysis (pp. 151–170). Urbana: University of Illinois Press.

Arnheim, Rudolf, & Bayne, Martha C. (1941). Foreign language broadcasts over local American stations. In
Paul F. Lazarsfeld & Frank N. Stanton (Eds.), Radio research 1941 (pp. 3–64). New York, NY: Duell, Sloan &

Page 37 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Pearce.

Aron, Betty. (1950). The Thematic Apperception Test in the study of prejudiced and unprejudiced individuals.
In Theodor W. Adorno; Else Frenkel-Brunswick; Daniel J. Levinson; & R. Nevitt Sanford, The authoritarian
personality. New York, NY: Harper.

Ash, Philip. (1948). The periodical press and the Taft-Hartley Act. Public Opinion Quarterly, 12, 266–271.

Asheim, Lester. (1950). From book to film. In Bernard Berelson and Morris Janowitz (Eds.), Reader in public
opinion and communication (pp. 299–306). New York, NY: Free Press.

Atkinson, J. Maxwell, & Heritage, John. (Eds.). (1984). Structures of social action: Studies in conversation
analysis. Cambridge, MA: Cambridge University Press.

Averill, James R. (1985). The social construction of emotions with special reference to love. In Kenneth J.
Gergen & Keith E. Davis (Eds.), The social construction of the person (pp. 89–107). New York, NY: Springer-
Verlag.

Baldwin, Alfred L. (1942). Personal structure analysis: A statistical method for investigating the single
personality. Journal of Abnormal and Social Psychology, 37, 163–183.

Bales, Robert F. (1950). Interaction process analysis. Reading, MA: Addison-Wesley.

Barcus, Francis E. (1959). Communications content: Analysis of the research 1900–1958: A content analysis
of content analysis. Unpublished doctoral dissertation, University of Illinois.

Barnett, George A., & Doerfel, Marya L. (1997, May). A semantic network analysis of the International
Communication Association. Paper presented at the 47th Annual Meeting of the International Communication
Association, Montreal, Canada.

Barton, Allen H. (1968). Bringing society back in: Survey research and macro-methodology. American
Behavioral Scientist, 12(2), 1–9.

Bateson, Gregory. (2000). Steps to an ecology of mind. New York, NY: Basic Books.

Bauer, Christian, & Scharl, Arno. (2000). Quantitative evaluation of Web site content and structure. Internet
Research, 10, 31–41.

Baxter, Leslie A., & Montgomery, Barbara M. (1996). Relating: Dialogues and dialectics. New York, NY:
Guilford Press.

Page 38 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Becker, Howard P. (1930). Distribution of space in the American Journal of Sociology, 1895–1927. American
Journal of Sociology, 36, 461–466.

Becker, Howard P. (1932). Space apportioned forty-eight topics in the American Journal of Sociology,
1895–1930. American Journal of Sociology, 38, 71–78.

Bengston, David N., & Xu, Zhi. (1995). Changing national forest values: A content analysis (Research
Paper NC-323). St. Paul, MN: U.S. Department of Agriculture, Forest Service, North Carolina Forest
Experimentation Station.

Bennett, Edward M.; Alpert, R.; & Goldstein, A. C. (1954). Communications through limited response
questioning. Public Opinion Quarterly, 18, 303–308.

Benoit, William L.; Blaney, Joseph R.; & Pier, P. M. (1998). Campaign ‘96: A functional analysis of acclaiming,
attacking, and defending. Westport, CT: Praeger.

Berelson, Bernard. (1949). What “missing the newspaper” means. In Paul F. Lazarsfeld & Frank N. Stanton
(Eds.), Communications research 1948–1949 (pp. 111-129). New York, NY: Harper Brothers.

Berelson, Bernard. (1952). Content analysis in communications research. New York, NY: Free Press.

Berelson, Bernard, & Lazarsfeld, Paul F. (1948). The analysis of communication content. Chicago, IL:
University of Chicago Press.

Berelson, Bernard, & Salter, Peter J. (1946). Majority and minority Americans: An analysis of magazine fiction.
Public Opinion Quarterly, 10, 168–190.

Berelson, Bernard, & Steiner, George A. (1964). Human behavior: An inventory of scientific findings. New
York, NY: Harcourt Brace Jovanovich.

Berger, Peter L., & Luckmann, Thomas. (1966). The social construction of reality: A treatise in the sociology
of knowledge. Harmondsworth, England: Penguin.

Berkman, Dave. (1963). Advertising in Ebony and Life: Negro aspirations vs. reality. Journalism Quarterly,
40, 53–64.

Bermejo, Fernando. (1997). Press self-presentation: Self-reference and reflexivity in newspapers.

Unpublished master’s thesis, University of Pennsylvania, Annenberg School for Communication.

Bernard, H. Russell, & Ryan, Gery W. (1998). Text analysis, qualitative and quantitative methods. In H.
Russell Bernard (Ed.), Handbook of methods in cultural anthropology (pp. 595–646). Walnut Creek, CA:

Page 39 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

AltaMira.

Best, Michael L. (1997). Models for interacting populations of memes: Competition and niche behavior.
Journal of Memetics—Evolutionary Models of Information Transmission, 1, 80–96.

Best, Michael L. (1998). Corporal ecologies and population fitness on the Net. Journal of Artificial Life, 3,
261–287.

Bishop, Stephanie. (1998). Driving dangerously in the Prozac nation—Road rage: The making of a modern
malady. Unpublished manuscript, University of Pennsylvania, Annenberg School for Communication.

Blalock, Herbert M. (1964). Causal inferences in non-experimental research. Chapel Hill: University of North
Carolina Press.

Bloch, Daniel A., & Kraemer, Helena Chmura. (1989). 2 × 2 kappa coefficients: Measures of agreement or
association. Biometrics, 45, 269–287.

Bonfantini, Massimo, & Proni, Giampaolo. (1988). To guess or not to guess? In Umberto Eco & Thomas
A. Sebeok (Eds.), The sign of three: Dupin, Holmes, Peirce (pp. 119–134). Bloomington: Indiana University
Press.

Boot, N. (1980). Homography and lemmatization in Dutch texts. ALLC Bulletin, 8, 175–189.

Boulding, Kenneth E. (1978). Ecodynamics. Beverly Hills, CA: Sage.

Brennan, Robert L., & Prediger, Dale J. (1981). Coefficient kappa: Some uses, misuses, and alternatives.
Educational and Psychological Measurement, 41, 687–699.

Brin, Sergey, & Page, Lawrence. (1998). The anatomy of a large-scale hypertextual Web search engine.
Computer Networks and ISDN Systems, 30, 107–117. Accessed at http://infolab.stanford.edu/pub/papers/
google.pdf

Broder, David P. (1940). The adjective-verb quotient: A contribution to the psychology of language.
Psychological Record, 3, 310–343.

Broom, Leonard, & Reece, Shirley. (1955). Political and racial interest: A study in content analysis. Public
Opinion Quarterly, 19, 5–19.

Brouwer, Marten; Clark, Cedric C.; Gerbner, George; & Krippendorff, Klaus. (1969). The television world
of violence. In Robert K. Baker & Sandra J. Ball (Eds.), Mass media and violence (Vol. 9, pp. 311–339,
519–591). Washington, DC: Government Printing Office.

Page 40 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Bruner, Jerome S., & Allport, Gordon W. (1940). Fifty years of change in American psychology. Psychological
Bulletin, 37, 757–776.

Bruns, Axel. (2007). Methodologies for mapping the political blogosphere: Explorations using the
IssueCrawler research tool. First Monday, 12, 5. http://firstmonday.org/ojs/index.php/fm/article/view/1834/
1718

Budd, Richard W. (1964). Attention score: A device for measuring news “play.” Journalism Quarterly, 41,
259–262.

Budge, Ian; Robertson, David; & Hearl, Derek. (1987). Ideology, strategy and party change: Spatial analyses
of post-war election programmes in 19 democracies. Cambridge, MA: Cambridge University Press.

Cahnman, Werner J. (1948). A note on marriage announcements in the New York Times. American
Sociological Review, 13, 96–97.

Campbell, Donald T. (1957). Factors relevant to the validity of experiments in social settings. Psychological
Bulletin, 54, 297–311.

Campbell, Donald T., & Fiske, Donald W. (1959). Convergent and discriminant validation by the multitrait-
multimethod matrix. Psychological Bulletin, 56, 81–105.

Cappella, Joseph N.; Price, Vincent; & Nir, Lilach. (2002). Argument repertoire as a reliable and valid measure
of opinion quality: Electronic dialogue during campaign 2000. Political Communication, 19, 73–93.

Cappella, Joseph N.; Turow, Joseph; & Jamieson, Kathleen Hall. (1996). Call-in political talk radio:
Background, content, audiences, portrayal in mainstream media (Report series no. 5). Philadelphia:
University of Pennsylvania, Annenberg Public Policy Center.

Carletta, Jean; Isard, Amy; Isard, Stephen; Kowtko, Jacqueline C.; Doherty-Sneddon, Gwyneth; & Anderson,
Anne H. (1997). The reliability of a dialogue structure coding scheme. Computational Linguistics, 23, 13–31.

Carley, Kathleen M. (1997). Network text analysis: The network positions of concepts. In Carl W. Roberts
(Ed.), Text analysis for the social sciences: Methods for drawing statistical inferences from texts and
transcripts (pp. 79–100). Mahwah, NJ: Erlbaum.

Carmines, Edward G., & Zeller, Richard A. (1979). Reliability and validity assessment. Beverly Hills, CA:
Sage.

Cartwright, Dorwin P. (1953). Analysis of qualitative material. In Leon Festinger & Daniel Katz (Eds.),

Page 41 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Research methods in the behavioral sciences (pp. 421–470). New York, NY: Holt, Rinehart & Winston.

Chung, Cindy.K., & Pennebaker, James.W. (2007). The psychological function of function words. In K. Fiedler
(Ed.), Social communication: Frontiers of social psychology (pp. 343–359). New York: Psychology Press.

Cheng, Na; Chandramouli, R.; & Subbalakshmi, K. P. (2011). Author gender identification from text. Accessed
at http://www.ece.stevens-tech.edu/~mouli/gender-id.pdf

Chomsky, Noam. (1959). Review of B. F. Skinner Verbal Behavior. Language, 35, 26–58.

Cicchetti, Domenic V., & Feinstein, Alvan R. (1990). High agreement but low kappa: II. Resolving the
paradoxes. Journal of Clinical Epidemiology, 43(6), 551–559.

Clark, Cedric, C. (1966). Cultural differences in reaction to discrepant communication: An exploratory study.
Dissertation. East Lansing: Michigan State University.

Cohen, Bernard C. (1957). The political process and foreign policy: The making of the Japanese peace
settlement. Princeton, NJ: Princeton University Press.

Cohen, Jacob. (1960). A coefficient of agreement for nominal scales. Educational and Psychological
Measurement, 20, 37–46.

Cohen, Jacob; Cohen, Patricia; West, Stephen G.; & Aiken, Leona S. (2003). Applied multiple regression/
correlation analysis for the behavioral sciences, 3rd ed. Mahwah, NJ: Erlbaum.

Content analysis: A new evidentiary technique. (1948). University of Chicago Law Review, 15, 910–925.

Coombs, Clyde H. (1964). A theory of data. New York, NY: Wiley.

Cooren, François. (2010). Action and agency in dialogue: Passion, incarnation and ventriloquism.
Amsterdam, The Netherlands: John Benjamins.

Corsaro, William, & Heise, David R. (1990). Event structure models from ethnographic data. In Clifford Clogg
(Ed.), Sociological methodology, 1990 (pp. 1–57). Oxford, England: Blackwell.

Council on Interracial Books for Children. (1977). Stereotypes, distortions and omissions in U.S. history
textbooks. New York, NY: Racism and Sexism Resource Center for Educators.

Craig, Robert T. (1981). Generalization of Scott’s index of intercoder agreement. Public Opinion Quarterly,
45, 260–264.

Page 42 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Cronbach, Lee J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.

Cuilenburg, Jan J. van. (1991). Inhoudsanalyse en computer. In Roel Popping & Jules L. Peschcar (Eds.),
Goed Geïnformeerd (pp. 71–82). Houten, The Netherlands: Bohn Stafleu Van Loghum.

Cuilenburg, Jan J. van; Kleinnijenhuis, Jan; & De Ridder, Jan A. (1986). A theory of evaluative discourse:
Toward a graph theory of journalistic texts. European Journal of Communication, 1, 65–96.

Cuilenburg, Jan J. van; Kleinnijenhuis, Jan; & De Ridder, Jan A. (1988). Artificial intelligence and content
analysis: Problems of and strategies for computer text analysis. Quality and Quantity, 22, 65–97.

Dale, Edgar. (1937). The need for the study of newsreels. Public Opinion Quarterly, 1, 122–125.

D’Andrade, Roy. (1991). The identification of schemas in naturalistic data. In Mardi J. Horowitz (Ed.), Person
schemas and maladaptive interpersonal patterns (pp. 279–301). Chicago, IL: University of Chicago Press.

D’Andrade, Roy. (1995). The development of cognitive anthropology. Cambridge, MA: Cambridge University
Press.

Danielson, Wayne A.; Lasorsa, Dominic L.; & Im, Dal S. (1992). Journalists and novelists: A study of diverging
styles. Journalism Quarterly, 69, 436–446.

Danowski, James A. (1982). A network-based content analysis methodology for computer-mediated

communication: An illustration with a computer bulletin board. In Michael Burgoon (Ed.), Communication
yearbook 6 (pp. 904–925). Beverly Hills, CA: Sage.

Danowski, James A. (1993). Network analysis of message content. In William D. Richards, Jr., & George A.
Barnett (Eds.), Progress in communication sciences (Vol. 4, pp. 197–221). Norwood, NJ: Ablex.

Darnton, Robert. (1999, June 12). No computer can hold the past. New York Times, p. A15.

Dawkins, Richard. (1976). The selfish gene. New York, NY: Oxford University Press.

Dawkins, Richard. (1982). The extended phenotype: The long reach of the gene. San Francisco. CA: W. H.
Freeman.

Deerwester, Scott C.; Dumais, Susan T.; Furnas, George W.; Landauer, Thomas K.; & Harshman, Richard
A. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41,
391–407.

Denzin, Norman K., & Lincoln, Yvonna S. (2000). Introduction: The discipline and practice of qualitative

Page 43 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

research. In Norman K. Denzin & Yvonna S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp.
1–28). Thousand Oaks, CA: Sage.

DeWeese, L. Carroll, III. (1977). Computer content analysis of “day-old” newspapers: A feasibility study.
Public Opinion Quarterly, 41, 91–94.

Dibble, Vernon K. (1963). Four types of inferences from documents to events. History and Theory, 3,
203–221.

Diefenbach, Donald L. (2001). Historical foundation of computer-assisted content analysis. In Mark D. West
(Ed.), Theory, method, and practice in computer content analysis (pp. 13–41). Westport, CT: Ablex.

Dodds, Peter S.; Harris, Kameron D.; Kloumann, Isabel M.; Bliss, Catherine A.; & Danford, C. M. (2011).
Temporal patterns of happiness and information in a global social network: Hedonometrics and Twitter. PloS
ONE, 6, 12, e26752. http://arxiv.org/abs/1101.5120v3, accessed 2017.9.15.

Dollard, John, & Auld, Frank, Jr. (1959). Scoring human motives: A manual. New Haven, CT: Yale University
Press.

Dollard, John, & Mowrer, O. Hobart. (1947). A method of measuring tension in written documents. Journal of
Abnormal and Social Psychology, 42, 3–32.

Doreian, Patrick. (Ed.). (1993). Narrative methods [Special issue]. Journal of Mathematical Sociology,
18(1–2).

Dovring, Karin. (1954–1955). Quantitative semantics in 18th century Sweden. Public Opinion Quarterly, 18,
389–394.

Dumais, Susan T. (1992). LSI meets TREC: A status report. In Donna K. Harman (Ed.), The First Text
REtrieval Conference (TREC-1) (NIST Special Publication 500–207). Washington, DC: U.S. Department of
Commerce, National Institute of Standards and Technology.

Dumais, Susan T. (1993). Latent semantic indexing (LSI) and TREC-2. In Donna K. Harman (Ed.), The
Second Text REtrieval Conference (TREC-2) (NIST Special Publication 500–215). Washington, DC: U.S.
Department of Commerce, National Institute of Standards and Technology.

Dunphy, Dexter C. (1966). The construction of categories for content analysis dictionaries. In Philip J. Stone;
Dexter C. Dunphy; Marshall S. Smith; & Daniel M. Ogilvie, The General Inquirer: A computer approach to
content analysis (pp. 134–168). Cambridge, MA: MIT Press.

Page 44 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Durig, Alex. (1995). The event frame. Research Studies in Symbolic Interactionism, 7, 243–266.

Dziurzynski, Patricia S. (1977). Development of a content analytic instrument for advertising appeals used in
prime time television commercials. Unpublished master’s thesis, University of Pennsylvania.

Eco, Umberto. (1994). The limits of interpretation. Bloomington: Indiana University Press.

Eigen, Manfred J.; McCaskill, John; & Schuster, Peter. (1988). Molecular quasi-species. Journal of Physical
Chemistry, 92, 6881–6891.

Ekman, Paul, & Friesen, Wallace V. (1968). Nonverbal behavior in psychotherapy research. In John Shlien
(Ed.), Research in psychotherapy (Vol. 3, pp. 179–216). Washington, DC: American Psychological
Association.

Ekman, Paul; Friesen, Wallace V.; & Taussig, Thomas G. (1969). Vid-R and SCAN: Tools and methods for
the automated analysis of visual records. In George Gerbner; Ole R. Holsti; Klaus Krippendorff; William J.
Paisley; & Philip J. Stone (Eds.), The analysis of communication content: Developments in scientific theories
and computer techniques (pp. 297–312). New York, NY: Wiley.

Ellison, John W. (1965). Computers and the testaments. In Proceedings, Conference on Computers for the
Humanities (pp. 64–74). New Haven, CT: Yale University Press.

Esbensen, Kim H. (2010). Multivariate data analysis in practice (5th ed.). Esbjerg, Denmark: Aalborg
University.

Eshbaugh-Soha, Matthew (2010). The tone of local presidential news coverage. Political Communication, 27,
2: 121–140.

Evans, William. (2002). Computer environments for content analysis: Reconceptualizing the roles of humans
and computers. In Orville Vernon Burton (Ed.), Computing in the social sciences and humanities (pp. 67–83).
Urbana: University of Illinois Press.

Eysenbach, Gunther, & Till, James E. (2001). Ethical issues in qualitative research in Internet communities.
British Medical Journal, 323(7321), 1103.

Fan, David P. (1988). Predictions of public opinion from the mass media: Computer content analysis and
mathematical modeling. New York, NY: Greenwood.

Fan, David P., & Cook, R. Dennis. (1997). Predictions of the Michigan Index of Consumer Sentiment from the
press. Unpublished manuscript, University of Minnesota, St. Paul.

Page 45 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Farrall, Kenneth, & Delli Carpini, Michael X. (2005, October). Issue politics, social networks, and the Web
graph. Paper presented at the annual meeting of the Association of Internet Researchers, Chicago, IL.

Feigl, Herbert. (1952). Validation and vindication: An analysis of the nature and the limits of ethical arguments.
In Wilfried Sellars & John Hospers (Eds.), Readings in ethical theory (pp. 667–680). New York, NY: Appleton-
Century-Crofts.

Feinstein, Alvan R., & Cicchetti, Domenic V. (1990). High agreement but low kappa: I. The problem of two
paradoxes. Journal of Clinical Epidemiology, 43(6), 543–549.

Fellbaum, Christiane. (Ed.). (1998). WordNet: An electronic lexical database. Cambridge, MA: MIT Press.

Feng, Guangchao C. (2015). Mistakes and how to avoid mistakes in using intercoder reliability indices.
Methodology, 11, 1:13-22.

Fenton, Frances. (1910). The influence of newspaper presentations on the growth of crime and other anti-
social activity. American Journal of Sociology, 16, 342–371, 538–564.

Fleck, Ludwik. (1979). Genesis and development of a scientific fact. Chicago: University of Illinois Press.
(Original work published 1935)

Fleiss, Joseph L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76,
378–382.

Fleiss, Joseph L. (1981). Statistical methods for rates and proportions. New York, NY: Wiley.

Flesch, Rudolph. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221–233.

Flesch, Rudolph. (1951). How to test readability. New York, NY: Harper & Row.

Flesch, Rudolph. (1974). The art of readable writing (Rev. ed.). New York, NY: Harper & Row.

Foltz, Peter W. (1990). Using latent semantic indexing for information filtering. In Proceedings of the 5th
Conference on Office Information Systems [Special issue]. ACM SIGOIS Bulletin, 11(2–3).

Ford, Sarah; Fallowfield, Leslie; & Lewis, Shon. (1996). Doctor-patient interactions in oncology. Social
Science & Medicine, 42, 1511–1519.

Foster, Charles R. (1938). Editorial treatment of education in the American press. Cambridge, MA: Harvard
University Press.

Page 46 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Freeman, Steven F. (2001). Patterns of executive attention in U.S. auto industry letters to shareholders
1963–1987. Unpublished manuscript, University of Pennsylvania, Wharton School.

Furnas, George W.; Deerwester, Scott C.; Dumais, Susan T.; Landauer, Thomas K.; Harshman, Richard A.;
Streeter, Lynn A.; & Lochbaum, Karen E. (1988). Information retrieval using a singular value decomposition
model of latent semantic structure. In Yves Chiaramella (Ed.), Proceedings of the 11th International ACM
SIGIR Conference on Research and Development in Information Retrieval (pp. 465–480). New York, NY:
Association for Computing Machinery.

Garfield, Eugene. (1955, July 15). Citation indexes for science: A new dimension in documentation through
association of ideas. Science (New Series), 122(3159), 108–111. Accessed at http://scimaps.org/static/docs/
Garfield1955cit.pdf

Garfield, Eugene. (1979). Citation indexing: Its theory and application to science, technology and humanities.
New York, NY: Wiley.

Geller, A.; Kaplan, D.; & Lasswell, Harold D. (1942). An experimental comparison of four ways of coding
editorial content. Journalism Quarterly, 19, 362–370.

Gentzkow, Matthew, & Shapiro, Jesse M. (2010). What Drives Media Slant? Evidence from U.S. Daily
Newspapers. Econometrika, 78, 1, 35–71.

George, Alexander L. (1959a). Propaganda analysis: A study of inferences made from Nazi propaganda in
World War II. Evanston, IL: Row, Peterson.

George, Alexander L. (1959b). Quantitative and qualitative approaches to content analysis. In Ithiel de Sola
Pool (Ed.), Trends in content analysis (pp. 7–32). Urbana: University of Illinois Press.

Gerbner, George. (1958). The social role of the confession magazine. Social Problems, 6, 29–40.

Gerbner, George. (1964). Ideological perspectives and political tendencies in news reporting. Journalism
Quarterly, 41, 495–508.

Gerbner, George. (1969). Toward “cultural indicators”: The analysis of mass mediated public message
systems. In George Gerbner; Ole R. Holsti; Klaus Krippendorff; William J. Paisley; & Philip J. Stone (Eds.),
The analysis of communication content: Developments in scientific theories and computer techniques (pp.
123–132). New York, NY: Wiley.

Gerbner, George. (1985). Mass media discourse: Message system analysis as a component of cultural
indicators. In Teun A. Van Dijk (Ed.), Discourse and communications: New approaches to the analyses of

Page 47 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

mass media discourse and communication (pp. 13–25). Berlin, Germany: Walter de Gruyter.

Gerbner, George; Gross, Larry; Morgan, Michael; & Signorielli, Nancy. (1994). Television violence profile no.
16: The turning point: From research to action. Philadelphia: University of Pennsylvania, Annenberg School
for Communication.

Gerbner, George; Gross, Larry; Morgan, Michael; & Signorielli, Nancy. (1995). Growing up with television:
The cultivation perspective. In Jennings Bryant & Dolf Zillmann (Eds.), Media effects: Advances in theory and
research (pp. 17–41). Mahwah, NJ: Erlbaum.

Gerbner, George; Gross, Larry; Signorielli, Nancy; Morgan, Michael; & Jackson-Beeck, Marilyn. (1979).
Violence profile no. 10: Trends in network television drama and viewer conceptions of social reality,
1967–1978. Philadelphia: University of Pennsylvania, Annenberg School of Communications.

Gerbner, George; Holsti, Ole R.; Krippendorff, Klaus; Paisley, William J.; & Stone, Philip J. (Eds.). (1969). The
analysis of communication content: Developments in scientific theories and computer techniques. New York,
NY: Wiley.

Gerbner, George, & Marvanyi, George. (1977). The many worlds of the world’s press. Journal of
Communication, 27(1), 52–75.

Gergen, Kenneth J. (1985). Social constructionist inquiry: Context and implications. In Kenneth J. Gergen &
Keith E. Davis (Eds.), The social construction of the person (pp. 3–18). New York, NY: Springer-Verlag.

Gergen, Kenneth J. (1991). The saturated self: Dilemmas of identity in contemporary life. New York, NY: Basic
Books.

Gieber, Walter. (1964). News is what newspapermen make it. In Lewis A. Dexter & David M. White (Eds.),
People, society, and mass communication (pp. 173–182). New York, NY: Free Press.

Glaser, Barney G., & Strauss, Anselm L. (1967). The discovery of grounded theory: Strategies for qualitative
research. Chicago, IL: Aldine.

Goodenough, Ward H. (1972). Componential analysis. In James P. Spradley (Ed.), Culture and cognition (pp.
327–343). San Francisco, CA: Chandler.

Goodwin, Charles. (1977). Some aspects of the interaction of speaker and hearer in the construction of the
turn at talk in natural conversation. Unpublished doctoral dissertation, University of Pennsylvania, Annenberg
School of Communications.

Page 48 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Goodwin, Charles. (1981). Conversational organization: Interaction between speakers and hearers. New
York, NY: Academic Press.

Gottschalk, Louis A. (1995). Content analysis of verbal behavior: New findings and clinical applications.
Hillsdale, NJ: Erlbaum.

Gottschalk, Louis A., & Bechtel, Robert J. (1982). The measurement of anxiety through the computer analysis
of verbal samples. Comprehensive Psychiatry, 23, 364–369.

Gottschalk, Louis A., & Gleser, Goldine C. (1969). The measurement of psychological states through the
content analysis of verbal behavior. Berkeley: University of California Press.

Graham, Todd, & Witschge, Tamara. (2003). In search of online deliberations: Towards a new method for
examining the quality of online discussions. Communications, 28, 173–204.

Groth, Otto. (1948). Die Geschichte der deutschen Zeitungswissenschaft, Probleme und Methoden. Munich,
Germany: Konrad Weinmayer.

Guetzkow, Harold. (1950). Unitizing and categorizing problems in coding qualitative data. Journal of Clinical
Psychology, 6, 47–58.

Guo, Lei; Mays, Kate; Sameki, Mehrnoosh; & Betke, Margit. (2017, May). From crowdsourcing to
crowdcoding: An alternative approach to annotate big data in communication research. Paper presented at
the 67th Annual Conference of the International Communication Association, San Diego, CA.

Gwet, Kilem (2002). Kappa statistic is not satisfactory for assessing the extent of agreement between
raters. Statistical Methods for Inter-Rater Reliability Assessment, 1. Accessed at http://www.agreestat.com/
research_papers/kappa_statistic_is_not_satisfactory.pdf

Gwet, Kilem Li. (2008). Computing inter-rater reliability and its variance in the presence of high agreement.
British Journal of Mathematical and Statistical Psychology, 61, 29–48.

Hackett, Robert, & Zhao, Yaezhi. (1994). Challenging the master narratives: Peace protest and opinion/
editorial discourse in the U.S. during the Gulf War. Discourse & Society, 5, 509–511.

Harary, Frank; Norman, Robert Z.; & Cartwright, Dorwin P. (1965). Structural models: An introduction to the
theory of directed graphs. New York, NY: Wiley.

Harris, Karen L. (1996). Content analysis in negotiation research: A review and guide. Behavioral Research
Methods, Instruments, & Computers, 28, 458–467.

Page 49 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Hart, Roderick P. (1985). Systematic analysis of political discourse: The development of DICTION. In Keith
R. Sanders; Lynda Lee Kaid; & Dan Nimmo (Eds.), Political communication yearbook 1984 (pp. 97–134).
Carbondale: Southern Illinois University Press.

Harwood, Jake, & Giles, Howard. (1992). Don’t make me laugh: Age representations in a humorous context.
Discourse & Society, 3, 403–436.

Hatch, David L., & Hatch, Mary. (1947). Criteria of social status as derived from marriage announcements in
the New York Times. American Sociological Review, 12, 396–403.

Hawk, Lee. (1997). “Listenability” of network newscasts: The change over sixteen years. Feedback:
Broadcast Education Association, 38(2), 18–21.

Hayes, Andrew F. (n.d.). My macros and code for SPSS and SAS. Accessed at http://www.afhayes.com/spss-
sas-and-mplus-macros-and-code.html

Hayes, Andrew F., & Krippendorff, Klaus. (2007). Answering the call for a standard reliability measure for
coding data. Communication Methods and Measures, 1, 77–89. Accessed at http://www.afhayes.com/public/
cmm2007.pdf

Hays, David C. (1960). Automatic content analysis. Santa Monica, CA: Rand Corporation.

Hays, David C. (1969). Linguistic foundations for a theory of content analysis. In George Gerbner; Ole R.
Holsti; Klaus Krippendorff; William J. Paisley; & Philip J. Stone (Eds.), The analysis of communication content:
Developments in scientific theories and computer techniques (pp. 57–67). New York, NY: Wiley.

Heclo, Hugh. (1978). Issue networks and the executive establishment. In Anthony King (Ed.), The New
American Political System. Washington, DC: American Enterprise Institute for Public Policy Research.

Heise, David R. (1965). Semantic differential profiles for 1,000 most frequent English words. Psychological
Monographs, 79(8, Whole No. 601).

Heise, David R. (1995). Specifying event content in narratives. Unpublished manuscript, Indiana University.
Accessed at http://www.indiana.edu/~socpsy/papers/EventContent.html

Hellsten, Iina, & Leydesdorff, Loet. (2017). Automated analysis of topic-actor networks on Twitter: New
approach to the analysis of socio semantic networks. Unpublished manuscript, University of Amsterdam,
Amsterdam School of Communication Research (ASCoR). Preprint available at https://arxiv.org/abs/
1711.08387

Page 50 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Herdan, Gustav. (1960). Type-token mathematics: A textbook of mathematical linguistics. The Hague, The
Netherlands: Mouton.

Herma, Hans; Kriss, Ernst; & Shor, Joseph. (1943). Freud’s theory of the dream in American textbooks.
Journal of Abnormal and Social Psychology, 38, 319–334.

Herrnstein, Richard J., & Murray, Charles. (1994). The bell curve; Intelligence and class structure in American
life. New York, NY: Simon and Schuster.

Hillman, James. (1995). Kinds of power: A guide to its intelligent uses. New York, NY: Doubleday.

Holley, W., & Guilford, J. P. (1964). A note on the G-index of agreement. Educational and Psychological
Measurement, 24, 749–754.

Holsti, Ole R. (1962). The belief system and national images: John Foster Dulles and the Soviet Union.
Unpublished doctoral dissertation, Stanford University.

Holsti, Ole R. (1969). Content analysis for the social sciences and humanities. Reading, MA: Addison-Wesley.

Holsti, Ole R.; Brody, Richard A.; & North, Robert C. (1965). Measuring affect and action in international
reaction models: Empirical materials from the 1962 Cuban crisis. Peace Research Society Papers, 2,
170–190.

Hopper, Robert; Koch, Susan; & Mandelbaum, Jennifer. (1986). Conversation analysis methods. In Donald G.
Ellis & William A. Donahue (Eds.), Contemporary issues in language and discourse processes (pp. 169–186).
Hillsdale, NJ: Erlbaum.

Houle, Paul. (2002). [Contribution to a discussion of neural network content analysis programs on
[email protected]], March 30.

Howe, Jeff. (2006, June 1). The rise of crowdsourcing. Wired.

Howe, Jeff. (2008). Crowdsourcing: How the power of the crowd is driving the future of business. New York,
NY: Random House.

Hsueh, Pei-Yun; Melville, Prem; & Sindhwani, Vikas. (2009). Data quality from crowdsourcing: A study
of annotation selection criteria. Proceedings of the NAACL HLT Workshop on Active Learning for Natural
Language Processing. Boulder, CO. Pages 27-35.

Hubert, Lawrence. (1977). Kappa revisited. Psychological Bulletin, 84, 289–297.

Page 51 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Iker, Howard P. (1974). Select: A computer program to identify associationally rich words for content analysis:
I. Statistical results. Computers and the Humanities, 8, 313–319.

Iker, Howard P. (1975). Words system manual. Rochester, NY: Computer Printout.

Iker, Howard P., & Harway, Norman I. (1969). A computer system approach toward the recognition and
analysis of content. In George Gerbner; Ole R. Holsti; Klaus Krippendorff; William J. Paisley; & Philip J. Stone
(Eds.), The analysis of communication content: Developments in scientific theories and computer techniques
(pp. 381–405). New York, NY: Wiley.

Innis, Harold A. (1951). The bias of communication. Toronto, Canada: University of Toronto Press.

Institute for Propaganda Analysis. (1937). How to detect propaganda. Propaganda Analysis, 1, 5–8.

Jamieson, Kathleen Hall. (1984). Packaging the presidency: A history and criticism of presidential campaign
advertising. New York, NY: Oxford University Press.

Jamieson, Kathleen Hall. (1998). A preliminary report on so-called “negative” ads and campaigning, October
7, 1998. Philadelphia: University of Pennsylvania, Annenberg School for Communication.

Janda, Kenneth. (1969). A microfilm and computer system for analyzing comparative politics literature. In
George Gerbner; Ole R. Holsti; Klaus Krippendorff; William J. Paisley; & Philip J. Stone (Eds.), The analysis
of communication content: Developments in scientific theories and computer techniques (pp. 407–435). New
York, NY: Wiley.

Janis, Irving L. (1965). The problem of validating content analysis. In Harold D. Lasswell; Nathan Leites; &
Associates (Eds.), Language of politics: Studies in quantitative semantics (pp. 55–82). Cambridge, MA: MIT
Press. (Original work published 1943)

Janis, Irving L., & Fadner, Raymond H. (1965). The coefficient of imbalance. In Harold D. Lasswell; Nathan
Leites; & Associates (Eds.), Language of politics: Studies in quantitative semantics (pp. 153–169).
Cambridge, MA: MIT Press. (Original work published 1943)

Janson, Svante, & Vegelius, Jan. (1979). On generalizations of the G index and the phi coefficient to nominal
scales. Multivariate Behavioral Research, 14, 255–269.

Jefferson, Gail. (1978). Sequential aspects of storytelling in conversation. In Jim Schenkein (Ed.), Studies in
the organization of conversational interaction (pp. 219–248). New York, NY: Free Press.

Jensen, Klaus Bruhn. (2006). News as ideology: Economic statistics and political ritual in television network

Page 52 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

news. Journal of Communication, 37(1), 8–27.

Johnson-Laird, Philip N.; Herrmann, Douglas J.; & Chaffin, Roger. (1984). Only connections: A critique of
semantic networks. Psychological Bulletin, 96, 292–315.

Josephson, John R., & Josephson, Susan G. (1994). Introduction. In John R. Josephson & Susan G.
Josephson (Eds.), Abductive inference: Computation, philosophy, technology. New York, NY: Cambridge
University Press.

Kamada, Tomihisa, & Kawai, Satoru. (1989). An algorithm for drawing general undireted graphs. Information
Processing Letters, 31(1), 7-15.

Kaplan, Abraham. (1964). The conduct of inquiry: Methodology for behavioral science. San Francisco, CA:
Chandler.

Kaplan, Abraham, & Goldsen, Joseph M. (1965). The reliability of content analysis categories. In Harold D.
Lasswell; Nathan Leites; & Associates (Eds.), Language of politics: Studies in quantitative semantics (pp.
83–112). Cambridge, MA: MIT Press.

Katz, Elihu; Gurevitch, Michael; Danet, Brenda; & Peled, Tsiyona. (1969). Petitions and prayers: A content
analysis of persuasive appeals. Social Forces, 47, 447–463.

Katz, Elihu; Gurevitch, Michael; Peled, Tsiyona; & Danet, Brenda. (1969). Exchanges with clients: A
diagnostic approach to organizations and professions. Human Relations, 22, 309–324.

Katz, Jonathan N. (1995). The invention of heterosexuality. New York, NY: Dutton.

Kelly, Edward F., & Stone, Philip J. (1975). Computer recognition of English word senses. Amsterdam, The
Netherlands: North-Holland.

Kenski, Kate (with Jamieson, Kathleen Hall, & Romer, Dan). (1999). Public smarter than press, pundits, or
academics about so-called “negative” ads and campaigning [Survey, October 6]. Philadelphia: University of
Pennsylvania, Annenberg Public Policy Center.

Kim, Joohan, & Gamson, William A. (1999, May). Computer assisted frame analysis (CAFA) of abortion
issues: A pilot study. Paper presented at the 49th Annual Meeting of the International Communication
Association, San Francisco, CA.

Klausner, Samuel Z. (1968). Two centuries of child-rearing manuals [Technical report to the Joint Commission
on Mental Health of Children]. Philadelphia: University of Pennsylvania, Department of Sociology.

Page 53 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Klein, Hans K. (1999). Tocqueville in cyberspace: Using the Internet for citizen associations. The Information
Society, 15(4), 213–220. Accessed at http://www.indiana.edu/~tisj/readers/full-text/15-4%20klein.pdf

Klein, Malcolm W., & Maccoby, Nathan. (1954). Newspaper objectivity in the 1952 campaign. Journalism
Quarterly, 31, 285–296.

Kleinnijenhuis, Jan. (1990, June). Applications of graph theory to cognitive communication research. Paper
presented at the 40th Annual Meeting of the International Communication Association, Dublin, Ireland.

Kleinnijenhuis, Jan; De Ridder, Jan A.; & Rietberg, Edwald M. (1997). Reasoning in economic discourse: An
application of the network approach to the Dutch press. In Carl W. Roberts (Ed.), Text analysis for the social
sciences: Methods for drawing statistical inferences from texts and transcripts (pp. 191–207). Mahwah, NJ:
Erlbaum.

Klir, Jiri, & Valach, Miroslav. (1965). Language as a means of communication between man and machine. In
Jiri Klir & Miroslav Valach, Cybernetic modeling (pp. 315–373). Princeton, NJ: D. Van Nostrand.

Kolbe, Richard H., & Burnett, Melissa S. (1991). Content-analysis research: An examination of applications
with directives for improving research reliability and objectivity. Journal of Consumer Research, 18, 243–250.

Koppel, Moshe; Argamon, Shlomo; & Shimoni, Anat Rachel. (2002). Automatically categorizing written texts
by author gender. Literary and Linguistic Computing, 17(4), 401–412.

Kracauer, Siegfried. (1947). Propaganda and the Nazi war film. In From Caligari to Hitler: A psychological
history of the German film (pp. 308–331). Princeton, NJ: Princeton University Press.

Kracauer, Siegfried. (1952–1953). The challenge of quantitative content analysis. Public Opinion Quarterly,
16, 631–642.

Krendel, Ezra S. (1970). A case study of citizen complaints as social indicators. IEEE Transactions on
Systems Science and Cybernetics, 6, 267–272.

Krippendorff, Klaus. (1967). An examination of content analysis: A proposal for a general framework and an
information calculus for message analytic situations. Unpublished doctoral dissertation, University of Illinois,
Urbana.

Krippendorff, Klaus. (1969a). Models of messages: Three prototypes. In George Gerbner; Ole R. Holsti;
Klaus Krippendorff; William J. Paisley; & Philip J. Stone (Eds.), The analysis of communication content:
Developments in scientific theories and computer techniques (pp. 69–106). New York, NY: Wiley.

Page 54 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Krippendorff, Klaus. (1969b). Theories and analytical constructs: Introduction. In George Gerbner; Ole R.
Holsti; Klaus Krippendorff; William J. Paisley; & Philip J. Stone (Eds.), The analysis of communication content:
Developments in scientific theories and computer techniques (pp. 3–16). New York, NY: Wiley.

Krippendorff, Klaus. (1970a). Bivariate agreement coefficients for reliability data. In Edgar F. Borgatta &
George W. Bohrnstedt (Eds.), Sociological methodology 1970 (pp. 139–150). San Francisco, CA: Jossey-
Bass.

Krippendorff, Klaus. (1970b). Estimating the reliability, systematic error and random error of interval data.
Educational and Psychological Measurement, 30, 61–70.

Krippendorff, Klaus. (1970c). The expression of value in political documents. Journalism Quarterly, 47,
510–518.

Krippendorff, Klaus. (1970d). On generating data in communication research. Journal of Communication,

20(3), 241–269. Accessed at http://repository.upenn.edu/asc_papers/273

Krippendorff, Klaus. (1971). Reliability of recording instructions: Multivariate agreement for nominal data.
Behavioral Science, 16, 222–235.

Krippendorff, Klaus. (1978). Reliability of binary attribute data. Biometrics, 34, 142–144. Accessed at
http://www.jstor.org/pss/2529602

Krippendorff, Klaus. (1980a). Clustering. In Peter R. Monge & Joseph N. Cappella (Eds.), Multivariate
techniques in communication research (pp. 259–308). New York, NY: Academic Press.

Krippendorff, Klaus. (1980b). Content analysis: An introduction to its methodology. Beverly Hills, CA: Sage.

Krippendorff, Klaus. (1986). Information theory: Structural models for qualitative data. Beverly Hills, CA: Sage.

Krippendorff, Klaus. (1987). Association, agreement and equity. Quality and Quantity, 21, 109–123.

Krippendorff, Klaus. (1991). Reconstructing (some) communication research methods. In Frederick Steier
(Ed.), Research and reflexivity (pp. 115–142). Newbury Park, CA: Sage.

Krippendorff, Klaus. (1992, May). Recent developments in reliability analysis. Paper presented at the 42nd
Annual Meeting of the International Communication Association, Miami, FL. Accessed at
http://repository.upenn.edu/asc_papers/44

Krippendorff, Klaus. (1993). The past of communication’s hoped-for future. Journal of Communication, 43(3),
34–44.

Page 55 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Krippendorff, Klaus. (1995a). On the reliability of unitizing continuous data. In Peter V. Marsden (Ed.),
Sociological methodology, 1995 (Vol. 25, pp. 47–76). Cambridge, MA: Blackwell.

Krippendorff, Klaus. (1995b). Undoing power. Critical Studies in Mass Communication, 12, 101–132.

Krippendorff, Klaus. (2004a). Content analysis (2nd ed.). Thousand Oaks, CA: Sage.

Krippendorff, Klaus. (2004b). Measuring the reliability of qualitative text analysis data. Quality and Quantity,
38, 787–800. Accessed at http://repository.upenn.edu/asc_papers/42

Krippendorff, Klaus. (2005). Writing: Monologue, dialogue, and ecological narrative. In Michael B. Hinner
(Ed.), Introduction to business communication (pp. 119–159). Frankfurt, Germany: Lang. Accessed at
http://repository.upenn.edu/asc_papers/93

Krippendorff, Klaus. (2006). The dialogical reality of meaning. American Journal of SEMIOTICS, 19(1–4),
17–34. Accessed at http://repository.upenn.edu/asc_papers/51

Krippendorff, Klaus. (2008). Systematic and random disagreement and the reliability of nominal data.
Communication Methods and Measures, 2(4), 323–338. Accessed at http://repository.upenn.edu/asc_papers/
205

Krippendorff, Klaus. (2011a). Agreement and information in the reliability of coding. Communication Measures
and Methods, 5(2), 93–112. Accessed at http://repository.upenn.edu/asc_papers/278

Krippendorff, Klaus. (2011b). A dissenting view on so-called paradoxes of reliability coefficients.

Communication Yearbook 36: Chapter 20, pp. 481-499 in C. T. Salmond, (Ed.). New York, NY: Routledge.

Krippendorff, Klaus. (2016). Algorithm for bootstrapping a distribution of α. Retrieved from

http://web.asc.upenn.edu/usr/krippendorff/boot.c-Alpha.pdf

Krippendorff, Klaus. (2017). Three concepts to retire. Annals of the International Communication Association
41(1), 92-99.

Krippendorff, Klaus, & Bock, Mary Angela. (Eds.). (2009). The content analysis reader. Thousand Oaks, CA:
Sage.

Krippendorff, Klaus, & Craggs, Richard. (2016). The reliability of multi-valued coding of data. Communication
Methods and Measures 10(4), 181-198.

Krippendorff, Klaus, & Eleey, Michael. (1986). Monitoring the symbolic environment of organizations. Public
Relations Review, 12(1), 13–36.

Page 56 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Krippendorff, Klaus; Mathet, Yann; Bouvry, Stéphane & Widlöcher, Antoine (2016). On the reliability of
unitizing textual continua: Further developments. Quality &Quantity 50, 2347-2364. Retrieved from
https://link.springer.com/article/10.1007/s11135-015-0266-1

Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika 2(3),
151–160.

Kuhn, Deanna. (1991). The skills of argument. New York, NY: Cambridge University Press.

Labov, William. (1972). Sociolinguistic patterns. Philadelphia: University of Pennsylvania Press.

Laffal, Julius. (1993). A concept dictionary of English. Essex, CT: Galley.

Lakoff, George. (1987). Women, fire, and dangerous things: What categories reveal bout the mind. Chicago,
IL: University of Chicago Press.

Landis, H. H., & Burtt, Harold E. (1924). A study of conversations. Journal of Comparative Psychology, 4,
81–89.

Landis, J. Richard, & Koch, Gary H. (1977). An application of hierarchical kappa-type statistics in the
assessment of majority agreement among multiple observers. Biometrics, 33, 363–374.

Lashner, Marilyn A. (1990). Content analysis: A method for analyzing communications. The Expert and the
Law, 10(1), 2–5.

Lasswell, Harold D. (1927). Propaganda technique in the world war. New York, NY: Knopf.

Lasswell, Harold D. (1938). A provisional classification of symbol data. Psychiatry, 1, 197–204.

Lasswell, Harold D. (1941). The World Attention Survey: An exploration of the possibilities of studying
attention being given to the United States by newspapers abroad. Public Opinion Quarterly, 5, 456–462.

Lasswell, Harold D. (1960). The structure and function of communication in society. In Wilbur Schramm (Ed.),
Mass communications (pp. 117–130). Urbana: University of Illinois Press.

Lasswell, Harold D. (1963). Politics: Who gets what, when, how. New York, NY: Meridian.

Lasswell, Harold D. (1965a). Detection: Propaganda detection and the courts. In Harold D. Lasswell; Nathan
Leites; & Associates (Eds.), Language of politics: Studies in quantitative semantics (pp. 173–232).
Cambridge, MA: MIT Press.

Page 57 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Lasswell, Harold D. (1965b). Why be quantitative? In Harold D. Lasswell; Nathan Leites; & Associates (Eds.),
Language of politics: Studies in quantitative semantics (pp. 40–52). Cambridge, MA: MIT Press. (Original
work published 1949)

Lasswell, Harold D., & Kaplan, Abraham. (1950). Power and society: A framework for political inquiry. New
Haven, CT: Yale University Press.

Lasswell, Harold D.; Leites, Nathan; & Associates. (Eds.). (1965). Language of politics: Studies in quantitative
semantics. Cambridge, MA: MIT Press.

Lasswell, Harold D.; Lerner, Daniel; & Pool, Ithiel de Sola. (1952). The comparative study of symbols.
Stanford, CA: Stanford University Press.

Latour, Bruno, & Woolgar, Steve. (1986). Laboratory life: The construction of scientific facts. Princeton, NJ:
Princeton University Press.

Lazarsfeld, Paul F.; Berelson, Bernard; & Gaudet, Hazel. (1948). The people’s choice: How the voter makes
up his mind in a presidential campaign. New York, NY: Columbia University Press.

Lee, Chin-Chuan; Chan, Joseph Man; Pan, Zhongdang; & So, Clement Y. K. (2002). Global media spectacle:
News war over Hong Kong. Albany: State University of New York Press.

Leetaru, Kalev H. (2011). Data mining methods for the content analyst: An introduction to the computational
analysis of content. New York, NY: Routledge.

Lehmann, Pola; & malisazobel (2017). Chancen und Grenzen der Schwarnintelligenz. Werkstattbericht aus
dem Crowd-Coding-Project. WissenschaftsZentrum Berlin für Sozialforschung. March 9, 2017.

Leites, Nathan; Bernaut, Elsa; & Garthoff, Raymond L. (1951). Politburo images of Stalin. World Politics, 3,
317–339.

Leites, Nathan, & Pool, Ithiel de Sola. (1942). Communist propaganda in reaction to frustration (Document
No. 27). Washington DC: Library of Congress, Experimental Division for Study of Wartime Communications.

Leskovec, Jure; Backstrom, Lars; & Kleinberg, Jon. (2017). Meme-tracking and the dynamics of the news
cycle. Paper presented at the 67th Annual Conference of the International Communication Association, May
25-29, San Diego, CA.

Lindsay, Robert K. (1963). Inferential memory as the basis of machines which understand natural language.
In Edward A. Feigenbaum & Julian Feldman (Eds.), Computers and thought (pp. 217–233). New York, NY:

Page 58 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

McGraw-Hill.

Lippmann, Walter. (1922). Public opinion. New York, NY: Macmillan.

Löbl, Eugen. (1903). Kultur und Presse. Leipzig, Germany: Duncker & Humblot.

Loeventhal, Leo. (1944). Biographies in popular magazines. In Paul F. Lazarsfeld & Frank N. Stanton (Eds.),
Radio research 1942–1943 (pp. 507–548). New York, NY: Duell, Sloan & Pearce.

Lombard, Matthew; Snyder-Duch, Jennifer; & Bracken, Cheryl Campanella. (2002). Content analysis in
mass communication research: An assessment and reporting of intercoder reliability. Human Communication
Research, 28, 587–604.

Lorr, Maurice, & McNair, Douglas M. (1966). Methods relating to evaluation of therapeutic outcome. In Louis
A. Gottschalk & Arthur H. Auerbach (Eds.), Methods of research in psychotherapy (pp. 573–594). Englewood
Cliffs, NJ: Prentice Hall.

Lowe, Will. (2003). Software for content analysis: A review. Cambridge, MA: Harvard University. Accessed at
http://kb.ucla.edu/system/datas/5/original/content_analysis.pdf

Lynch, Kevin. (1965). The image of the city. Cambridge, MA: MIT Press.

Maccoby, Nathan; Sabghir, F. O.; & Cushing, B. (1950). A method for the analysis of news coverage of
industry. Public Opinion Quarterly, 14, 753–758.

MacQueen, Kathleen M.; McLellan, Eleanor; Kay, Kelly; & Milstein, Bobby. (1998). Codebook development
for team-based qualitative analysis. Cultural Anthropology Methods, 10(2), 31–36.

Mahl, George F. (1959). Exploring emotional states by content analysis. In Ithiel de Sola Pool (Ed.), Trends in
content analysis (pp. 89–130). Urbana: University of Illinois Press.

Mann, Mary B. (1944). Studies in language behavior: III. The quantitative differentiation of samples of written
language. Psychological Monographs, 56(2), 41–47.

Manning, Christpher D.; Raghavan, Prabhakar; & Schütze, Hinrich. (2008). Introduction to information
retrieval. Cambridge, MA: Cambridge University Press.

Markov, Andrei A. (1913). Essai d’une recherche statistique sur le texte du roman Eugene Onegin illustrant la
liaison des epreuves en chaine (Russian). Bulletin de L’Académie Impériale des Sciences de St. Pétersbourg
6(7), 153–162.

Page 59 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Martin, Helen. (1936). Nationalism and children’s literature. Library Quarterly, 6, 405–418.

Martindale, Colin. (1990). The clockwork muse: The predictability of artistic change. New York, NY: Basic
Books.

Maruyama, Margoroh. (1963). The second cybernetics: Deviation-amplifying mutual causal processes.
American Scientist, 51, 164–179.

Mathews, Byron C. (1910). A study of a New York daily. Independent, 68, 82–86.

Maxwell, A. E. (1970). Comparing the classification of subjects by two independent judges. British Journal of
Psychiatry, 116, 651–655.

McClelland, David C. (1958). The use of measures of human motivation in the study of society. In John W.
Atkinson (Ed.), Motives in fantasy, action and society (pp. 518–552). Princeton, NJ: D. Van Nostrand.

McClelland, David C.; Atkinson, John W.; Clark, Russell A.; & Lowell, Edgar L. (1992). A scoring manual for
the achievement motive. In Charles P. Smith (Ed.), Motivation and personality: Handbook of thematic content
analysis (pp. 153–178). Cambridge, MA: Cambridge University Press.

McCombs, Maxwell, & Shaw, Donald L. (1972). The agenda-setting function of mass media. Public Opinion
Quarterly, 36, 176–187.

McCombs, Maxwell; Shaw, Donald L.; & Weaver, David. (1997). Communication and democracy: Exploring
the intellectual frontiers in agenda-setting theory. Mahwah, NJ: Erlbaum.

McDiarmid, John. (1937). Presidential inaugural addresses: A study in verbal symbols. Public Opinion
Quarterly, 1, 79–82.

McKnight, Katherine S., & Walberg, Herbert J. (1994). Neural network analysis of student essays. Journal of
Research and Development in Education, 32, 26–31.

McMillan, Sally J. (2000). The microscope and the moving target: The challenge of applying content analysis
to the World Wide Web. Journalism and Mass Communication Quarterly, 77, 80–98.

McTavish, Donald D.; Litkowski, Kenneth C.; & Schrader, Susan. (1997). A computer content analysis
approach to measuring social distance in residential organizations for older people. Social Science Computer
Review, 15, 170–180.

McTavish, Donald D., & Pirro, Ellen B. (1990). Contextual content analysis. Quality and Quantity, 24,
245–265.

Page 60 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Merrill, John C. (1962). The image of the United States in ten Mexican dailies. Journalism Quarterly, 39,
203–209.

Merritt, Richard L. (1966). Symbols of American community, 1735–1775. New Haven, CT: Yale University
Press.

Merten, Klaus. (1991). Inhaltsanalyse: Eine Einführung in Theorie, Methode und Praxis. Opladen, Germany:
Westdeutscher Verlag.

Miles, Josephine. (1951). The continuity of English poetic language. In University of California publications in
English (pp. 517–535). Berkeley: University of California Press.

Miller, George A. (1951). Language and communication. New York, NY: McGraw-Hill.

Miller, George A.; Beckwith, Richard; Fellbaum, Christiane; Gross, Derek; Miller, Katherine J.; & Tengi,
Randee I. (1993). Five papers on WordNet (CSL Report 43). Princeton, NJ: Princeton University, Cognitive
Science Laboratory.

Miller, Kevin J.; Fullmer, Steven L.; & Walls, Richard T. (1996). A dozen years of mainstreaming literature: A
content analysis. Exceptionality, 6(2), 99–109.

Mokotoff, Gary. (1997). Soundexing and genealogy. Avotaynu. Accessed at http://www.avotaynu.com/

soundex.htm

Monge, Peter R., & Contractor, Noshir S. (2003). Theories of communication networks. New York, NY: Oxford
University Press.

Monge, Peter R., & Eisenberg, Erik M. (1987). Emergent communication networks. In Fredric M. Jablin; Linda
L. Putnam; Karlene H. Roberts; & Lyman W. Porter (Eds.), Handbook of organizational and management
communication (pp. 204–242). Newbury Park, CA: Sage.

Montgomery, Michael. (1989). Protocol for using LAGS CodeMap and LagsMap microcomputer programs:
CodeMap for a lexical item. Accessed at http://hyde.park.uga.edu/lags/protocol.txt

Montgomery, Michael. (1993). Review: The linguistic atlas of the Gulf states. American Speech, 68(3),
263–318.

Morton, Andrew Q. (1963, November 3). A computer challenges the church. Observer.

Morton, Andrew Q., & Levinson, Michael. (1966). Some indications of authorship in green prose. In Jacob
Leed (Ed.), The computer and literary style (pp. 141–179). Kent, OH: Kent State University Press.

Page 61 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Mosteller, Frederick, & Wallace, David L. (1963). Inference in an authorship problem. Journal of the American
Statistical Association, 58, 275–309.

Mosteller, Frederick, & Wallace, David L. (1964). Inference and disputed authorship: The Federalist. Reading,
MA: Addison-Wesley.

Murray, Edward J.; Auld, Frank, Jr.; & White, Alice M. (1954). A psychotherapy case showing progress but no
decrease in the discomfort-relief quotient. Journal of Consulting Psychology, 18, 349–353.

Murray, Henry A. (1943). Thematic Apperception Test manual. Cambridge, MA: Harvard University Press.

Nacos, Brigitte L.; Shapiro, Robert Y.; Young, John T.; Fan, David P.; Kjellstrand, Torsten; & McCaa, Craig.
(1991). Content analysis of news reports: Comparing human coding and a computer-assisted method.
Communication, 12, 111–128.

Namenwirth, J. Zvi. (1973). Wheels of time and the interdependence of value change in America. Journal of
Interdisciplinary History, 3, 649–683.

Namenwirth, J. Zvi, & Weber, Robert P. (1987). Dynamics of culture. Boston, MA: Allen & Unwin.

Naureckas, Jim (1995, January 1). Racism resurgent: How media let the Bell Curve’s pseudo-science define
the agenda on race. Fair.

Neuendorf, Kimberly A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.

Newell, Allen, & Simon, Herbert A. (1956). The logic theory machine. IRE-Transactions on Information Theory
2(3), 61–79.

Newell, Allen, & Simon, Herbert A. (1963). General Problem Solver: A program that simulates human thought.
In Edward A. Feigenbaum & Julian Feldman (Eds.), Computers and thought (pp. 279–293). New York, NY:
McGraw-Hill.

Niederer, Sabine, M. C. (2013). Global warming is not a crisis! Studying climate change skepticism on the
web. Necsus, 3 (Spring).

Niederer, Sabine, M. C. (2016). Networked content analysis. Amsterdam, The Netherlands: Dissertation
defended at the Amsterdam University of Applies Sciences.

Niederer, Sabine, M. C. (2018, forthcoming). The study of networked content: Five considerations for digital
research in the humanities. In G. Schiuma & D. Carlucci (Eds.). Big data in the arts and humanities: Theory
and practice (pp. 89-100). Boca Raton, FL: CRC Press-Taylor & Francis Group.

Page 62 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Nixon, Raymond B., & Jones, Robert L. (1956). The content of non-competitive newspapers. Journalism
Quarterly, 33, 299–314.

North, Robert C.; Holsti, Ole R.; Zaninovich, M. George; & Zinnes, Dina A. (1963). Content analysis: A
handbook with applications for the study of international crisis. Evanston, IL: Northwestern University Press.

Ogden, Charles Kay. (1937). Basic English and grammatical reform. Cambridge, MA: Orthological Institute.

Osgood, Charles E. (1959). The representation model and relevant research methods. In Ithiel de Sola Pool
(Ed.), Trends in content analysis (pp. 33–88). Urbana: University of Illinois Press.

Osgood, Charles E. (1974a). Probing subjective culture: Part 1. Cross-linguistic tool-making. Journal of
Communication, 24(1), 21–35.

Osgood, Charles E. (1974b). Probing subjective culture: Part 2. Cross-cultural tool using. Journal of
Communication, 24(2), 82–100.

Osgood, Charles E.; Saporta, Sol; & Nunnally, Jum C. (1956). Evaluative assertion analysis. Litera, 3,
47–102.

Osgood, Charles E.; Suci, George J.; & Tannenbaum, Percy H. (1957). The measurement of meaning.
Urbana: University of Illinois Press.

O’Sullivan, Thomas C., Jr. (1961). Factor analysis concepts identified in theoretical writings: An experiment
design. Lexington, MA: Itek Laboratories.

Paisley, William J. (1964). Identifying the unknown communicator in painting, literature and music: The
significance of minor encoding habits. Journal of Communication, 14(4), 219–237.

Palmquist, Michael E.; Carley, Kathleen M.; & Dale, Thomas A. (1997). Applications of computer-aided
text analysis: Analyzing literary and nonliterary texts. In Carl W. Roberts (Ed.), Text analysis for the social
sciences: Methods for drawing statistical inferences from texts and transcripts (pp. 171–189). Mahwah, NJ:
Erlbaum.

Parsons, Talcott. (1951). The social system. New York, NY: Free Press.

Parsons, Talcott, & Bales, Robert F. (1953). The dimensions of action-space. In Talcott Parsons; Robert F.
Bales; & Edward A. Shills (Eds.), Working papers in the theory of action (pp. 63–109). New York, NY: Free
Press.

Patterson, Brian R.; Neupauer, Nichola C.; Burant, Patricia A.; Koehn, Steven C.; & Reed, April T. (1996).

Page 63 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

A preliminary examination of conversation analytic techniques: Rates of inter-transcriber reliability. Western

Journal of Communication, 60, 76–91.

Pearson, Karl et al. (1901). Mathematical contributions to the theory of evolution: IX. On the principle of
homotyposis and its relation to heredity, to variability of the individual, and to that of race. Part I: Homotyposis
in the vegetable kingdom. Philosophical Transactions of the Royal Society, 197(Series A), 285–379.

Pederson, Lee; McDaniel, Susan Leas; Adams, Carol; & Liao, Caisheng. (Eds.). (1989). Linguistic atlas of the
Gulf states: Vol. 3. Technical index. Athens: University of Georgia Press.

Péladeau, Normand. (1996). SimStat for Windows: User’s guide. Montreal, Canada: Provalis Research.

Pennebaker, James W. (1997). Writing about emotional experiences as a therapeutic process. Psychological
Science, 8, 162–166.

Pennebaker, James W.; Francis, Martha E.; & Booth, Roger J. (2001). Linguistic inquiry and word count
(LIWC) (2nd ed., PDF manual). Mahwah, NJ: Erlbaum.

Pennebaker, James W., & Stone, Lori D. (2001, May). LIWC 2001. Contribution to a CATA workshop
presented at the 51st Annual Meeting of the International Communication Association, Washington, DC.

Pennings, Paul, & Keman, Hans. (2002). Towards a new methodology of estimating party policy positions.
Quality and Quantity, 36, 55–79.

Perreault, William D., & Leigh, Lawrence E. (1989). Reliability of nominal data based on qualitative judgments.
Journal of Marketing Research, 26, 135–148.

Pescosolido, Bernice A.; Grauerholz, Elisabeth; & Milkie, Melissa A. (1996). Culture and conflict: The
portrayal of blacks in U.S. children’s picture books through the mid- and late twentieth century. American
Sociological Review, 62, 443–464.

Peter, Jochen, & Lauf, Edmund. (2002). Reliability in cross-national content analysis. Journalism & Mass
Communication Quarterly, 79, 815–832.

Phillips, David P. (1978). Airplane accident fatalities increase just after newspaper stories about murder and
suicide. Science, 201, 748–749.

Piault, Collette. (1965). A methodological investigation of content analysis using electronic computers for data
processing. In Dell Hymes (Ed.), The use of computers in anthropology (pp. 273–293). The Hague, The
Netherlands: Mouton.

Page 64 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Pierce, Bessie L. (1930). Civic attitudes in American school textbooks. Chicago, IL: University of Chicago
Press.

Ploughman, Penelope. (1995). The American print news media “construction” of five natural disasters.
Disasters, 19, 308–326.

Pocklington, Richard, & Best, Michael L. (1997). Cultural evolution and units of selection in replicating text.
Journal of Theoretical Biology, 188, 79–87.

Pool, Ithiel de Sola. (1951). Symbols of internationalism. Stanford, CA: Stanford University Press.

Pool, Ithiel de Sola. (1952a). The prestige papers: A survey of their editorials. Stanford, CA: Stanford
University Press.

Pool, Ithiel de Sola. (1952b). Symbols of democracy. Stanford, CA: Stanford University Press.

Pool, Ithiel de Sola. (Ed.). (1959a). Trends in content analysis. Urbana: University of Illinois Press.

Pool, Ithiel de Sola. (1959b). Trends in content analysis today: A summary. In Ithiel de Sola Pool (Ed.), Trends
in content analysis (pp. 189–233). Urbana: University of Illinois Press.

Pool, Ithiel de Sola; Abelson, Robert P.; & Popkin, Samuel L. (1964). Candidates, issues and strategies: A
computer simulation of the 1960 presidential election. Cambridge, MA: MIT Press.

Popping, Roel. (1988). On agreement indices for nominal data. In William E. Saris & Irmtraut N. Gallhofer
(Eds.), Sociometric research: Data collection and scaling (Vol. 1, pp. 90–105). New York, NY: St. Martin’s.

Popping, Roel. (1997). Computer programs for the analysis of texts and transcripts. In Carl W. Roberts (Ed.),
Text analysis for the social sciences: Methods for drawing statistical inferences from texts and transcripts (pp.
209–221). Mahwah, NJ: Erlbaum.

Popping, Roel. (2000). Computer-assisted text analysis. London, England: Sage.

Posen, Solomon. (1997). The portrayal of the doctor in non-medical literature: The impaired doctor. Medical
Journal of Australia, 166, 48–51.

Potter, W. James, & Levine-Donnerstein, Deborah. (1999). Rethinking reliability and reliability in content
analysis. Journal of Applied Communication Research, 27, 258–284.

Potter, W. James, & Vaughan, Misha. (1997). Antisocial behaviors in television entertainment: Trends and
profiles. Communication Research Reports, 14(1), 116–124.

Page 65 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Quillian, M. Ross. (1968). Semantic memory. In Marvin L. Minsky (Ed.), Semantic information processing (pp.
216–270). Cambridge, MA: MIT Press.

Rainoff, T. J. (1929). Wave-like fluctuations of creative productivity in the development of West-European

physics in the eighteenth and nineteenth centuries. Isis, 12, 287–307.

Rapoport, Anatol. (1969). A system-theoretic view of content analysis. In George Gerbner; Ole R. Holsti;
Klaus Krippendorff; William J. Paisley; & Philip J. Stone (Eds.), The analysis of communication content:
Developments in scientific theories and computer techniques (pp. 17–38). New York, NY: Wiley.

Reynolds, Henry T. (1977). Analysis of nominal data. Beverly Hills, CA: Sage.

Rice, Ronald E. (1990). Hierarchies and clusters in communication and library and information science
journals, 1978–1987. In Christine Borgman (Ed.), Scholarly communication and bibliometrics (pp. 138–153).
Newbury Park, CA: Sage.

Rice, Ronald E.; Borman, Christine L.; & Reeves, Bryan. (1988). Citation networks of communication journals,
1977–1985. Human Communication Research, 15, 256–283.

Rice, Ronald E., & Richards, William D., Jr. (1985). An overview of network analysis methods and programs.
In Brenda Dervin & Melvin J. Voigt (Eds.), Progress in communication sciences (Vol. 6, pp. 105–165).
Norwood, NJ: Ablex.

Richards, William D., Jr., & Rice, Ronald E. (1981). NEGOPY network analysis program. Social Networks, 3,
215–223.

Riffe, Daniel, & Freitag, Alan. (1996). Twenty-five years of content analyses in Journalism & Mass
Communication Quarterly. Paper presented at the annual meeting of the Association for Education in
Journalism and Mass Communication, Anaheim, CA.

Riffe, Daniel; Lacy, Stephen; & Fico, Frederick G. (1998). Analyzing media messages: Using quantitative
content analysis in research. Mahwah, NJ: Erlbaum.

Rijsbergen, Cornelis J. (1979). Information retrieval. London, England: Butterworth.

Roberts, Carl W. (1989). Other than counting words: A linguistic approach to content analysis. Social Forces,
68, 147–177.

Roberts, Carl W. (Ed.). (1997). Text analysis for the social sciences: Methods for drawing statistical inferences
from texts and transcripts. Mahwah, NJ: Erlbaum.

Page 66 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Rogers, Everett M., & Kincaid, D. Lawrence. (1981). Communication networks: Toward a new paradigm for
research. New York, NY: Free Press.

Rogers, Everett M., & Shoemaker, Floyd F. (1971). Communication of innovations. New York, NY: Free Press.

Rogers, Richard. (2002). The Issue Crawler: The makings of live social science on the Web. EASST Review,
21, 3–4, 8–11. Retrieved from http://www.easst.net/review/sept2002/issuecrawler.shtml. Accessed 2017.9.9.

Rogers, Richard. (2004). Information politics on the Web. Cambridge, MA: MIT Press.

Rogers, Richard. (2010). Mapping public Web space with the IssueCrawler. Pp. 89-99 in B. Reber &
C. Brossaud (Eds.). Digital cognitive technologies: Epistemology and the knowledge economy. London,
England: Wiley.

Rogers, Richard, (2017). Foundations of digital methods: Query design. Pp. 75-95 in M. T. Schäfer & K. van
Es (Eds.). The datafied society: Studying culture through data. Amsterdam, The Netherlands: Amsterdam
University Press.

Rogers, Richard; Jansen, Fieke; Stevenson, Michael; & Weltevrede, Esther. (2010). Mapping democracy:
2009—Access to online information and knowledge. Amsterdam, The Netherlands: Digital Methods Initiative.
Retrieved from http://www.giswatch.org/fr/node/158

Romme, A. Georges L. (1995). Boolean comparative analysis of qualitative data. Quality and Quantity, 29,
317–329.

Rosch, Eleanor. (1983). Prototype classification and logical classification: The two systems. In Ellin Kofsky
Scholnick (Ed.), New trends in conceptual representation: Challenges to Piaget’s theory? (pp. 73-86).
Hillsdale, NJ: Erlbaum.

Rosenberg, Stanley D.; Schnurr, Paula P.; & Oxman, Thomas E. (1990). Content analysis: A comparison of
manual and computerized systems. Journal of Personality Assessment, 54, 298–310.

Ruesch, Jürgen, & Bateson, Gregory. (1951). Communication: The social matrix of psychiatry. New York, NY:
W. W. Norton.

Sacks, Harvey. (1974). An analysis of the course of a joke’s telling in conversation. In Joel Sherzer & Richard
Bauman (Eds.), Explorations in the ethnography of speaking (pp. 337–353). London, England: Cambridge
University Press.

Salgado, Jesus F., & Moscoso, Silvia. (1996). Meta-analysis of interrater reliability of job performance ratings

Page 67 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

in validity studies of personnel selection. Perceptual and Motor Skills, 83, 1195–1201.

Salmond, Ann. (1982). Theoretical landscapes. In David Parkin (Ed.), Semantic anthropology (pp. 65–87).
New York, NY: Academic Press.

Samarel, Nelda; Fawcett, Jacqueline; Krippendorff, Klaus; Piacentino, Jayne C.; Eliasof, Barbara; Hughes,
Phyllis; . . . Ziegler, Elaine. (1998). Women’s perceptions of group support and adaptation to breast cancer.
Journal of Advanced Nursing, 28, 1259–1268.

Saris-Gallhofer, Irmtraut N., & Morton, E. L. (1978). A validity study of Holsti’s content analysis procedure.
Quality and Quantity, 12, 131–145.

Schank, Roger C., & Abelson, Robert P. (1977). Scripts, plans, goals and understanding: An inquiry into
human knowledge structures. Hillsdale, NJ: Erlbaum.

Schnurr, Paula P.; Rosenberg, Stanley D.; & Oxman, Thomas E. (1992). Comparison of TAT and free speech
techniques for eliciting source materials in computerized content analysis. Journal of Personality Assessment,
58, 311–325.

Schnurr, Paula P.; Rosenberg, Stanley D.; & Oxman, Thomas E. (1993). Issues in the comparison of
techniques for eliciting source material in computerized content analysis. Journal of Personality Assessment,
61, 337–342.

Schrodt, Philip A.; Davis, Shannon G.; & Weddle, Judith L. (1994). KEDS: A program for the machine coding
of event data. Social Science Computer Review, 12, 561–588.

Schutz, William C. (1958). On categorizing qualitative data in content analysis. Public Opinion Quarterly, 22,
503–515.

Scott, William A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion
Quarterly, 19, 321–325.

Searle, John. (1969). Speech acts: An essay in the philosophy of language. Cambridge, MA: Cambridge
University Press.

Sebeok, Thomas A., & Orzack, Louis H. (1953). The structure and content of Cheremis charms. Anthropos,
48, 369–388.

Sebeok, Thomas A., & Zeps, Valdis J. (1958). An analysis of structured content with application of electronic
computer research in psycholinguistics. Language and Speech, 1, 181–193.

Page 68 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Sedelow, Sally Y. (1967). Stylistic analysis. Santa Monica, CA: SDC.

Sedelow, Sally Y. (1989). The interlingual thesaurus model for global technical communication: Research
results. (ERIC Document Reproduction Service No. ED324936).

Sedelow, Sally Y., & Sedelow, Walter A., Jr. (1966). A preface to computational stylistics. In Jacob Leed (Ed.),
The computer and literary style (pp. 1–13). Kent, OH: Kent State University Press.

Sedelow, Sally Y., & Sedelow, Walter A., Jr. (1986). Thesaural knowledge representation. In Advances in
lexicology: Proceedings of the Second Annual Conference of the UW Centre for the New Oxford English
Dictionary (pp. 29–43). Waterloo, ON, Canada: UW Centre for the New Oxford English Dictionary.

Selltiz, Claire; Jahoda, Marie; Deutsch, Morton; & Cook, Stuart W. (1964). Research methods in social
relations. New York, NY: Holt, Rinehart & Winston.

Shanas, Ethel. (1945). The American Journal of Sociology through fifty years. American Journal of Sociology,
50, 522–533.

Shannon, Claude E., & Weaver, Warren. (1949). The mathematical theory of communication. Urbana:
University of Illinois Press.

Shapiro, Gilbert. (1997). The future of coders: Human judgments in a world of sophisticated software. In Carl
W. Roberts (Ed.), Text analysis for the social sciences: Methods for drawing statistical inferences from texts
and transcripts (pp. 225–238). Mahwah, NJ: Erlbaum.

Shapiro, Gilbert, & Markoff, John. (1997). A matter of definition. In Carl W. Roberts (Ed.), Text analysis for the
social sciences: Methods for drawing statistical inferences from texts and transcripts (pp. 9–31). Mahwah, NJ:
Erlbaum.

Shneidman, Edwin S. (1963). The logic of politics. In Leon Arons & Mark A. May (Eds.), Television and human
behavior (pp. 178–199). Englewood Cliffs, NJ: Prentice Hall.

Shneidman, Edwin S. (1966). The logics of communication: A manual for analysis. China Lake, CA: U.S.
Naval Ordnance Test Station.

Shneidman, Edwin S. (1969). Logical content analysis: An explication of styles of concludifying. In George
Gerbner; Ole R. Holsti; Klaus Krippendorff; William J. Paisley; & Philip J. Stone (Eds.), The analysis of
communication content: Developments in scientific theories and computer techniques (pp. 261–279). New
York, NY: Wiley.

Page 69 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Shneidman, Edwin S., & Farberow Norman L. (1957). Some comparisons between genuine and simulated
suicide notes. Journal of General Psychology, 56, 251-256.

Shuman, Ronald B. (1937). Identification elements of advertising slogans. Southwestern Social Science
Quarterly, 17, 342–352.

Siegel, Sydney, & Castellan, N. John. (1988). Nonparametric statistics for the behavioral sciences (2nd ed.).
Boston, MA: McGraw-Hill.

Simon, Herbert A. (1957). Models of man. New York, NY: Wiley.

Simonton, Dean K. (1994). Computer content analysis of melodic structure: Classical composers and their
compositions. Psychology of Music, 22, 31–43.

Simpson, George E. (1934). The Negro in the Philadelphia press. Unpublished doctoral dissertation,
University of Pennsylvania.

Singer, J. David. (1964). Soviet and American foreign policy attitudes: A content analysis of elite articulations.
Journal of Conflict Resolution, 8, 424–485.

Sixsmith, Judith, & Murray, Craig D. (2001). Ethical issues in the documentary data analysis of Internet posts
and archives. Qualitative Health Research, 11(3), 423–432.

Skalski, Paul D. (2002). Message archives. In Kimberly A. Neuendorf, The content analysis guidebook (pp.
215–218). Thousand Oaks, CA: Sage.

Sloman, Steven, & Fernbach, Philip. (2017). The knowledge illusion; Why we never think alone. New York,
NY: Riverhead Books.

Sluga, Hans D., and David G. Stern (Eds.). (1996). The Cambridge companion to Wittgenstein. Cambridge,
MA: Cambridge University Press.

Smith, Charles P. (1992a). Introduction: Inferences from verbal material. In Charles P. Smith (Ed.), Motivation
and personality: Handbook of thematic content analysis. (pp. 1–17). Cambridge, MA: Cambridge University
Press.

Smith, Charles P. (Ed.). (1992b). Motivation and personality: Handbook of thematic content analysis.
Cambridge, MA: Cambridge University Press.

Smythe, Dallas W. (1954). Some observations on communications theory. Audio-Visual Communication

Review, 2, 248–260.

Page 70 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

So, Clement Y. K. (1988). Citation patterns of core communication journals. Human Communication
Research, 15, 236–255.

So, Clement Y. K. (1995). Mapping the intellectual landscape of communication studies: An evaluation of
its disciplinary status. Unpublished doctoral dissertation, University of Pennsylvania, Annenberg School for
Communication.

Spearman, Charles (1904). The proof and measurement of association between two things. American Journal
of Psychology. 15, 72–101.

Speed, Gilmer J. (1893). Do newspapers now give the news? Forum, 15, 705–711.

Spiegelman, Marvin; Terwilliger, Charles; & Fearing, Franklin. (1953a). The content of comics: Goals and
means to goals of comic strip characters. Journal of Social Psychology, 37, 189–203.

Spiegelman, Marvin; Terwilliger, Charles; & Fearing, Franklin. (1953b). The reliability of agreement in content
analysis. Journal of Social Psychology, 37, 175–187.

Stempel, Guido H., III. (1952). Sample size for classifying subject matter in dailies: Research in brief.
Journalism Quarterly, 29, 333–334.

Stempel, Guido H., III. (1961). The prestige press covers the 1960 presidential campaign. Journalism
Quarterly, 38, 157–170.

Stevens, Stanley S. (1946). On the theory of scales of measurement. Science, 103, 677–680.

Stone, Philip J. (1975). Report on the Workshop on Content Analysis in the Social Sciences, Pisa, 1974.
Social Science Information, 14, 107–111.

Stone, Philip J.; Dunphy, Dexter C.; Smith, Marshall S.; & Ogilvie, Daniel M. (1966). The General Inquirer: A
computer approach to content analysis. Cambridge, MA: MIT Press.

Stone, Philip J., & Hunt, Earl B. (1963). A computer approach to content analysis using the General
Inquirer system. In E. C. Johnson (Ed.), American Federation of Information Processing Societies, conference
proceedings (pp. 241–256). Baltimore, MD: American Federation of Information Processing Societies.

Street, Arthur T. (1909, July 25). The truth about newspapers. Chicago Tribune.

Strodthoff, Glenn G.; Hawkins, Robert P.; & Schoenfeld, A. Clay. (1985). Media roles in a social movement: A
model of ideology diffusion. Journal of Communication, 35(2), 134–153.

Page 71 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Sullivan, Dan. (2001). Document warehousing and text mining. New York, NY: Wiley.

Surowieki, James. (2005). The wisdom of crowds. New York, NY: Doubleday.

Szykiersky, Dorit, & Raviv, Amiram. (1995). The image of the psychotherapist in literature. American Journal
of Psychotherapy, 49, 405–415.

Tannenbaum, Percy H., & Greenberg, Bradley S. (1961). J. Q. references: A study of professional change.
Journalism Quarterly, 38, 203–207.

Taylor, Wilson L. (1953). “Cloze procedure”: A new tool for measuring readability. Journalism Quarterly, 30,
415–433.

ten Have, Paul. (1999). Doing conversation analysis: A practical guide. Thousand Oaks, CA: Sage.

Tenney, Alvan A. (1912). The scientific analysis of the press. Independent, 73, 895–898.

Tesch, Renata. (1990). Qualitative research: Analysis types and software tools. Bristol, PA: Falmer.

Thome, Helmut, & Rahlf, Thomas. (1996). Dubious cycles: A mathematical critique of the Namenwirth/Weber
thesis on cultural change with an introduction into filter design methods. Quality and Quantity, 30, 427–448.

Thompson, Stith. (1932). Motif-index of folk literature: A classification of narrative elements in folk-tales,
ballads, myths, fables, mediaeval romances, exempla, fabliaux, jest-books, and local legend. Bloomington:
Indiana University Studies.

Tijssen, Robert J. W., & Van Raan, Anthony F. J. (1994). Mapping changes in science and technology:
Bibliometric co-occurrences analysis of the R&D literature. Evaluation Review, 18, 98–115.

Todorov, Tzvetan. (1988). Mikhail Bakhtin: The dialogical principle. Wlad Godzich (Trans.). Minneapolis:
University of Minneapolis Press.

Toulmin, Stephen E. (1958). The uses of argument. Cambridge, MA: Cambridge University Press.

Treichler, Paula A. (1988). AIDS, homophobia, and biomedical discourse: An epidemic of signification. In
Douglas Crimp (Ed.), AIDS, cultural analysis, cultural activism (pp. 31–70). Cambridge, MA: MIT Press.

Truzzi, Marcello. (1988). Sherlock Holmes, applied social psychologist. In Umberto Eco & Thomas A. Sebeok
(Eds.), The sign of three: Dupin, Holmes, Peirce (pp. 55–80). Bloomington: Indiana University Press.

Tuggle, C. A. (1997). Differences in television sports reporting of men’s and women’s athletics: ESPN

Page 72 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

SportsCenter and CNN Sports Tonight. Journal of Broadcasting & Electronic Media, 41, 14–24.

Tukey, John W. (1980). Methodological comments focused on opportunities. In Peter R. Monge & Joseph N.
Cappella (Eds.), Multivariate techniques in human communication research (pp. 489–528). New York, NY:
Academic Press.

Turow, Joseph. (1989). Playing doctor: Television, storytelling, and medical power. New York, NY: Oxford
University Press.

Valente, Thames W. (1993). Diffusion of innovations and policy decision-making. Journal of Communication,
43(1), 30–45.

Van Dijk, Teun A. (1977). Text and context: Explorations in the semantics and pragmatics of discourse. New
York, NY: Longman.

Van Dijk, Teun A. (1991). Racism and the press. New York, NY: Routledge.

Van Dijk, Teun A. (1993). Principles of critical discourse analysis. Discourse & Society, 4, 249–283.

Van Rijsbergen, Cornelius J. (1979). Information Retrieval. London, England; Boston, MA: Butterworths.

Vernadsky, Vladimir I. (1945). The biosphere and the noosphere. Scientific American, 33(1), 1–12.

Wallace, Anthony F. C. (1961). Culture and personality. New York, NY: Random House.

Walworth, Arthur. (1938). School histories at war: A study of the treatment of our wars in the secondary school
history books of the United States and in those of its former enemies. Cambridge, MA: Harvard University
Press.

Wang, Rong. (2017). Marginality or strategic selection? Collaborative crowdsourcing through team building.
Paper presented at the 67th Annual Conference of the International Communication Association, May 25-29,
San Diego, CA.

Waples, Douglas, & Berelson, Bernard. (1941). What the voters were told: An essay in content analysis.
Unpublished manuscript, University of Chicago, Graduate Library School.

Watzlawick, Paul; Beavin, Janet H.; & Jackson, Don D. (1967). Pragmatics of human communication: A study
of interaction patterns, pathologies, and paradoxes. New York, NY: W. W. Norton.

Weaver, Donald H.; Buddenbaum, Judith M.; & Fair, Jo Ellen. (1985). Press freedom, media, and
development, 1950–1979: A study of 134 nations. Journal of Communication, 35(2), 104–117.

Page 73 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Webb, Eugene J.; Campbell, Donald T.; Schwartz, Richard D.; & Sechrest, Lee. (1966). Unobtrusive
measures: Nonreactive research in the social sciences. Chicago, IL: Rand McNally.

Weber, Max. (1911). “Geschäftsbericht” in Verhandlungen des Ersten Deutschen Soziologietages vom
19.–22. Oktober 1910 in Frankfurt A. M. In Schrift der Deutschen Gesellschaft für Soziologie (pp. 39–62).

Weber, René; Mangus, J. Michael; Huskey, Richard; Amir, Ori; Swanson, Reid; Gordon, Andrew;
Khooshabeh, Peter; Hahn, Lindsay; & Tamborini, Ron. (2017). Extracting moral foundations from text
narratives: Relevance, challenges, and solutions. Paper presented at the 67th Annual Conference of the
International Communication Association, May 25–29, San Diego, CA.

Weber, Robert P. (1984). Computer-generated content analysis: A short primer. Qualitative Sociology, 7,
126–174.

Weber, Robert P. (1990). Basic content analysis (2nd ed.). Newbury Park, CA: Sage.

Weick, Karl E. (1968). Systematic observational methods. In Gardner Lindzey & Elliot Aronson (Eds.), The
handbook of social psychology (pp. 357–451). Reading, MA: Addison-Wesley.

Weitzman, Eben A., & Miles, Matthew B. (1995). Computer programs for qualitative data analysis: A software
sourcebook. Thousand Oaks, CA: Sage.

Wellman, Barry, & Berkowitz, Stephen D. (Eds.). (1988). Social structures: A network approach. Cambridge,
MA: Cambridge University Press.

Wells, Robert A., & King, Erika G. (1994). Prestige newspaper coverage of foreign affairs in the 1990
congressional campaign. Journalism Quarterly, 71, 652–664.

Whissell, Cynthia M. (1980). The Dictionary of Affect in Language. In Robert Plutchik & Henry Kellerman
(Eds.), Emotion: Theory, research, and experience (pp. 113–131). New York, NY: Academic Press.

Whissell, Cynthia M. (n.d.). Whissell’s Dictionary of Affect in Language technical manual and user’s guide.
Accessed at http://www.hdcus.com/manuals/wdalman.pdf

Whissell, Cynthia M.; Fournier, M.; Pelland, R.; Weir, D.; & Makarek, K. A. (1986). Dictionary of Affect in
Language: IV. Reliability, validity, and applications. Perceptual and Motor Skills, 62, 875–888.

White, David M. (1964). The “gatekeeper”: A case study in selection of news. In Lewis A. Dexter & David M.
White (Eds.), People, society and mass communication (pp. 160–172). New York, NY: Free Press.

White, Paul W. (1924, May 31). Quarter century survey of press content shows demand for facts. Editor and

Page 74 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Publisher, 57.

White, Ralph K. (1947). Black Boy: A value analysis. Journal of Abnormal and Social Psychology, 42,
440–461.

Wilcox, Dennis F. (1900). The American newspaper: A study in social psychology. Annals of the American
Academy of Political and Social Science, 16, 56–92.

Willey, Malcolm M. (1926). The country newspaper: A study of socialization and newspaper content. Chapel
Hill: University of North Carolina Press.

Winograd, Terry, & Flores, Fernando. (1986). Understanding computers and cognition: A new foundation for
design. Norwood, NJ: Ablex.

Wittgenstein, Ludwig. (1922). Tractatus Logico-Philosophicus. New York, NY: Harcourt, Brace & Co.

Wittgenstein, Ludwig. (1958). Philosophical investigations (English text of the 3rd edition). G. E. M. Anscombe
(Trans.). New York, NY: Macmillan.

Woelfel, Joseph. (1993). Artificial neural networks for policy research. Journal of Communication, 43(1),
63–80.

Woelfel, Joseph. (1997). Attitudes as nonhierarchical clusters in neural networks. In George A. Barnett &
Franklin J. Boster (Eds.), Progress in communication sciences (Vol. 13, pp. 213–227). Norwood, NJ: Ablex.

Woelfel, Joseph, & Fink, Edward L. (1980). The measurement of communication processes: Galileo theory
and method. New York, NY: Academic Press.

Wolfers, Justin. (2017, August 20). Exposing a toxic milieu for women in economics. New York Times, p. 3.

Wonsek, Pamela. (1992). College basketball on television: A study of racism in the media. Media, Culture,
and Society, 14, 449–461.

Woodward, Julian L. (1934). Quantitative newspaper analysis as a technique of opinion research. Social
Forces, 12, 526–537.

Wright, Charles E. (1964). Functional analysis and mass communication. In Lewis A. Dexter & David M. White
(Eds.), People, society, and mass communication (pp. 91–109). New York, NY: Free Press.

Wu, Alice H. (2017). Gender stereotyping in academia: Evidence from economics job market rumors forum.
Retrieved from https://www.dropbox.com/s/v6q7gfcbv9feef5/Wu_EJMR_paper.pdf?dl=0

Page 75 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

Yule, George U. (1944). The statistical study of literary vocabulary. London, England: Cambridge University
Press.

Zeldow, Peter B., & McAdams, Dan P. (1993). On the comparison of TAT and free speech techniques in
personality assessment. Journal of Personality Assessment, 60, 181–185.

Zhao, Xinshu; Liu, Jun S.; & Deng, Ke. (2012). Assumptions behind inter-coder reliability indices. In C. T.
Salmon (Ed.), Communication Yearbook 36. New York, NY: Routledge.

Zillmann, Dolf. (1964). Konzept der Semantischen Aspektanalyse. Unpublished manuscript, Institut für
Kommunikationsforschung, Zurich, Switzerland. (Mimeo)

Zipf, George Kingsley. (1935). The psycho-biology of language: An introduction to dynamic philology. Boston,
MA: Houghton Mifflin.

Zucker, Harold G. (1978). The variable nature of news media influence. In Brent D. Ruben (Ed.),
Communication yearbook 2 (pp. 225–240). New Brunswick, NJ: Transaction.

Züll, Cornelia; Weber, Robert P.; & Mohler, Peter. (1989). Computer-assisted text analysis for the social
sciences: The General Inquirer III. Mannheim, Germany: Zentrum für Umfragen, Methoden und Analysen.

Zwick, Rebecca. (1988). Another look at interrater agreement. Psychological Bulletin, 103, 347–387.

Page 76 of 77 Content Analysis: An Introduction to Its Methodology

Sage Sage Research Methods
© 2019 by SAGE Publications, Inc.

About the Author

Klaus Krippendorff (Grad. Ulm School of Design 1962; PhD, communication, University of Illinois, Urbana,
1967, and PhDhc Linnaeus University, Kalmar, Sweden, 2012) is Professor of Communication and Gregory
Bateson Professor Emeritus for Cybernetics, Language, and Culture at the University of Pennsylvania’s
Annenberg School for Communication. In addition to publishing numerous articles in journals of
communication, sociological methodology, cybernetics, system theory, and design, he authored Information
Theory: Structural Models for Qualitative Data; A Dictionary of Cybernetics; On Communicating: Otherness,
Meaning, and Information; and The Semantic Turn: A New Foundation for Design. He edited Communication
and Control in Society and Design in the Age of Information: A Report to the National Science Foundation
(NSF) and coedited The Analysis of Communication Content: Developments in Scientific Theories and
Computer Techniques. The first edition of Content Analysis: An Introduction to Its Methodology was published
in 1980, the second in 2004, and the third in 2013. The first edition was translated into Italian, Persian,
Japanese, Spanish, Indonesian, and Hungarian, and the third into Chinese. In 2004, it received the
International Communication Association (ICA) Fellows Book Award for its lasting contribution to the
advancement of communication studies. In 2017, Klaus Krippendorff received the first methodological
innovation award from the Mass Communication Division of ICA for his contributions to content analysis
and reliability statistics. To broaden the range of examples and solutions to practical problems in the
book, in 2009, he and Mary Angela Bock coedited The Content Analysis Reader. Klaus Krippendorff
currently pursues five exciting projects: With epistemology in mind, he explores how realities are socially
constructed in public, political, professional, and scientific discourses. As a critical scholar, he seeks to
understand conditions of social entrapment and possibilities of liberation from various forms of oppression,
including by conceptions of power, artificial intelligence as superior human intelligence, and attributing
agency to physical phenomena and abstractions used in language. His cybernetic background leads him
to recursive conceptions of conversations—interpersonal, public, and organizational—and their erosion into
diverse forms communication, discourses, and algorithms. Part of this project is to come to grips with the
ongoing algorithmization of social institutions. As a former designer, now design scholar, he made it his
mission to move human communication conceptions into the center of design considerations: conceptualizing
technology through human interfaces; stakeholder networks within which technologies are realized; how
media, texts, talk, and interactions constitute the social nature of artifacts; and the ecological interactions of
technological artifacts that their human use sets in motion. In addition to these conceptual contributions, he
continues to generalize his reliability statistics, Krippendorff’s alpha, to a variety of data-making efforts.

Page 77 of 77 Content Analysis: An Introduction to Its Methodology