Criteria For Assessing Interpretive Validity in Qualitative Research

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/232589193
Criteria for assessing interpretive validity in qualitative research
Article · January 1988
CITATIONS READS
887 15,631
2 authors, including:
David L. Altheide
Arizona State University
170 PUBLICATIONS 9,166 CITATIONS
SEE PROFILE
All content following this page was uploaded by David L. Altheide on 21 February 2017.
The user has requested enhancement of the downloaded file.

35
REFLECTIONS ON INTERPRETIVE
ADEQUACY IN QUALITATIVE RESEARCH
David L. Altheide and John M. Johnson
A line is a dot that went for a walk.

—Paul Klee
O
ver 15 years ago, we published “Criteria for Assessing debate about using cyberspace for research generates new
Interpretive Validity in Qualitative Research” (Altheide & issues and perspectives. In the context of this creative flux, con-
Johnson, 1994). In this work, we continued the develop- servative countermovements emerge to standardize or normal-
ment of our ideas about qualitative research, many of which ize the practice and evaluation. In the United States, a scientifi-
were developed during a long series of professional meetings in cally based research (SBR) countermovement has arisen, also
the 1980s. Our animating questions were how interpretive meth- called scientific inquiry in education (SIE), and this is joined by
odologies should be judged by readers (audiences) who share similar movements in the United Kingdom, called the research
the perspective that how knowledge is acquired, organized, assessment exercise (RAE), or in Australia the research quality
interpreted, and presented is relevant for the substance of those framework (RQF). Two reports by the U.S. National Science
claims. We presented our ideas about how to make the claims Foundation have proven especially troublesome and problem-
and narratives of qualitative research more trustworthy to read- atic for American scholars (Lamont & White, 2009; Ragin,
ers and audiences. We called our approach “analytic realism,” to Nagel, & White, 2004). These movements have generated heated
identify how reflexive and interpretive methods could be pre- debates about “the politics of evidence,” and inspired claims
sented to enhance their credibility, relevance, and importance. about qualitative inquiry being “under fire” by these state-
Much has happened in the world of qualitative methods dur- sponsored efforts to constrain and control the criteria of scien-
ing the last two decades, and important questions and issues tific inquiry (Denzin, 2009).
now span many new disciplines, venues, arenas, perspectives, Our advocacy of analytic realism in the 1994 paper was
theories, and problem areas. Methodological issues once con- intended to align our efforts with philosophical realism, argu-
sidered relevant for only a minority of anthropologists and ably the dominant philosophical position in the social sciences
sociologists are now discussed (and disputed) in many other for many decades. The basic idea of realism is that there is a real
disciplines, especially education, policy studies, the health sci- world with which we act and interact (an “obdurate” world in
ences, gender studies, communication, cultural studies, justice the words of George Herbert Mead), that individuals and groups
studies, and others. Many new models of representation and create meaning in this world, and that while our theories, con-
interpretation have arisen during this period, including those cepts, and perspectives may approach some kind of valid
from advocates for linking qualitative research to justice values, understanding, they cannot and do not exhaust the phenomena
issues, and agendas. Performative writing and performance of our interest. All theories, concepts, and findings are grounded
ethnography emerge as new positions, and advocates for stand- in values and perspectives; all knowledge is contextual and par-
point epistemologies cross disciplinary boundaries. A robust tial, and other conceptual schemas and perspectives are always
2– 581
582– 2– PART V THE ART AND PRACTICES OF INTERPRETATION, EVALUATION, AND REPRESENTATION
possible. We are heartened that many other scholars have each of these areas have been grappling with issues of truth,
advanced some related version of this perspective, such as validity, verisimilitude, credibility, trustworthiness, dependabil-
“critical realism” (Bhaskar, 1979; Harré & Madden 1975; Manicas ity, confirmability, and so on. What is valid for clinical studies or
& ebrary, Inc. 2006; Maxwell 2008), “experimental realism” policy studies may not be adequate or relevant for ethnography
(Lakoff, 1987), “subtle realism” (Emerson, Fretz, & Shaw, 1995; or autoethnography or performance ethnography. We return to
Hammersley, 1992), “ethnographic realism” (Lofland, 1995), this point later.
“innocent realism” (Haak, 2003), “natural realism” (Putnam,
1999), and “emergent realism” (Henry, Julnes, & Mark, 1998). Clinical Studies
These different versions of realism share certain basic ideas:
that human social life is meaningful, and that it is essential to In recent years, researchers in several clinical fields have
take these meanings into account in our explanations, concepts, utilized qualitative methods to grasp and articulate invisible or
and theories; furthermore, to grasp the importance of the val- taken-for-granted realities they experience in clinical settings.
ues, emotions, beliefs, and other meanings of cultural members, Miller and Crabtree (2005) write,
it is imperative to embrace an interpretivist approach in our
scientific and theoretical work. According to Maxwell (2008), Qualitative clinical researchers bring several power perspectives to
the clinical encounter that help surface the unseen and the
these versions of realism reflect an ontological realism while
unheard and also add depth to what is already present. These
simultaneously accepting a form of epistemological construc- include understanding disease as a cultural construction . . . pos-
tionism and relativity. They oppose the radical constructivist sessing knowledge of additional medical models such as biopsy-
view, which denies the existence of any reality apart from our chosocial and humanistic models, homeopathy, and non-Western
constructions of it (or them). This kind of realist perspective models that include traditional Chinese, Ayurvedic, and shaman-
has proven very valuable in grasping the relationship(s) between ism, and recognizing the face and importance of spirituality in
the meanings and perspectives of cultural members and the human life. (p. 612)
social contexts in which they are embedded, and especially for
understanding conflicts or differences in meanings for actors Clinical practitioners who utilize observational, narrative, or
located in the same situation or context. discourse methods are seeking to articulate standards of valid-
When knowledge and evidence are viewed from a symbolic ity (or truthfulness) that can be shared with others in their field,
interactionist perspective, evidence is seen as part of a commu- and hence are subject to independent tests and verification. Of
nication process that symbolically joins an actor, an audience, a these efforts, Rolfe (2004) writes,
point of view, assumptions, and claims about the relations
between two or more phenomena. This view of evidence-as- [For some,] validity and reliability [in clinical studies] are achieved
process is termed the “evidentiary narrative,” and draws atten- when the researcher rigorously follows a number of verification
tion to the ways in which credible information and knowledge strategies in the course of the research process. “Together, all these
verification strategies incrementally and interactively contribute to
are buffeted by symbolic filters, including distinctive “epistemic
and build reliability and validity, thus ensuring rigor. Thus, the
communities,” or collective meanings, standards, and criteria rigor of qualitative inquiry should be beyond question, beyond
that govern sanctioned action (see Altheide, 2008). It is hearten- challenge, and provide pragmatic scientific evidence that must be
ing to see that numerous qualitative researchers in a number of integrated into our developing knowledge base” [Morse et al., 2002,
these epistemic communities have made impressive strides in emphasis added by Rolfe]. This statement of intent exemplifies
addressing their validity issues within their own perspectives, very strongly the aspirations of some qualitative researchers to the
and have developed many useful ideas to create more trustwor- values, approaches, terminologies, and hence, to the certainties of
thy knowledge to be shared with other audiences. We briefly the “hard” sciences. Rigor is clearly the key to success. . . . [But oth-
review some of these developments as a prelude to saying how ers] argue that issues of validity in qualitative studies should be
this has changed our views in recent years. linked not to “truth” or “value,” as they are for the positivists, but
rather to “trustworthiness,” which becomes a matter of persuasion
whereby the scientist is viewed as having made those practices
visible, and therefore, auditable. (p. 305)
2 Framing Validity
Issues in Interpretive Research These comments illustrate the ongoing debates of the
evidence-based practice movement in clinical studies, where for
There are many ways to use, practice, promote, and claim quali- many years now qualitative practitioners and researchers have
tative research, and in each there is a proposed or claimed rela- been discussing the applicability of such ideas as truth, validity,
tionship between some field of human experience, a form of reliability, trustworthiness, and so on. While many anthropolo-
representation, and an audience. Researchers and scholars in gists or sociologists who practice qualitative research might be
Chapter 35 Reflections on Interpretive Adequacy in Qualitative Research– 2– 583
primarily motivated to make fundamental contributions to the subject to further scrutiny and debate. The process cannot be
basic knowledge of their disciplines, or contributions to a sub- ensconced in a single research method or a once-and-for-all set
stantive problem area, those who practice qualitative methods of standards. (p. 73)
in clinical studies typically intend a different audience, those
interested in advancing effective clinical practice. These com- Others in the policy field also affirm the necessity for com-
ments illustrate that there is a diversity of purposes that ani- paring research reports with prior reports, and for comparing
mate qualitative research, and that criteria of usefulness in settings with other settings (Hammersley, 1992). This is a com-
research are tied to these practical purposes and disciplinary/ mon theme in what is now called “the new public management
occupational values. movement,” which seeks greater transparency and clarity of the
policy-making and policy-implementation processes. This sug-
gests a dual approach to the issue of validity: certain expecta-
Policy Studies tions for the researcher or research team to show readers the
Qualitative research is increasingly used in policy studies, grounds for trusting their report, on the one hand, in conjunc-
where the intention is to study how various actors bring and tion with a measured and realistic skepticism by readers, on the
make meaning in actual concrete settings, and the conse- other hand, to place the claims of any given research report in a
quences of these actions. The goals of this type of research are context of many other reports, even one’s life experiences.
crisply stated by Hammersley (2005): “Qualitative policy
research is aimed at having an impact on current programs and Action Research
practices” (p. 3). The focus here can be on the impact or conse-
quences of policy, but additionally the processes of how official Action research or participatory action research is another
law or policies are translated and interpreted, from the heights emergent form of qualitative research, which usually involves
of inception down to the points of implementation, to the one researcher or a research team in the field, participating with
“street-level” realities. There is little doubt that qualitative societal members to produce social change or implement a
research can be more flexible than traditional quantitative social policy or organized response to a problem. Kemmis and
research, and has the potential to adjust research agendas to McTaggart (2005) propose their vision of this kind of research:
meet changing demands in the field. While the focus of quanti-
Through participatory action research, people can come to under-
tative research is usually on “outcome measures,” or metrics, stand that—and how—their social and educational practices are
qualitative research has the potential to study the complex located in, and are the product of, particular material, social, and
social and bureaucratic processes whereby laws and policies are historical circumstances that produced them and by which they are
actually implemented in daily life. In the United Kingdom, cen- produced in everyday social interaction in a particular setting. By
tral policy makers have proposed their own set of standards for understanding their practices as the product of particular circum-
what they expect of qualitative research in these areas (Cabinet stances, participatory action researchers becomes alert to clues
Office, 2003). These should include a detailed description of the about how it may be possible to transform the practices they are
contexts in which the study was conducted, a discussion of how producing and reproducing through their current ways of working.
fieldwork settings or methods may have influenced data collec- (p. 565, emphasis original)
tion, descriptions of background or historical developments
and social/organizational characteristics of study sites, descrip- Social action researchers do not deny the relevance or
tion and illumination of diversity/multiple perspectives or posi- importance of foundational or basic knowledge, but often insist
tions, and discussion/evidence of the ideological perspectives/ that they seek experiential knowledge as well, often expressing
values/philosophies that guide the researcher or research team the hope that theory and pragmatics together can achieve a
(see Torrance, 2007, pp. 55–79). whole that is greater than its parts. To provide grounds for trust-
The tensions and debates in policy studies mirror those in ing participatory action research, Ladkin (2004) proposes that
other areas; some feel that validity or truth is better served by research done in this vein should include accounts to demon-
affirming a set of research standards (such as those offered by strate emergence and enduring consequences of actions or
the UK Cabinet Office above), whereas others think that truth is policies, accounts of how the research dealt with pragmatic
better served by making comparative assessments between issues of practice and practicing, accounts of how the research
studies, over time, and between diverse settings. Harry Torrance deals with questions of significance, and accounts showing how
(2007) writes, the research considers a number of different ways of knowing.
While it would be easy to criticize Ladkin’s proposals as being
Assuring the quality of research, and particularly the quality of very abstract and insubstantial, or for emphasizing certain
qualitative research in the context of policy making, must be things (like the consequences) that are not within the control of
conceptualized as a vital and dynamic process that is always the researcher or research team, our main point here is to show
that those working in this vein are struggling with issues of many autoethnographers seek to elucidate the changes in
truthfulness and validity, and seek to engage this debate within meaning or perspective over many years, even decades, where
the confines of their own expertise. the long passage of time itself produces new or altered under-
standings of past “facts.”
Autoethnography and Expressive Frames Performance ethnography and expressive artists often share
the autoethnographer’s desire to explore and communicate the
A diversity of current research seeks to break down the prior
deeply personal or taken-for-granted aspects of personal or
barriers between subject and object, between the knower and
daily life, but in addition they often seek to engage their audi-
the known, between the self and the social, between the spiri-
ence in a more direct manner, often seeking to evoke an emo-
tual and the empirical, and between the writer and the audience.
tional response. This may be done with acted performances,
Autoethnography is only one of several names given to this
poems, photography, multimedia collages, or readings. Szto,
emerging enterprise, and autoethnographers commonly seek to
Furman, and Langer (2005) write,
integrate the storyteller and the story. Laurel Richardson (1997)
says that “writing stories about our ‘texts’ is a way of making
The photographer is an ethnographer in this sense [of trying to
sense of and changing our lives” (p. 5). Carolyn Ellis (2009)
capture a subject’s reality]. You try to capture the context. You have
seeks to “open up conversations about emotions in romantic to take poetic license and select context. . . . In the role of researcher,
and family relationships” (p. 17). She additionally says, the poet must engage in conscious and constant self-exploration.
When he (or she) writes about a subject in front of himself (her-
Thus, reexamining the events we have lived through and the stories self), or when he (she) is reducing data from narratives, he (she)
we have told about them previously allows us to expand and has to be very clear to stay faithful to the data. His (her) notes serve
deepen our understandings of the life we have led, the culture in as both data to be worked with, as well as ethnographic notes that
which we have lived, and the work we have done. This review pro- explore their reactions. Many times, these biases should be pre-
vides new possibilities for understanding ourselves and keeps us sented so the readers can decide for themselves how to interpret
from remaining stuck in the interpretations we have settled on in the poem. The first allegiance of the researcher, as poet, has to be
the past. (p. 13) to the subject’s experience. In a sense, there are two types of poems
for the researcher. There are poems in which they merely present
On many occasions, autoethnographers grapple with the the subjects’ experience as accurately as possible, hopefully utiliz-
issue of “memory,” and how to contextualize or re-contextualize ing their words, and then there are interpretive poems, in which
people and events from long ago. This issue is larger than the they deconstruct the meaning of the experience and consciously
vagaries of remembering empirical facts, because it includes the allow for interpretation.[AU: Page # needed]
many issues of interpretation and perspective. On the standards
involved in this kind of research, Bochner (2007) states, There is great diversity of qualitative research, and there is
diversity in the ways to justify or legitimize each of the above
Of course, my gravest obligation is not to lie. But the space between approaches. While these approaches differ, they also share an
lying and telling the truth can be vast. If telling the truth is merely
ethical obligation to make public their claims, to show the
saying what I remember, then I have set the bar of obligation
reader, audience, or consumer why they should be trusted as
extremely low. Once the past was there, now it is gone. I want to be
faithful to the past, but what I remember of my history is anchored faithful accounts of some phenomenon. Moreover, each of these
by what summons me now to remember, and my memory is, in approaches reflects the context and purposes for the practitio-
part, a response to what inspired my recollections. (p. 198, empha- ners and audiences, including clients. The pragmatic utility of
sis original) validity as “good for our present intents and purposes” cuts
through all of the methodological approaches and authoritative
For many of its practitioners, autoethnography becomes a claims. In other words, whether it is truthful, accurate, on the
disciplined way to interrogate one’s memory, to contextualize or mark, and so forth is framed by an ecology of knowing tied to
re-contextualize empirical facts or memories within interpreta- practices and intentions, and ultimately, “our justifications” for
tions or perspectives that “make sense” of them in new or newly using this method. What is common to each of these approaches,
appreciated ways. Many of these studies deal with intimate and and by implication all forms of inquiry, is a process of acquiring
family relations, and commonly seek to make explicit what is information, organizing it as data, and then analyzing and inter-
usually taken for granted within these relationships, the many preting those data with the help of refractive (conceptual, theo-
seen-but-unspoken or known-but-not-acknowledged complex- retical, perhaps political) lenses.
ities of our lives. In some autoethnographic studies, the time We have noted that validity has been referred to many
frame of interest is longer than that of a traditional, observa- ways, including successor validity, catalytic validity, interro-
tional ethnography, which might take as long as 10 years gated validity, transgressive validity, imperial validity, simula-
between the time of inception and the time of the final report; cra/ironic validity, situated validity, and voluptuous validity
(see, e.g., Atkinson, 1990; Atkinson, 1992; Guba, 1990; Hammersley, validity-as-reflexive-accounting (VARA), which places the
1990, 1992; Lather, 1993; Wolcott, 1990). Our effort to clarify the researcher, the topic, and the sense-making process in interaction.
logic-in-use by many qualitative researchers suggested that a We identified these hyphenated validities (above) as illustra-
heuristic view of “hyphenated validity” could help clarify the tions of the range of attention the “problem of validity” has
methodological discourse at the time (early 1990s). received. But another standard for validity has appeared. The
SBR, SIE, and RAE movements noted above represent examples
Types of Validity of an expanded context of control involving disparate audiences
with oversight interests in corralling and regulating qualitative
Validity-as-culture (VAC) is well known to social science students. research in accordance with conventional formats of communi-
A basic claim is that the ethnographer reflects, imposes, repro- cation and regulation, and ultimately legitimacy associated with
duces, writes, and then reads their cultural point of view for the
more positivistic methodology. This has led to a new version of
“others.” Point of view is the culprit in validity. The solution
includes efforts to include more points of view, including reassess- validity, validity-as-marketable-legitimacy (VAML), which
ing how researchers view the research mission and the research refers to the negotiated order of socially sanctioned (and
topic. Atkinson (1992, p. 34 ff.) suggests that ethnographies can be respectable) research methodology.
mythologized: “But the sense of class continuities is hardly surpris- This latest version of validity is being promoted for bureau-
ingly stronger in the British genre than in the American which is cratic, rational, and organizational purposes and not to enhance
more preoccupied with a sense of place.” inquiry, creativity, or discovery, but rather, accountability, as in,
Validity-as-ideology (VAI) is very similar to VAC, except the funding this is warranted according to our “guidelines,” there-
focus is on the certain specific cultural features involving social fore no individual will be accountable for any errors, and so
power, legitimacy, assumptions about social structure, e.g., forth (Denzin, 2010; Kvale & Brinkmann, 2008). An unintended
subordinate/superordinate. consequence of VAML is to dampen the creative search for var-
Validity-as-gender (VAG), like the previous two, focuses on ied forms of truth and relevant search. Openness and the tre-
taken-for-granted assumptions made by “competent” researchers
mendous success of the explorations of various approaches to
in carrying out their conceptual and data collection tasks, including
some issues about power and domination in social interaction. qualitative work across many disciplines have, paradoxically,
One concern is that these asymmetrical aspects of social power contributed to its utility, use, and imposed limitations for prac-
may be normalized and further legitimated. tical purposes. In recent years, the debate has intensified and
Validity-as-language/text (VAL) resonates with all that have changed, as qualitative research has become more accepted for
come before, particularly how cultural categories and views of the funding and “practical” applications, e.g., policy research. One
world, as implicated in language, and more broadly, “discourse,” interpretation is that qualitative research is now in the market-
restricts decisions and choices by how things are framed. place of ideas, where the coinage is not just intellectual prowess,
Validity-as-relevance/advocacy (VAR) stresses the utility and but actual coin. The approaches to validity that we delineated
“empowerment” of research to benefit and uplift those groups often were consistent with the pursuit of truth and a logic of discov-
studied, relatively powerless people, e.g., the poor, peasants, etc. ery. However, the success of varied approaches to qualitative
Validity-as-standards (VAS) asserts that the expectation about research has opened up market possibilities not only for publi-
a distinctive authority for science, or the researchers legitimized by
cation and teaching, but also for funding and sponsored proj-
this “mantle of respectability,” is itself suspect, and that truth-
claims are so multiple as to evade single authority or procedure. In ects that are accountable to administrators and overseers of
the extreme case, science ceases to operate as a desirable model of agencies and organizations that must answer to other scientific
knowledge, because it is, after all, understanding rather than and political constituencies. The standardization and reduction
codified, theoretically integrated information—as knowledge— of an array of approaches leads to more than smoothing out
that is to be preferred. (Altheide & Johnson, 1994, p. 488) sharp edges; domains become sacrificed for the sake of an
established lexicon, rhetoric, and narrative of authorized know-
These approaches to validity, while certainly not definitive, ing, ultimately as “objective.” The push to the linear criteria and
reflected the purpose and audiences for research approaches decision making—recall Klee’s dot that went for a walk!—
and applications. The subtext was openness and engagement. means that approved criteria and checklists of acceptability and
Notwithstanding intense debates that occurred among the prac- standardization matter. We will say a bit more about this later in
titioners, the tone was inclusive and the spirit was to not over- the essay after discussing the complexities of evidence and the
look important segments of, say, audiences that might not be so importance of tacit knowledge in understanding the social con-
well served by unintended limitations of the generation of struction of reality. For now, we wish to emphasize that the
knowledge. Informed by our basic assumption that the social standardization of qualitative work is also risk avoidance and
world is an interpreted world, not a literal world, always under demonstrates a “risk society” approach to solving the problem
symbolic construction, with emphasis on awareness of the process of evaluating research, and so forth (Ericson & Doyle, 2003).
of the ethnographic work, we offered another inclusive view: However, evidence is not that simple.
The Problem of Evidence systems in everyday life are tied to epistemic communities that
provide perspectives, scenarios, and scripts that reflect sym-
Much of the foregoing discussion rests on an understanding
bolic social and moral orders.
of “evidence,” or agreed-upon—or potentially agreeable—
information that would serve as a basis or foundation. There is An “evidentiary narrative” symbolically joins an actor, an audience,
a rich literature on the control of information, research subjects, a point of view (definition of a situation), assumptions, and a claim
and topics (Van den Hoonaard, 2002), as well as the politics of about a relationship between two or more phenomena. If any of
evidence (Altheide, 2008b; Denzin & Giardina, 2008). Evidence these factors are not part of the context of meaning for a claim, it
and facts are similar but not identical. We can often agree on will not be honored, and thus, not seen as evidence. Moreover, only
facts, e.g., there is a rock, it is harder than cotton candy. Evidence the claim is discursive, or potentially problematic, but it need not
involves an assertion that some facts are relevant to an argument be so. (Altheide, 2009, p. 65)
or claim about a relationship. Since a position in an argument is
The idea is that evidence is not about facts per se, but is
likely tied to an ideological or even an epistemological position,
about an argument, a narrative that is appropriate for the
evidence is not completely bound by facts, but is more problem-
purpose-at-hand. That means it is contextualized and part of a
atic and subject to disagreement. Indeed, until the 1990s, most
bounded project, with accompanying assumptions, criteria,
qualitative research was not taken seriously by many sociolo-
rules of membership, participation, and so on.
gists, and was regarded as second-rate social science; editors
From a sociology-of-knowledge perspective, the active
and reviewers for the discipline’s major journals would rarely
reception of a point “of information” is contingent on the “media
publish qualitative reports. The basic problem, of course, was
logic” of legitimacy (acceptability) of the information source,
that qualitative research, with exceptions, was not regarded as
the technology, medium, format and logic through which it is
being based on data; quotes and observations were not regarded
delivered (Altheide & Snow, 1979). What is meant by “evidence”
as appropriate evidence, especially if there were not a large
can be viewed as “information that is filtered by various sym-
enough number or “N” of these, thus rendering qualitative
bolic filters and nuanced meanings compatible with member-
claims as akin to quantitative estimates. The situation was so
ship” (Altheide, 2009, p. 65). Only then can the information be
dire that several groups of qualitative researchers started their
interpreted as evidence in juxtaposition with an issue, problem,
own journals, including Urban Life (later, Journal of Contempo-
or point of contention. Conversely, information that is not suit-
rary Ethnography), Symbolic Interaction, and Qualitative Sociol-
ably configured and presented is likely to be resisted, if not
ogy. In 2011—as this handbook attests—the situation has
rebuffed, within a prevailing discourse.
much improved, as qualitative methods, data, and “evidence”
Earlier we argued (Altheide & Johnson, 1994) that a more
have come to be more accepted. Yet, as the foregoing discussion
encompassing view of the ethnographic enterprise would take
indicates, there remains a kind of impatience with the wide-
into account the process by which the ethnography occurred,
ranging epistemologies that certain interest groups want to
which must be clearly delineated, including accounts of the
compartmentalize and regulate.
interaction between the context, researcher, methods, setting,
The problem that sociologists have with conflicts over evi-
and actors. The broad term that we offered, “analytic realism,” is
dence, however, is minimal compared to the nonacademic set-
based on the view that the social world is an interpreted world,
tings. When the president of the United States uses rhetoric and
not a literal world, always under symbolic construction (even
photographs to demonstrate/show that Iraq had weapons of
deconstruction!). We can also apply this perspective to under-
mass destruction (WMD) and therefore should be invaded in
stand how situations in everyday life are informed by social
order to keep the world safe, and when this “proof ” turns out to
contexts and uses of evidence. This application illuminates the
be false, then it is even more important that evidence be exam-
process by which evidence is constituted. We can now see how
ined and critically analyzed. Accordingly, qualitative researchers
any effort to standardize and limit qualitative research criteria
have focused on evidence and its social contexts (Denzin &
is doomed to failure and irrelevance. As long as the core of
Giardina 2008).
qualitative research is extended to more specific audiences and
We wish to stress that communication strategies, formats,
uses, criteria for validity will be linked with the evidence appro-
and paradigmatic boundaries can cloud our vision. The sym-
priate for specific tastes and uses. It is useful, then, to consider
bolic meaning filters that are called forth all stem from various
the following elements of what we termed “an ethnographic
memberships. Ultimately, evidence is bound up with our iden-
ethic” (Altheide & Johnson, 1994, p. 489) when trying to under-
tity in a situation. The multiple memberships we hold in various
stand evidence that is stated or affirmed in a situation:
epistemic communities are situationally shuffled and joined for
a particular purpose (e.g., when an assumption or value is chal- (1) The relationship between what is observed (behaviors, rituals,
lenged or called into question). An “evidentiary narrative” meanings) and the larger cultural, historical, and organizational
emerges from a reconsideration of how knowledge and belief contexts within which the observations are made (the substance);
(2) The relationship between the observer, the observed, and the and checklists, but such efforts are risky, since this is likely to be
setting (the observer); mainly regulative but not constitutive of the process that gave
(3) The issue of perspective (or point of view), whether that of the rise to the innovations in the first place. The upshot is nothing
observer or the member(s), used to render an interpretation of less than treating ethnography and qualitative research as a
the ethnographic data (the interpretation); commodity—a product—to be bought and sold in a market, but
this market, like all markets, is reflexive of the process and inter-
(4) The role of the reader in the final product (the audience); and
est that gives rise to all markets. Nico Stehr’s (2008) insights
(5) The issue of representational, rhetorical, or authorial style used about knowledge markets are instructive:
by the author(s) to render the description or interpretation
(the style). [W]hen viewed from a contemporary sociological perspective—
and this is the much more common and developed critique—
Each of these areas includes questions or issues that must be markets are not so much responsible for widespread affluence;
addressed and pragmatically resolved by any particular observer rather, they represent a rather harsh, impersonal institution that
in the course of his or her research. As originally formulated, has put into practice what major classical sociological theorists
these five dimensions of qualitative research include problem- have always anticipated and, of course, feared. From this per-
atic issues pertaining to validity. Indeed, we argued that the spective market relations are nothing but power relations.
“ethnographic ethic” calls for ethnographers to substantiate Market relations are a form of pure power relationship, pitting
their interpretations and findings with a reflexive account of the owners of the means of production against the owners of
themselves and the process(es) of their research (Altheide & labor power. (p. 85)
Johnson, 1993).
The evidentiary narrative is built on several arrays of The power relations inherent in funding agencies and sanc-
meaning: what we know, who we are, and what we consider as tioning boards that sanctify appropriate methodology can,
evidence of either our most basic assumptions (e.g., that there perhaps unintentionally, purge qualitative research of the subtle
is order in the world), or a specific claim about part of that but important distinctions noted above.
order (e.g., my beliefs are legitimate and truthful). Prior to The creative logic-in-use of scientific discovery that has
delving into a solution, let’s attempt an overview of the prob- shaped much of qualitative inquiry and development is pushed
lem. We live as social beings and are accountable to some aside as qualitative research becomes recast as “expertise” and
people but not others. Why do we accept certain claims but becomes a resource for knowledge producers to employ for
reject others, and what would lead us to change our minds? In practical purposes and for diverse audiences with limited
the modern era, this involved scientific authority about certain understanding (and interest) in the complexities of qualitative
empirical truths that were based on “data” as evidence, or as a research, including varieties of validity. Steven Fuller (2008)
kind of fact-guided proof. The modernist project relied on argues that historically, science and expertise have been anti-
rationality, including some formal rules of logic grounded in thetical forms of knowledge, with the former associated with
an objective view of things. This view has been seriously ques- creativity and contemplation, and in a sense, examining the
tioned by extensive research and writings encompassing such nature and realms of order and possibilities, while the latter
approaches as ethnomethodology, phenomenology, existential was more associated with application, and “doing” practical
sociology, symbolic interactionism, feminism, literary criti- things. Focusing on research in science technological studies
cism, performance studies, and autoethnography, to name a (STS), he asserts,
few (Denzin & Lincoln, 1994). These approaches contributed
to the “reflexive turn” in the social sciences, and the examina- Science and expertise are historically opposed ideas: The former
tion of how the research process, including the “act of writing” evokes a universalistic ideal meant to be pursued in leisure, while
the latter consists of particular practices pursued to earn a living.
partially produces the research result (Marcus & Clifford,
However, expertise can serve the universalistic ideal of science by
1986; Van Maanen, 1988). undermining the authority of other expertises that would cast
The reflexive turn is central to the problem of evidence, not doubt on the viability of this ideal. Put bluntly, expertise is
just why people believe “crazy things” and won’t easily consider “progressive” only when it serves as the second moment of a Hege-
evidence that would lead them to reject such beliefs, but more lian dialectic. . . . I see the modern university—specifically through
basically, why—and how—do researchers and scientists accept its teaching function—as the place where this moment most often
information as evidence for a particular matter? The subject happens. (p. 115)
matter under investigation still matters, but mainly as a “product”
that is socially constructed. Fuller’s concern is to save inquiry, exploration, and discovery as
Efforts like SBR and SIE referred to above attempt to formu- process from a steamrolling rhetoric and organizational push
late what qualitative research should look like, in terms of criteria for completion and results that can flatten subtleties as useful
products and procedures. This issue is apparent in current memories, and checking these against new interpretations
discussions about the nature and uses of evidence. gleaned from various documents about his family history,
including a Bible; a diary; and his father’s agency codebook,
The Great Gatsby.
2 The Evidentiary Narrative as Process We draw on this extraordinary project to illuminate the
sometimes circular path of the dimensions of qualitative inquiry.
We began with the quote from Klee, about a dot that went for a We argue that in one form or another, qualitative research,
walk, and before one realizes, there is a line. The qualitative especially ethnography, involves some data collection, analysis,
researcher shows us a “line” by describing and telling about the and interpretation, although these may not always be as appar-
meanings in order to make them visible. Because of the reflexiv- ent and transparent as a reader may desire. Indeed, a distinc-
ity of all research and the indexicality of all communication, this tive aspect of qualitative research designs that emerge from
is often problematic. The nature and meaning of a person’s lived experience is the blurring of data collection and analysis,
experience is not isomorphic with the researcher’s account of since the latter often informs what new data or examples/
that experience. As Schutz (1967) noted, the lifeworld is a world comparisons to seek out, clarify, and compare. For example,
interpreted by social actors. These are first-order cognitions and Professor Goodall (2006) began with his personal experiences,
constructs; the social scientist or qualitative research must but later came to rely on various documents, including photo-
interpret the actors’ meanings and provide second-order constructs graphs, family and military records, and so forth, for more
and accounts. As noted above, these second-order constructs information. All research involves collecting, organizing, ana-
are made within a social/cultural/historical context, with an lyzing, and interpreting data. Sometimes this involves recog-
intended audience. Our emphasis here is on the ways audience nizing that we have something worthwhile, and then finding
members might critically assess research reports or other repre- out what it is. The most basic concern is perhaps an epistemo-
sentations in order to apply their criteria of validity, adequacy, logical one: How do we know? But this involves several features
or truthfulness. Some of the dimensions we will here examine of the “knowing process,” including what is it that we know,
are basic, yet they provide tools to assist us in assessing qualita- how did we learn or come to know something about it, and
tive research. This raises the issue of the transparency of quali- how do we make sense of this? We are not suggesting any par-
tative research, the ways used by researchers to “connect the ticular approach, mode, or representation for these activities,
dots” of their efforts, from inception to final report or represen- but only that they help distinguish a social science “telling” or
tation. It is not possible to have complete or total transparency, “accounting” from other genres.
again because of the inherent reflexivity and indexicality of the Qualitative research should provide a window for a critical
research process itself, yet most qualitative reports contain reading, or at the very least, permit an informed reader’s queries
guidance about the relatedness of the observations, findings, about what is being read, or seen, or heard. Our position is that
claims, explanations, or conclusions. any claim for veracity, validity, adequacy, or truthfulness turns
Practitioners of qualitative methods routinely encounter cer- on the transparency of these dimensions, and their personal
tain problems and issues in the conduct of their observations, relevance, pertinence, and significance for the audience mem-
experiences, or research. While many of these problems and ber (e.g., reader, listener, viewer). Transparency promotes
issues are legendary in traditional social science ethnography, empathic and sympathetic understanding and participation
rather than drawing on these well-known examples from classi- between the author and the audience. We wish to stress, then,
cal observational research, we will use illustrations from an that it is not enough to “like” a research account/narrative, but
exceptional recent autoethnography, H. Lloyd (Bud) Goodall’s that for social science purposes, such “liking” can be linked to
book A Need to Know: The Clandestine History of a CIA Family these dimensions.
(2006), which reports his experiences as a child growing up in a The power of extensive involvement with the lifeworld one is
family governed by secrecy and paranoia, and routinely under experiencing is apparent from journalistic accounts as well.
the surveillance of the U.S. government. Only upon the death of Journalists do not have a method as such. They inquire and
his father did Goodall learn that he had been a CIA operative. search out leads for a story. Their main connection to a kind of
Bud’s subsequent impassioned quest for answers about his realism is to have “facts checked” by someone else, which often
father’s work, life, and commitments provided some clarity involves someone finding out if a person actually told the jour-
about family history, his mother’s illness, and numerous moves, nalist something, but not whether what that person said was
as well as documenting the institutional and organizational con- actually true. While social scientists operate with a theoretical
texts for real family events, personal lives, and a young boy’s fears orientation that guides data collection and interpretation, being
and insecurities. Bud Goodall’s problem was solved through close to the data, the actual experience, is critical for under-
careful weaving of subtle and tacit knowledge of real-life events standing. Journalists are often closer to the action, and their
that were constructively revealed by replaying narratively shaped descriptions provide great insight, even though their rendering
is less theoretically guided and informed. This can be seen with he came to identify with the character played by Al Pacino. With a
journalist Roberto Saviano’s (2007) statement of how he knows bit of imagination, Schiavone’s hollowed face could actually be
about the corruption of the Camorra, Italy’s brutal criminal superimposed on the actor’s. The story has all the makings of a
organization that dominates commerce in southern Italy. In legend. People say Schiavone even gave his architect a copy of the
describing his familiarity with the criminal control of the film; he wanted the Scarface villa, exactly as it was in the movie.
(pp. 344–345)
cement industry, Saviano states,
I know and I can prove it. I know how economies originate and Many insights found in qualitative research originate in a
where they get their odor. The odor of success and victory. I know researcher’s personal experience. A bit later in the chapter, we
what sweats of profit. I know. And the truth of the war takes no discuss the importance of tacit knowledge. As we move from
prisoners because it devours everything and turns everything into insights or hunches to more robust claims-making, we run into
evidence. It doesn’t need to drag in cross-checks or launch investi- the issue the issue of sampling, that is, how cases were selected.
gations. It observes, considers, looks, listens. . . . The proofs are not Qualitative researchers do not discount research from small
hidden in some flash drive concealed in a hole in the ground. I samples, even a sample of one—a case—but it is helpful if this
don’t have compromising videos hidden in a garage in some inac- information is available. Indeed, we could say that most ideas—
cessible mountain village. Nor do I possess copies of secret service no matter what method is used to develop them—are not dis-
documents. The proofs are irrefutable because they are partial, connected from personal experience. But there is more to it.
recorded with my eyes, recounted with words, and tempered with
Unless it is clear that the account being offered is completely
emotions that have echoed off iron and wood. I see, hear, look, talk,
and in this way I testify, an ugly word that can still be useful when
idiosyncratic or unique, then we also would like to know how a
it whispers, “It’s not true,” in the ear of those who listen to the “case” was selected, as well as whether other cases were exam-
rhyming lullabies of power. The truth is partial; after all, if it could ined, other comparisons made, and if not, are others implied or
be reduced to an objective formula, it would be chemistry. I know suggested? In other words, qualitative researchers are trying to
and I can prove it. And so I tell. About these truths. (p. 213) make statements about the world of experience such as how
things are organized and the consequences of this action. We do
We wish to emphasize that those portions of experience that not know of a single piece of research in which the researcher
are the basis for a particular project, argument, or account need avowed that the work being examined was not relevant for any-
to be available or at least be a point of reference for the reader or thing, any situation, context, application, and the like. The rele-
audience. What, for example, was the basis for a claim or expla- vance might be limited, but the work matters because it is
nation or rendering or narrative with which we are engaged? Is assumed to shed some light on a specific or related problem that
it personal experience (e. g., biographical), a critical event/ goes beyond the particular case being illuminated, referred to,
experience shared with others, an observation—or series of or scrutinized. Again, Roberto Saviano’s (2007) account of the
observations such as an extended ethnography, informal or banality of fear and intimidation illustrates the taken-for-
formal interviews, reflections on documentary “evidence” or granted moral order dictated by the Camorra. Here, he discusses
accounts, and so on? Even if the critical insight is personal, the violation of the code of morality by a witness to a street
emergent from a creative consciousness, is the referent in any killing, who was willing to testify. And there were consequences
sense actual, and if not, in what ways is it connected to the life- for this brave woman:
world presumably shared by readers, viewers, or listeners? An
example is insights about metaphors of life or how identities are It wasn’t testifying in itself that generated such fear, or her identify-
shaped by popular culture. Saviano (2007) reports how domi- ing a killer that caused such a scandal. The logic of Omerta isn’t so
nant Camorra leaders mimic Hollywood bad guys as part of a simple. What made the young teacher’s gesture scandalous is that
she considered being able to testify something natural, instinctive,
lifestyle of domination and control:
and vital. In a land where [lying] is considered to be what gets you
something and [truth] what makes you lose, living as if you actu-
Camorra villas are pearls of cement tucked away on rural streets,
ally believe truth can exist is incomprehensible. So the people
protected by walls and video cameras. There are dozens and doz-
around you feel uncomfortable, undressed by the gaze of one who
ens of them. Marble and parquet, colonnades and staircases,
has renounced the rules of life itself, which they have fully accepted.
granite fireplaces with the boss’s initials. One, the most sumptu-
(pp. 279–280)
ous, is particularly famous, or perhaps it has merely generated the
most legends. Everyone calls it Hollywood. Just saying the word
makes you understand why. . . . Walter Schiavone’s villa really does Saviano’s gathering of various examples for his story provides
have a link to Hollywood. People in Casal di Principe say the boss data for other investigators, criminal and sociological.
told his architect he wanted a villa just like Tony Montana’s, the The expanded focus to other types of data is well illustrated
Miami Cuban gangster in Scarface. He’d seen the film countless with Goodall’s (2006) experience, which began with his own
times and it had made a deep impression in him, to the point that family, but he subsequently branched out to explore how CIA
operatives were dealt with by the government, how their fami- intent here to say which is better or which ones you should trust
lies were treated, and so on. more than the others, but we mainly want to emphasize that the
reader/listener/viewer should be able to discern what was used
I didn’t go into this research project believing that I would be able in general, as well as in specific instances. Any audience mem-
to discover the “truth” about my father’s clandestine career, or even ber will discern his or her own criteria for being “convinced” or
about the meaning of my family’s cold war life. I hoped for some- “skeptical” of the connection between what is being reported
thing less grand. I hoped to find an adequate, even if partial, expla- and its avowed source. But the main concern is that the connec-
nation for what happened to us, and why. (p. 24)
tion be apparent, and to the extent possible, transparent. This
can even be aided by adding a methodological appendix in
This work involved interviews, historical records and other
which parts of the research process are delineated, which we
documents, and even connections with other social science
address shortly below, or even an occasional footnote.
literature, but he was able to relate a unique case—his family—
These general principles of transparency can be applied to
with other cases, and was able to track how organizational
data analysis as well. Data analysis seems self-evident in some
practices and cultures—including the unique culture of the
first-person qualitative reports, but it is seldom as straightfor-
CIA—and how it contributed to family dysfunction, on the one
ward as it seems. The popular applications of “grounded the-
hand, but also foreign policy missteps, on the other.
ory” provide one rationale for coding and comparison, but there
Moreover, it is clear from Goodall’s (2006) excellent mono-
are other modes of comparison as well. The skilled writer can
graph what the different sources of data were, how they were
weave in the creative processes of data explication, comparison,
interwoven, and the reader is given a good understanding of
and triangulation that are usually involved in qualitative
how his subsequent interpretations and conclusions are more or
reports, while in other instances it is helpful to have a section
less closely tied to the various data, while other considerations
(e.g., footnote) delineating “what I did” and “how I came up
may be a bit more speculative and less data bound.
with this.” Again, Goodall’s (2006) use of official government
Saviano (2007) understands what any ethnographer might,
records about his father’s bogus job descriptions clearly illus-
but the way that he knows is not transparent, partly because, as
trated the pervasive bureaucratic duplicity and lies, even when
a journalist, he answers to a different epistemological canon. Yet,
he was trying to understand his deceased father’s life, troubles,
it is compelling enough to motivate researchers to know as
and perspective. His use of these materials helped make an
much as he does, but to be able to offer an account of how we
organizational bureaucratic process of duplicity more visible.
know what we claim. Listen to his testament of authenticity in
The challenge of making our research approach visible underlies
the final pages of his riveting book:
validity issues.
I was born in the land of the Camorra, in the territory with the
most homicides in Europe, where savagery is interwoven with
commerce, where nothing has value except what generates power. 2 Tacit Knowledge and
Where everything has the taste of a final battle. It seemed impos- an Ecology of Understanding
sible to have a moment of peace, not to live constantly in a war
where every gesture is a surrender, where every necessity is trans- Good ethnographies display tacit knowledge. We focus on the
formed into weakness, where everything needs to be fought for
dimensions of “an ecology of understanding.” Contextual, taken-
tooth and nail. In the land of the Camorra, opposing the clans is not
for-granted, “tacit knowledge” plays a constitutive role in pro-
a class struggle, an affirmation of a right, or a expropriation of
one’s civic duty. It’s not the realization of one’s honor or the preser- viding meaning. Goodall (2006) reports some bewilderment as
vation of one’s pride. . . . To set oneself against the clans becomes a a child about what his father did for a living, and where he
war of survival, as if existence itself—the food you eat, the lips you would disappear to for weeks at a time. His parents’ answer, “It’s
kiss, the music you listen to, the pages you read—were merely a complicated,” would be given to other queries over the years.
way to survive, not the meaning of life. Knowing is thus no longer Social life is spatially and temporally ordered through experi-
a sign of moral engagement. Knowing—understanding—becomes ences that cannot be reduced to spatial boundaries, as numer-
a necessity. (pp. 300–301) ous forms of communication attempt to do, especially those
based on textual and linear metaphors. More specifically, expe-
While we are convinced, ethnography must provide more to rience is different from words and symbols about those experi-
maintain authority. ences. Words are always poor representations of the temporal
We have addressed mainly personal experiences and very and evocative lifeworld. Words and texts are not the primary
close—if not biographical—observations and recollections to stuff of the existential moments of most actors in what Schutz
this point. But the same logic about the veracity and relevance (1967) termed the “natural attitude.” They are very significant
of a report resonates with various kinds of interviews, addi- for intellectuals and wordsmiths who claim to represent such
tional observations, and even documents. Again, it is not our experiences. Yet, as those word-workers have come to rely on
and substitute words and other texts for the actual experiences, There is a distinction to be drawn between interesting, pro-
their procedures of analysis have been reified to stand for the vocative, and insightful accounts of ethnographic research, on
actual experience. Therein lies much of the problem that some the one hand, and high-quality ethnographic work. Given our
have termed the “crisis in representation.” emphasis on the reflexive nature of social life, it will not surprise
Capturing members’ words alone is not enough for ethnog- the reader that we prefer those studies that enable the ethno-
raphy. If it were, ethnographies would be replaced by inter- graphic audience to symbolically engage the researcher and
views. Goodall (2006) could not simply have told his story to enter through the research window of clarity (and opportunity).
an interviewer, because part of the story was not clear to him While no one is suggesting a “literal” accounting, our work and
until he reflected more on his past, sought other information that of many others suggests that the more a reader (audience
sources, compared different versions of events, and pieced the member) can engage in a symbolic dialogue with the author
meanings together into a coherent narrative. His book, in other about a host of routinely encountered problems that compro-
words, is not the foundation of his story, but is actually part of mise ethnographic work, the more our confidence increases.
the method he used in discovering, selecting, and interpreting Good ethnographies increase our confidence in the findings,
experiences. Good ethnographies, like Goodall’s, reflect tacit interpretations, and accounts offered.
knowledge, the largely unarticulated, contextual understand-
ing that is often reflected in nods, silences, humor, and
naughty nuances. This is the most challenging dimension of
ethnography, and gets to the core of the members’ perspective,
2 Accounting for Ourselves
or for that matter, the subtleties of membership itself. As Laura A key part of the ethnographic ethic is how we account
Nader (1993) suggests, this is the stuff of ethnography: for ourselves. Good qualitative research—and particularly
“Anthropology is a feat of empathy and analysis” (p. 7). But, ethnographies—shows the hand of the ethnographer. The
without doubting the wisdom of Professor Nader, it is neces- effort may not always be successful, but there should be clear
sary to give an accounting of how we know things, what we “tracks” that the attempt has been made. We are in the midst
regard and treat as empirical materials—the experiences— of a rediscovery that social reality is constructed by human
from which we produce our second (or third) accounts of agents—even social scientists—using cultural categories and
“what was happening.” language in specific situations or contexts of meaning. This
interest is indeed welcomed because it gives us license to do yet
another elucidation of the “concept of knowing.”
2 Reflexive Accounting for Substance Our collective experience in reading a literature spanning
more than 50 years, along with our own work on numerous top-
As we learn more about other significant and essentially invari- ics and projects, suggests that there is a minimal set of problem
able dimensions of settings, such as hierarchical organization, areas that are likely to be encountered in most studies. We do
these are added. In order to satisfy the basic elements of the not offer a solution to the problems that will follow, but only
ethnographic ethic, the following “generic” topics can be found suggest that these can offer a focus for providing a broader and
in ethnographic reports. Goodall’s (2006) descriptions of his more complete account of the reflexive process through which
mother’s practiced attempts to appear more sophisticated and something is understood (Altheide, 1976; Denzin, 1991; Douglas,
the tensions and problems that this created are apt illustrations. 1976; Johnson, 1975). Such information enables the reader to
After noting that his mother’s modest West Virginia childhood engage the study in an interactive process that includes seeking
hardly prepared her for the duties and cultural performances of more information and contextualizing findings, reliving the
a vice consul’s wife, often before the critical eye of Ambassador report as the playing out of the interactions between the
to Italy Clare Boothe Luce, we read the following: researcher, the subjects, and the topic in question.
The idea for the critical reader of an ethnography is to ask
Embassy social gatherings were an American cultural perfor- whether or not any of the basic issues of data collection and
mance. They relied heavily on appearances and the careful cultiva- analysis were likely to have been relevant problems, were they
tion of approved patterns of perceptions. My mother was given explicitly treated as problematic by the researcher, and if so,
detailed instruction on appropriate behavior at social functions
how were they addressed, resolved, compromised, avoided, and
during a State Department orientation for spouses. . . . She also had
so forth. Because these dimensions of ethnographic research
been coached on conversation. She knew how to appear interested
in a topic when she was uninterested in it; how to deflect talk when are so pervasive and important for obtaining truthful accounts,
confronted with an unwanted question or merely something they should be implicitly or explicitly addressed in the report.
tedious or dull; how to appear happy and energetic when she felt Drawing on such criteria enables the ethnographic reader to
tired and unhappy; and how to appear helpless or clueless to get approach the ethnography interactively and critically, and to
out of trouble if the case required it. (p. 141) ask what was done, and how it was done, and what are the likely
and foreseen consequences of the particular research issue, and should accept the inevitability that all statements are reflexive,
how was it handled by the researcher. These dimensions repre- and that the research act is a social act. Indeed, that is the essen-
sent one range of potential problems likely to be encountered by tial rationale for research approaches grounded in the contexts of
an ethnographer. experience of the people who are actually involved in their set-
No study avoids all of these problems, although few research- tings and arenas.
ers give a reflexive account of their research problems and Studies of popular culture and symbolic reality construction
experience. One major problem is that the phenomenon of our are, for some people, ethnographically on the edge; the life-
interest reflects multiple perspectives; there are usually a multi- worlds under investigation are experiential, but representa-
plicity of modes of meanings, perspectives, and activities, even tional of entertainment-oriented media personas and styles.
in one setting. Indeed, this multiplicity is often unknown to Considerable care must be taken to show something of the
many of the official members of the setting. Thus, one does not process, the contexts of understanding and the social relation-
easily “become the phenomenon” in contemporary life. As we ships connecting the symbolic to the actual or the real, as in
strive to make ourselves, our activities and claims, more everyday life discipline, language, and actions involving enforce-
accountable, a critical feature is to acknowledge our awareness ment and resistance.
of a process that may actually impede and prevent our adequate Recent work employing qualitative document analysis (also
understanding of all relevant dimensions of an activity. referred to as “ethnographic content analysis”) illustrates the
Our experience suggests that the subjects of ethnographic application of an ethnographic ethic and tacit knowledge where
studies are invariably temporally and spatially bounded. That the emphasis is on discovery and description, including search-
the range of activities under investigation occurs in time and ing for contexts, underlying meanings, patterns, and processes,
space (which becomes a “place” when given a meaning) pro- rather than mere quantity or numerical relationships between
vides one anchorage, among many others, for penetrating the two or more variables. Michael Coyle’s (2007) study of “The
hermeneutic circle. One feature of this knowledge, of course, is Language of Justice” examined numerous documents, including
its incompleteness, its implicit and tacit dimensions. The quali- dictionaries, to trace the etiology of the terms victim—especially
tative researcher seeks to draw on the tacit dimension in order innocent victim—and evil. His focus was the penetration of
to make meanings and order more explicit, and in a sense, more these terms into public discourse about crime and punishment,
visible. Our subjects always know more than they can tell us, including the popular phrase “tough on crime” that was widely
usually even more than they allow us to see; likewise, we often used by moral entrepreneurs and politicians.
know far more than we can articulate. Even the most ardent Chris Schneider (2008) drew on a lifelong interest and study
social science wordsmiths are at a loss to transform the nuances, of popular music to track how digitized and embedded tech-
subtleties and the sense of the sublime into symbols! For this nologies can simultaneously resist and assist authority and
reason, we acknowledge the realm of tacit knowledge, the inef- social control. His examination of a range of documents, espe-
fable truths, unutterable partly because they are between mean- cially popular music (e.g., rap music), as well as school contexts
ings and actions, the glue that separates and joins human for controlling students’ use of interactive portable technology
intentionality to more concretely focused symbols of practice. (e.g., cell phones, iPods), illuminated how the communication
As we have stressed, the key issue is not to capture the infor- of control is embedded in products marketed for entertainment.
mant’s voice, but to elucidate the experience that is implicated Similarly, as Tim Rowlands (2010) has masterfully shown,
by the subjects in the context of their activities as they perform the world of virtual experience and reality can be investigated,
them, and as they are understood by the ethnographer. Harper’s described, and rendered theoretically through intensive field-
(1987) explanation of how he used photography in a study of a work in cyberspace. As with Coyle’s and Schneider’s studies,
local craftsman illustrates this intersection of meaning: Rowlands’ personal experience was significant in providing
resources and questions to uncover data; years of experience as
The key, I think, is a simple idea that is the base of all ethnography. a serious player of various “computer games” opened up experi-
I want to explain the way Willie has explained to me. I hope to show ences with other players and provided opportunities for infor-
a small social world that most people would not look at very closely. mal interviews to clarify the semiotic scenarios of, for example,
In the process I want to tell about some of the times between Willie EverQuest, that could be conceptually treated as exploration of
and me, thinking that at the root of all sociology there are people one virtual utopia, which hinted of Saviano’s Camorra. What
making connections, many like ours. (p. 14) were the rules, Rowlands asked, what were the underlying
assumptions of order, and above all, what does justice look like
One approach to making ourselves more accountable and in virtual space? He found violence and the iconography of vio-
thereby sharing our experience and insights more fully with read- lence, but also a technologically rendered cosmology of alien-
ers, is to locate inquiry within the process and context of actual ated confrontations as task completion for role fulfillment that
human experience. Our experience suggests that researchers players recognized and pursued as pastimes of boredom, fun,
work, and as an inevitable feature of the computer environment can become tighter within the community of scholars who
that reflected a capitalistic order. Examining the evidence employ certain methods and criteria. While we do not recom-
within the confines of interpretive validity suggests, “MMOs mend “recipe” methods, we have offered several “lists” of items
[massive multiplayer online games] such as EverQuest serve in to consider when focusing on the presentation of research prob-
important ways to shape our understanding of what virtual lems and solutions in a given project (Altheide & Johnson,
worlds currently are” (p. 369). 1994). For example, many approaches to interviewing (e.g.,
Let us bring the parts of the argument closer together. Evi- focused, life history, etc.), ethnography (e.g., grounded theory),
dence is a feature of the interaction between an audience, a and document analysis (qualitative content analysis) have
claim, and practical epistemologies of everyday life. Whether developed criteria and procedures for optimal work. But our
evidence is convincing follows along the same lines; is the evi- task is not to refine and squeeze the novelty and richness out of
dence “good enough,” or is there enough evidence relevant to the experience in favor of some bygone notion of rigor and effi-
topic? The key question is, what is relevant for the topic or ques- ciency, nor is it to make sure that creative problem solving and
tion under investigation? A reader’s (listener’s) perspective discovery are compromised in order to dot the i’s and cross the
about what is “evidence” is informed by biography, culture, and t’s. We want to see more dots flourish and evolve into creative
so forth. insights. Our task is to continue pushing the line in new direc-
tions to illuminate our humanity and communicative worlds.
2 Conclusion
The great artist Paul Klee, who stated that a line is a dot that went 2 References
for a walk, could enjoy the task of trying to capture the intersec-
Altheide, D. L. (2008). The evidentiary narrative: Notes toward a sym-
tion of many lines—borne of numerous dots—that are heading bolic interactionist perspective about evidence. In N. K. Denzin &
in different directions. We seek, among other things, to under- M. D. Giardina (Eds.), Qualitative inquiry and the politics of evi-
stand the nature, process, and consequences of social interac- dence (pp. 137–162). Walnut Creek, CA: Left Coast Press.
tion, on the one hand, and how this promotes individual and Altheide, D. L. (2009). Terror post 9/11 and the media. New York: Peter
joint renderings of a definition of a situation, on the other hand. Lang.
A positivistic view of validity works fine in a different social Altheide, D. L., & Johnson, J. M. (1993). The ethnographic ethic. In
universe where there are not multiple perspectives, vastly differ- N. K. Denzin (Ed.), Studies in symbolic interaction (pp. 95–107).
ent methods and materials with which to work, and myriad uses Greenwich, CT: JAI Press.
and audiences. But that is not our social research world. The Altheide, D. L., & Johnson, J. M. (1994). Criteria for assessing interpre-
social world and its human actors make and interpret meanings tive validity in qualitative research. In N. K. Denzin & Y. S. Lincoln
through an interaction process that contributes to the construc- (Eds.), Handbook of qualitative research (pp. 485–499). Newbury
Park, CA: Sage.
tion, reification, and resistance of social reality. Any method that
Altheide, D. L., & Snow, R. S. (1979). Media logic. Beverly Hills, CA: Sage.
obliterates the essential role of emergence, negotiation, and tacit Atkinson, P. (1990). The ethnographic imagination: Textual constructs
knowledge will not be valid. And any effort to remake this world of reality. New York: Routledge.
to comply with an idealized model is folly, will lack credibility, Bhaskar, R. (1979). The possibility of naturalism: A philosophical cri-
and is doomed to failure. Our overview of validity issues in tique of the human sciences. Brighton, UK: Harvester.
qualitative research, with a focus on ethnographic reports, sug- Bochner, A. P. (2007). Notes toward an ethics of memory in autoethno-
gests that a proper set of standards or criteria for assessing valid- graphic inquiry. In N. K. Denzin & M. D. Giardina (Eds.), Ethical
ity entails considering the place of evidence in an interaction futures in qualitative research: Decolonizing the politics of knowl-
process between the researcher, the subject matter (phenome- edge (pp. 197–208). Walnut Creek, CA: Left Coast Press.
non to be investigated), the intended effect or utility, and the Cabinet Office. (2003). Quality in qualitative evaluation: A framework
audience for which the project will be evaluated and assessed. for assessing research evidence [Full report]. London: Author.
We continue to hold that focusing on the process of investi- Coyle, M. J. (2007). The language of justice: Exposing social and crimi-
nal justice discourse. Unpublished doctoral dissertation, School of
gation and communicating that process, the problems and solu-
Justice and Social Inquiry, Arizona State University, Tempe.
tions encountered in accessing, collecting, analyzing, and inter- Denzin, N. K. (2009). Qualitative inquiry under fire. Walnut Creek, CA:
preting data—to the best of our ability—is quite consistent Left Coast Press.
with analytic realism, or the general notion that the social world Denzin, N. K. (2010). A qualitative stance: Remembering Steinar Kvale
is an interpreted one. We are not opposed to using evidence, but (1938–2008). International Journal of Qualitative Studies in Edu-
prefer that the spirit of the evidentiary narrative be considered. cation, 23(2), 125–127.
We do not intend to leave the question of validity completely Denzin, N. K., & Giardina, M. D. (Eds.). (2008). Qualitative inquiry and
open-ended. There are parameters, and the criteria and process the politics of evidence. Walnut Creek, CA: Left Coast Press.
Denzin, N. K., & Lincoln, Y. S. (Eds.). (1994). Handbook of qualitative Mancias, P. T., & ebrary, Inc., 2006. A realist philosophy of social science
research. Newbury Park, CA: Sage. explanation. Cambridge, UK: Cambridge University Press.
Ellis, C. (2004). The ethnographic I. Walnut Creek, CA: AltaMira Press. Marcus, G. E., & Clifford, J. (1986). Writing culture: The poetics and
Ellis, C. (2009). Revision: Autoethnographic reflections on life and work. politics of ethnography. Berkeley: University of California Press.
Walnut Creek, CA: Left Coast Press. Maxwell, J. A. (2008). The value of a realist understanding of causality
Emerson, R. M., Fretz, R. I., & Shaw, L. L. (1995). Writing ethnographic for qualitative research. In N. K. Denzin & G. D. Giardina (Eds.),
fieldnotes. Chicago: University of Chicago Press. Qualitative inquiry and the politics of evidence (pp. 163–181).
Erikson, R. V., & Doyle, A. (2003). Risk and morality. Toronto, ON, Walnut Creek, CA: Left Coast Press.
Canada: University of Toronto Press. Miller, W. L., & Crabtree, B. F. (2005). Clinical research: Participatory
Fuller, S. (2008). Science democratized—Expertise decommissioned. action research. In N. K. Denzin & Y. S. Lincoln (Eds.), The
In N. Stehr (Ed.), Knowledge and democracy (pp. 105–117). New SAGE handbook of qualitative research (3rd ed., pp. 605–639).
Brunswick, NJ: Transaction. Thousand Oaks, CA: Sage.
Goodall, H. L., Jr. (2006). A need to know: The clandestine history of a Nader, L. (1993). Paradigm busting and vertical linkage. Contemporary
CIA family. Walnut Creek, CA: Left Coast Press. Sociology, 33, 6–7.
Guba, E. G. (1990). Subjectivity and objectivity. In E. W. Eisner & Putnam, H. (1999). The threefold cord: Mind, body, and the world.
A. Peshkin (Eds.), Qualitative inquiry in education (pp. 74–91). New York: Columbia University Press.
New York: Teachers College Press. Ragin, C., Nagel, J., & White, P. (2004). Workshop on scientific founda-
Haak, S. (2003). Defending science—within reason. Amherst, NY: tions of qualitative research. Washington, DC: National Science
Prometheus. Foundation.
Hammersley, M. (1990). Reading ethnographic research. London: Richardson, L. (1997). Fields of play. New Brunswick, NJ: Rutgers
Longman. University Press.
Hammersley, M. (1992). What’s wrong with ethnography? Methodologi- Rolfe, G. (2004). Validity, trustworthiness, and rigor: Quality and the
cal explorations. London & New York: Routledge. idea of qualitative research. Journal of Advanced Nursing, 53(3),
Hammersley, M. (2005). The myth of research-based practice: The 304–310.
critical case of educational inquiry. International Journal of Social Rowlands, T. E. (2010). Empire of the hyperreal. Unpublished doctoral
Research Methodology, 8(4), 317–330. dissertation, School of Social Transformation, Arizona State
Harper, D. (1987). Working knowledge: Skill and community in a small University, Tempe.
shop. Berkeley: University of California Press. Saviano, R. (2007). Gomorrah. New York: Farrar, Straus and Giroux.
Harré, R., & Madden, E. (1975). Causal powers. Oxford, UK: Basil Schneider, C. J. (2008). Mass media, popular culture and technology:
Blackwell. Communication and information formats as emergent features of
Henry, G., Julnes, G. J., & Mark, M. (Eds.). (1998). Realist evaluation. social control. Unpublished doctoral dissertation, School of Jus-
San Francisco: Jossey-Bass. tice and Social Inquiry, Arizona State University, Tempe.
Johnson, J. M. (1975). Doing field research. New York: Free Press. Schutz, A. (1967). The phenomenology of the social world. Evanston, IL:
Kemmis, S., & McTaggart, R. (2005). Participatory action research: Northwestern University Press.
Communicative action in the public sphere. In N. K. Denzin & Stehr, N. (2008). Moral markets. Boulder, CO: Paradigm.
Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research Szto, P., Furman, R., & Langer, C. (2005). Qualitative research in sociol-
(3rd ed., pp. 599–603). Thousand Oaks, CA: Sage. ogy in Germany and the U.S. Focus: Qualitative Social Work, 4(2),
Kvale, S., & Brinkmann, S. (2008). InterViews: Learning the craft of 135–156.
qualitative research interviewing. Thousand Oaks, CA: Sage. Torrance, H. (2007). Building confidence in qualitative research:
Ladkin, D. (2004). Action research. In C. Sale, G. Gobo, J. F. Gubrium, & Engaging the demands of policy. In N. K. Denzin & M. D. Giardina
D. Silverman (Eds.), Qualitative research practice (pp. 536–48). (Eds.), Qualitative inquiry and the politics of evidence (pp 55–79).
New York: Routledge. Walnut Creek, CA: Left Coast Press.
Lakoff, G. (1987). Women, fire, and other dangerous things: What catego- Van den Hoonaard, W. C. (2002). Walking the tightrope: Ethical issues
ries reveal about the mind. Chicago: University of Chicago Press. for qualitative researchers. Toronto. ON, Canada: University of
Lamont, M., & White, P. (2009). Workshop on interdisciplinary stan- Toronto Press.
dards for systematic qualitative research. Washington, DC: National Van Maanen, J. (1988). Tales of the field: On writing ethnography.
Science Foundation. Chicago: University of Chicago Press.
Lather, P. (1993). Fertile obsession: Validity after poststructuralism. Wolcott, H. F. (1990). On seeking—and rejecting—validity in qualitative
Sociological Quarterly, 34, 673–93. research. In E. W. Eisner & A. Peshkin (Eds.), Qualitative inquiry in
Lofland, J. (1995). Analytic ethnography. Journal of Contemporary education: The continuing debate (pp. 121–152). New York: Teachers
Ethnography, 24(1), 30–67. College Press.
View publication stats

Criteria For Assessing Interpretive Validity in Qualitative Research

Uploaded by

Copyright:

Available Formats

Criteria For Assessing Interpretive Validity in Qualitative Research

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Criteria For Assessing Interpretive Validity in Qualitative Research

Uploaded by

Copyright:

Available Formats

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

Criteria for assessing interpretive validity in qualitative research

Article · January 1988

The user has requested enhancement of the downloaded file.

A line is a dot that went for a walk.

View publication stats

You might also like