Critical Realism A Way Forward in IS Research
Critical Realism A Way Forward in IS Research
net/publication/221407194
CITATIONS READS
20 1,891
1 author:
Sven Carlsson
Lund University
143 PUBLICATIONS 2,150 CITATIONS
SEE PROFILE
All content following this page was uploaded by Sven Carlsson on 30 May 2014.
Abstract
Although different strands of “post-approaches” and “post-theories”, for example, grounded
theory and structuration theory, have gained popularity in Information Systems (IS) research
they are not without problems. We present critical realism as an alternative philosophical
underpinning for IS research. Critical realism starts from an ontology that identifies
structures and mechanisms through which they are generated as being fundamental to the
constitution of our natural and social reality. The paper presents critical realism and Derek
Layder’s adaptive theory—this is based on the principles of critical realism—and exemplifies
how they can be used in IS-research, for example, in theory development and evaluation
research.
Keywords
Critical realism, after postmodernism, IS research, theory development, evaluation research
1. Introduction
Commentators on IS-research have pointed out weaknesses in positivist, realist, and other
modernist approaches and theories for research on the design, development, and use of ICT-
based Information Systems (IS). In response, IS-researchers have used different types of
“post-approaches” and “post-theories”, for example, grounded theory and ethnography as
well as structuration theory and actor network theory. The alternatives overcome some of the
problems noted by the commentators. Given that we in the last years seen an increase in
published IS-research utilizing postmodern, posstructuralist, postrealist, and nonpositivistic
approaches and theories, these alternative approaches and theories as well as IS-research
based on them need to be scrutinized. (For simplicity, we will refer to these different
approaches and theories as “post-approaches” and “post-theories” when distinction is not
required.)
The purpose of this paper is threefold. First, to point out some of the limitations and
weaknesses in different post-approaches and post-theories for studying IS. Second, to present
critical realism as an alternative philosophical underpinning for IS research. Third, to
exemplify how critical realism and Derek Layder’s (1998) adaptive theory approach can be
used in IS research— Layder’s adaptive theory builds on the principles of critical realism.
The remainder of the paper is organized as follows: the next section briefly presents and
discusses different “post-approaches” and “post-theories” and points out some limitations and
weaknesses in these theories and approaches. Next we present critical realism. This is
Sven A. Carlsson Critical Realism: A Way Forward in IS Research
followed by a discussion of how critical realism and Derek Layder’s adaptive theory
approach can be used in IS research.
2. Problems in Paradise
In response to the cry for the use of post-approaches and post-theories in IS research,
researchers have used, for example, approaches like qualitative approaches, interpretive
approaches and grounded theory as well as theories like Anthoy Giddens’ structuration
theory—for different IS-examples, see, Lee et al. (1997), Trauth (2001), Whitman and
Woszczynski (2004), and Michael Myers’ “Qualitative Research in Information Systems”
(http://www.qual.auckland.ac.nz/).
We will not do an exhaustive review of different post-approaches and post-theories, but will
point out limitations and weaknesses in: 1) one approach for generating theories, grounded
theory, 2) one “theory” (description) of human action and social organization, structuration
theory, and 3) the suggestions to integrate and combine different approaches in IS research,
for example combining positivist and interpretive approaches.
Several IS-scholars have suggested the use of grounded theory and a number of IS studies
using grounded theory have been published—for a good example, see Urquhart (2001).
Generally, grounded theory is an approach to the analysis of qualitative data aiming at
generating theory out of research data by achieving a close fit between the two (Glaser &
Strauss, 1967). Said Strauss and Corbin, “… theory that was derived from data,
systematically gathered and analyzed through the research process. In this method, data
collection, analysis, and eventual theory stand in close relationship to one another.” (Strauss
& Corbin, 1998). One of the weaknesses in grounded theory is its concentration on micro
phenomena: “The very fixity of this concentration is a factor which prevents grounded theory
from attending to historical matters of macro structure as a means of enriching contemporary
or, as I [Layder] shall call them, present-centred forms of research on micro phenomena. It
should be possible to augment the processual and dynamic analyses of interactional
phenomena by a parallel focus on the historically antecedent forms that provide their
institutional backdrop.“ (Layder, 1993). Macro phenomena have no validity to IS-researchers
using grounded theory unless these macro phenomena emerge directly from the field data.
But, research suggests that macro phenomena, like national culture, influence IS-designers
(Hunter & Beck, 1996) and how IS are used and evaluated (Tan et al., 1995; Leidner et al.,
1999). Macro phenomena (structural/systemic factors) can hardly emerge in IS research
focusing on agents’ perceptions, meanings, and actions. Other weaknesses pointed out by
Layder include how power can be handle in grounded theory. Grounded theory focuses
situated and interpersonal aspects. This means that a researcher using grounded theory will
most likely miss the importance of power ”behind the scenes” of situated activities.
A theory having gained popularity among IS-researchers is Anthony Giddens’ (1984)
structuration theory. Most notably is Orlikowski’s work on applying structuration theory to
the development and use of IS in organizations (Orlikowski, 1992, 2000). Figure 1 depicts
Orlikowski’s structurational model of technology and shows the relationship between
technology, human agents, and institutional properties.
According to Orlikowski (1992), technology is identified as the “product of human action”
(arrow a), coming into existence and being sustained through human action, and being
constituted through use. Only through the appropriation of technology by humans does it
exert influence. However, technology is also “the medium of human action” (arrow b).
Technology constrains and enables social practices. Institutional properties influence human
Sven A. Carlsson Critical Realism: A Way Forward in IS Research
Institutional Properties
d
Technology
c
a
b
Human Agents
Structuration theory overcomes some of the problems associated with realist and structuralist
theories, but Giddens’ view on agency and structure is problematic when studying artifacts
like IS. Giddens’ conception of agency and structure means that structure cannot be separated
from agency. It is an attempt to treat human action and social structure as a duality rather
than a dualism; action and structure are seen as two aspects of the same whole (a duality).
According to Reed, it is “a single-level social ontology that conflates ‘agency’ and ‘structure’
in such a way that they are analytically rendered down to localized social practices bereft of
any institutional underpinnings or contextualization. The ontological status and explanatory
power of ‘structure’—i.e., as a concept referring to a relatively enduring institutionalised
relationships between social positions and practices located at different levels of analysis that
constrain actors’ capacities to ‘make a difference’—is completely lost in a myopic analytical
focus on situated social interaction and the local conversational routines through which it is
reproduced.” (Reed, 1997). Jones (1999) after reviewing IS research based on Giddens’
structuration theory concludes: “…it is evident that the specific attempts to adapt
structuration to incorporate material aspects of IS have encountered a number of serious
problems which remain as yet unresolved”.
Some IS-scholars advocate that IS-researchers should integrate or combine positivist and
interpretive approaches (Lee, 1991; Trauth & Jessup, 2000), integrate case study and survey
research methods (Gable, 1994), or combine qualitative and quantitative methods (Kaplan &
Duchon, 1988). These suggestions seem to have some similarities with what critical realism
writers suggest in terms of using methods and techniques, but there is one major difference.
Critical realism writers are based in a specific philosophy of social sciences and in discussing
how to use different methods and techniques in research they start from the ontology and
epistemology of critical realism. IS-writers arguing for integrating and mixing different
methods and techniques do this without discussing their ontological and epistemological
views.
Summarizing, from an IS research perspective we can identify at least four major problems
with the use of different strands of post-approaches and post-theories as well as the idea to
integrate and combine different approaches and methods. First, the post-approaches and post-
theories fascination with the voices of those studied have lead to an increase in IS research as
Sven A. Carlsson Critical Realism: A Way Forward in IS Research
mere reportages and local narratives. Second, their focus on agency leads to that they ignore
the structural (systemic) dimension. Third, their rejection of objectivist elements leads to
problem when researching artifacts like IS. Fourth, the idea to integrate and combine
different approaches and methods, founded in different and incompatible ontologies, are
presented without an elaborate discussion of how to “handle” the incompatible ontologies.
We are not claiming that the criticized approaches and theories cannot be useful in IS
research, but that they have a number of limitations and that they certainly not are panaceas.
To overcome some of the noted problems, Jones (1999), Walsham (1995), and Mingers
(2001a) have argued that the development of a “position” based on critical realism could be a
valuable avenue to explore in IS research. Next section presents critical realism.
3. Critical Realism
Critical realism was developed as an alternative to traditional positivistic models of social
science as well as an alternative to postmodern approaches and theories. The most influential
writer on critical realism is Roy Bhaskar (1978, 1989, 1998). Archer et al. (1998) and Lòpez
and Potter (2001) contain chapters focusing on different aspects of critical realism, ranging
from fundamental philosophical discussions to how statistical analysis can be used in critical
realism research.
Critical realism can be seen as a specific form of realism. Its manifesto is to recognize the
reality of the natural order and the events and discourses of the social world. It holds that “we
will only be able to understand—and so change—the social world if we identify the
structures at work that generate those events and discourses … These structures are not
spontaneously apparent in the observable pattern of events; they can only be identified
through the practical and theoretical work of the social sciences.” (Bhaskar, 1989). Bhaskar
(1978) outlines what he calls three domains: the real, the actual, and the empirical (Table 1).
The real domain consists of underlying structures and mechanisms, and relations; events and
behavior; and experiences. The generative mechanisms, residing in the real domain, exist
independently of but capable of producing patterns of events. Relations generate behaviors in
the social world. The domain of the actual consists of these events and behaviors. Hence, the
actual domain is the domain in which observed events or observed patterns of events occur.
The domain of the empirical consists of what we experience, hence, it is the domain of
experienced events.
Table 1. Ontological assumptions of the critical realist view of science (Bhaskar 1978).
Xs indicate the domain of reality in which mechanisms, events, and experiences,
respectively reside, as well as the domains involved for such a residence to be possible.
Bhaskar argues that “…real structures exist independently of and are often out of phase with
the actual patterns of events. Indeed it is only because of the latter we need to perform
experiments and only because of the former that we can make sense of our performances of
them. Similarly it can be shown to be a condition of the intelligibility of perception that
Sven A. Carlsson Critical Realism: A Way Forward in IS Research
Element Focus
We will briefly present the different elements and, for convenience, start with the self and
work towards the macro elements. The first level is self, which refers “... primarily to the
individual’s relation to her or his social environment and is characterized by the intersection
of biographical experience and social involvements.“ (Layder, 1993). Self focuses on how an
individual is affected by and responds to social situations. In encountering social situations
individuals use strategies and tactics, based on their ”theories” (mental models), to handle the
situations. In general, the self and situated activity have as their main concern “...the way
individuals respond to particular features of their social environment and the typical
situations associated with this environment.“ (Layder, 1993).
In situated activity the focus is on the dynamics of social interaction. The area of self focuses
how individuals are affected and respond to certain social processes whereas situated activity
focus on the nature of the social involvement and interactions. This means that interactions
and processes have features that are the result of how the participating individuals’ behaviors
intermesh and coalesce.
The focus in setting is on the intermediate forms of social organization. A setting provides
the immediate arena for social activities. A setting can be things like the culture of the
organization, artifacts like ICT-based IS that are used in situated activities, power and
authority structures. It should be stressed that setting is not just a particular patterns of
activity. The wider macro social forms that provide the more remote environment of social
activity is refereed to as the context. Although there is not a clear border between settings and
context and some social forms straddle the two elements it can be fruitful to distinguish them.
In general, context refers to large-scale and society-wide features.
Viewing the design, development, implementation, and use of IS as layers of human activity
and social organization that are interdependent has two major advantages. It enables a
researcher to be sensitive to the different elements with their distinctive features. Critical
realism and Layder’s framework stress that the layers operate on different ”time scales”. This
means that a researcher has to view the operation of the elements not only vertically but also
horizontally.
Critical realism has influenced a number of social sciences fields, e.g., organization studies.
With few exceptions, critical realism is almost invisible in the IS-field. Mutch (1997, 2002)
and Dobson (2001) argue for the use of critical realism in IS research and discuss how critical
realism can overcome problems associated with postmodern approaches and theories, e.g.
Mutch (2000) notes how critical realism can overcome problems in actor network theory.
Mingers (2001a) used, in part, critical realism to argue for the use of pluralist methodologies
in IS research. He also used an approach influenced by critical realism in reviewing the use
of multimethod research in the IS literature (Mingers 2001b).
are loosely and flexibly positioned in relation to each other. Layder stresses that theorizing
should be a continuous process accompanying the research at all stages.
Choice of topic/problem
Theoretical deliberations
Data analysis and theorizing
To exemplify how Layder’s adaptive theory can be used in IS research, we will use a study
on Executive Information Systems (EIS) carried out in the US, Sweden and Mexico. (The
study was done together with Dorothy Leidner and the results have been published in several
papers and articles.) Here a new discussion of the research is carried out. The overall purpose
of the study was to increase our knowledge on the development and use of EIS.
Layder’s adaptive theory approach has eight overall parameters. One parameter says that
adaptive theory “uses both inductive and deductive procedures for developing and
elaborating theory.” (Layder, 1998). The adaptive theory suggests the use of both forms of
theory-generation within the same frame of reference and particularly within the same
research project and time-frame. In our project we used both forms of theory-generation.
Based on previous EIS studies and theories as well as Huber’s (1990) propositions on the
effects of advanced ICT on organizational design, intelligence, and decision making, we
generated a number of hypotheses (a deductive procedure). These were empirically tested.
Critical realism proceeds by trying to discover underlying structures that generate particular
events and patterns. Statistical analysis can be used to, for example, detect patterns
(tendencies) within the data (Porpora, 2001; Mingers, 2000). The result was used as a starting
point for more substantive investigations. In the same project we also used an inductive
procedure. Although, previous theories as well as the results from other parts of the project
were fed into the inductive procedure we primarily used and inductive approach to from the
data generate patterns of EIS use.
Another parameter says that adaptive theory “rests upon an epistemological position which is
neither positivist nor interpretivist.” (Layder, 1998). Adaptive theory, in order to look for the
most powerful forms of explanation, draws upon both positivist and interpretivist theories. In
an effort to transcend the limitations of the theories, adaptive theory occupies the
intermediate ground between both theories. In our study this was, in part, accomplished by
using and combining positivistic research and interpretive research, but this was done from
critical realism ontology.
One parameter says that adaptive theory “embraces both objectivism and subjectivism in
terms of its ontological presuppositions” (Layder, 1998). The adaptive theory conceives the
social world as including both subjective and objective aspects and mixtures of the two. In
our project, one objective aspect was the ICT used in the different EIS—viewed as a
generative mechanism—and one subjective aspect was perceived effects of EIS use. (This
can be contrasted with IS-research based on structuration theory, which views technology as
“instantiations.”)
Sven A. Carlsson Critical Realism: A Way Forward in IS Research
Two other parameters say that adaptive theory “assumes that the social world is complex,
multi-faceted (layered) and densely compacted” and “focuses on the multifarious
interconnections between human agency, social activities and social organization (structures
and systems)” (Layder, 1998). The idea that the world comprises both subjective and
objective features means that adaptive theory assumes that “the social world is complex and
dense. Furthermore, it also assumes that the texture of this complexity and density is formed
from the multifarious interconnections between agency and structure.” (Layder, 1998). In our
study we focused the interconnections between agency and structure. We addressed self, e.g.,
perceptions of EIS, situated activity, e.g., use of EIS in day-to-day work, setting, e.g.
organizational structure and culture, and context, e.g., national culture and economic
situation. We found, for example, that national culture influenced how EIS were used and
how they were perceived. We also found “interconnections” between EIS use and
organizational strategy and organizational structure. The “effects” of these generative
mechanisms “showed up” in the domain of experienced events.
Our study and the results (theory) were influenced by, e.g. Huber’s propositions and the
theory saying that EIS are systems for providing top-managers with critical information, as
well as we were influenced by Robert Quinns’ competing values approach (Quinn, 1988;
Quinn & Rohrbaugh, 1983; Quinn et al,.1996). The latter theory was brought in to theorize
around the data from the interpretive part of the study. Adaptive theorizing was ever present
in the research process. Theorizing was done before “hypotheses-testing” was done. We used
theories about national culture (extant theory) to generate hypotheses (theorizing) on how
national culture (context) could influence how EIS were used and perceived (situated activity
and self). Also in the “interpretive” part was theorizing present. It was present in generating
questions—using extant theory and the results from the hypotheses-testing—as well as in
analyzing the collected data. Based on our study, we argued that it is a misconception to think
of EIS as systems that just provide top-managers with information. EIS are systems that
support managerial cognition and behavior—providing information is only one of several
means—as well as it can be one important means in organizational focusing and change. The
above is related to the parameter saying that adaptive theory “both shapes, and is shaped by
the empirical data that emerges from research. It allows the dual influence of extant theory
(theoretical models) as well as those that unfold from (and are enfolded in) research.
Adaptive theorizing is an ever-present feature of the research process” (Layder, 1998).
We also envision that critical realism can make a major contribution in IS evaluation
research. Work has started on developing a realistic IS evaluation perspective (Carlsson,
2003) which builds on critical realism and realistic evaluation (Pawson & Tilley, 1997).
Driving this is the aim to produce ever more detailed answers to the question of why an IS
initiative works for whom and in what circumstances. This means that evaluators attend to
how and why an IS initiative has the potential to cause (desired) change.
In this perspective an evaluator works as an experimental scientist, but not according to the
logics of the traditional experimental evaluation research school. Said Bhaskar: “The
experimental scientist must perform two essential functions in an experiment. First, he must
trigger the mechanism under study to ensure that it is active; and secondly, he must prevent
any interference with the operation of the mechanism. These activities could be designated as
‘experimental production’ and ‘experimental control’.” (Bhaskar 1998). Figure 4 depicts the
realist experiment.
Sven A. Carlsson Critical Realism: A Way Forward in IS Research
Experimental
production: fires
mechanism
Context (C)
Mechanism (M)
Experimental
Regularity (X-Y) control: disables
extraneous
mechanism
Other mechanisms
Realist evaluators do not conceive that IS initiatives “work”. It is the action of stakeholders
that makes them work, and the causal potential of an IS initiative takes the form of providing
reasons and resources to enable different stakeholders and participants to “make” changes.
This means that a realist evaluator seeks to understand why an IS initiative (IS
implementation) works through an understanding of the action mechanisms. It also means
that a realist evaluator seeks to understand for whom and in what circumstances (contexts) an
IS initiative works through the study of contextual conditioning.
Realist evaluators orient their thinking to context-mechanism-outcome pattern configurations
(CMO configurations). This leads to the development of transferable and cumulative lessons
from IS evaluation research. A CMO configuration is a proposition stating what it is about an
IS initiative (IS implementation) which works for whom in what circumstances. A refined
CMO configuration is the finding of an IS evaluation.
Realist evaluators examine outcome patterns in a theory-testing role. This means that a realist
evaluator tries to understand what are the outcomes of an IS initiative and how are the
outcomes produced. Hence, a realist evaluator is not just inspecting outcomes in order to see
if an IS initiative works, but are analyzing the outcomes to discover if the conjectured
mechanism/context theories are confirmed.
In terms of generalization, a realist evaluator through a process of CMO configuration
abstraction creates “middle range” theories. These theories provide analytical frameworks to
interpret differences and similarities between types of IS initiative (IS implementations).
Given that the goal is develop theories—construct and test context-mechanism-outcome
pattern explanations—for practitioners, stakeholders, and participants, IS evaluators need to
engage in a teacher-learner relationship with these IS practitioners, stakeholders, and
participants.
Sven A. Carlsson Critical Realism: A Way Forward in IS Research
Realistic IS evaluation research design employs no standard formula. The base strategy is to
develop a clear theory of IS initiative mechanisms, contexts and outcomes. Given the base
strategy, an evaluator has to design appropriate empirical methods, measures, and
comparisons. Realistic IS evaluation research is supportive of the use of both quantitative and
qualitative evaluation methods.
References
Archer, M, Bhaskar, R, Collier, A, Lawson T & A. Norrie, A (eds)(1998), Critical realism:
essential readings, Routledge, London.
Bhaskar, R (1978), A realist theory of science, Harvester Press, Sussex.
Bhaskar, R (1989), Reclaiming reality, Verso, London.
Bhaskar, R (1998), The possibility of naturalism, 3rd edn, Routledge, London.
Bryman, A (2001), Social research methods, Oxford University Press, Oxford.
Carlsson, SA (2003), ‘Information systems evaluation research: alternative perspectives and
schools’, in Proceedings of The Second European Conference on Research
Methodology for Business and Management Studies, Reading, pp. 59-66.
Dobson, PJ (2001), ‘The philosophy of critical realism—an opportunity for information
systems research’, Information Systems Frontier, vol. 3, no. 2, pp. 199-201.
Gable, G (1994), ‘Integrating case study and survey research methods: an example in
information systems’, European Journal of Information Systems, vol. 3, no. 2, pp. 112-
126.
Giddens, A (1984), The constitution of society, Polity Press, Cambridge, UK.
Glaser, BG & Strauss, AL (1967), The discovery of grounded theory, Aldine, Chicago.
Huber, GP (1990), ‘A theory of the effects of advanced information technologies on
organizational design, intelligence, and decision making’, Academy of Management
Review, vol. 15, no. 1, pp. 47-71.
Hunter, MG & Beck, JE (1996), ‘A cross-cultural comparison of ‘excellent’ systems
analysts’, Information Systems Journal, vol. 6, no. 4, pp. 261–281.
Jones, M (1999), ‘Structuration theory’ in Rethinking management information systems, ed
WL Currie & B Galliers, Oxford University Press, Oxford, pp. 103-135.
Sven A. Carlsson Critical Realism: A Way Forward in IS Research
Strauss, A & Corbin, JM (1998), Basics of qualitative research: techniques and procedures
for developing grounded theory, Sage, Thousand Oaks, CA.
Tan, BCY, Watson, RT & Wei, K-K (1995) ‘National culture and group support systems:
filtering communication to dampen power differentials’, European Journal of
Information Systems, vol. 4, pp. 82–92.
Trauth, EM (ed)(2001), Qualitative research in IS: issues and trends, Idea Group Publishing,
Hershey, PA.
Trauth, E & Jessup, L (2000), ‘Understanding computer-mediated discussions: positivist and
interpretive analyses of group support system use’, MIS Quarterly, vol. 24, no. 1, pp.
43-79.
Urquhart, C (2001), ‘An encounter with grounded theory: tackling the practical and
philosophical issues’ in Qualitative research in IS: issues and trends, ed EM Trauth,
Idea Group Publishing Hershey, PA, pp. 104-140
Walsham, G (1995), ‘Interpretive case studies in IS research: nature and method’, European
Journal of Information Systems, vol. 4, no. 1, pp. 74–81.
Whitman, ME & Woszczynski, AB (eds.)(2004), Handbook for information systems
research, Idea Group Publishing, Hershey, PA, forthcoming.