Extensive Theory Testing Using Case Study

Extensive Theory Testing Using Case Study

Completed Research Paper

Ignatius Chukwudi Meng Zhang

Queensland University of Technology Queensland University of Technology
Y706C-12, QUT Gardens Point Y706C-34, QUT Gardens Point
Campus, Brisbane, QLD, Australia Campus, Brisbane, QLD, Australia
[email protected] [email protected]

Guy Gable
Queensland University of Technology
Y704, QUT Gardens Point Campus
Brisbane, QLD, Australia
[email protected]

Abstract
Extensive theory testing refers to the joint conduct of theory testing and extension in a
single empirical study. One of the types of extensive theory testing is using case study
approach. While quantitative approach to theory testing, such as survey method,
focuses on corroborating or falsifying theory, extensive theory testing aims to both
corroborate and extend theory at the same time. By employing extensive theory testing,
researchers can simultaneously develop deeper understandings of an existing theory
situated within a context and generate new insights based on the effort to test the
existing theory. Thus, extensive theory testing offers a useful alternative,
complementing quantitative approach to theory testing. Although case study has been
used for theory testing, the rationale and procedures of extensive theory testing have
not been adequately understood. In this paper, we clarify the rationale of extensive
theory testing and propose a framework for the conduct of extensive theory testing.

Keywords: Theory testing, qualitative research method, case study research, post-
positivism.

Introduction
The importance of theory testing in information systems (IS) research cannot be overstated. Whereas
researchers build theory to identify relationships between concepts and why such relationships exist (e.g.
Barratt et al. 2011; Ketokivi and Choi 2014), theory testing corroborates or falsifies the relationships by
evaluating what works, where, for whom, and under what conditions (e.g. Acton et al. 1991; Alexander
1958; Jenson et al. 2016; Miller and Tsang 2011; Rushmer et al. 2014; Ulriksen and Dadalauri 2016).
When relationships are not supported in theory testing, researchers can also learn from the empirical
investigation experience (Ridder 2017)
Theory can be tested using both quantitative (e.g. survey) and qualitative (e.g. case study) approaches.
However, most theory testing studies employ quantitative methods and as a consequence selectively
emphasize quantitative relationships in theory (Bitektine 2008; Ketokivi and Choi 2014). This dogma has
engendered scepticism toward qualitative approaches to theory testing (Burton-Jones and Lee 2017).
Overreliance on quantitative approaches tend to encourage a narrow focus on relationships that can only
be verified with respect to quantity, magnitude, or degree (Bitektine 2008).

Fortieth International Conference on Information Systems, Munich 2019 1

 Extensive Theory Testing Using Case Study

This ‘overreliance’ on quantitative methods for testing IS theories has faced criticisms. Lee and
Baskerville (2003) argued that quantitative approaches to theory testing often emphasize statistical
generalisations but dismiss generalisations across contexts. Similarly, Argyris (1979) and Bitektine (2008)
posited that contextual factors, such as group dynamics and inter-group relations among members within
an organisation, are difficult to ascertain using quantitative methods. Markus and Robey (1988) noted
that testing emergent relationships requires examining dynamic processes and human intensions in
natural settings.
The case study approach has been used for theory testing in the IS discipline (Benbasat et al. 1987; Cavaye
1996; Lee 1989). It offers researchers opportunities to examine the phenomenon of interest in situ (Løkke
and Sørensen 2014; Miller and Tsang 2011). Further, case studies enable researchers to establish causal
claims that otherwise might not be possible through quantitative methods (e.g. Benbasat et al. 1987;
Cavaye 1996; Lee 1989; Paré 2001; Yin 2014) and provide additional insights into theoretical
relationships specific to contexts (Maxwell 2004; Miller and Tsang 2011).
Despite the use of case studies for theory testing, few guidelines have been provided on how to test
theories using case studies (Cavaye 1996; Miller and Tsang 2011; Paré 2001). A lack of explicit
understanding of the theory testing process makes it difficult to recognize and justify its merits and
limitations (Pan and Tan 2011). Further, existing studies that employ case studies for theory testing are
often from a positivist perspective (Bhattacherjee 2012; Paré 2001; Sarker and Lee 2002). Case study
theory testing from a post-positivist perspective is less understood. Positivism assumes the existence of
objective reality and that there can be law-like generalizable theories across contexts (Guba and Lincoln
1994; Maxwell 2004). By contrast, post-positivism assumes theories are influenced by cultural and social
contexts. By adopting a post-positivist perspective, researchers are more attuned to the socio-technical
systems in which humans and information technologies interact (Markus and Robey 1988).
We propose an extensive theory testing approach using case study. The goal of this paper is to clarify the
rationale and procedures of extensive theory testing using case study. Extensive theory testing aims to
both corroborate (or falsify) and extend theory in a single empirical study. Thus, extensive theory testing
entails corroborating or falsifying theoretical relationships (Irwin 2004; Johnston et al. 1999; Miller and
Tsang 2011), identifying new concepts or theoretical relationships (Barratt et al. 2011; Colquitt and
Zapata-Phelan 2007; Tsang 2013; Zhang and Gable 2017), and uncovering potential areas for future
theory development (Crowe et al. 2011; Pereira et al. 2013). By employing extensive theory testing,
researchers can simultaneously develop deeper understandings of an existing theory situated within a
context and generate new insights based on efforts to test the theory.
Extensive theory testing employs a post-positivist lens. Post-positivism offers a researcher the ability to
test theories in their natural context. Ontologically, post-positivism relaxes the rigid objectivity of
positivism and the subjectivity of interpretivism. Further, post-positivism ensures the determination of
causal relationships between the independent concept(s) and dependent concept(s) in a given theory and
enables a researcher to study problems by examining the causes that affect results (Denzin and Lincoln
2011; Guba and Lincoln 1994). Thus, by applying a post-positivist lens on extensive theory testing, we
recognise an objective reality, but such reality cannot be certain given the contextual nature of IS
research. Thus, there are mechanisms responsible for the validity (or lack thereof) of a theory’s
proposition that exist beyond what can be observed from a positivist viewpoint (Miller and Tsang 2011;
Ryan 2016).
By synthesizing the literature (Bitektine 2008; Johnston et al. 1999; Løkke and Sørensen 2014; Yin 2014),
we outline the rationale of extensive theory testing using case study, elucidate its main procedures, and
discuss the criteria for evaluating extensive theory testing research.

Define Qualitative Theory Testing with Case Study

The debate over what qualifies as theory testing research has persisted in both IS research (e.g. Weber
2012) and other related literatures (e.g. Chinn 1986; Colquitt and Zapata-Phelan 2007; Khan 2011).
Earlier efforts to define theory testing have focused on determining whether the constructs, propositions,
boundary conditions and assumptions in a giving theory are logically consistent (Colquitt and Zapata-
Phelan 2007). A set of theoretical statements are logically consistent if the statements can be true at the
same time (Bhattacherjee 2012; Jenson et al. 2016). Generally, theory testing is concerned with

Fortieth International Conference on Information Systems, Munich 2019 2

 Extensive Theory Testing Using Case Study

establishing the validity (or lack thereof) of the relationship between the independent concepts and
dependent concepts in a theory’s proposition within a selected sample or population (Argyris 1979; Dul
and Hak 2007; Sarker and Lee 2002). A theory’s proposition is the causal relational statements between
the independent and dependent concepts in the theory (Dul and Hak 2007). For example, a typical
proposition can be stated in the form: if there is ‘A’ then there must be ‘B’; where A and B are the
independent and dependent concepts respectively.
Qualitative theory testing using cases study involves the process of ascertaining whether empirical
evidence in a case or group of cases supports (or does not support) a theory’s proposition(s) (Dul and Hak
2007). One of the key determinants for the use of case study for theory testing is investigating phenomena
in their natural context and where the researcher has little or no control (Cavaye 1996; Shanks and Parr
2003; Yin 2014). Using case studies for theory testing starts with identifying a set of testable propositions,
and operationalising the identified propositions by translating the abstract concepts into observable
concepts (hypotheses creation) (Bhattacherjee 2012; Johnston et al. 1999). A testable proposition is a
statement that describes the relationship between observable concepts. The existence or absence of the
concepts, and the causes that affect their existence are then compared against empirical data (Dul and
Hak 2007; Johnston et al. 1999).
In qualitative theory testing using case studies, theoretical conditions or outcomes are not only considered
rather, causal mechanisms proposed by the theories are also examined in their different contexts (Miller
and Tsang 2011). Causal mechanisms are pathways through which an outcome is achieved. A causal
mechanism can be a sequence of events or conditions that relate the independent concept(s) to the
dependent concept(s) (Lewis-Beck et al. 2004). Causal mechanisms may differ from one context to
another. The identification of causal mechanisms is central to theory testing using case studies (Goertz
2012; Miller and Tsang 2011) because it provides a stronger argument into the relationship between the
independent and dependent concept. This is also important given the post-positivist lens adopted in this
paper (the theory testing process is discussed later in this paper).
Qualitative theory testing using case study can go beyond just confirming or falsifying a theory. Rather, it
can form the basis for theory expansion or refinement (Voss et al. 2002). Theory refinement is concerned
with the process of updating a domain knowledge in light of new knowledge or cases (Buntine 1991).
During the course of theory testing, qualitative case study research offers a researcher the opportunity to
further investigate a phenomenon of interest in the event that the conditions specified in the existing
theory are not supported by empirical data. This process could result in the development of alternative
theories or explanations as to why the original theory was not supported. For example, consider the work
of Orlikowski (1992). Having started with an existing theory – which assumed that technology has a
deterministic impact on organisational properties (e.g. structure), Orlikowski (1992) went further to
develop three new theoretical models to examine the interaction between technology and organisation.
Qualitative theory testing using case studies can be conducted from both the positivist and post-positivist
perspective. Much theory testing within IS has been dominated by the positivist philosophical paradigm
(Paré 2001; Sarker and Lee 2002; Sarker and Lee 2003). A positivist approach to theory testing using case
study advocates: (a) formulating testable propositions, (b) selecting cases that align with the theoretical
propositions, (c) collecting data based on the selected case(s) and (d) comparing the observed pattern
with available data (Johnston et al. 1999; Sarker and Lee 2002). Thus, the positivist theory testing
approach aligns with the natural science approach where theoretical elements are considered to be
discrete or dichotomous (i.e. having two possible values such as ‘present’ or ‘absent’) (Dul and Hak 2007)
and reality is considered to be absolute.
A positivist view at theory testing using case studies aims only at verifying or falsifying a theory. As an
example, consider the work of Sarker and Lee (2002) who used a single used a single, positivist case study
to test three competing theories of business process redesign (i.e. technocentric, sociocentric and
sociotechnical theories). The result of their research indicates that only one of the theories (the
sociotechnical theory of business process redesign) was supported by empirical data while the
technocentric and sociocentric theories were not supported. Thus, criticisms such as (a) selection bias and
(b) ambiguity of inferred hypothesis (Bitektine 2008, p. 161) are common in such studies. Selection bias
arises from selecting cases which the researcher feels are most likely to confirm or refute the existing
theory; while ambiguity in inferred hypothesis might arise due to the fact that positivism does not
consider contextual factors when validating the relationship between concepts in a theory. Thus, given

Fortieth International Conference on Information Systems, Munich 2019 3

 Extensive Theory Testing Using Case Study

that IS research focuses on studying information technology (IT) within organisations (Benbasat et al.
1987), the neutrality of IT has shown that socio-technical systems can rarely be separated from their social
and technical contexts (Markus and Robey 1988; Orlikowski 1992).
The post-positivist approach to theory testing using case studies emanated as a reaction to the failures of
positivism. As mentioned, positivist researchers seem to conceive theory testing as involving ‘a specified
and near-conclusive procedure for falsification or verification’ (Løkke and Sørensen 2014, p. 68).
However, a post-positivist lens on theory testing using case studies conceives theory testing as a more
inclusive approach – which involves not only identifying causal relationships, but also identifying
circumstances where such relationships are active (i.e. the context of the relationships) (Smith 2010). By
adopting a post-positivist perspective, a researcher acknowledges the fallibility of human judgement and
recognises that theoretical propositions in the IS discipline cannot be separated from their context
(Maxwell 2004; Miller and Tsang 2011; Paré 2001).
Post-positivism associates a level of uncertainty to what can be known about a theory’s proposition
especially in the association between independent and dependent concepts (Guba and Lincoln 1994;
Shanks and Parr 2003). A post-positivist lens on theory testing emphasises that researchers properly
identify and understand the research perspectives and directions. It also ensures that researchers
acknowledge in their data analysis the uncertainty associated with the knowledge of theoretical elements.
Another highpoint of the post-positivist approach is its consideration of human factors in theory testing
using case study. To ascertain causal effects in any explanation for the validity (or lack thereof) of a
theory, human factors have to be accounted for because IS processes and outcomes are likely to be
influenced by the interests or goals of the people who operate the systems.
The post-positivist approach to theory testing using case study can be used to study patterns of behaviours
of individuals at the highest level of social organisation. By studying behavioural patterns, researchers can
be able to identify intensions or unintended consequences of human actions that affects the outcome of a
giving theoretical relationship and can lead to the explanation of causality. As an example of the
interaction between causal relationships and explanations, consider the case of the people-determined
theory (Markus 1983). The people-determined theory predicted that replacing individuals who are
resistant to the financial information system (FIS) or co-opting them with other users who are less
resistant to the system, would reduce resistance to the FIS (Markus 1983). Markus reported that GTC (a
pseudonym) implemented a rotation policy (with the hope of reducing resistance to the FIS) where
accountants are moved from and to resistant divisions. However, Markus further noted that an
accountant – who happens to be one of the system’s designers and an advocate of the FIS, started
resisting the system after he was moved from the corporate accounting division to the divisional
controller’s office. By resisting the system after being moved to another division, the accountant
demonstrated the impact of context and the influence of behavioural patterns to theoretical relationships.

The Place of Case Study Research in Extensive Theory Testing

As mentioned, extensive theory testing is the joint conduct of theory testing and theory extension in one
empirical study. At the point of testing a theory, possible outcomes may include (a) the original theory is
corroborated by empirical data; (b) part of the original theory is falsified – in which case, some
relationships cannot be verified by empirical data and (c) new relationships are identified from empirical
data that cannot be explained using the original theory. Each instance offers the researcher the
opportunity to develop a deeper understanding of the empirical data and provide explanation for the
anomalies identified. Anomalies are misrepresentations of an existing theory. Thus, the identification of
new relationships or the falsification of existing relationships within the bounds of existing theory
indicates the presence of an anomaly.
The presence of an anomaly in the existing theory can be a consequence of the imperfection in human
understanding and the dynamic nature of socio-technical relationships within organizational context.
Because reality is not absolute and independent of human perception, existing theories are susceptible to
misrepresentations. As noted by Popper (2002, p. 50), theories are always changing, perhaps explaining
the difficulty some branches of science face in forming ‘an elaborate and logically well-constructed system
of theories’. Popper further explained that a good quality theory is imperative to sound, informed practice
and the continued maturation of the field. Given the pervasive nature of technology (e.g. digital

Fortieth International Conference on Information Systems, Munich 2019 4

 Extensive Theory Testing Using Case Study

technology), there are bound to be misconceptions of the dynamics of socio-technical relationships within
organizations.
The role of case study in extensive theory testing can be better understood by considering the theory
continuum. The theory continuum involves the different stages of the life cycle of a theory. While there
are continuing efforts to better define the theory continuum (Colquitt and Zapata-Phelan 2007), the
various stages include: theory building, theory development, theory testing and theory
extension/refinements (Ridder 2017; Voss et al. 2002). Within the theory continuum, theory can serve as
both a final product and as a continuum. A theory plays the role of a ‘final product’ when the aim of the
researcher is to identify and describe relationships between independent concepts and dependent
concepts (e.g. in theory building). This process is accomplished by examining empirical data within a
context. As a continuum, the aim of the researcher is not just to identify relationships between concepts,
but to identify failures in the relationships that cannot be explained by the existing theory. Thus, as a
continuum, theory serves as an input to the research process (e.g. in theory testing) – which could
subsequently lead to theory extension.
The implications of extensive theory testing using case study can be explained using the generativity of
digital technology artefacts within the digital innovation literature. Generativity refers to the overall
capacity of a system to produce unprompted changes driven by large, varied and uncoordinated audiences
(Zittrain 2006). By system we mean technical systems (a subset of the socio-technical system). Digital
innovation has become a house-hold concept within information systems as physical artefacts are
augmented with or in some cases replaced with digital components (Lyytinen et al. 2016). For instance,
the creation by Nikon engineers of digital cameras integrated with google maps, exemplifies the
generative properties of physical artefacts. Such cameras in addition to their usage for photographic
purposes, can also be used as navigation systems. While the digital innovation theory has linked
generativity with physical artefacts and with digital innovation, examining the digital properties of
artefacts does not paint a complete picture of the influence of generativity on digital innovation. A system
(e.g. a digital camera) consists of actors who contribute with their creativity and skills, and of suitable
artefacts that help those actors accomplish their goals. Thus, the generative properties that influence
digital innovation cannot only be associated with the physical artefacts, but also with the human actors
that interface with the physical artefacts to bring about their generativity.

The Extensive Theory Testing Process

As noted, guidelines for conducting case study research in the IS discipline are not new (Gerring 2013;
Meyer 2001; Paré 2001; Yin 2011). However, little attention has been paid to specific instances such as the
use of case study for extensive theory testing. Thus, we have included here a set of guidelines that we
believe are appropriate particularly for the conduct of extensive theory testing using case study. The
guidelines provided here are based on previous work with case study research and synthesis of existing
literature on theory testing using case studies within and outside the IS discipline (Bitektine 2008; Cavaye
1996; Johnston et al. 1999; Yin 2014).

Step 1. Establish Theory

Theory testing using the case study approach starts with a theory (Johnston et al. 1999; Yin 2014). In this
study we conceive theory as: systems of concepts and interrelationships among concepts that jointly
explain the constituent elements of a phenomenon and show how and/or why a phenomenon of interest
occurs within a set of boundary assumptions and constraints (Bacharach 1989; Gioia and Pitre 1990).The
theory could result from a comprehensive literature review – during which existing work on the
phenomenon of interest are codified to generate the theory to be tested. The literature review process
enables a researcher to develop a deep appreciation of the phenomena of interest (Johnston et al. 1999).
An example of this type of theory could be seen in the work of Sarker and Lee (2002) who codified the
literature on business process re-engineering (BPR) to ascertain three competing theories-in-use of BPR.
Similarly, a theory generating case study (i.e. ‘theory development’ in the theory continuum) could form
the source of the theory to be tested (Dul and Hak 2007). When case studies are used for theory
development, theories generated from such studies could be tested using single or multiple cases to
ascertain the fitness (or lack thereof) of such theory to empirical data (Johnston et al. 1999). Established
theories (e.g. Orlikowski 1992) could also form the basis of theory testing study using case studies. In this

Fortieth International Conference on Information Systems, Munich 2019 5

 Extensive Theory Testing Using Case Study

scenario, the aim could be to test such theories to ascertain if the theory could be generalized beyond its
immediate boundary.

Step 1.1. Establish Testable Propositions

Testable propositions are statements that can be investigated empirically (Gregor 2006). According to
Goode and Hatt (1952), testable propositions are imaginative ideas, a statement of a solvable problem or
any thought that can be subjected to empirical tests to ascertain its validity. Propositions are defined at
the theory level (i.e. they operate at the realm of the theory) (Dul and Hak 2007). Khan (2011) identified
four characteristics that a testable proposition should have: (a) the concepts should be observable and
measurable; (b) should provide solution to the identified problem in the problem space and be
conceptually comprehensible; (c) should relate to existing body of knowledge; (d) should be logically
comprehensive. In using case studies for theory testing, the testable propositions should be converted into
relational statements – in the form: ‘If not A, then not B’ – where A and B are the antecedents and
outcomes respectively. These relational statements which define the ‘necessary conditions’ and the
expected outcomes are known as the hypothesis (Acton et al. 1991; Dul and Hak 2007). A hypothesis
should indicate the expected outcome in the chosen research problem (Rajasekar et al. 2006). The
hypothesis operates at the realm of the study – which takes into consideration the context and boundary
where the theory operates.

Step 1.2. Identify Causal Mechanisms that Affect Results

This stage of the research process involves the identification of the causal mechanisms which might affect
the hypothesized relationship in the theory under study (Miller and Tsang 2011). Personal actions or
behavioral patterns to phenomena can be key indicators of causal mechanisms. Given the possibility of
having different causes for different hypothesized outcomes, care must be taken at this stage to select
mechanisms that are related to the context in which the hypothesis is considered.

Step 1.3. Operationalize Testable Propositions

One of the properties of qualitative data is that they cannot be used in correlational analysis. Thus, for
hypotheses to be tested using qualitative data, such hypotheses would have to be stated such that they
propose the existence of a condition or an observable characteristic of the object of study (Bitektine
2008). Consider the technocentric theory of business process redesign (Sarker and Lee 2002). Sarker and
Lee proposed that: successful design (and installation) of enabling IT guarantees the effectiveness of
business process redesign (and the effectiveness of the implementation of redesigned business processes)
(Sarker and Lee 2002, p. 9). To operationalize the theoretical proposition, they translated abstract
concepts contained in the proposition (e.g. enabling IT) into more concrete and directly observable
concepts. For instance, enabling IT were taken to mean the installation of computerized BPR tools for
process mapping such as: ‘Process Maker’, ‘Process Charter’, ‘COSMO’ and Pen Analysis Intelligent
Whiteboard (Sarker and Lee 2002, p. 12). Table 1 Summarizes the guidelines for establishing the
theoretical base.

Table 1. Summary of the Guidelines to Establish Theory

Research Stage Research Main Procedures
Purpose
Establish Theory Conceptualize the • Identify and specify the theory to be tested using
theoretical domain qualitative case study.
• Develop a concise and conceptual definition of the
theoretical constructs.
• Establish the theoretical domain (i.e. the boundary within
which the theory operates)
• Develop a set of conceptual propositions.

Fortieth International Conference on Information Systems, Munich 2019 6

 Extensive Theory Testing Using Case Study

Establish Testable • Restate the set of conceptual propositions to testable

Proposition propositions – stating observable characteristics of the
object of study.
Identify causes that • Identify the different conditions under which the
affects results. hypothesized relationships occur. This involves identifying
‘what works, for whom and why’ (Goertz 2012)
Operationalize the • Identify abstract concepts in the testable proposition and
conditions of the replace them with concrete indicators.
propositions

Table 1. Summary of Guidelines to Establish Theory

Step 2. Design Case Study Research

The design of the theory testing case study research involves several considerations which include (a)
identifying the unit of analysis, (b) selecting the appropriate case for study and (c) designing the research
protocol. After the theoretical base has been established, an important decision to be made is identifying
the unit of analysis. The unit of analysis according to Yin (2014) could be a person, an organization, a
group, events, activities, and/or relationships (this list is not exhaustive). The selection of the unit of
analysis is tightly coupled with the research question under consideration. However, given that theory
testing using qualitative case studies seeks to support or refute the relationships between the independent
concept(s) and dependent concept(s) in each theory, the unit of analysis is most likely to be the
relationship stated in the hypothesis. This is important because the expected pattern of observable
phenomena will form the basis of the actual case studies and the data collection decisions that will be
made during the research process.
To select the appropriate case to be studied, several factors would have to be taken into consideration (Yin
2014). A case is selected for study if it belongs to the domain of the theory. The theory domain consists of
possible cases where the conditions of the theory is said to apply (Hak and Dul 2010; Johnston et al.
1999). There are two different designs for selecting the appropriate case to be tested. Each design differs
based on the presence or absence of the dependent or independent concept. Table 2 outlines the different
criteria for selecting a case or group of cases to be studied when testing theories using qualitative case
studies.

Table 2. Case Selection Designs for Extensive Theory Testing.

Criteria Description
Independent Given the following necessary condition hypothesis: A exists only if B exists – where
Variable Design A is the independent construct (expected phenomena or pattern) and B is the
dependent construct (i.e. expected outcome based on the presence or absence of A).
A case is selected in which the outcome (B) is present. An expected pattern is then
formulated about the value of A (Hak and Dul 2010). This expected pattern is then
compared against the observed pattern from data during the analysis phase. The
independent constructs could be human actions, episodes, events or assumptions.
Dependent In the dependent variable design, a case is selected if the independent variable is
Variable Design absent. For instance, if the hypothesis is stated as: B exists only if A exists – where A
and B are independent and dependent constructs respectively, a case is selected if A
is absent. The expected pattern is then formulated about the dependent variable (B).

Table 2. Case Selection Design for Extensive Theory Testing.

Step 3. Prepare for Data Collection and Analysis

The main sources of data for theory testing case study research include interviews (structured and
unstructured), observations, open-ended surveys, informal conversations, archival documents, meeting

Fortieth International Conference on Information Systems, Munich 2019 7

 Extensive Theory Testing Using Case Study

and attendance to events (Voss et al. 2002; Yin 2014). To prepare for data collection, a researcher needs
to first design a case study protocol. The case study protocol is a formal document capturing the entire set
of procedures involved in the data collection and analysis phase of the case study research project (Mills et
al. 2010). A well-designed case study protocol will enhance the validity and reliability of the study (Yin
2014). Similarly, the case study protocol could serve as criteria for evaluating the research process. During
theory testing using case studies, an explicit description of the protocol elements is important. This is to
ensure the rigour of the process, and to give the readers and reviewers of the research a better sense of the
logic that supports the data collection and analysis of the study (Ketokivi and Choi 2014; Mills et al.
2010).
The different elements that should be contained in the case study protocol includes a set of questions that
are expected to be addressed with respect to the stated hypotheses (Johnston et al. 1999). Each question is
expected to inform the researcher on what form of data is needed to address the aims of the study
(Johnston et al. 1999; Voss et al. 2002). When developing the protocol, the researcher should also specify
how the data will be collected and what appropriate techniques are to be used to analyze the data (Yin
2014).

Step 4. Collect Empirical Data

The data to be collected in a theory testing case study is influenced by the theory under consideration and
the nature of the research question(s) (Kuhn 2012; Seale 1997; Yin 2014). Data can be collected using
multiple sources such as interviews, observation, documentation and archival records (Johnston et al.
1999; Keutel et al. 2014; Walsham 2006). The multiple sources of data enable data triangulation and
conforms with the post-positivist lens adopted in this paper. In IS, interviews have been used extensively
as a source of data for both theory testing and theory building studies (Dubé and Paré 2003; Paré and
Elam 1997). For instance, Sarker and Lee (2002) interviewed 17 key stakeholders of the business process
reengineering initiative at MANCO (a pseudonym). Interviews enable the researcher to gather contextual
information relating to different entities associated with the theory under investigation.
Another technique for collecting theory testing data is participant observation (Hirschman 1986).
Participant observation is a technique used to collect qualitative data and involves a researcher ‘personally
emerging into the phenomenon of interest to get a first-hand account’ (Johnston et al. 1999, p. 207). In
participant observation, the researcher is exposed to similar conditions experienced by the researched.
This method enhances the collection of contextual evidence that is related to the phenomenon of interest.
Other methods through which data can be collected include non-participant observation (where the
researcher is an unobtrusive observer of the phenomenon of interest), meetings, archival records and/or
documents. The multiple sources of data ensure that rigour is maintained throughout the research
process.
Further, another consideration to be made during the data collection stage is the data sampling strategy.
Sarker and Lee (2002) suggests the use of purposeful sampling (Patton 2002). Data sampling ensures
that new information is obtained about the phenomenon of interest and to enhance researcher’s
confidence on the observed constructs through constant data triangulation (Sarker and Lee 2002). Table
3 shows a description of the different data sampling techniques that could be employed while collecting
data for extensive theory testing using case studies. Given the overarching goal of extending the theory,
considerations should also be made for identifying new insights within the bounds of existing theory. For
instance, the researcher should look more broadly beyond what is considered in the existing research.

Table 3. Data Sampling Techniques for Extensive Theory Testing

Sampling Description
Technique
Criterion Sampling Criteria sampling involves the reliance on pre-determined sampling criteria.
For instance, during interviews, interviewees could be selected according to
criteria that was stipulated in the case study protocol (Patton 2002; Sarker and
Lee 2002). An example of a pre-determined criterion could be interviewing the
leadership of an organization based on their experience (Sarker and Lee 2002).

Fortieth International Conference on Information Systems, Munich 2019 8

 Extensive Theory Testing Using Case Study

Theory Based or Theory based data sampling involves the selection of data sources (e.g.
Operational interviewees, types of document) to directly test a particular theory of interest
Construct Sampling (Patton 2002). In this type of sampling, the researcher simultaneously collects
data, codes the data and analyses the data in order to decide the next data to be
collected. Although theory-based sampling is mostly used in theory building
studies, it is a perfect fit for extensive theory testing. Theory based sampling
offers the researcher the opportunity to identify anomalies in the existing
theory and then decides what further data needs to be collected to explain such
anomalies. This process leads to theory extension.
Chain Sampling Involves the selection of additional data sources that might have been
identified by prior data sources. For example, an interviewee could identify
another employee as a useful source of information on a subject area of
interest (Patton 2002; Sarker and Lee 2002).
Opportunistic Involves the use of emergent opportunities to collect additional data. For
Sampling example, casual discussion with employees during lunch breaks, and other
recreational periods (Sarker and Lee 1998).

Table 3. Data Sampling Technique for Extensive Theory Testing

Step 5. Analyze Empirical Data

Analyzing qualitative data in a theory testing study involves the identification of useful information that
helps a researcher draw informed conclusions that either support or refute the hypothesized relationship
(Selene Xia and Gong 2014). Yin (2014) notes that the least-developed aspect of case study research is the
procedures for linking data to propositions and the criteria for interpreting research findings (Yin 2014,
pp. 35-36). Data analysis forms an integral aspect of theory testing using case studies (Houghton et al.
2015). Table 4 outlines the various actions to be taking during data analyses.

Table 4. Analyzing Empirical Data Extensive Theory Testing

Research Action Description
Check the Presence During this process, the case researcher is required to consider the presence of
of the Proposed the identified mechanism in the empirical setting. The test for mechanisms
Causal Mechanism involves identifying any implications of the relationships other than those
specified in the theory. This might include events, episodes, assumptions,
predictions or utility to practitioners.
Test Causal Effects In the event that the theorized mechanism(s) are present in the empirical
setting, the researcher can then go ahead to test the causal effects stated in the
hypothesized relationships (Goertz 2012; Miller and Tsang 2011). This could
involve combining antecedents under different conditions.

Table 4. Analyzing Empirical Data in Extensive Theory Testing

Different analytical techniques have been proposed for qualitative data in the literature. In this section,
we will explore some of the techniques that could be used in analyzing qualitative data when testing
theories using case studies.

Step 5.1. Pattern Matching

One of the qualitative analytical techniques for testing theories is pattern matching (Pereira et al. 2013;
Yin 2014). Pattern matching is a technique used to compare expected patterns from theory with observed
patterns from empirical data (Almutairi et al. 2014). The pattern matching technique uses the
confirmation/falsification logic (Almutairi et al. 2014; Johnston et al. 1999; Yin 2014). The
confirmation/falsification logic involves a researcher articulating the theories into testable hypotheses

Fortieth International Conference on Information Systems, Munich 2019 9

 Extensive Theory Testing Using Case Study

which is later compared against empirical data to confirm their fitness (Barratt et al. 2011; Dul and Hak
2007).
When testing theories using qualitative data, an expected pattern is formulated prior to data collection
based on the hypothesis to be tested (Hak and Dul 2010). The formulated pattern could be in the form of
an expected phenomena, an activity or a set of activities that must be carried out for an outcome to occur
(Bitektine 2008). Likewise, in using the pattern matching technique, competing theories are put forward.
The competing theories are intended to predict pattern(s) that are contrary to the dominant theory of
interest (Hyde 2000; Yin 2014). Data is collected from the selected case(s) and compared to predictions
from the dominant theory as well as predictions from the competing theory (Hyde 2000; Yin 2014).
If empirical data matches the predictions of the dominant theory more than it matches that of the
competing theory, support is demonstrated for the dominant theory (Almutairi et al. 2014) otherwise, the
propositions of the dominant theory might be modified (Yin 2014). A theory cannot be completely
rejected or supported based on a single case. However, the confirmation of a theory in a single case
enhances confidence in the validity of the theory. Subsequent confirmations or disconfirmation of the
theoretical propositions could lead to generalizing the theory to a different context or rejecting the theory
based on empirical evidence (Almutairi et al. 2014; Hyde 2000). Table 5 outlines the procedures for
conducting pattern matching.

Table 5. Summary of Procedures for Conducting Pattern Matching (Almutairi et al.

2014; Hyde 2000; Yin 2014).
Analytical Main Research Procedures
Technique
Pattern • Establish and state dominant theoretical proposition(s) prior to data gathering.
Matching • Establish competing theoretical propositions
• Collect empirical data from selected cases
• Identify observable pattern from empirical data
• Compare observed pattern with predicted pattern from the dominant and
competing theories.
• Record the number of matches or lack thereof (hit or miss).
• Draw inference based on the number of matches or lack of matches (i.e.
whether the theory is supported or not supported by empirical data).
Table 5. Summary of Procedures for Conducting Pattern Matching
As an example, consider the work of Sarker and Lee (2002). Sarker and Lee (2002) identified a dominant
theory-in-use (technocentric theory of business process redesign) and two competing theories (the socio-
centric and sociotechnical theories of business process redesign). While testing the dominant
technocentric theory Sarker and Lee (2002)) predicted a pattern of activities that should influence the
role of technology in effective business process redesign. This pattern include (a) business designers must
first identify new technologies and comprehend their properties, and (b) business designers must actively
and creatively find business problems to solve, and opportunities where they can take advantage of their
understanding (Sarker and Lee 2002, p. 9). The presence/absence of these patterns were then observed
during the data collection phase of the research. The observed patterns were subsequently compared with
the predicted pattern from the theory. The technocentric theory was not supported as the predicted
patterns did not match the observed patterns from empirical data (Sarker and Lee 2002).

Step 5.2. Classification and Clustering Analysis.

In theory testing using qualitative data, one of the best ways to learn about your data is by classifying it
based on identifiable concepts or characteristics of individual cases considered during data collection.
Originally developed for use in quantitative analysis, the classification and clustering analysis technique
involves identifying observable concepts (specified in the testable proposition) and mechanisms that
influence the relationship between independent and dependent concepts in a theory. The identified
concepts are then matched with individual cases examined in the theory testing study. Further, the
concepts are transformed into binary data (i.e. 0 or 1) – where zero (0) and one (1) indicate the absence

Fortieth International Conference on Information Systems, Munich 2019 10

 Extensive Theory Testing Using Case Study

and presence of a concept or mechanism in each theoretical relationship respectively. A clustering

analysis is then used to determine the relationship between antecedents and outcomes in the theoretical
relationship based on empirical data.
Before implementing the clustering analysis, a researcher should decide the relevant concepts to be
included in the analysis. This decision should be dependent on the research question. Care should also be
taken to avoid irrelevant concepts that bias towards a certain cluster solution. Irrelevant concepts that can
be avoided include characteristics that are common to the cases being investigated. For instance, if the
cases are individuals, information like gender, job status (e.g. white or blue collar) might need to be
avoided as such information might introduce bias in the clusters. Relevant concepts that might be
included are information about behavioral patterns or individual perception of the phenomenon of
interest.
After the qualitative data is transformed into binary data, a similarity or relational decision is made using
a contingency table. The contingency table helps to compare and identify characteristics or mechanisms
that are common in the existing theory and those observed in empirical data. Table 6 indicates a sample
contingency table. Looking at table 6, ‘a’, ‘b’, ‘c’, and ‘d’ are attributes which are either present or absent in
the existing theory and empirical data respectively.

Table 6. Contingency Table for Testing Relationships in Existing Theory

Empirical Data
Present (1) Absent (0)
Existing Theory Present (1) a b
Absent (0) c d

Table 6. Contingency Table for Testing Relationships in Existing Theory

By comparing the concepts and mechanisms identified in both the existing theory and empirical data, a
researcher might be able to confirm or falsify the validity of a theory. Consequently, it offers the
researcher the ability to identify relationships that might be present in empirical data that cannot be
explained using the existing theory. A full-blown discussion of clustering analysis technique is beyond the
scope of this paper.

Step 5.3. Process Tracing and Causal Process Observations

Process tracing has become one of the fundamental tools of qualitative data analysis especially in ‘within-
case’ study analysis (Collier and Politics 2011). Process tracing is defined as the systematic examination of
evidence selected and analyzed considering research questions and hypotheses posed by the researcher
(Collier and Politics 2011, p. 823). The use of process tracing has been argued to contribute to causal
inference in qualitative theory testing studies through the discovery of causal process observations (CPO)
(George and Bennett 2005). CPO’s are non-comparable observations related to the link between cause
and outcome. CPO’s contribute an insight or piece of data that provides information about context,
process or causes that affect theoretical relationships. Gerring (2007) noted that the aim of process
tracing is to understand the processes linking the different factors (independent concepts) to the expected
outcome (dependent concepts) (Bailey and Jackson 2003; Gerring 2007).

Step 6. Extend the Theory

When the empirical data in a specified theoretical setting fails to explain the identified causes or
relationships in a theory, there is value in extending the theory after testing. In IS, theory extension is not
new. For example, the work of Orlikowski (1992) exemplified how case studies can be used to extend
theories when the hypothesized relationships or mechanisms are not present in the empirical settings.
However, the procedure for extending such theory have not been well articulated. In the following section
we outline the procedure for extending theory using case study.

Fortieth International Conference on Information Systems, Munich 2019 11

 Extensive Theory Testing Using Case Study

Step 6.1. Document the Anomaly in the Empirical Data

The first thing to do when trying to extend existing theory whose hypothesized relationship cannot be
explained by the empirical data is to detect and document the anomaly. This involves identifying things of
interest that the existing theory failed to explain when compared with the empirical data (Burawoy 1998).
We recommend that this process should follow from the data analysis during theory testing. In detecting
the anomaly in the existing theory, the researcher should look at the existing theory through the lens of
new developments within the bounds of the theory. For instance, consider the case of modularity of
physical products. Researchers have argued that modularity influences product innovation (Baldwin and
Clark 2014; Ethiraj et al. 2008; Sanchez and Shibata 2018). However, the digitization of physical products
and the subsequent generativity of digital artefacts have added a new dimension to the theory of
modularity as it relates to product innovation. Innovation of physical products is not only influenced by
their modular architecture but also by the generative properties of the digital artefacts attached to the
physical products (Henfridsson and Bygstad 2013; Lyytinen et al. 2017; Yoo et al. 2012; Zittrain 2006).
Thus, anomalies within the modularity theory could be identified by considering new developments
within digital artefacts.

Step 6.2. Collect Additional Data

The researcher at this point should engage in further dialogue and consultation with stakeholders within
the empirical setting. By using qualitative sources of data collection – such as interviews, the researcher
aims at discovering further knowledge and relationships. Interviews are one of the recommended
approaches for establishing the relationship between the observed outcomes and the antecedents within
the empirical context. Grunig (2002) recommends starting off by asking interview participants high level
questions about their knowledge of the phenomena under investigation without mentioning specific
characteristics of the phenomena. This approach will help the researcher to assess why people assess a
given relationship the way they do (Grunig 2002). The interview data can also be complemented by
documents and observations. If the researcher’s access to new evidence is limited or terminated, the
researcher can draw from the existing knowledge/theory base. This could include revisiting the already
collected data for insights or patterns that might have been overlooked during the theory testing phase.

Step 6.3. Analyze (New) Empirical Data

The additional data collected through interviews, participant observations and in some cases documents,
is then analyzed and triangulated to ensure the validity of the information. The pattern matching and
process tracing methods (discussed earlier) can also be employed to identify the relationship between
antecedents and outcomes. This gives the researcher the opportunity to provide a better understanding
and explanation of the relationship which could not be accounted for in the original theory.

Step 6.4. Reformulate the Theory’s Proposition

Based on the outcome of the data analysis, the theory’s proposition can then be reformulated to reflect the
causes and relationships that better explain the anomalies not considered in the original theory (Burawoy
1998; Ridder 2017). The focus of the researcher at this stage is aimed at further understanding the
phenomena of interest within a context and to extend and codify the theory considering the patterns
observed in the new evidence.

Criteria for Evaluating Extensive Theory Testing Using Case Study

A key indicator of the quality of a research is the measures adopted in evaluating the research. Thus, to
ensure that the extensive theory testing process is of high quality, the researcher needs to employ strong
evaluation criteria. For example. Ketokivi and Choi (2014) noted that case study research needs to clearly
describe underlying premises and analysis that leads researchers to conclusions. Table 7 outlines the
evaluation criteria for the theory testing process.

Fortieth International Conference on Information Systems, Munich 2019 12

 Extensive Theory Testing Using Case Study

Table 7. Evaluation Criteria for Extensive Theory Using Case Study.

Evaluation Criteria Rationale
Define theoretical It is important for the researcher to clearly explicate the theoretical
concepts concepts. Theoretical concepts are the characteristics of the object(s) of
study (Dul and Hak 2007). When using qualitative case study, abstract
theoretical concepts do not lend themselves to empirical measurement
(Acton et al. 1991). Hence, it is important that these concepts are clearly
defined, and appropriate phenomena identified to ‘measure’ such
concepts. Measure as used in this context is to assign a value to a
concept/construct (Burton-Jones and Lee 2017) or to observe the
occurrence of a phenomenon of interest in a selected case.
Ensure that measuring The validity of the concepts and how they are measured is an important
instruments are concept of theory testing research. The researcher must ensure that
theoretically valid evidence of the assignment of values or the formulation of expected
patterns are included in the case study report (Acton et al. 1991).
Ensure the use of To ensure the validity of the process, the researcher should ensure that the
holdout samples dataset used in establishing the hypothesis is not the same dataset used in
testing the relationships between the independent concepts and dependent
concepts (Hyde 2000). This will ensure the integrity of the theory testing
process.

Table 7. Evaluation Criteria for Extensive Theory Testing

Discussion
This discussion section is aimed at reflecting what we believed to be some of the rationale for testing
theories with case studies and the need for extensive theory testing in information systems.

Presence of Multiple Competing Theories-in-Use

Theories-in-Use are a form of knowledge that falls within the realms of everyday people (e.g. managers,
practitioners, business executives, etc.) as opposed to scientific theories (which are mostly empirically
tested theories) that falls within the realm of academics (Sarker and Lee 2002). Theories in use play a
significant role in both research and practice as they sometimes inform policy and decision making within
organizations (Ryan and Rutty 2018). In IS for instance, Sarker and Lee (2002) identified three
competing theories-in-use of business process redesign: the technocentric theory, the sociocentric theory,
and the socio-technical theory of business process redesign. Propositions generated based on these
theories indicated that practitioners and business executives hold competing beliefs about the role of
technological, sociological and socio-technical factors to the implementation and success of business
process redesign (cf. Sarker and Lee 2002). Given that quantitative methods such (e.g. experiment) are
focused on theory testing/development through repeated testing of related hypotheses, a comprehensive
test of competing practitioner theories-in-use could only be explained by exploring multiple aspects of the
theoretical process. Given that these relationships are not always validated quantitatively (Bitektine
2008), qualitative case study design remains the best option for explaining competing theories-in-use.

The Nature of the Phenomenon Under Study

The nature of theories in IS contributes to the choice of a suitable method for evaluating such theory. IS
theories may be described as comprising three essential elements: (1) a set of concepts/constructs
(Offermann et al. 2010); (2) the ‘association’ between concepts and (3) and the boundary within which the
theory apply (Dubin 1978; Neuman 2014; Weber 2003; Weber 2012). Theoretical associations may vary
depending on their representations such as whether they represent simple relationships, sequences of
events and/or causal relationships (Shanks et al. 2012). Similarly, depending on the nature of the
associations, theories can be defined as variance theories, process theories, or systems theories (Burton-

Fortieth International Conference on Information Systems, Munich 2019 13

 Extensive Theory Testing Using Case Study

Jones et al. 2011; Shanks et al. 2012). While ‘variance theories (which are mostly aligned with quantitative
methods) focus on the relationship between the values of attributes of constructs’ (analyzing correlations
between the attribute values) (Shanks et al. 2012, p. 2), process theories focuses on event chains and
understanding the processes linking the relevant factors to the outcomes (Gerring 2007; Shanks et al.
2012; Ulriksen and Dadalauri 2016). Thus, the evaluation of process theories is closely associated with
qualitative case studies (e.g. longitudinal case studies) as it is problematic to try to evaluate such theories
using quantitative approaches (e.g. survey, experiment) (Shanks et al. 2012; Ulriksen and Dadalauri
2016).

Conclusion
The outcome of a theory testing does not often result in a rejection or confirmation of the theory. A
hypothesized relationship or causal effect may fail to be explained by the empirical data contained within
a context. The case study approach at this point offers the researcher an opportunity to further investigate
the phenomena by collecting context-specific information through interviews and other sources of data
collection (observation, documents). This process could then result in a better understanding of the
phenomena under investigation.
Extensive theory testing using case study have not been discussed in the extant IS literature. In this study,
we explored the potential values of extensive theory testing using case studies in the IS discipline. We
argued that a post-positivist approach to extensive theory testing using case studies offers a promising
opportunity to ascertaining the causal relationship and context dependencies in the relationship between
independent and dependent variable in a theory. We also argued that case studies can be used to test
complex socio-technical phenomena especially when the phenomena involved are context dependent and
cannot be readily explained using quantitative relationships. Further, we explored the procedures for
conducting extensive theory testing using case studies in information systems. We provided guidelines on
how to proceed when conducting extensive theory testing. Finally, we clarified the rationale for testing
theories with case studies and presented actionable steps for implementing qualitative case study design
for theory testing. This study will help researchers to better understand and implement extensive theory
testing using case studies within the IS discipline.

References
Acton, G. J., Irvin, B. L., and Hopkins, B. A. 1991. "Theory-Testing Research: Building the Science," ANS.
Advances in nursing science (14:1), pp. 52-61.
Alexander, P. 1958. "Theory-Construction and Theory-Testing," The British Journal for the Philosophy of
Science (9:33), pp. 29-38.
Almutairi, A. F., Gardner, G. E., and McCarthy, A. 2014. "Practical Guidance for the Use of a Pattern‐
Matching Technique in Case‐Study Research: A Case Presentation," Nursing & Health Sciences
(16:2), pp. 239-244.
Argyris, C. 1979. "Using Qualitative Data to Test Theories," Administrative Science Quarterly (24:4), pp.
672-679.
Bacharach, S. B. 1989. "Organizational Theories: Some Criteria for Evaluation," The Academy of
Management Review (14:4), pp. 496-515.
Bailey, D. M., and Jackson, J. M. 2003. "Qualitative Data Analysis: Challenges and Dilemmas Related to
Theory and Method," American Journal of Occupational Therapy (57:1), pp. 57-65.
Baldwin, C. Y., and Clark, K. B. 2014. Design Rules the Power of Modularity. Cambridge: MIT Press.
Barratt, M., Choi, T. Y., and Li, M. 2011. "Qualitative Case Studies in Operations Management: Trends,
Research Outcomes, and Future Research Implications," Journal of Operations Management
(29:4), pp. 329-342.
Benbasat, I., Goldstein, D. K., and Mead, M. 1987. "The Case Research Strategy in Studies of Information
Systems," MIS Quarterly (11:3), pp. 369-386.
Bhattacherjee, A. 2012. "Social Science Research Principles, Methods, and Practices," A. Bhattacherjee
(ed.). Minneapolis: Open Textbook Library.
Bitektine, A. 2008. "Prospective Case Study Design - Qualitative Method for Deductive Theory Testing,"
Organizational Research Methods (11:1), pp. 160-180.

Fortieth International Conference on Information Systems, Munich 2019 14

 Extensive Theory Testing Using Case Study

Buntine, W. 1991. "Theory Refinement on Bayesian Networks," Proceedings of the Seventh conference on
Uncertainty in Artificial Intelligence: Morgan Kaufmann Publishers Inc., pp. 52-60.
Burawoy, M. 1998. "The Extended Case Method," Sociological Theory (16:1), pp. 4-33.
Burton-Jones, A., and Lee, A. S. 2017. "Thinking About Measures and Measurement in Positivist
Research: A Proposal for Refocusing on Fundamentals," Information Systems Research (28:3),
pp. 451-467.
Burton-Jones, A., McLean, E. R., and Monod, E. 2011. "On Approaches to Building Theories: Process,
Variance and Systems," Working Paper, Sauda School of Business, University of British
Colombia, Canada).
Cavaye, A. L. M. 1996. "Case Study Research: A Multi-Faceted Research Approach for Is," Information
Systems Journal (6:3), pp. 227-242.
Chinn, L. P. 1986. "What Does “Theory Testing Research” Mean?," Advances in Nursing Science (9:1), pp.
viii-viii.
Collier, D., and Politics. 2011. "Understanding Process Tracing," PS: Political Science (44:4), pp. 823-830.
Colquitt, J. A., and Zapata-Phelan, C. P. 2007. "Trends in Theory Building and Theory Testing: A Five-
Decade Study of the Academy of Management Journal," Academy of Management Journal
(50:6), pp. 1281-1303.
Crowe, S., Cresswell, K., Robertson, A., Huby, G., Avery, A., and Sheikh, A. 2011. "The Case Study
Approach," BMC Medical Research Methodology (11:1), pp. 100-100.
Denzin, N. K., and Lincoln, Y. S. 2011. The Sage Handbook of Qualitative Research, (4th ed.). Thousand
Oaks: Sage.
Dubé, L., and Paré, G. 2003. "Rigor in Information Systems Positivist Case Research: Current Practices,
Trends, and Recommendations," MIS Quarterly (27:4), pp. 597-636.
Dubin, R. 1978. Theory Building, (Revised ed.). New York: Free Press.
Dul, J., and Hak, T. 2007. Case Study Methodology in Business Research. Routledge.
Ethiraj, S. K., Levinthal, D., and Roy, R. R. 2008. "The Dual Role of Modularity: Innovation and
Imitation," Management Science (54:5), pp. 939-955.
George, A. L., and Bennett, A. 2005. Case Studies and Theory Development in the Social Sciences. MIT
Press.
Gerring, J. 2007. Case Study Research: Principles and Practices. New York: Cambridge University Press.
Gerring, J. 2013. "The Case Study: What It Is and What It Does," in The Oxford Handbook of
Comparative Politics. Oxford University Press.
Gioia, D. A., and Pitre, E. 1990. "Multiparadigm Perspectives on Theory Building," Academy of
management review (15:4), pp. 584-602.
Goertz, G. 2012. "Case Studies, Causal Mechanisms, and Selecting Cases," Unpublished manuscript,
Version (5).
Goode, W., and Hatt, P. 1952. Methods in Social Science. New York: Mc-Gray Hill.
Gregor, S. 2006. "The Nature of Theory in Information Systems," MIS Quarterly (30:3), p. 611.
Grunig, J. E. 2002. Qualitative Methods for Assessing Relationships between Organizations and Publics.
Gainesville, FL: The Institute for Public Relations, Commission on Public Relations Measurement
and Evaluation.
Guba, E. G., and Lincoln, Y. S. 1994. "Competing Paradigms in Qualitative Research," in Handbook of
Qualitative Research. Thousand Oaks, CA, US: Sage Publications, Inc, pp. 105-117.
Hak, T., and Dul, J. 2010. "Theory-Testing with Cases," Encyclopedia of case study research), pp. 938-
943.
Henfridsson, O., and Bygstad, B. 2013. "The Generative Mechanisms of Digital Infrastructure Evolution,"
MIS Quarterly (37:3), p. 907.
Hirschman, E. C. J. J. o. m. R. 1986. "Humanistic Inquiry in Marketing Research: Philosophy, Method,
and Criteria," (23:3), pp. 237-249.
Houghton, C., Murphy, K., Shaw, D., and Casey, D. 2015. "Qualitative Case Study Data Analysis: An
Example from Practice," Nurse Researcher (22:5), pp. 8-12.
Hyde, K. F. 2000. "Recognising Deductive Processes in Qualitative Research," Qualitative Market
Research: An International Journal (3:2), pp. 82-90.
Irwin, T. J. B. 2004. "Testing and Extending Theory in Strategic Information Systems Planning through
Literature Analysis," Information Resources Management Journal (IRMJ) (17:4), pp. 20-48.

Fortieth International Conference on Information Systems, Munich 2019 15

 Extensive Theory Testing Using Case Study

Jenson, I., Leith, P., Doyle, R., West, J., and Miles, M. P. 2016. "Testing Innovation Systems Theory Using
Qualitative Comparative Analysis," Journal of Business Research (69:4), pp. 1283-1287.
Johnston, W. J., Leach, M. P., and Liu, A. H. 1999. "Theory Testing Using Case Studies in Business-to-
Business Research," Industrial Marketing Management (28:3), pp. 201-213.
Ketokivi, M., and Choi, T. 2014. "Renaissance of Case Research as a Scientific Method," Journal of
Operations Management (32:5), pp. 232-240.
Keutel, M., Michalik, B., and Richter, J. 2014. "Towards Mindful Case Study Research in Is: A Critical
Analysis of the Past Ten Years," European Journal of Information Systems (23:3), pp. 256-272.
Khan, J. A. 2011. Research Methodology. New Delhi: APH Publishing Corporation.
Kuhn, T. S. 2012. The Structure of Scientific Revolutions, (4th ed.). Chicago, IL: University of Chicago
Pres.
Lee, A. S. 1989. "A Scientific Methodology for Mis Case Studies," MIS Quarterly (13:1), pp. 33-50.
Lee, A. S., and Baskerville, R. L. 2003. "Generalizing Generalizability in Information Systems Research,"
Information Systems Research (14:3), pp. 221-243.
Lewis-Beck, M. S., Bryman, A., and Liao, T. F. 2004. The Sage Encyclopedia of Social Science Research
Methods. Thousand Oaks, Calif: Sage.
Løkke, A.-K., and Sørensen, P. D. 2014. "Theory Testing Using Case Studies," Electronic Journal of
Business Research Methods (12:1), pp. 66-74.
Lyytinen, K., Sørensen, C., and Tilson, D. 2017. "Generativity in Digital Infrastructures: A Research Note,"
in The Routledge Companion to Management Information Systems. Taylor and Francis, pp. 253-
275.
Lyytinen, K., Yoo, Y., and Boland Jr, R. J. 2016. "Digital Product Innovation within Four Classes of
Innovation Networks," Information Systems Journal (26:1), pp. 47-75.
Markus, M. 1983. "Power, Politics, and Mis Implementation," Communicartions of the ACM (26:6), pp.
430-444.
Markus, M. L., and Robey, D. 1988. "Information Technology and Organizational Change: Causal
Structure in Theory and Research," Management Science (34:5), pp. 583-598.
Maxwell, J. A. 2004. "Using Qualitative Methods for Causal Explanation," Field Methods (16:3), pp. 243-
264.
Meyer, C. B. 2001. "A Case in Case Study Methodology," Field Methods (13:4), pp. 329-352.
Miller, K. D., and Tsang, E. W. K. 2011. "Testing Management Theories: Critical Realist Philosophy and
Research Methods," Strategic Management Journal (32:2), pp. 139-158.
Mills, A., Durepos, G., and Wiebe, E. 2010. Encyclopedia of Case Study Research. Thousand Oaks,
California: SAGE Publications.
Neuman, L. W. 2014. Social Research Methods: Qualitative and Quantitative Approaches, (7th ed.). New
Delhi: Pearson Education.
Offermann, P., Blom, S., Schönherr, M., and Bub, U. 2010. "Artifact Types in Information Systems Design
Science – a Literature Review," in Global Perspectives on Design Science Research: 5th
International Conference, Desrist 2010, St. Gallen, Switzerland, June 4-5, 2010. Proceedings., R.
Winter, J.L. Zhao and S. Aier (eds.). Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 77-92.
Orlikowski, W. J. 1992. "The Duality of Technology: Rethinking the Concept of Technology in
Organizations," (3:3), pp. 398-427.
Pan, S. L., and Tan, B. 2011. "Demystifying Case Research: A Structured–Pragmatic–Situational (Sps)
Approach to Conducting Case Studies," Information and Organization (21:3), pp. 161-176.
Paré, G. 2001. "Using a Positivist Case Study Methodology to Build and Test Theories in Information
Systems: Illustrations from Four Exemplary Studies,").
Paré, G., and Elam, J. J. 1997. "Using Case Study Research to Build Theories of It Implementation," in
Information Systems and Qualitative Research. Springer, pp. 542-568.
Patton, M. Q. 2002. Qualitative Research and Evaluation Methods, (3rd ed.). Thousand Oaks, Calif:
Sage.
Pereira, R., Almeida, R., and da Silva, M. M. 2013. "How to Generalize an Information Technology Case
Study," Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 150-164.
Popper, K. R. 2002. The Logic of Scientific Discovery. London: Routledge Classics.
Rajasekar, S., Philominathan, P., and Chinnathambi, V. 2006. "Research Methodology," available at:
https://arxiv.org/abs/physics/0601009 (accessed 01 September 2017).

Fortieth International Conference on Information Systems, Munich 2019 16

 Extensive Theory Testing Using Case Study

Ridder, H.-G. 2017. "The Theory Contribution of Case Study Research Designs," Business Research
(10:2), pp. 281-305.
Rushmer, R. K., Hunter, D. J., and Steven, A. 2014. "Using Interactive Workshops to Prompt Knowledge
Exchange: A Realist Evaluation of a Knowledge to Action Initiative," Public Health (128:6), pp.
552-560.
Ryan, G. S. 2016. "Enhancing Nursing Student Success: A Critical Realist Framework of Modifiable
Factors," Archives of Nursing Practice Care (2:1), pp. 57-70.
Ryan, G. S., and Rutty, J. 2018. "Post-Positivist, Critical Realism: Philosophy, Methodology and Method
for Nursing Research," Nurse Researcher), p. (In Press).
Sanchez, R., and Shibata, T. 2018. "Modularity Design Rules for Architecture Development: Theory,
Implementation, and Evidence from Development of the Renault-Nissan Alliance "Common
Module Family" Architecture." St. Louis: Federal Reserve Bank of St Louis.
Sarker, S., and Lee, A. S. 1998. "Using a Positivist Case Research Methodology to Test a Theory About It-
Enabled Business Process Redesign," Proceedings of the international conference on
Information systems, Helsinki, Finland: Association for Information Systems, pp. 237-252.
Sarker, S., and Lee, A. S. 2002. "Using a Positivist Case Research Methodology to Test Three Competing
Theories-in-Use of Business Process Redesign," Journal of the Association for Information
Systems (2:1), p. 7.
Sarker, S., and Lee, A. S. 2003. "Using a Case Study to Test the Role of Three Key Social Enablers in Erp
Implementation," Information & Management (40:8), pp. 813-829.
Seale, C. 1997. "Ensuring Rigour in Qualitative Research," European Journal of Public Health (7:4), pp.
379-384.
Selene Xia, B., and Gong, P. 2014. "Review of Business Intelligence through Data Analysis,"
Benchmarking: An International Journal (21:2), pp. 300-311.
Shanks, G., Bekmamedova, N., and Johnston, R. 2012. "Exploring Process Theory in Information Systems
Research," Proceedings of the Information Systems Foundations Workshop, ANU, Canberra,
ANU, Canberra.
Shanks, G. G., and Parr, A. N. 2003. "Positivist, Single Case Study Research in Information Systems: A
Critical Analysis," 11th European Conference on Information Systems (ECIS, 2003), Naples,
Italy, pp. 16-21.
Smith, M. 2010. "Testable Theory Development for Small-N Studies: Critical Realism and Middle-Range
Theory," International Journal of Information Technologies and Systems Approach (IJITSA)
(3:1), pp. 41-56.
Tsang, E. W. K. 2013. "Case Study Methodology: Causal Explanation, Contextualization, and Theorizing,"
Journal of International Management (19:2), pp. 195-202.
Ulriksen, M. S., and Dadalauri, N. 2016. "Single Case Studies and Theory-Testing: The Knots and Dots of
the Process-Tracing Method," International Journal of Social Research Methodology (19:2), pp.
223-239.
Voss, C., Tsikriktsis, N., and Frohlich, M. 2002. "Case Research in Operations Management,"
International Journal of Operations & Production Management (22:2), pp. 195-219.
Walsham, G. 2006. "Doing Interpretive Research," European journal of information systems (15:3), pp.
320-330.
Weber, R. 2003. "Theoretically Speaking1," MIS Quarterly (27:3), pp. III-XII.
Weber, R. 2012. "Evaluating and Developing Theories in the Information Systems Discipline," Journal of
the Association for Information Systems (13:1), pp. 1-30.
Yin, R. K. 2011. "Case Study Research: Design and Methods," Modern Language Journal (95), p. 474.
Yin, R. K. 2014. Case Study Research: Design and Methods, (Fifth ed.). Los Angeles: Sage.
Yoo, Y., Boland, R., Lyytinen, K., and Majchrzak, A. 2012. "Organizing for Innovation in the Digitized
World," Organization Science (23:5), pp. 1398-1408.
Zhang, M., and Gable, G. G. 2017. "A Systematic Framework for Multilevel Theorizing in Information
Systems Research," Information Systems Research (28:2), pp. 203-224.
Zittrain, J. L. 2006. "The Generative Internet," Harvard Law Review (119:7), pp. 1974-2040.

Fortieth International Conference on Information Systems, Munich 2019 17