Steps in Conducting a Scholarly

Mixed Methods Study

John W. Creswell
University of Nebraska - Lincoln,
[email protected]
11-14-2013
 University of Nebraska - Lincoln
DigitalCommons@University of Nebraska - Lincoln

Discipline-Based Education Research Group

Follow this and additional works at: https://digitalcommons.unl.edu/dberspeakers

Creswell, John W., "Steps in Conducting a Scholarly Mixed Methods Study" (2013). DBER Speaker
Series. 48.
https://digitalcommons.unl.edu/dberspeakers/48

This Presentation is brought to you for free and open access by the Discipline-Based Education Research
Group at DigitalCommons@University of Nebraska - Lincoln. It has been accepted for inclusion in
DBER Speaker Series by an authorized administrator of DigitalCommons@University of Nebraska -
Lincoln.
Presenter(s), Department(s):
John Creswell Professor
Department of Educational
Psychology University of Nebraska‐Lincoln
Title:
Steps in Conducting a Scholarly Mixed Methods Study

Abstract:
Mixed methods research is a rapidly expanding methodology in the social and human sciences in the US and around the
world. In this presentation I will first define mixed methods research (combining both quantitative and qualitative methods
of research) and discuss what it is and what it is not. Then I will review a brief history of its development, and why it is
important today. I will discuss several of the scientific developments in mixed methods that have occurred over the last ten
years, such as the specification of types of designs, the formation of mixed methods questions, and the use of innovative
approaches to jointly display quantitative and qualitative results. Finally, I will talk about the future of this methodology ‐
where it is headed and some important worldwide developments that have encouraged mixed methods research.
Steps in Conducting a Scholarly Mixed
Methods Study

John W. Creswell
University of Nebraska‐Lincoln
 How We Make Sense

Personal
Numbers Experiences

A Mixed
Methods
Approach
 A Simple Idea

Let’s use quantitative and qualitative data (or quantitative research and
quantitative research) together to gain a more complete understanding of our
research questions.
Mixed methods is a research approach, popular in the social, behavioral, and
health sciences, in which researchers collect, analyze, and integrate both
quantitative and qualitative data in a single study or in a sustained long‐term
program of inquiry to address their research questions.
 POPULARITY ‐ FEDERAL
FUNDING 5

New NIH funding awards that self-identify as mixed

methods (N = 226) (1997-2008)
70

60

Number of New Awards

50

40

30

20

10

0
1996 1998 2000 2002 2004 2006 2008 2010
Year

Source: CRISP database; Plano Clark (2010)

DISSERTATIONS

Haines, C., 2011 unpublished dissertation, University of Nebraska‐

Lincoln
Mixed Methods Social Science
Articles
Mixed Methods Health Science Articles
Select Mixed Methods Books
 When did mixed methods develop in
the social sciences and health/behavioral sciences?

(1985-90)
Jennifer Greene – USA- University of Illinois – evaluator

John Hunter and Allen Brewer – USA - Northwestern and Boston

College – sociologists

Alan Bryman - England – management John Creswell –

USA – education

Nigel and Jane Fielding – sociologist in UK Jan Morse – nursing in

Canada
If you walked into my office, with the idea of conducting a
mixed methods study, I would take you through a series of
questions to help you design your project.

I will take the steps out of order of what you typically see in a
research project, but we will put them in order at the end.
 Question1:-
Do You Have A Mixed Methods
Study? (or Simply A
Quantitative Study Or A
Qualitative Study)
What I am looking for (core characteristics
of a mixed methods study):

1. Do you have a quantitative database?

(closed‐ ended)
2. Do you have a qualitative database?
(open‐ ended)
3. Do you plan on bringing the two
databases together? (integration)
4. What procedures (design) will you use?
What I am NOT looking for:
• Keeping the databases separate
• The collection of BOTH quantitative
and qualitative data (not content analysis)
• Multiple forms of quantitative data or multiple
forms of qualitative data (multi‐method)
• Only the transformation of qualitative
data into quantitative (in analysis)
• Only formative/summative evaluation
• Only adding in qualitative data into an
experiment
• Doing mixed methods because it is trendy
Question 2. Does your
research problem or question
merit a mixed methods study?

They say, choose your method based on your

problem.
When is mixed methods suitable for
a problem or question?

• When qualitative research or

quantitative research is insufficient
to fully understand the problem
 Other types of problems, when:

• We need to explore before we administer

instruments
• We need to explain our statistical results
by talking to people
• We need to see if our quantitative results
and our qualitative results match
• We need to enhance our experiments by
talking with people
• We need to develop new instruments by
gathering qualitative data
Question 3: What is the overall intent of your
study?
My goal:

• To learn whether you are

quantitatively‐ or qualitatively‐
oriented by the way you state
your intent
Key wor ds

Quantitative Qualitative
factors explore
determinants generate
relationship develop
causes create
influences meaning
effects personal experiences
 Your Keywords:

1. Will help me think about what

mixed methods design you will likely
use
2. Will help me think about your
discipline field background that you
bring to mixed methods
3. Will help me think about what skills you
have to begin conducting a mixed methods
study
• “This article reports on the study conducted to understand
students’ persistence in the Distributed Doctoral Program in
Educational Leadership in Higher Education (ELHE) offered by the
University of Nebraska‐Lincoln (UNL).” (Ivankova & Stick, 2007)

• “We conducted an interview‐based study of terminally ill

individuals who were receiving palliative care.” (Kutner, Steiner,
Corbett, et al., 1999)

• “…to understand patients’ experiences of transitions from

hospital to a homeless shelter and determine aspects of these
experiences associated with perceived quality of these
transitions.” (Greysen, Allen, Lucas, et al., 2012)
Question 4: What quantitative
and qualitative data will you
collect to address this intent?
I will ask you to list your quantitative and
qualitative data sources:

Quantitative Data Qualitative Data

Site and sample?

How many?
Types of data to
collect
Specific
instruments/protoc
ols

Types of
scales/questions
I will ask you to list your quantitative and
qualitative data analysis steps:

Quantitative Qualitative
Analysis Analysis

Getting data ready

for analysis

Preliminary analysis

Steps in analysis
(including statistics,
themes)
 Qualitative Data to be Collected and Analyzed

QUANTITATIVE RESEARCH QUALITATIVE RESEARCH

• Quantitative • Qualitative data collection
data (open‐ended)
collection(closed‐ – Interviews
ended) – Observations
– Instruments – Documents
– Behavioral – Audio‐visual materials
checklists
– Records
• Quantitative data • Qualitative data analysis
analysis – Use text and image data,
– Use numeric • For coding
data,• For description • For theme development
• For comparing groups • For relating themes
• For relating variables

27
 Are These Procedures Realistic and Feasible?

• Time
• Economically
• Skills
• Useful for stakeholders
Question 5. Why are you
collecting both quantitative
and qualitative data?
 Basic reasons for using mixed methods
(called the rationale in mixed methods):
• Need different, multiple perspectives, or more
complete understandings
• Need to confirm our quantitative measures with
qualitative experiences
• Need to explain quantitative results
• Need better contextualized instruments, measures, or
interventions to reach certain populations
• Need to enhance our experiments
• Need to gather trend data and individual perspectives
from community members
• Need to evaluate the success of a program by using a
needs assessment AND a test of the success of the program
Questions 6. Do you plan on
using a theory? Do you plan
on advancing your philosophical
stance?
 Frame the study within theory/philosophy:

Paradigm/Worldview (epistemology, ontology, axiology,

methodology)

Theoretical lens
(e.g., feminist, racial, social science theories)

Methodological approach (e.g., ethnography,

experiment, mixed methods)

Methods of data collection (e.g., interviews,

checklists, instruments)
32

Adapted from Crotty M.

(1998)
 Further questions about philosophy:

• What is a philosophy?

• Which philosophies are typically

used as the foundation in
mixed methods research?
 Philosophies to consider in mixed methods:

• Pragmatism
• Transformative worldview
• Critical realism
• Dialectic pluralism

• Multiple worldviews/paradigms
• Match worldview to design
• Worldview comes from your scholarly
community
 Further questions about theory:

• What is a theory?

• How do you find a theory?

• How will you use it in your mixed methods

study?
Question 7. What mixed
methods design will you use?
Parsimonious set of designs: Basic and Advanced

Basic Designs
Convergent Design
Explanatory Sequential Design Advanced Designs
Exploratory Sequential Design Basic
Designs
Advanced Designs
Intervention Design
Transformative Design
Multiphase Design
 Convergent Parallel Design

Quantitative Data Quantitative

Collection and Results
Analysis
Merge
Results Interpret or Explain
For Convergence/Divergence
Qualitative Data Qualitative Compar
Collection and Results ison
Analysis

Explanatory Sequential Design

Quantitative Data Determine Qualitative Data Interpret How

Quantitative Quantitative Qualitative Qualitative Data
Collection and Collection and
Results Results to Results Explains
Analysis Analysis Quantitative Results
Explain

Exploratory Sequential Design

Quantitative Data
Use Results to Collection and Interpret How
Qualitative Data Qualitative Form Variables, Analysis Based on Quantitative Quantitative Results
Collection and Results Instruments, Variables, Results Provide New Results,
New, Better Instruments,
Analysis Interventions Instruments, and Better Interventions
Interventions
 Intervention Mixed Methods Design
Experimental Study

Qualitative
Data Collection, Experiment Group Pre‐test Intervention Post‐Test Qualitative
Analysis, and Data Collection,
Results Control Group Pre‐test Post‐Test Analysis, and Results
(Exploratory) (Explanatory)
Qualitative Data Collection,
Analysis, and Results (Convergent)

Social Justice Design (using an Explanatory Sequential Design example)

Quantitative Qualitative Data Interpret How

Theory Research Quantitative Qualitative Qualitative Data
Data Collection Collection and Results
Questions Results Analysis
Explains
e.g., survey Quantitative Results
And Calls for Action
Promote Social Justice

Multistage Evaluation Design (using an Exploratory Sequential Design example)

Single Program Objective
Summative
Formative Program
Formative Needs Theory/ Instrument Program
Program Revision
Assessment Conceptual Framework Development Evaluation
Assessment
(qualitative (based on qualitative (based on (pre‐post
(qualitative
data collection) results) quantitative tests) quantitative
data collection)
tests)
 How do you choose your design?

• First select your basic design, then

add if you have an advanced design
• Consider your scholarly community:
quantitative or qualitative? (how to begin
your study)
• Consider your skills: quantitatively‐
strong or qualitatively‐strong (what will be
emphasized)
• Consider your resources (collect at one time
or space out)
• Consider the complexity of the design (what
is the easiest? the hardest?)
refugees in Boston

Community‐Based Participatory Research Approach

Intervention Design

Exploratory
Sequential Design
Question 8. Can you draw
a diagram of your design?
Source: based on Wittink et al.(2006)
 the management of anti‐psychotic medication)

Flow of the experiment

Experimental Methodology
qual qual Overall
before QUAN QUAN after results
intervention Pre- Intervention Post- & inter and
Follow-up Follow-up vent interpr
Measure Measures ion etation

Procedures: Procedures:
-on- •Three groups: control group, compliance • One-on-one semi- Discuss treatment
one semi-structured Intervention group, alliance intervention structured interviews – effectiveness
interviews group – group comparisons exiting the trial, •Discuss themes
• Thematic analysis •Outcome measures: 1) attitudes toward participants from two in context of
medication 2) adherence to treatment experimental conditions; interventions
3) avoidance of relapse • Thematic analysis and outcomes
•DAI measure completed 3 times (pre,
post, & follow up) Products: Products:
• Transcripts Discuss
Products: Products: • Themes and quotes
• Transcripts • Numerical item scores
• Developed intervention • Change scores
treatment • Test statistics
Question 9. Can you write a good
purpose statement (or study aim)
that fits your design?
 How to write a mixed methods study aim:

• This mixed methods study will address [overall content

aim]. A convergent parallel mixed methods design will Intent
be used, and it is a type of design in which qualitative
and quantitative data are collected in parallel, analyzed Design
separately, and then merged. In this study,
[quantitative data] will be used to test the theory of
[the theory] that predicts that [independent variables]
will [positively, negatively] influence the [dependent
variables] for [participants] at [the site]. The [type of Data
qualitative data] will explore [the central phenomenon] Collection
for [participants] at [the site]. The reason for collecting
both quantitative and qualitative data is to [the mixing Rationale
reason].
Question 10. Can you write
research questions for your
mixed methods study?
Write 3 types of questions:

• Quantitative questions or
hypotheses

• Qualitative questions

• A mixed methods
question
 Basic design mixed methods questions:

Convergent Design – To what extent do the

quantitative and qualitative
results converge?
Explanatory Design – In what ways do the
qualitative data help to
explain the quantitative
results?
Exploratory Design – In what ways do the
quantitative results generalize
the qualitative findings?
 Advanced design mixed methods questions:

• Intervention Design – How do the qualitative findings

provide an enhanced understanding
of the quantitative results?

– How do the qualitative findings

provide an enhanced understanding
• Transformative Social Justice of the quantitative results in order
Design to explore inequalities?

– How to the different phases in

the project to address the overall
• Multistage Design research goal?
for a good mixed methods project:
• Problem
• Theory and/or philosophy
• Purpose (study aim)
• Rationale for gathering both
quantitative and qualitative data
• Research questions (quantitative,
qualitative, mixed)
• Types of data/analysis
• Definition of mixed methods
• Type of mixed methods design
• Diagram of the design
Steps in Conducting a Scholarly Mixed Methods
Study

John W. Creswell
University of Nebraska‐Lincoln