Today's Plan

Touching base on the past few weeks (and turn in your reflection paper!) Share your diagram of your study or some other details related to your data collection procedures Analyzing and interpreting data within a mixed methods study Preparing for the rest of the semester
Creswell & Plano Clark (2007) 1

Sharing your data collection plans

What should you look for when reviewing someone's procedural diagram?

Mixed Methods Research

EDPS 936 Chapter 7 -- Analyzing and Interpreting Data in Mixed Methods Research The Basics of Quantitative and Qualitative Data Analysis and Interpretation

Researchers analyze data to address their research questions

!

Preparing your data for analysis

!

General quantitative and qualitative data analysis procedures include: ! Preparing the data for analysis ! Exploring the data ! Analyzing the data ! Representing the analysis ! Interpreting the results ! Validating the data and findings "We are mixing apples and oranges and it can be done, a fruit salad does taste good, but you have to be careful that the fruits aren't overripe or under ripe when you make the fruit salad." (Plano Clark, 2005, p. 149)
Creswell & Plano Clark (2007) 5

Quantitative
!

Qualitative
!

! ! !

Assign numeric value to each response in a database (e.g., SPSS, SAS) Clean the database Create special variables Develop codebook

Organize and transcribe the data Check transcripts for accuracy Enter data into qualitative data analysis software (e.g., NVivo, MAXQDA, Atlas.ti) Or, if coding by hand, print transcripts with wide margins

Creswell & Plano Clark (2007)

Exploring your data

!

Analyzing your data

!

Quantitative
!

Qualitative
!

Quantitative
!

! !

Conduct descriptive analyses Inspect trends in the data Plot data and check if data are normally distributed

! !

Read through the data to obtain a sense of it all Memo initial thoughts Develop codebook (if appropriate)

Analyze data to answer quantitative research questions Select inferential statistical tests based on: ! research questions, ! scale type, ! number of variables, and ! distribution Consult with the NEAR Center

Qualitative
!

! ! !

Analyze data to answer qualitative research questions Code the data Develop description and themes from codes Interrelate themes Consult with the NEAR Center
8

Creswell & Plano Clark (2007)

Creswell & Plano Clark (2007)

Representing your data analysis

!

Validating your data and results

!

Quantitative
!

Qualitative
!

Quantitative
! !

Qualitative
!

! !

Summarize statistical results Tables Figures

Discuss evidence for the themes/description Interrelate the themes in a model (figure) or with tables (e.g., compare groups)

Reliability - extent to which the scores are consistent and stable Validity - extent to which one can draw meaningful inferences Discuss validity, reliability of the instruments you use in the methods section (from past uses) and in your results (from current study) Reduce threats to internal validity (extent to which cause-and-effect claims can be made) and external validity (extent to which the results can be generalized to other persons, settings, or times)

Reliability - minor role, perhaps the extent to which multiple coders agree on codes Validity - extent that the information is accurate and credible Try to use minimum of 3 qualitative validation techniques, such as:
! ! ! ! ! !

member checking, triangulating data from several types of data, triangulating data from several individuals, spending extended time in field, reporting disconfirming evidence, or having an external review of the data and procedures
10

Creswell & Plano Clark (2007)

Creswell & Plano Clark (2007)

Interpreting your results

!

Quantitative
!

Qualitative
!

Compare results to research questions and hypotheses Compare results with prior predictions or explanations drawn from the literature

Discuss how the findings answer the research questions Compare findings to past literature Bring in the researcher's personal experiences and personal assessment of the meaning of the findings
11

Data Analysis and Interpretation within Mixed Methods Designs

Creswell & Plano Clark (2007)

Analyzing data within mixed methods designs

!

Initial discussions in the literature focused on generic approaches

!

Use mixed methods analysis to answer the mixed methods research question Data analysis can occur at a single point in the study or at multiple points After analyses complete, assess how the information addresses the mixed methods question (mixed methods interpretation)

For example, Caracelli & Graham (1993):

! ! ! !

Data transformation Typology development Extreme case analysis Data consolidation or merging

We now see analytic procedures depending on the type of design

Creswell & Plano Clark (2007)

13

Creswell & Plano Clark (2007)

14

Convergent/Concurrent Triangulation Design

!

Explanatory Design
!

Basic steps include:

!

Basic steps include:

! !

! !

Collecting both quantitative and qualitative data concurrently Analyzing the two data sets separately and independently Merging the two databases by: ! comparing the results from the two data sets or ! transforming one type of result into the other type of data and conducting further analyses Comparing the merged results with the research questions

Collecting and analyzing quantitative data Using the results to inform the follow-up qualitative data collection by shaping:
" " "

The research questions Participant selection Data collection protocols

! !

Collecting and analyzing qualitative data Interpreting how the qualitative analysis helps to explain the initial quantitative results (i.e., addressing the mixed methods question)

Creswell & Plano Clark (2007)

15

Creswell & Plano Clark (2007)

16

Exploratory Design
!

Embedded Design
! !

Basic steps include:

! !

Collecting and analyzing qualitative data Using the results to inform the follow-up quantitative data collection by shaping:
" " "

Quantitative and qualitative strands may be implemented concurrently and/or sequentially Data analysis depends on how the embedded data is used Basic steps include:
! ! ! !

The research questions The identification of the variables of interest Data collection instruments

! !

Collecting and analyzing quantitative data Interpreting how the quantitative analysis helps to generalize or extend the initial exploratory qualitative findings (i.e., addressing the mixed methods question)
Creswell & Plano Clark (2007) 17

Analysis of the primary data Analysis of the secondary data Further mixed methods analyses Interpretation of how and in what ways the secondary data and results support or augment the primary design

Creswell & Plano Clark (2007)

18

Mixed Methods Data Analysis Falls within Two Broad Categories

!

Merged Data Analysis for Concurrent Approaches

!

Concurrent approaches (e.g., triangulation, some embedded and transformative designs) call for: ! merged data analysis Sequential approaches (e.g., explanatory, exploratory, some embedded and transformative designs) call for: ! connected data analysis

Strategies for comparing results:

! !

Side-by-side comparison of results on a topic Joint display (often a table) that relates quantitative information (categories, scores) to qualitative information (themes, quotes) Data transformation to facilitate easy comparisons and further analyses

Creswell & Plano Clark (2007)

19

Creswell & Plano Clark (2007)

20

Creswell & Plano Clark (2007)

21

Creswell & Plano Clark (2007)

22

Wittink et al. (2006, p. 307)

Data transformation
! !

Convert one form of data into the other form so that it can be easily merged More often done by converting qualitative results into numbers
! ! ! !

Develop codes and themes Count the occurrences Output the values Relate the quantized qualitative data with the quantitative data in statistical analyses
Creswell & Plano Clark (2007) 24

Creswell & Plano Clark (2007)

23

Interpreting Merged Results

! !

Connected Data Analysis for Sequential Approaches

!

Interpretation relates to the design Most common: interpret extent to which the two databases converge by looking for:
! ! !

Strategies for connecting results QUAN ! QUAL:

!

Congruent and discrepant evidence Consistencies and inconsistencies Conflicts

Options for resolving discrepancies:

! ! ! !

Review methodological procedures Collect additional data Reexamine databases Suggest future research
Creswell & Plano Clark (2007) 25

Include quantitative analyses to facilitate the selection of best participants for the follow-up phase. For example: ! Identify typical scores ! Graphically display scores to identify extremes ! Identify groups based on significant differences ! Select individuals who scored differently on significant predictor variables

Creswell & Plano Clark (2007)

26

Connected Data Analysis for Sequential Approaches

!

Interpreting Connected Results

!

Strategies for connecting results QUAL ! QUAN:

!

Although interpretations are being made in each phase, be sure to also make a larger interpretation across the two phases at the end Interpret:
!

Include qualitative analyses to facilitate the design of a typology or instrument. For example: ! Look for natural differences in responses to help identify categories ! Identify good quotes, codes, and themes to guide instrument development (items, sub-scales, and variables); develop a table that illustrates connection from qualitative results to quantitative items

Whether the qualitative follow-up information explains the initial results and whether the combination provides a better understanding of the overall problem Whether the quantitative follow-up information provides a more generalized understanding of the overall problem

Creswell & Plano Clark (2007)

27

Creswell & Plano Clark (2007)

28

Validate the data within the mixed methods design

!

Outline of the Structure of a Proposal for a Dissertation, Thesis, or Study

Title Introduction The research problem Past research on the problem Deficiencies in past research and one deficiency related to a need to collect both quantitative and qualitative data The audiences that will benefit from the study Purpose statement Quantitative, qualitative, and mixed methods research questions/hypotheses Philosophical Foundations for using mixed methods Literature Review (include quantitative, qualitative, and mixed methods studies, if they are available) Methods A definition of mixed methods research The type of design used, and its definition * Challenges in using this design Examples of using the type of design (in your field, if possible) * Reference to and inclusion of a procedural diagram in appendix * Quantitative data collection and analysis (preparing, exploring, analyzing, representing, interpreting, and validating) * Qualitative data collection and analysis (preparing, exploring, analyzing, representing, interpreting, and validating) Mixed methods data analysis procedures Validity approaches in quantitative, qualitative, and mixed methods research Potential Ethical Issues Researchers Resources and Skills Timeline for Completing the Study References Appendices with instruments/protocols

Report and discuss validity within context of the quantitative and qualitative strands Define mixed methods validity as employing strategies that address potential issues in data collection, data analysis, and the interpretations that might compromise the merging or connecting the quantitative and qualitative strands of the study and the conclusions drawn from the combination Consider how validity relates to the different designs (more work needs to be done in this area!) Identify potential threats to validity and discuss how they will be minimized
Creswell & Plano Clark (2007) 29

30

Activity
!

Upcoming Schedule
!

Let's examine the data analysis strategies (merged or connected) used in different mixed methods studies
!

April 6: Writing and evaluating mixed methods research

!

Read: D&CMMR: Ch. 8; MMReader Ch 12 (Sandelowski, 2003) Read: MMReader: Ch. 13 & 14 Student presentations All project papers due! Student presentations Student presentations

If you are using a concurrent approach:

!

April 13: Reflections on mixed methods

! !

Examine: MM Reader chapter 15 or 16

!

If you are using a sequential QUAN ! QUAL approach:

!

April 20:
! !

Examine: MM Reader chapter 20 or 21

If you are using a sequential QUAL ! QUAN approach:

!

Examine: MM Reader chapter 22

April 27:
!

Creswell & Plano Clark (2007)

31

Creswell & Plano Clark (2007)

32