Journal of Mixed

Methods Research
Volume 1 Number 3
July 2007 226-241
Ó 2007 Sage Publications
A Recursive Approach to 10.1177/1558689807301101
http://jmmr.sagepub.com
Mixed Methods Research hosted at
http://online.sagepub.com

in a Longitudinal Study of
Postsecondary Education
Disability Support Services
Thomas W. Christ
University of Hawaii, Honolulu

Mixed methods research has increased in popularity over the past 20 years. Literature reveals that
exploratory qualitative analysis followed by confirmatory survey research is common and concurrent
studies outnumber longitudinal design. Longitudinal studies using quantitative and qualitative methods
in sequence for exploratory purposes are rare, and no studies were found that combine exploratory
quantitative analysis followed by both an exploratory cross-case analysis, and an exploratory longitudi-
nal analysis. This design was used in the present study of postsecondary disability support services, and
it is being presented to highlight how recursive analysis at each stage can be used to refine subsequent
research questions.
Keywords: mixed methods; qualitative; disability; longitudinal; grounded theory

L eading constructivists such as Lincoln and Guba (2005) have indicated that mixed
methods are prominently influenced by postpositivist philosophical viewpoints. This
example will demonstrate how an exploratory multistage mixed methods design can
embrace the constructivist view that studies should be flexible, refuting theorists such as
Yin’s (2006) advice that the researcher use preconceived procedures including overarch-
ing research questions that cover both the quantitative and qualitative phases of the study.
Yin’s approach appears most appropriate for confirmatory research but may be restrictive
in exploratory studies and any mixed methods design that incorporates a longitudinal
phase of analysis.
Mixed methods research has increased in popularity since Bryman (1988), Brewer and
Hunter (1989), and Greene, Caracelli, and Graham (1989) first published their views about
combining qualitative and quantitative methods in social science research. According to
Creswell (2006), Tashakkori and Teddlie (2003), Onwuegbuzie and Johnson (2004), and
Yin (2006), multiple method evaluations have great potential but are time-consuming and
complicated, often requiring teams of researchers to undertake these tasks. Sequential
mixed methods research is most apparent in designs where one method prominently
informs the next. Meta-analysis of articles using mixed methods research published in the
social sciences indicated that concurrent designs are more prominent than sequential
studies (Collins, Onwuegbuzie, & Sutton, 2006; Plano Clark, 2006). Many mixed designs
begin with an exploratory qualitative component followed by confirmatory survey
research. This sequence is logical when generalization is the researcher’s goal. Review of
the literature also revealed that concurrent collection of data for exploratory purposes is
226
 Christ / A Recursive Approach to Mixed Methods Research 227

common in mixed designs, and occasionally longitudinal studies use multiple methods to
gauge changes over time. Regardless of sequential or concurrent design, longitudinal
studies using quantitative and qualitative methods in sequence for exploratory purposes
are rare, and no studies were found that utilize quantitative exploratory research to inform
a longitudinal qualitative cross-case analysis. This unique methodological sequence was
recently performed in a longitudinal case study of disability support services using
multiple methods research (Christ, 2006).
Mixed methods studies, according to prominent constructivists such as Lincoln and
Guba (2005) and Howe (2004), are often influenced by postpositivist philosophical view-
points. In essence, these authors are challenging the concept of utilizing what many meth-
odological experts would see as diametrically opposed paradigms and philosophical
assumptions in a single study. This article provides the opportunity to challenge this con-
troversy by illustrating the complementary and logical nature of using exploratory quanti-
tative and qualitative analyses in one longitudinal study of disability support services.
Through example, this article will illustrate how qualitative and quantitative methods were
used at separate phases of analysis to logically explore institutional change over time.
Furthermore, this example will highlight how sequential and recursive processes were
used to analyze data that informed when it would be advantageous to make changes in
subsequent phases of the design. This example demonstrates the importance of flexibility
coveted by constructivists when making logical decisions about when a study should be
altered to better understand the phenomenon in question. In the case of the presented
example, changes that occurred over time in disability support services.
This article further highlights how conclusions in an initial stage of a study can be used
to logically guide critical components, including the research question used in the con-
struction and analysis of subsequent phases of a study. This concept challenges theorists
such as Yin (2006), who indicated that mixed method case studies should be developed
based upon preconceived procedures, including overarching research questions that cover
both the quantitative and qualitative phases of the study. Although preconceived proce-
dures may be most appropriate in a confirmatory mixed methods design, they may be
inappropriate or even restrictive when both the qualitative and quantitative phases of the
research study are exploratory and conducted in sequence. Finally, this example highlights
the strength of utilizing key grounded theory techniques including constant comparative
method of data analysis (Charmaz, 2006) and recursive coding strategies (Strauss &
Corbin, 1998) to identify core theoretical conceptions that emerge from themes. The core
theoretical conceptions that emerge can be used to inform decisions about the focus,
design, and analysis of subsequent phases in a project. This process of using core knowl-
edge gained from earlier phases of analysis to logically inform decisions follows the origi-
nal intent of grounded theory, that of developing theory from data (Glaser, 1978; Glaser &
Strauss, 1967; Strauss & Corbin, 1998). The presented example will further highlight the
importance of methodological flexibility and the recursive analysis process which allowed
the research focus and questions to logically evolve at each phase. As Huberman and
Miles (2002) explained, ‘‘If a new data collection opportunity arises or if a new line of
thinking emerges during the research, it makes sense to take advantage by altering data
collection, if such an alteration is likely to better ground the theory or to provide new
theoretical insights’’ (p. 16). Finally, if the example presented in this article had followed
228 Journal of Mixed Methods Research

Yin’s suggestion of creating a predetermined research question that drives the quantitative
and qualitative analysis in one study, there would have been no opportunity to modify the
design and research questions to capture important details of how an unanticipated critical
incident had a severe impact over time.

Mixed Methods Study Overview

This methodological overview summarizes the logic and procedures used in the design
of this multistage exploratory study. The first phase of research began in 1999 with
a national survey of 1,500 disability support coordinators. The survey was repeated in
2001 to determine if significant changes occurred over time. Exploratory factor analysis
revealed trends and constructs, but many questions that emerged as a result of the analysis
could not be answered using traditional quantitative research methods. The survey analy-
sis was extremely helpful in two primary ways. First, the results helped frame criteria used
in purposeful selection of three exemplary postsecondary institutions. Second, the results
guided the construction of key semistructured interview questions used to collect data for
the qualitative cross-case analysis at the three sites.
The second phase of the research study consisted of a qualitative cross-case analysis.
Three sites were purposefully selected using criteria that included information from the
national survey to inform site selection. This criterion was presented to experts in the field
of disability studies to solicit nominations of applicable exemplary sites. Once the sites
were selected, information from the literature, advice from experts, and findings from the
national survey were compiled into a set of semistructured interview questions that
reflected the intent of the research questions. Interviews at the three sites were conducted
over several months in 2002. Data including the transcribed interviews were analyzed for
each case individually and then together to perform the cross-case analysis. Transcripts
were analyzed over several months using an open-coding technique as first described by
Glaser (1978) to develop the one set of codes for the three cases. This was the first step in
the recursive analysis process that allowed the opportunity to locate core categories. Next,
core categories for each case were analyzed to determine ‘‘conceptual connectors,’’ which
highlighted how categories are related to their properties (Glaser, 1992, p. 38). This pro-
cess followed the constant comparison format first described by Lincoln and Guba (1985)
that uses a form of inductive logic to develop emerging themes and categories from a
mass of narrative data. Modified version of Glaser’s comparative coding technique (1992)
helped guide the final cross-case analysis of the three sites. This modified version focused
primarily upon the theoretical constructs, not earlier open codes, to generate overarching
theoretical conclusions used to highlight similarities and anomalies in services between
sites.
The third phase of this analysis involved studying a critical single site faced with a sig-
nificant budget reduction. This site analysis highlights the value of methodological flex-
ibility in a longitudinal study as the follow-up was warranted when funding at the site was
drastically cut. Thus, if a critical aspect such as the research questions was predetermined
as is suggested by Yin (2003) and common in typical quasi and true experimental designs,
this third phase would have to be treated separately. This highlights the flexibility and
 Christ / A Recursive Approach to Mixed Methods Research 229

usefulness of mixed method designs in a longitudinal study where researchers are not
forced to conform to one methodological paradigm or predetermined overarching research
questions. In the example presented, compelling circumstances at the one site undergoing
a significant budget reduction prompted change to the study design. By adding a longitudi-
nal phase and modifying two of the four research questions, the opportunity to explore
how the organization offset damaging effects of decreased funds strengthened the entire
study and was much more meaningful for the intended audience of policy makers and dis-
ability support coordinators. Thus, circumstances of the condition under examination and
flexibility in analysis allowed the study to expand and develop the critical theme of leader-
ship that emerged during the cross-case analysis. By extending and reformulating the
study of one site over time, leadership qualities could further be examined from four dif-
ferent levels within the organization (coordinator, supervisor, support staff, and students),
adding credibility and depth to the results (Patton, 1999).
The three phases of this study (see Figure 1) remained focused upon analyzing charac-
teristics that constitute appropriate and effective support services. The quantitative survey
analysis provided the opportunity to describe national trends in disability related services.
The qualitative cross-case analysis allowed for the investigation of what strengthened dis-
ability related services and how they were utilized. The longitudinal phase provided the
opportunity to compare the coordinator’s, supervisors’, service providers’, and students’
perspectives of what happened to services in light of a 40% budget reduction. Each phase
offered information that was useful when framing the design and research questions used
in this exploratory process. Some could argue that the design was ad hoc rather than each
phase informing the next. Regardless of this opinion, the flexibility in design resulted in
much more important findings than would have emerged if the entire design was estab-
lished at the beginning of the 6-year study. Several critical features important to under-
standing exemplary disability related services in postsecondary education were not
imagined in the first and second phase and only emerged in the longitudinal analysis.
What turned out to be critical features, that leadership skills and collaborative efforts con-
tributed to organizational strengths, were never considered in the early stages of analysis.
Thus, the three phases made possible the exploration of what comprised support services
in the first phase, how legislation and funding were related to support services in the
second phase, and how staff utilized innovative techniques to offset declining funds in the
third phase. Each phase will now be presented in summary form as a way to describe
the methodological procedures and decisions made over the course of a 6-year study of
disability support services.

Phase I: National Survey Analysis

The first phase of this mixed methods study began with the analysis of a repeatedly
administered national survey of disability support coordinators. Christ and Stodden (2005)
analyzed a total of 1,067 surveys from 1,530 randomly selected postsecondary institutions
(650 in 1999 and 417 in 2001). Statistical analysis determined if the survey items grouped
together into meaningful constructs (Strategies, Assistive Technology, Accommodations,
Vocation/Work Supports), if 2-year and 4-year institutions were significantly different in
230 Journal of Mixed Methods Research

Figure 1
Overview of Longitudinal Mixed Method Study

1999 Survey 2001 Survey Exploratory factor analysis

Surveys of 1,500 Postsecondary Institutions

El Camino Quinsigamond George Brown Cross Case Analysis

Purposeful Selection: Three Case Studies

El Camino: Support coordinator,Supervisors, Support staff, and students

Longitudinal Analysis One Site–Four Levels

the way they provided support services, and if they were different over time. The research
questions used for this phase of the analysis were (a) Can survey items be grouped
together into meaningful constructs? (b) Are support constructs significantly different in
comparison of varying 2-year and 4-year institutions? (c) Are support constructs different
in comparison of varying time periods (1999 and 2001)? and (d) Is there an interaction
effect between, institutions (2-year and 4-year) and time (1999 and 2001)?
First, item loadings were examined for retention followed by exploratory factor ana-
lysis (Hair, Anderson, Tathem, & Black, 1992) to determine if items grouped into the
constructs as designated by the survey authors (Christ & Stodden, 2005). Specifically,
principal component analysis was used in the development of constructs, reliability was
assessed, and regression was used to determine if the constructs were significantly differ-
ent between the institutions surveyed (2-year and 4-year) and over two points in time
(1999 and 2001). Multivariate analysis of variance (MANOVA) was then conducted to
determine if institution type and time exhibited main or interaction effects.

Phase II: Cross-Case Analysis

The qualitative cross-case analysis emerged in part from inherent limitations in the
survey design. Specifically, by selecting the coordinators to be surveyed, it reduced the
opportunity to determine true differences in the amount of supports being utilized at post-
secondary institutions as they reported what supports were offered, not how they were
used, or their perceived value by staff and students. These limitations alone raised many
questions that were difficult if not impossible to answer using traditional quantitative
research methods.
According to Yin (2003), multiple-site case studies are a viable option when attempting
to address questions that are not readily discernable using traditional statistical analysis.
For the purpose of exploring concepts including how policy is related to funding, how
 Christ / A Recursive Approach to Mixed Methods Research 231

collaboration efforts are supported by institutional leaders, and how supports are per-
ceived, a qualitative approach was warranted at three purposefully selected institutions.
The interviews were conducted at each institution with a variety of personnel. Raw tran-
scripts were sent to participants to check accuracy and then the data was subsequently ana-
lyzed using a cross-case approach (Creswell, 2003) to explore several aspects of disability
support services including how funding, legislation, and collaboration contributed to ser-
vices at the three purposefully selected exemplary colleges. The specific research ques-
tions that directed the second phase of this study were as follows: (1) How are support
services provided to students with disabilities in exemplary postsecondary institutions?
and (2) How does leadership and staff cohesion relate to the provision of disability related
support services in postsecondary institutions?
The second phase began with purposeful site selection driven by survey findings that
established the parameters to include 2-year postsecondary schools in North America with
a reputation for excellence and active record of funding from a variety of sources. The
exemplary sites were purposefully selected (Maxwell, 2005; Merriam, 2001) in 2002 as a
way to represent best practices in the field of postsecondary support services. The three
sites were chosen based upon a pool of recommended institutions generated by experts at
the National Center on Secondary Education Transition (NCSET) at the University of
Minnesota and the National Center for the Study of Postsecondary Educational Supports
(NCSPES) at the University of Hawaii. Site selection was based upon the following cri-
teria: (a) 2-year postsecondary institutions that provide all of the services indicated in the
National Supports Survey, (b) institutions that have a reputation for providing excellent
service and support to students, and (c) institutions that actively pursue funding for ser-
vices from a variety of sources.
The three sites chosen from a pool of expert recommendations were El Camino College
(Los Angeles, California), Quinsigamond College (Worcester, Massachusetts), and George
Brown College (Toronto, Canada). El Camino College, 1 of 109 California 2-year
colleges, is located in Los Angeles and was recommended due to the longevity of the pro-
gram (32 years), a strong reputation for providing excellent services, extensive collabora-
tion efforts, and disability-specific funding provided by the state. Quinsigamond College, 1
of 8 colleges in the state of Massachusetts, was nominated for several reasons including
an excellent record of attaining state grants and outstanding collaborative efforts with
secondary education. George Brown College in Toronto was elected as it has a stellar repu-
tation for serving a broad range of students with disabilities while providing excellent sup-
port services. Furthermore, Ontario’s prescriptive basis for providing funds is unique and
vastly different from the other two sites located in the United States.
The analysis of the three case studies relied upon aspects of axial and thematic coding
(Strauss & Corbin, 1998) as a way to examine the conditions and actions of categories
when confirming, cross-validating, and corroborating themes that emerged during indivi-
dual and subsequent cross-case analysis of the three sites (Huberman & Miles, 2002). The
majority of data used in the cross-case analysis came from 17 interviews generating 436
pages of raw transcripts in 2002. The interviews were repeatedly analyzed, revealing 22
open codes that were used to create code reports. Recursive analysis of the code reports
resulted in six major dimensions: funding, legislation, collaboration, self-determination,
barriers, and innovative techniques. Coding the raw data helped refine the exploration of
232 Journal of Mixed Methods Research

the phenomenon in question, the core categories that eventually lead to theoretical con-
cepts that emerge during the final stage of the cross-case analysis. These core findings
were then provided to key participants for comment and clarification as necessary.

Phase III: Longitudinal Analysis

The conception of adding a longitudinal phase to this study first emerged when it was
learned that the state of California, site of El Camino College, reduced funding for disabil-
ity support services in 2003 by close to 40% shortly after the second phase in this analysis.
Thus, a follow-up study would allow the examination of how an exemplary site dealt with
a significant budget reduction and how the services compared before and after the event.
Some of the specific features that were analyzed during the third phase emerged during the
final stages of the cross-case analysis. Specifically, it became apparent that leadership was
a critical feature at all three of the exemplary sites warranting special attention. Revisiting
the site that had a significant budget reduction allowed the opportunity to examine in detail
an example of how leadership was perceived from several levels within the organization.
The third phase of the study was driven by two research questions used to distinguish
longitudinal changes at one institution drastically effected by a significant budget reduc-
tion: (1) How have disability support services at an institutional level changed as a result
of declining funds? and (2) Are there differences in the way students and staff view the
effectiveness of disability support services?
The longitudinal phase relied upon 23 interviews with participants at four levels within
the organization. This included the disability support coordinator, support supervisors, sup-
port providers, and students. The concurrent collection of data at multiple levels (Creswell,
2003) utilized inductive content logic as described by Patton (2002) to analyze a single site.
Themes from the survey and cross-case analysis results were used to create semistructured
interview questions for the follow-up site visit. Elements not addressed in the previous
phases were addressed including fluctuations in budget, cost-saving procedures, and how
the role of leadership had an effect upon the provision of services. Some of the semistruc-
tured interview questions used when interviewing staff were written to address specific fea-
tures of the budget reduction, and others were generated in a recursive fashion as they
emerged from results of the survey research and the three case studies. The semistructured
interview questions for the coordinator included items that addressed budget, morale,
vision, technology, and collaboration. Supervisor questions focused upon structural change,
leadership qualities, and vision. Support staff questions asked about changing roles, pro-
cesses, and dynamics of the disability support center. Questions for the students focused
upon the organizational climate, service delivery, collaborative efforts, barriers, and transi-
tion activities.
The 23 interviews were held on-site, conducted in person, digitally recorded, and tran-
scribed generating 406 pages of transcripts. Transcribing began immediately after the inter-
view process was complete. All transcripts were sent to the participants to cross-check
accuracy, and then they were printed for the purpose of writing comments and indicating
potential codes (nodes) handwritten in the margins. Quotable passages were highlighted
for possible inclusion in the final write-up. Salient points from the side bar notes and
 Christ / A Recursive Approach to Mixed Methods Research 233

memos created from the raw transcripts and a combination of the survey and cross-case
analysis findings were used to generate 24 open codes (Glazer, 1992). The transcripts were
converted to Rich Text Format (RTF) and imported into a case file used by NVivo.
The code report indicated the name of the participant file and paragraph number assuring
accuracy. The 24 code reports were then converted to word documents with page numbers
added and printed. Coded reports ranged in length from 49 pages to less than 1. Seven of
the 24 codes were eliminated from the original list as they produced so few quotes that
analysis was impossible. From the 17 remaining code reports, four code categories were
distinct in the number of coded sections included in the report drawn across all the
interviews.
Review of the code reports resulted in four code categories as they emerged directly
from the data (Denzin & Lincoln, 2003; Strauss & Corbin, 1998) and were used to frame
this longitudinal summary. The four categories are as follows: ‘‘Changes’’ was operation-
ally defined as the positive and negative effects to disability support services that occurred
over a 3-year period and as a result of significant budget reduction. ‘‘Leadership’’ was the
individual characteristics and actions exhibited by the coordinator, the supervisors, and
the support staff that guided the disability support center. ‘‘Technology’’ included techno-
logical supports that assist students with a variety of disabilities to function in the college
setting. ‘‘Interpreter’’ includes all aspects of costly interpreter services provided.
These four categories had several subsumed codes that significantly overlapped when
examined using NVivo coding stripes. This option allows the visual examination of the
coded transcripts to determine how codes overlapped. Specifically, ‘‘Changes’’ had the
following subsumed categories: Accountability, Barriers, Commitment, Efficiency, Fund-
ing, Grouping, and Responsibility. ‘‘Leadership’’ had the following subsumed categories:
Collaboration, Communication, Decisions, Dedication, Philosophy, and Vision. Inter-
preter services and Technology are specific categories that were examined separately as
they represent specific exemplary support services noted to be critical and important by
experts when serving postsecondary students. Findings from the four categories were then
sent to key participants for comment and clarification before proceeding with the final
analysis of the data from the three phases together.

Synopses of Results
Phase I Results
Principal component analysis (Heck, 1998) revealed that 25 items adequately loaded into
four factors operationally defined as Strategies, Assistive Technology, Accommodations,
and Vocation/Work support. Regression revealed there was a difference when institution
type and survey years were compared. Two-year and 4-year institutions were significantly
different in Assistive Technology (2-year > 4-year), Accommodations (4-year > 2-year),
and Vocation/Work support (4-year > 2-year). Only Accommodations was significantly dif-
ferent (2001 > 1999) over time (see Table 1). Multivariate analysis revealed no interaction
between institution type and time when all the constructs were combined. Only the main
effect for institution type (F = 18.50, 4 df, p = .000) was found to be statistically
234 Journal of Mixed Methods Research

Table 1
Regression (Strategies, Assistive Technology, Accommodations, Vocation/Work)
B SE Beta t Sig. Adjusted R2

Strategies −.002
Institution type: 2- and 4-year .016 .064 .008 0.246 .806
Survey years 1999/2001 .024 .065 .021 0.649 .516
Assistive technology .023
Institution type: 2- and 4-year −.254 .063 −.127 −4.054 .000∗
Survey years 1999/2001 .182 .064 .089 2.864 .004∗
Accommodations .014
Institution type: 2- and 4-year .224 .059 .118 3.775 .000∗
Survey years 1999/2001 .100 .060 .052 1.666 .096
Vocation/work .007
Institution type: 2- and 4-year .164 .062 .083 2.651 .008∗
Survey years 1999/2001 .111 .062 .055 1.771 .077
∗
Significant at p < .01.

significantly different (see Table 2). These findings suggest that 2-year and 4-year institu-
tions differ on the level of supports provided. Thus, the survey research indicated that in a
population of postsecondary institutions there was indeed a difference between institution
types in the way they are providing services. These findings were informative in the purpo-
seful selection process, when determining how to ‘‘bound’’ the cases (Gomm, Hammersley
& Foster, 2002), and when establishing potential codes (Strauss & Corbin, 1998), con-
structs, and potential semistructured interview questions used in the qualitative cross-case
and longitudinal phases of this research study.

Phase II Results
The cross-case analysis revealed that supports at the three sites were remarkably simi-
lar. All offered academic, social, and physical supports to self-identified students, but the
quantity and way services were administered varied significantly according to differences
in legislation and funding. Both California and Ontario have disability specific funding
sources as required by legislation while Massachusetts does not. Cross-case analysis
revealed that the three sites approached funding issues differently. At Quinsigamond, the
coordinator openly disseminated information to the staff about the budget. This was not
true at El Camino and George Brown Colleges, which had much larger budgets. The
director at El Camino College did not disseminate specifics about the budget amount or
how funds were allocated out of concern for possible interdepartmental resentment over
differences in how the funds were distributed. The director indicated ‘‘interpreter services
are extremely costly and there was concern that other departments would resent how much
is spent on interpreters.’’ Conversely, the director at Quinsigamond with a much smaller
budget indicated that specifics about funding were open to all staff. ‘‘We regularly have
meetings together to share information about the budget. Everyone is responsible for their
own budget and at least once a month we go over the budgets in prescheduled meetings.’’
 Christ / A Recursive Approach to Mixed Methods Research 235

Table 2
Multivariate Tests (1999/2001 Surveys of 2-/4-Year Institutions)
Effect F df Sig.

Institution type: 2- and 4-year

Wilks’s Lambda 18.54 4 .000∗
Hotelling’s trace 18.54 4 .000∗
Survey year: 1999 and 2001
Wilks’s Lambda 1.814 4 .124
Hotelling’s trace 1.814 4 .124
Institution type: 2- and 4-year by survey year 1999 and 2001
Wilks’s Lambda 1.959 4 .099
Hotelling’s trace 1.959 4 .099
∗
Significant at p < .01.

Collaboration emerged as a major theme in the cross-case analysis phase of this study.
Each site had distinctly different ways of collaborating within and between departments,
but all three sites exhibited strong collaborative ties to the various departments within
their colleges. Thus, examining collaboration revealed the importance of strong and effec-
tive leaders who promote collaborative efforts within and between departments as a way
of integrating a unified and motivated staff. According to the coordinator at El Camino
College, ‘‘Collaboration efforts are essential. Successful student integration on campus
begins with the partnerships they build as they integrate into whatever major they want.
This center is like an invisible support in many cases.’’ Collaborative efforts cross the
institutions hierarchical boundaries; ‘‘They cross the lines from departments to the dean,
staff to administration, and even peer to peer. The internal collaborative efforts help to
develop a sense of institutional commitment.’’
Dedication to quality and efficiency was a theme that emerged at the three sites in the
cross-case analysis. This was most apparent in the way each site created innovative techni-
ques as a way to increase efficiency and improve staff morale. Many innovative techni-
ques were seen at the three sites. As example, El Camino and George Brown colleges had
interpreter training programs. At El Camino, interns in the program were hired to provide
in-class note-taking services for hearing-impaired students as a way to save funds, while
Quinsigamond College had a Community Transition Program (CTP) that provided specia-
lized training for students who were preparing to graduate and transfer to work. Although
the three sites had similar technology, El Camino College was extraordinarily innovative
in the way they provided technology services to students with disabilities. For example,
through the use of middleware, a technology that links several software programs
together, sophisticated switching devices can be linked to other software for activating the
computers to allow students with significant physical disabilities to utilize sophisticated
programs simultaneously. According to the technology coordinator, these programs can
now be linked:

Dragon-Naturally Speaking, a widely used voice recognition program and Joz, a program uti-
lized by the Blind to verbalize scanned materials can be linked by JawBone, a type of middle-
ware that allows the separate programs to operate together. Dragon-Naturally Speaking, Joz,
236 Journal of Mixed Methods Research

and Handi-word, a word-prediction software can also be linked to Easy-Keys, a basic switch
for retainers, for your chin or foot as a means of control. If someone is able to blink, they can
use this technology.

An innovative and economical technique has been utilized at Quinsigamond College to link stu-
dents requesting note-taking services to other students willing to volunteer their notes. By using
highly qualified peers as volunteer note-takers at the beginning of the semester, it is much more
efficient, less disruptive, and cheaper to provide note-taking services. According to the director at
Quinsigamond College, requesting volunteer note-takers is an unusual cost-saving measure that
provides a superior level of support:

We try to find a good student in a class who is willing to take notes. We saved about $12,000
this year by doing that. A lot of times, they could be honor students who are doing very well
and they might receive a stipend. We had a lot of students from this semester who did that. It
is a huge cost saving and helps us.

Phase III Results

Analysis of the data collected for the longitudinal study performed in 2005 highlighted
reactions to a significant reduction in funding. Foremost, leader-prompted change made
the organization structurally efficient with less staff, yet the services remained student-
centered. According to the coordinator at El Camino College, staff must remain student-
centered:

Every time there’s a decision that needs to be made whether it’s long term planning, strategic
planning, or troubleshooting and solving problems, the first question that I always ask every-
one is what is the impact on the students and how can we resolve this dilemma and be more
student centered.

By reframing staff responsibilities, replication of services was avoided. The only way
this was successful according to the director was sophisticated interdepartmental commu-
nication. ‘‘Without this level of communication between departments it would have been
impossible to maintain and share scarce resources.’’ Critical links were made between
departments including the tutoring lab, counseling, and career placement center in order
to maintain services and support for all attending students, not just those with disabilities.
The longitudinal analysis highlighted how individually administered services were
replaced with those provided to students in groups. For example, technology and strategy
training, career and vocational counseling, interpreting, and test accommodations were all
individually administered in 2002. With the 40% budget reduction and elimination of all
nonessential full-time staff in 2003, group-administered services were enacted through a
mandate from the coordinator. This increased efficiency yet made little difference in the
applicability of the support students receive. Grouping students was a viable and efficient
alternative to providing individual services, but unless the supports are well executed by
competent and dedicated staff, there is potential for serious degradation of services.
Although virtually all the respondents in the longitudinal study indicated that grouping stu-
dents had no visible negative impact upon disability-related supports, it would be negligent
and potentially damaging for this finding to be taken out of context. Grouping students is
 Christ / A Recursive Approach to Mixed Methods Research 237

only a viable alternative if skillfully executed by knowledgeable staff. By virtue of the

coordinator’s and supervisors’ planning and expertise and the extraordinary efforts to
maintain services by staff, the services remained remarkably similar to those provided
before the budget reduction. The students agreed that they saw no difference in the way the
services were provided, although staff were acutely aware of the changes.

Mixed Methods and Longitudinal Design Implications

The sequential design provided the opportunity to determine how leadership character-
istics and staff cohesion had a direct effect upon the provision of disability support ser-
vices. This finding would not have emerged if the study were limited to survey research or
concurrent collection of mixed data. The sequential multilevel design in the final phase of
analysis helped validate how the coordinator’s positive tone had influence upon the entire
operation of the support service office. This tone, according to the respondents, helped the
supervisors and support staff members maintain their dedication in a time of transition
and limited resources. All staff consistently indicated the coordinator was a great leader
who was willing to share in the duties of creating values and goals. Thus, by collaborating
to create a shared vision, staff became vested in their role despite increased duties and
responsibilities. According to staff, the coordinator and supervisors promoted open com-
munication and the opportunity for everybody to learn from each other. As one counselor
stated ‘‘We are all learning a little bit about the different roles and responsibilities. Cer-
tainly after the budget cut this has increased.’’ Major changes were designed through col-
laborative efforts between the supervisors and the staff following the directives of the
coordinator to ‘‘do things smarter, better, and cheaper,’’ that was her charge. According to
one supervisor ‘‘We were directed to group, orient, and determine the students needs and
that’s what we did. A whole integrated process to help students prepare to get off the
ground and running from the get-go. We loved it.’’
Declining financial support was another critical topic that could only be studied through
a longitudinal design. Concurrent data collection techniques cannot reveal trends or be
used to explore hypotheses about change. Through a longitudinal examination, surprising
results were revealed. Not all aspects of the budget reduction were negative as initially
hypothesized. Staff indicated that many of the services were made more efficient and
improved despite the loss of personnel. The budget reduction did require significant modi-
fications to the format and content of services, but as the modifications were purposeful
and executed with care, the quality of services did not degrade.
In summary, change occurred as a result of a significant budget reduction. Some
changes were devastating to highly coveted and valuable supports, while other changes
resulted in unforeseen benefits including staff cohesion and increased efficiency. Much of
what was learned in this study could not have been revealed unless the quantitative and
qualitative phases were exploratory and sequential. By utilizing mixed methods and a
longitudinal design in a single study, many of the critical themes that emerged would not
have been evident. This study highlights how a pragmatic approach to research is critical.
In this case, the research questions, not ideologies or the researcher’s preference for quan-
titative or qualitative analysis, drove the study. Both methods served their purposes. The
quantitative survey phase helped establish a nationally representative description of the
238 Journal of Mixed Methods Research

types of services provided to students with disabilities in postsecondary education.

Exploratory factor analysis also established several constructs that were useful throughout
all phases of the study while revealing parameters used in the purposeful selection of
exemplary sites for further analysis. The qualitative cross-case analysis revealed similari-
ties and anomalies in each of the three purposefully selected exemplary sites. This phase
of the analysis was particularly useful for gaining a better understanding of how policy is
related to funding and how leadership qualities influence staff cohesion and effectiveness.
Finally, the longitudinal phase was the only way to reveal how services changed over
time. In particular, the sequential collection of data allowed the analysis of a particularly
relevant subject: reactions to a reduction in funding. This sequential phase that emerged
as a result of a legislative mandate appears to be most meaningful for those directing and
supervising support service as they face the challenge of providing more students man-
dated services with reduced financial support.
Although the cross-case analysis was particularly helpful in describing similarities and
anomalies at exemplary sites, the concurrent collection of data could not reveal how ser-
vices and supports evolved over time. For example, the cross-case analysis indicated that
many of the services provided were quite similar despite completely different locations,
policy, and legislation at the three exemplary sites, but funding did have an effect upon
the quantity of supports made available to the students. The final phase revealed just what
happens when a budget reduction occurs. Contrary to expectations, when changes were
executed by dedicated individuals following a well-thought-out plan, the services did not
degrade as initially expected.
The final analysis in this study examined the findings from all three phases collectively.
Several themes were evident when the results were merged. One example is that exemp-
lary services can be attributed to distinct goals and drive established by dedicated coordi-
nators who exhibited an extraordinary degree of motivation to provide the best possible
service for students. These leadership qualities did not stop with the coordinators, as they
extended directly to the supervisors, the specialists, and the support staff. Although com-
petency was expected in this study that focused upon exemplary support services, most
revealing was that leadership qualities appeared infectious in the organizations crossing
boundaries and extending all the way to the students.

Limitations
There were a number of limitations for this study. Foremost is that the sites were
purposefully selected from expert nominations of what constitutes exemplary disability
support centers. Although this study was designed to explore best practices and innovative
techniques that promote successful disability services, the choice and number of sites
reduces any opportunity to generalize the results to other institutions.
The unique nature of the sites, in terms of the legislation, location, size, and funding
sources further preclude the opportunity to generalize findings. This may not be a problem
if the findings are examined for the purpose of why they were analyzed, a way of explor-
ing and describing specific features that make an exemplary disability service delivery
system. Although the findings by design are limited in how they can be used, they describe
in detail specific features of the disability support centers, the characteristics of those
 Christ / A Recursive Approach to Mixed Methods Research 239

involved in the organizations, and the types and quality of services that are provided in
exemplary programs. Thus, these findings may not easily apply to other programs that are
immature, poorly developed, or otherwise viewed as less than exemplary. However, read-
ers of the study may find that some of the elements and service delivery strategies seen in
the exemplary programs are applicable to their local circumstances.
A more specific limitation is the possibility of the Hawthorne effect as a result of
defining the sites as exemplary when soliciting participation. By framing the request for
inclusion as a way to examine exemplary disability services, the respondents may have
presented a more positive picture of the disability support services than they otherwise
would have. Thus, the expectancy effect had potential for misleading the researcher as to
how significant was the effect of the budget reduction upon services. Staff may have been
reluctant to present negative information about the budget reduction given that the frame-
work of the study was focused upon exemplary institutions and the services they provide.
Another limitation specific to the longitudinal study is that there were only two site vis-
its and each visit lasted one week. This reduced the opportunity to fully understand the
culture of the environment or how the changes evolved over time.

Discussion
This longitudinal mixed methods analysis of disability support services serves as an
example of how quantitative and qualitative phases in a sequential analysis are related in a
single exploratory study. This example indicates how evidence can converge in the final
phase of analysis to produce rich themes and conclusions that are more compelling than
what can be produced separately in a single method study. Yin (2006) argued for a single
overarching focus to be used when framing a mixed methods study; otherwise, there is
risk that the methodological sections essentially create decomposed multiply related stu-
dies. Yin indicated the value of integrating mixed methods findings to produce convergent
evidence that is more compelling than evidence produced from a single method alone. But
Yin did not provide direction when considering a longitudinal analysis, as is appropriate
whenever the intent of the research is to gauge changes over time. Researchers including
Stake (2006) and Tashakkori and Teddlie (1998) also stated the importance of logically
planning methods and procedures in a mixed methods study. There is no argument here;
logically planning and executing research is essential; otherwise the study will potentially
be compromised. Yin suggested that five procedures should be integrated to ensure that
the study does not decompose into separate related studies. These steps resemble the post-
positivist linear approach commonly used in quantitative and some qualitative research
studies excluding those embracing a grounded theory approach. The five procedures Yin
proposed to help maintain the integrity in a mixed methods study include (a) research
questions, (b) units of analysis, (c) samples for study, (d) instrumentation and data collec-
tion methods, and (e) analysis strategies. This pattern is most useful when some aspect of
the mixed methods study is confirmatory. Yin’s linear framework may not be appropriate
when all phases in a mixed method study are for exploratory or longitudinal purposes, as
was the case in the example of disability support services. Following the inductive logic
noted in grounded theory, research questions in the longitudinal cross-case study of
240 Journal of Mixed Methods Research

support services emerged and were refined in a recursive process at each phase of analysis.
Yin’s argument is that when researchers treat qualitative and quantitative research ques-
tions separately, there is a greater chance the study will decompose. This may be the case
in many instances, but at what point does methodological rigidity supersede opportunities
to gain insight when a phenomenon worthy of study prompts changes in design? From a
pragmatic stance, the opportunity to gain knowledge should outweigh methodological
preferences or norms. Logic must be maintained in all research endeavors, but not all
domains have been fully explored or defined, as is the case with a longitudinal multistage
multimethod research project as described in this example.
Yin’s (2006) linear model does not address how constructivist concepts common in
various forms of qualitative research can be extremely beneficial in a longitudinal multiple-
level exploratory study, as the recursive process of analysis can be used to create and refine
the research questions after each sequential phase of analysis. As is the case with grounded
(Glaser, 1978; Strauss & Corbin, 1998) and evolved grounded theory concepts (Charmaz,
2000) that rely upon the recursive examination of data to refine constructs and theories, a
longitudinal multistage mixed methods analysis can also use a logical recursive process to
shape research questions throughout each phase of analysis.
Yin’s (2006) suggestion of following a five-step linear procedure may not be appropriate
when all phases of the study are exploratory, as a linear design has the potential to limit the
possibilities of incorporating unique emergent findings into subsequent phases of a study.
A recursive approach to designing the research questions at subsequent phases of analysis
was used in the longitudinal study of disability support services. In this sense, information
from the national survey nested in the overall study helped define institutions that represent
best practices and the initial research questions used in the cross-case analysis. Findings
from the survey research and subsequent analysis of data collected from the three exemp-
lary institutions also provided themes used to refine the research questions used in the long-
itudinal phase of this study. Following a linear preplanned approach as suggested by Yin
would have potential to restrict the opportunity to let emerging themes drive the direction
of the study. For example, findings from the survey research helped refine the research
questions in the cross-case analysis, and findings from the cross-case analysis helped frame
the research questions used to direct the final stage of analysis. These refined research ques-
tions were much more meaningful as each stage helped reconfigured and refine the focus
of the study. Thus, a recursive approach helped define important aspects and findings that
would not have emerged if a linear model had been followed.

References
Brewer S., & Hunter, A. (1989). Multimethod research: A synthesis of styles. Newbury Park, CA: Sage.
Bryman, A. (1988). Quantity and quality in social research. London: Routledge.
Charmaz, K. (2000). Grounded theory: Objectivist and constructivist methods. In N. K. Denzin & Y. S. Lincoln (Eds.),
The SAGE handbook of qualitative research (2nd ed., pp. 675-694). Thousand Oaks, CA: Sage.
Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. Thousand Oaks, CA:
Sage.
Christ, T. (2006). Longitudinal cross case analysis of support services for students with disabilities in postsecondary educa-
tion. Dissertation Abstracts International, 64 (05). (UMI No. 3216053)
Christ, T., & Stodden, R. (2005). Advantages of developing survey constructs when comparing educational supports offered
to students with disabilities in postsecondary education. Journal of Vocational Rehabilitation, 22(1), 23-31.
 Christ / A Recursive Approach to Mixed Methods Research 241

Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods design (2nd ed.). Thousand Oaks,
CA: Sage.
Creswell, J. (2006, April 10). Continuing the discourse: Advocates for and challengers to mixed methods research. Sympo-
sium conducted at the American Education Research Association Mixed Methods SIG Business Meeting, San Francisco.
Collins, K., Onwuegbuzie, A., & Sutton, I. (2006). A model incorporating the rationale and purpose for conducting mixed
methods research in special education and beyond. Learning Disabilities: A Contemporary Journal, 4(1), 67-100.
Denzin, N., & Lincoln, Y. (2003). Strategies of qualitative inquiry. London: Sage.
Glaser, B. (1978). Theoretical sensitivity: Advances in the methodology of grounded theory. Mill Valley, CA: Sociology
Press.
Glaser, B. (1992). The basics of grounded theory analysis: Emergence vs. forcing. Mill Valley, CA: Sociology Press.
Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine.
Gomm, R., Hammersley M., & Foster, P. (Eds.). (2002). Case study method. Thousand Oaks, CA: Sage.
Greene, J., Caracelli, V., & Graham, W. (1989). Toward a conceptual framework for mixed-method evaluation designs.
Educational Evaluation and Policy Analysis, 11(3), 255-274.
Hair, J., Anderson, R., Tathem, R., & Black, W. (1992). Multivariate data analysis withreadings (3rd ed.). New York:
Macmillan.
Heck, R. (1998). Factor analysis: Exploratory and confirmatory approaches. In G. A. Marcoulides (Ed.), Modern methods
of business research (pp. 177-215). Mahwah, NJ: Lawrence Erlbaum.
Howe, K. (2004). A critique of experimentalism. Qualitative Inquiry, 10(4), 42-61.
Huberman, A., & Miles, M. (2002). The qualitative researcher’s companion. Thousand Oaks, CA: Sage.
Lincoln, Y., & Guba E. (2005). Paradigmatic controversies, contradictions, and emerging confluences. In N. K. Denzin &
Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (pp. 191-214). Thousand Oaks, CA: Sage.
Maxwell, J. (2005). Qualitative research design: An interactive approach (3rd ed). Thousand Oaks, CA: Sage.
Merriam, S. (2001). Qualitative research and case study application in education. San Francisco: Jossey-Bass.
Onwuegbuzie, A., & Johnson, R. (2004). Mixed method and mixed model research. In R. Johnson & L. Christansen (Eds.),
Educational research: Quantitative, qualitative, and mixed approaches (pp. 408-431). Neeham Heights, MA: Allyn &
Bacon.
Patton, M. (2002). Qualitative research and evaluation methods (3rd ed). Thousand Oaks, CA: Sage.
Plano Clark, V. L. (2006, April). Discipline perspectives on mixed methods research: Case analyses from counseling psychol-
ogy, primary care, and physical education research. Paper presented at the annual meeting of the American Educational
Research Association, San Francisco.
Stake, R. (2006). Multiple case study analysis. New York: Guilford.
Strauss, A., & Corbin, J. (1998). Basics of qualitative research. Thousand Oaks, CA: Sage.
Tashakkori, T., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand
Oaks, CA: Sage.
Tashakkori, A., & Teddlie, C. (2003). Issues and dilemmas in teaching research methods courses in social and behavioral
sciences: US perspective. International Journal of Social Research Methodology, 6, 61-77.
Yin, R. (2003). Case study research: Design and methods (3rd ed). Thousand Oaks, CA: Sage.
Yin, R. (2006, April). Mixed methods research: Are methods genuinely integrated or merely parallel? Paper presented at
the annual meeting of the American Educational Research Association, San Francisco.