UNIVERSIDAD INDUSTRIAL DE SANTADER

SERGIO ANDRÉS RODRÍGUEZ GÓMEZ

QUALITATIVE RESEARCH METHODS IN EDUCATION
WEEK 1 June 8th, 2020
Griffee (2012): PART TWO Design
THE QUESTION THE ANSWER

Introduction Research design

(Purpose and RQs)

Figure 1. The structure of a research paper

 TREE: Teacher-Researcher-Educator-Evaluators.
What is research design?
Creswell (2013) uses the term strategy to explain design, as well as tradition, method,
approach, procedure, and process.
Razavieh (1990) say a research design is “a description of the procedures to be followed in
testing the hypothesis”
Babbie (2004), research design “involves a set of decisions regarding what topic is to be
studied among what populations with what research methods for what purpose.”
Gay and Airasian (2000) define research design as “a general strategy for conducting a
research study.”
Vogt (1999) defines research design as “the science (and art) of planning procedures for
conducting studies.”
For Mohn (1996), a research design is the basis for causal conclusions
In other words, a research design is a set of instructions for data collection and analysis. If
the introduction in a research paper raises the question to be investigated, the research
design contains directions to determine the answer. The research design stipulates the parts
of the research project, how they are arranged, and how they function. However, the
research design does not determine the type of data, how the data are collected, or how they
are analyzed.
How many research designs are there?
Correlation is a statistical procedure -an example of data analysis.
1. Experimental
2. Survey
3. Ethnographic
4. Correlational*
5. Case study
6. Action research
7. Qualitative
8. Ex post facto
9. Descriptive
10. Introspection
11. Grounded theory
12. Narrative
13. Historical
14. Evaluation
15. Causal - comparative
16. Interactional analysis
17. Phenomenology
18. Critical theory
19. Constructivism
20. Content analysis
21. Ethnomethodology
22. Feminist research design
23. Hermeneutics
How to select a research design
1. To select one that addresses your purpose and answers your research questions.
2. You can use the design you already know. We tend to use what we know, and if we
only know one design, then that is the one we will select.
3. To use the design promoted by a senior colleague. This political decision has the
advantage that the professor under or with whom you are working can give cogent
advice on the research because of familiarity with the design.
4. It might be that your academic specialty has a preference toward doing a research
using a certain design or want to publish in a certain journal and it seems to publish
a certain type of research using a certain type of design. Therefore, you might use
this design to gain admission to your area of specialization or, to increase your
chances of publication.
5. You might be intrigued by a certain design and want to learn how it works.
The only rationale that the author would advise against is picking a design thinking it to
be easy. There is no such thing as an easy design.
Mixed-method design
Method could refer to a way of organizing research (in which case the term design is
preferable),
Method could refer to a way of collecting data (data collection instrument is better in
this case),
Method could refer to a way of analyzing data,
Method is frequently used in close proximity to other terms such as paradigm,
quantitative and qualitative.
The term mixed methods means the use of more than one kind of data (Barkaoui,
2010).
Design Measurement Analysis
1.Experimental Quantitative Statistical
2.Naturalistic Qualitative Content
3.Experimental Qualitative Content
4.Experimental Qualitative Statistical
5.Naturalistic Qualitative Statistical
6.Naturalistic Quantitative Statistical
Table 2. Examples of mixed methods as a result of triangulation based on Patton (1990)
Design #1 of the table is the classical experimental design, or what Patton calls the pure
deductive approach.
Design #2, naturalistic inquiry, uses qualitative data and content analysis. This is pure
(or at least typical) inductive approach, which Patton terms qualitative strategy.
Designs #3, #4, #5, and #6 could be called mixed forms.
Is there such a thing as qualitative research?
There is no such a thing as qualitative research design, or quantitative research design.
Any design can accommodate any type of data. To identify research by the type of data
gathered is not only confusing and unhelpful, it is not possible if multiple types of data
(qualitative and quantitative) are gathered, which is frequently done.
Instead of using the term qualitative or quantitative research, let us come down a level
of abstraction and identify the research design we propose to use.
What conclusions can be drawn about research designs?
Various research designs are inconsistently named. These inconsistencies seem to be
examples of what Pedhazur and Schmelkin (1991) call the “Jingle Jangle fallacy”. In
other words, just because authors call things by the same name does not mean they are
the same, and just because authors call things by different names does not mean they
are different.
We might conclude that if a TREE were to elect to use a popular research design, such
as a variation of experimental design or survey design, for a paper, thesis, or
dissertation, the decision would be met with understanding and probably approval
(Lazaraton, 2005). On the other hand, if a TREE elected a less-popular research design,
such as action research or grounded theory, the decision might be met with less
understanding. The lesson is to carefully consider design selection, remembering that
the decision may be as much political as it is academic.

WEEK 2 June 18th, 2020

Newby (2014) Research Methods for Education p. 96-111 / 128-137
The discussion between the scholars who claimed ‘quantitative’ or ‘qualitative’
approaches to be the best one for educational research approaches has been so
argumentative that has been called ‘paradigm wars’. However, in this text, the intention
while using these terms is just descriptive.
 Quantitative research bases its evidence on numerical data, which is analyzed using
statistical data to draw conclusions.
 Qualitative research implies seeking for the reasons or the processes that drive the
behavior and experiences of life (a concern with feelings and values).
 Mixed methods research aims to fuse quantitative and qualitative approaches
arguing that research issues in education tend to be so complex that the insights of
both approaches are necessary to gain a good understanding.
Have into account that:
- No one approach is absolutely better or worse than another.
- A research approach should arise primarily out of the research question, not out of a
researcher’s conviction.
- Circumstances can lead to a researcher modifying a style to meet particular
constraints or needs.
- Sharing ideas and borrowing techniques reflects a researcher’s skill.
Truth is central to the notion of research. Epistemology, as a philosophical idea, is the
study of knowledge and, by implication, how we know what we know. The ‘truth’ that
results from belief in a faith is very different from what I believe to be true from the
perspective of a social scientist and both are likely to be different from what a physicist
would accept as truth. The standards we apply to determining truth depend on where we
are coming from, what we are using the truth for and, how many people agree with us.
You should not be so focused on your research issue or research paradigm that, when
you analyze your data, you miss what it is really trying to tell you. Make this your
guiding principle: Always look for the gorillas.
(i) The character of quantitative research
The guiding principle of quantitative research is the concept of positivism. It was
developed by the French philosopher, Auguste Compte ‘the father of sociology’, in the
middle of the nineteenth century (Course of Positive Philosophy, 1830-1842), and later
split into ‘logical positivism’ and ‘post-positivism’. At the heart of his thesis was the
belief that truth arose from verifiable facts. He saw positivism as the culmination of
mankind’s ability to understand the world.
Positivism’s legacy has been the responsibility for establishing principles to guide
scientific research. The character of quantitative research can be summarized as the
identification and explanation of pattern and order, with two significant aspects, the
derivation of theory and the nature of proof.
(a) A concern with theory
The belief of the Quantitative research in an observable and measurable reality
places it firmly in the positivist camp. Because the things that quantitative
researchers examine are measurable and because the conditions for data collection
and analysis are specified, quantitative research can be replicated. The
reproducibility of its research results is not only a test of the effectiveness of its
procedures, but it is also a platform for the generalization of research findings. The
ultimate purpose of quantitative research is to generate theory. Truths about
behavior and relationships that are applicable in a range of situations; however, it is
not necessary for every piece of quantitative research to lead to a theory-type
statement, the education world just could not cope with that much theory!.
The development of theory advances step by small step. An idea is modified
according to this circumstance and then another. And then everything has to be
replicated to ensure that there are no spurious claims as so relationships and
predictions.
 Duplication of the research: It is a test of the extent to which individual
conclusions are more generally valid.
The research would again be replicated and duplicated; in this way the research
community would build up a picture of how significant the various potential
influences actually are.
(b) A concern with proof: the role of the hypothesis
The nature of proof is the second issue that makes quantitative research distinctive.
The concept of proof lies at the heart of quantitative research. Prediction here means
just that, if something happens then the result that is predicted will also happen, it
means that there is a defined and stable relationship. For quantitative research there
are ways of convincing others that our findings constitute a proof.
 In science we understand that a theory is proved when there is certainty in a
relationship. While in education, the best that can be said of people is that there is a
likelihood of something happening. In other words, we can use probability to say
something about behavior.
1. We have to specify the factors that might influence demand. When these
factors are combined, we might have a means of predicting demand for
places in any one school.
2. We have to present these factors in a particular way. As they stand, they
effectively constitute a theory of what is likely to happen. We can express
this theory as a proposition, called HYPOTHESIS. We can be really quite
specific about how the causation works to influence parental choice.
Can we prove our hypothesis? In terms of logic, the only truth that we can establish
is whether or not a conclusion is false and not whether a conclusion is true (because
we cannot possibly test every situation for truth). We have to phrase our hypothesis
in a particular way, as an inverse of what we are trying to prove (null hypothesis).
False conclusions are only compatible with premises being false. True conclusions
are compatible with any combination of truth or falsity in the ‘premise’. Testing a
null hypothesis leaves us with the possibility of demonstrating that it may be false
and if the null hypothesis is false, then we have a basis for accepting that there is a
likelihood of the positive hypothesis being true.
(ii) The character of qualitative research
Qualitative research is concerned with understanding how people choose to live their lives,
the meanings they give to their experiences and their feelings about their condition. It can
include approaches such as:
 Ethnography – the processes of observing individuals or groups either as
participants or non-participants and of analyzing and structuring the record.
 Action research – a cyclical research and development procedure that moves
from problem to goal, through action to reflection on the result in relation to
the goal, and then moves forward by revising action or goal or both.
 Case study – an investigation of a single instance, usually with the goal of
identifying and perhaps understanding how an issue arose, how a problem
was resolved, often with the purpose of isolating critical incidents that act as
decision points for change.
They are more usually associated with qualitative strategies because they do not embrace
positive principles.
Qualitative research lacks a clear definition. Despite this, qualitative methodologies are not
weak and research results are significant. Qualitative approaches give guidance but allow
researchers considerable freedom of choice. This does highlight the necessity for researcher
excellence.
Qualitative approaches are often described as being ‘other than’ something else.

QUANTITATIVE APPROACHES QUALITATIVE APPROACHES

-Having hard edge and being concerned -They are soft, descriptive and
with process outcomes, explanation, concerned with how and why things
generalization and the derivation of happen as they do.
laws.
-They are held to stem a humanistic
-They are objective, experimental and, tradition. They dray on insight and
value the empirical observation of interpretation and allow researchers to
cause and effect. draw on their subjective responses to
evidence.
-Numerical evidence is the basis for
drawing conclusions. -The emphasiss is on wholeness and
detailed connections between our social
worlds, emotional and cognitive
processes and economic circumstances,
all which have to be understood in
order to recreate the meaning that we
give to our lives.

(a) A holistic and integrative approach

Qualitative approaches have the willingness to use data of different types and from
different sources and combine them into an analysis and interpretation of a situation,
this is a true strength of the method. Qualitative researchers should be confident in
their response for two reasons.

1. Individual methodologies, (for example grounded theory) have

established their own procedures that allow other researchers to see
clearly what a researcher’s decisions have been.
2. Probabilities do not tell us what we have to do to change this situation. It
is necessary to understand the people. Understanding people does mean
that you have to deal with people’s feelings, values and emotions as well
as their behaviors, their attachments to place and people, their fears,
hopes and motivations as well as their perceptions of the world, the
organizations with which they have contact and their relationships with
them. It requires sorting, evaluating, juxtaposing, contrasting and
rejecting before accepting. The insights that create the integration that
lead to understanding are made by researchers themselves.
(b) A naturalistic form of enquiry
It means obtaining data in as natural a setting as possible, to minimize the influence
of an unrealistic research environment. However, it is not always possible to sustain
a pure naturalistic approach; certain research methods require a degree of separation
of subjects from their natural world, other methods impose an artificial situation on
respondents.
(c) Not one reality
Qualitative researchers often assert that they are not concerned with ‘one reality’.
To understand what they mean by this we have to go back again to quantitative
approaches and the assumption that there is a single truth to be discovered, the
implication of this is that there is a ‘real world’ that has an independent, verifiable
existence. But does all of this apply in relation to people? Is there only one
explanation, only one set of relationships to be explained?
Example: In England and Wales, the original model was that a school inspection
would normally occur every five or six years, would be of a week’s duration and
would involve a team of between five and twelve inspectors; from the school’s point
of view the inspection disrupted school life for at least a quarter of the academic
year; the inspections were a clear example of the research interfering with the
process being researched. Sweden, however, provides an interesting contrast, the
tone of the visit is not an inquisition but a dialogue; the whole process is based on
the assembly of data from different sources and of different types, it is a typical
qualitative research approach. “while the qualitative methods may have certain
inherent weaknesses, it is the personal visits ‘in reality’ which make the material
unique and interesting”. This is a more mature and more accurate and effective way
of assessing the quality of education than was the case in the UK.
There is no single rationality that determines the way people behave. The task of the
qualitative researcher is not to look at how people behave as an outsider but to
understand how individuals see the world.
(d) The place of theory
 Quantitative approaches are deductive: from the evidence we draw logical
conclusions as to relationships. The process of quantitative research presents them
first as theories and having tested them in a variety of situations, validates them as
laws.
The qualitative approach is conventionally held to be inductive. In this case the
evidence is brought together, reviewed, and patterns and processes identified that
lead to the specification of theory. No social scientist gets beyond theory. Grounded
theory is an established methodology for generating theory. However, qualitative
research is not always inductive. It is possible for qualitative research to explore
theoretical statements and even to test hypotheses, though not in the same way that
we would do with quantitative procedures.
WE OUGHT TO BE PREPARED TO CHALLENGE ORTHODOXY AND
CONVENTION IF OUR RESEARCH PROBLEM REQUIRES IT.

The quantitative test specifies the nature of the difference in the research hypothesis
and the hypothesis that is tested is the null hypothesis. A quantitative approach
would feed the data into a statistical procedure in order to determine proof.
The qualitative hypothesis explores not only whether the levels of motivation are
different but also why they are different. A qualitative research project would seek
to see if these issues played a part differentially between ethnic groups but, if not,
the cultural dimension rather than the level of motivation could be the prime
influence on attainment and this would have to be explored further. In a qualitative
approach all we have are the judgement and the argument of the researcher. Proof is
whether our professional colleagues or tutors accept our conclusions.
(iii) The character of mixed methods research
Mixed methods research brings together quantitative and qualitative research approaches. It
could include, for instance, using survey and experimental approaches together or an
ethnographic approach with a case study.
It has been controversial to mix quantitative and qualitative approaches in a single research
design. In education the debate appears to have been particularly loud. For some
researchers on either side of the quan/qual divide, the incorporation of a procedure from
‘the other side’ is tantamount to supping with the devil. A number of reasons can be
identified.
 Some researchers so identify with an approach that any transgression against its
principles is seen as an attack.
 Some subjects have a research tradition that is dominated by one approach to
research.
 Some researchers regard epistemology not as a description of how research is
conducted but as a rule book that specifies how all research should be conducted.
 Some researchers follow the implicit or even explicit guidance of those who fund
research.
 Some traditions are given more weight when students are taught about research
methods. For professor Stephen Gorard and Dr. Chris Taylor, both British
educational researchers, the small number of quantitative studies in education was a
consequence of this over-emphasis on qualitative research approaches in research
methods courses, as well as of the attitude of students to numeracy and their
capability in mathematics. Students are taught more about qualitative approaches
than about quantitative and are less able or inclined to use quantitative.
 Quantitative designs fit more easily than others into cost-benefit analysis; and this is
just the sort of approach to decision making that governments use.
(a) Mixed methods research: is it a new approach?
Burke Johnson and Anthony Onwegbuzie (2004) referred to it as ‘a paradigm whose
time has come’. Gorard and Taylor (2004) have called it ‘a third methodological
movement’.
Combining methods was relatively unproblematic until the 1980s and 1990s. As the
numbers of academics and the scale of academic work grew, an intellectual iron
curtain grew up behind which the two research traditions defined and refined their
operating principles in such a way as to promote exclusion. Those who brought
methods together in order to understand an issue had to argue their case. The
appropriate basis on which to test their case was the construction of a third way with
its own distinctive epistemology.
It should have rules of procedure, though it is fair to say that those for qualitative
research are less prescriptive than for quantitative and apply at the level of
methodology rather than epistemology, while quantitative analysis rests on the
principle of randomness. Some order is emerging in the ways in which methods can
be combined, though these are either conceptual typologies or operational guidance
based on what seems to work. The need to be flexible in the way we use research
methods to resolve specific research issues is now much more widely accepted.
 (b) The argument that mixed methods are fundamentally positivist
Lynne Giddings calls mixed methods research ‘positivist dressed up in drag’, she is
critical of its being positioned as a new research style because of its continuing
reference to its parent research traditions and because quantitative principles must
strongly influence the research design.
(c) The argument that mixed methods are not a new approach
The absence of an accepted conceptual infrastructure for mixed methods certainly
supports the argument against the approach being a new tradition, as the world
evolves, frameworks that do not change will gradually lose touch with what they
seek to manage. Innovation is a necessity in a changing environment, in research as
in anything else. In research, as in life, the past is another country.

Qualitative approaches
(a) A framework of practices
We are faced with a myriad of approaches and a willingness amongst researchers to mix
and match to meet particular needs. Qualitative researchers do not limit themselves to
numbers in order to create understanding and find answers. Qualitative research seeks
understanding from any evidence that reflects our motives, our values, our attitudes, the
bonds that tie us, in other words the deep personal, social and cultural drivers of behavior,
because understanding behavior is often the route to solving a problem.
To see qualitative researchers just have to accept what people through their natural
behavior provide; they can stimulate a response by asking people questions, or by creating
situations in which behavior can be observed, or by asking people to express themselves
through a particular medium. Whether we stimulate a response or not, the processes for
collecting data are the same (watching, asking, listening, reading and perhaps even tasting
and smelling. Qualitative researchers are prepared to take data from any source and
evaluate it as evidence. It is this preparedness to break conventions that gives
qualitative research its identity.
(b) What can go wrong?
The criteria for judging a quantitative approach or method are validity, reliability,
and objectivity.
 Validity means that the processes of collecting data accurately reflect the
aspects that they are meant to measure
 Reliability means that the outcomes of measurement are stable overtime
always assuming that other things remain the same
 Objectivity means that the researchers judgments are dispassionate and by
implication that another disinterested researcher would reach the same
conclusion when faced with the same evidence.
For qualitative research the criteria is composed by credibility, dependability and
confirmability.
 Credibility requires that the subjects who provided the data believe that the
interpretation is credible from their own perspective.
 In the case of dependability the only real test for it is whether the researcher
explains the context for the research sufficiently for the audience to agree
with their conclusions which appears to be just another route to ensuring that
the research and conclusions are reliable.
 Confirmability is the assumption that others would reach the same
conclusion.
we have to believe ultimately in the honesty and integrity of the researcher. With
sufficient information on how data were collected and analyzed, we can make this
judgment and understand the personal framework within which the researcher was
objective.
Reliability and validity
We have to be sure that what we gather in represents the situation that we intended
to examine and that, if another researcher were to investigate using our approach,
the results would be the same.
One particular issue for qualitative researchers to be aware of is the influence of the
relationship between the researcher and the subject. Even if the social dynamics do
not evolve, it is possible that the very act of being investigated may affect the
subject and the outcomes.
The Hawthorne effect
The Hawthorne effect describes a situation in which outcomes contrary to
those expected are caused either by a contrariness or a desire to please or
achieve. What the researchers in the 1920s appeared to find was that as
lighting quality was reduced worker productivity increased when the
opposite was expected; the researchers concluded that the workers felt they
 were important because they were being tested and so responded with
greater efforts.

How can a qualitative researcher demonstrate reliability and validity? The answer is by a
process called triangulation. It seeks to validate a claim, a process or an outcome through at
least two independent sources. How can triangulation be implanted?
 Repeat data collection by another researcher as a test of reliability.
 Obtaining information about the same process from two or more different people
involved with the process.
 Corroborating interview evidence with documentary evidence or vice versa.
While there may be differences in detail or nuance these are acceptable if the thrust from
both sources is the same.
Objectivity and integrity
Objectivity must always be a goal for every researcher, though limits can be placed on it.
We can be partial in relation to, and allow our values to influence, the research we choose
to do but we should strive to be impartial in the way we use evidence to reach conclusions.
If this is done, then the researcher is demonstrating integrity and integrity is the very
foundation of research quality. Lack of integrity corrupts an investigation.
To make full use of the qualitative approach we have to:
 Believe in its value to resolve research issues.
 Understand the implications of using a particular approach.
 Demonstrate how we ensure quality in our research.
 Be open in all we do.
Mixed methods research
(a) Frameworks for combining methods (Tashakkori and Teddlie, 2003)
 Mixed method design, the research approach includes both qualitative and
quantitative data collection and data analysis. They also include the conversion of
data from qualitative to quantitative.
 Mixed model design, the trigger to a mixed model design is the research question,
which should take the research in directions that require both approaches.
 Multi-level design, in which one approach dominates at one level and the other at
another.
(b) What can mixing methods achieve? (Mason, 2006)
1. To give context to or illustrate a bigger picture.
2. To answer questions that are related but which require different research
approaches.
3. To ask questions that intersect at different points in a hierarchy or spatial
relationship. Going from a broad picture to detailed studies.
4. To test the accuracy of evidence. With the flexibility that mixed methods offers
there is an increased potential for new ways of triangulating evidence. The use of
quantitative and qualitative approaches together allows researchers to compare
results in a complementary way. They cannot validate each other with any precision
but they can reinforce each other.
5. To ask contrasting but related questions. Educational issues are frequently multi-
dimensional, and a comprehensive evaluation will often require us to mix and match
methods.
6. To let the issue determine the approach. Where we have an issue perhaps not even a
clear understanding of a question, we can only start and seek to resolve the issue
stage by stage. Situations where exploration or unfolding is appropriate (Petter and
Gallivan, 2004):
1) Refining our research.
2) Looking at things that are different from what was expected or at unusual or
extreme results.
3) Expanding our investigation to expose and assess more issues and factors at
work.
 The implication of an unfolding research process is that an entire plan of action is
not possible. Stage by stage, new decisions are taken to move the project forward in
ways that are informed by results already obtained or questions that arise.
(c) What can go wrong?
Double trouble
The answer is that anything that can go wrong with either pure qualitative or pure
quantitative research can also go wrong when you mix the approaches. By far the best
solution is to collaborate with someone whose expertise complements ours.
Linking research question(s) with research method(s)
The actual specification of the research question requires great care on our part. We must
explore the issue through existing research and theoretical perspectives before we can set
out the research question or questions. The nature of the research question will shape the
character of our research design. We will need a robust link between question and method;
it means that we have to be sure that the method will generate data appropriate to the
question and, with a mixed method approach. Our research question is likely to be a short
paragraph outlining the complexity of the issue to be explored.
Going off-piste
Doing things in the way that the research question wants them done is what a mixed
methods approach should be all about. We should look at guidelines put forward by other
researchers, judging whether you should work within them and, if you do not, then make
clear why you have not and explain how you determined your way forward.
 If there is conflict or the potential for conflict between the research methods that
you choose to generate your data, then those from the dominant method should
generally be preferred.
 Data sets should talk to and inform each other. In some cases, our qualitative data
informs the typology created by our survey.
 It is preferable to have flexible designs that can respond to unexpected findings or
that can accommodate additional data.
Mixed approaches are particularly appropriate for certain types of research:
 Going beyond showing cause and effect to understand how the cause creates the
effect.
 Getting to grips with complex issues involving the interplay of behavior, attitudes,
culture and values or understanding what are called ‘wicked problems’, problems
where attempted solutions merely produce more problems (social exclusion, pupil
attainment and intergenerational reproduction of social values, non-attendance at
school, changing the culture in a university, school or college and drug taking and
alcohol abuse amongst young people.
  Evaluation of project or activity processes and outcomes.
 Long-term longitudinal studies of behavior or performance or attitudes and parallel
studies of changes in the group’s social and cultural context.
 Action research, particularly where the research takes the form of an experiment
involving pre-test, intervention and post-test and where the insights of more
qualitative assessments can add considerable value.
To sum up, mixed methods research are cost-effective ways of dealing with complex
issues. However, they do require us to take great care at the stage of specifying the
research issue. An important member of any research project is a critical friend.

WEEK 3 June 27th, 2020

Dörnyei, Z. (2007). Chapter 7: Mixed methods research: purpose and design.
Research Methods in applied linguistics: Quantitative, qualitative, and mixed
methodologies. Pp. 163-173.
Mixed methods research involves the mixing of quantitative and qualitative research
methods or paradigm characterictics (Johnson and Christensen, 2004)
1) Why do we want to mix methods?
2) Isn’t there a danger that the paradigmatic collision cancels out any potential
benefits?
3) What are the best ways of mixing methods?
4) Why don’t people mix methods more?
I. The purpose of mixed methods research
a) To achieve a fuller understanding of a target phenomenon.
Combining and increasing the number of research strategies used within a
particular project would broaden the scope of the investigation and enrich
the scholars’ ability to draw conclusions about the problem under study.
Four specific functions by which mixed methods research can produce a
fuller picture of an issue (Greene, Caracelli and Graham, 1989)
 Complementarity function: QUAL – QUAN ‘division of labour’
whereby qualitative research is used to explore a new phenomenon
and develop an initial hypothesis, which is then tested in terms of the
breadth of its distribution in the population by a quantitative method.
 Development function: Qualitative and quantitative methods are used
sequentially so that the results of the first method inform the
development of the second.
  Initiation function: Researchers may intentionally utilize varied
methods to generate discrepancies, paradoxes, or contradictions,
which are meant to be provocative through the recasting of questions,
leading hopefully to new perspectives.
 Expansion function: Qualitative methods can be used to explore the
processes of a certain instructional program and quantitative methods
to assess the program outcomes.
b) To verify one set of findings against the other.
This is done through triangulation, defined by Denzin (1978) as “the
generation of multiple perspectives on a phenomenon by using a variety of
data sources, investigators, theories, or research methods with the purpose of
corroborating an overall interpretation”. It is an effective strategy to ensure
research validity.
How we should interpret any divergence in the triangulated findings.
Exploring the conflicting results can lead to enhanced understanding,
especially if it also involves further data collection (initiation function).
c) To reach audiences that would not be sympathetic to one of the approaches if
applied alone.
A well-executed mixed methods study has multiple selling points. One of
them is generating an overall level of trustworthiness for the researcher.
II. The compatibility of different research paradigms
Can different forms of data and knowledge about the world be integrated?
Everything depends on what exactly we are after in our research. The crucial point is that
one’s world view, research methodology, and nature of interpretation should be consistent
in order to produce high quality research results both in qualitative and quantitative ways.
The goal of mixed methods research is not to replace either of these approaches but rather
to draw from the strengths and minimize the weaknesses of both in single research studies
and across studies.
How can we achieve consistency in an approach that is based on mixing up very different
components? The key processes are ‘principled mixing’ and ‘additive mixing’.
The fundamental principle of mixed methods research is that researchers should collect
multiple data using different strategies, approaches, and methods in such a way that the
resulting mixture or combination is likely to result in complementary strengths and non-
overlapping weaknesses.
The mixing process is centered around the purpose of the investigation (the research topic
or question). Today’s research scene is becoming increasingly interdisciplinary and
complex, as a result of the growing desire of social researchers to link micro- and macro-
level analyses.
In the past, social scientists exploring the relationship between the individual and the
surrounding social world have typically adopted one of two perspectives:
 The ‘individualistic perspective’ views the social world through the individual’s
eyes.
 From the ‘societal perspective’ the individual’s behavior is seen to be determined by
the more powerful forces at large.
Mixed methods research seems ideally suited to operationalize the theoretical
transcendence of the micro and macro perspectives at the research level.
“Applied social inquirers appear to ground inquiry decisions primarily in the nature of the
phenomena being investigated and the contexts in which the studies are conducted. Inquiry
decisions are rarely, if ever, consciously rooted in philosophical assumptions or beliefs”.
(Greene & Caracelli, 2003, p.107)
III. Main types of mixed methods designs
While the typologies done by researchers can be beneficial in organizing and labelling
varied practices and thus conveying a sense of rigor, it is not clear how far it is worth
pursuing such a formalized typological approach. The most used combinations are
questionnaires and semi-structured interviews.
Typological organization
The two most widely accepted typological principles have been the sequence and
the dominance of the method constituents.
 Capital letters denote priority or increased weight.
 Lowercase letters denote lower priority or weight.
 A plus sign (+) represents a concurrent collection of data.
 An arrow ( ) represents a sequential collection of data.
If a study has only two components, a qualitative and a quantitative, both the
sequence and the dominance dimensions have three categories (qualitative first,
quantitative first or concurrent; and qualitative dominant, quantitative dominant or
equal status), resulting in nine possible combinations:
1. QUAL + QUAN
2. QUAL + quan
3. QUAN + qual
4. QUAL QUAN
5. QUAN QUAL
6. QUAL quan
7. qual QUAN
 8. QUAN qual
9. quan QUAL
Mixed methods research truly opens up an exciting and almost unlimited potential for
future research.
Exemplar-based typology
Listing design types in this way has a positive and a negative side: on the positive side, it
uses descriptive labels to facilitate understanding. On the negative side, the list is selective
rather than comprehensive; that is, it only includes the most prominent basic combinations.
 Questionnaire survey with follow-up interview or retrospection (QUAN
qual)
In a questionnaire survey the respondents’ engagement tends to be rather shallow
and therefore we cannot explore complex meaning directly with this technique; if
we find some unexpected results (there are always some unexpected results) we
cannot usually interpret those on the basis of the questionnaire data. Even if an
observed relationship makes sense, the questionnaire data usually reveals little about
the exact nature of the relationship. Adding a subsequent qualitative component
(follow-up interview) to the study can remedy this weakness.
Creswell et al. (2003) labelled this combination a ‘sequential explanatory design’, it
can benefit almost every quantitative study. Conducting a ‘retrospective interview’,
using the respondents’ own survey responses as the retrospective prompts for
further open-ended reflection about what they really meant.
 Questionnaire survey facilitated by preceding interview (qual QUAN)
A frequently recommended procedure for designing a new questionnaire involves
conducting a small-scale exploratory qualitative study first to provide background
information on the context, to identify or arrow down the focus of the possible
variables and to act as a valuable source of ideas for preparing the item pool for the
purpose of questionnaire scale construction.
 Interview study with follow-up questionnaire survey (QUAL quan)

One of the strengths of qualitative research is its exploratory nature, allowing us to

gain new insights and formulate new theories. Qualitative data cannot inform us
about how widely what is discovered exists in the rest of the world. Combining a
qualitative interview study with a follow-up survey can offer the best of both
worlds, as the questionnaire can specifically target the issues uncovered in the first
phase of the research and investigate the generalizability of the new hypotheses in
wider populations.
 Interview study facilitated by preceding questionnaire survey (quan
QUAL)

Qualitative vulnerability is the usually small sample sizes of the respondents

examined. One way of dealing with this issue is to apply purposive sampling and
this procedure can be made more principled if we include an initial questionnaire in
the study whose role is to help to select the participants for the subsequent
qualitative phase systematically. The design can help to establish the
representativeness of the cases presented. It does not work if the initial
questionnaire is anonymous because then we cannot identify the appropriate survey
participants.

 Concurrent combinations of qualitative and quantitative research (QUAL/qual

+ QUAN/quan)

‘Concurrent designs’, in which we use two methods in a separate and parallel

manner and the results are integrated in the interpretation phase. The main purpose
of this design is to broaden the research perspective and thus provide a general
picture or to test how the different findings complement or corroborate each other.
QUAL + quan is frequent in case studies; QUAN + qual to describe an aspect of a
quantitative study that cannot be quantified or to embed a qualitative component
within a larger, primarily quantitative study such as a program evaluation; and
QUAL + QUAN in a traditional triangulation design conducted for validation
purposes.

Concurrent designs are invaluable when we examine a phenomenon that has several
levels. They are also useful for combining micro and macro perspectives.

 Experiments with parallel interviews (QUAN + qual)

We can sometimes improve experiments even further by conducting interviews to
get at the research participants’ perspectives and meanings that lie behind the
experimental research findings.
 Longitudinal study with mixed methods components (QUAN + QUAL)

Longitudinal studies lend themselves to various combinations of qualitative and

quantitative methods at different stages of the project.

 Combining self-report and observational data (QUAL + QUAN)

 Two fundamental ways of gaining information about people are: through self-
report, that is, through the individuals’ own accounts and through external
observation of the individual.
IV. Why don’t people mix methods more?
There are two main factors at the heart of it: a lack of sufficient knowledge about method
mixing and a lack of expertise to implement a mixed design. Mixed methods research has a
severe ‘shortcoming’: it requires the competent handling of both qualitative and
quantitative research.
Most researchers have a natural inclination towards either qualitative or quantitative
research, which is most probably related to differences in their cognitive styles and certain
personality traits.
The real potential for the implementation of mixed methods research lies in working in
teams that consist of members with different research orientations.
Greene et al. (2005) provide a detailed description of how such teamwork was initiated and
structured as part of the ‘Harvard Family Research Project’; as the authors conclude, by
mixing methods they arrived at a unique, complex, and nuanced understanding of their
research topic.
The publication pressures of the current research climate promote piecemeal publications in
general and therefore even when a project involves methods mixing, the authors might
choose to try and publish the results of the different phases separately.

QUESTIONS:
 How would conclude a mixed method study with an expansion function?