A GUIDE TO QUALITATIVE RESEARCH –

WHY, WHEN, WHAT AND HOW?

February 2019
 i

INTRODUCTION
Who is this Note for?
This note is for DFAT staff and commissioners who are responsible for planning or procuring assessments,
evaluations or research.
What are the objectives of this Note?

• To increase understanding of the value (attributes, limits and quality standards) of qualitative
research and when to commission it.
• To highlight a selection of different qualitative research methods and tools, their purpose, and the
strengths and weaknesses of these.
• To assist commissioners in assessing whether a qualitative research approach is rigorous and ethical.

SECTION
SECTION1:1: WHAT IS THE VALUE OF QUALITATIVE RESEARCH?

Section 1.1: What are the are the main strengths & advantages of qualitative research?

Qualitative research uses open-ended questions and probing, which gives participants the opportunity to
respond in their own words, rather than forcing them to choose from fixed responses, as quantitative
methods do. Open-ended questions have the ability to evoke responses that are:

- meaningful and culturally salient to the participant

- unanticipated by the researcher
- rich and explanatory in nature

Another advantage of qualitative methods is that they allow the researcher the flexibility to probe initial
participant responses – to ask why or how. This makes qualitative research especially effective in obtaining
culturally specific information about the values, opinions, behaviours, relationships, and social contexts of
particular populations. Qualitative methods are also effective in identifying intangible factors, such as social
norms, power, status, gender roles, ethnicity, and religion, whose role in development outcomes may not be
readily apparent. Without understanding these issues, it is very difficult to design policy and program
interventions that provide good Value for Money

When used along with quantitative research (called a ‘Q-square’ approach), qualitative research can help us
to interpret and better understand the complex reality of a given situation and the implications of
quantitative data. While Q-squared approaches are generally superior to using one or the other approach in
isolation, where some information on an issue already exists (or time and resources are constrained), it may
be prudent to commission only one type of study.

i
 ii

Section 1:2 What are the main difference between quantitative and qualitative research?

The key difference between quantitative and qualitative methods is their flexibility. As we’ve seen,
qualitative methods are flexible, allowing greater spontaneity and adaptation of the interaction between the
researcher and the study participant. Participants have the opportunity to respond more elaborately and in
detail than is typically the case with quantitative methods. In turn, researchers have the opportunity to
respond immediately to what participants say by tailoring subsequent questions to information the
participant has provided.

Generally, quantitative methods are inflexible. With quantitative methods such as surveys and
questionnaires, for example, researchers ask all participants identical questions in the same order. The
response categories from which participants may choose are “closed-ended” or fixed. The advantage of this
inflexibility is that it allows for meaningful comparison of responses across participants and study sites.
However, it requires a thorough understanding of the important questions to ask, the best way to ask them,
and the range of possible responses.

While quantitative research is particularly good at answering questions regarding “what” phenomena exist
and “to what extent” they exist, qualitative research is essential for understanding “why” phenomena exist,
and “how” they work, (including why and how they change or are resistant to change). Table 1, below, gives
examples of the kind of information and understanding that the two approaches – qualitative and
quantitative - can provide.

TIP: Why is qualitative data critical for program and policy design? Without it, you might make some wrong
assumptions about WHY or HOW things occur/don’t occur that lead you to design a program that does not
address problems, or capitalize on opportunities. Worse, you might actually design a program that does harm.
This is a serious risk.

Table 1: Quantitative and qualitative research snapshot

Qualitative Quantitative

General overview • Seek to confirm hypotheses about • Seek to explore social phenomena
phenomena Seek to explore • Instruments use more flexible,
phenomena iterative style of eliciting and
• Instruments use more rigid style of categorizing responses to
eliciting and categorizing responses questions
to questions • Use semi-structured methods
• Use highly structured methods such such as in-depth
as questionnaires, surveys, and • interviews, focus groups, and
structured observation participant observation

ii
 iii

• To quantify variation • To describe variation

Analytical objectives
• To predict causal relationships • To describe and explain
• To describe characteristics of a relationships
population • To describe individual
experiences
• To describe group norms

Examples of information • Who is poor in absolute terms • Understanding why certain

generated • Understanding relative poverty – groups are poor and vulnerable –
how income, consumption, assets the underlying structures and
and opportunities are distributed processes that lead to poverty.
across a population • Understanding how people cope
• Understanding the response to risks, with shocks, risks, and life course
shocks and life course events events and why
• Understanding the prevalence of • Understanding why people have
different attitudes, norms and the attitudes, norms and
behaviours behaviours they do, what holds
these in place and how they can
shift.
• Understanding how power is
exercised within and between
different groups.
• Understanding the politics of why
some people remain poor and
others move out of poverty

We discuss these distinctions further below, under “When should you commission qualitative research?”

Section 1:3 When should you commission qualitative research?

So, you have a gap in your knowledge that needs filling by information…should you commission a qualitative
or quantitative exercise? Figure 1, below, suggests a simple a decision making tree regarding what type of
study to commission. A narrative explanation is provided below the Figure.

Figure 1 - Decision making tree for commissioning research

iii
 iv

If there is very little available data on the phenomenon you want to explore, you would probably start with
qualitative research (1). This will allow you to better describe and explore the phenomenon, enabling you to
define the questions you want to ask in a more focused piece of research.

If sufficient qualitative research exists to understand and explain the phenomenon, but you do not know the
extent to which this phenomenon exists and for whom, whether there is causality, or how different
phenomena are related to each other (e.g. if there is causality), then you will want to proceed with
quantitative research (2).

If quantitative research exists, but there is insufficient data to enable you to understand and explain
variation, relationships, individual experiences, or group norms then you need to commission qualitative
research (3).

SECTION 2: HOW DO YOU DECIDE WHAT TYPE OF

QUALITATIVE RESEARCH TO COMMISSION?

There are a range of different qualitative research methods and approaches. Each method has a different
purpose, and different approaches have different strengths and weaknesses. You don’t need to be an expert,
but it’s helpful to be an informed consumer.

Section 2:1 What different qualitative methods are there, and what is their aim?
Before discussing different approaches or methods for qualitative research and their strengths and
weaknesses, it us useful to have a basic knowledge of the different tools that these approaches employ, and
why they are used. The most common tools are: in-depth interviews, focus group discussions, participant
observation, and participatory methods.

Table 2: Different types of qualitative methods

Qualitative research methods Characteristics

In-depth key informant interviews KII’s are optimal for collecting data on individuals’ personal histories,
perspectives, and experiences, particularly when sensitive topics are being
explored.

In-depth interviews are generally semi-structured, which means that forgo

preconceived questions to instead focus on the dynamic flow of conversation
between researcher and participant(s).

In-depth semi-structured interviews have a number of advantages:

iv
 v

- Participants can answer in as much detail as they want;

- They gather valid information about participants’ views, opinions,
attitudes and experiences, and how people explain and contextualize
these issues;
- Participants are encouraged to be open and honest due to the more
relaxed and conversational atmosphere created;
- The researcher can be flexible, adjusting questions and changing
direction as the interview takes place;
- The researcher is able to probe, explore, challenge, and ask for
clarification.

Key informant interviews are a type of in-depth semi-structured interview used

with participants that have particularly informed perspectives and specialized or
first-hand knowledge of issues.

Focus Group Discussions A focus group discussion gathers together a group of (generally 6 – 8) people
from similar backgrounds or experiences to discuss a specific topic of interest.
The moderator facilitates the discussion using a discussion guide. A good
moderators is able to create an environment where all members of the group are
encouraged to participate in a lively and natural discussion amongst themselves.

A central strength of focus groups discussions is that they allow the participants
to agree or disagree with each other so that it provides an insight into how a
group thinks about an issue, about the range of opinion and ideas, and the
inconsistencies and variation that exists in a particular community in terms of
beliefs and their experiences and practices. As such, the focus group discussion is
a good method to use prior to designing a questionnaire - to ensure that the
questionnaire includes relevant topics and frames questions in a way that will be
understandable to respondents – and after a questionnaire has been
administered - to explore the meanings of survey findings that cannot be
explained statistically, and the reasons behind common or outlier opinions,
views, and experiences.
Participant observation Participant observation is the process of enabling researchers to learn about the
activities of the people under study through observing and participating in those
activities. Its aim is to gain a close and intimate familiarity with a given group of
individuals (such as a religious, occupational, sub cultural group, or a particular
community) and their practices through an intensive involvement with people in
their natural setting, usually over an extended period of time. It is the main till
utilised within ethnographic research.

Conversations about the thematic areas the research wishes to explore are
generally unstructured. The focus is on relaxed, informal and participant led
interactions. Participant observation generally takes place over an extended
period of time, and thus the researcher is able to:

- Include quantitative dimensions (though the method is generally

characterized as qualitative research);
- obtain more detailed and accurate information about the individuals,
community, and/or population under study;
- Collect data on observable details (like daily time allotment) and more
hidden details (like taboo behaviour) that are best interpreted over a
longer period of time;

v
 vi

- Discover discrepancies between what participants say—and often

believe—should happen (the formal system) and what actually does
happen, or between different aspects of the formal system

Ethnographic research is a method of qualitative research where researchers

observe and/or interact with a study’s participants in their real life environment.
While ethnography was popularised by anthropology, it is used across a range of
social science disciplines.

Participatory Methods Participatory tools and exercises have the objective of handing power from the
researcher to research participants. Once the researcher has explained the
exercise, research participants generate their own data through the use of the
tool. This data is often visual (so these tools are good for using with children and
less literate participants), and in group settings the data is generated in an
interactive manner, with group members working and discussing together.
Participants are encouraged by the researcher to analyse and reflect on the
information generated through the exercise, in order to obtain any findings and
insights. Examples of participatory tools include ranking and scoring exercises,
social mapping and body mapping.

Participatory research tools can be used alone - as part of a broader participatory

research exercise - or can be embedded within other qualitative methods. For
example, a researcher might use a ranking exercise within an in-depth interview
or a focus groups discussion, or might use a mapping exercise when conducting
participant observation.

Section 2:2 What are the strengths and weaknesses of different qualitative methods?
There are dozens of different qualitative research approaches. This section looks at 3 different methods for
qualitative research and presents an assessment of their strengths and weaknesses. The three approaches
we assess are: 1. Participatory Ethnographic Evaluation and Research (PEER), 2. SenseMaker, 3. “General”
qualitative research.
We first describe the method, and then rate it using a value for money approach that looks at outputs (how
efficiently do inputs translate into outputs?) and outcome (how effective is the product, i.e. what is the
quality of the research insights, presentation of findings, and advocacy that results from each approach?).
We use a +, ++, +++ system, with +++ being the best VfM and + being the poorest VfM. We are only able to
assess VfM at output and outcome level. To assess overall cost-effectiveness at the impact level – in terms of
the impact on policies and programs on people’s lives – would require research exercises to report on their
ultimate impacts, which could be several years after a study has concluded. Unfortunately, this ‘evaluation of
research’ is almost never done in practice, and is not available for our comparators.
It should be noted that VfM relies not only on the intrinsic approach employed, but also on extrinsic factors,
such as sufficient budget, clear commissioning, well-trained researchers, etc. Thus there can be considerable
variation in VfM between two different research products generated using the same methodology. Further,
we reviewed only a small sample of research studies for each methodology. This assessment, therefore, is
indicative and not representative.

vi
 vii

Figure 2 summarizes the main objective of each method, and how they are generally used. We then describe
each approach in more detail.

Figure 2 - Objectives and uses of comparators

Table 3: Different qualitative Poverty and Social Analysis Approaches

Qualitative Approach Strengths Value for Money Assessment

Participatory Ethnographic The PEER method is based on the ethnographic Peer Vfm Assessment
Evaluation and Research method, but uses members of a community to
(PEER) generate data, as ‘peer’ researchers. It is more • Output +++
rapid and ‘light touch’ than traditional ethnographic • Outcome +++
approaches, which require long field research
phases to allow the (outside) researcher to gain
trust and understanding.

The stated strength of the method is in its ability to

tap into the existing established relationships with
the individuals whom peers interview. Importantly,
the method uses ‘third person’ interviewing
techniques, whereby respondents are asked about
people like themselves in their communities, rather
than about their own personal experiences and
opinions. In doing so, it can yield rich narrative data
to help understand health and risk perceptions and
behaviours from an insider’s point of view,
generate detailed understanding of the context in

vii
 viii

which these behaviours occur, and provide a more

intimate engagement with the realities of
participants’ lives.
While ‘sensemaking’ more broadly is often referred Sensemaker VfM Assessment
Sensemaker
to as an approach to synthesizing and analyzing Ouput +/++
qualitative data (indeed many approaches include a Outcome +/++
‘sensemaking’ stage), here we are referring to the
application of an approach developed with
accompanying software by Cognitive Design, which
draws on complex adaptive systems thinking as well
as cognitive science and anthropology.

The crux of the approach is the idea that

participants provide short stories related to the
topic of interest – ‘micronarratives’ – and in so
doing they reveal a range of diverse perspectives.
These are then interpreted by the participants
themselves using a set of pre-defined questions (or
the ‘signification framework’). The software then
filters and analyses these micro-narratives and
signifiers to identify patterns and trends that may
be of interest, allowing an understanding to emerge
from a large amount of different experiences that
might not be possible using other methods. Visual
patterns and a set of individual stories are then
used in an iterative sense, to understand what is
significant.
General Qualitative Research Beyond these particular approaches, there is a long General Qualitative VfM
methods tradition of qualitative research more generally, Assessment
which can use a range of tools, from in-depth Output ++/+++
interviews to focus group discussions to any Outcome ++/+++
number of participatory exercises or, indeed, a
combination of all these. The research approach
can be flexibly designed to respond to whatever
scale and scope is required, depending on the
research requirements; this makes their strengths
and weaknesses inherently difficult to assess.
Section 3, on research quality, will help
commissioners here.

SECTION 3: HOW DO YOU KNOW IF THE RESEARCH IS GOOD

QUALITY?
Section 3:1 What does “good quality” mean?
As a commissioner, it is your job to assess the quality of the research design, monitor the quality of research
implementation, and plan for good quality communication and dissemination of research findings. There are

viii
 ix

two elements of quality that need to be assessed at each stage of the research process, no matter what
method of research is employed: rigor and ethics.
As a concept, rigor is perhaps best thought of in terms of the quality of the research process; a more
rigorous research process will result in findings that have more integrity, and that are more trustworthy,
valid, plausible and credible. For qualitative research, there are 10 aspects of rigor that we suggest can be
assessed by commissioners (we describe these below in Table 4).
Those who commission, manage, conduct or review research and evaluation, particularly in relation to
poverty reduction, development and social justice, should consider ethical standards for responsible
conduct. Ethical practice in research and evaluation relies on active self-reflection, discretion, judgement and
appreciation of context (ACFID, 2017). We suggest 6 foundational questions that commissioners can ask
researchers concerning ethics (Table 4).

TIP: Both good ethical standards and practices, and attention to rigor in research and evaluation processes,
practices and products are necessary for quality research. Not paying attention to rigor reduces value for money.
Not following principles of ethics introduces and increases risk. Insufficient attention to both can harm evaluation
and research outcomes.

Section 3.2 How do you assess “good quality”?

Table 4 presents a checklist to help commissioners 1) commission rigorous and ethical research, and 2)
assess research quality across the different stages of a research activity.
Table 4: Assessing quality: a rigor and ethics checklist for commissioners

What to As How to Tell

Rigor in Preparation & Design

Does the research take a multi-disciplinary

Researchers from a range of disciplines can help to surface different insights
approach?
and perspectives and challenge biases in analysis and interpretation, all of
which strengthens research design and analysis. Check whether the team is
sufficiently multi-disciplinary.
Are researchers experienced, reflexive, and Ask for CVs of all researchers, including local researchers. Check whether
well-trained? researchers have:

- Previous experience conducting qualitative research;

- Good knowledge of the issues to be researched
- Previous work and/or living experience in the geographies that the
research will take place;
- Deep professional or personal experience with the populations who are
being researched;

ix
 x

- Language proficiency and cultural fluency, which usually – but not

always – suggests the inclusion of strong local researchers on the team.

Ask to see training materials. Check whether:

- The training is being led by an experienced researcher;

- It is of sufficient length (5 days – 2 weeks depending on the experience
of the researchers);
- The content of the training is adequate, i.e. not just a training on the
research tools, but training researchers to interrogate their own and
other team members biases and blind spots; better understand their
perceived (by research participants) role, motives, and power, how
assumptions and biases might influence the research process and
results, and how these can be mitigated

Do researchers have sufficient contextual

Check whether the research proposal includes a sufficiently good understanding
understanding?
of the context (issues, geographies, population to be studied) and/or whether
the approach includes a review of secondary research (and potentially expert
interviews).
Contextual understanding is critical for identifying what is “known” (and how
this is known, in order to check these assumptions) and what is not known, and
in developing contextually appropriate approaches to data collection.

Is there a framework to guide inquiry?

Check whether the research proposal has any conceptual framework to guide
enquiry? This helps to focus the research, while still enabling it to remain
flexible enough for interesting insights and theories to emerge.

Rigour in fieldwork

How will researchers reduce the obtrusiveness Unobtrusiveness can be reduced by: using local researchers, spending longer in
of the research? the field sites (and returning multiple times to field sites to build trust), ensuring
that researchers enter field sites on foot or using local transport, ensuring that
researchers wear appropriate clothing and behave appropriately.

How will data be triangulated? Triangulation is the use of different methods and multiple observers to increase
confidence in the findings. This can be done by collecting data over time
(different times of the day and the year), collecting data from different types of
informant (older, younger, male, female, different ethnic or religious groups,
etc.), and by collecting data using different methods (interviews, observations,
discussions with groups, participating in activities, etc.).

Rigor in fieldwork

How will researchers reduce the obtrusiveness Unobtrusiveness can be reduced by: using local researchers, spending longer in
of the research? the field sites (and returning multiple times to field sites to build trust), ensuring
that researchers enter field sites on foot or using local transport, ensuring that
researchers wear appropriate clothing and behave appropriately

How will data be triangulated? Triangulation is the use of different methods and multiple observers to increase
confidence in the findings. This can be done by collecting data over time

x
 xi

(different times of the day and the year), collecting data from different types of
informant (older, younger, male, female, different ethnic or religious groups,
etc.), and by collecting data using different methods (interviews, observations,
discussions with groups, participating in activities, etc.).

Will respondents be involved in validation? Data should be validated in the field with participants. Does the research
How? include time and suitable methods to encourage local meanings to emerge and
the researcher’s own emerging hypotheses to be validated, and to enable
people’s own analyses to be surfaced through discussion and reflection?

How will the data be recorded? Faithful and accurate recording of data is essential in qualitative research
exercises. While some ethnographic approaches to research involve
considerable participant observation, which does not allow for immediate
recording of data, the research approach to detail plans for recording data as
objectively and comprehensibly as possible, including the use of note taking,
audio, video, photographs, drawings, and different levels of detail in the
transcription of data.

Rigor in analysis and reporting

How will analysis be carried out and by whom? There are several approaches for analysing qualitative data after fieldwork that
ensure that generalizations are supported by adequate evidence – that they are
reliable and dependable – and that enable insights to emerge from the data.
Ask:
- Whether analysis is iterative, and whether it starts in the field (note: it
is important to include field researchers in the analysis process);
- How charting and coding of data is done in analysis;
- How comparative analysis is conducted, and how “deviant” cases are
taken into account;
- Whether there can be use of quasi-statistics (counts of events)
You don’t need to know what these things are in any depth, but you need to
know whether they are being considered carefully by researchers.

How will researchers ensure the transparency Transparency in reporting enables researchers demonstrate the credibility of
of results? findings. Reports should provide readers with a thorough description of the
steps taken in conducting their research.
Documentation of research data and the subsequent steps of synthesis, analysis
and interpretation (including making primary research material available),
should be made freely available for a number of reasons: first, if others want to
replicate the research to see whether they achieve similar results, they can;
second, it enables readers to assess whether the method chosen was the most
appropriate for answering the chosen research question; third, it enables
commissioners and consumers to trace findings, insights, implications and
recommendations back to source data; and fourth, it enables reflection on how
the research process itself was limited, and what the implications of this are.

Ethical considerations

xi
 xii

How will privacy and confidentiality be Research participants should have the right to remain anonymous and to have
assured? their rights to privacy and confidentiality respected. There are a number of ways
to do this that commissioners can ask about:
- How are participants represented in the research? Is this respectful?
Have participants themselves been asked?;
- How are researchers ensuring that there is no link between the data
(responses) and the source (the participant);
- How will data to be stored securely?

How will informed consent be obtained, and All research participants must give informed consent. Ask researchers what
how? official guidelines they are following for this process, and what specific
procedures and considerations will be observed in the case of particular groups
such as children and young people, and people with disabilities.

What additional considerations will be put Work with vulnerable populations, particularly on sensitive issues – requires
understanding of the “risk context” for those researchers to follow proper procedures for obtaining truly informed consent.
participants. By this we are in place for
Working with vulnerable participants also requires researchers to have a sound
vulnerable populations?
understanding of the “risk context” for those participants. By this we are
referring to the risks that participants face in their everyday social worlds, but
also the risks that the research can exacerbate or even introduce. Researchers
should be well equipped to handle a disclosure of abuse (in terms of training
and skills) and should have a reporting or referral plan in place to be able to
respond.
Researchers should be cognizant to ensure that research processes are inclusive
of vulnerable people, for example people with disabilities, and that their specific
participation limitations are addressed Commissioners should carefully
interrogate the policies and procedures in place for working with vulnerable
populations.

How does the research design take context and Research design should reflect the context in which the research will take place;
culture into consideration? this is an issue of ethics, as well as of rigor. Research cannot be assumed to
have beneficial outcomes for host communities or relevant research
participants. What is considered appropriate in one context might not be in
another; research design requires a firm grounding in the relevant local cultural
values, norms and the local historical and political context. Special consideration
should be taken of gender, and the intersection of gender with other facts, such
as age, ethnicity, (dis)ability and religion. Commissioners should ask how the
research considers these things, and also how it is envisaged that the research
will be of benefit to participants.

How is researcher safety being considered? Ethical research also includes ensuring that the research is designed and
implemented in a way that does not unduly compromise researcher safety. At a
minimum, a risk assessment should be done that takes into account not only
risks to researchers in general, but also looks at possible risks to researchers
from different ethnic, linguistic and religious backgrounds, as well as specific
risks to researchers because of their sex or sexual identity. Ask to see this.

What ethical review and other review processes Where research is assessed to be “negligible risk” (where any foreseeable risk is
are anticipated? no more than inconvenience) or ‘low risk’ (where the only foreseeable risk is
discomfort), then a reduced or internal assessment of ethical issues may be

xii
 xiii

considered rather than a formal ethical review and approval. There are a
number of circumstances where a more formal ethical review process should be
considered:
- where evaluation or research processes address sensitive issues or
topics, involve vulnerable groups, or use significant participant time.;
- where less well-established research or evaluation methods are being
used;
- where the aim or purpose of the research goes beyond improving the
implementation of an established intervention or program (quality
assurance), or where research is largely exploratory in nature.
Beyond formal review, increasingly, the quality of outputs are subject to review
by groups of peers, ideally independent (and sometimes anonymous or at least
anonymized) review processes. Good practice suggests that review processes
be set up at research inception, to enable expert inputs into research design
and preparation, as well as to comment on research products. It would be wide
for commissioners to put these internal review processes in place.

xiii