Tips & Tools #17: Analyzing Qualitative Data

Introduction

Qualitative evaluation methods yield narrative data—often describing experiences, perceptions,

or opinions from key informant interviews, open ended survey questions, focus group
interviews, etc.—that are less easily summarized in numerical form [See Tips & Tools #5 for
further discussion on Qualitative Methods.] This guide outlines the common methods and basic
approach for analyzing qualitative data in the context of tobacco control.

Sources of Qualitative Data

The narrative data that will need to be analyzed can come from a variety of sources. In
essence, these are the ways how the data may be collected.

• Key informant interviews may generate data in the form of notes, a summary of an
interview, or verbatim transcripts. Example: Interviewing city officials about their
position on an outdoor smoke-free policy.

• Focus groups involve text and notes or full transcripts from a moderator or observer.
Example: A guided roundtable discussion facilitated with Native Americans about the
best way to pursue a smoke-free casino policy.

• Media activity/policy record reviews might be used to examine published written

material or notes of city council meetings. Example: Review of local media to take the
pulse of community sentiment on local government regulation to see if the time is right
to pursue a TRL policy.

• Observations can be recorded as field notes or descriptive accounts of watching and

listening at a particular site. Example: Notes and narratives of who is using parks, when
and locations where tobacco litter is most prevalent.

• Open-ended questions can produce text such as brief feedback or full ideas in the
form of paragraphs from questionnaires. Example: Questions on survey to capture
“other” responses.

Be Systematic—The Analysis Process

It is important to be systematic in your approach to analyze qualitative data. The elements

below describe the basic elements in conducting a qualitative data analysis. It is important to
note that this process is fluid, so you need to think about them (and move back and forth
between them) before, during and after your data has been collected. Essentially, during the
entire process—from beginning to end—you want to think about and strategize your end
product in order to better inform your project and project’s next steps. The steps are as
follows:

• Step 1: Get to know your data

• Step 2: Focus the analysis

• Step 3: Categorize information

• Step 4: Identify patterns, themes, connections

• Step 5: Interpret meaning and present results

Get to Know Your Data

A strong analysis depends on knowing your data, inside and out. The first step is to read
through or listen to all of your data just to get a sense of what you’ve got. Mark up the hard
copy – highlight, underline, and note any impressions as you go through the data (have an
analysis “note” sheet where you can jot down notes and ideas). It is really useful to have this
record so you don’t forget later on, but also to refer to as your impressions of the data change.

After your initial readthrough, it is good to revisit your research questions to re-acquaint
yourself with what you wanted the data to tell you (remember, you will constantly be thinking
about the purpose of your project). With those questions in mind, you’ll want to read/listen
again to the data several more times – to kind of saturate yourself until it begins to percolate in
your mind and make connections almost unconsciously.

Finally, think about whether your data has anything worthwhile to say. Decide how much time
for analysis your data merits. If it seems rather weak, then you probably won’t want to spend a
great deal of time on it. Instead look for ways to supplement your data with other sources.

• Listen to recordings repeatedly

• Read, read and reread through your data

• Write down your initial impressions

• Consider data quality

Focus the Analysis

With a general sense of the extent and quality of your data, return to your original purpose in
collecting these data. What did your project want to find out? Identify a few key questions
that your analysis should answer. Write these down. These will help you decide how to
progress and keep you from drifting off course. These questions may change as you work with
the data. Therefore, it’s important to define them first to serve as the anchor for your analysis
process.

Once you have prioritized your research questions, you are ready to make your first analysis
decision. Does it make more sense to analyze the data by question or topic or by timeframe,
event, group or individual? There are different ways you can do it. Some people will organize
data by group. So if you conducted six focus groups, a file for all the data from group one and
another file for group two, and so on, can be created. Or, you can organize data by question or
topic, so you’d create a file for all of the responses to the first question asked in the focus group
and paste in the responses from all of the groups to that file. You can decide what format
supports your purpose best, but our recommendation is to keep it simple and use your
interview protocol to create a separate file or page for each question and paste in relevant data
from all respondents or sources. From there, it’s easy look across all the answers in order to
identify consistencies and differences. Later, you may explore the connections and
relationships between questions (topics, time periods, events) or look at data by groups after
separating data by individual questions.

Finally, in focusing the analysis, we would recommend that you start with the big picture.
Essentially, in focusing your analysis, you need to think about end-use (the big picture!). What
do you want to be able to say or show with your data? It also helps to think about how you
plan to use the results. This will help you refine the focus of the analysis.

• Review the purpose of the data collection

• What did you want to find out?

• Identify key questions analysis should answer

• Decide on analysis approach

Categorize Information

The step of categorizing information is often referred to as “coding the data.” If you’ve ever
done a full qualitative analysis, you know that the process of categorizing data is extremely
labor intensive, but it is the foundation of your analysis, so it’s got to be done right. Coding
involves reading and re-reading the text (data) and identifying coherent categories. The first
part of this process is to:

 Identify themes or patterns—ideas, concepts, behaviors, interactions, etc.

 Organize them into coherent categories that summarizes the data

As you read each segment of text, you are underlining, highlighting things that jump out at you
as important. Also think about how you would characterize or categorize the sentence or
paragraph and jot the code in the margin. This will help organize the data into categories.
Some people use colored highlighters to code portions of text. Others flag sections with post
its. Still others cut text into strips of paper and move them around to see where they best fit.
There is also the qualitative software like NVivo or ATLAS ti. Everyone has their own way of
categorizing data, and looking for patterns and meaning.

As you categorize the data, you might identify other themes that serve as subcategories.
Continue to categorize until you have identified and labeled all relevant themes. [See Tips and
Tools #18—“Coding Qualitative Data” for more information on the coding process.]

• Codes assign meaning

• Codes are used to retrieve and organize the data

• Collapse codes into themes

The Analysis Process—Writing Analytic Memos

Writing analytic memos is really Step 3.5 because it is an important part of the coding process,
but it almost puts us in Step 4—identifying patterns and connections within and between
categories.

As you read and code data, it is critical to stop from time-to-time and write analytical memos,
which is just research jargon for “notes to self.” These are your running commentary and
analyses of what you’re seeing in the data. Memos trace the steps and decisions you’ve taken
so far, lump similar codes into larger themes, note any patterns you are beginning to see, and
capture hunches that you have about what something means. Everyone does this differently,
but we like to think of it as stream of consciousness writing where you simply get all your
thoughts on paper regarding what the data mean, any connections you’ve been seeing, and so
on.

If you’re analyzing key informant interviews, you may want to assign yourself analytical memos
for every handful of interviews. So, for instance, if you have 20 interviews, you may want to
write an analytical memo after every four interviews are coded. Essentially, these are mini-
analyses that will help you tremendously when you sit down to write up your findings. [See
Tips and Tools #20—“Writing Analytic Memos” for more information on this important step.]

• Trace the steps you’ve taken

• Tie codes into larger themes

• Describe emerging patterns

• Record hunches about what the data may mean

Identify Patterns

With a qualitative analysis—in contrast to a quantitative one—you’re making connections as you

go. You’re coding your data, looking for themes, patterns and relationships, and writing analytic
memos along the way. However, once you’re done coding the data, you can start the
important step of identifying patterns and connections within and between categories.

As you organize your data into categories, you will begin to see patterns and connections, and
it’s good to think about different ways to link the patterns and connections to really understand
what your data are saying. Some ways that this can be done are by making distinctions within
categories (subcategories), creating larger supercategories (between two or more major
themes), measuring the relative importance of responses by tabulating them, and finding
relationships (correlations) between themes. For all these different ways of making
connections, it is important to look for examples of responses that can be compared and also
run counter to the prevailing themes. What do the contrasting ideas suggest? Are they
important to the analysis? Often, you learn a great deal from looking and trying to understand
items and themes that do not necessarily fit into your categorized patterns. This is fine, and
actually, a valuable part of doing good qualitative analysis.

• Make connections…

 Within category description

 Creating larger categories

 Tabulating relative importance

 Finding relationships

 Contrasting ideas that emerge

Interpret the Data—“Telling Your Story”

You have identified the themes and patterns in your data. Next, use your analysis of the
themes and patterns to explain your findings. In doing so, it is important to recognize that
interpretation is more than mere description. Think about the significance of your findings.

It can be easy to get side tracked by the details and rich descriptions of the data. But what
does it all mean? What is really important? Revisit your research questions one last time to
make sure you answer the questions you set out to investigate.

How do you go about attaching meaning and significance to the analysis? A good place to start
is to summarize the main points of the data based on your coding and analytic memos. Think
about what you have learned. What are the major lessons? What new things emerged? How
might this apply to other settings? Be sure to bring in multiple perspectives for each theme;
don’t just include those that support your view. Use metaphors and analogies to help you
interpret the data.

It is also important to incorporate representative quotes to illustrate your points and bring the
data to life. Provide rich details that will offer context to what you are presenting. Sometimes
it is important to explain who made the comment and in what context.
In terms of your final write up, it is recommended that you start with an outline or a diagram of
your findings and use it as your guidepost to your writing. This will help you organize
everything. Creating an outline may also reveal gaps in your investigation and connections that
remain unclear.

• Summarize

• Include multiple perspectives

• Use metaphors and analogies

• Insert quotes (raw data)

• Provide details for context

• Address tensions and contradictions

Data Management

Because analyzing qualitative data is about organizing and keeping track of the text, it is
important to appropriately manage your data [See Tips and Tools #19 “Managing Qualitative
Data” for a full listing of helpful hints in this area.]

Much of the data management will depend on your personal preference and the amount and
type of data you have. With small amounts of narrative data, you may work directly from the
original hard copy. However, you will usually need to create distinct Word or Excel files based
on answers to questions, text from certain groups, or answers that fall into different themes.
For this process, you may wish to use a special qualitative data analysis program like NVivo or
ATLAS ti, depending on the size of your data and the level of analysis needed. Below is a listing
of things you can do to help manage your data:

• Check your data

• Add ID numbers

• Prepare data for analysis

• Make copies

• Identify sources of all data

• Create data files based on categories and themes

Pitfalls to Avoid

We wanted you to keep in minds some of the following cautions. For one, avoid over-
generalizing. The goal of qualitative work is not to generalize across a population. Rather, a
qualitative approach seeks to provide understanding from the respondent’s perspective.
Remember to think about multiple perspectives and the tensions and contradictions from the
various respondents. This differentiates the analysis from a quantitative one.

In terms of quotes, choose them carefully. You do not want to only use quotes that support
one argument or illustrate success. This can lead to using people’s words out of context. Think
about the purpose of the quote—do you want to show the differences in people comments, give
examples of a typical response to a certain topic, or something else? You want to specify why
you chose the selected quotes.

Additionally, don’t fall into the trap of quantifying qualitative data by simply reporting the
number of people who provided a certain answer or response to an open ended question. This
falsely gives the impression of frequencies to samples that are not meant to be representative.
If you want to provide frequency, think of it as supplementary to the real story that your
qualitative data offers.

Confidentiality and anonymity are also concerns when using quotes and writing up results.
Even if you do not give the person’s identity, others may be able to tell who made the remarks.
For instance, if we quote the “Project Director for Yolo County” it would not be difficult for
readers to find the information. Be careful: even seemingly innocuous quotes can be
misconstrued and we would therefore recommend making sure all quotes are completely
confidential and anonymous in how they are reported (and the original data be kept in
safekeeping).

Finally, every report or listing of findings has limitations. Presenting the problems or limitations
you had while collecting and analyzing the data helps others better understand how you arrived
at your conclusions. And, recognize that you will incur problems at part of the process of
analyzing data. It’s simply part and parcel with doing this type of analysis.

• Don’t over-generalize

• Choose quotes carefully

• Avoid quantifying

• Maintain confidentiality and anonymity

• Address limitations and alternatives

Conclusion:

As we noted, qualitative analysis can be labor intensive, but it can also be a rich and
enlightening experience. If you think of it like a jigsaw puzzle and follow a systematic
approach, an evocative and insightful story is bound to emerge. And, like many things in life,
the more you practice, the easier and more rewarding it will become. Good luck in your analytic
journey!
References:

Barbour, Rosaline. 2007. Doing Focus Groups.

Gibbs, Graham. 2007. Analyzing Qualitative Data.

Krueger, Richard A. 2009. Focus Groups: A Practical Guide for Applied Research.

Miles, Matthew B., and Huberman, Michael A. 1994. Qualitative Data Analysis.

Rapley, Tim. 2007. Doing Conversation, Discourse and Document Analysis.

Taylor-Powell, Ellen and Marcus Renner. 2003. “Analyzing Qualitative Data.” Program
Development and Evaluation, University of Wisconsin-Extension.