CHANGING

APPETITES

CHANGING
MINDS
Measuring the Impact
of Food, Inc.
Authors:
Johanna Blakley, Grace Huang,
Sheena Nahm, Heesung Shin
 ABOUT
MEDIA IMPACT PROJECT
The Media Impact Project is a hub for collecting, developing and sharing approaches for measuring the
impact of media. Based at the USC Annenberg Norman Lear Center, we seek to better understand the role
that media plays in changing knowledge, attitudes and behavior among individuals and communities,
large and small, around the world. The Media Impact Project brings together a unique team of researchers
including social and behavioral scientists, journalists, analytics experts and other specialists to collaborate
to test and create new ways to measure the impact of media. Content creators, distributors and media
funders can ultimately apply these techniques to improve their work and strengthen engagement. The
Lear Center’s Media Impact Project is funded by a grant from the Bill & Melinda Gates Foundation, with
additional funding from the John S. and James L. Knight Foundation and the Open Society Foundation. For
more information, please visit www.mediaimpactproject.org.

THE NORMAN LEAR CENTER

The Norman Lear Center is a nonpartisan research and public policy center that studies the social, political,
economic and cultural impact of entertainment on the world. The Lear Center translates its findings into
action through testimony, journalism, strategic research and innovative public outreach campaigns. On
campus, from its base in the USC Annenberg School for Communication and Journalism, the Lear Center
builds bridges between schools and disciplines whose faculty study aspects of entertainment, media and
culture. Beyond campus, it helps bridge the gap between the entertainment industry and academia, and
between them and the public. Through scholarship and research; through its conferences, public events
and publications; through its role in the formulation of the academic field of entertainment studies; and
in its attempts to illuminate and repair the world, the Norman Lear Center works to be at the forefront of
discussion and practice in the field. For more information, visit www.learcenter.org.

PARTICIPANT MEDIA
Participant Media (www.participantmedia.com) is a leading media company dedicated to entertainment
that inspires and compels social change. Founded in 2004 by Jeff Skoll, Participant combines the power
of a good story well told with opportunities for viewers to get involved. Participant’s more than 70 films
include Spotlight, Contagion, Lincoln, The Help, He Named Me Malala, The Look of Silence, CITIZENFOUR,
Food, Inc., and An Inconvenient Truth. Participant has also launched more than a dozen original series,
including “Please Like Me,” “Hit Record On TV with Joseph Gordon-Levitt,” and “Fortitude,” for its television
network, Pivot (www.pivot.tv). Participant’s digital hub, TakePart (www.TakePart.com), serves millions
of socially conscious consumers each month with daily articles, videos and opportunities to take action.
Follow Participant Media on Twitter at @Participant and on Facebook.

2 MEAS U R IN G T H E I MPAC T O F FO O D, I NC .
 CONTENTS
INTRODUCTION KEY FINDINGS METHODOLOGY

4 CAN MOVIES REALLY

CHANGE PEOPLE?
7 PROFILING
VIEWERS
FOOD, INC. 16 OVERVIEW
7 Demographics
5 WHY STUDY FOOD,
INC.?
9 Media Exposure & 17 QUANTITATIVE ANALYSIS
Preferences
11 Politics
6 HOW CAN IMPACT
17 Advantages of PSM
ON VIEWERS BE 12 SOCIAL IMPACT:
Using Control Groups 18 Disadvantages of PSM
MEASURED?
to Measure Changes in
Knowledge & Behavior
18 QUALITATIVE ANALYSIS
12 Creating the Control
Group
12 The Propensity Model 18 ACKNOWLEDGEMENTS
13 Comparing Outcomes

14 SOCIAL IMPACT:
Self-Reported Changes
in Knowledge &
Attitudes

15 RECOMMENDATIONS
FROM RESPONDENTS

15 Overwhelmed
vs. Empowered
Respondents

CHANGING APPETITES & CHANGING MINDS 3

 INTRODUCTION
CAN MOVIES REALLY CHANGE PEOPLE?
For media researchers, it is really hard to figure out the impact of a TV show, a song, a film, a game. There is a general feeling that
representations have an impact on our lives, but it seems like an impossible thing to measure. In this research study we sought to
understand which variables influence someone’s likelihood of watching a particular film or television show and whether there was any
impact on viewers’ knowledge, attitudes and behavior that could be attributed to that media exposure.

Participant Media approached the Norman Lear Center to help them answer these questions about their film, Food, Inc. Participant
Media is a production company whose goal is to make films that change society and they have made dozens of critically acclaimed films,
both documentaries and fictional feature films, that deal with serious social issues in entertaining and engaging ways. Participant Media
wanted the Lear Center’s help figuring out whether their films were having the impact they had hoped for.

Lear Center researchers began to answer these questions by developing an online survey methodology that could evaluate the impact of
Participant Media’s films and their social action campaigns on the general public. The Center looked at three of Participant Media’s films
— Food, Inc., Waiting for ‘Superman’ and Contagion — and this report is part of a series of impact evaluations of those films.

Our research questions included:

● What do people learn about issues depicted in a film?

● Did a film encourage someone to take action?
● Which elements of Participant Media’s social action campaigns are most likely to encourage people to take action?
● Is there a relationship between emotional engagement with a film and taking action?
● Can we associate enjoyment or appreciation of a film with taking action?
● Is there a relationship between people’s inclination to take action and their beliefs about the potential impact that a film can
have on individuals, the media, public opinion and public policy?
● What do survey respondents believe Participant Media should do to motivate people to take social action?

Each of these three reports provides highlights from our findings. Please contact the Norman Lear Center at [email protected] to inquire
about additional results.

4 MEAS U R IN G T H E I MPAC T O F FO O D, I NC .
 U SC L E A R CE NTE R M E D I A I M PACT P RO J E CT | w w w. m e di a i m p a c t proj e c t .o rg

WHY STUDY FOOD, INC?

Food, Inc. is an Oscar-nominated documentary film by Robert Kenner that was released in the United States in 2009. In the film, Kenner
explores the food industry’s highly-mechanized and profit-driven approach to agribusiness in the United States. The film reveals the long-
term impact that these business practices have on the American consumer and highlights how government regulatory agencies, such as
the United States Department of Agriculture (USDA) and the Food and Drug Administration (FDA), have encouraged and sustained these
practices through their policies.

Food, Inc. was an ideal film for an impact evaluation because its content encouraged viewers to make simple behavior changes in their
everyday lives, such as drinking soda less often, eating at home instead of eating out, eating less meat, buying organic or sustainable
foods grown with little to no pesticide use, shopping at local farmers markets and reading food labels. It also encouraged viewers to try
to effect broader social change by telling schools to stop selling junk food, telling lawmakers that food safety is important, demanding job
protection for farm workers and food processors by ensuring fair wages and other protections, and to discuss these issues with family and
friends. Along with the film, Participant Media launched a social change campaign called “Hungry for Change,” a food-specific outreach
effort organized through Participant Media’s digital media arm, TakePart. The TakePart model was designed to enhance the impact of the
film by creating a channel for continued conversation about socially relevant news and information, both online and off.

CHANGING APPETITES & CHANGING MINDS 5

 Two questions guided our study of this film:

● Which variables influenced someone’s likelihood of watching Food, Inc.?

● What was the impact of Food, Inc. on knowledge and behavior?

Funding for this study, which was independently designed, conducted and released by the Norman Lear Center, was provided by
Participant Media, who also co-financed the making of Food, Inc.

HOW CAN IMPACT ON VIEWERS BE MEASURED?

Although the film was very successful for Participant Media, very few people around the country actually saw it. Therefore answering
questions about the impact of the film on a nationally representative sample of viewers would be very expensive to do and probably
ill-advised. The main problem is that people who decide to see a social-issue documentary are highly “self-selected” — that is, the
vast majority of the film’s viewers are probably biased toward the perspective of the film, and probably more likely than an average non-
viewer to take the actions recommended in the film. In short, niche films attract niche audiences and so trying to construct national
representative samples is neither cost-effective nor helpful if the goal is to understand what kind of impact a documentary has had on its
viewers.

The Lear Center developed an innovative survey instrument that could assess the impact of Food, Inc. on its viewers while taking into
account these issues of self-selection bias. We used propensity score matching (PSM) to help determine whether the different results that
we see between viewers and non-viewers are associated with watching Food, Inc., rather than pre-existing differences between these two
groups. We believe PSM results are more accurate in an assessment of media impact than traditional, non-PSM results, which are reports
of percentages of responses. More details on our
approach and an explanation of how we used
We used propensity score matching (PSM) to propensity score matching can be found in our
help determine whether the different results Methodology section.
that we see between viewers and non-viewers
This research began with a link to a survey
are associated with watching Food, Inc., rather about Participant Media films that was posted
than pre-existing differences between these on various Participant Media sites and an email
newsletter. We did not mention the survey was
two groups. specifically for Food, Inc. because our goal was
to attract respondents who had not seen the film
as well as those who had. The survey contained many traditional questions: demographic questions, questions about political affiliations
and attitudes toward the issues depicted in the film. However, we also asked survey respondents how likely it was that they would take
specific actions recommended in the film — whether they had seen the film or not.

This report describes the findings of this survey. It is our hope that these results will be useful for filmmakers, funders, activists and
media researchers who are eager to more accurately measure the impact of any type of media content on viewers, listeners, readers,
participants or players.

6 MEAS U R IN G T H E I MPAC T O F FO O D, I NC .
 KEY FINDINGS
PROFILING FOOD, INC. VIEWERS
This section provides a snapshot of all the Food, Inc. viewers who responded to our survey. Not everyone answered every question and so
the number of respondents ranges from 8,480 to 21,790.

Demographics
● The majority of Food, Inc. viewers were female (73%), Caucasian (74%) and did not have children (62%).
● Most viewers completed at least some college: 32% completed some college; 33% were college graduates and 16% attended
graduate school.
● Viewer employment was concentrated in health (30%) and education (29%).
● Viewer income varied across categories with the largest group of viewers reporting a yearly income of $75k or more (26%).
● Food, Inc. viewers were highly concentrated in California, New York, Texas and Florida. Since they were not evenly distributed around
the U.S., it would have been very difficult and expensive to find a random national sample for the survey.
● The survey was administered through existing Participant Media sites, social media channels and an email list, which made it
unnecessary to pay for access to a survey panel.
FOOD, INC. VIEWERS

73%
female

62% 0-.92%
no children .92%-2.08%
Percent of viewers in 2.08%-4.32%
4.32%-7.00%
74% survey population
7.00%-13.77%
Caucasian

CHANGING APPETITES & CHANGING MINDS 7

 Do you work
FOOD, INC. VIEWERS in any of the
following?**

What is your highest

level of education What is your approximate 30%
completed?* household income?* health

Less than 29%

6% some high school or less $25,000 17% education

8% completed high school

$25,000 —
$49,999 20%
20%
food industry
32% some college/trade school $50,000 —
$74,999 17%
33% college graduate
18%
non-profit
$75,000 or
more 26%
16% graduate school
Declined to 12%
4% answer 19% media/advertising

declined to answer

12%
government
*Due to rounding, total does not equal 100%.
**This was a check-all-that-apply question and these were all the options.

8 MEAS U R IN G T H E I MPAC T O F FO O D, I NC .
 U SC L E A R CE NTE R M E D I A I M PACT P RO J E CT | w w w. m e di a i m p a c t proj e c t .o rg

Media Exposure & Preferences

● The majority of viewers saw the film through online streaming (64%) or watched on DVD/Blu-ray (25%).
● Food, Inc. viewers had some previous exposure to other Participant Media films — 30% watched Fast Food Nation and 34% watched
An Inconvenient Truth.
● Food, Inc. viewers frequently watched social issue documentaries and
79% of viewers said a film can
feature films:
● 27% watch social issue documentaries ‘very often,’ compared to impact public opinion, compared
19% of non-viewers. to only 38% of non-viewers.
● 38% watch social issue scripted films ‘very often,’ compared to
28% of non-viewers.
● Viewers were exposed to the film’s outreach primarily through Participant Media’s Hungry for Change website (62%), as well as
through film previews (30%) and Facebook (28%).
● Viewers believe that a film can have an impact — either ‘moderate’ or ‘large’ — on individual attitudes, individual behavior and
public opinion:
● 95% of viewers said a film can impact individual attitudes, compared to 88% of non-viewers.
● 89% of viewers said a film can impact individual behavior, compared to 76% of non-viewers.
● 79% of viewers said a film can impact public opinion, compared to only 38% of non-viewers.
● Over 78% of viewers reported engaging in conversations about food safety in the last year. This suggests that the majority of viewers
were not passively absorbing information but were actively producing it in dialogue with their peers.

In the last year, do you recall

seeing or hearing anything about
MEDIA
food safety issues in any of the
following?
Where did you watch Food, Inc.?

64%

78%
Conversations with
friends, family, colleagues

73%
The Internet

68% 25%
Television

57%
Newspapers/Magazines 12%
4%
1% 1%
32%
Radio Classroom/ Special Regular TV DVD Online
House Party Theater Theater (Netflix, iTunes)
Screening Screening Screening

CHANGING APPETITES & CHANGING MINDS 9

 MEDIA Exp
Foodosure t
, Inc o
. out
Do you think a film could have a moderate reac
h
or large impact on any of the following?

95% Hungry for Change 62%

Preview 30%

89% Facebook 28%

Media Coverage 21%

Petition 21%

High School Lesson 9%

79% Book 8%

PSA 8%

Screening Invite 5%

69% Twitter 3%

51% Where did they

Individual Attitudes take the survey?
Individual Behavior 64% Food, Inc. Email List
Public Opinion
Media Coverage 31% TakePart Website
Public Policy
5% Food, Inc. Facebook Page
FOOD, INC.

THE COVE
FAST FOOD

WILSON’S WAR

INFORMANT

WAITING FOR
INCONVENIENT

‘SUPERMAN’

Which
THE KITE
NATION

RUNNER

CRAZIES

of the
SOLOIST
CHARLIE
TRUTH

following
AN

films have
THE
THE

THE

you seen?

100% 34% 30% 19% 17% 16% 15% 14% 13% 4%

10 MEAS U R IN G T H E I MPAC T O F FO O D, I NC .
 U SC L E A R CE NTE R M E D I A I M PACT P RO J E CT | w w w. m e di a i m p a c t proj e c t .o rg
Politics
We had assumed that viewers of a film like Food, Inc. would be politically engaged, but the results demonstrated that only a third were
involved in social causes or were willing to call themselves “politically active.”
● 26% of viewers claimed no political affiliation, and 24% said they had never donated any money or time to a political cause.
● 33% said they were not politically active and another 30% were not sure.
● While 36% of viewers said that they were strong supporters of social and environmental causes, only 15% reported strong support
for political or economic causes.

POLITICS
Food, Inc. Viewers: Political Identification

CAUSES
4%
Libertarian
5%
Other
THEY social
36%
6%
Green SUPPORT environmental

30%
Democratic
13%
Independent
economic
15%
26% 16% political
No political Republican
affiliation

“I consider
myself a
politically 7% 30% 30% 19% 14%
active
person.” strongly agree neutral disagree strongly
agree disagree

CHANGING APPETITES & CHANGING MINDS 11

 SOCIAL IMPACT:
USING CONTROL GROUPS TO MEASURE DIFFERENCES IN KNOWLEDGE & BEHAVIOR

Creating the Control Group

Given that people choose which films they will see based on their personal taste and interests, it is difficult for media researchers to
determine a film’s impact without dealing with some degree of bias. In this study, propensity score matching (PSM) techniques were used
to control for self-selection bias among survey respondents. PSM helped identify factors that predict the likelihood of a person seeing
Food, Inc. The Lear Center’s research team performed a statistical analysis of survey responses from all the respondents who watched
Food, Inc., and determined what personal characteristics increased their likelihood — or propensity — to see the film.

The Propensity Model

We discovered that viewers with a high propensity to watch Food, Inc. shared 17 characteristics:

Demographics: 1 No child
2 Not working in the media industry
3 Slightly more likely to work in education

Media Recalls seeing information about food safety on:

Exposure: 4 TV
5 news websites
6 radio
N
7 Visited the Hungry for Change website ATIO
!E DUC
I L D FREE ST FOOD
H A N
AT C IAL F TIO
8 Democratic affiliation E M OCR O F SOC OCS NA NT
Ideology & D N D
A FAFILMS & ONVENI
E
9 Believes that sustainable agriculture is important U E I N C H S
Taste: IS S T NEWES
TRU D I O S I T Y FO
R
10 Supports organized efforts to improve food safety/ RA GR
TV O
HUN GE
D CHAN LE AG
F O AB
sustainable agriculture ORT TAIN
Y IS
UPP SUS
T
SAFE I
11 Frequently watches social issue feature films FOOD STRY
IA INDU
MED
12 Frequently watches social issue documentaries
Believes that a film can impact:
13 individual attitudes
14 individual behavior
15 media coverage
16 Watched An Inconvenient Truth
17 Watched Fast Food Nation

We used these characteristics to generate a propensity score for survey

respondents who had seen the film, and those who had not. People with all 17 of these characteristics received the highest score and
those with the fewest received the lowest. However, just because someone has a high score does not mean that they have seen the film —
it just makes it more likely that they have seen it. Therefore, a person with a very high score may not have seen the film yet, and a person
with a very low score may have seen it despite themselves (for instance, a teacher may have screened the film in a class where students
with low scores saw it).

12 MEAS U R IN G T H E I MPAC T O F FO O D, I NC .
 U SC L E A R CE NTE R M E D I A I M PACT P RO J E CT | w w w. m e di a i m p a c t proj e c t .o rg
Once scores were assigned, we created two groups: people who had watched Food, Inc. and those who had not. Next we compared the
range of scores in each group and then performed “one-to-one matching,” which allowed us to use an automated method to remove
subjects from each group until both groups were composed of the same number of respondents with the same range of propensity scores
(e.g., each person who saw the film was paired with a person who did not see the film, but was equally likely to see the film based on their
propensity scores). The salient difference between the two groups was whether or not they had viewed Food, Inc.

This method allowed us to create something similar to an experimental study design where subjects are randomly assigned to a control
group and a treatment group. Here, the “treatment” group is comprised of those who had seen Food, Inc., and the “control” group is
comprised of those who had not seen the film but were equally likely to. By making these groups completely parallel, we were able to
examine whether differences in knowledge and behavior are attributable to exposure to the film.

Comparing Outcomes
We wanted to find out whether seeing the film Food, Inc. increased
people’s knowledge about food safety and, perhaps even more
importantly, whether it encouraged people to change any behaviors
around buying or eating food. We asked: did the Food, Inc. viewer
change somehow, due to exposure to this documentary film?*

Knowledge
● The film significantly impacted knowledge about genetically
modified foods and sustainable agricultural practices. Viewers
of the film knew significantly more about genetically modified
corn, for instance.

Behaviors
● Among those with the lowest propensity to see Food, Inc.,
viewers were significantly more likely than non-viewers
to support organized efforts to improve food safety and,
specifically, to support legislation that improves food safety.
These findings suggest that the film is not just speaking to the
choir but creating new converts to the movement.
● Among those with low- or mid-level propensity, viewers were
significantly more likely than non-viewers to look for information
about food safety and consistently buy organic or sustainable
food.
● Overall, viewers were significantly more likely to encourage their
friends, family and colleagues to learn more about food safety
and shop at their local farmers market. Additionally, almost all
viewers had significantly higher odds of eating healthy food.
● Viewers were significantly more likely to contribute time or
money to support organized efforts around:
● improving the treatment of animals in the food industry

● improving food and drinks served in schools

● passing legislation that improves food safety
● passing legislation that offers fair wages and job
protection to farm workers and food processors

*While we cannot measure “change” since we did not collect data from viewers
before viewing, comparing them to matched non-viewers allows us to suggest that
the difference may be attributed to viewing the film.
CHANGING APPETITES & CHANGING MINDS 13
 SOCIAL
IMPACT:
SELF-REPORTED
3% NO
CHANGES IN
KNOWLEDGE &
Food,
ATTITUDES Inc.
changed
84%
Some questions in the survey
could only be asked of people my life! YES 13%
who had seen the film.
Therefore, these findings did not NEUTRAL
involve the use of the control
group, which means there is
no correction for self-selection
bias.
After watching the film, do Did Food, Inc. explain to
● The vast majority of viewers you feel like you could be a you what you could do to
— an astounding 84% —
part of a social movement help solve the problems
said “this film changed my
to reform agribusiness? * addressed in the film?*
life.”
● Even though only 37% of
viewers said “I consider
YES 80% YES 79%

myself a politically active NO 4% NO 7%

person,” 80% said they
could be part of a social UNSURE 17% UNSURE 14%
movement to reform * due to rounding, total does not equal 100% * due to rounding, total does not equal 100%
agribusiness.
● 79% of viewers said that
the film explained to them
what they could do to solve
Viewers said they learned ‘a lot’ or ‘very
the problems addressed in much’ about the following issues:
the film.
● Over two-thirds of viewers
said they learned ‘a lot’
or ‘very much’ about the
following issues: 87% 85%
● Agribusiness policies 79% 79% 73%
●

●
Sustainable agriculture
Food safety issues
67%
● Treatment of animals
in U.S. agribusiness
● Treatment of workers
in U.S. agribusiness
● Genetically modified
foods
Agricultural Sustainable Food Safety Treatment of Treatment of Genetically
Policies Agriculture Issues Animals in Workers in Modified
U.S. Agriculture U.S. Agriculture Foods

14 MEAS U R IN G T H E I MPAC T O F FO O D, I NC .
 U SC L E A R CE NTE R M E D I A I M PACT P RO J E CT | w w w. m e di a i m p a c t proj e c t .o rg

RECOMMENDATIONS FROM RESPONDENTS

Participant Media expressed interest in understanding what more they could do to help their network get involved in efforts to reform
agribusiness. To address this issue, the Lear Center created a simple open-ended question geared toward capturing a broad range of
responses rather than guiding respondents to describe a particular practice or discuss a specific issue (see the Methodology section for
more details on the qualitative aspects of the survey).

From over 20,000 respondents, we received 4,835 responses to the open-ended

CONCEPTUAL CATEGORIES question; some responses were quite lengthy and contained scores of suggestions.

(top 30 out of 308) ● The top suggestion for Participant Media and TakePart was to advertise more.
1. advertise/media coverage Many respondents expressed frustration that they would have missed this film if it
2. want more info & ways to get involved had not appeared in their Netflix queue.
● Food, Inc. viewers were more likely to be childless, but they were highly likely to
3. show/teach in schools
suggest that Participant target kids and schools with their Food, Inc. messaging.
4. spread the word/get the word out Respondents felt that this film should be screened in schools and that the messages
5. start early with kids/youth shared in the film needed to be delivered to children, specifically, since they have a
6. organic lot of power to change the behavior of their parents.
● Respondents felt that Participant could be helpful to viewers in the future by:
7. buy local/organic
● Better explaining how individual action leads to system change
8. make film available
● Being very blunt about what to do about food safety issues and how to fix the
9. policy change problems
10. agribusiness ● Providing information about support available within viewers’ communities
11. keep doing (what you’re doing) ● Clearly depicting success stories
● Respondents also suggested that Food, Inc. viewers would benefit from maps or
12. local involvement
lists of local farmers markets and information on how to read product labels.
13. N/A ● Lastly, respondents felt Participant should do the following to stimulate action
14. affordability/cost of food from its viewers:
15. social media ● Promote actions their viewers can make every day
16. more documentaries ● Ask viewers to contact their representatives
● Given that viewers want to vote with their dollars, tell them what products to
17. educate
buy
18. government
19. college students Overwhelmed vs. Empowered Respondents
20. recommend companies/products
Some viewers felt empowered after seeing the film but others felt overwhelmed by
21. contact legislature
the information presented. Overwhelmed or immobilized respondents did not see a
22. farmers markets
clear path for action and expressed a general sense of helplessness. Empowered or
23. broader than USA/global perspective mobilized respondents seemed hopeful that they could apply what they had learned and
24. list of local farms/farmers markets incorporate specific actions into their everyday lives.
25. reform ● Empowered respondents typically discussed the connection between the micro
and the macro, the local and the global. They understood, for instance, how their
26. animal cruelty
individual purchasing power, in aggregate, can affect a powerful industry like
27. individual change agribusiness.
28. small/local farmers ● Empowered respondents often mentioned the importance of having a strong sense
29. eat healthy of community and access to social support.
30. Monsanto

CHANGING APPETITES & CHANGING MINDS 15

 METHODOLOGY
OVERVIEW
The Lear Center’s impact
evaluation of Food, Inc. and
its campaign began over a
year after the film’s release —
beginning in November 2010
and concluding in April 2011.
The year between the release
of the film and data collection
allowed for the population of
viewers to grow well beyond
moviegoers to those who
were exposed to the film
through television, video/
DVD rental, or online media in
their homes, classrooms and
communities. Additionally,
waiting a year made it
possible to capture sustained
changes in knowledge,
attitudes and behavior, as
opposed to the short-lived
or aspirational changes that
might register in a survey
taken immediately after a
screening of a film.

The research began with a 5-

10-minute online survey that
was disseminated through a
link placed in a Participant
Media email blast. The survey
was also posted on the film’s
promotional website, Hungry
for Change, the Participant
Media website, the TakePart

16 MEAS U R IN G T H E I MPAC T O F FO O D, I NC .
 U SC L E A R CE NTE R M E D I A I M PACT P RO J E CT | w w w. m e di a i m p a c t proj e c t .o rg

website and the Facebook and Twitter accounts associated with the film. All surveys were completed online; participation was voluntary
and all survey items were in English.

QUANTITATIVE ANALYSIS
The survey methodology in this study of Food, Inc. adapts propensity score matching (PSM) techniques used in clinical research as well as
communication studies. PSM specifically addresses the key problem of “selection bias” among movie viewers: only certain people choose
to see certain films, making it very difficult for researchers to expose people randomly to a movie and to determine the actual impact of
the film.

Propensity score matching is a method of statistical

analysis that controls for simple selection bias in
Using this methodology allowed the Lear
studying the effect of exposure to a treatment or Center’s researchers to create a detailed
intervention. In the case of a media campaign, the
“intervention” may be a feature film, a song, a TV profile of likely viewers of the film and
episode, a billboard, a game, a pamphlet, a PSA or a to compare viewers who saw the film
news report.
with very similar people who did not.
In this research, the first phase of PSM entailed finding
the factors that would predict the likelihood of a subject
being exposed to the film, Food, Inc. These factors might include some combination of personal taste, ideology, media preferences, past
behavior patterns and demographics. Using logistic regression, we created a model based upon those predictors which included 17
variables.

In the second phase, subjects are assigned propensity scores: subjects who did not view Food, Inc. are matched and compared with
subjects with the same propensity score who did view Food, Inc. In studies of this type, we are typically looking for differences in awareness,
knowledge, attitudes and behavior based upon exposure to the film.

Using this methodology allowed the Lear Center’s researchers to create a detailed profile of likely viewers of the film and to compare
viewers who saw the film with very similar people who did not. Unlike typical survey research, this method allows researchers to construct
something similar to a classic study design where individuals are randomly assigned to a treatment group and a control group.

The PSM results are based on a subset of 15,157 respondents who answered all of the PSM questions in the survey (from a total of 22,489).
All of these survey respondents were assigned a propensity score indicating the likelihood that they would view Food, Inc. The scores were
based on 17 variables such as demographics, prior viewership of social issue films, and exposure to Food, Inc. promotional materials.
After performing one-to-one matching, both the exposed and the control groups were composed of respondents with the same range of
propensity scores. There were 708 people in each of these groups. The difference between the two groups was whether or not they had
viewed Food, Inc.

Advantages of PSM
● PSM is an accepted tool for dealing with adjustments for bias in online surveys (the online population is not representative of the
general population.)
● In multimedia evaluations, there are often many variables influencing outcomes, making simple weighting schemes difficult to
determine. PSM allows for control of multiple variables so that the impact of the campaign can be examined more specifically.
● Pre-post testing can be problematic in that the survey is administered in two different time periods. Subjects will inevitably be
exposed to a variety of other media messages between the pre- and post-test, which could contribute to altered outcomes in the

CHANGING APPETITES & CHANGING MINDS 17

 post-test. Also, the pre-test can serve to prime subjects for the post-test, encouraging them to think more about the issues related
to the intervention than they may have otherwise.

Unexposed PSM respondents = pre-intervention respondents

Exposed PSM respondents = post-intervention respondents

Disadvantages of PSM
● It relies on a relatively large sample size that contains enough variety for an exposed cohort to have a comparable non-exposed
cohort. Although the sample size on this study was very large, we have successfully performed a PSM analysis with as few as 1,000
respondents.

QUALITATIVE ANALYSIS
The survey included an open-ended question which asked people, “Do you have any suggestions about what Participant Media or
TakePart could do to help people get involved in efforts to reform agribusiness?” The advantage of using an open-ended question is that
the assumptions of the researchers would not limit the range of responses. In the 4,835 responses we received, respondents had the
opportunity to provide information and feedback that they thought was relevant to mobilization and reform.

Responses were systematically coded based upon recurring themes within the data by a single coder. Open coding was used for any
items that were stressed by respondents, regardless of whether the coder thought it was important or not. Open coding generated a
code list that was then reviewed by the authors for recurring themes that could be ranked by frequency of occurrence through ATLAS.ti, a
qualitative data analysis tool. These themes were then consolidated into conceptual categories when open codes were similar and, once
combined, resulted in 308 categories. Next, the codes were analyzed to determine linkages that respondents made between concepts in
their descriptions of the opportunities and challenges to making changes in their own lives and in society more broadly.

ACKNOWLEDGEMENTS
The research team included principal investigator Johanna Blakley, PhD, who is the managing director and director of research at the
Norman Lear Center; chief research consultant Sheena Nahm, PhD, MPH, and USC doctoral students Grace Huang, MPH, and Heesung
Shin, MPH. Special thanks to USC doctoral student LeeAnn Sangalang for providing research assistance with this report; the Norman Lear
Center’s Veronica Jauriqui for report design and layout with assistance from Adam Amel Rogers; and the Media Impact Project’s Nandi
Jordan, PhD, for research and report editing.

18 MEAS U R IN G T H E I MPAC T O F FO O D, I NC .