H o m e w o r k 5 - A s s i g n m e n t

1. Suppose you are looking at some survey data, and you find that people who report
reading newspapers more regularly are also more likely to vote.

2. What is the unit of analysis in this correlation?

3. What is the dependent variable? What is the independent variable?

4. What are some confounders or issues of reverse causality that might be driving this
relationship?

5. How would you control for these confounders? Think carefully about how you would
measure the confounding variables.

6. Are there some confounders or problems that you may not be able to fully control for?
Why?

7. Now, suppose you decided to run an experiment to see if there is a causal relationship
between reading the newspaper and voting. Let’s say you had a large amount of money
to run the experiment. Describe how you would do it. Make sure you answer the
following questions:

a. What is the population of interest?

b. How would you select a sample?
c. What would your treatment or intervention be?
d. How would you randomize?

8. How or why does the experiment address the confounders or problems you were not
fully able to control for as discussed question 6?

9. What problems of non-compliance might you face? How would you address them?

10. What problems of attrition might you face? How would you address them?

11. Read the paper by Gerber et al (2009). It is attached to the end of this document. Discuss
the research design, answering the same questions you did in question 7.

12. In what ways does the author’s design and your design differ? How might the design
differences affect the results and interpretation?

13. What are the author’s main results? Are they what you expected? How did the results
change your beliefs about media consumption and voting?

14. What future research questions do these results inspire?

Does the Media Matter? A Field Experiment Measuring the Effect of Newspapers on
Voting Behavior and Political Opinions
Author(s): Alan S. Gerber, Dean Karlan and Daniel Bergan
Source: American Economic Journal: Applied Economics , April 2009, Vol. 1, No. 2 (April
2009), pp. 35-52
Published by: American Economic Association

Stable URL: https://www.jstor.org/stable/25760159

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

is collaborating with JSTOR to digitize, preserve and extend access to American Economic
Journal: Applied Economics

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
American Economic Journal: Applied Economics 2009, 1:2, 35-52
http://www. aeaweb. org/articles.php?doi?10.1257/app. 1.2.35

Does the Media Matter? A Field Experiment

Measuring the Effect of Newspapers
on Voting Behavior and Political Opinions*

By Alan S. Gerber, Dean Karl an, and Daniel Berg an*

We conducted a field experiment to measure the effect of exposure

to newspapers on political behavior and opinion. Before the 2005
Virginia gubernatorial election, we randomly assigned individuals to
a Washington Post/ree subscription treatment, a Washington Times
free subscription treatment, or a control treatment. We find no effect
of either paper on political knowledge, stated opinions, or turnout in
post-election survey and voter data. However, receiving either paper
led to more support for the Democratic candidate, suggesting that
media slant mattered less in this case than media exposure. Some evi
dence from voting records also suggests that receiving either paper led
to increased 2006 voter turnout. (JEL D72, L82)

Citizensvision
learn and read
about in newspapers.
politics Recent from
and government work has
the examined
news theyhow theon
watch media
tele
shapes the public's political knowledge, attitudes, and behavior (Timothy J. Besley
and Robin Burgess 2002, Alexander Dyck and Luigi Zingales 2002, and James
T. Hamilton 2003). Media sources may influence the public not only through the
slant of a particular report (Stefano DellaVigna and Ethan Kaplan 2007) but also by
choosing what to cover (Lisa M. George and Joel Waldfogel 2006).
This paper reports the results of a natural field experiment1 to measure the effect
of political news content on political behavior and opinions. The Washington, DC
area is served by two major newspapers, the conservative Washington Times and the
more liberal Washington Post (Tim Groseclose and Jeffrey Milyo 2005). The presence
of a liberal and conservative paper serving the same region creates an outstanding
opportunity to study the effect of media slant in a naturalistic setting within a single
population.2 Approximately one month prior to the Virginia gubernatorial election in

* Gerber: Department of Political Science, Yale University, 77 Prospect St, New Haven, CT 06520, and
National Bureau of Economic Research (e-mail: [email protected]); Karlan: Yale University, 27 Hillhouse
Ave, New Haven, CT 06511, Innovations for Poverty Action and Massachusetts Institute of Technology Jameel
Poverty Action Lab (e-mail: [email protected]); Bergan: Department of Communications, Michigan State
University, 463 Communications Arts & Sciences Building, East Lansing, MI 48824 (e-mail: [email protected]).
We would like to thank Stefano DellaVigna, Don Green, Tim Groseclose, and Brian Knight for very helpful com
ments. We also thank Chris Mann and Brenner Tobe for providing the turnout data.
f To comment on this article in the online discussion forum visit the articles page at:
http://www.aeaweb.org/articles.php?doi=10.1257/app. 1.2.35.
1 As per the taxonomy put forth in Glenn W. Harrison and John A. List (2004).
2 The Washington Post is also a more nationally prominent newspaper than the Washington Times. Thus, read
ers may perceive a difference in quality and trustworthiness, not just slant, between these two newspapers.

35

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
 36 AMERICAN ECONOMIC JOURNAL: APPLIED ECONOMICS APRIL 2009

November 2005, we administered a short survey to a random selection of households

in Prince William County, a northeastern Virginia county. From the 3,347 households
reporting that they received neither the Post nor the Times, we randomly assigned
households to get subscriptions to either the Post or the Times for approximately ten
weeks, or to a control group that was not sent either paper. A week after the election,
we conducted a follow-up survey in which we asked individuals whether they voted
in the November 2005 election; which candidate they selected (or preferred, if they
said they did not vote); their attitudes toward the president, the political parties, and
national political issues; their attitudes toward news events of the previous weeks; and
their knowledge about recent news events. We also obtained voter turnout data for the
November 2005 and 2006 elections from state administrative records.

Our research contributes to a large and growing literature on the effect of news
media on political attitudes and behavior. The earliest media studies discounted the
impact of mass communications, but most recent studies conclude that media expo
sure can have a sizable effect. The most common method of measuring the effect
of media content is to use data from surveys to measure the association between a
respondent's reported media exposure and his or her political views. Among the find
ings adduced to suggest media effects, researchers frequently document strong asso
ciations between exposure to media with a distinctive slant and the viewers' political
attitudes and information (Peter Clarke and Eric Fredin 1978; Arthur Miller, Edie N.
Goldenberg, and Lutz Ebring 1979; Carl R. Bybee et al. 1981; Gina M. Garramone
and Charles K. Atkin 1986; Joel Lieske 1989; Craig Leonard Brians and Martin P.
Wattenberg 1996; Russell J. Dalton, Paul A. Beck and Robert Huckfeldt 1998; and
John R. Hibbing and Elizabeth Theiss-Morse 1998). While suggestive, this research
design may produce upwardly biased estimates of media influence due to biased
reports of media exposure and to selection bias from the tendency for individuals
to seek out information that agrees with their pre-existing views (Timothy C. Brock
1965, Paul D. Sweeney and Kathy L. Gruber 1984, Matthew Gentzkow and James
M. Shaprio 2005, and Sendhil Mullainathan and Andrei Shleifer 2005).
Several recent papers employ natural experiments to measure media effects on
voter turnout. Gentzkow (2006) studies the introduction of television and shows that
the introduction of television was associated with a decline in voter turnout, a sharp
drop in the number of newspapers, and a drop in newspaper readership. Noting the
political science literature linking newspaper readership to political participation
(Michael Morgan and James Shanahan 1992), he argues that the rise in television is
responsible for between one-quarter and one-half of the total decline in voter turnout
from the 1950s to the 1970s. George and Waldfogel (2002) study the expansion of
the national edition of the New York Times. They find that, as the New York Times
displaces local newspaper readership, turnout in nonpresidential elections falls rela
tive to turnout in presidential elections.
A recent study of the persuasive effect of political news (DellaVigna and Kaplan
2007) uses variation in the availability of the Fox News Channel across cable sys
tems to measure the effects of the channel's news coverage on the Republican Party
vote share in presidential elections (as well as the Republican share in Senate races
and voter turnout). They estimate that Fox News caused about one-half of a per
centage point shift toward George W. Bush in the 2000 presidential election. Since

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
VOL. 1 NO. 2 GERBER ETAL.: DOES THE MEDIA MATTER? 37

Fox News is viewed by only a modest portion of the population, depending on

which measure of viewership the authors use (either recall of watching Fox News
or diary records of viewership, which show a smaller audience) this translates into
a persuasion rate among those exposed to the channel of between approximately 5
percent and nearly 30 percent of the audience not already voting Republican in the
presidential race. Given that citizens are unusually well informed and consistent
in their presidential voting behavior, compared to voting for lower offices, these
results suggest media slant can have a powerful political influence.
The large behavioral effects reported in DellaVigna and Kaplan (2007) accord
with recent laboratory experimental evidence in which political advertising has sub
stantial persuasive effects in gubernatorial and senate races (Stephen Ansolabehere
and Shanto Iyengar 1995). In addition, a growing literature has employed random
ized field experiments to measure the turnout effects of campaign mailings, phone
calls, and face-to-face canvassing (Alan S. Gerber and Donald R Green 2000, and
Green and Gerber 2008). According to meta-analysis of dozens of studies of each of
the alternative methods of voter mobilization, door-to-door canvassing prior to the
election often has a large effect on voter turnout, raising turnout by approximately
8 percentage points in a typical election, while phone calls and mailings have more
modest but still notable effects. A live phone call from a commercial firm raises
turnout around 0.5 percentage points, a call from a volunteer raises turnout 2.5 per
centage points, and several pieces of campaign mail boosts turnout by approximately
1 percentage point (Green and Gerber 2008).
In our experiment, we find no effect of receiving either paper on knowledge of
political events, opinions of those events, or on voter turnout in the 2005 gubernatorial
election. However, receiving either paper led to more support for the Democratic can
didate, suggesting that media slant mattered less in this case than exposure to media.
There was also some evidence of increased voter turnout in the 2006 election among
those receiving either paper.
We contribute to the literature on the effect of media on politics by performing
what appears to be the first field experiment measuring the effect of newspapers on
political attitudes and behavior. Field experimentation has some advantages over the
previous research strategies, namely use of a naturalistic setting and an assurance
of orthogonality of treatment to observable and unobservable characteristics. Our
study has some important limitations, the most important of which is the relatively
small sample size. The sample size for the post-election survey was 1,081 interviews
and the sample size for the data merged with administrative voting records is still
only 2,571. Consequently, the standard errors of the estimated treatment effects were
often large. Our standard errors are such that we fail to reject (two-sided, at the five
percent level) the null hypothesis of no effect if we obtain a treatment effect estimate
of less than a 3.6 percentage point change in voter turnout (using state voting data),
a 7 percentage point increase in likelihood of voting for the Democratic candidate
(using survey data), or less than an 8.6 percentage point differential between the Post
and Times groups in likelihood of voting for the Democratic candidate.
The paper proceeds as follows. Section I describes the experiment in more detail.
Section II presents the results. Section III discusses the implications of the findings,
limitations of the research, and directions for future work.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
38 AMERICAN ECONOMIC JOURNAL: APPLIED ECONOMICS APRIL 2009

I. Experimental Design and Data

A. Experimental Procedures

Households were drawn from a sample of residents in Prince William County,

VA, a county 25 miles outside of Washington, DC. The county was suitable for
our study because it is within the circulation of both the Times and the Post but far
enough away from the Washington, DC area that the sample is not dominated by
citizens involved professionally with politics. We sampled individuals from two lists:
a list of registered voters and a consumer database list. Roughly equal proportions
were included in the sample from each list.
We performed a baseline survey in September 2005.3 We asked individuals if
someone at the household received either the Post or the Times, and we excluded
from the study those who said they received either newspaper. This perhaps is the
most important issue to note regarding the formation of the sample frame. We are
studying individuals who do not already subscribe to a newspaper, hence are exam
ining the effect of exposing individuals who, on average, are less exposed to the media
than the average individual. We also asked a number of other questions about news
paper readership and politics. Only individuals who completed all questions in the
initial phone survey were included in the experimental sample.
Individuals were then randomly assigned to one of three groups: the Post, the
Times, or a control group.4 Participants received a postcard in the mail that said
"Congratulations! You have won a free ten-week subscription to the Washington
Times(Post)r5 Table 1A shows sample statistics from the baseline interview for
the entire sample, broken down by treatment group. The lowest /?-value for a test of
independence across groups was 0.18 (for gender). Using the treatment assignment
as the dependent variable in a multinomial logit model produced a p-value for the
joint significance of the covariates of p = 0.95.
There were three noncompliance issues to note regarding treatment administra
tion. First, 6 percent of households in the treatment groups opted out of the free sub
scription. In our analysis, we focus on intent to treat effects and include all treatment
group subjects even if they cancelled. Second, some addresses (76 for the Times, 1 for
the Post) were deemed "undeliverable." 6 Third, when we sent the list of households
to the Washington Post, we were informed that 75 (out of 965) were already on the

3 The complete survey is available at http.7/karlan.yale.edu/p/index.php.

4 The study was launched in two waves each a week apart. This was done primarily due to capacity constraints
in starting new subscriptions at the newspapers. Prior to the first wave, 50 households were removed at random for
a small pilot study to gauge the refusal rate and get experience with the logistics of starting and stopping newspa
per delivery. These households are excluded from the analysis.
5 The postcard continued, "We have held a drawing to award free ten-week subscriptions of the Washington
Times to households in Prince William County. Delivery begins this week. Delivery will automatically end after
ten weeks, you do not need to call to cancel. However, if you want to c ancel before the end of the ten weeks, please
call 1-800-xxx-xxxx and we will remove you from this promotion. Thank you for trying out the newspaper."
6 We verified that the papers we had ordered were actually delivered by having a research assistant observe a
random sample of the treatment group households during the first wave. The Times, after reviewing the full list,
reported that there were 76 addresses to which they were unable to deliver. The Post was able to deliver to all but
one of the addresses. Undeliverable addresses are included in all analyses, but it is useful to note that these 76
(8 percent) addresses may be different along important characteristics, such as income.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
VOL. 1 NO. 2 GERBER ETAL.: DOES THE MEDIA MATTER? 39

Table 1A?Summary Statistics from Baseline Survey

(Sample frame: all baseline survey respondents, mean, standard errors, and standard deviations)

All Control Post Times /7-value

(1) (2) (3) (4) (5)
> female 34.76 34.44 33.01 37.02 0.18
(0.84) (1.28) (1.53) (1.59)
[47.63] [47.54] [47.05] [48.31]

% voted in 2004 (self-report) 88.62 88.51 88.82 88.57 0.99

(0.78) (1.22) (1.44) (1.45)
[31.77] [31.91] [31.54] [31.86]

% voted in 2002 (self-report) 48.08 49.04 45.76 49.06 0.48

(1.23) (1.92) (2.27) (2.28)
[49.98] [50.03] [49.87] [50.04]
% voted in 2001 (self-report) 7.30 7.07 7.66 7.28 0.93
(0.64) (0.98) (1.21) (1.19)
[26.03] [25.65] [26.62] [26.00]
% from consumer list 50.91 52.58 49.95 49.37 0.24
(0.86) (1.32) (1.61) (1.62)
[50.00] [49.95] [50.03] [50.02]

> get news or political magazine 9.20 9.36 8.81 9.37 0.88
(0.50) (0.77) (0.91) (0.95)
[28.91] [29.13] [28.36] [29.15]

% prefers Democratic candidate for governor in VA 14.43 14.53 14.61 14.11 0.94
(0.61) (0.93) (1.14) (1.13)
[35.15] [35.25] [35.34] [34.83]

% no preference in VA governor race 14.82 14.18 15.54 15.05 0.63

(0.61) (0.92) 0.17) (1.16)
[35.53] [34.89] [36.25] [35.78]

> in wave 2 of random assignment 37.14 36.87 37.31 37.37 0.96

(0.84) (1.28) (1.56) (1.57)
[48.32] [48.26] [48.39] [48.40]
% participating in follow-up survey 32.30 31.70 32.02 33.47 0.65
(0.81) (1.23) (1.50) (1.53)
[46.77] [46.55] [46.68] [47.21]

Number surveyed?baseline 3,347 1,432 965 950

Notes: Standard errors reported in parentheses; standard deviations in brackets. Column 5 reports the p-values
for chi-squared tests of independence between treatments for each variable. The second through fourth rows (per
cent voted) apply only to the voter registration (i.e., nonconsumer) sample frame. All regressions in Tables 2-4
include controls for which sample frame provided the observation. A multinomial logit model predicting assign
ment to treatment using all of the above baseline variables yields a chi-squared test value of 9.21 (d.f. 18, p-value
of 0.95).

Post subscription list (although it may be the case that these households were receiv
ing only Sunday delivery). The Times has a lower subscription rate and reported only
five households already subscribing. As group assignment was random, this suggests
that some portion of the control group and Times treatment group, perhaps around 8
percent, was getting the Post at least on Sunday, and a much smaller portion of the
Post treatment group and the control group was getting the Times. Since the treatment
effect estimates are based on the difference in treatment rates between the treatment
and control group, this suggests the treatment effect should be interpreted as the effect
of boosting the household exposure rate to the Post by approximately 92 percentage
points rather than 100 percentage points. Thus, any observed difference between the
Post treatment group and the other groups will tend to underestimate, most likely by
a small amount, the impact of exposure to the Post.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
 40 AMERICAN ECONOMIC JOURNAL: APPLIED ECONOMICS APRIL 2009

B. Background on Newspapers and Media Coverage

We followed the news reporting of both papers throughout the study period and
recorded the choice of topics and headlines in each paper. Previous researchers
have found that, as is widely believed, the Post leans left and the Times leans right
(Groseclose and Milyo 2005). Our analysis of coverage confirms this assessment.
While both papers gave extensive attention to the Iraq War, the Times had three above
the fold headlines mentioning Iraqi efforts at forging a constitution and only one men
tion of the controversies involving Iraq detainees. The pattern for the Washington
Post was the opposite, featuring three stories on detainees and one on the constitu
tion. The Post gave much greater attention to Republican political difficulties. The
disclosure of the identity of Valerie Plame, a Central Intelligence Agency (CIA)
officer married to an administration critic, was given very extensive coverage in the
Post and much less prominence in the Times. In place of the Posfs emphasis on the
Plame issue and other administration controversies, the Times gave more coverage to
the filling of the impending Supreme Court vacancy. The Times had about twice as
many above the fold stories on the nominations of Harriet Miers and Samuel Alito
for the Supreme Court. Both papers covered Hurricane Wilma, but the Post also had
several stories about preparations for and the response to Hurricane Katrina.7
The manner in which subjects were covered also suggests the Times was more
favorable to the administration than was the Post. A comparison of same day head
lines illustrates this. On Tuesday, October 18, 2005, both papers had front page
stories on Iraq. The Times story had the headline "No tears for Saddam in Iraq,"
while the Post ran the headline "Iraqis Say Airstrikes Kill Many Civilians." On
November 4, 2005, the Times ran a story with the headline "Recruits Join Armed
Forces Seeking War?A Sort of Vendetta Spurs Youth to Enlist After 9/11," whereas
on November 7, 2005, the Post ran a story with the headline "Youths in Rural U.S.
Are Drawn to Military?Worries About Jobs Outweigh War Fears." When Miers
nomination to the Supreme Court was withdrawn, a Times headline read simply
"Miers Rules Herself Out" (October 28, 2005), while the Post used the headlines
"Miers Withdrawn as Nominee for the Court" and "Nomination was Plagued by
Missteps from the Start."
Both newspapers gave the governor's race extensive coverage. The Post had 15
stories on the front page or first page of the Metro section while the Times had ten
articles. National events may have affected the gubernatorial race in Virginia as
well. News reports cited national issues as persuasive to many voters in the Virginia
election.8

7 Two tables summarizing the front page, above-the-fold stories, by topic, for the Post and the Times, along
with a listing of every headline as well as every headline on the Metro page that related to the Virginia gubernato
rial election, are available at http://karlan.yale.edU/p/index.php.
8 "Moments foreshadowing a political collapse" The Virginian-Pilot, November 13, 2005; '"Twas a Famous
Victory, & Republicans Have Some Issues" Richmond Times Dispatch, November 13, 2005; "New GOP Agenda,
Many Things Combined to Cripple Kilgore's Gubernatorial Hopes" Richmond Times Dispatch, November 13,
2005.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
 VOL. 1 NO. 2 GERBER ETAL.: DOES THE MEDIA MATTER? 41

Table IB?Summary Statistics from Baseline Survey

(Sample frame: only those who completed the follow-up survey, mean, standard errors, and standard deviations)

All Control Post Times /7-value

(1) (2) (3) (4) (5)
% female 32.86 31.54 36.84 30.89 0.21
(1.44) (2.20) (2.77) (2.61)
[46.99] [46.52] [48.32] [46.28]
? voted in 2004 (self-report) 90.70 92.58 89.24 89.53 0.44
(1.23) (174) (2.47) (2.34)
[29.07] [26.27] [31.09] [30.70]

% voted in 2002 (self-report) 55.99 57.64 50.63 58.72 0.27

(2.10) (3.27) (3.99) (3.76)
[49.68] [49.52] [50.15] [49.38]

> voted in 2001 (self-report) 8.41 9.17 8.23 7.56 0.84

(1.17) (1.91) (2.19) (2.02)
[27.78] [28.92] [27.57] [26.51]
j from consumer list 48.29 49.56 48.87 45.91 0.59
(1.52) (2.35) (2.85) (2.80)
[49.99] [50.05] [50.07] [49.91]

% get news or political magazine 11.29 10.35 11.00 12.89 0.54

(0.96) (1.43) (178) (1.88)
[31.66] [30.50] [31.34] [33.57]
% prefers Democratic candidate for governor in VA 19.43 19.60 21.04 17.61 0.55
(1.20) (1.87) (2.32) (2.14)
[39.58] [39.74] [40.82] [38.15]

% no preference in VA governor race 12.86 13.22 10.03 15.09 0.16

(1.02) (1.59) (171) (2.01)
[33.49] [33.90] [30.09] [35.86]

? in wave 2 of random assignment 35.06 35.02 38.51 31.76 0.21

(1.45) (2.24) (2.77) (2.61)
[47.74] [47.76] [48.74] [46.63]

Number surveyed?follow-up 1,081 454 309 318

Notes: Standard errors reported in parentheses; standard deviations in brackets. Column 5 reports the p-values for
chi-squared tests of independence between treatments for each baseline variable. A multinomial logit model pre
dicting assignment to treatment using all of the above baseline variables yields a chi-squared test value of 17.62
(d.f. 18,p-value of 0.48).

C. Outcome Data

During the week after the November election, we reinterviewed 1,081 of the 3,347
individuals in our sample for the follow-up survey. Response rates of 30 or 40 percent
are typical in the public opinion literature (Herbert Asher 2004).9 Survey respondents
were not told of any link between the free subscriptions and the phone surveys. The
follow-up survey asked questions about the 2005 Virginia gubernatorial election (e.g.,
did the subject vote, which candidate was voted for or preferred), national politics (e.g.,

9 Public opinion literature suggests that increasing the response rate from 30-40 percent to 60 percent pro
duces similar results for many topics including attention to media, engagement in politics, and social and political
attitudes (Scott Keeter et al. 2000). The complete set of dispositions was: Survey completed 31.8 percent, Refused
to answer/Not interested 29.7 percent, Person not available 10.3 percent, Answering machine 9.8 percent, Partial
survey/refused 6.0 percent, Disconnect 4.1 percent, Do not call/irate 3.1 percent, Ring no answer 1.7 percent,
Wrong number 1.5 percent, Language barrier 1.1 percent, Busy signal 0.8 percent, Rescheduled call 0.1 percent,
Deceased 0.03 percent.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
42 AMERICAN ECONOMIC JOURNAL: APPLIED ECONOMICS APRIL 2009

Table 1C?Summary Statistics for Outcome Measures

(Sample frame: only those who completed the follow-up survey, mean, standard errors, and standard deviations)

All Control Post Times

(1) (2) (3) (4)
% voted 2005?self-reported from survey 0.73 0.73 0.72 0.74
(0.01) (0.02) (0.03) (0.02)
[0.44] [0.45] [0.45] [0.44]
> voted 2005?administrative state voting dataa 0.57 0.56 0.57 0.56
(0.01) (0.02) (0.02) (0.02)
[0.50] [0.50] [0.49] [0.50]
% voted 2006?administrative state voting data3 0.64 0.63 0.65 0.66
(0.01) (0.01) (0.02) (0.02)
[0.48] [0.48] [0.48] [0.47]
% voted for Democrat in 2005 VA election 0.45 0.41 0.49 0.45
(0.02) (0.03) (0.04) (0.03)
[0.50] [0.49] [0.50] [0.50]
% did not vote, but preferred Democrat 0.40 0.42 0.41 0.35
(0.03) (0.05) (0.06) (0.05)
[0.49] [0.50] [0.50] [0.48]
% voted for or preferred Democrat 0.43 0.41 0.47 0.42
(0.02) (0.02) (0.03) (0.03)
[0.50] [0.49] [0.50] [0.50]
Most important problem 0.92 0.92 0.93 0.91
(1 = issue other than scandals, 0 = scandals) (0.01) (0.01) (0.01) (0.02)
[0.27] [0.27] [0.25] [0.28]
Most important issues in Iraq 0.52 0.53 0.49 0.55
(1 = Iraq constitution or Hussein trial, 0 = other) (0.02) (0.02) (0.03) (0.03)
[0.50] [0.50] [0.50] [0.50]
Progress in Iraq 1.35 1.36 1.33 1.35
(3 = going very well, 0 = going very badly) (0.03) (0.04) (0.06) (0.06)
[0.97] [0.93] [1.01] [1.00]
Leak case 1.75 1.74 1.72 1.79
(3 = did nothing wrong, 1 = something illegal) (0.02) (0.04) (0.05) (0.04)
[0.74] [0.74] [0.74] [0.74]
Alito confirmation 2.34 2.37 2.27 2.38
(3 = should confirm, 1 = should not confirm) (0.02) (0.03) (0.04) (0.04)
[0.65] [0.65] [0.67] [0.63]

Specific issue index 0.02 0.03 -0.03 0.04

(standardized values of above 5?higher scores conservative) (0.02) (0.03) (0.04) (0.04)
[0.67] [0.67] [0.70] [0.65]
Republican favorable 2.47 2.50 2.41 2.48
(4 = very favorable, 1 = very unfavorable) (0.03) (0.05) (0.06) (0.06)
[1.01] [1.02] [1.00] [1.02]
Democrat unfavorable 2.62 2.63 2.57 2.65
(4 = very unfavorable, 1 = very favorable) (0.03) (0.05) (0.06) (0.05)
[0.95] [0.97] [0.95] [0.93]

Bush approval 2.43 2.49 2.37 2.41

(4 = strong approval, 1 = strong disapproval) (0.04) (0.06) (0.08) (0.08)
[1.32] [1.31] [1.31] [1.34]

Conservatism 4.51 4.56 4.38 4.58

(7 = extreme conservative, 1 = extreme liberal) (0.05) (0.07) (0.09) (0.08)
[1.45] [1.43] [1.50] [1.44]

Broad policy index 0.00 0.03 -0.06 0.02

(standardized values of above 4?higher scores conservative) (0.02) (0.04) (0.04) (0.04)
[0.79] [0.79] [0.79] [0.78]

% knew number dead in Iraq 0.78 0.78 0.78 0.79

(0.01) (0.02) (0.02) (0.02)
10.41] [0.41] [0.42] [0.41]

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
 VOL. I NO. 2 GERBER ETAL.: DOES THE MEDIA MATTER? 43

Table 1C? Summary Statistics for Outcome Measures (Continued)

All Control Post Times

(1) (2) (3)0.70(4)
0.74 0.75
0.75
% identified Libby as involved in leak
(0.01) (0.02) (0.03) (0.02)
[0.44] [0.43] [0.46] [0.43]
) identified Miers as Supreme Court nominee 0.78 0.79 0.73 0.81
(0.01) (0.02) (0.03) (0.02)
[0.42] [0.41] [0.45] [0.39]
Fact index -0.01 0.01 -0.08 0.04
(standardized values of above 3) (0.02) (0.03) (0.04) (0.04)
[0.74] [0.73] [0.76] [0.71]

Number surveyed?follow-up 1,081 454 309 318

Number merged (administrative voting data) 2,571 1,087 748 736

Notes: Standard errors in parentheses; standard deviation in brackets. Number of observations varies due to refusal
to answer or "no opinion" responses.
a Sample frame for the administrative data outcomes in rows 2 and 3 include all individuals for whom we suc
cessfully matched state administrative data with the baseline, whereas all other rows include only the individuals
who completed the follow-up survey.

favorability ratings for Bush, the Republicans, the Democrats, support for Supreme
Court nominee Alito), and knowledge of news events (e.g., does subject know the num
ber of Iraq war dead, has subject heard of I. Lewis "Scooter" Libby).
For analyzing the effects on voter turnout, we also obtained administrative records
of individual voter turnout and successfully merged these data for 2,571 (76.8 per
cent) of the individuals in the baseline (not all individuals in the sample frame were
registered voters).
Table IB shows baseline sample statistics for the subsample of subjects that
completed the post-election survey, and Table 1C shows the summary statistics for
all outcome measures analyzed in further tables. Table IB shows that assignment
to treatment appears orthogonal to all covariates, even after attrition. Appendix
Table Al provides further evidence of this by examining whether the interaction of
baseline covariates and assignment to treatment predicts attrition. We do not find
that either treatment led to a sample selection bias in terms of the characteristics of
individuals who responded to the follow-up survey. Individuals who voted in 2002
and subscribed to a news magazine (hence are more engaged in politics), as well as
those who preferred the Democratic candidate for governor in the baseline, were
more likely to complete the follow-up phone survey. However, this sample selection
bias is not correlated with assignment to treatment (Appendix Table Al, column 2).
Regardless, all results we present later in the paper include these same baseline
covariates in order to address potential bias from the sample response. If there is
attrition based on unobserved variables that are correlated with the outcome mea
sures but not predicted by the observables, our results may be biased.
One limitation of this study is that while we know which households received
newspapers, we cannot be sure that the newspapers were read. Our follow-up sur
vey provides three measures of the effect of newspaper provision on newspaper
reading: whether subjects receive a newspaper, which newspaper they receive, and
the frequency with which they read a newspaper. Table 2 shows the relationship
between treatment assignment and five distribution and readership outcomes. All

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
 44 AMERICAN ECONOMIC JOURNAL: APPLIED ECONOMICS APRIL 2009

Table 2?Paper Distribution and Readership

(OLS)

Frequency reads a
Receives Receives Receives paper (0 = Never, Reads at least
a newspaper the Post the Times 3 = Every day) several times/week
(5)
_0)_(2) (3)_(4)
Panel A: Separate treatment effects estimated for Washington Post an d Washington Times
Washington Post treatment 0.287 0.344 -0.006 0.151 0.089
(0.035) (0.035) (0.021) (0.084) (0.038)
Washington Times treatment 0.100 0.031 0.133 0.086 0.057
(0.034) (0.035) (0.021) (0.083) (0.037)
Adjusted/?2 0.32 0.18 0.19 0.24 0.18
F-test: Post = Times 24.76 66.96 37.81 0.51 0.63
p-value 0.00 0.00 0.00 0.48 0.43
Panel B: Pooled treatment effect estimated for receiving either newspaper
Received either Post or Times 0.191 0.183 0.066 0.118 0.072
treatment (0.029) (0.031) (0.018) (0.070) (0.032)
Adjusted/?2 0.30 0.11 0.15 0.24 0.18
Observation counts for both panels
Observations 1,080 1,080 1,080 1,075 1,075
Refused/no opinion 1116 6
Total surveyed in follow-up 1,081 1,081 1,081 1,081 1,081
Notes: Standard errors in parentheses. Dependent variables are dummy variables based on responses to the
post-election survey. We include the following covariates: gender; reported age; three separate indicators for vot
ing in the 2001, 2002, and 2004 general elections; an indicator for whether the respondent was drawn from a con
sumer list; self reports of receiving any news or political magazines; baseline survey self reports of preferring the
Democratic candidate in the gubernatorial election and having no preference in the gubernatorial election; and an
indicator for wave of the study. If a covariate value was missing, an indicator variable was included and the cova
riate was coded as zero. We include strata indicators, variables for each strata formed prior to the randomization,
which included unique combinations of the following: intention to vote, receive a paper (non-Post/non-Times),
mentions ever reading a paper, gets a magazine, and asked whether they wish they read the paper more. We also
include surveyor/date indicators, a set of indicator variables for each unique combination of surveyor, and date
for the follow-up survey.

pooled treament effect estimates are statistically significant at the .01, .05, or .1 level
and the pattern of results matches the subject treatment group assignment (i.e., the
Post treatment group reported receiving the Post and not the Times, and the Times
treatment group reported receiving the Times and not the Post). The coefficients,
however, suggest less than full readership of those in the treatment groups. No doubt
some of this is due to individuals ignoring the free subscription altogether, but it may
partly be due to the wording of the question. Individuals may have answered "no" to
"receiving a newspaper" because they did not see their free trial as a true subscrip
tion. Additionally, questions about whether one reads the paper may have been inter
preted as inquiring about typical rather than very recent behavior. We also found in
the baseline survey that some respondents reported not receiving a newspaper, when
in fact the newspaper was delivered to them regularly. There is further evidence that
the newspapers were not disregarded. The Post informed us that, as of March 2006
(three months after the free subscription ended), approximately 17 percent of the
treatment group had decided to subscribe to the Post}0

The Times did not provide us the comparable resubscription figure.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
VOL. 1 NO. 2 GERBER ETAL.: DOES THE MEDIA MATTER? 45

Table 3?Effect of Treatment on Political Knowledge and Attitudes

(OLS)

Fact Accuracy Index Specific Issue Index Broad Policy Index

(higher is more accurate) (higher is conservative) (higher is conservative)
(1) (2) (3)
Panel A: Separate treatment effects estimated for Washington Post and Washington Times
Washington Post treatment -0.023 -0.045 -0.085
(0.057) (0.049) (0.055)
Washington Times treatment 0.047 -0.027 -0.051
(0.056) (0.048) (0.054)
Adjusted R2 0.16 0.25 0.32
F-test: Post -- Times 1.32 0.12 0.33
p-value 0.25 0.73 0.57
Panel B: Pooled treatment effect estimated for receiving either newspaper
Received either Post or Times 0.013 ?0.036 -0.068
treatment (0.047) (0.041) (0.046)
Adjusted R2 0.15 0.25 0.32
Observation counts for both panels
Observations 1,080 1,081 1,076
Refused/no opinion 1 0 5
Total surveyed in follow-up 1,081 1,081 1,081

Notes: Standard errors in parentheses. Dependent variables are indexes constructed by summing the standard
deviations from the mean for each of the specific questions for each index. See Appendix Table A2 for regressions
showing the treatment effects for each question used in the construction of the indices. The Fact Accuracy Index
is based on responses to three factual questions (identified number dead in Iraq in a closed-ended question, iden
tified "Scooter" Libby from a list of four individuals as Dick Cheney's chief of staff who recently resigned, iden
tified Harriett Miers from a list of four individuals as a recent female US Supreme Court nominee). The Specific
Issue Index is based on five questions on political issues (three questions about Iraq and the war, a question on the
Plame leak, a question about the Alito confirmation). The Broad Policy Index is based on four questions about atti
tudes toward the political parties, President Bush, and ideological self-placement on a 7 point scale.
We include the following covariates: gender; reported age; three separate indicators for voting in the 2001,
2002, and 2004 general elections; an indicator for whether the respondent was drawn from a consumer list; self
reports of receiving any news or political magazines; baseline survey self reports of preferring the Democratic
candidate in the gubernatorial election and having no preference in the gubernatorial election; and an indicator
for wave of the study. If a covariate value was missing, an indicator variable was included and the covariate was
coded as zero. We include strata indicators, variables for each strata formed prior to the randomization, which
included unique combinations of the following: intention to vote, receive a paper (non-Post/non-Times), mentions
ever reading a paper, gets a magazine, and asked whether they wish they read the paper more. We also include
surveyor/date indicators, a set of indicator variables for each unique combination of surveyor, and date for the
follow-up survey. All results remain qualitatively similar, and statistical significance remains as-is, using probit
or ordered probit specifications instead of OLS.

II. Results

We measured the effect of the newspapers on political knowledge and attitudes

(Table 3) and voting behavior (Table 4). All models include the baseline covariates,
and fixed effects for strata, survey date, and surveyor.
First, we examine whether the treatments led to greater knowledge of political
issues in the news. The dependent variable in Table 3, column 1 is an index of
whether the respondent correctly answered three factual questions. The effects on
the individual questions that comprise the indices are shown in Appendix Table A2.
Neither of the newspapers improved the subject's ability to answer factual questions
about the recent news, and no effect is found in aggregate. Table 3, columns 2 and
3 report two indices of political attitudes. Column 2 is a "Specific Policy" index of

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
 46 AMERICAN ECONOMIC JOURNAL: APPLIED ECONOMICS APRIL 2009

Table 4?Effect of Treatment on Voting Behavior in Virginia Governors Race

(OLS)

Voted for Democrat Voted for Democrat

Voted in 2005 Voted in 2005 Voted in 2006 (set to missing (set to zero
election3 election5 electionb if did not vote)a if did not vote)a
_0)_(2)_(3)_(4)_(5)
Panel A: Separate treatment effects estimated for Washington Post and Washington Times
Washington Post -0.001 0.011 0.025 0.112 0.072
treatment (0.033) (0.019) (0.019) (0.045) (0.035)
Washington Times 0.005 -0.006 0.031 0.074 0.060
treatment (0.033) (0.019) (0.020) (0.045) (0.035)
Adjusted/?2 0.21 0.39 0.31 0.31 0.26
F-test: Post = Times 0.03 0.65 0.10 0.58 0.09
p-value 0.86 0.42 0.75 0.44 0.76
Panel B: Pooled treatment effect estimated for receiving either newspaper
Received either Post 0.002 0.003 0.028 0.093 0.066
or Times treatment (0.028) (0.016) (0.016) (0.038) (0.029)
Adjusted/?2 0.21 0.39 0.31 0.31 0.26
Observation counts for both panels
Observations 1,079 2,571 2,571 718 1,003
Refused/not asked 2 363 78
Total not merged 776 776
(columns 2 and 3)
Total surveyed in 1,081 1,081 1,081
follow-up

Notes: Standard errors in parentheses. The following covariate variables are included in all specifications: gender;
reported age; three separate indicators for voting in the 2001, 2002, and 2004 general elections; an indicator for
whether the respondent was drawn from a consumer list; self report of receiving any news or political magazines;
baseline survey self reports of preferring the Democratic candidate in the gubernatorial election and having no
preference in the gubernatorial election; and an indicator for wave of the study. If a covariate value was missing,
an indicator variable was included and the covariate was coded as zero. We include strata indicators, which are
variables for each strata formed prior to the randomization, which included unique combinations of the following:
intention to vote, receive a paper (non-Post/non-Times), mentions ever reading a paper, gets a magazine, and asked
whether they wish they read the paper more. All results remain qualitatively similar, and statistical significance
remains as-is, using probit specifications instead of OLS.
Data source:
a Survey.
b Administrative voting records.

five specific issues (most important problem in America, most important problem in
Iraq, progress in Iraq, investigation regarding Plame, and the Alito confirmation).
Column 3 is a "Broad Policy" index of four broad political views (Bush's approval
rating, Republican favorable rating, Democrat favorable rating, and respondent's
reported political ideology). For each of these indices, more conservative is a higher
number.
The Post and Times treatment had no significant effect on shifting either the
Broad Policy index toward the Democrats (Post coefficient of -0.085 std, p < 0.12;
Times coefficient of -0.051 std, p < 0.34; Post and Times groups, -0.068 std, p <
0.14) or the Specific Policy index (Post coefficient of -0.045 std, p < 0.35; Times
coefficient of -0.027 std, p < 0.57). Considering the newspaper treatment groups
together, we also find negative (toward liberal) point estimates, but they are not sta
tistically significant.
Table 4 shows the effect of the newspapers on voter turnout (self-reported and
from administrative data) and which candidate individuals voted for or preferred.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
 VOL. 1 NO. 2 GERBER ETAL. : DOES THE MEDIA MATTER? 47

The first set of regressions measure the turnout effects. There was no effect on either
self-reported or administratively measured turnout for the 2005 election (panel B,
column 1: coefficient for self-reported outcome is 0.2 percent, s.e. = 2.8 percent;
column 2: coefficient for administrative outcome is 0.3 percent, s.e. = 1.6 percent).
In November 2006 (column 3), however, the administrative data show a 2.8 percent
age point higher voter turnout, with a standard error of 1.6 percentage points (p <
0.11). It is surprising to see a result in 2006 but not in 2005. This could be a result of
the post-election receipt of the remainder of the ten-week newspaper subscriptions
or the treatment that resulted from the 17 percent of the Post treatment group who
renewed their subscription after the free period ended.
The dependent variables in columns 4 and 5 of Table 4 are dummy variables equal to
one if the subject reported voting for the Democratic candidate in the 2005 gubernato
rial election. In column 4, the sample is restricted to those who reported voting, while
column 5 includes all respondents (coding those who did not vote at all as zero, i.e., as not
voting for the Democratic candidate). The newspapers had an effect on which candidate
the subject supports. Getting the Post is estimated to increase the probability of selecting
the Democrat by 11.2 percentage points (p < 0.014) among those who reported voting
(column 4) and by 7.2 percentage points (p < 0.043) across all respondents. Contrary to
initial expectations, the right leaning Times was also associated with an increase in the
probability of a Democratic vote in the Virginia governor's race. The effect was about %
as large as that estimated for the Post treatment (7.4 percentage points and p < 0.10, and
6.0 percentage points and p < 0.084, respectively for columns 4 and 5). The difference
between the Post and Times point estimates is not statistically significant.

III. Conclusion

Our investigation of the effect of newspapers on political attitudes, behavior, and

subject knowledge of news events found that even short exposure to a daily newspa
per appears to influence voting behavior and may affect turnout behavior.
Contrary to our expectations, despite the slant of the newspapers the effects were
qualitatively similar for the Post and the Times. One explanation for our findings is
the particular news environment, which was politically challenging for Republicans.
During the period that the subjects received the papers, Bush's approval rating fell from
approximately 40 to 37 percent nationwide.11 There was a clear difference in the way
a right-leaning paper and a left-leaning paper covered the news, but both papers cov
ered war casualties and political controversies such as the Plame investigation and the
widely criticized Miers Supreme Court nomination. It may be that what the coverage
had in common was more important than any differences between the newspapers.
A second explanation for why the Times and Post had similar effects may be that
the Democratic candidate for governor was a conservative leaning Democrat, and
thus even though the Times endorsement went to the Republican, the articles and

11 These numbers are based on an average of the polls compiled by the Web site www.pollingreport.com. The
average percentage of respondents approving of Bush's job performance using all polls in the field with starting
dates between October 12-October 18 (TV = 4 polls) was 40.0, starting October 19-October 25 (N = 3) was 41.3,
and starting November 9-November 16 (N = 3) was 37.3.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
48 AMERICAN ECONOMIC JOURNAL: APPLIED ECONOMICS APRIL 2009

opinions put forward were not heavily against the Democratic candidate. A third
explanation is sampling error. Given the 4 percentage point standard errors for each
treatment group, it is possible that there are meaningful differential effects, consis
tent with the news slant of the papers, which we did not detect due to inadequate
power. Hence, while prior beliefs about media and bias should be updated in light of
the evidence we present, it would be useful for future work to obtain more precise
measurement of the differential treatment effects through use of larger samples.
Finally, there is some evidence that getting either newspaper produced an increase
in voter turnout especially in the 2006 national elections. Our finding that turnout
among the treatment groups was about 3 percentage points higher than the con
trol group is consistent with previous work showing positive turnout effects from
newspaper reading (Gentzkow 2006) and suggests newspaper exposure might have
an important long term effect on the level of political interest. However, this result
was nonexistent for the immediate election in 2005 and of only borderline statisti
cal significance for 2006. Therefore, both the existence of an effect and its potential
mechanism should be treated guardedly until further study.
There were also some important outcomes that were not affected by treatments.
There is only limited evidence that the newspapers increased the subject's factual
knowledge of politics. Since conveying facts about politics is a plausible channel
for how the papers might induce change in opinions about candidates and the deci
sion to vote, this null result stands in contrast to the measured effects on candidate
preference and turnout. However, while greater political information among subjects
in the treatment groups would have helped to explain the movement in candidate
preference among the subject groups, changes in opinion often occur without a sub
ject being able to recall the facts that caused opinions to shift. Results of this type
are commonplace in political science. Drawing on work from psychology (Norman
H. Anderson and Stephen Hubert 1963, and W. A. Watts and W. J. McGuire 1964),
political scientists have constructed models of online processing of political infor
mation, where citizens update their judgments in response to the flow of information
but do not retain memory of the particular facts that caused them to revise their
views (Milton Lodge, Kathleen M. McGraw, and Patrick Stroh 1989; Lodge and
Stroh 1993). These models receive substantial empirical support (Lodge, Marco R.
Steenbergen, and Shawn Brau 1995).
Our field experiment directly addresses the problem of selection bias in standard
observational studies. As in all empirical work, experimental or not, there is still the
important question of generalizing from our particular findings. Any broad inferences
from this study to the effects of media bias on political decisions, in general, should
recognize that the results may depend on several specific features of our experiment
such as the political context, choice of subjects, intensity of treatment, length of the
study, timing of the study, and choice of media outlets. Of particular interest would be
to investigate whether the findings we report, which suggest that the common effects
of greater news exposure are of greater consequence than the news slant, generalize to
periods with a more balanced news flow and are confirmed in larger studies that mea
sure differential effects across treatment groups with greater precision. To address these
issues, we suggest that this field experiment approach should be applied in different
political contexts, with different subjects, for different durations, using different media.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
VOL. 1 NO. 2 GERBER ETAL.: DOES THE MEDIA MATTER? 49

Appendix

Table Al?Analysis of Participation in the Follow-Up Survey, Probit

(Dependent variable = 1 if survey successfully completed in follow-up phone call)

_0)_(2)_
Post treatment group 0.003 -0.048
(0.020) (0.046)
Times treatment group 0.018 0.052
(0.020) (0.050)
Female -0.026 -0.040
(0.017) (0.026)
Voted in 2002 0.095 0.103
(0.024) (0.038)
From consumer database sample frame 0.044 0.046
(0.021) (0.032)
Subscribes to news magazine 0.069 0.026
(0.029) (0.043)
Reported preferring Democratic candidate for governor 0.126 0.126
(0.026) (0.040)
Wave 2 of experiment -0.037 -0.035
(0.017) (0.026)
Post x female 0.094
(0.045)
Post x voted in 2002 -0.037
(0.054)
Post x from consumer database sample frame 0.011
(0.050)
Post x subscribes to news magazine 0.053
(0.071)
Post x reported preferring Democratic candidate for governor 0.032
(0.059)
Post x wave 2 of experiment 0.043
(0.043)
Times x female -0.040
(0.040)
Times x voted in 2002 0.014
(0.057)
Times x from consumer database sample frame ?0.018
(0.049)
Times x subscribes to news magazine 0.092
(0.072)
Times x reported preferring Democratic candidate for governor ?0.031
(0.056)
Times x wave 2 of experiment -0.048
(0.039)

Pseudo-fl2 0.018 0.023

Mean dependent variable 0.32 0.32
N 3,347 3,347

Pr (Times interaction variables = 0) 0.23

Ft (Post interaction variables = 0) 0.55

Notes: Standard errors in parentheses. Indicator variable included (but not reported) if gender
information is missing (applicable for 134 observations). All variables (except assignment to
treatment and gender) are from the baseline survey.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
50 AMERICAN ECONOMIC JOURNAL: APPLIED ECONOMICS APRIL 2009

Table A2? The Effect of Treatment on Knowledge and Attitudes

Outcomes on Specific Questions
(OLS)

Panel A: Fact Accuracy Index (Table 3, column I)

Knew number Identified Libby Identified Miers
dead as involved as Supreme Court
in Iraq = 1 in leak = 1 nominee = 1
(1) (2) (3)
Separate treatment effects estimated for Washington Post and Washington Times
Washington Post treatment 0.024 -0.026 -0.035
(0.034) (0.035) (0.033)
Washington Times treatment 0.020 0.006 0.021
(0.033) (0.035) (0.033)

Adjusted R2 0.04 0.10 0.11

F-test: Post ? Times 0.01 0.70 2.43
p-value 0.92 0.40 0.12

Pooled treatment effect estimated for receiving either newspaper

Received either Post or Times treatment 0.022 -0.009 -0.006
(0.028) (0.029) (0.028)

Adjusted R2 0.04 0.11 0.11

Observation counts for both panels

Observations 1,077 1,067 1,074
Refused/no opinion 4 14 7
Total surveyed in follow-up 1,081 1,081 1,081

Panel B: Specific Issue Index (Table 3, column 2)

Most important Most Progress Leak case Alito

problem important in Iraq (3 = no one confirmation
(1 = issue issues in Iraq (1 = very did anything (3 = should
other than (1 = constitution badly, wrong, confirm,
scandals, or Hussein 4 = very 1 = something 1 = should

(4) (5) (6) (7) (8)

0 = scandals) trial) well) illegal) not confirm)

Separate treatment effects estimated for Washington Post and Washington Times

Washington Part treatment 0.024 -0.039 -0.035 0.015 -0.059

(0.023) (0.041) (0.074) (0.066) (0.054)
Washington Times treatment -0.016 -0.017 -0.052 0.020 0.029
(0.022) (0.040) (0.073) (0.062) (0.052)

Adjusted R2 -0.01 -0.17 0.22 0.16 0.18

F-test: Post ? Times 2.68 0.24 0.04 0.00 2.32
/7-value 0.10 0.63 0.83 0.94 0.13
Pooled treatment effect estimated for receiving either newspaper
Received either Post or Times 0.003 -0.028 -0.044 0.017 -0.013
treatment (0.019) (0.034) (0.062) (0.053) (0.045)

Adjusted R2 -0.01 0.17 0.22 0.16 0.18

Observation counts for both panels
Observations 1,074 982 1,042 899 971
Refused/no opinion 7 99 39 182 110
Total surveyed in follow-up 1,081 1,081 1,081 1,081 1,081

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
VOL. 1 NO. 2 GERBER ETAL.: DOES THE MEDIA MATTER? 51

Table A2? The Effect of Treatment on Knowledge and Attitudes

Outcomes on Specific Questions
(OLS) (Continued)

Panel C: Broad Policy Index (Table 3, column 3)

Republican Democrat Bush approval Conservatism
favorable unfavorable rating (7 = extreme
(4 = very (4 = very (4 = strong conservative,
favorable, 1 = very unfavorable, approval, 1 = strong 1 = extreme
unfavorable) 1 = very favorable) disapproval) ,:i-n
(9) (10) nn
liberal)
(11) (12)
Separate treatment effects estimated for Washington Post and Washington Times
Washington Post treatment -0.078 -0.004 -0.148 -0.145
(0.080) (0.073) (0.099) (0.115)
Washington Times treatment -0.119 0.069 -0.146 -0.014
(0.079) (0.072) (0.097) (0.114)

Adjusted R2 0.18 0.23 0.30 0.16

F-test: Post = Times 0.24 0.87 0.00 1.11
/7-value 0.63 0.35 0.98 0.29
Pooled treatment effect estimated for receiving either newspaper
Received either Post or Times -0.099 0.033 -0.147 -0.078
treatment (0.066) (0.060) (0.082) (0.096)

Adjusted R2 0.18 0.23 0.30 0.16

Observation counts for both panels

Observations 1,021 1,022 978 1,033
Refused / no opinion 60 59 103 48
Total surveyed in follow-up 1,081 1,081 1,081 1,081

Notes: Standard errors in parentheses. Dependent variables include factual questions (ability to identify the num
ber dead in Iraq in a closed-ended question, identification of "Scooter" Libby from a list of four individuals as
Dick Cheney's chief of staff who recently resigned, identification of Harriett Miers from a list of four individu
als as a recent US Supreme Court nominee), political opinion quesitons (a closed-ended question about the most
important problem facing the country, a closed-ended question about the most important problems in the Iraq
War, attitudes about the leak case, the Alito confirmation), and attitudes about general national issues (Bush
approval, favorability towards Republicans and Democrats, and Conservatism). We include covariates: gender,
reported age, three separate indicators for voting in the 2001, 2002, and 2004 general elections, an indicator for
whether the respondent was drawn from a consumer list, self report of receiving any news or political magazines,
and baseline survey self reports of preferring the Democratic candidate in the gubernatorial election and having
no preference in the gubernatorial election, and an indicator for wave of the study. If a covariate value was miss
ing, an indicator variable was included and the covariate was coded as zero. We also include strata indicators,
variables for each of the strata formed prior to the randomization, which included unique combinations of the
following: intention to vote, receive a paper (non-Post/non-Times), mentions ever reading a paper, gets a maga
zine, and asked whether they wish they read the paper more. We also include surveyor/date indicators, which are
a set of indicator variables for each unique combination of surveyor, and date for the follow-up survey. All results
remain qualitatively similar, and statistical significance remains as-is, using probit or ordered probit specifica
tions instead of OLS.

REFERENCES

Anderson, Norman H., and Stephen Hubert. 1963. "Effects of Concomitant Recall on Order Effects in
Personality Impression Formation." Journal of Verbal Learning and Verbal Behavior, 2: 379-91.
Ansolabehere, Stephen, and Shanto Iyengar. 1995. Going Negative. New York: The Free Press.
Asher, Herbert B. 2004. Polling and the Public: What Every Citizen Should Know. Washington, DC:
CQ Press.
Besley, Timothy J., and Robin Burgess. 2002. "The Political Economy of Government Responsive
ness: Theory and Evidence from India." Quarterly Journal of Economics, 117(4): 1415-51.
Brians, Craig Leonard, and Martin P. Wattenberg. 1996. "Campaign Issue Knowledge and Salience:
Comparing Reception from TV Commercials, TV News and Newspapers." American Journal of
Political Science, 40(1): 172-93.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms
 52 AMERICAN ECONOMIC JOURNAL: APPLIED ECONOMICS APRIL 2009

Brock, Timothy C. 1965. "Commitment to Exposure as a Determination of Information Receptivity."

Journal of Personality and Social Psychology, 2: 10-19.
Bybee, Carl R., Jack M. McLeod, William D. Luetcher, and Gina Garramone. 1981. "Mass Commu
niation and Voter Volatility." Public Opinion Quarterly, 45(1): 69-90.
Clarke, Peter, and Eric Fredin. 1978. "Newspapers, Television and Political Reasoning." Public Opin
ion Quarterly, 42(2): 143-60.
Dalton, Russell J., Paul A. Beck, and Robert Huckfeldt. 1998. "Partisan Cues and the Media: Informa
tion Flows in the 1992 Presidental Election." American Political Science Review, 92(1): 111-26.
DellaVigna, Stefano, and Ethan Kaplan. 2007. "The Fox News Effect: Media Bias and Voting." Quar
terly Journal of Economics, 122(3): 1187-1234.
Dyck, Alexander, and Luigi Zingales. 2002. "The Corporate Governance Role of the Media." In The
Right to Tell: The Role of the Media in Development, ed. Roumeen Islam, 107-40. Washington,
DC: The World Bank.
Garramone, Gina M., and Charles K. Atkin. 1986. "Mass Communication and Political Socializa
tion: Specifying the Effects." Public Opinion Quarterly, 50(1): 76-86.
Gentzkow, Matthew. 2006. "Television and Voter Turnout." Quarterly Journal of Economics, 121(3):
931-72.
Gentzkow, Matthew, and Jesse M. Shaprio. 2006. "Media Bias and Reputation." Journal of Political
Economy, 114(2): 280-316.
George, Lisa M., and Joel Waldfogel. 2002. "Does the New York Times Spread Ignorance and Apa
thy?" Unpublished.
George, Lisa M., and Joel Waldfogel. 2006. "The New York Times and the Market for Local Newspa
pers." American Economic Review, 96(1): 435-47.
Gerber, Alan S., and Donald P. Green. 2000. "The Effects of Canvassing, Telephone Calls, and Direct
Mail on Voter Turnout: A Field Experiment." American Political Science Review, 94(3): 653-63.
Green, Donald P., and Alan S. Gerber. 2008. Get Out The Vote: How To Increase Voter Turnout.
Washington, DC: Brookings Institution Press.
Groseclose, Tim, and Jeffrey Milyo. 2005. "A Measure of Media Bias." Quarterly Journal of Eco
nomics, 120(4): 1191-1237.
Hamilton, James T. 2003. All the News That's Fit to Sell: How the Market Transforms Information
into News. Princeton, NJ: Princeton University Press.
Harrison, Glenn W., and John A. List. 2004. "Field Experiments." Journal of Economic Literature,
42(4): 1009-55.
Hibbing, John R., and Elizabeth Theiss-Morse. 1998. "The Media's Role in Public Negativity Toward
Congress: Distinguished Emotional Reactions and Cognitive Evaluations." American Journal of
Political Science, 42(2): 475-98.
Keeter, Scott, Carolyn Miller, Andrew Kohut, Robert M. Groves, and Stanley Presser. 2000. "Con
sequences of Reducing Nonresponse in a National Telephone Survey." Public Opinion Quarterly,
64(2): 125-48.
Lieske, Joel. 1989. "The Political Dynamics of Urban Voting Behavior." American Journal of Politi
cal Science, 33(1): 150-74.
Lodge, Milton, Kathleen M. McGraw, and Patrick Stroh. 1989. "An Impression-Driven Model of
Candidate Evaluation." American Political Science Review, 83(2): 399-419.
Lodge, Milton, Marco R. Steenbergen, and Shawn Brau. 1995. "The Responsive Voter: Campaign
Information and the Dynamics of Candidate Evaluation." American Political Science Review,
89(2): 309-26.
Lodge, Milton, and Patrick Stroh. 1993. "Inside the Mental Voting Booth: An Impression-Driven
Model of Candidate Evaluation." In Explorations in Political Psychology, ed. Shanto Iyengar and
William J. McGuire, 225-63. Durham, NC: Duke University Press.
Miller, Arthur H., Edie N. Goldenberg, and Lutz Erbring. 1979. "Type-Set Politics: Impact of News
papers on Public Confidence." American Political Science Review, 73(1): 67-84.
Morgan, Michael, and James Shanahan. 1992. "Television Viewing and Voting 1972-1979." Elec
toral Studies 11(1): 3-20.
Mullainathan, Sendhil, and Andrei Shleifer. 2005. "The Market for News." American Economic
Review, 95(4): 1031-53.
Sweeney, Paul D., and Kathy L. Gruber. 1984. "Selective Exposure: Voter Information Preferences
and the Watergate Affair." Journal of Personality and Social Psychology, 46(6): 1208-21.
Watts, W. A., and W. J. McGuire. 1964. "Persistence of Induced Opinion Change and Retention of the
Inducing Message Contents." Journal of Abnormal and Social Psychology, 68: 233-41.

This content downloaded from

71.178.44.92 on Wed, 02 Nov 2022 17:27:40 UTC
All use subject to https://about.jstor.org/terms