Joinson&Reips CHB
Joinson&Reips CHB
Joinson&Reips CHB
Computers in
Human Behavior
Computers in Human Behavior xxx (2005) xxx–xxx
www.elsevier.com/locate/comphumbeh
Abstract
Three studies were conducted to examine the effect of personalized salutation and sender
power on signing up to an online survey panel, and subsequent survey response rates. In
the first study, significantly more people joined a panel if addressed by a personalized saluta-
tion. In Study 2, this effect was replicated using an invitation to leave a second panel. In the
final study, a significant salutation effect was found when power of the sender was high, and
not when power of the sender was neutral. It is argued that for this sample, power of audience
and participant identifiability linked to create a compliance-based motivation to join and
maintain membership of an online panel. Implications for the maintenance of online panels,
survey response rates, and the collection of sensitive personal information, are discussed.
2004 Elsevier Ltd. All rights reserved.
1. Introduction
0747-5632/$ - see front matter 2004 Elsevier Ltd. All rights reserved.
doi:10.1016/j.chb.2004.12.011
ARTICLE IN PRESS
2 A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx
inates data entry errors, and virtual researchers can collect data 24 h a day (Reips,
2000, 2002). Comparisons of data collected online and via more traditional modes
tends to confirm that no major differences occur (Krantz & Dalal, 2000; Krantz, Bal-
lard, & Scher, 1997), although computerized methods have proved to be successful at
eliciting sensitive information (Tourangeau, 2004), while reducing socially desirable
responding (Joinson, 1999).
A further advantage of online research is the control it offers researchers to per-
sonalize the research experience of participants. This personalization can take a
number of forms – for example, by providing feedback or study results to volunteers,
by adapting subsequent questions on the basis of previous answers, by pre-filling
items based on an individualÕs records, or by enabling invitations to participate to
be personalized using mail merge techniques.
The impact of such personalization of invitations on response rates and partic-
ipant motivation is not well understood, and research findings are highly equivo-
cal. In part this is due to the various differing manipulations of personalization
used in previous mail survey studies, including the use of handwritten envelopes,
letters, signatures and by varying formal and personal salutations (e.g. Andreasen,
1970; Edwards et al., 2002; Simon, 1967). A number of studies have found an
improvement in response rates if the salutation is ÔDear <personÕs name>Õ com-
pared to ÔDear Sir/Madam’’ (Brennan, 1992), and the personalization of corre-
spondence forms part of the ÔTailored Design MethodÕ (Dillman, 2000) which
aims to maximize response rates. However, Simon (1967) found that response rates
both increased and declined due to personalization in different survey situations,
and Pearson and Levine (2003) found no effect of varied salutation on response
rates in an e-mail invitation to complete an online survey, although they did find
some interactions between age and preferred salutation. Most recently, Heerwegh,
Vanhove, Matthijs, and Loosveldt (in press) and Joinson, Woodley, and Reips (in
press) did find an effect of personalized salutation on response rates to an online
survey.
Clearly then, the impact of personalized salutation on response rates is moder-
ately robust, but there has been little progress in identifying possible reasons for
the variation in results, or indeed why a personalized salutation (at least sometimes)
increases response rates. Dillman (2000), drawing on social exchange theory, argues
that personalization increases the reward of a survey to a participant by making
them feel more important and valued. Since this economic model assumes that the
balance of costs and rewards (and trust in the likely reward in the long term) is
weighed by the potential participant before action, then any increase in reward will
tip the scales toward participation.
Indeed, there is other evidence that manipulations which make the respondent feel
more ÔspecialÕ or responsible will increase response rates. For instance, Barron and
Yechiam (2002) showed that addressing an e-mail request for assistance to a single
addressee (as opposed to five people) led to more responses, and for those responses
to be both more helpful and longer in length. However, explaining this effect does
not depend on social exchange theory, and fits more closely to social psychological
research on helping behavior (e.g. Latane & Darley, 1970).
ARTICLE IN PRESS
A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx 3
Moreover, Andreasen (1970) argues that for participants wishing to protect their
anonymity, increasing a personalized salutation may well reduce the likelihood of
them responding to a survey (although the results of his study were inconclusive).
Joinson et al. (in press) similarly report that using a personalized salutation, while
increasing response rates, also serves to reduce disclosure to a sensitive personal
question. This finding suggests that a personalized salutation may serve to reduce
participantsÕ perception of anonymity. Heewegh et al. (in press) also found some evi-
dence that personalized surveys also tended to be associated with increased socially
desirable responding.
However, this reduction in anonymity might suggest a further reason for the var-
ied impact of personalized salutation on survey response rates. Although in some
cases reduced anonymity may reduce response rates, in others it is plausible that it
will improve response rates. This is because, for an identifiable individual, respond-
ing to a survey may well involve impression management in itself. This is particularly
likely when the status of the sponsor or researcher is high (for example, in employee
surveys, see Reips & Franek, in press) – a technique found to improve response rates
(Brennan, 1990; Edwards et al., 2002). In such cases, acting in a socially desirable
manner – effectively to please the researcher – is more likely to occur. As such, we
would expect that a personalized salutation will increase response rates only when
the power or status of the requestor is high.
In the present paper, three studies that examine the impact of personalization
of survey invitation on response rates, and possible causes, are presented. All
three studies involve the use of e-mail to invite responses to either join a panel
of students giving their opinion (Study 1), leave the panel (Study 2), or invitations
to panel members to complete a survey (Study 3). Across all studies, the depen-
dent variable is response rate. The sample is derived from the student body of
the Open University UK – a large (200,000 student) adult distance education
institution.
2. Study 1
2.1. Overview
2.2. Methods
2.2.1. Participants
Participants were a stratified sample of 10,000 Open University students with
e-mail addresses, selected to match demographically the entire undergraduate study
body by gender, age and location. This sample represents 6.3% of the entire student
body.
ARTICLE IN PRESS
4 A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx
2.2.2. Procedure
The sample of 10,000 students was randomly split into four groups of 2500. Each
group was assigned a salutation: ÔDear StudentÕ, ÔDear Open University StudentÕ,
ÔDear ForenameÕ (e.g. Dear John), and ÔDear Forename SurnameÕ (e.g. Dear John
Doe). Each student was sent an e-mail signed by the (female) vice-chancellor of
the Open University offering them the chance to volunteer to join a student survey
panel (called ÔPRESTOÕ). Membership of PRESTO would entail receiving up to
six invitations to Web-based surveys per annum. The subject line of the e-mail
was ÔGive the OU your opinionsÕ, the e-mail sender was ÔOU Electronic Survey
TeamÕ, with the reply address Ô[email protected]Õ. Sign on period was 14 days,
as most responses to e-mail invitations occur within nine days (Welker, 2001).
2.3. Results
Of the 10,000 e-mails, 706 were returned as ÔdeadÕ e-mail accounts (equally spread
across conditions, p > 0.7). The total number of people signing up to the panel was
1405. This represents an overall response rate of 15.12%. A small sub-sample (nine
people) who had not been invited directly onto the panel signed up – presumably be-
cause of either shared family e-mail accounts or automatic forwarding of e-mails. All
these people were addressed as ÔDear StudentÕ or ÔDear Open University StudentÕ.
The percentage of females signing onto the panel (15.1% of total invited) was sig-
nificantly higher than the proportion of males (12.9%) (v2 = 10.23, df = 1, p < 0.01).
The response rates for each salutation condition are shown in Table 1. A Chi-
square test confirmed a significant association between salutation and response rate
(v2 = 24.39, df = 3, p < 0.000).
Odds ratio (OR) analyses found that using the salutation ÔDear JohnÕ significantly
increased the odds of a response compared to ÔDear John DoeÕ (OR = 1.16,
p = 0.056), ÔDear Open University StudentÕ (OR = 1.39, p < 0.001) and ÔDear
StudentÕ (OR = 1.40, p < 0.001). Using the salutation ÔDear John DoeÕ significantly
increased the odds of a response compared to ÔDear Open University StudentÕ
(OR = 1.20, p < 0.05) and ÔDear StudentÕ (OR = 1.21, p < 0.05).
Table 1
Salutation and response rates
Salutation
Dear Dear Open Dear Dear
Student University Student John Doe John
Number of responses 311 313 366 415
Response rate (from 2500) 12.4 12.5 14.6 16.6
Response rate (adjusted for bounced e-mails) 13.38 13.47 15.75 17.86
ARTICLE IN PRESS
A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx 5
2.5. Discussion
3. Study 2
3.1. Overview
In the second study, salutation was once again manipulated, but this time when
inviting existing panel members to exit the panel. This methodology allows an exam-
ination of the hypothesis that personalized messages are more likely to be responded
to simply because they are more likely to be read or attended to. It is predicted that
fewer people will respond to leave the panel when a personalized salutation is used.
3.2. Methods
3.2.1. Participants
Participants were 2247 students who had signed on to the PRESTO panel 12
months previously. The method to recruit the panel members was the same as used
in Study 1 for the second panel.
3.2.2. Procedure
The sample of 2247 students was randomly split into four groups of 562 (with
Dear John Doe being sent to 561). Each group was assigned a salutation: ÔDear Stu-
dentÕ, ÔDear Open University StudentÕ, ÔDear ForenameÕ (e.g. Dear John), and ÔDear
ARTICLE IN PRESS
6 A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx
Forename SurnameÕ (e.g. Dear John Doe). Each student was sent an e-mail signed by
the vice-chancellor of the Open University offering them the chance to leave the
PRESTO student survey panel. The subject line of the e-mail was ÔYour PRESTO
panel membershipÕ. The e-mail sender was ÔOU Electronic Survey TeamÕ, with the
reply address Ô[email protected]Õ.
3.3. Results
Of the 2247 e-mails sent, 146 were ÔdeadÕ, leading to a sample of 2101. The number
of dead e-mails was evenly distributed across conditions (p > 0.72). A total of 103
people opted to leave the panel (response rate: 4.9%). Response rate by salutation
is shown is Table 2. A Chi-square found no significant association between saluta-
tion and response (v2 = 3.14, (df = 3), p > 0.3), although this is most likely due to
the small number of people choosing to leave the panel.
The personalized (Dear John, John Doe) and impersonal (Dear Student, Open
University Student) groups were combined to create a two level factor. Using this
two level factor, the odds-ratio of salutation on leaving was 1.416 (v2 = 2.93,
p = 0.054). This means that using a non-personalized salutation increases the chances
of a person leaving the panel by 1.4 times.
Analyses of the average age and gender distributions of those choosing to remain
versus leave the panel revealed no significant differences between the two groups.
Age (t = 0.29, p > 0.75, means 41.9 and 42.2 for leavers and non-leavers respec-
tively) and gender (t = 1.59, p > 0.10, percentage females were 59.8% in the leavers
group, 51.8% in the non-leavers group). A logistic regression found a non-significant
effect of the model (salutation, age, gender: p = 0.11), and a marginally significant
influence of salutation (Wald = 3.57, p = 0.06) and gender (Wald = 2.71, p = 0.1),
but not age (Wald = 0.41, p > 0.5).
3.4. Discussion
The partial replication of Study 1 strongly suggests that higher response rates in
the personalized salutation condition found in Study 1 were not due to a higher like-
lihood of a message being read or actively attended to. If this were the case, then we
would expect to find a higher number of people opting to leave the panel when a per-
sonal salutation is used – while in the present study, the opposite effect was found.
Although the differences in responses do not reach significance due to the small over-
all response rate, using a personalized salutation reduced opt-outs by around a third.
Table 2
Salutation and response rates (leaving the panel)
Salutation
Dear Student Dear Open University Student Dear John Doe Dear John
Number of responses 32 28 21 22
Response rate (%) 5.7 5.0 3.7 3.9
ARTICLE IN PRESS
A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx 7
If more people from the whole sample were minded to opt out, this clearly would be
a substantial effect.
The pattern of responses across the first two studies suggests that a personalized
salutation may be encouraging socially desirable behaviour amongst participants.
Specifically, in choosing to join a panel, or remain a panel member, participants
are acting in a manner likely to please the source of the message. In both cases,
the source was high power – the vice-chancellor (UK equivalent of Principal) of their
university. The combination of personalized salutation and a powerful requestor
may exert compliance pressure on potential responders, in much the same manner
in which a powerful outgroup and identifiability tends to inhibit undesirable behav-
ior (e.g. Reicher & Levine, 1994). In the third study, both salutation and power of
the sender were manipulated to examine whether the effect of personalized salutation
is independent of the power of the sender.
4. Study 3
4.1. Overview
In the third study, the interaction between salutation and power of the sender is
examined using a request to complete a survey sent to members of the PRESTO
panel retained following the opt-out survey in Study 2. It is predicted that the salu-
tation and power/status interact such that the highest response rates will be when a
personalized invitation originates from a high power/status source, the lowest when
an impersonal invitation originates from a neutral power/status source.
4.2. Method
4.2.1. Participants
Participants were the remaining 2137 members of the first PRESTO panel follow-
ing the opt-out conducted during Study 2. The panel was randomly divided into
equal sub-groups (356 per group, one more in the final group) and each group
assigned a condition.
8 A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx
Three questions about studentsÕ identification with the Open University were also
collected at the end of the survey (for use in predicting later response rates, and not
analysed here).
4.3. Results
There were 146 dead e-mails (6.8%), evenly distributed across conditions (p > 0.6).
The survey comprised of a start page and 14 separate HMTL pages. The first ques-
tion was a skip question, meaning that the response rate measure used previously
(first page submitted) is not appropriate since many participants did not submit this
page, but instead ÔskippedÕ to three quarters of the way through the survey. In light
of this, two measures of response rate are possible – the number of people visiting the
start page and clicking the continue button, or those who completed the first page,
i.e. the page skipped to. 1054 participants visited the start page and clicked Ôcon-
tinueÕ, and 961 (48.26%) responded to the first page of questions. Given that
response in the earlier studies was taken as submission of some answers, the more
conservative second measure was used to study response rates.
Drop out was low: 63 of the panel members who began the questionnaire did not
complete it as far as the final non-optional question, and all but 124 completed the
survey (including optional items). There were no differences in drop out across con-
ditions (see Table 3, v2 all ps > 0.10).
Table 3
Drop out by condition (raw n)
Dear John Dear John Doe Dear Student
High power 14 6 16
Neutral power 11 6 10
ARTICLE IN PRESS
A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx 9
Table 4
Power, salutation and response rates (raw and %)
Dear John Dear John Doe Dear Student
High power 190 (53.4) 154 (43.3) 150 (42.1)
Neutral power 166 (46.6) 158 (44.4) 143 (40.1)
p = 0.2, ns). According to Cohen (1988), an effect size of between 0.07 and 0.2 should
be judged to be a small effect (for 2 df). This supports the hypothesis that the effect of
salutation on response is in part dependent upon the power of the source.
4.5. Discussion
As predicted, the highest response rate was when a personalized invitation came
from a high power source and the lowest when an impersonal invitation came from a
neutral power source. Moreover, the effect of salutation was confined to the high
power sources of the invitation. Power itself exerted only a minor (non-significant)
effect on response rates. This pattern of results supports the hypothesis that a com-
bination of personalized salutation and high power source leads to a strategic imper-
ative to respond to a survey.
5. General discussion
Research on the effect of personalization on survey response rates has been some-
what equivocal – although a general improvement in responses to mail surveys has
been recorded, the causes, possible contra-indications and limits of such an effect are
not well understood. The research in the present paper is the first successful investi-
gation of salutation, power and response rates in all electronic surveys.
Across the three studies, the effect of personalized salutation in improving re-
sponse rates and panel retention has been shown to be moderately robust for both
recruitment and maintenance of panel membership and completion of surveys once
a member of a panel. This research is important since general reductions in response
rates have been a concern to survey methodologists (Tourangeau, 2004), and increas-
ingly panels are seen as one mechanism for the improvement of responses generally
(Göritz, Reinhold, & Batinic, 2002). However, online panels can similarly suffer from
low recruitment, retention and response rates (Göritz, 2004), so research that
addresses methods to improve responses is timely and valuable.
There are a number of unresolved issues that arise from the research presented
here that require further consideration. First, the precise nature of the interaction be-
tween a powerful source and personalized salutation on response rates is unclear.
According to models derived from social psychological research on groups (e.g. Rei-
cher & Levine, 1994), it is the potential for censure or reprisal that leads identifiable
individuals to repress undesirable ingroup behaviour. It is unclear if this pattern ap-
plies here, and a number of further studies will need to be conducted to establish if it
ARTICLE IN PRESS
10 A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx
is indeed the case (for instance, by manipulating the risk of censure and personaliza-
tion). An alternative explanation might be somewhat more prosaic – to receive a per-
sonalized salutation from a high power source might simply increase the importance
or seriousness with which any request is treated. However, the data from Study 2
suggests that even if this were possible, this combination would still lead to audi-
ence-led behaviour.
Second, the differences between the two personalized salutations are difficult to
interpret. Although ÔDear JohnÕ would seem to be more ÔpersonalÕ, ÔDear John
DoeÕ would on first sight seem to be a more identifiable salutation. However,
the pattern of results (excepting Study 2) would suggest that ÔDear JohnÕ is the
more influential salutation (particularly when combined with power as in Study
3). One possibility is that the interaction effect between personalized salutation
and power is not related to identifiability. Alternatively, ÔDear John DoeÕ might
be received as the norm for direct mail or Ôimpersonal personalÕ salutations. Future
studies will need to measure identifiability independently (or via social desirability)
in response to salutation to confirm the interpretation proposed in the present
paper.
Third, the present studies have not examined individual characteristics in survey
response patterns. A number of individual differences, specifically the age of the par-
ticipant, their gender and their years as a student, also predicted willingness to sign
up to the panel. Further research should investigate these individual differences in
more detail, with particular attention paid to possible interactions between survey
mode, personalization and other manipulations, and individual characteristics.
Finally, the present series of studies have tended to equate social desirability with
response rates – but people may well respond to surveys for a number of other rea-
sons (e.g. level of interest in the topic). However, whenever a role of the audience in
social behaviour is found, it does tend to be accompanied by an increase in impres-
sion management concerns, and associated socially desirable responding and face-
saving motivations (Joinson, 2005; Paulhus, 1984).
6. Conclusion
The results of the present research replicate previous research on salutation and
response rates conducted in print surveys in an electronic environment, and extend
our understanding of the mechanisms by which personalized salutations improve
response rates. The interaction between salutation and power of the source was con-
firmed – such that a personalized salutation only exerts an effect on response rates
when the source of the invitation is high power/status. One possible reason for this
is that personalized salutations increase peopleÕs sense of identifiability – which when
combined with a high power audience increase socially desirable, strategic behav-
iour. As such, studies which require sensitive information, or evaluation of an entity
associated with the high power source, would be advised to exercise caution when
combining a personalized invitation to participate in research with a high power
source.
ARTICLE IN PRESS
A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx 11
Acknowledgment
This research was supported by the granting of a visiting fellowship to the second
author by the Institute of Educational Technology (Open University, UK).
References
Andreasen, A. R. (1970). Personalizing mail questionnaire correspondence. Public Opinion Quarterly, 34,
273–277.
Barron, G., & Yechiam, E. (2002). Private e-mail requests and the diffusion of responsibility. Computers in
Human Behavior, 18, 507–520.
Brennan, M. (1992). Techniques for improving mail survey response rates. Marketing Bulletin, 3,
24–37.
Brennan, M. (1990). The effect of researcher status on mail survey response rates. Marketing Bulletin, 1,
50–52.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Dillman, D. A. (2000). Mail and Internet surveys: The tailored design method (2nd ed.). New York, NY:
John Wiley & Sons.
Edwards, P., Roberts, I., Clarke, M., DiGuiseppi, C., Pratrap, S., Wentz, R., & Kwan, I. (2002).
Increasing response rates to postal questionnaires: systematic review. British Medical Journal, 324,
1183.
Göritz, A. S., Reinhold, N., & Batinic, B. (2002). Online panels. In B. Batinic, U. Reips, & M. Bosnjak
(Eds.), Online social sciences (pp. 27–47). Seattle: Hogrefe & Huber.
Göritz, A. S. (2004). The impact of material incentive quantity, response quality, sample composition,
survey outcome, and cost in online access panels. International Journal of Market Research, 46,
327–345.
Heerwegh, D., Vanhove, T., Matthijs, K., & Loosveldt, G., (in press). The effect of personalization on
response rates and data quality in web surveys. International Journal of Social Research Methodology:
Theory and Practice.
Joinson, A. N. (1999). Anonymity, disinhibition and social desirability on the Internet. Behaviour Research
Methods, Instruments and Computers, 31, 433–438.
Joinson, A. N., Woodley, A., & Reips, U-R., (in press). Personalization, authentication and self-disclosure
in self-administered Internet surveys. Computers in Human Behavior.
Joinson, A. N. (2005). Internet behaviour and the design of virtual methods. In C. Hine (Ed.), Virtual
Methods: issues in social research on the Internet (pp. 32–34). Berg: Oxford.
Krantz, J. H., Ballard, J., & Scher, J. (1997). Comparing the results of laboratory and world-wide web
samples on the determinants of female attractiveness. Behavioral Research Methods, Instruments, &
Computers, 29, 264–269.
Krantz, J. H., & Dalal, R. (2000). Validity of Web-based psychological research. In M. H.
Birnbaum (Ed.), Psychological experiments on the Internet (pp. 35–60). San Diego, CA:
Academic Press.
Latane, D., & Darley, J. M. (1970). The unresponsive bystander: Why doesnÕt he help?. New York:
Appleton-Century-Crofts.
Pearson, J., & Levine, R. A., (2003). Salutations and response rates to online surveys. In Association for
survey computing fourth international conference on the impact of technology on the survey process, 19
September 2003, University of Warwick, England.
Paulhus, D. L. (1984). Two-component models of socially desirable responding. Journal of Personality and
Social Psychology, 46(3), 598–609.
Reicher, S. D., & Levine, M. (1994). Deindividuation, power relations between groups and the expression
of social identity: The effects of visability to the out-group. British Journal of Social Psychology, 33,
145–163.
ARTICLE IN PRESS
12 A.N. Joinson, U.-D. Reips / Computers in Human Behavior xxx (2005) xxx–xxx
Reips, U.-D. (2000). The Web experiment method: Advantages disadvantages and solutions. In M. H.
Birnbaum (Ed.), Psychological experiments on the Internet (pp. 89–117). San Diego, CA: Academic
Press.
Reips, U.-D. (2002). Standards for Internet-based experimenting. Experimental Psychology, 49(4),
243–256.
Reips, U.-D., & Franek, L., (2004). Mitarbeiterbefragungen per Internet oder Papier? Der Einfluss von
Anonymität, Freiwilligkeit und Alter auf das Antwortverhalten [Employee surveys via Internet or
paper? The influence of anonymity, voluntariness and age on answering behavior]. In: A. Schütz, & J.
Krems, (Hrsg.), Neue Medien im (Arbeits) autag. Sonderheft der Zeitschrift. Wirtschaftspsychologie,
6(1), 67–83.
Simon, R. (1967). Responses to personal and form letters in mail surveys. Journal of Advertising Research,
7, 28–30.
Tourangeau, R. (2004). Survey research and societal change. Annual Review of Psychology, 55(1), 775–802.
Welker, M. (2001). E-mail surveys: Non-response figures reflected. In U.-D. Reips & M. Bosnjak (Eds.),
Dimensions of Internet science (pp. 231–237). Lengerich: Pabst.