Psychological Assessment © 2013 American Psychological Association

2013, Vol. 25. No. 4, 1137-1145 1040-3590/13/$12.00 DOI: 10.1037/a0033323

The Impact of Acquiescence on the Evaluation of Personahty Structure

Beatrice Rammstedt Richard F. Farmer

GESIS-Leibniz Institute for the Social Sciences, Mannheim, Oregon Research Institute, Eugene, Oregon
Germany

Acquiescence, or the tendency to respond to descriptions of conceptually distinct personality attributes with
agreement/affirmation (acceptance acquiescence) or disagreement/opposition (counter-acquiescence), has
been widely recognized as a source of bias that can substantially alter interitem correlations within scales.
Acquiescence is also known to operate differently among some groups of persons; it is, for example, more
pronounced among individuals with less formal education. Consequently, the biasing effects of acquiescence
are of particular concern when the dimensionality underlying the item set of a measure is examined with
representative samples comprised of persons with varying levels of educational attainment and evaluated with
correlation-based statistical methods such as factor analysis. In the present study, we extended our earlier
research by investigating the biasing effect of acquiescence on personality factor structures derived from the
full-scale version ofthe Big Five Inventory (BFI) when administered to a large sample {N = 1,427) selected
to be representative of Germany's adult population. Consistent with previous findings based on a short-scale
version of the BEI, factor analyses of the unadjusted BEI item set failed to replicate the expected Big
Five-factor structure in the low/medium and high educational groups, with distortions in factor structure more
pronounced in the former group. Once acquiescence was controlled in the item responses for both groups,
however, the obtained factor structures were consistent with the Big Five framework. The implications of
acquiescence on the evaluation of the factor structure of personality inventories and for the validity of
personality assessments are discussed.

Keywords: personality assessment. Big Five, factor structure, acquiescence, education

Responses to personality inventory items have historically been verge on the observation that even modest levels of acquiescent
regarded as a joint function of construct-related item content and responding can have substantial effects on item covariations and
non-content-related response sets or styles, including social desir- factor structures (McCrae, Herbst, & Costa, 2C101; Rammstedt,
ability, impression management, faking good or bad, and acquies- Goldberg, & Borg, 2010; Rammstedt cfe Kemper, 2011; Rammst-
cence. Response sets and styles are generally considered problem- edt, Kemper, & Borg, 2013; Soto, John, Gosling, & Potter, 2008;
atic, as they constitute a source of noncontent variance and Ware, 1978; Winkler, Kanouse, & Ware, 1982).
correlated measurement errors. When responses to construct-
related content are confounded with response sets or styles, the
validity of personality assessment can be diminished or substan- Historical Development of the Acquiescence Response
tially undentiined. Style Concept
Acquiescence, or the tendency to respond to descriptions of Within the fields of psychological and educational measure-
conceptually distinct personality attributes with agreement/affir- ment, definitions of acquiescence have shifted over time, as have
mation (acceptance acquiescence) or disagreement/opposition perspectives on the importance of acquiescence on the assessment
(counter-acquiescence), is an example of a response style whose process. Cronbach's (1946, 1950) scholarly reviews are often
relevance to personality assessment has been passionately debated credited as important catalysts for subsequent research investiga-
historically. Accumulating data, however, are beginning to con- tions into the potential biasing influence of response sets on item
responding and test scores. Cronbach (1946, p. 476) deñned a
response set as "any tendency causing a person consistently to give
different responses to test items than he [or she] would when the
This article was published Online First July 1, 2013. same content is presented in different form." Acquiescence, re-
Beatrice Rammstedt, GESIS-Leibniz Institute for the Social Sciences, garded by Cronbach as a specific type of response set, was char-
Mannheim, Germany; Richard F. Farmer, Oregon Research Institute, Eu- acterized as the general tendency to agree with items by selecting
gene, Oregon.
"yes" or "true" response options. His detailed analyses focused
The authors thank Ingwer Borg for assistance with some of the data
analyses reported in this article and Lewis R. Goldberg for his comments
attention on the possibility that factors other than item content
on an early draft of this report. substantially influence responding, that individual differences in
Correspondence concerning this article should be addressed to Beatrice response sets such as the tendency to agree or guess affirmatively
Rammstedt, GESIS-Leibniz Institute for the Social Sciences, P.O. Box 12 (i.e., acquiescence in his framework) were substantial and reliable,
21 55, Mannheim D-68072, Germany. E-mail: beatrice.rarrunstedt@ and that biases associated with response sets can limit the validity
gesis.org of a test if not statistically controlled.
1137
1138 RAMMSTEDT AND FARMER

In their subsequent review of research on response sets, Jackson 2008; McCrae et al., 2001) can, for example, substantially alter
and Messick (1958) distinguished between the content of items covariations among item pairs regardless of item content or the
and the style of responding. Whereas content referred to the topic direction of response keying (Bentler et al., 1971; McCrae et al.,
addressed by the item, style referred to one's characteristic mode 2001; Soto et al, 2008; Winkler et al., 1982), whereby positive
or style of expression. In making this discrimination, Jackson and correlations become greater in magnitude in the positive direction
Messick also distinguished response style from response set, with and negative correlations decrease in size by becoming less neg-
the former referring to potentially important modes of responding ative. The effect of acquiescence on item covariations can also
and the latter denoting potential sources of measurement error that spuriously infiate the internal consistency of items sets and thus
should be minimized or eliminated. Acquiescence was regarded as imply greater content-related measurement accuracy than is war-
a response style and defined by Jackson and Messick in bipolar ranted.
terms as the tendency to either agree/endorse or disagree/oppose Individual differences in acquiescence can also differentially
items regardless of item content. Controversially, they also as- infiuence scale scores (e.g., infiate them among item acceptors),
serted that most of the major factor analytically derived dimen- thus presenting some challenges for test interpretation (Bentler et
sions from personality inventories were largely infiuenced by al., 1971; McCrae et al., 2001). Similarly, when scores on predic-
response styles rather than item content. tor and criterion measures are each confounded with acquiescence,
Jackson and Messick's (1958) suggestion of the centrality of the resultant models might suggest greater predictive accuracy
response style as a major determinant of item responding stimu- related to content than is actually justified. Tests of group equiv-
lated considerable debate and dissent, most notably from Rorer alence in construct measurement may also be biased when item
(1965) and Block (1965, 1971, 1972). These commentators responses or scale scores are more strongly confounded with
pointed to a collection of item-reversal studies that demonstrated acquiescence in one of the comparison groups (e.g., among those
that when the keying of inventory items was reversed, responses to with less education, as is discussed later; see also Ware, 1978).
questionnaire items were largely consistent across original and With respect to the primary focus of the present research,
reverse-keyed item sets. Such findings implied that the content of acquiescence can also present significant problems when evaluat-
the item was the primarily responsible for consistency in item ing the underlying factor structure of a measure, either during the
responding, not a response style. Specifically, acquiescence— scale development process or during assessment of the underlying
when conceptualized as the tendency to respond "yes" or "true" to theories of personality inventories. Considerable distortions in
test items (e.g., Cronbach, 1946, p. 479) or as a "a tendency to factor structure can result (McCrae et al., 2001; Rammstedt et al,
select some response category a disproportionate amount of the 2010; Rammstedt & Kemper, 2011; Rammstedt et al., 2013; Soto
time independently of the item content" (Rorer, 1965, p. 134)— et al., 2008; Ware, 1978; Winkler et al., 1982), even when acqui-
was shown to have minimal infiuence on responding. escence has a relatively small effect on individual item responses
Bentler, Jackson, and Messick (1971) subsequently conceded (McCrae et al., 2001). When item responses are strongly infiu-
that the tendency to consistently endorse the same response cate- enced by method, for example, resultant factor dimensions may be
gory, which they termed agreement acquiescence, had at best only primarily infiuenced by method-related features such as the direc-
a minor influence on responses to personality items. They sug- tion of item keying in unbalanced item sets, whereby the highest
gested, however, that acceptance acquiescence, or the tendency to loading items on one factor are of a positive sign and the corre-
endorse or deny conceptually distinct personality attributes as sponding loadings on another factor are of a negative sign (Jackson
self-descriptive, constitutes a personality variable that can exert & Messick, 1958; McCrae et al., 2001; Winkler et al., 1982).
considerable infiuence on item responding. Whereas agreement
acquiescence in their framework is posited to be response style
The Current Study
based on a true- or agree-response tendency and thus alterable
through item keying, acceptance acquiescence is related to the Perhaps because of the previously noted effects on interitem
tendency to ascribe various conceptually distinct characteristics to correlations, the acquiescence response style might be a central
oneself, regardless of the direction of item keying. With accep- factor underlying failures to replicate important findings in per-
tance acquiescence, then, extreme "item acceptors" would accept sonality research with diverse and representative samples. For
heterogeneous descriptions of personality attributes as "true" of example, recent studies on the predominant dimensional model of
them and respond "false" to items that deny such characteristics, personality structure, the Big Five factor structure, revealed that
whereas extreme "item rejecters" would respond in an opposite self-report assessments of these personality domains are con-
fashion by denying various personality attributes while endorsing founded by acquiescence, particularly among persons with lower
as "true" items that deny or negate such attributes. With respect to levels of educational attainment, resulting in failures to replicate
the present research, the conceptualization of the acquiescence the Big Five factor organization (Rammstedt et al., 2010; Rammst-
response style is most consistent with Bentler et al.'s (1971) notion edt & Kemper, 2011; Rammstedt et al., 2013). These studies, as
of acceptance acquiescence. well as a study involving children and adolescents (Soto et al.,
2008), subsequently demonstrated unequivocal support for the Big
Five model once acquiescence was controlled, with this observa-
Potential Effects of Acquiescence on Personality
tion extended to different assessment modes (self-report and in-
Measurement
terview; Rammstedt & Kemper, 2011) and to persons from differ-
From a measurement perspective, the acquiescent response style ent countries (Rammstedt et al., 2013).
can present several challenges to the validity of personality assess- Although these studies have provided extensive evidence for the
ment. This temporally stable response style (Billiet & Davidov, distorting effects of acquiescence on Big Five factor structures.
 ACQUIESCENCE AND ITEM FACTOR STRUCTURE 1139

they are also limited as they have almost exclusively relied on a sampling methods and study procedures are detailed in Schumann
single abbreviated measure, the BFI-10 (Rammstedt & John, (2004).
2007), a 10-item short-scale version of the widely used Big Five
Inventory (John, Donahue, & Kentle, 1991; see also Benet-
Martinez & John, 1998; John & Srivastava, 1999). Soto et al.
Measures and Procedures
(2008) provided the first evidence that acquiescence also affects Education. The highest level of educational attainment was
the psychometric quality of the full scale BFI. Their study, how- assessed during the interview. Participants were categorized ac-
ever, was based on data collected from children and adolescents cording to classifications consistent with the German educational
(mean age = 16.6 years), with evidence of acquiescence most system into one of the following five groups: no secondary edu-
pronounced prior to age 18. Consequently, it is presently unclear if cation (Group 1; n = 31); lower secondary education (Group 2;
the acquiescence response style is pervasive in the item endorse- n = 653); intermediate secondary education (Group 3; n = 455);
ments of the full scale BFI among adults and, correspondingly, higher secondary education, which is the entrance qualification for
whether acquiescence distorts the factor structure of the 44-item general or applied universities (Group 4; n = 149); and completed
BFI item set. university degree (Group 5; n = 139).
As the extant evidence of the biasing effect of acquiescence on Our previous research (Rammstedt et al., 2010; Rammstedt &
Big Five factor structures is limited to a single abbreviated mea- Kemper, 2011) has produced highly similar results for the factor
stire with adults and a single study with children and adolescents, structure matrices for personality items for the low to medium
the present research was designed to investigate the generalizabil- education groups (i.e.. Groups 1, 2, and 3 in the present study; n =
ity of previously reported findings with the full-scale BFI admin- 1,139); consequently, these groups were collapsed into a single
istered to a large community-based sample of adults with varying group {low/medium education) for data analytic purposes. Simi-
levels of educational attainment. Previous research has also indi- larly, the two highest educational groups (i.e.. Groups 4 and 5;n =
cated that the acquiescence response style is more commonly 288) have also been observed in our earlier studies to produce
observed among persons who have less formal education (Billiet & comparable results and were hence combined into a single group
McClendon, 2000; Rammstedt et al., 2010; Rammstedt & Kemper, {high education).
2011; Ware, 1978; Winkler et al., 1982). Consistent with these Big Five Inventory (BFI). Although not as widely used as the
earlier findings, we hypothesized that (a) the Big Five factor NEO Personality Inventories (Costa & McCrae, 1992) and the
structure would be distorted when uncorrected item scores were International Personality Item Pool Big Five Inventories (Gold-
subjected to a factor analysis; (b) distortion of the Big Five factor berg, 1999; Goldberg et al., 2006), the relatively brief BFI (John et
structures would be more pronounced among those with lower al., 1991,as reprinted in Benet-Martinez & John, 1998; John &
levels of educational attainment; and (c) the expected factor struc- Srivastava, 1999) is nonetheless a popular and widely used measure of
tures would emerge once acquiescence response tendencies were the Big Five model of personality. The BFI consists of 44 short-phrase
controlled at the item level. items, rated on a 5-point scale (from 1 = disagree strongly to 5 =
agree strongly), with each of the Big Five personality domains
(i.e., Extraversion, Agreeableness, Conscientiousness, Neuroti-
Method cism, and Openness) assessed with between eight and 10 items. As
will be described in more detail later, most items in the BFI are
Sampling Methods and Participants balanced in content and keying (32 of 44 items, or 73%), with the
remaining items keyed in a positive direction (i.e., agreement with
Data for this research were collected as part of the Personality the item is an endorsement of the personality characteristic).
and Election Behavior Survey conducted in Germany in 2003 The German version of the BFI used here was carefully trans-
(Schumann, 2004). The sampling procedure targeted adults (age ä lated and adapted to the German language based on translation and
18) randomly selected from cotnmunities spanning all regions of back-translation methods by experts followed by reconciliation.
the country. A two-stage sampling procedure was followed. The resulting German version is fully comparable to the original
Households selected for interview were first randomly identified. English-language version and demonstrates satisfactory psycho-
Following household selection, the Kish grid method (Kish, 1949) metric properties (Rammstedt, 1997; Rammstedt & John, 2005;
was used to randomly select household members for interview. A see also Lang, Liidtke, & Asendorpf, 2001). Internal consistencies
commercial vendor (Marplan) organized the sampling procedure (coefficient alpha) for the items belonging to the Big Five scales in
and collected interview and questionnaire data. Participation in this the present research ranged between .74 (for Neuroticism) and .82
study was voluntary, and no financial compensation was offered. (for Openness), with an average of .79.
As part of this survey, a total of 2,508 persons initially took part Acquiescence response style. In the present study, an overall
in a 35-min in-person interview that largely emphasized the as- indicator of the acquiescent response style was derived from 16
sessment of demographic information, following which partici- matched pairs of items from each of the five personality domains
pants were provided with a packet of questionnaires (including the that are logically opposite in content and balanced in their keying
BFI) to complete on their own and return in a self-addressed and (e.g., "Is talkative" vs. "Tends to be quiet"). Mean acquiescence
stamped envelope. Overall, 57% of the participants initially inter- scores were computed for each participant according to the pro-
viewed {N - 1,427; 48% women; mean age = 49, SD = 17, range = cedures outlined by Soto et al. (2008). The use of balanced
18-90) provided usable demographic data and subsequently com- matched pairs is particularly well suited for the study of acquies-
pleted and returned the questionnaire, and collectively serve as the cence, as this approach involves the evaluation of consistent re-
reference sample for this research. Complete descriptions of the sponding to items that are similar in content but expressed in
1140 RAMMSTEDT AND FARMER

opposite or balanced terms (Bentler et al., 1971; Winkler et al., Results

1982). Acceptance acquiescence would be evident, for example, if
tbe mean score for a pair of balanced items was greater than the Big Five Factor Structures for Low/Medium and
value corresponding to the middle response option (i.e., coded as High Education Groups Based on Unadjusted
3) along the 5-point Likert-response format, indicating that accep-
Item Responses
tance of the positive item is stronger than the rejection of the
negative item. Counter-acquiescence, or nay-saying, would be The 44 unadjusted BFI items were subjected to exploratory
evident if the mean score for a pair of balanced items was below factor analyses, separately for each of the education-based groups.
the value corresponding to the middle response option. In survey For botb subsamples, scree tests suggested factor structures with
research, acceptance acquiescence appears to be considerably six rather than five dimensions (upper half of Figure 1). To keep
more common than counter-acquiescence (Winkler et al., 1982). our analyses aligned with the Big Five model, we extracted five
As operationalized in tbe current research, the acquiescence factors for both education groups, resulting in 47% of the variance
response style is similar to Bentler et al.'s (1971, pp. 190-191) being explained in both subsamples. Tbe varimax-rotated factor
notion of acceptance acquiescence described previously. Because loading matrices for botb education groups are presented within
the BFI does not contain items that are strict negations of specific the first 10 columns of Table 1.
experiences (e.g., "Not happy," "Not sad"), acquiescence on the For the low/medium education group, tbe resulting factor struc-
BFI would be evident when respondents demonstrate a tendency to ture is severely distorted, with no single Big Five dimension
(a) consistently agree with items that describe personality charac- unambiguously replicated. Departures from tbe expected factor
teristics that are logical opposites (acceptance acquiescence) or (b) structures are also evident by the congruence coefficients of the
consistently disagree with personality-descriptive items that are observed factors with the idealized factor structures. Congruence
logical opposites (counter-acquiescence). coefficient values ranged between .53 and .86 for the rotated
Five-Factor solutions, with an overall matrix congruence of .72.
For the high education subsample, the matrix of factor loadings
Statistical Analyses similarly failed to conform to the expected Big Five pattern.
Except for the Agreeableness dimension, one or more items cor-
Ipsatization of individual item responses based on idio-
responding to each of the remaining four personality dimensions
graphically derived acquiescence scores. Ipsatization is a com-
demonstrated greater secondary loadings relative to tbe primary
mon technique for removing variance due to acquiescent response
loading. Overall, however, factor structures for the high education
tendencies from personality questionnaires (Clemans, 1966; Cun-
subsample demonstrated greater correspondence to the idealized
ningham, Cunningham, & Green, 1977; Fischer, 2004; Ostendorf,
factor matrix compared with the low/medium education group.
1990; ten Berge, 1999). In the present research, each respondent's
Factor congruence coefficients ranged from .58 to .93, with an
mean score derived from the 16 matched item pairs was subtracted
overall matrix congruence of .79.
from bis or her score on each single BFI item, including the 12 BFI
items that were not among the 32 items used in the computation of
acquiescence scores. In analyses we will describe, botb unadjusted Individual Differences in Acquiescence Response Bias
BFI item responses and acquiescence-corrected BFI item values
Research that we have reviewed previously suggests tbat level
were used in evaluations of the Big Five factor structure.
of education attainment is related to individual differences in
Factor analyses. To facilitate comparisons across similar acquiescence. We investigated this issue with the present sample
studies, we used the factor analytic procedures that were consistent by comparing acquiescence response tendencies between the two
with the assumptions of the Big Five model and with methods used educational groups. As expected, variance in acquiescence scores
in related studies (Rammstedt et al., 2010; Rammstedt & Kemper, was larger in the low/medium education group compared with the
2011; Rammstedt et al., 2013). Specifically, we conducted a set of high education group, according to Levene's test for the equality of
principal component exploratory factor analyses for which we variances (variances = .11 and .07, respectively), F(l, 1425) =
specified Five-Factor extractions and orthogonal (varimax) simple 4.63, p < .05.
structure rotations. Noncorrected and acquiescence-corrected so-
lutions were generated separately for tbe two education groups,
low/medium and high. Big Five-Factor Structures after Controlling for
Evaluation of factor structure congruence. As advocated by
Acquiescence
Hopwood and Donnellan (2010) for evaluations of tbe internal To evaluate the possible infiuence of acquiescence on the dis-
structure of personality inventories, tbe resultant varimax rotated tortion of the Big Five factor structure, we repeated the factor
factors described were subsequently compared with the idealized analytic procedures used previously, this time replacing unadjusted
Big Five pattern (consisting of -I-1, - 1 , and 0 loadings) by use of item scores with mean-corrected item scores adjusting for acqui-
Procrustes rotation. The similarities between the unadjusted and escence. When based on acquiescence-controlled item values,
acquiescence-controlled five-dimensional factor solutions to the scree tests clearly revealed a five factorial solution for each
idealized patterns were evaluated with the congruence coefficient education-based subsample (see lower half of Figure 1). The
c (Burt, 1948). Guidelines provided by Lorenzo-Seva and ten resultant Five-Factor solutions accounted for 44% and 47% of the
Berge (2006) for interpreting factor structure congruence were explained variance in the low/medium and high education groups,
followed, with congruence coefficients of at least .85 needed to respectively. Varimax-rotated factor loading matrices for botb
infer "fair" replication. groups are presented in the last 10 columns of Table 1, each of
 ACQUIESCENCE AND ITEM FACTOR STRUCTURE 1141

Scree plot Scree plot

raw scores; low and medium educated raw scores: high educated

s 11 13 IS 17 19 21 23 2S 27 29 31 33 35 37 39 4( 43 1 3 5 7 9 11 13 15 17 19 21 23 25 2T 29 31 33 35 37 3S 41 43

Factor Factor

Scree plot Scree plot

mean-corrected scores; tow and medium educated mean-corrected scores; high educated

1 3 5 7 9 11 13 IS 17 19 21 23 25 27 29 31 33 35 37 39 41 43 1 3 5 7 9 11 13 ÎS 17 IS 21 23 25 27 2?) 31 33 35 37 33 41 43

Factor Factor

Figure 1. Scree plots from principal components analyses of raw data for low/medium and high education
groups (Panels a and b, respectively) and mean-corrected data for low/medium and high education groups
(Panels c and d, respectively).

which demonstrates clear and consistent Big Five patterns. Each sures, summarized in McCrae et al. (2001), Rammstedt et al.
BFI item in the acquiescence-controlled analyses demonstrated its (2010), and Hopwood and Donnellan (2010), have occasionally
highest primary loading on the factor to which it conceptually failed to reveal or confirm the expected Five-Factor internal sti-uc-
belonged, with only a single exception (Item 29, "I can be moody," ture. The acquiescent response tendency has been recognized as a
for the high education group). In each instance, congruence coef- potential source of bias that can create difficulties when one is
ficients for single factors equaled or exceeded .85, with an overall evaluating the internal structure of assessment measures, including
matrix congruence of .86 and .87 for the low/medium and high Big Five inventories. Findings irom this study in conjunction with
education groups, respectively. our previous adult-based studies with an abbreviated version of the
BFI (Rammstedt et al., 2010; Rammstedt & Kemper, 2011;
Discussion Rammstedt et al., 2013) and a single study with the full-scale BFI
The internal structure of a personality measure, described by with children and adolescents (Soto et al., 2008) clearly indicate
Hopwood and Donnellan (2010, p. 332) as the "orderiiness of the that acquiescence can have adverse impact on interitem correla-
actual clustering of related elements . . . within and across theo- tions that, in turn, produces distortions in the underlying factor
retically substantive dimensions," can be regarded as an indication structure of a measure. In unadjusted analyses in the current study,
of the plausibility of the psychological theory underlying a scale or considerable cross-loadings among item sets were evident, and no
inventory (see also Loevinger, 1957). Previous exploratory and factor solution met or surpassed the critical congruence benchmark
confirmatory factor analytic studies of Big Five personality mea- of at least .85 with reference to the idealized Big Five factor
1142 RAMMSTEDT AND FARMER

t n n t m .Dr»nrm
ppp—'p— p p p p p p p r) cs
' ''' r' ' ' 'r ' ' ' ' ' ' ' ' ' ' ' ' ' ' ' p p p p p
r ' I i.H roooo^ONCNoo*;i-io^
ppp p p p —CN — — CNrn ^ „ _ p p —• p p —p — o

Tf • * O < n — O4 CN — >C Os ^ ON VO CN l~ 00 VD t~. t - r - M rt CN OMn « CN m t^ O — M CN CN in i/"! CN ^ in ÇS en — OO ^ ON

' '''' r '''' r
— pcNp^-rfcnp p — — p p p — — — r^inr^Nosci^vcTtin — — ppcNCjcNcN p — — p p p p — ppoo
. . . . ^. . ^. ^. ^. . ^. ^. . . . ^. . . ^. . ^. ^. . . . ^. ^. . ^. . . ^. . ^. . . . ^. . ^. ^. . . ^. .

.2
8 I I I I I

I ^ in ^ ^ *# in t
1 c*^ o ^^ ON ^^ csinvo — t^t^oor^r<i
oo2c I — csp p p p p p p p — — rnp
r ' r i'
p —
ooo—'r--cNor-'<tcN^i.
•^CN — c J r n - ^ p c s p p c
' r ' ' i' \' ' \' {
•s i t~~ in CN in o cN ^ '"' r~- cs r^
r '' I I
rrr
I I I I I

I r^ i ^ CN "^ "^
o o o ——
NomoNONOrtin«
rfvoin^NOTTinin

I I

I ^H t^ Tf (
in in >n ^
I I I I I I
r 'r ooo o o.
oocvivo
l p — CN C
o o oC
I I I II'

- f*^ CO ^ 00 o ^-
5 —. o o — —' CN OOOOO
I I I I I I I
ONr-inwoof'ïcNin
8 c*^pr
Ov —
in —vem —m —>o
I I I I r i r I I I
-a O'^ooç4.^c*^rn^ - —O\ —gin — g g
I I I I I I I I I I
!• CN
I
Ö—— <

I I I I I II II
§
^n^QO^O^'Or^CS

I
00 íN ^ t-^ 00 O —' 00 ^ ^ ^D r^ ^ cs o "^ ^ Vi I
O.—'iS'— — o o — o o—'O—'OCSCN'' n-¡priO'^fN004^- IOC
I

ONOONOrnO — —
O ~" o Ô o o íN p p pp5pp' ; p CN p

r ' r ' ' I' r ' r r r r r ' ' r I I I

rr
c§ I 00 m 00 in ^ m oo (^
in p in_ f i — ve — ve
I I r l'
—«inin^HTt^^c^^cN V j í. ^ ^ " ^ " O> O^ O^ O^ ^ ' O\ oo ©^ oo 00 r-- CN '—'CT\'—' O m
OCNCN — — T f O O
S —^So^or^o — 8 o o
1 1 1 1 1 1 1 1 1 1

t ~ - ' * C N N O ' * — CNOO 00 ^- VO 00 •<í O »n O r^ -^ 00

— <N — ^-CN^-CN — £2 12o — cN (N o •—1 es ^^ oo 00

I 1 II II

v^ CO r*^ ^^^ ""^ ^P c^ ""^ f^

1

£S cNin S ? £
1

in ^-' ^ tn ^ r^
's I r
Oo

-5?
T3

•a
II
s •a
§ gg I
to
II o
I 3 's: -K C

I *E°3»§22
I i3 S
# BBS'S « B S -^.
JJ M)
o t«
.5Í
S
"1
c
Si"
5 M
••a 'S
es
° i "u
onië'âS.-sl « O

<§ « .H I
S'a
^ va «5 t/3 H H V) "
j.ü(
• *CT\• * C
— I — I CN C

< U X
 ACQUIESCENCE AND ITEM FACTOR STRUCTURE 1143

solution. Both personality characteristics (content) and method al, 1982). As demonstrated here and elsewhere (Rammstedt et al.,
(noncontent) likely infiuenced item covariations and the resultant 2010; Rammstedt & Kemper, 2011; Rammstedt et al., 2013; Soto
factor structures in these unadjusted analyses. Adjusted analyses, et al, 2008; Winkler et al, 1982), the inclusion of matched item
however, revealed that distortions in factor structure could be pairs within scales also provides a built-in means for assessing and
almost entirely eliminated following corrections for acquiescence controlling acquiescent responding. Even when scales are balanced
as evident by resultant factor structures that were unambiguously in item keying, however, acquiescent responding is still possible
interpretable in terms of the Big Five theory of personality orga- (Ware, 1978), particularly among individuals who are likely to
nization. The recovery of the hypothesized factor structure follow- endorse a broad array of heterogeneous characteristics while op-
ing the control for the acquiescent response style in the adjusted posing statements that negate such qualities (Bentler et al, 1971).
analyses likely resulted from the predominant infiuence of content Statistical control of acquiescence may therefore be necessary in
with minimal infiuence from method on item covariations and some circumstances, even when scales are balanced.
factor structures. Future research into the infiuence of acquiescence on Big Five
The present research also demonstrated that persons with less factor structures or other dimensional models of personality might
formal education display greater variability in acquiescence re- evaluate the extent to which this response style biases responding
sponse tendencies and greater associated distorting effects on across measures. The present study was based on data of the
factor structure compared with those with higher levels of educa- German version of the BFl. Earlier research (Rammstedt et al.,
tional attainment. These results, which are consistent with those 2013) has demonstrated the generalizability of the distorting effect
from other investigations (Billiet & McClendon, 2000; Rammstedt of acquiescence on the BFT across cultures. In need of further
et al., 2010; Ramtnstedt & Kemper, 2011; Rammstedt et al., 2013; evaluation, however, is the extent of the generalizability of acqui-
Ware, 1978; Winkler et al., 1982), suggest that findings frequently escence across different Big Five questionnaires (e.g., the NEO-
obtained with highly educated samples (e.g., university students) PI-R, NEO-FFI, or IPIP inventories) as well as among personality
may not replicate in more representative samples such as we used inventories that assess different models of personality. In addition,
here unless acquiescence response tendencies are measured and further research is needed to investigate the generalizability of the
controlled. Despite greater acquiescence variability among the effects across different response formats (e.g., a 4-point Likert
low/medium education group, for example, the expected Big Five format instead of the 5-point scale utilized here that included a
factor structures were revealed with near-textbook-like clarity neutral response option), and to corroborate limited reports that
once items were corrected for this response style. suggest some response option formats more frequently occasion
Developers of personality scales and inventories can take steps acquiescent responding than others (Hui & Triandis, 1989).
to minimize the potential impact of acquiescence during the initial Future research might also seek to illutninate psychological
stages of scale development. During the item-writing phase, for processes that underlie the tendency to be acquiescent and to
example, scale developers should avoid items that are ambiguous, identify when and among whom acquiescence is more probable.
abstract, and moderately socially desirable content, as these item Relatively little is presently known, for example, about individuals
features may increase the likelihood of acquiescent responding who are especially prone to acquiescence. In addition to being
(Cloud & Vaughan, 1970; Jackson & Messick, 1958). During the more common among persons with less formal education, acqui-
item selection and retention process, items might be considered for escence appears to be more frequent among persons who produce
exclusion if they demonstrate relatively large associations with lower IQ scores (Gudjonsson, 1990; Gudjonsson & Young, 2011)
acquiescence, as was done during the development of the NEO or who reside at either end of the age continuum (Billiet &
inventories (McCrae et al., 2001). McClendon, 2000; Soto et al, 2008; Winkler et al, 1982). In
Some test developers have included in their inventories scales accounting for age differences among youth, for example. Soto et
that explicitly measure response styles. One such example is the al. (2008) suggested that acquiescence might be less common
True Response Inconsistency (TRIN) scales of the Multidimen- among persons with more clearly developed and differentiated
sional Personality Questionnaire (Tellegen, 1982), the Minnesota self-concepts who, in turn, have greater certainty about their pref-
Multiphasic Personality Inventory-2 (MMPI-2; Butcher et al., erences, opinions, and personal experiences. Racial and cultural
2001), and MMPI-2-Restructured Form (MMPI-2-RF; Ben- factors also appear to be related to acquiescence (Bachman &
Porath & Tellegen, 2008). The TRIN scales consist of item pairs O'Malley, 1984; Hui & Triandis, 1989; Johnson, Kulesa, Cho, &
that contain semantically opposite content and are particularly Shavitt, 2005; Marin, Gamba, & Marin, 1992; Rammstedt et al,
useful for identifying persons whose responses are highly con- 2013; Winkler et al, 1982). Smith (2004), for example, has sug-
founded with acquiescence responding. McCrae et al. (2001) pro- gested that acquiescence is a style of communication that is more
vided examples in which TRIN scale scores were used to control common in cultures high on family collectivism or whose mem-
for acquiescence prior to factor analysis as a method for producing bers express preferences for highly structured lives with few
unconfounded factor structures. Alternatively, researchers could opportunities for uncertainty. Examinations of individual and cul-
also simply elect to discard or deem as invalid data supplied by tural differences in acquiescence can identify groups of persons
individuals who score high on acquiescence measures such as more or less inclined to utilize this response style and stimulate the
TRIN, although this approach may be less successful in eliminat- development of testable theories of acquiescent responding.
ing the overall infiuence of acquiescent responding than statistical Finally, to establish the broader implications of acquiescence on
approaches such as partialling (Winkler et al., 1982). the validity of personality assessment, additional investigations are
Another approach for minimizing acquiescence is to balance needed on the effects of acquiescence beyond those related to
item content and the keying of scales (Cloud & Vaughan, 1970; factor structures. Similar to studies on the effects of social desir-
McCrae, Costa, & Martin, 2005; McCrae et al., 2001; Winkler et ability in personnel selection (e.g.. Ones & Viswesvaran, 1998),
1144 RAMMSTEDT AND FARMER

acquiescence could also be examined as putative predictor, mod- Clemans, W. V. (1966). An analytical and empirical examination of some
erator, mediator, or suppressor variables with respect to criterion properties of ipsative measures. Richmond, VA: Byrd Press.
behaviors of interest. Evidence has been presented, for example, Cloud, J., & Vaughan, G. M. (1970). Using balanced scales to control
which suggests that the importance of social desirability relative to acquiescence. Sociometry, 33, 193-202. doi: 10.2307/2786329
these various roles is, at best, modest and relatively trivial in Costa, P. T., & McCrae, R. R. (1992). Revised NEO Personality Inventory
(NEO PI-R) and NEO Five Factor Inventory: Professional mantMl.
personnel selection contexts (Ones & Viswesvaran, 1998; Ones,
Odessa, FL: Psychological Assessment Resources.
Viswesvaran, & Reiss, 1996). It is presently unclear if acquies-
Cronbach, L. J. (1946). Response sets and test validity. Educational and
cence would similarly display negligible importance in relation to
Psychological Measurement, 6, 475-494.
these roles in various personality assessment contexts.
Cronbach, L. J. (1950). Further evidence on response sets and test design.
In conclusion, the present research and our earlier related find- Educational and Psychological Measurement, 10, 3-31. doi:10.1177/
ings suggest that acquiescence can have deleterious effects on the 001316445001000101
internal structure of personality measures. When not taken into Cunningham, W., Cunningham, 1. C. M., & Green, R. T. (1977). The
account or controlled, the distorting effects of acquiescence on ipsative process to reduce, response set bias. Public Opinion Quarterly,
internal structure can, in turn, produce uncertainty about the utility 41, 379-394. doi: 10.1086/268394
of a inventory, raise questions about the validity of the personality Fischer, R. (2004). Standardization to account for cross-cultural response
theory underlying a measure, pose problems in interpretation of bias: A classification of score adjustment procedures and review of
summary scores, or create difficulties in evaluations of correlations research in JCCP. Journal of Cross-Cultural Psychology, 35, 263-282.
between summary scores and external predictor or criterion vari- doi:10.1177/0022022104264122
ables. Findings from the present study suggest that replicable and Goldberg, L. R. (1999). A broad-bandwidth, public-domain personality
theoretically coherent internal structures of Big Five measures can inventory measuring the lower-level facets of several five-factor models.
be achieved with diverse and representative samples when acqui- In I. Mervielde, I. J. Deary, F. De Fruyt, & F. Ostendorf (Eds.),
Personality psychology in Europe (Vol. 7, pp. 7-28). Tilburg, the
escence response tendencies are measured and controlled. Given
Netherlands: Tilburg University Press.
findings from this and other studies that show that acquiescence is
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C ,
generally a greater concern among those with less formal educa-
Cloninger, C. R., & Gough, H. G. (2006). The international personality
tion, caution is also suggested with respect to the exclusive use of
item pool and the future of public-domain personality measures. Journal
well-educated convenience samples during the development and
of Research in Personality, 40, 84-96. doi:10.1016/j.jrp.2005.08.007
evaluation of personality questionnaires. Gudjonsson, G. H. (1990). The relationship of intellectual skills to sug-
gestibility, compliance and acquiescence. Personality and Individual
References Differences, U, 227-231. doi:10.1016/0191-8869(90)90236-K
Gudjonsson, G. H., & Young, S. (2011). Personality and deception: Are
Bachman, J. G., & O'Malley, P. M. (1984). Yea-saying, nay-saying, and suggestibility, compliance, and acquiescence related to socially desirable
going to extremes: Black-white differences in response styles. Public responding? Personality and Individual Differences, 50, 192-195. doi:
Opinion Quarterly, 48, 491-509. doi:10.1086/268845 10.1016/j.paid.2010.09.024
Benet-Martfnez, V., & John, O. P. (1998). Los Cinco Grandes across Hopwood, C. J., & Donnellan, M. B. (2010). How should the internal
cultures and ethnic groups: Multitrait multimethod analyses of the Big structure of personality inventories be evaluated? Personality and Social
Five in Spanish and English. Journal of Personality and Social P.9ychol- Psychology Review, 14, 332-346. doi:10.1177/1088868310361240
ogy, 75, 729-750. doi: 10.1037/0022-3514.75.3.729 Hui, C. H., & Tdandis, H. C. (1989). Effects of culture and response format
Ben-Porath, Y. S., & Tellegen, A. (2008). MMPI-2-RF (Minnesota Mui- on extreme response style. Journal of Cross-Cultural Psychology, 20,
tiphasic Personality Inventory-2-Restructured Form): Manual for ad- 296-309. doi:10.1177/0022022189203004
ministration and scoring. Minneapolis: University of Minnesota Press. Jackson, D. N., & Messick, S. (1958). Content and style in personality
Bentler, P. M., Jackson, D. N., & Messick, S. (1971). Identification of assessment. Psychological Bulletin, 55, 243-252. doi:10.1037/h0045996
content and style: A two dimensional interpretation of acquiescence.
John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The Big Five
Psychological Bulletin, 76, 186-204. doi:10.1037/h0031474
Inventory-Versions 4a and 54. Berkeley: University of California,
Billiet, J. B., & Davidov, E. (2008). Testing the stability of an acquiescence
Berkeley, Institute of Personality and Social Research.
style factor behind two interrelated substantive variables in a panel
John, O. P., & Sdvastava, S. (1999). The Big Five trait taxonomy: History,
design. Sociological Methods & Research, 36, 542-562. doi:10.1177/
measurement, and theoretical perspectives. In L. A. Pervin & O. P. John
0049124107313901
(Eds.), Handbook of personality: Theory and research (2nd ed., pp.
Billiet, J. B., & McClendon, M. J. (2000). Modeling acquiescence in
102-138). New York, NY: Guiiford Press.
measurement models for two balanced sets of items. Structural Equation
Modeling, 7, 608-628. doi:10.1207/S15328007SEM0704_5 Johnson, T., Kulesa, P., Cho, Y. I., & Shavitt, S. (2005). The relationship
between culture and response styles: Evidence from 19 countries. Jour-
Block, J. (1965). The challenge of response sets. New York, NY: Apple-
ton-Century-Crofts. nal of Cross-Cultural Psychology, 36, 264-277. doi:10.1177/
Block, J. (1971). On further conjectures regarding acquiescence. Psycho- 0022022104272905
logical Bulletin, 76, 205-210. doi:10.1037/h0031475 Kish, L. (1949). A procedure for objective respondent selection within the
Block, J. (1972). The shifting definitions of acquiescence. Psychological household. Journal of the American Statistical Association, 44, 380-
Bulletin, 78, 10-12. doi:10.1037/h0033043 387. doi:10.1080/01621459.1949.10483314
Burt, C. (1948). Factor analysis and canonical correlations. British Journal Lang, F. R., Lüdtke, O., & Asendorpf, J. (2001). Testgüte und psy-
of Statistical Psychology, 1, 95-106. chometdsche Äquivalenz der deutschen Version des Big Five Inventory
Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., Dahlstrom, (BFI) bei jungen, mittelalten und alten Erwachsenen. [Validity and
W. G., & Kaemmer, B. (2001). MMPI-2 (Minnesota Muitiphasic Per- psychometric equivalence of the German version of the Big Five Inven-
sonality Inventory-2): Manual for administration and scoring (Rev. ed.). tory in young, middle-aged, and old adults.] Diagnostica, 47, 111-121.
Minneapolis: University of Minnesota Press. doi:10.1026//0012-1924.47.3.111
 ACQUIESCENCE AND ITEM FACTOR STRUCTURE 1145

Loevinger, J. (1957). Objective tests as instruments of psychological the- Rammstedt, B., & John, O. P. (2007). Measuring personality in one minute
ory. Psychological Reports, 3, 635-694. or less: A 10-item short version of the Big Five Inventory in English and
Lorenzo-Seva, U., & ten Berge, J. M. F. (2006). Tucker's congruence German. Joumai of Research in Personality, 41, 203-212. doi:10.1016/
coefficient as a meaningful index of factor similarity. Methodology, 2, j.jrp.2006.02.001
57-64. Rammstedt, B., & Kemper, C. J. (2011). Measurement equivalence of the
Marin, G., Gamba, R. J., & Marin, B. V. (1992). Extreme response style Big Five: Shedding further light on potential causes of the educational
and acquiescence among Hispanics: The role of acculturation and edu- bias. Joumai of Research in Personality, 45, 121-125. doi:10.1016/j.jrp
cation. Journal of Cross-Cultural Psychology, 23, 498-509. doi: .2010.11.006
10.1177/0022022192234006 Rammstedt, B., Kemper, C. J., & Borg, I. (2013). Correcting Big Five
McCrae, R. R., Costa, P. T., Jr., & Martin, T. A. (2005). The NEO-PI-3: personality measurements for acquiescence: An 18-country cross-
A more readable revised NEO personality inventory. Journal of Person- cultural study. European Journal of Personality, 27, 71-81. doi:
ality Assessment, 84, 261-270. doi:10.1207/sl5327752jpa8403_05 10.1002/per.l894
McCrae, R. R., Herbst, J. H., & Costa, P. T., Jr. (2001). Effects of Rorer, L. G. (1965). The great response-style myth. Psychological Bulletin,
acquiescence on personality factors structures. In R. Riemann, F. M. 63, 129-156. doi:10.1037/h0021888
Spinath, & F. Ostendorf (Eds.), Personality and temperament: Genetics, Schumann, S. (2004): ZA4052: Persönlichkeit und Wahlverhalten 2003
evolution, and structure (pp. 217-231). Berlin, Germany: Pabst Science. [ZA4052: Personality and voting behavior 2003] [Data file. Version
Ones, D. S., & Viswesvaran, C. (1998). The effects of social desirability 1.0.0]. Retrieved from the GESIS Data Archive (Cologne) at http://
and faking on personality and integrity assessment for personnel selec- info 1 .gesis.org/dbksearch 19/SDesc2.asp?no=4052&tab = 3&II = I0&
tion. Human Performance, 11, 245-269. notabs=&af=&nf=l&db=E. doi: 110.4232/1.4052
Ones, D. S., Viswesvaran, C , & Reiss, A. D. (1996). Role of social Smith, P. B. (2004). Acquiescent response bias as an aspect of cultural
desirability in personality testing for personnel selection: The red her- communication style. Joumai of Cross-Cultural Psychology, Í5, 5 0 - 61.
ring. Journal of Applied Psychology, 81, 660-679. doi:10.1037/0021- doi:10.1177/0022022103260380
9010.81.6.660 Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2008). The develop-
Ostendorf, F. (1990). Sprache und Persönlichkeitsstruktur. Zur Validität mental psychometrics of Big-Five self-reports: Acquiescence, factor
des Fünf-Faktoren-Modells der Persönlichkeit. [Language and person- structure, coherence, and differentiation from ages 10 to 20. Joumai of
ality structure. On the structural validity of the Five Factor Model of Personality and Social Psychology, 94, 1\%-11,1. doi: 10.1037/0022-
personality]. Regensburg, Germany: Roderer. 3514.94.4.718
Rammstedt, B. (1997). Die deutsche Version des Big Five Inventories Tellegen, A. (1982). Brief manual for the Differential Personality Ques-
(BFI): Übersetzung und Validierung eines Fragebogens zur Erfassung tionnaire. Unpublished manuscript. Department of Psychology, Univer-
des Fünf-Faktoren-Modells der Persönlichkeit. [The German version of sity of Minnesota, Minneapolis.
the Big Five Inventory (BFI): Translation and validation of a question- ten Berge, J. M. F. (1999). A legitimate case of component analysis of
naire assessing the Five Factor Model of personality] (Unpublished ipsative measures, and partialling the mean as an alternative to ipsati-
master's thesis [Diplomarbeit]). University of Bielefeld, Bielefeld, Ger- zation. Multivariate Behavioral Research, 34, 89-102. doi:10.1207/
many. sl5327906mbr3401_4
Rammstedt, B., Goldberg, L. R., & Borg, I. (2010). The measurement Ware, J. E., Jr. (1978). Effects of acquiescent response set on patient
equivalence of Big-Five factor markers for persons with different levels satisfaction ratings. Medical Care, 16, 327-336. doi: 10.1097/00005650-
of education. Journal of Research in Personality, 44, 53-61. doi: 197804000-00005
10.1016/j.jrp.2009.10.005 Winkler, J. D., Kanouse, D. E., & Ware, J. E., Jr. (1982). Controlling for
Rammstedt, B., & John, O. P. (2005). Kurzversion des Big Five Inventory acquiescence response set in scale development. Journal of Applied
(BFI-K): Entwicklung und Validierung eines ökonomischen Inventars Psychology, 67, 555-561. doi:10.1037/0021-9010.67.5.555
zur Erfassung der fünf Faktoren der Persönlichkeit. [The short-scale Big
Five Inventory (BFI-K): Development and validation of an economic Received October 8, 2012
instrument for the assessment of the Big Five factors of personality.] Revision received April 4, 2013
Diagnostica, 51, 195-206. doi:10.1026/0012-1924.51.4.195 Accepted April 11, 2013 •
Copyright of Psychological Assessment is the property of American Psychological
Association and its content may not be copied or emailed to multiple sites or posted to a
listserv without the copyright holder's express written permission. However, users may print,
download, or email articles for individual use.
Copyright of Psychological Assessment is the property of American Psychological
Association and its content may not be copied or emailed to multiple sites or posted to a
listserv without the copyright holder's express written permission. However, users may print,
download, or email articles for individual use.