Michael Paige 2003

Download as pdf or txt
Download as pdf or txt
You are on page 1of 20

ARTICLE IN PRESS

International Journal of Intercultural Relations


27 (2003) 467–486

Assessing intercultural sensitivity: an empirical


analysis of the Hammer and Bennett Intercultural
Development Inventory
R. Michael Paigea,*, Melody Jacobs-Cassutob,
Yelena A. Yershovaa, Joan DeJaegherea
a
Department of Educational Policy and Administration, University of Minnesota, 330 Wulling Hall,
86 Pleasant Avenue, SE, Minneapolis, MN 55455, USA
b
Department of Psychology, University of Minnesota, Minneapolis, MN 55455, USA

Abstract

This article reports the results of the authors’ psychometric analysis of the Intercultural
Development Inventory (IDI) (The intercultural development inventory manual. Intercultural
Communication Institute, Portland, OR, 1998). The study had two major research objectives:
to examine the empirical properties of the IDI and to generate a single, composite IDI score
that could be used for research and training (participant profiling/diagnostic) purposes.
In May, 1998 and January, 1999, the IDI was administered to 378 high school students,
college students, and instructors in foreign language, language and culture, and intercultural
education courses. IDI data from the final sample of 353 were analyzed using a standard set of
psychometric procedures including factor analysis, reliability and validity testing, and social
desirability analysis.
The results demonstrate that the IDI is a reliable measure that has little or no social
desirability bias and reasonably, although not exactly, approximates the developmental model
of intercultural sensitivity (Towards ethnorelativism: A developmental model of intercultural
sensitivity. In: R. M. Paige (Ed.), Cross-cultural orientation: New conceptualizations and
applications, University Press of America, New York, 1986, pp. 27–69; Towards
ethnorelativism: a developmental model of intercultural sensitivity. In: R. M. Paige (Ed.),
Education for the intercultural experience, Intercultural Press, Yarmouth, ME, 1993, pp. 21–
71) upon which it is based. The study also produced a weighted mean composite measure, the

*Corresponding author. Tel.: +1-612-626-7456; fax: +1-612-624-3377.


E-mail address: r-paig@tc.umn.edu (R. Michael Paige).

0147-1767/03/$ - see front matter r 2003 Elsevier Ltd. All rights reserved.
doi:10.1016/S0147-1767(03)00034-8
ARTICLE IN PRESS
468 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

IDI developmental score, which should be of particular value for profiling and diagnosis.
r 2003 Elsevier Ltd. All rights reserved.

Keywords: Intercultural sensitivity; IDI; Weighted mean IDI score; Instrument validity and reliability;
Social desirability; High school and college students

1. Introduction

1.1. Overview of the study

This article reports on the empirical evaluation of a new measurement scale, the
Intercultural Development Inventory (IDI) (Hammer & Bennett, 1998), designed to
index intercultural sensitivity. The first objective of the study was to determine the
psychometric properties of the IDI: social desirability, reliability, and factorial and
construct validity (in reference to the underlying theoretical model). The second
objective was to devise an empirical IDI scale score out of the six individual stage
scores in order to test the discriminant validity of the IDI and to enable other
researchers to examine statistical relationships involving intercultural development
in a more parsimonious way than using the six individual stage scores.
To accomplish these objectives, the IDI was administered in May, 1998 and
January, 1999 to 378 US high school students, college students, and instructors in
foreign language, language and culture, and intercultural education courses. The
subjects were selected to represent diversity of demographic and intercultural
experiences.

1.2. Organization of the article

The article begins with the rationale for the study, followed by descriptions of the
developmental model of intercultural sensitivity (DMIS) (Bennett, 1986, 1993), the
underlying theoretical construct, and the IDI (Hammer & Bennett, 1998). The next
sections report on the research methods and findings. The article concludes with a set
of observations regarding the implications for intercultural researchers, educators,
and trainers.

1.3. Rationale for the study

Assessing intercultural sensitivity has been a key issue in the intercultural field in
recent years. There is a large theoretical literature on what intercultural sensitivity
means (Landis & Bhagat, 1996; Martin, 1989, special issue of International Journal of
Intercultural Relations; Lustig & Koester, 2003), but a much smaller body of
knowledge on how to assess or measure it. Much of the conceptual literature has
focused on traits hypothesized to comprise intercultural sensitivity (Adler, 1976;
Bochner, 1977; Dinges, 1983; Gudykunst & Hammer, 1987). A smaller body of work
has viewed intercultural sensitivity in more dynamic and developmentally oriented
ARTICLE IN PRESS
R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486 469

terms (Adler, 1975; Hoopes, 1979; Lysgaard, 1955); however, most of those models
have lacked conceptual specificity. Most importantly for the purposes of this article,
the empirical evidence on instruments and other methods of assessing intercultural
sensitivity has been very limited to date.
Bennett’s (1986, 1993) DMIS represents an important theoretical advance from
those earlier works. He presents a complex model of intercultural development,
framed in terms of the phenomenology of an individual’s affective, cognitive, and
behavioral construal of, as well as response to, cultural difference. Bennett
conceptualizes six stages along a continuum of intercultural development of which
three are ethnocentric (Denial, Defense, Minimization) and three are ethnorelative
(Acceptance, Adaptation, Integration). The DMIS will be discussed in greater detail
in Section 2.
Interculturalists have also been interested in scales that could be used to measure
intercultural sensitivity for research purposes as well as in intercultural education,
training, and personnel selection. A number of instruments do exist, two of the best
known and most widely used being the Culture Shock Inventory (Reddin, 1994) and
the Cross-Cultural Adaptability Inventory (Kelley & Meyers, 1992). While both
measure specific human characteristics thought to be associated with intercultural
sensitivity (e.g., flexibility and openness, emotional resilience, cultural knowledge),
neither purports to measure intercultural sensitivity as a developmental construct.
The IDI, published in May, 1998, was designed to assess intercultural sensitivity
along the lines of the constructs embedded in the DMIS.
The rationale for this study was to determine how well the IDI measures
intercultural sensitivity. Does the IDI correspond to the underlying conceptual
model? Is it compromised by social desirability? Does it possess discriminant
validity? These are significant questions for researchers and trainers who might be
interested in using this instrument.

2. The developmental model of intercultural sensitivity

Bennett’s (1986, 1993) DMIS consists of three ethnocentric stages (Denial,


Defense, Minimization) and three ethnorelative stages (Acceptance, Adaptation,
Integration). The six stages constitute a theoretical continuum ranging from Denial
of difference to Integration of difference.

2.1. Stage one: Denial of Difference

This initial ethnocentric stage consists of benign neglect, indifference to, or


ignorance regarding cultural difference. It is characterized by naive observations
about culturally different others and superficial statements of tolerance. Persons in
the Denial stage have generally grown up in culturally homogeneous environments
and have had limited contact with people outside their own culture group.
There are two substages of Denial. The first is isolation, which is the unintentional
isolation from other culture groups due to life circumstances. The second is
ARTICLE IN PRESS
470 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

separation, the intentional separation from other culture groups to maintain the
condition of isolation.
Sample IDI Denial items include:
Society would be better off if culturally different groups kept to themselves.
I do not like to socialize very much with people from different cultures.

2.2. Stage two: Defense against difference

This is the stage characterized by recognition and negative evaluation of


difference. Persons in Defense feel threatened by difference and respond by
protecting their worldview. Dualistic ‘‘we—they’’ thinking and overt, negative
stereotyping are common at this stage.
There are three dimensions of Defense. In the first, superiority, the virtues of one’s
own group are compared to all others, the positive aspects of one’s group are
exaggerated, and criticism of one’s culture is interpreted as an attack. This substage
might be viewed as positive in-group evaluation. The second substage is denigration
where persons evaluate other cultures as inferior, use derogatory terms to describe
other groups, and apply negative stereotypes to other groups. This can be referred to
as negative out-group evaluation. The third substage of reversal consists of viewing
the other culture as superior to one’s own and feeling alienated from one’s own
culture group. It can be viewed as ‘‘going native,’’ the phenomenon of negative in-
group combined with positive out-group evaluation.
Sample IDI Defense items include:
My culture’s way of life should be a model for the rest of the world.
People from other cultures are not as open-minded as people from my own
culture.

2.3. Stage three: Minimization of difference

In this stage, people recognize superficial cultural differences, but they hold to the
view that basically human beings are the same. The emphasis is on similarities, not
differences. The similarities are those people see in others that resemble what they
know about themselves.
There are two substages of Minimization, the first being physical universalism
where the emphasis is on physiological similarities; similarity is based on the fact of
our all being human beings with similar needs, etc. The second substage of
transcendent universalism represents the assumption that people are similar due to
spiritual, political, or other overarching commonalties.
Sample IDI Minimization items include:
People are the same despite outward differences in appearance.
I am sick and tired of hearing all the time about what makes people different; we
need to recognize that we are all human beings, after all.
ARTICLE IN PRESS
R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486 471

2.4. Stage four: Acceptance of difference

This is the first of three ethnorelative stages. Persons in Acceptance recognize and
appreciate cultural differences. Culture is understood as a viable way of organizing
human behavior. Cultural differences in behaviors and values are accepted as normal
and desirable. Difference is no longer judged by the standards of one’s own group;
difference is examined within its own cultural context. The guiding principle of
Acceptance is cultural relativism: one culture is not inherently better or worse than
another.
There are two substages of Acceptance. In behavioral relativism persons accept the
idea that behavior varies across culture groups and according to cultural context,
behavioral patterns are valid for those who share and understand them, and
acceptance of behavioral difference does not mean that one is necessarily
comfortable about specific differences. Value relativism means accepting the
perspective that values and beliefs also exist in a cultural context and vary across
cultural communities, notions of ‘‘good’’ and ‘‘bad’’ are value orientations that can
differ according to the culture group that holds those views.
Sample IDI Acceptance items include:

I generally enjoy the differences that exist between myself and people from other
cultures.

It is appropriate that people from other cultures do not necessarily have the same
values and goals as people from my own culture.

2.5. Stage five: Adaptation to difference

Persons in this stage consciously try to imagine how the other person is thinking
about things. They shift their mental perspective into the ‘‘insider’s’’ point of view.
They employ alternative ways of thinking when they are solving problems and
making decisions. They can communicate and interact effectively with people from
other cultures. They can shift their frames of reference.
The two dimensions of Adaptation are empathy and pluralism. Empathy refers to
the ability to shift perspective into alternative cultural worldviews. Pluralism means
the internalization of more than one complete worldview. Behavior shifts completely
into different frames of reference without much conscious effort.
One difference between the IDI and the DMIS emerged in the instrument
development process. The factor analysis of Adaptation delineated two-factors
comprised of items related to (1) the pluralism form of Adaptation and (2) items
related to what the authors referred to as the ‘‘importance of culture specific
understanding’’ (Hammer & Bennett, 1998, p. 67–68). In the model, pluralism is the
more advanced form of Adaptation; it measures its behavioral aspects. The authors
renamed this scale Behavioral Adaptation to more accurately represent the content of
its items. A factor analysis of the sixth and final DMIS Integration stage revealed
one factor, which includes items from the contextual evaluation form of Integration
ARTICLE IN PRESS
472 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

and the empathy form of Adaptation. This substage was renamed Cognitive
Adaptation.
Sample IDI Cognitive Adaptation items include:

I feel there are advantages in identifying with more than one culture.

In evaluating an intercultural situation, it is better to be able to draw from more


than one cultural perspective.

Sample IDI Behavioral Adaptation items include:

Although I am a member of my own culture, I am nearly as comfortable in one or


more other cultures.

When I come in contact with people from a different culture, I find I change my
behavior to adapt to theirs.

2.6. Stage six: Integration of difference

Persons in this stage have internalized more than one cultural worldview into their
own. Their identity includes but, more importantly, transcends the cultures of which
they are a part. They see themselves as persons ‘‘in process’’. They define themselves
as persons at the margin of cultures (‘‘cultural marginals’’) and as facilitators of
cultural transition.
There are two substages of Integration. The first is contextual evaluation, which is
defined as the ability to employ different cultural frames of reference in evaluating a
given situation. The second is constructive marginality, i.e., acceptance of an identity
that is not based primarily on one culture. Persons in this substage have the ability to
facilitate constructive contact between cultures and they are likely to participate in a
‘‘marginal reference group.’’ As was pointed out, no items were found to form pure
contextual evaluation or constructive marginality substages, nor did items from the
two substages form an Integration construct.

3. The IDI instrument development process

In order to develop the IDI items, Hammer and Bennett (1998) developed a
qualitative interview designed to elicit how respondents made sense out of their
experiences with cultural difference. The interview consisted of six questions that
stimulated discussion around the six stages of the DMIS. Forty individuals
representing diverse experiences and cultural backgrounds were interviewed. The
interviews generated more than 350 statements relevant to intercultural sensitivity.
From these, 200 were viewed by four independent raters (with interrater reliability
greater than 0.66 for stage ratings) as representing the six stages and 13 forms of the
DMIS. These items were then reexamined by seven experts familiar with the DMIS.
Setting the interrater agreement criterion at greater than 0.60 in order for the item to
ARTICLE IN PRESS
R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486 473

be selected for the instrument, the expert round resulted in the first IDI with 145
items.
To validate the instrument, the pilot instrument was administered to 226
respondents. Within-stage factor analysis was conducted on these items to examine
their suitability for the final IDI version. Reliability analysis was performed on each
of the six scales with a minimum scale reliability of 0.80 as the criterion. The authors
assessed construct validity by correlating the IDI items with items from the
Worldmindedness Scale (Sampson & Smith, 1957) and the Intercultural Anxiety
Scale (Hammer & Bennett, 1998; Stephan & Stephan, 1985). This analysis
demonstrated that higher ethnorelativism scores correlated with higher Worldmind-
edness and lower Social Anxiety Scale scores as theoretically predicted. Finally, an
interscale correlational matrix showed that the scales were related to each other in
the manner suggested by the conceptual model. These analyses provide evidence that
the IDI is measuring what it is purporting to measure. For a more detailed
description, see Hammer, Bennett, and Wiseman (2003).
Having discussed how the IDI was constructed and originally validated, we now
turn to the current study.

4. Method

4.1. Subjects

The following principles guided the sampling: (1) sample a large enough group to
ensure sufficient variability on intercultural sensitivity, (2) sample for variability on a
set of personal characteristics that could theoretically be expected to correlate with
intercultural sensitivity (e.g., prior international experience, amount of exposure to
language education), (3) sample respondents who would be participating in common
learning experiences of interest to the project team (in this instance, foreign language
and intercultural education coursework), but who would have had varying levels of
exposure to those experiences.
The 353 subjects in the present study included foreign language students both at
the high school and college level, college students in an intercultural education
course, and four college language instructors. Data were collected twice. In the first
data set there were 139 high school students studying varying levels of French and 38
high school students studying varying levels of German. This subject pool also
included 47 college students in beginning Spanish, 16 studying beginning Portuguese
for Spanish speakers, and seven in Intermediate Portuguese. Finally, there were four
college level language instructors for a total of 251 subjects in the first data set.
The second data set consisted of only college students. It was added to contribute
to the variability of responses by introducing more subjects with potentially higher
IDI scores into the sample, to even out the high school and college ratio, and to
build a larger sample for the purposes of statistical analysis. There were 23 students
in Beginning Spanish, 22 in Intermediate Spanish, 40 in Advanced Spanish
Composition, 16 studying Introduction to Hispanic Cultures, and 26 in an
ARTICLE IN PRESS
474 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

intercultural education and training course. The second data set (n ¼ 127) brought
the total number of respondents to 378. Missing data accounted for the loss of 25
respondents for a final sample size of 353.

4.2. Missing data

Not all subjects filled out the IDI completely. Because the scales are 10 items in
length, each item lost is a 10% reduction in the scale. For individuals who had
missing data for a single item within a scale, a value could be estimated. In this study,
missing values were replaced with predicted values using a regression procedure
(RMV Trend, SPSS, 1994). Of the 251 in the initial data set, 202 subjects had no
missing data and 25 subjects had only one missing score per scale. These 25 had the
single item estimated using the SPSS procedure RMV Trend. The remaining 23
subjects with two or more missing items were dropped from the analyses. In the
second data set of 127, three subjects had only one missing score per scale and two
individuals had to be dropped.
The final tally of 353 subjects consists of 123 high school French students, 32 high
school German students, 69 Beginning Spanish students, 22 Intermediate Spanish
students, 7 Intermediate Portuguese students, 16 Portuguese for Spanish Speakers,
38 Advanced Spanish Composition students, 16 Hispanic Culture students, 26
Teaching Culture students, and 4 Spanish Instructors.

4.3. Instruments

Three instruments were administered to the respondents. The first was the IDI
(Hammer & Bennett, 1998). The IDI consists of 60 items, 10 for each of the six
stages. The second instrument was a demographic questionnaire developed by the
authors. The demographic questionnaire included items about the students’ prior
and current intercultural experiences, their motivation to take foreign languages,
their level of interest in the language they were taking, and the progress that they felt
they were making in the foreign language class. The third was a shortened version of
the Marlowe-Crown Social Desirability Scale (Strahan & Gerbasi, 1972). Both
college and high school students received the IDI and the demographic questionnaire
but only the college students received the social desirability index.

4.4. Procedures

Five procedures were used to analyze the IDI. First, all six scales were examined
for internal consistency reliability. Second, all 60 items were factor analyzed to
determine how well a six-factor structure could explain the data. Both orthogonal
and oblique solutions were considered. Third, the predictive validity of the IDI was
determined by studying the relationships between IDI scores and a set of
background variables. Fourth, the potential for a social desirability bias was
examined by correlating IDI scores with the Marlowe-Crown Social Desirability
ARTICLE IN PRESS
R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486 475

Scale (Strahan & Gerbasi, 1972). And finally, a weighted mean IDI score was created
that enabled the researchers to conduct other parts of the analysis.

5. Results

5.1. Scale reliability

Table 1 lists the internal consistency reliability for each of the six stages.
Four of the scales had Cronbach a internal consistency reliability coefficients of
0.80 and above, thus met the lower bound for internal consistency reliability
(Crocker & Algina, 1986). The a coefficients of 0.77 for Acceptance and 0.74 for
Behavioral Adaptation came close to meeting the minimum standard. Hence the
scales are acceptable in terms of internal consistency reliability.

5.2. Factor analysis

Fig. 1 is a scree plot using all 60 items of the IDI. The ‘‘elbow’’ appears to be
somewhere between four and six factors.
In the factor analysis, a theoretical six-factor orthogonal model was used to test
the six-stage model. This model accounts for 49.4% of the variance, but there are 424
(23.0%) residuals (above the diagonal) with absolute values >0.05. This is to be

Table 1
Cronbach’s a for the 6 IDI stages

Denial Defense Minimization Acceptance Cognitive Adaptation Behavioral Adaptation

a a a a a a
0.8749 0.9096 0.8332 0.7711 0.8715 0.7437

Factor Scree Plot


IDI (N = 353)
16
14
12
Eigenvalue

10
8
6
4
2
0
1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58
Factor Number
Fig. 1. Factor scree plot IDI (N ¼ 353).
ARTICLE IN PRESS
476 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

expected, since reduction in the number of factors will reduce the amount of variance
accounted for and increase the number of large residuals. While the reduction of
variance (3.8%) is minimal, an increase by 20% of large residuals indicates a less
than optimal fit (Tabachnick & Fidell, 1996). An additional problem of factor
analysis is the size of the data set. In order to get stable results, there should ideally
be a 10:1 ratio of people to variables (i.e. N ¼ 600). The more ‘‘lenient’’ criterion is a
5:1 ratio (N ¼ 300). Our data set fits the lenient criterion. Despite the residual and
sample size issues, one of the objectives of extracting six factors was to see if the
factors could be interpreted within the framework of the theoretical model. Even
with a smaller sample, factor loadings can be examined.
Table 2 lists the factor loadings for the rotated six-factor solution. Factor 1 is a
combination of items from the Denial and Defense stage. 17 of the 20 items from
those two scales loaded 0.50 or higher on Factor 1. In addition, there were five
Acceptance items with moderately negative loadings on Factor 1. Factor 2 consists
of eight Cognitive Adaptation and four Behavioral Adaptation items; our
interpretation of this factor was that the Behavioral Adaptation items tended to
use cognitive vocabulary. Minimization is split between Factors 3 and 4. Factor 3
consists of five physical universalism items, which contain phrases such as ‘‘people
are basically the same.’’ Factor 4 (n=three items) reflects transcendent universalism
loaded highest on items containing the phrase ‘‘children of a spiritual being.’’ Factor
5 contained three Behavioral Adaptation items which loaded highly. Factor 6
contained two Acceptance and two Denial items, hence Acceptance splits with
Factor 1. However, Factor 6 does not fit the theoretical model.
While these results do not present a factor structure that is an exact fit with the six
distinct stages of the DMIS, they do follow the model to a considerable degree.
There are five stages suggested by the factor analysis: (1) an ethnocentric stage which
combines the adjacent categories of Denial and Defense; (2) a Minimization stage
consisting of physical universalism items; (3) a second Minimization stage consisting
of transcendent universalism items; (4) an Adaptation stage which combines
cognitive and behavioral items and (5) a separate and purely Behavioral Adaptation
stage.
In addition, a two-factor solution was also examined to see if the more global
categories of ethnocentric and ethnorelative stages manifest themselves. The results
are quite clear in a two-factor solution. Fig. 2 below shows the mean factor loadings
on an unrotated two-factor solution.
The three ethnorelative stages (Cognitive Adaptation, Behavior Adaptation, and
Acceptance) cluster on one factor and two of the three ethnocentric stages, Denial
and Defense cluster on the other factor with Minimization in the middle between the
two. They are also orthogonal.

5.3. Predictive validity

The predictive validity (Black, 1999) of the IDI was examined on the basis of its
ability to discriminate persons in a manner consistent with theory and prior research.
In order to test for the predictive validity of the IDI, one way analysis of variance
ARTICLE IN PRESS
R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486 477

Table 2
Rotated factor loadings for a six-factor orthogonal solution

Stage Item Factor 1 Factor 2 Factor 3 Factor 4 Factor 5 Factor 6

DEFENSE 41 0.79073
DENIAL 40 0.78029
DENIAL 43 0.74664
DEFENSE 28 0.74589
DEFENSE 16 0.70892
DEFENSE 44 0.70621
DEFENSE 55 0.70468
DENIAL 30 0.69849
DEFENSE 20 0.67753
DENIAL 38 0.67624
DEFENSE 56 0.67237
DENIAL 14 0.66952
ACCEPT 47 0.66564
DENIAL 49 0.66393
DEFENSE 11 0.66051
DEFENSE 39 0.65377
DEFENSE 10 0.63804
DENIAL 57 0.61163
ACCEPT 27 0.55740
ACCEPT 19 0.54092
ACCEPT 29 0.53878
ACCEPT 33 0.53666
COGADP 53 0.76121
COGADP 25 0.74512
COGADP 24 0.71818
COGADP 3 0.69179
COGADP 52 0.68436
COGADP 18 0.65595
COGADP 60 0.65096
COGADP 46 0.62014
BHVADP 9 0.60942
BHVADP 26 0.55805
BHVADP 35 0.53760
BHVADP 36 0.51186
MINIM 22 0.74940
MINIM 4 0.71531
MINIM 34 0.71315
MINIM 51 0.67837
MINIM 8 0.64848
MINIM 12 0.79523
MINIM 23 0.79048
MINIM 59 0.78284
BHVADP 54 0.70597
BHVADP 58 0.69324
BHVADP 13 0.62808
ACCEPT 48 0.58427
ACCEPT 2 0.56940
DENIAL 15 0.54565
DENIAL 1 0.53209
ARTICLE IN PRESS
478 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

Mean Factor Loadings of IDI


Unrotated Solution (N = 353) CASE_LBL
0.4
0.3 MINLOAD
Ethnocentrism 0.2 DENLOAD
0.1
0.0 DEFLOAD
-0.1 COGLOAD
-0.2 BHVLOAD
-0.3
-0.4 ACCLOAD
-0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4 0.6

Ethnorelativism
Fig. 2. Mean factor loading of IDI unrotated solution (N ¼ 353).

Table 3
Analysis of variance: criterion variables and intercultural development

Variable F score Significance

Prior intercultural experience 11.888 0.000


Prior Language–culture study 12.133 0.000
Friends from other cultures 31.339 0.000
Socialize–other cultural 12.206 0.000
Age 8.436 0.000
Gender 2.514 0.114

tests were run on six background variables, five of which were expected to be
associated with intercultural sensitivity (age, prior intercultural experience, prior
language and culture study, friends with people from other cultures, and socialize
with people from other cultures) and one of which (gender) was not. Table 3 presents
the results. All of the variables differentiate respondents’ intercultural development
in the predicted direction. Gender is not associated with intercultural development in
a statistically significant manner.

5.4. Social desirability

In any type of attitude survey, there is always the possibility that subjects may
rate items in a way that reflects socially acceptable answers rather than their
own opinions. This response set is referred to as social desirability. In order
to test for social desirability bias, subjects were given 10- or 20-item shortened
versions of the Marlowe-Crown Social Desirability Scale (Strahan & Gerbasi,
1972). High school respondents in the first IDI administration were given a
10-item version and the university students in the second administration completed
a 20-item version of the scale (which included the original 10 items). The procedure
is to score each item as true or false. Items scored in the keyed direction of
ARTICLE IN PRESS
R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486 479

social desirability are assigned a score of 1, those in the unkeyed direction were
assigned a score of 0. The score for each subject is the sum of 1 and 0 s from the
individual items.
Tables 4 and 5 list the means and standard deviations of the sample for the
individual items as well as the total summed score.
In general, subjects scored low on social desirability. Items 2, 3, 11, 13, 14, and 15
are the only items for which subjects answered more than half the time in the keyed
direction. Tables 6 and 7 list the correlations between social desirability and IDI
scores for the two different length inventories.
The low correlations with the IDI indicate that there is no relationship between the
way subjects answered the IDI and their level of social desirability. The only
correlation worth noting is the correlation between Social Desirability and Mean 3
(Minimization). While it is low and not statistically significant, it is much larger in
magnitude than the other correlations, which tend to hover around zero. There is
some indication that there is a relationship between the items on the Minimization
scale and the items on the Social Desirability Index.

5.5. Scoring algorithms

One goal of the study was to create a single score representing the respondent’s
overall intercultural development level. The theoretical criteria were that the overall
score would take all of the stage scores into account, assign a greater weight to the
larger theoretical gap between Minimization and Acceptance, and negatively weight
the ethnocentric scores. The scoring algorithm principle was that taking the overall
response pattern, the response magnitude, and the theoretical structure of the model
into account would generate a score that is a reasonable representation of where the
person is actually situated on the developmental continuum. The result is what the
researchers refer to as the IDI score.

Table 4
Means, standard deviations, and N’s of the social desirability index (10 items)

Item Mean SD N

1 0.4857 0.5034 70
2 0.8143 0.3917 70
3 0.6857 0.4676 70
4 0.1429 0.3525 70
5 0.2000 0.4029 70
6 0.2286 0.4229 70
7 0.2714 0.4479 70
8 0.4571 0.5018 70
9 0.1429 0.3525 70
10 0.3571 0.4826 70
Total 3.7857 1.9329 70
ARTICLE IN PRESS
480 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

Table 5
Means, standard deviations, and N’s of the social desirability index (20 items)

Item Mean SD N

1 0.5169 0.5018 118


2 0.8898 0.3144 118
3 0.6525 0.4782 118
4 0.1356 0.3438 118
5 0.2966 0.4587 118
6 0.1949 0.3978 118
7 0.3305 0.4724 118
8 0.4915 0.5021 118
9 0.1017 0.3035 118
10 0.3220 0.4692 118
11 0.5847 0.4949 118
12 0.2966 0.4587 118
13 0.7288 0.4465 118
14 0.5593 0.4986 118
15 0.8644 0.3438 118
16 0.2881 0.4548 118
17 0.3644 0.4833 118
18 0.3305 0.4724 118
19 0.1949 0.3978 118
20 0.4831 0.5018 118
Total 8.6271 4.0378 118

Table 6
Correlations between social desirability (10 items) and the IDI

Denial Defense Minimiz Accept Cog Adp Bhv Adp IDI score

Social desirability 0.0397 0.1092 0.1326 0.0045 0.0046 0.0461 0.0395

Table 7
Correlations between social desirability (20 items) and the IDI

Denial Defense Minimiz Accept Cog Adp Bhv Adp IDI Score

Social desirability 0.0305 0.0022 0.2050 0.0422 0.0709 0.0666 0.0597

5.5.1. IDI score construction


The major question with any weighted mean score is what to use for weights.
Mark Davison, an authority on multidimensional scaling, (personal communication,
October, 1998) suggested using the weights of 3, 2, 1, 1, 2, 3 to (1) take into
account the larger gap between Minimization and Acceptance and (2) allow for a
symmetric, integer set of weights. Moreover, from a psychometric perspective,
ARTICLE IN PRESS
R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486 481

having symmetric positive and negative weighting preserves the pattern of responses
but ‘‘washes away’’ some of the effect of person response sets (i.e. individuals who
avoid using the endpoints of a Likert scale). Since the scoring goes from 3 to 3, the
single score was then converted to a score from 1 to 6 based on a linear
transformation. (Note: This consists of adding a constant of 3.5 of scores greater
than or equal to 1 or less than or equal to 1. Otherwise, divide the score by 2 and
then add 3.5.) Table 8 lists some actual data examples including mean scale scores
and the weighted mean using Davison’s weights.
This approach to a weighted mean had the limitation of a restricted range of IDI
scores from 3.2 to 4.6 in these real cases. When the Davison weights are applied to
the extreme hypothetical example below, the score is

3ð1Þ þ 2ð1Þ þ 1ð1Þ þ 1ð1Þ þ 2ð1Þ þ 3ð7Þ=ð1 þ 1 þ 1 þ 1 þ 1 þ 7Þ ¼ 1:5:

Applying the linear transformation yields a score 5.0. From this example, it
appears that the hypothetical upper limit is 5.0 instead of 6.5. The equation was
adjusted to take into account the fact that the IDI classification is based on a scale
from 1 to 6 and the Likert scale used for the six individual stage score ranges from 1
to 7. In the adjustment, 1 is subtracted from each mean in the denominator or a total
of 6 is subtracted from the sum of the means (see Eq. (1) below)
P6
wi m
Score ¼ P i¼1  i :
 ð1Þ
6
i¼1 mi  6

The resulting score is then converted back into a score ranging from 0.5 to 6.5.
Applying this formula to our hypothetical example of 1, 1, 1, 1, 1, 7 results in the
following: 3ð1Þ þ 2ð1Þ þ 1ð1Þ þ 1ð1Þ þ 2ð1Þ þ 3ð7Þ=ð1 þ 1 þ 1 þ 1 þ 1 þ 7Þ  6 ¼
3 or 6.5 when a linear transformation is made. Clearly, this is more satisfactory
solution. Table 9 lists the mean scale scores, Davison’s weights, and the IDI score
from Eq. (1) for those same individuals.
While the results are similar, only Eq. (1) allows for the full range of the spectrum.
Hence, Eq. (1) became the formula for the IDI score. Table 10 shows several

Table 8
Sample IDI scores

Person Denial Defense Minimiz Accept Cog Adp Bhv Adp Davison

1 4.6 4.5 3.5 4.2 3.0 4.2 3.427


2 1.9 1.9 4.2 6.4 5.3 5.3 3.884
3 1.0 1.6 4.5 6.8 7.0 6.5 4.580
4 5.2 4.5 4.8 5.1 4.6 5.2 3.508
5 1.0 1.8 3.3 6.8 4.9 6.5 4.578
6 1.2 1.3 6.1 5.8 4.1 6.1 3.906
7 1.6 2.2 6.4 7.0 6.4 4.6 3.819
8 1.9 1.7 4.4 5.7 5.3 4.8 3.861
ARTICLE IN PRESS
482 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

Table 9
Actual mean scale scores, Davison’s weighted mean, and the IDI weighted mean score

Person Denial Defense Minimiz Accept Cog Adp Bhv Adp Davison IDI score

1 4.6 4.5 3.5 4.2 3.0 4.2 3.427 3.4023


2 1.9 1.9 4.2 6.4 5.3 5.3 3.884 4.5105
3 1.0 1.6 4.5 6.8 7.0 6.5 4.580 4.8832
4 5.2 4.5 4.8 5.1 4.6 5.2 3.508 3.5104
5 1.0 1.8 3.3 6.8 4.9 6.5 4.578 4.9317
6 1.2 1.3 6.1 5.8 4.1 6.1 3.906 4.5743
7 1.6 2.2 6.4 7.0 6.4 4.6 3.819 3.9054
8 1.9 1.7 4.4 5.7 5.3 4.8 3.861 3.9831

Table 10
IDI Scores—hypothetical extreme cases

Person Denial Defense Minimiz Accept Cog Adp Bhv Adp Davison IDI score

1 7 1 1 1 1 1 2.00 0.5000
2 1 7 1 1 1 1 2.50 1.5000
3 1 1 7 1 1 1 3.25 2.5000
4 1 1 1 7 1 1 3.75 4.5000
5 1 1 1 1 7 1 4.50 5.5000
6 1 1 1 1 1 7 5.00 6.5000
7 1 1 1 7 7 1 4.50 5.0000
8 1 1 1 7 7 7 5.00 5.5000
9 1 1 1 1 7 7 5.17 6.0000
10 1 1 7 7 1 1 3.50 3.5000

hypothetical scores and the different scores they yield using the different scoring
algorithms.
The Davison scoring algorithm yielded scores restricted in range. Only the
theoretical scoring allows values that range from 0.5 to 6.5.

6. Discussion

6.1. Scale analysis

This study has examined the potential of the IDI to measure a person’s
intercultural sensitivity. To that end, we have (1) analyzed the psychometric
properties of the instrument, in order to assess its internal validity and reliability;
(2) refined the IDI scoring by creating a weighted mean (‘‘developmental’’) score, a
composite measure that situates a person on the intercultural development
continuum, and (3) explored the sensitivity of the IDI items to a social desirability
response set.
ARTICLE IN PRESS
R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486 483

The analyses of the internal structure of the IDI have shown it to be a reasonable
approximation of the theoretical model of intercultural development. Reliability
coefficients for individual stages were found to be quite high, similar to the original
IDI validation study results. This indicates that the items within each individual scale
correlate well with each other in measuring the intended trait. Two scales of concern
were Acceptance (a ¼ 0:77) and Behavioral Adaptation (a ¼ 0:74). In the case of the
Acceptance scale, the relatively lower internal consistency might be explained by
the complexity and multidimensionality of this first ethnorelative stage, which, in the
operational terms, is reflected in a larger variety of statement types used to measure
this construct. The items of the scale ranged from ‘‘describing difference’’ (in non-
evaluative ways) and ‘‘enjoying difference’’ to ‘‘learning difference’’ to ‘‘value
relativity’’ (Hammer & Bennett, 1998, p. 40). Given that each IDI scale is comprised
of only ten items, such a range of statement types might have compromised the
internal consistency of the Acceptance scale. That Acceptance appears in several
different factors supports this interpretation.
A similar argument can be employed in the analysis of the Behavioral Adaptation
scale. The two primary constructs measured by this scale, ‘‘behavioral shift’’ and
‘‘bicultural identification,’’ seem to be significantly different in nature. The
behavioral shift items test the extent to which respondents change their behavior
according to the cultural context in which they find themselves; the bicultural
identification items address how respondents make intellectual sense of their
intercultural experiences in identity terms. Thus, we think the first set of items
measures behavior and the second cognition. It should also be kept in mind that
Behavioral Adaptation, being the highest stage of intercultural development,
requires a substantial amount of intercultural experience, as well as a high level of
maturity and sophistication in processing this experience. Thus, it is very likely that
our sample did not have very many respondents scoring high in this stage of
intercultural development; this accounted for the low variance and ultimately
affected the reliability coefficient.
Overall, the factor analyses provide strong empirical support for the broader two-
factor (ethnocentric and ethnorelative) structure of the developmental model and
modest support for the six-factor structure of intercultural sensitivity that the IDI is
purporting to measure. While the six factors do not exactly mirror the six stage
structure of the IDI, upon closer examination there is nothing particularly surprising
or problematical about the factor loadings. Denial and Defense items loaded
together on Factor 1. Eight of ten Cognitive Adaptation items loaded significantly
on Factor 2 along with the four Behavioral Adaptation items (that we argue employ
a cognitive vocabulary). Minimization items split along the theoretical lines of
physical and transcendent universalism (Factors 3 and 4). The unambiguously
phrased Behavioral Adaptation items form Factor 5. Factor 6, the weakest factor
statistically, does not follow a predictable conceptual pattern in combining two
Acceptance and two Denial items. Acceptance items split between factor 1 (n ¼ 5
items) and Factor 6 (n ¼ 2 items). The Factor 1 finding can be explained to a degree
by the fact that the Acceptance item loadings are negative, consistent with the
author’s theoretical position that ‘‘in terms of cultural difference, acceptance is the
ARTICLE IN PRESS
484 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

opposite of defense’’ (p. 38). In effect, all of the stages of the model are accounted for
in our factor structure, albeit some more strongly than others.
The fact that most items of the Denial and Defense scales loaded on Factor 1 can
be explained in several ways. First, we observed that there was an extremely low
number of the respondents identifying with those stages and, therefore, a
correspondingly low variance. Second, it could be reasoned that the respondents
were acting predictably in rejecting Denial and Defense due to the fact that they were
foreign language students who presumably had been exposed to cultural differences
and cultural variables in their language courses. Third, the finding might be the result
of the possible transparency of the items, which made them readily identifiable as
socially unacceptable or ‘‘politically incorrect.’’ However, there is no evidence of this
in our social desirability analysis.
The Minimization split is interesting because it follows exactly the form structure
of the DMIS. The IDI in our study distinguished between the set of items that
trivialize difference and stress basic humanness of all people, i.e. their physical
similarities, and the items that emphasize the universal principles transcending
cultures, such as spirituality or belief in God. Interestingly, it is the items of the latter
kind that seemed to present a problem for our respondents. A review of selected
answer sheets showed that most subjects were not sure how to deal with these items
and therefore marked their responses as neutral (4 on the 7-point scale). It is very
likely that many individuals of the age groups represented in our sample have not
given any serious thought to their position on the issues of spirituality and religious
beliefs. The lack of strong responses to the statements of this kind might have
accounted for the differences in the results that we observed.
Finally, the finding that Behavioral Adaptation split into two factors is explained,
in our view, by the cognitive vocabulary of the bicultural identification statements
versus the behavioral emphasis of the behavioral shift items.

6.2. Single score construction

The other significant part of this study consisted of constructing a single,


composite score representing the respondent’s overall performance on the IDI. Work
on developing a single score was to a large extent fueled by the research interest in
correlating a person’s intercultural development with other potentially salient
variables, including age, number of years of foreign language study, living abroad
experiences, etc.
Creating the IDI score was complicated by a major theoretical challenge. The
researchers felt very strongly about reflecting the theoretical assumptions of the
DMIS in the final score. This meant accounting for the ‘‘paradigmatic barrier’’
(Bennett, 1993, p. 45) between ethnocentrism and ethnorelativism, i.e. the movement
out of Minimization into Acceptance. That issue was resolved by the manner in
which the stage scores were weighted. A second problem emerged when we applied
our IDI score to hypothetical ideal types on the continuum (see Table 7). The IDI
score assumes that the DMIS is a mutually exclusive stage model; progress means
leaving earlier stages as one becomes more interculturally sensitive. Therefore, the
ARTICLE IN PRESS
R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486 485

highest IDI score, 6.5, goes to the person who scores a 1 1 1 1 1 7 on the six stages.
The structure of the theoretical model, although not stated explicitly, is actually
stage-exclusive in the ethnocentric stages, but stage-cumulative in the ethnorelative
stages, i.e., Acceptance is the foundation for further development into Cognitive and
Behavioral Adaptation. Our conclusion is that the IDI, while not an exact
representation of the theoretical model, is a very satisfactory diagnostic score for
training and education programs. However, it should be used with caution in
research programs where the relationships being examined are theoretical in nature.
In such instances, the assumptions of the theory need to be represented as accurately
as possible by the statistical models being employed. The IDI comes close, but it
remains for future research to refine it further. At this point, the six stage scores are
better research measures.
In summary, our research suggests that Hammer & Bennett’s Intercultural
Development Inventory is a sound instrument, a satisfactory way of measuring
intercultural sensitivity as defined by Bennett (1993) in his developmental model. As
with most instrument measuring a multidimensional human quality, it will
undoubtedly undergo revisions.

References

Adler, P. (1975). The transition experience: An alternative view of culture shock. Journal of Humanistic
Psychology, 15(4), 13–23.
Adler, P. (1976). Beyond cultural identity: Reflections upon cultural and multicultural man. In L. A.
Samovar, & R. E. Porter (Eds.), Intercultural communication: A reader (2nd ed.) (pp. 180–362).
Belmont, CA: Wadsworth.
Bennett, M. J. (1986). Towards ethnorelativism: A developmental model of intercultural sensitivity. In
R. M. Paige (Ed.), Cross-cultural orientation: New conceptualizations and applications (pp. 27–69). New
York: University Press of America.
Bennett, M. J. (1993). Towards ethnorelativism: A developmental model of intercultural sensitivity. In
R. M. Paige (Ed.), Education for the intercultural experience (pp. 21–71). Yarmouth, ME: Intercultural
Press.
Black, T. R. (1999). Doing quantitative research in the social sciences. London: Sage.
Bochner, S. (1977). The mediating man and cultural diversity. In R. W. Brislin (Ed.), Culture learning:
Concepts, application, and research. Honolulu: University of Hawaii Press.
Crocker, L., & Algina, J. (Eds.), (1986). Introduction to classical and modern test theory. Orlando, FL:
Harcourt, Brace & Janovich College Publishers.
Dinges, N. (1983). Intercultural competence. In D. Landis, & R. W. Brislin (Eds.), Handbook of
intercultural training, Vol. 1. (pp. 176–202). Elmsford, NY: Pergamon.
Gudykunst, W. B., & Hammer, M. R. (1987). Strangers and hosts: An uncertainty reduction based theory
of intercultural adaptation. In Y. Y. Kim, & W. B. Gudykunst (Eds.), Cross Cultural Adaptation-
Current Approaches (pp. 106–139). Newbury Park: Sage.
Hammer, M. R., & Bennett, M. J. (1998). The intercultural development inventory (IDI) manual. Portland,
OR: Intercultural Communication Institute.
Hammer, M. R., Bennett, M. J., & Wiseman, R. (2003). Measuring intercultural sensitivity: The
Intercultural Development Inventory. International Journal of Intercultural Relations, 27(4),
421–443.
Hoopes, D. S. (1979). Intercultural communication concepts and the psychology of intercultural
experience. In M. D. Pusch (Ed.), Multicultural education: A cross-cultural training approach (pp. 9–38).
La Grange Park, IL: Intercultural Press.
ARTICLE IN PRESS
486 R. Michael Paige et al. / International Journal of Intercultural Relations 27 (2003) 467–486

Kelley, Meyers (1992). The Cross-Cultural Adaptability Inventory. Yarmouth, ME: Intercultural Press.
Landis, D., & Bhagat, R. S. (1996). A model of intercultural behavior and training. In D. Landis, & R. S.
Bhagat (Eds.), Handbook of intercultural training (2nd ed.) (pp. 1–13). Thousand Oaks, CA: Sage.
Lysgaard, S. (1955). Adjustment in a foreign society: Norwegian Fulbright Grantees visiting the United
States. International Social Science Bulletin, 7, 45–51.
Lustig, M. W., & Koester, J. (2003). Intercultural competence: Interpersonal communication across cultures,
(4th Ed.). New York: Allyn & Bacon/Longman.
Martin, J.N. (Guest Ed.), (1989). Special issue on Intercultural communication competence. International
Journal of Intercultural Relations, 13, 227–428.
Reddin, W. J. (1994). Using test to improve training: The complete guide to selecting, developing and using
training instruments. Englewood Cliffs, NJ: Prentice-Hall.
Sampson, D. L., & Smith, H. P. (1957). A scale to measure world-minded attitudes. Journal of Social
Psychology, 45, 99–106.
SPSS (1994). SPSS 6.1 Syntax Reference Guide. Chicago, IL: Author.
Stephan, W. G., & Stephan, C. W. (1985). Intergroup anxiety. Journal of Social Issues, 41, 157–176.
Strahan, R., & Gerbasi, K. (1972). Short, homogenous version of the Marlowe-Crowne Social Desirability
Scale. Journal of Clinical Psychology, 28, 191–193.
Tabachnick, B., & Fidell, L. (1996). Using multivariate statistics (3rd ed.). Northridge, CA: HarperCollins
College Publishers.

You might also like