Archives of Clinical Neuropsychology 29 (2014) 60–74

Spanish Multicenter Normative Studies (NEURONORMA Project):

Normative Data and Equivalence of Four BNT Short-Form Versions
Marta Casals-Coll1,2, Gonzalo Sánchez-Benavides 1, Sandra Meza-Cavazos 1,3, Rosa M. Manero4,
Miguel Aguilar5, Dolors Badenes5, José Luis Molinuevo6, Alfredo Robles7, Marı́a Sagrario Barquero8,
Carmen Antúnez9, Carlos Martı́nez-Parra 10, Anna Frank-Garcı́a 11, Manuel Fernández12, Rafael Blesa13,
Jordi Peña-Casanova1,3,*, for the NEURONORMA Study Team
1
Behavioral Neurology Group, Institut Hospital del Mar d’Investigacions Médiques, Barcelona, Spain
2
Universitat Autû˙noma de Barcelona, Spain
3
Service of Geriatrics, Internal Medicine, Hospital Universitario Dr. José E. Gonzalez, Monterrey, México
4
Section of Behavioral Neurology and Dementias, Hospital del Mar, Barcelona, Spain
5
Service of Neurology, Hospital Mútua de Terrassa, Terrassa, Spain
6
Service of Neurology, Hospital Clı́nic, Barcelona, Spain
7
Service of Neurology, Hospital Clı́nico Universitario, Santiago de Compostela, Spain
8
Service of Neurology, Hospital Clı́nico San Carlos, Madrid, Spain
9
Service of Neurology, Hospital Virgen Arrixaca, Murcia, Spain
10
Service of Neurology, Hospital Virgen Macarena, Sevilla, Spain
11
Department of Neurology, Hospital Universitario La Paz, Madrid, Spain
12
Service of Neurology, Hospital de Cruces, Bilbao, Spain
13
Service of Neurology, Hospital de la Santa Creu i Sant Pau, Barcelona, Spain
*Corresponding author at: Section of Behavioral Neurology and Dementias, Hospital del Mar, IMIM, Biomedical Research Park Building,
Carrer Dr. Aiguader, 88, 08003 Barcelona, Spain. Tel.: +34-933160778; fax: +34-933160723.
E-mail address: [email protected] (J. Peña-Casanova).
Accepted 9 October 2013

Abstract
The application of the Boston Naming Test (BNT) is time-consuming and shortened versions need to be developed for screening purposes. The
aims of this study were to develop four equivalent 15-item forms of a Spanish adaptation of the BNT, to test the equivalence of the new versions in
a clinical sample, and to provide normative data. The normative sample consisted of 340 subjects. The clinical sample included 172 patients
(76 Mild Cognitive Impairment and 96 Alzheimer’s disease). An empirical procedure was used to develop the shortened versions. All new
versions demonstrated satisfactory internal consistency. Pearson’s coefficient analysis showed strong relationships among the four short-
form versions as well as between each of them and the 60-item test. The inferential confidence interval method demonstrated the equivalence
between the four shortened versions. Age and education affected the score of all short-form versions, but sex was found to be unrelated to
the performance. Normative data were calculated for midpoint age groups. This paper proposes four 15-item equivalent versions that could
be useful and time-saving tools for screening purposes.

Keywords: Boston Naming Test; Short forms; Equivalence; Normative data

Introduction

Word retrieval difficulty is one of the most prevalent symptoms associated with neuropsychological impairment. Anomia has
been related to many neurological disorders such as dementia, left temporal lobe epilepsy, or cerebrovascular events. The Boston
Naming Test (BNT; Kaplan, Goodglass, & Weintraub, 1978) is one of the most commonly used tests for the assessment of con-
frontation naming (Rabin, Barr, & Burton, 2005). Several reports have shown that the BNT efficiently discriminates the healthy

# The Author 2013. Published by Oxford University Press. All rights reserved. For permissions, please e-mail: [email protected].
doi:10.1093/arclin/act085 Advance Access publication on 11 November 2013
 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74 61

elderly from those with dementia (LaBarge, Edwards, & Knesevich, 1986; Lansing, Ivnik, Cullum, & Randolph, 1999; Martin &
Fedio, 1983; Welch, Doineau, Johnson, & King, 1996).
A first experimental version of the BNT included 85 items (Kaplan et al., 1978). The modified version, published in 1983, was
limited to 60 drawings (Kaplan, Goodglass, & Weintraub, 1983). In 2001, Kaplan, Goodglass, and Weintraub published the second
edition of the test, with multiple-choice options and error codes to categorize incorrect responses, which included a 15-item short-
form developed by Mack, Freed, Williams, and Henderson (1992). There is a large volume of published normative studies con-
cerning the BNT in various languages (see Lezak, Howieson, & Loring, 2004; Mistrushina, Boone, Razani, & D’Elia, 2005;
Strauss, Sherman, & Spreen, 2006; for a review). The contribution of the MOANS and MOAANS projects to this test has been
considerable providing normative data for elder Americans and Afro-Americans subjects (Lucas et al., 2005; Steinberg,
Bieliauskas, Smith, Langellotti, & Ivnik, 2005). Zec, Burkett, Markwell, and Larsen (2007) also provided normative data from
a sample of 1026 subjects aged over 49 years.
There are a number of versions of the BNT in Spanish. A first 60-item version was proposed as an adaptation to Spanish from the
modified version (Kaplan, Goodglass, & Weintraub, 1986). A second modified version was proposed as a new adaptation for
Spanish speaking populations (Kaplan, Goodglass, & Weintraub, 2005). Previous studies have presented normative data in
Spanish, one from an Argentinean sample (Allegri et al., 1997), others from a young adult Spanish sample (Aranciva et al.,
2012; Quiñones-Ubeda, Peña-Casanova, Böhm, Gramunt-Fombuena, & Comas, 2004). Recently, two studies have published nor-
mative data from a sample of elderly Spanish subjects (Peña-Casanova, Quiñones-Ubeda, et al., 2009; Rami et al., 2008).
Various short-form versions of BNT have been proposed. They could offer many advantages in clinical assessment, especially
when impaired or low educational level patients are examined (e.g., short forms may be less taxing for low educational level
patients). Short-form versions are also essential in the test– retest designs in treatment monitoring or in research studies
(Calero, Arnedo, Navarro, Ruiz-Pedrosa, & Carnero, 2002; Fastenau, Denburg, & Mauer, 1998; Mack et al., 1992). The admin-
istration of the 60-item version of the BNT requires 15– 30 min, time that is beyond the scope of a routine screening for cognitive
impairment (Deveugele, Derese, van den Brink-Muinene, Bensing, & De Maeseneer, 2002; Villarejo & Puertas-Martı́n, 2011). In
order to improve the efficiency of the assessment, shorter versions of the BNT have been developed. A variety of methods was used
to construct abbreviated versions. Specifically, for the 30-item versions, an even and an odd version were developed from the
60-item test (Williams, Mack, & Henderson, 1989). The same authors built an empirical version by examining differences
between group means of individual items from normal controls and patients with Alzheimer disease (Williams et al., 1989).
These two versions have been analyzed by a large number of authors (Fisher, Tierney, Snow, & Szalai, 1999; Graves, Bezeau,
Fogarty, & Blair, 2004; Jefferson et al., 2007; Kent & Luszcz, 2002; Lansing et al., 1999; Mack et al., 1992; Tombaugh &
Hubley, 1997). Fastenau and colleagues (1998) combined the four 15-item versions developed by Mack and colleagues (1992)
to construct two 30-item versions. Saxton and colleagues (2000) created two empirical versions on the basis of achieving equiva-
lent word frequency. Using item response theory, Graves and colleagues (2004) developed another 30-item version treating the
difficulty of each item as information to be incorporated in the selection of items.
Regarding 15-item versions, the first proposal was suggested in 1989 by the Consortium to Establish a Registry for Alzheimer’s
disease (CERAD). This version was later assessed by several authors (Kent & Luszcz, 2002; Lansing et al., 1999; Mack et al., 1992).
In addition, Teng and colleagues (1989) developed four 15-item versions by using item redistribution according to its difficulty.
Afterwards, Mack and colleagues (1992) published four 15-item versions in order to establish equivalence among them. An empirical
version was created on the basis of the item discrimination capacity between normal controls and Alzheimer’s disease patients
(Lansing et al., 1999). Graves and colleagues (2004) made a 15-item version using the same methodology as in their 30-item version.
Although a considerable amount of shortened versions of the test have been developed for diverse populations, most of the
authors conclude that a reliable shortened version should be specially designed and validated for each target population. As all
these shortened versions of BNT were devised for English-speaking communities, several items were not appropriate for popula-
tions with a different cultural background. On these grounds, Serrano and colleagues (2001) developed a 12-item shortened
version adapted to Argentina. Later, Calero and colleagues (2002) tested the shortened version proposed by Mack and colleagues
(1992) in southern Spain. Recently, Nebreda and colleagues (2011) proposed a short-form version for language screening in de-
mentia in a bilingual rural community in Galicia (Fernández-Blázquez et al., 2012).
The present report is part of the NEURONORMA project, a multicenter study of normalization and validation of neurocognitive
and functional tools with genetic and neuroimaging correlation for the detection, diagnosis, and follow-up of cognitive deterioration
in aging and dementia (Peña-Casanova, Blesa, et al., 2009), which represents the most comprehensive and largest normative study for
a Spanish population on selected neuropsychological tests. This project was designed following a specific methodology allowing the
co-normalization of a battery of tests covering different cognitive areas. Accordingly, the hypothesis we tested in this study was that
four 15-item specially designed shortened versions of the BNT could be used equally in a dementia screening context.
The aims of the study were therefore: (a) to design four shortened versions as an alternative to the 60-item BNT; (b) to study
equivalence among each other; and (c) to provide normative data for the four short versions from a Spanish adult control sample.
62 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74

Materials and Methods

Research Participants

A total sample of 535 subjects was included in the entire NEURONORMA project. Participants included 356 cognitively normal
adults, 79 patients diagnosed with Mild Cognitive Impairment (MCI), and 100 patients diagnosed with AD. The ethnic background of
all participants was Caucasian. The project was reviewed and approved by the Research Ethics Committee of the Municipal Institute
of Medical Care of Barcelona, Spain, and was conducted in accordance with the Declaration of Helsinki (World Medical Association,
1977) and its subsequent amendments, and the European Union regulations concerning medical research. All participants were
required to have an informant who could answer questions about their cognition, function, and health.
The diagnosis, included the selection of controls, was based on a clinical interview with the patient and the informant and phys-
ical and neurological examination. An initial cognitive assessment was completed by trained neuropsychologists. The selection
tests were the Mini-Mental State Examination (Folstein, Folstein, & McHugh, 1975) in a validated Spanish version (Blesa et al.,
2001), the abbreviated Barcelona Test (Peña-Casanova, 2005; Quintana et al., 2011, 2012), and the Interview for Deterioration of
Daily Living in Dementia (IDDD; Teunisse, Derix, & Crever, 1991) in its validated Spanish version (Böhm et al., 1998) was used
to evaluate functional status.

Cognitively normal adult definition. Entry criteria for the control group included consecutive individuals according to a series of
inclusion and exclusion criteria. Recruitment procedures and socio-demographic characteristics of the sample have been reported
in a previous paper (Peña-Casanova, Blesa, et al., 2009). Volunteers did not need to be completely medically healthy to participate
(Pedraza et al., 2005). Patients with active, chronic medical, psychiatric, or neurological conditions or physical disabilities or han-
dicaps were included if the researcher judged that the condition was correctly controlled or resolved and not causing cognitive
impairment. The same criterion was applied in the case of use of psychoactive medications. Three hundred and forty subjects
were included in the BNT control sample (Peña-Casanova, Quiñones-Ubeda, et al., 2009), 202 women and 138 men, between
50 and 85 years old, with a mean age of 65 years (SD ¼ 9.3) and a mean education of 10.5 years (SD ¼ 5.5). The BNT normative
sample demographic information is presented in Table 1.

The clinical sample definition. All patients were recruited from dementia consultations at nine hospitals from different regions of
Spain (see Peña-Casanova, Blesa, et al., 2009 for details). As part of the recruitment protocol subjects were interviewed by skilled
clinicians to explore relevant medical antecedents and the presence of depressive symptoms, functional disability, and cerebro-
vascular risk. For such purpose, the Hamilton Depression Rating Scale (Hamilton, 1960), the IDDD, and the Modified Ischemia
Score (Rosen, Terry, Fuld, Katzman, & Peck, 1980) were administered.
The current sample was composed of patients with cognitive impairment who came from the NEURONORMA project
(Quintana et al., 2011) and to whom the BNT was administered. One hundred and seventy-two subjects were included in the

Table 1. Normative sample sociodemographic characteristics

Count Percent of total

Sex
Men 138 40.6
Women 202 59.4
Groups of age (years)
50–56 75 22.0
57–59 51 15.0
60–62 33 9.7
63–65 18 5.3
66–68 25 7.3
69–71 49 14.4
72–74 31 9.1
75–77 29 8.5
78–80 19 5.6
.80 10 2.9
Education (years)
0 –5 69 20.3
6 –12 151 44.4
.12 120 35.3
 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74 63

BNT clinical sample, 105 women and 67 men, with a mean age of 73 years (SD ¼ 7.13) and a mean education of 7.83 years
(SD ¼ 4.78). Seventy-six patients were diagnosed with amnestic MCI according the following criteria: (a) presence of
subjective memory complaints; (b) functional abilities essentially preserved confirmed by an informant; (c) a minimal evolu-
tion of 6 months; (d) evidence of objective memory impairment, defined as a score below the 15th percentile of the age and
education-matched control group on a test of verbal delayed recall (Barcelona Test paragraph recall), but no other significant
deficits on neuropsychological examination; (e) National Institute of Neurological and Communication Disorders and
Stroke-Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA; McKhann et al., 1984) criteria for probable
AD not met; and (f) a Global Deterioration Scale (GDS; Reisberg, Ferris, de Leon, & Crook, 1982) score of 3. Ninety-six patients
with a diagnosis of probable Alzheimer disease fulfilled Diagnostic and Statistical Manual for the Mental Disorders-Fourth
Edition criteria (DSM-IV; American Psychiatric Association, 1994) and NINCDS-ADRDA criteria; they were at mild to moderate
dementia stages and were at GDS stages 4 and 5.

Neuropsychological Measures

The neuropsychological tests of the entire project were administered as part of the NEURONORMA battery (Peña-Casanova,
Blesa, et al., 2009). Although other measures were included as part of this larger study, the authors chose only to investigate the
BNT for the purpose of this paper. Tests were administered and scored by neuropsychologists specifically trained for this project.
The test consists of a book with 60 cards with black and white drawings to be named. On the back of each card there are four
printed name choices (semantically, phonologically, or perceptively related to the word target). Following the BNT instructions,
participants were allowed up to 20 s to name each item. A stimulus cue was provided when no response was given after 20 s, the
participant stated that he/she did not know the name, or the item was incorrectly perceived. First, he/she was given the semantic
stimulus cue and again allowed up to 20 s to name the picture. If the participant still did not recognize the picture after receiving the
semantic cue, or misnamed the picture, the examiner proceeded to phonemic cueing. In those cases in which the participant named
the semantic class, or superordinate, he/she was prompted to give the specific name. Finally, after the test was completed, the exam-
iner returned to each item incorrectly named after a phonemic cue and presented the card with that item and the four printed name
choices. The examiner read each word and asked the participant to indicate the correct name. The total score was the number of
correct responses produced spontaneously and with the aid of semantic cue. Maximum score was 60 for the 60-item version and
15 for the four shortened versions that were developed specifically for this study.
A Spanish experimental adaptation (Quiñones-Úbeda et al., 2004) of the latest BNT version (Kaplan et al., 2001) was applied in
this study. This latest BNT version was slightly modified in order to adapt the test to Spain. The process of adaptation was described
in previous papers (Peña-Casanova, Quiñones-Ubeda, et al., 2009; Quiñones-Úbeda et al., 2004). Briefly, the procedure was the
following: A multiple choice form was developed following a three-step process. First, three translations into Spanish were pre-
pared by researchers. Second, the three translations were reconciled into one version, and words checked in the official dictionary
of the Royal Academy of Spanish Language (2001). Finally, this version was tested empirically for clarity and acceptability of
wording with four individuals who were native in Spanish. In order to adapt the vocabulary to Spain, an extra item was added:
“magdalena” (a cupcake or fairy cake made in a small paper cup container [muffin]), as an alternative to “pretzel.” Standard
instructions indicate beginning with Item 30 and proceeding forward unless an error is made prior to achieving eight consecutive
correct responses. Instead of standard instructions, as this was a normative study, the test was administered entirely from Item 1 to
Item 60 to know the specific contribution of each item on the final score.

Data Management and Statistical Analyses

With the purpose of developing four 15-item short versions (A, B, C, and D), BNT items were arranged by correct answer per-
centage in decreasing order. An item named spontaneously or after being provided with the semantic cue was considered a correct
answer. Subsequently, an item balanced assignment to short forms was completed (Table 2). It was performed by carrying out the
following sequence: A, B, C, D – D, A, B, C – C, D, A, B – B, C, D, A, and so on, to balance difficulty among the four versions
(Table 3). Basic descriptive analyses included mean and standard deviation. In order to obtain a measure of internal consistency,
Cronbach’s alpha was calculated.
Following the suggestion of a reviewer and the indications of Ross, Furr, Carter, and Weinberg (2006), this study employed
Tryon’s (2001) method for examining statistical equivalence between all shortened versions using inferential confidence intervals
(CIs) as an adjunct to null hypothesis significance testing (NHST). This first method compares the maximum probable difference
between two means, also called Rg, to delta, a value that represents an amount considered inconsequential. Rg is the range of two
appropriately reduced inferential CIs. Rg begins with the lower CI limit of the lesser mean and extends to the upper CI limit of the
greater mean. If Rg is less than delta, then the maximum amount of difference between test forms can be considered
64 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74

Table 2. Correct answer percentage from the results in the normative sample (n ¼ 340) and item assignment to A, B, C, and D short-form versions

Correct answer (%) Item Word Version

100.000 1 Cama (bed) A

100.000 2 Árbol (tree) B
100.000 4 Casa (house) C
100.000 6 Tijeras (scissors) D
100.000 7 Peine (comb) D
100.000 12 Escoba (broom) A
99.709 8 Flor (flower) B
99.709 10 Cepillo de dientes (toothbrush) C
99.709 22 Caracol (snail) C
99.128 9 Sierra (saw) D
99.128 20 Banco (bench) A
98.837 5 Silbato (whistle) B
98.837 37 Escaleras mecánicas (escalator) B
98.547 16 Silla de ruedas (wheelchair) C
98.256 17 Camello (camel) D
97.965 13 Pulpo (octopus) A
97.965 14 Seta (mushroom) A
97.674 15 Percha (hanger) B
97.384 3 Lápiz (pencil) C
97.384 46 Embudo (funnel) D
97.093 21 Raqueta (racquet) D
96.802 11 Helicóptero (helicopter) A
96.802 44 Bozal (muzzle) B
96.221 18 Máscara (mask) C
95.349 47 Acordeón (accordion) C
95.058 49 Espárrago (asparagus) D
94.477 28 Corona de flores (wreath) A
92.733 23 Volcán (volcano) B
92.442 32 Bellota (acorn) B
91.860 43 Pirámide (pyramid) C
90.698 35 Dominó (dominoes) D
90.116 54 Pinzas (tongs) A
88.372 50 Compás (compass) A
85.756 36 Cactus (cactus) B
84.302 30 Armónica (harmonica) C
84.302 31 Rinoceronte (rhinoceros) D
83.721 24 Caballito de mar (seahorse) D
81.977 61 Magdalena (muffin) A
80.523 38 Arpa (harp) B
79.942 27 Globo terráqueo (globe) C
72.384 52 Trı́pode (tripod) C
69.477 39 Hamaca (hammock) D
66.279 40 Aldaba (knocker) A
65.988 25 Dardo (dart) B
62.791 48 Soga (noose) B
62.500 53 Pergamino (scroll) C
62.500 58 Paleta (palette) D
61.337 26 Canoa (canoe) A
59.012 41 Pelı́cano (pelican) A
58.721 34 Zancos (stilts) B
57.558 56 Yugo (yoke) C
55.233 51 Pestillo (latch) D
53.779 33 Iglú (igloo) D
52.035 45 Unicornio (unicorn) A
47.965 55 Esfinge (sphinx) B
30.523 42 Estetoscopio (stethoscope) C
26.744 60 Ábaco (abacus) C
19.767 29 Castor (beaver) D
14.826 59 Transportador (protractor) A
6.977 57 Pérgola (trellis) B
6.686 19 Pretzel (pretzel)
 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74 65

Table 3. Shortened versions A, B, C, and D

Item A B C D

1 Cama Árbol Casa Tijeras

2 Escoba Flor Cepillo de dientes Peine
3 Banco Silbato Caracol Sierra
4 Pulpo Escaleras mecánicas Silla de ruedas Camello
5 Seta Percha Lápiz Embudo
6 Helicóptero Bozal Máscara Raqueta
7 Corona de flores Volcán Acordeón Espárrago
8 Pinzas Bellota Pirámide Dominó
9 Compás Cactus Harmónica Rinoceronte
10 Magdalena Arpa Globo terráqueo Caballito de mar
11 Aldaba Dardo Trı́pode Hamaca
12 Canoa Soga Pergamino Paleta
13 Pelı́cano Zancos Yugo Pestillo
14 Unicornio Esfinge Estetoscopio Iglú
15 Transportador Pérgola Ábaco Castor

inconsequential, and the 95% CI test for statistical equivalence is satisfied (for an extensive description of the method, see Tryon,
2001). The present study predetermined a delta value of 3.2076 as a result of the following calculation: The lesser mean of the
control sample minus the greater mean of the clinical sample of the shortened versions scores. We opted for this value because
no agreed upon rationale for determining this value currently exists for the BNT shortened forms.
To test equivalence using NHST the analysis of variance (ANOVA) and a post hoc analysis (Scheffé), and Pearson’s correlation
coefficients were conducted to examine differences and correlations in the four shortened versions scores. To analyze the adequacy
of these shortened versions as an alternative to the 60-item BNT, Pearson’s correlation coefficients were calculated. All these ana-
lyses were carried out within the clinical sample.
Considering that the ability to compare all co-normed test scores directly with each other facilitates clinical interpretation of
neuropsychological test profiles, a uniform normative procedure was applied to all measures in the NEURONORMA project. Data
management and characteristics of statistics analyses for the normative procedures of the entire project have been reported exten-
sively in a previous paper (Peña-Casanova, Blesa, et al., 2009). Briefly the procedure was the following: (a) the overlapping inter-
val strategy (Pauker, 1988) was adopted to maximize the number of subjects contributing to the normative distribution at each
midpoint age interval. Each midpoint age group provides norms for individuals of that age, plus or minus 1 year; (b) determination
of age, sex, and education effects on raw subtest scores. Coefficients of correlation (r) and determination (R 2) of raw scores with
age, years of education, and sex were determined for the BNT; (c) creation of age-adjusted normative tables. To ensure a normal
distribution, the frequency distribution of the raw scores was converted into age-adjusted scaled scores, NSSA (NEURONORMA
scaled score age-adjusted). For each age range, a cumulative frequency distribution of the raw scores was generated. Raw scores
were assigned percentile ranks in function of their place within a distribution. Subsequently, percentile ranks were converted to
scaled scores (from 2 to 18) based on percentile ranges. This transformation of raw scores to NSSA produced a distribution close to
the normal distribution (mean ¼ 10; SD ¼ +3) on which linear regressions could be applied; (d) education adjustments. Years of
education were modeled using the following equation: NSSA ¼ k + (b × Education). The regression coefficient (b) from this ana-
lysis was used as the basis for education adjustments. A linear regression was employed to derive age- and education-adjusted
scaled scores. The following formula outlined by Mungas, Marshall, Weldon, Haan, and Reed (1996) was employed:

NSSA&E = NSSA − b × [Education − 12].

The obtained value was truncated to the next lower integer.

Results

Table 1 summarizes the normative sample’s demographic characteristics. Age distribution of this control sample made it pos-
sible to calculate norms for the following 10 midpoint age groups: 50– 56, 57– 59, 60 – 62, 63– 65, 66– 68, 69– 71, 72– 74, 75– 77,
78– 80, .80. Sample sizes resulting from midpoint age intervals are presented in A, B, C, and D normative tables (Tables 9 – 12
and 14– 17).
Means and standard deviations from the short-form versions of the normative and clinical sample are provided in Table 4.
No significant statistical differences were found between all short-form versions when an ANOVA and a post hoc were
66 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74

Table 4. Mean comparisons of the four shortened versions

A B C D ANOVA

Normative sample (N ¼ 340)

Mean (SD) 12.0424 (2.0858) 11.8970 (1.9086) 11.9515 (2.0296) 12.1000 (2.1591) N.S.
Clinical sample (N ¼ 172)
Mean (SD) 8.4410 (2.9492) 8.3416 (2.9519) 8.6398 (2.7353) 8.6894 (2.6838) N.S.

Table 5. Pearson’s correlation coefficient between the four shortened versions and the 60-item test within the clinical sample (N ¼ 172)
60-item test A B C D

60-item test 1 — — — —
A .919a 1 — — —
B .933a .804a 1 — —
C .926a .814a .803a 1 —
D .928a .779a .846a .822a 1
a
p , .0001.

Table 6. Descriptive statistics for scores for A, B, C, and D shortened versions for the clinical sample
Statistic A B C D

Mean 8.4410 8.3416 8.6398 8.6894

SD 2.9492 2.9519 2.7353 2.6838
N 161 161 161 161
df (N 2 1) 160 160 160 160
t (df) 1.9749 1.9749 1.9749 1.9749
SE 0.2324 0.2326 0.2156 0.2115
Notes: t(df) ¼ t value associated to N –1 for an alfa of 0.05.

Table 7. 95% inferential confidence intervals and Rg values for scores for A, B, C, and D shortened versions
B–A B–C C–D A –C A–D B–D

E 0.7071 0.7076 0.7071 0.7076 0.7079 0.7079

tx 1.3965 1.3975 1.3965 1.3974 1.3980 1.3980
A (95% CI) Lower 7.0445 7.0435 7.04300
Upper 9.8375 9.8384 9.8390
B (95% CI) Lower 6.9451 6.9441 6.9435
Upper 9.7381 9.7391 9.7396
C (95% CI) Lower 7.2423 7.2433 7.2423
Upper 10.0373 10.0363 10.0372
D (95% CI) Lower 7.2929 7.2914 7.2913
Upper 10.0859 10.0874 10.0874
Rg 2.8923 3.0931 2.8426 2.9937 3.0444 3.1439
Notes: E and tx are necessary values to calculate the 95% inferential CI. For more details see Tryon, 2001; CI ¼ confidence interval; Rg ¼ the maximum probable
difference associated with using different versions.

applied. The four short-form versions presented a satisfactory internal consistency when Cronbach’s alpha was calculated
(A version: a . 0.75; B version: a . 0.76; C version: a . 0.74; D version: a . 0.78). Table 5 shows strong relationships
among the four short-form versions as well as between each of them and the 60-item test based on Pearson’s correlation coefficient
analysis tested within the clinical group. Table 6 shows the required statistics to carry out calculations to test equivalence. Based on
these data, pairwise comparisons between the four shortened versions with the related inferential CIs and the resultant Rg are pre-
sented in Table 7. All the Rg values were less than 3.2076 for all combinations. Fig. 1 illustrates the inferential CIs plotted pairwise.
We can appreciate that all the groups presented overlapping inferential CIs and an Rg less than the amount that substantively
defines equivalence (Delta).
Pearson’s correlations (r) and determination coefficient (R 2) of the four BNT short-form version scores with years of age, years
of education, and sex are presented in Table 8. Age and education accounted significantly for the raw-score variance of all short-
 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74 67

Fig. 1. Inferential CIs plotted pairwise.

Table 8. Correlations (r) and share variance (R2) of the four BNT shortened version scores with years of age, years of education, and sex
Age (years) Education (years) Sex
2 2
r R r R r R2

A 2.261 .068 .435 .189 2.096 .009

B 2.298 .089 .483 .233 2.085 .007
C 2.273 .075 .588 .346 2.085 .007
D 2.291 .085 .456 .208 2.093 .009

form versions (R 2 . 0.05), indicative of a significant statistic effect of these variables on BNT total score. Sex differences were not
observed in any of the four versions, indicating that no need to control this demographic variable.
Age-adjusted NEURONORMA scaled scores (NSSA) for each BNT short-form version are provided in Tables 9 – 12 and
include percentile ranks, ranges of ages contributing to each normative subsample, and the number of participants contributing
to each test’s normative estimates. To use the table, select the appropriate column corresponding to the patient’s age, find the
patient’s raw score, and subsequently refer to the corresponding NSSA and percentile rank (left part of the tables). As expected,
the normative adjustments reduce shared variance of age to virtually ,1%. Education, however, continues to account significantly
for age-adjusted test-score variance in all cases, up to 15% in all versions (Table 13). The transformation of raw scores to NSSA
produced a distribution close to the normal distribution on which linear regressions could be applied. Regression coefficients from
this analysis were used as the basis for education corrections. The resulting computational formulae were used to calculate NSSA&E
(b ¼ 0.25243 for A version; b ¼ 0.26599 for B version; b ¼ 0.34083 for C version; b ¼ 0.24325 for D version). From these data,
we have constructed adjustment tables for age and education for each version (Tables 14– 17). To use the tables, select the appro-
priate column corresponding to the patient’s years of education, find the patient’s NSSA, and subsequently refer to the correspond-
ing NSSA&E.
68 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74

Table 9. Age-adjusted NEURONORMA scores (NSSA) for the BNT A shortened version

Scaled score Percentile range Age range

50–56 57– 59 60– 62 63–65 66– 68 69– 71 72–74 75– 77 78– 80 81+

2 ,1 ≤7 ≤6 ≤6 ≤6 ≤6 ≤6 ≤5 ≤4 ≤4 ≤4
3 1 7 7 5 5
4 2 7 6 5
5 3 –5 8 8 8 7– 8 7 –8 8 7 6
6 6 –10 9 9 8 7 6 6
7 11– 18 10 9 9 9 9 –10 10 9 8 –9 7 7
8 19– 28 10 10 10 10 8– 9 8– 9
9 29– 40 11 11 11 11 11 11 10 10 10
10 41– 59 12 12 12 12 12 12 11–12 11 11 11
11 60– 71 13 13 13 13 13 13 12
12 72– 81 13 13 12 12
13 82– 89 14 13
14 90– 94 14 14 14 13
15 95– 97 14 14 14
16 98 14 14
17 99 14
18 .99 15 15 15 15 15 15 15 15 15 15
Age range 50–60 53– 63 56– 66 59–69 62– 72 65– 75 68–78 71– 81 74– 84 77–90
Sample size 134 132 123 105 118 124 125 100 67 42

Table 10. Age-adjusted NEURONORMA scores (NSSA) for the BNT B shortened version
Scaled score Percentile range Age range
50–56 57– 59 60– 62 63–65 66– 68 69– 71 72–74 75– 77 78– 80 81+

2 ,1 ≤7 ≤7 ≤7 ≤6 ≤6 ≤6 ≤5 ≤5 ≤5 ≤5
3 1 8 7 7 6
4 2
5 3 –5 9 8 8 7 7 6 6 6
6 6 –10 10 9 9 8 8 8 8 7 7
7 11– 18 10 10 9 9 9 9 8 8 7
8 19– 28 11 11 11 10 10 10 9 9 8
9 29– 40 11 11 10 10 10 9– 10
10 41– 59 12 12 12 12 12 11 11 11 11 11
11 60– 71 12 12 12
12 72– 81 13 13 13 13 13 13 12 12
13 82– 89 13 13
14 90– 94 13 13
15 95– 97 14 14 14 14
16 98 14 14 14 14 14
17 99 14
18 .99 15 15 15 15 15 15 15 15 15 15
Age range 50–60 53– 63 56– 66 59–69 62– 72 65– 75 68–78 71– 81 74– 84 77–90
Sample size 134 132 123 105 118 124 125 100 67 42

Discussion

The main purpose of this report was to design four shortened versions as an alternative to the 60-item BNT in order to facilitate
clinical assessment, to study the equivalence between the shortened versions in a clinical sample, and to provide normative data
from a Spanish adult sample.
Four shortened versions were proposed. Following an empirical procedure an item assignation to each version was carried out
according to item difficulty analysis. This procedure is in line with other works (Saxton et al., 2000) and, even though a rigorous
 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74 69

Table 11. Age-adjusted NEURONORMA scores (NSSA) for the BNT C shortened version

Scaled score Percentile range Age range

50–56 57– 59 60–62 63–65 66– 68 69–71 72–74 75– 77 78–80 81+

2 ,1 ≤9 ≤6 ≤6 ≤6 ≤5 ≤6 ≤5 ≤5 ≤5 ≤5
3 1 6 6 6 6
4 2 7 7 6
5 3 –5 8 –9 7 7 7 8 7 7
6 6 –10 8– 9 8 8 9 8 7 7
7 11– 18 10 10 9 9 –10 9 8 –9
8 19– 28 10 10 10 10 10 8– 9 8
9 29– 40 11 11 11 11 11 10 9 –10
10 41– 59 12 12 12 12 11 11 11 11 11
11 60– 71 13 12 12 12 12
12 72– 81 13 13 13 13 12 12
13 82– 89 14 14 14 13– 14 14 13 13 13
14 90– 94 14 14 14 13
15 95– 97 14
16 98 14
17 99
18 .99 15 15 15 15 15 15 15 15 15 15
Age range 50–60 53– 63 56–66 59–69 62– 72 65–75 68–78 71– 81 74–84 77–90
Sample size 134 132 123 105 118 124 125 100 67 42

Table 12. Age-adjusted NEURONORMA scores (NSSA) for the BNT D shortened version
Scaled score Percentile range Age range
50–56 57– 59 60–62 63–65 66– 68 69–71 72–74 75– 77 78–80 81+

2 ,1 ≤7 ≤6 ≤7 ≤6 ≤6 ≤5 ≤5 ≤4 ≤4 ≤4
3 1 7 6 5 5
4 2 7 7 5
5 3 –5 8 7 6 –7 6 –7
6 6 –10 9 8 8 8 8 8 8 8 6– 7 6 –7
7 11– 18 10 9 9 9 9 9 9 8 8
8 19– 28 11 10 10 10 10 10 9 9
9 29– 40 12 11 11 11 11 11 10–11 10 9
10 41– 59 12 12 12 12 12 12 11 10–11 10–11
11 60– 71 13 13 13 13 13 12 12 12
12 72– 81 14 13 13
13 82– 89 14 14 14 14 13 13
14 90– 94 14 13
15 95– 97 14 14
16 98
17 99
18 .99 15 15 15 15 15 15 15 15 14 14
Age range 50–60 53– 63 56–66 59–69 62– 72 65–75 68–78 71– 81 74–84 77–90
Sample size 134 132 123 105 118 124 125 100 67 42

Table 13. Correlations (r) and share variance (R 2) of age-adjusted NEURONORMA scores (NSSA) with age and education
Age (years) Education (years)
2
r R r R2

A 2.073 .005 .417 .174

B 2.082 .007 .459 .211
C 2.083 .007 .564 .318
D 2.083 .007 .389 .151
70 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74

Table 14. Education adjustment for BNT A shortened version scale scores

NSSA Education (years)

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

2 5 4 4 4 4 3 3 3 3 2 2 2 2 1 1 1 0 0 0 0 -1
3 6 5 5 5 5 4 4 4 4 3 3 3 3 2 2 2 1 1 1 1 0
4 7 6 6 6 6 5 5 5 5 4 4 4 4 3 3 3 2 2 2 2 1
5 8 7 7 7 7 6 6 6 6 5 5 5 5 4 4 4 3 3 3 3 2
6 9 8 8 8 8 7 7 7 7 6 6 6 6 5 5 5 4 4 4 4 3
7 10 9 9 9 9 8 8 8 8 7 7 7 7 6 6 6 5 5 5 5 4
8 11 10 10 10 10 9 9 9 9 8 8 8 8 7 7 7 6 6 6 6 5
9 12 11 11 11 11 10 10 10 10 9 9 9 9 8 8 8 7 7 7 7 6
10 13 12 12 12 12 11 11 11 11 10 10 10 10 9 9 9 8 8 8 8 7
11 14 13 13 13 13 12 12 12 12 11 11 11 11 10 10 10 9 9 9 9 8
12 15 14 14 14 14 13 13 13 13 12 12 12 12 11 11 11 10 10 10 10 9
13 16 15 15 15 15 14 14 14 14 13 13 13 13 12 12 12 11 11 11 11 10
14 17 16 16 16 16 15 15 15 15 14 14 14 14 13 13 13 12 12 12 12 11
15 18 17 17 17 17 16 16 16 16 15 15 15 15 14 14 14 13 13 13 13 12
16 19 18 18 18 18 17 17 17 17 16 16 16 16 15 15 15 14 14 14 14 13
17 20 19 19 19 19 18 18 18 18 17 17 17 17 16 16 16 15 15 15 15 14
18 21 20 20 20 20 19 19 19 19 18 18 18 18 17 17 17 16 16 16 16 15
Notes: Education adjustment by applying the following formula: NSSA&E ¼ NSSA – (b × [education – 12]), where b ¼ 0.25243.

Table 15. Education adjustment for BNT B shortened version scale scores
NSSA Education (years)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

2 2 5 4 4 4 4 3 3 3 3 2 2 2 2 1 1 1 0 0 0 0
3 3 6 5 5 5 5 4 4 4 4 3 3 3 3 2 2 2 1 1 1 1
4 4 7 6 6 6 6 5 5 5 5 4 4 4 4 3 3 3 2 2 2 2
5 5 8 7 7 7 7 6 6 6 6 5 5 5 5 4 4 4 3 3 3 3
6 6 9 8 8 8 8 7 7 7 7 6 6 6 6 5 5 5 4 4 4 4
7 7 10 9 9 9 9 8 8 8 8 7 7 7 7 6 6 6 5 5 5 5
8 8 11 10 10 10 10 9 9 9 9 8 8 8 8 7 7 7 6 6 6 6
9 9 12 11 11 11 11 10 10 10 10 9 9 9 9 8 8 8 7 7 7 7
10 10 13 12 12 12 12 11 11 11 11 10 10 10 10 9 9 9 8 8 8 8
11 11 14 13 13 13 13 12 12 12 12 11 11 11 11 10 10 10 9 9 9 9
12 12 15 14 14 14 14 13 13 13 13 12 12 12 12 11 11 11 10 10 10 10
13 13 16 15 15 15 15 14 14 14 14 13 13 13 13 12 12 12 11 11 11 11
14 14 17 16 16 16 16 15 15 15 15 14 14 14 14 13 13 13 12 12 12 12
15 15 18 17 17 17 17 16 16 16 16 15 15 15 15 14 14 14 13 13 13 13
16 16 19 18 18 18 18 17 17 17 17 16 16 16 16 15 15 15 14 14 14 14
17 17 20 19 19 19 19 18 18 18 18 17 17 17 17 16 16 16 15 15 15 15
18 18 21 20 20 20 20 19 19 19 19 18 18 18 18 17 17 17 16 16 16 16
Notes: Education adjustment by applying the following formula: NSSA&E ¼ NSSA – (b × [education – 12]), where b ¼ 0.26599.

method to determine item difficulty for creation of these short forms was not used, the methodology utilized was superior to any
arbitrary method. A satisfactory internal consistency was reported for each short-form version.
It is interesting to note that high correlations were found among each short-form version and the original 60-item version when
Pearson’s correlations were calculated with scores from the clinical sample. These results suggest that the four shortened versions
would be a good alternative to the 60-item version. This study also examined equivalence of versions A, B, C, and D of the shor-
tened BNT. Using traditional NHST, these versions were similar across scores. The results showed that there were no significant
differences between the mean raw scores between the four short-form versions. Taking into account that NHST is an insufficient
form of evidence to argue for form equivalence (Nickerson, 2000), we applied the Tryon’s method. Using the inferential CI
method, the maximum probable difference between two means (Rg values) were calculated for each shortened version. This
Rg was compared to the delta value that represents an amount considered inconsequential. In this study, it was specified a
 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74 71

Table 16. Education adjustment for BNT C shortened version scale scores

NSSA Education (years)

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

2 6 5 5 5 4 4 4 3 3 3 2 2 2 1 1 0 0 0 21 21 21
3 7 6 6 6 5 5 5 4 4 4 3 3 3 2 2 1 1 1 0 0 0
4 8 7 7 7 6 6 6 5 5 5 4 4 4 3 3 2 2 2 1 1 1
5 9 8 8 8 7 7 7 6 6 6 5 5 5 4 4 3 3 3 2 2 2
6 10 9 9 9 8 8 8 7 7 7 6 6 6 5 5 4 4 4 3 3 3
7 11 10 10 10 9 9 9 8 8 8 7 7 7 6 6 5 5 5 4 4 4
8 12 11 11 11 10 10 10 9 9 9 8 8 8 7 7 6 6 6 5 5 5
9 13 12 12 12 11 11 11 10 10 10 9 9 9 8 8 7 7 7 6 6 6
10 14 13 13 13 12 12 12 11 11 11 10 10 10 9 9 8 8 8 7 7 7
11 15 14 14 14 13 13 13 12 12 12 11 11 11 10 10 9 9 9 8 8 8
12 16 15 15 15 14 14 14 13 13 13 12 12 12 11 11 10 10 10 9 9 9
13 17 16 16 16 15 15 15 14 14 14 13 13 13 12 12 11 11 11 10 10 10
14 18 17 17 17 16 16 16 15 15 15 14 14 14 13 13 12 12 12 11 11 11
15 19 18 18 18 17 17 17 16 16 16 15 15 15 14 14 13 13 13 12 12 12
16 20 19 19 19 18 18 18 17 17 17 16 16 16 15 15 14 14 14 13 13 13
17 21 20 20 20 19 19 19 18 18 18 17 17 17 16 16 15 15 15 14 14 14
18 22 21 21 21 20 20 20 19 19 19 18 18 18 17 17 16 16 16 15 15 15
Notes: Education adjustment by applying the following formula: NSSA&E ¼ NSSA – (b × [education – 12]), where b ¼ 0.34083.

Table 17. Education adjustment for BNT D shortened version scale scores
NSSA Education (years)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

2 4 4 4 4 3 3 3 3 2 2 2 2 2 1 1 1 1 0 0 0 0
3 5 5 5 5 4 4 4 4 3 3 3 3 3 2 2 2 2 1 1 1 1
4 6 6 6 6 5 5 5 5 4 4 4 4 4 3 3 3 3 2 2 2 2
5 7 7 7 7 6 6 6 6 5 5 5 5 5 4 4 4 4 3 3 3 3
6 8 8 8 8 7 7 7 7 6 6 6 6 6 5 5 5 5 4 4 4 4
7 9 9 9 9 8 8 8 8 7 7 7 7 7 6 6 6 6 5 5 5 5
8 10 10 10 10 9 9 9 9 8 8 8 8 8 7 7 7 7 6 6 6 6
9 11 11 11 11 10 10 10 10 9 9 9 9 9 8 8 8 8 7 7 7 7
10 12 12 12 12 11 11 11 11 10 10 10 10 10 9 9 9 9 8 8 8 8
11 13 13 13 13 12 12 12 12 11 11 11 11 11 10 10 10 10 9 9 9 9
12 14 14 14 14 13 13 13 13 12 12 12 12 12 11 11 11 11 10 10 10 10
13 15 15 15 15 14 14 14 14 13 13 13 13 13 12 12 12 12 11 11 11 11
14 16 16 16 16 15 15 15 15 14 14 14 14 14 13 13 13 13 12 12 12 12
15 17 17 17 17 16 16 16 16 15 15 15 15 15 14 14 14 14 13 13 13 13
16 18 18 18 18 17 17 17 17 16 16 16 16 16 15 15 15 15 14 14 14 14
17 19 19 19 19 18 18 18 18 17 17 17 17 17 16 16 16 16 15 15 15 15
18 20 20 20 20 19 19 19 19 18 18 18 18 18 17 17 17 17 16 16 16 16
Notes: Education adjustment by applying the following formula: NSSA&E ¼ NSSA – (b × [education – 12]), where b ¼ 0. 24325.

generic delta of 3.2076. We also opted for presenting the Rg values to allow comparisons criteria considering other uses of the
shortened versions. All shortened versions showed Rg values below a delta of 3.2076; therefore, considering this value, the
four versions would be equivalent. Consequently, these forms could be used interchangeably for most practical purposes in
retest clinical assessment. These findings have important implications for neuropsychological assessment in a multidisciplinary
clinical context in which effectiveness and efficiency are crucial.
In order to obtain adjusted normative data from the four short-form versions, sociodemographic factors were analyzed. The
results showed that raw scores of all versions were influenced by education and age. This finding is in agreement with previous
studies that observed significant effects of age on shortened BNT version scores (Fastenau et al., 1998). We found a weak correl-
ation between age and the scores obtained in the four short-form versions of the BNT. This correlation was weak probably because
72 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74

of the range of age observed. We could hypothesize that more differences had been found in a sample with a wider range of age.
When normative adjustments were applied, the variance produced by age was minimized. Similarly, education had a significant
impact on raw scores. These results are consistent with other reports which found years of education to have a considerable influ-
ence on BNT scores (Jefferson et al., 2007; Lansing et al., 1999; Pontón et al., 1996; Saxton et al., 2000; Tombaugh & Hubley,
1997). It is important to clarify that we included a group of participants with 0 – 5 years of education. This group of low educated
subjects, even if commonly seen for the post-war generations in Spain, could have probably driven the results. In a similar manner
to previous reports (Calero et al., 2002), sex did not affect the raw scores of any of the shortened versions tested in this study.
The combination of these findings supports the use of these shortened versions for clinical assessment and research. Even so,
this study presents limitations. The most important one lies in the fact that these data stemmed from the 60-item version admin-
istration. However, in reviewing the literature (Lansing et al., 1999; Mack et al., 1992; Saxton et al., 2000), it could be hypothesized
that our results would have been similar if data had been originated from separate administrations. This procedure facilitates
control of some confounding variables, such as test retest variability. Additionally, as the BNT normative study of the
NEURONORMA project justified (Peña-Casanova, Quiñones-Ubeda, et al., 2009), this study utilized the word “magdalena”
as an alternative to “pretzel” because of its extremely low frequency of correct responses presented in a Spanish control
sample. As a result, the four short-form versions maintained the original items but changed “pretzel” for “magadalena.” This
condition confers some difficulties when comparing the results of this study to previous Spanish studies data. Finally, as in all
normative studies, the validity of these norms is clearly dependent upon the similarity between the characteristics of the
studied subjects and the demographic features of the normative sample. Despite the fact that some American countries share
the same language with Spain, cultural and linguistic differences do exist. Therefore, it would be advisable to apply these
norms with caution to these other Spanish speaking populations. For future research, and in order to establish the adaptation of
these BNT norms, it would be interesting to study the test performance in these populations.
In summary, this paper presents normative data for 15-item specially designed shortened versions of the BNT. Our data show
good equivalence among the shortened versions of the BNT. In addition, the four short-forms are presented as a good alternative to
the original BNT. As a result, the four 15-item versions can be used instead of the original BNT for screening purposes.

Funding

This study was mainly supported by a grant from the Pfizer Foundation and by the Medical Department of Pfizer, SA, Spain. It
was also supported by the Neurofunctionality of Brain and Language group of the Program of Neurosciences of the Institut
Hospital del Mar d’Investigacions Mèdiques, Barcelona, Spain. JP-C has received an intensification research grant from the
CIBERNED (Centro de Investigación Biomédica en Red sobre Enfermedades Neurodegenerativas), Instituto Carlos III
(Ministry of Health & Consumer Affairs of Spain).

Conflict of Interest

None declared.

Appendix: Members of the NEURONORMAES Study Team

Steering Committee: JP-C, Hospital del Mar, Barcelona, Spain; RB, Hospital de la Santa Creu i Sant Pau, Barcelona, Spain; MA,
Hospital Mútua de Terrassa, Terrassa, Spain.
Principal Investigators: JP-C, Hospital del Mar, Barcelona, Spain; RB, Hospital de la Santa Creu i Sant Pau, Barcelona, Spain;
MA, Hospital Mútua de Terrassa, Terrassa, Spain; JLM, Hospital Clı́nic, Barcelona, Spain; AR, Hospital Clı́nico Universitario,
Santiago de Compostela, Spain; MSB (deceased), Hospital Clı́nico San Carlos, Madrid, Spain; CA, Hospital Virgen Arrixaca,
Murcia, Spain; CM-P, Hospital Virgen Macarena, Sevilla, Spain; AF-G, Hospital Universitario La Paz, Madrid, Spain; MF,
Hospital de Cruces, Bilbao, Spain.
Genetics substudy: RO, Service of Genetics, Hospital Clı́nic, Barcelona, Spain.
Neuroimaging substudy: BG-A, Radiology Department and IDIBAPS, Hospital Clı́nic, Barcelona, Spain. Research Fellows:
Gemma Monte, Elena Alayrach, Aitor Sainz, and Claudia Caprile, Fundació Clı́nic, Hospital Clı́nic, Barcelona, Spain; Gonzalo
Sánchez, Behavioral Neurology Group, Institut Municipal d’Investigació Mèdica. Barcelona, Spain.
Clinicians, Psychologists, and Neuropsychologists: Nina Gramunt (coordinator), Peter Böhm, Sonia González, Yolanda
Buriel, Marı́a Quintana, Sonia Quiñones, Gonzalo Sánchez, Rosa M. Manero, Gracia Cucurella, Institut Municipal
d’Investigació Mèdica, Barcelona, Spain; Eva Ruiz, Mónica Serradell, Laura Torner, Hospital Clı́nic, Barcelona, Spain;
 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74 73

Dolors Badenes, Laura Casas, Noemı́ Cerulla, Silvia Ramos, Loli Cabello, Hospital Mútua de Terrassa, Terrassa, Spain; Dolores
Rodrı́guez, Clinical Psychology and Psychobiology Department, University of Santiago de Compostela, Spain; Marı́a Payno,
Clara Villanueva, Hospital Clı́nico San Carlos, Madrid, Spain; Rafael Carles, Judit Jiménez, Martirio Antequera, Hospital
Virgen Arixaca, Murcia, Spain; Jose Manuel Gata, Pablo Duque, Laura Jiménez, Hospital Virgen Macarena, Sevilla, Spain;
Azucena Sanz, Marı́a Dolores Aguilar, Hospital Universitario La Paz, Madrid, Spain; Ana Molano, Maitena Lasa, Hospital de
Cruces, Bilbao, Spain.
Administrative Management: Carme Pla (deceased), Romina Ribas, Department of Psychiatry and Forensic Medicine,
Universitat Autònoma de Barcelona, and Behavioral Neurology Group, Institut Municipal d’Investigació Mèdica, Barcelona,
Spain.

References

Allegri, R. F., Mangone, C. A., Fernández Villavicencio, A., Rymberg, S., Taragano, F. E., & Baumann, D. (1997). Spanish Boston naming test norms. The Clinical
Neuropsychologist, 11 (4), 416– 420.
American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.
Aranciva, F., Casals-Coll, M., Sánchez-Benavides, G., Quintana, M., Manero, R. M., Rognoni, T., et al. (2012). Estudios normativos españoles en población adulta
joven (Proyecto NEURONORMA jóvenes): Normas para el Boston Naming Test y el Token Test. Neurologı́a, 27 (7), 394– 399.
Blesa, R., Pujol, M., Aguilar, M., Santacruz, P., Bertran-Serra, I., Hernández, G., et al. (2001). Clinical validity of the “mini-mental state” for Spanish Speaking
communities. Neuropsychologia, 39 (11), 1150– 1157.
Böhm, P., Peña-Casanova, J., Aguilar, M., Hernández, G., Sol, J. M., & Blesa, R. (1998). Clinical validity and utility of the interview for deterioration of daily living
in dementia for Spanish-speaking communities. International Psychogeriatrics, 10 (3), 261–270.
Calero, M. D., Arnedo, M. L., Navarro, E., Ruiz-Pedrosa, M., & Carnero, C. (2002). Usefulness of a 15-Item version of the Boston naming test in neuropsychological
assessment of low-educational elders with dementia. Journals of Gerontology, Series B: Psychological Sciences and Social Sciences, 57 (2), 187– 191.
Deveugele, M., Derese, A., van den Brink-Muinene, A., Bensing, J., & De Maeseneer, J. (2002). Consultation length in general practice: Cross sectional study in six
European countries. British Medical Journal, 32 (7362), 472.
Fastenau, P. S., Denburg, N. L., & Mauer, B. A. (1998). Parallel short forms for the Boston naming test: Psychometric properties and norms for older adults. Journal
of Clinical and Experimental Neuropsychology, 20 (6), 828–834.
Fernández-Blázquez, M. A., Ruiz-Sánchez de León, J. M., López-Pina, J. A., Llanero-Luque, M., Montenegro-Peña, M., & Montejo-Carrasco, P. (2012). Nueva
versión reducida del test de denominación de Boston para mayores de 65 años: aproximación desde la teorı́a de respuesta al ı́tem. Revista de Neurologı́a, 55,
399–407.
Fisher, N. J., Tierney, M. C., Snow, W. G., & Szalai, J. P. (1999). Odd/even short forms of the Boston naming test: Preliminary geriatric norms. The Clinical
Neuropsychologist, 13 (3), 359– 364.
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). “Mini mental state”: A practical method for grading the cognitive state of patients for the clinician. Journal
of Psychiatric Research, 12 (3), 189–198.
Graves, R. E., Bezeau, S. C., Fogarty, J., & Blair, R. (2004). Boston naming test short forms: A comparison of previous forms with new item response theory based
forms. Journal of Clinical and Experimental Neuropsychology, 26 (7), 891– 902.
Hamilton, M. (1960). A rating scale for depression. Journal of Neurology Neurosurgery and Psychiatry, 23, 56–62.
Jefferson, A. L., Wong, S., Gracer, T. S., Ozonoff, A., Green, R. C., & Stern, R. A. (2007). Geriatric performance on an abbreviated version of the Boston naming
test. Applied Neuropsychology, 14 (3), 215 –223.
Kaplan, E. F., Goodglass, H., & Weintraub, S. (1978). The Boston naming test: Experimental edition. Boston: Lippincott Williams and Wilkins.
Kaplan, E. F., Goodglass, H., & Weintraub, S. (1983). The Boston naming test. Philadelphia: Lippincott Williams & Wilkins.
Kaplan, E. F., Goodglass, H., & Weintraub, S. (1986). Test de Vocabulario de Boston. Madrid: Editorial Médica Panamericana.
Kaplan, E. F., Goodglass, H., & Weintraub, S. (2001). The Boston naming test (2nd ed.). Philadelphia: Lippincott Williams and Wilkins.
Kaplan, E. F., Goodglass, H., & Weintraub, S. (2005). Test de Vocabulario de Boston (2nd ed.). Madrid: Editorial Médica Panamericana.
Kent, P. S., & Luszcz, M. A. (2002). A review of the Boston naming test and multiple-occasion normative data for older adults on 15-Item versions. The Clinical
Neuropsychologist, 16 (4), 555– 574.
LaBarge, E., Edwards, D., & Knesevich, J. W. (1986). Performance of normal elderly on the Boston naming test. Brain Language, 27 (2), 380–384.
Lansing, A. E., Ivnik, R. J., Cullum, C. M., & Randolph, C. (1999). An empirically derived short form of the Boston naming test. Archives of Clinical
Neuropsychology, 14 (6), 481–487.
Lezak, M. D., Howieson, D. B., & Loring, D. W. (2004). Neuropsychological assessment (4th ed.). New York: Oxford University Press.
Lucas, J. A., Ivnik, R. J., Smith, G. E., Ferman, T. J., Willis, F. B., Petersen, R. C., et al. (2005). Mayo’s Older African Americans Normative Studies: Norms for
Boston Naming Test, Controlled Oral Word Association, Category Fluency, Animal Naming, Token Test, Wrat-3 Reading, Trail Making Test, Stroop Test, and
Judgment of Line Orientation. The Clinical Neuropsychologist, 19 (2), 243– 269.
Mack, W. J., Freed, D. M., Williams, B. W., & Henderson, V. W. (1992). Boston naming test: Shortened versions for use in Alzheimer’s disease. Journal of
Gerontology, 47 (3), 154– 158.
Martin, A., & Fedio, P. (1983). Word production and comprehension in Alzheimer’s disease. The breakdown of semantic knowledge. Brain and Language, 19 (1),
124–141.
McKhann, G., Drachman, D., Folstein, M., Katzman, R., Price, D., & Stadlen, E. M. (1984). Clinical diagnosis of Alzheimer’s disease: Report of the
NINCDS-ADRDA work group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s disease. Neurology, 34 (7),
939–944.
Mistrushina, M., Boone, K. B., Razani, J., & D’Elia, J. F. (2005). Handbook of normative data for neuropsychological assessment (2nd ed.). New York: Oxford
University Press.
74 M. Casals-Coll et al. / Archives of Clinical Neuropsychology 29 (2014) 60–74

Mungas, D., Marshall, S. C., Weldon, M., Haan, M., & Reed, B. R. (1996). Age and education correction of Mini-Mental State Examination for English and
Spanish-speaking elderly. Neurology, 46, 700– 706.
Nebreda, M. C., Garcı́a-Caballero, A., Asensio, E., Revilla, P., Rodriguez-Girondo, M., & Mateos, R. (2011). A short-form version of the Boston Naming Test for
language screening in dementia in a bilingual rural community in Galicia (Spain). International Psychogeriatrics, 23 (3), 435–441.
Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241– 301.
Pauker, J. (1988). Constructing overlapping cell tables to maximize the clinical usefulness of normative test data: rationale and an example from neuropsychology.
Journal of Clinical Psychology, 44 (6), 930–933.
Pedraza, O., Lucas, J. A., Smith, G. E., Willis, F. B., Graff-Radford, N. R., Ferman, T. J. , et al. (2005). Mayo’s older African American normative studies:
Confirmatory factor analysis of a core battery. Journal of the International Neuropsychological Society, 11 (2), 184 –191.
Peña-Casanova, J. (2005). Programa Integrado de Exploración Neuropsicológica. Test Barcelona-Revisado. Barcelona: Masson.
Peña-Casanova, J., Blesa, R., Aguilar, M., Gramunt-Fombuena, N., Gómez-Ansón, B., Oliva, R., et al. (2009). Spanish Multicenter Normative Studies
(NEURONORMA project): Methods and sample characteristics. Archives of Clinical Neuropsychology, 24 (4), 307– 319.
Peña-Casanova, J., Quiñones-Ubeda, S., Gramunt-Fombuena, N., Aguilar, M., Casas, L., Molinuevo, J. L., et al. (2009). Spanish Multicenter Normative Studies
(NEURONORMA Project): Norms for Boston naming test and token test. Archives of Clinical Neuropsychology, 24 (4), 343–354.
Pontón, M. O., Satz, P., Herrera, L., Ortiz, F., Urrutia, C. P., Young, R., et al. (1996). Normative data stratified by age and education for the Neuropsychological
Screening Battery for Hispanics (NeSBHIS): Initial report. Journal of the International Neuropsychological Society, 2 (2), 96– 104.
Quintana, M., Guàrdia, J., Sánchez-Benavides, G., Aguilar, M., Molinuevo, J. L., Robles, A., et al. (2012). Using artificial neural networks in clinical neuropsych-
ology: High performance in mild cognitive impairment and Alzheimer’s disease. Journal of Clinical and Experimental Neuropsychology, 34 (2), 195– 208.
Quintana, M., Peña-Casanova, J., Sánchez-Benavides, G., Langohr, K., Manero, R. M., Aguilar, M., et al. (2011). Norms for the abbreviated Barcelona test.
Archives of Clinical Neuropsychology, 26, 144– 157.
Quiñones-Ubeda, S., Peña-Casanova, J., Böhm, P., Gramunt-Fombuena, N., & Comas, L. (2004). Estudio normativo piloto de la segunda edición del Boston
Naming Test en una muestra española de adultos jóvenes (20 a 49 años). Neurologı́a, 19 (5), 248 –253.
Rabin, L. A., Barr, W. B., & Burton, L. A. (2005). Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and
APA Division 40 members. Archives of Clinical Neuropsychology, 20 (1), 33–65.
Rami, L., Serradell, M., Bosch, B., Caprile, C., Sekler, A., Villar, A., et al. (2008). Normative data for the Boston naming test and the pyramids and palm trees test in
the elderly Spanish population. Journal of Clinical and Experimental Neuropsychology, 30 (1), 1–6.
Reisberg, B., Ferris, S. H., de Leon, M. J., & Crook, T. (1982). The Global Deterioration Scale for assessment of primary degenerative dementia. American Journal
of Psychiatry, 139 (9), 1136–1139.
Rosen, W. G., Terry, R. D., Fuld, P., Katzman, R., & Peck, A. (1980). Pathological verification of Ischemia Score in differentiation of dementias. Annals of
Neurology, 7 (5), 486–488.
Ross, T. P., Furr, A. E., Carter, S. E., & Weinberg, M. (2006). The psychometric equivalence of two alternate forms of the controlled oral word association test. The
Clinical Neuropsychologist, 20 (3), 414– 431.
Royal Academy of Spanish Language [Real Academia Española (RAE)]. (2001). Diccionario de la Lengua Española (22nd ed.). Madrid: Espasa Calpe.
Saxton, J., Ratcliff, G., Munro, C. A., Coffey, C. E., Becker, J. T., Fried, L., et al. (2000). Normative data on the Boston Naming Test and two equivalent 30-Item
short forms. The Clinical Neuropsychologist, 14 (4), 526 –534.
Serrano, C., Allegri, R. F., Drake, M., Butman, J., Harris, P., Nagle, C., et al. (2001). Version abreviada en español del test de denominación de Boston: su utilidad en
el diagnóstico diferencial de la enfermedad de Alzheimer. Revista de Neurologı́a, 33 (7), 624– 627.
Steinberg, B. A., Bieliauskas, L. A., Smith, G. E., Langellotti, C., & Ivnik, R. J. (2005). Mayo’s Older Americans Normative Studies: Age- and IQ-adjusted norms
for the Boston Naming Test, the MAE Token Test, and the Judgment of Line Orientation Test. The Clinical Neuropsychologist, 19 (3– 4), 280–328.
Strauss, E., Sherman, E., & Spreen, O. (2006). A compendium of neuropsychological tests: administration, norms, and commentary (3rd ed.). New York: Oxford
University Press.
Teng, E. L., Wimer, C., Roberts, E., Damasio, A. R., Eslinger, P. J., Folstein, M. F., et al. (1989). Alzheimer’s dementia: Performance on parallel forms of the
dementia assessment battery. Journal of Clinical and Experimental Neuropsychology, 11 (6), 899– 912.
Teunisse, S., Derix, M., & Crever, H. (1991). Assessing the severity of dementia. Patient and caregiver. Archives of Neurology, 48 (3), 274– 277.
Tombaugh, T. M., & Hubley, A. M. (1997). The 60-item Boston Naming Test: Norms for cognitively intact adults aged 25 to 88 years. Journal of Clinical and
Experimental Neuropsychology, 19 (6), 922–932.
Tryon, W. W. (2001). Evaluating statistical difference, equivalence, and indeterminacy using inferential confidence interval: An integrated alternative method of
conducting null hypothesis statistical tests. Psychological Methods, 6, 371– 386.
Villarejo, A., & Puertas-Martı́n, V. (2011). Utilidad de los test breves en el cribado de demencia. Neurologı́a, 26 (7), 425– 433.
Welch, L. W., Doineau, D., Johnson, S., & King, D. (1996). Educational and gender normative data for the Boston Naming Test in a group of older adults. Brain
Language, 53 (2), 260–266.
Williams, B. W., Mack, W., & Henderson, V. W. (1989). Boston Naming Test in Alzheimer’s disease. Neuropsychologia, 27 (8), 1073–1079.
World Medical Association. (1997). Declaration of Helsinki. Recommendations guiding physicians in biomedical research involving human subjects. Journal of
the American Medical Association, 277 (11), 925–926.
Zec, R. F., Burkett, N. R., Markwell, S. J., & Larsen, D. L. (2007). Normative data stratified for age education and gender on the Boston naming test. The Clinical
Neuropsychologist, 21 (4), 617– 637.