The Hopkins Verbal Learning Test An in Depth Analysis of Recall Patterns

Memory
ISSN: 0965-8211 (Print) 1464-0686 (Online) Journal homepage: https://www.tandfonline.com/loi/pmem20
The Hopkins Verbal Learning Test: an in-depth

analysis of recall patterns
Rachel Grenfell-Essam, Eef Hogervorst & Tri Budi W. Rahardjo
To cite this article: Rachel Grenfell-Essam, Eef Hogervorst & Tri Budi W. Rahardjo (2018) The
Hopkins Verbal Learning Test: an in-depth analysis of recall patterns, Memory, 26:4, 385-405, DOI:
10.1080/09658211.2017.1349804
To link to this article: https://doi.org/10.1080/09658211.2017.1349804
Published online: 11 Jul 2017.
Submit your article to this journal
Article views: 310
View related articles
View Crossmark data
Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=pmem20
MEMORY, 2018
VOL. 26, NO. 4, 385–405
https://doi.org/10.1080/09658211.2017.1349804
The Hopkins Verbal Learning Test: an in-depth analysis of recall patterns

Rachel Grenfell-Essama, Eef Hogervorsta,b,c and Tri Budi W. Rahardjoa,b,c
a
School of Sport, Exercise and Health Sciences, Loughborough University, Loughborough, UK; bCentre for Ageing Studies, Universitas
Indonesia, Depok, Indonesia; cFaculty of Health Sciences, Universitas Respati Indonesia, Jakarta Timur, Indonesia
ABSTRACT ARTICLE HISTORY

One of the earliest signs of dementia is memory issues and verbal word lists, such as the Hopkins Received 13 March 2017
Verbal Learning Test (HVLT), are successfully used for screening. To gain insight in how memory Accepted 27 June 2017
is affected in dementia, and to further improve the efficacy of the HVLT, in-depth analysis of the
KEYWORDS
recall patterns of dementia cases and controls was conducted. Dementia cases and controls Dementia; Hopkins Verbal
were matched for factors that can affect performance, such as age, gender and education Learning Test; HVLT;
level. Word frequency, syllable length, and orthographic neighbourhood size did not differ in memory; recall patterns
the Indonesian version of the HVLT, nor did these characteristics affect recall. However
dementia cases showed consistent and poor recall across the three trials; with the worst
recall for the “human shelter” category and best recall for the “animals” category. Dementia
cases also showed impaired accessibility of all categories with reduced subsequent recall
from accessed categories and reduced primacy and recency levels. Finally, dementia cases
exhibited lower levels of re-remembering and recalling new words, and higher levels of
immediate forgetting and never recalling words. It was concluded that utilising the extra
information provided by the in-depth analyses of the recall patterns could be beneficial to
improve dementia screening.
The majority of dementia cases are expected to reside in for dementia (see Benedict, Schretlen, Groninger, Dobraski,
developing countries (Hogervorst et al., 2011; Prince & Shpritz, 1996; Shapiro et al., 1999).
et al., 2015). Therefore, it is crucial that effective but The HVLT has been adapted and/or translated for use in
easily administered, low technology screening tests for multiple countries (e.g., French: Rieu, Bachoud-Lévi,
dementia are available which ideally could be adminis- Laurent, Jurion, & Dallabarba, 2006; Spanish: Cherner
tered without the need for specialist training or equipment. et al., 2007; Chinese: Shi, Tian, Wei, Miao, & Wang, 2012;
The Hopkins Verbal Learning Test (HVLT; Brandt, 1991) is Indonesian: Hogervorst et al., 2011). It is well tolerated by
one such screening test that we believe holds promise participants and is suitable for repeated testing due to its
for use in developing countries (Hogervorst et al., 2011; six different forms with minimal learning effects and no
Xu, Rahardjo, Xiao, & Hogervorst, 2014). ceiling effects (Benedict et al., 1998; Krebs, 1994; Rasmus-
The HVLT is a short test of verbal memory taking around son, Bylsma, & Brandt, 1995; Woods et al., 2005). An
10 minutes to administer (Brandt, 1991). The test was optimal cut-off score for the HVLT for dementia cases is
revised to include a delayed recall trial, which is copy- 16 to 17 words (total of 3 trials) immediately recalled, but
righted as HVLT-R (Benedict, Schretlen, Groninger, & this may vary depending on the country (and translation
Brandt, 1998). The HVLT has been favourably compared use) where it is used and/or the age of participants (see
to other dementia tests, such as: the Mini-Mental Status Xu et al., 2014 for a review). The HVLT displays good sensi-
Examination (MMSE) (e.g., de Jager, Schrijnemaekers, tivity and specificity for dementia in both Western (Fried-
Honey, & Budge, 2009; Frank & Byrne, 2000; Kuslansky man, Schinka, Mortimer, & Graves, 2002; Hester, Kinsella,
et al., 2004), the California Verbal Learning Test (CVLT) Ong, & Turner, 2004; Hogervorst et al., 2002) and non-
(Lacritz & Cullum, 1998; Lacritz, Cullum, Weiner, & Rosen- Western countries (Hogervorst et al., 2011; Shi et al.,
berg, 2001), CogState (de Jager et al., 2009), subtests of 2012; Xu et al., 2014).
the Wechsler Memory Scale-Revised (Shapiro, Benedict, Given the extent of previous research studying the HVLT
Schretlen, & Brandt, 1999), and the Brief Visuospatial it is perhaps surprising that research exploring in-depth
Memory Test – Revised (Shapiro et al., 1999). The HVLT performance of dementia patients is very limited. A
has also been compared to less commonly used tests highly similar test, the CVLT (Delis, Kramer, Kaplan, &
and has been shown to be more effective in screening Thompkins, 1987), had explored performance over a wide
CONTACT Rachel Grenfell-Essam [email protected] School of Sport, Exercise and Health Sciences, Loughborough University, Lough-
borough, Leicestershire, LE11 3TU, UK
© 2017 Informa UK Limited, trading as Taylor & Francis Group
386 R. GRENFELL-ESSAM ET AL.
range of in-depth analyses, such as semantic clustering clustering is well-known within dementia patients (e.g.,
(the degree to which items from the same category are Carlesimo et al., 1998; Delis et al., 1991; Glosser, Gallo,
recalled together), primacy and recency levels, learning Clark, & Grossman, 2002; Hodges et al., 1999; Perri, Carle-
rates, intrusion errors, etc. However, within the HVLT, the simo, Serra, & Caltagirone, 2005; Weingartner et al., 1981).
total recall score is the main measure used for assessment. As well as the aspects discussed above, there are several
It is useful for quick identification of probable dementia general characteristics that can affect the ability to remem-
patients (those who fall below the cut-off score), but the ber a word, such as word frequency, word length (number
total recall score fails to investigate whether there are of syllables), age-of-acquisition, and orthographic neigh-
any subtler differences within the recall of probable bourhood size. Surprisingly, although the HVLT allows for
dementia patients and controls. analysis of each word, no previous papers have investi-
Only a few papers have explored HVLT recall in more gated the effect of these characteristics on recall of the
depth. The most relevant of these is Schrijnemaekers, de individual words. Therefore it is currently unknown how
Jager, Hogervorst, and Budge (2006) who provided a these characteristics vary within the HVLT and what
brief examination of category recall and serial position effect dementia might have on recall. There is also a lack
effects in an Oxfordshire-based cohort of cases and con- of research investigating these characteristics within
trols. Serial position effects refer to the finding that the dementia patients’ performance in free recall tasks.
location of an item within the list affects its likelihood Word frequency is perhaps the most important charac-
of being remembered. Primacy effects refer to the teristic. However, most research conducted on dementia
recall advantage for words at the start of the list, and patients has focused on recognition tasks where the low
recency effects refers to the recall advantage for words frequency advantage observed in recall tasks dramatically
at the end of the list (e.g., Jahnke, 1965; Murdock, decreases or disappears (Balota, Burgess, Cortese, &
1962; Roberts, 1972). Schrijnemaekers et al. (2006) Adams, 2002; Wilson, Bacon, Fox, Kramer, & Kaszniak,
explored whether there were differences in the recall of 1983). Preserved word (syllable) length effects in memory
the three HVLT categories for trials 1 and 3 only. They span (serial recall) have been shown within dementia
did not find a significant difference for dementia cases, patients (Belleville, Peretz, & Malenfant, 1996; Morris, 1984).
however, the trend in both trials 1 and 3 was for The expected effect of age-of-acquisition is more
better recall of the category “animals” compared to the complex. Within healthy individuals some papers have
categories “semi-precious stones” and “human shelter”. found better recall for late acquired words (Dewhurst,
Dementia cases recall of all three categories was signifi- Hitch, & Barry, 1998; Morris, 1981) whilst others have
cantly poorer than controls. They also assessed primacy found no effect (Christian, Bickley, Tarka, & Clayton, 1978;
and recency effects via a grouped serial position analysis Coltheart & Winograd, 1986; Gilhooly & Gilhooly, 1979;
for trial 3 only. Recall was significantly lower for dementia Rubin, 1980). Studies investigating age-of-acquisition in
cases. Controls showed no primacy and recency effects, dementia patients have focused on word naming, picture
but dementia cases showed both. This paper provides naming and lexical decision tasks. They consistently find
an interesting, albeit limited, assessment of recall pat- an advantage for early acquired words on these tasks
terns within the HVLT. (e.g., Cuetos, Herrera, & Ellis, 2010; Forbes-McKay, Ellis,
Primacy and recency effects on other word lists have Shanks, & Venneri, 2005; Kremin, Hamerel, Dordain, De
been investigated within dementia research. Typically Wilde, & Perrier, 2000; Ralph, Graham, Ellis, & Hodges,
dementia cases show an impaired primacy effect in word 1998; Silveri, Cappa, Mariotti, & Puopolo, 2002) which is
list recall, but a relatively preserved recency effect and so in line with healthy individuals (e.g., Barry, Morrison, &
this may be diagnostically useful (e.g., Bayley et al., 2000; Ellis, 1997; Carroll & White, 1973; Gerhand & Barry, 1998;
Carlesimo, Fadda, Sabbadini, & Caltagirone, 1996; Foldi, Morrison & Ellis, 1995).
Brickman, Schaefer, & Knutelska, 2003; Gainotti & Marra, Dementia patients have shown improved performance
1994; Howieson et al., 2011; Massman, Delis, & Butters, for words with larger numbers of orthographic neighbours
1993; Tierney et al., 1994). within a recognition task using Spanish speakers (Dunabei-
Other papers have focused on semantic clustering tia, Marín, & Carreiras, 2009). This pattern has been mir-
within the HVLT. This can be assessed if the order in rored within the normal population in serial recall tasks
which the words are recalled is recorded (output order). (memory span and immediate serial recall) (Allen &
Gaines, Shapiro, Alt, and Benedict (2006) compared seman- Hulme, 2006; Glanc & Greene, 2012; Jalbert, Neath, & Sur-
tic clustering indexes in healthy U.S.-based adults, patients prenant, 2011; Jalbert, Neath, Bireta, & Surprenant, 2011;
with probable dementia of Alzheimer’s type and patients Roodenrys, Hulme, Lethbridge, Hinton, & Nimmo, 2002).
with vascular dementia. Both types of dementia patients In the current paper we sought to investigate whether
showed lower levels of semantic clustering than controls. these four characteristics varied at the category and indi-
Lower levels of semantic clustering within the HVLT have vidual word level in Form 1 of the HVLT in both English
also been found with amnestic mild cognitive impairment and Indonesian languages.1 We are the first to explore
(aMCI) patients compared to controls (Malek-Ahmadi, Raj, the effects of word frequency, word length, age-of-acqui-
& Small, 2011). This tendency for impaired semantic sition and orthographic neighbourhood size on
MEMORY 387
performance of dementia cases within the HVLT in either specifically which words are being learnt or forgotten
English or Indonesian and the first to compare the two from trial to trial within immediate recall. The words
languages.2 Whilst finalising the current paper, a highly rel- recalled in the final immediate recall trial may be com-
evant paper was published that looked at effects of word pletely different words to the ones recalled in the first
frequency, word length, and orthographic neighbourhood immediate recall trial, but if the number of words
size within controls and dementia cases utilising the is the same, then no learning will be the reported
German version of the CVLT (Hessler, Fischer, & Jahn, outcome.
2016). They found that word length had no effect on A few studies, however, have looked at intertrial per-
recall for either controls or dementia cases, that word fre- formance to assess learning and forgetting with dementia
quency showed an overall advantage for high frequency cases. Woodard, Dunlosky, and Salthouse (1999) examined
words for both controls and dementia cases, and that con- intertrial acquisition and consolidation levels within six
trols displayed better recall with high orthographic neigh- dementia cases. Utilising the Rey Auditory Verbal Learning
bourhood size, whereas dementia cases showed better test (Rey, 1964) they determined the levels of gained and
recall with low orthographic neighbourhood size. lost items across adjacent trials. Gained items reflect acqui-
In-depth analysis of the HVLT may allow for greater sition (learning) and lost items reflect impaired consolida-
understanding of the nature of the memory impairment tion (forgetting). Dementia cases had lower levels of
that causes reduced recall within dementia cases. As dis- learning and higher levels of forgetting suggesting
cussed earlier, dementia cases consistently show a impaired encoding and storage. These measures were
reduced ability to utilise semantic information indicating also not correlated with each other, indicating that these
an encoding issue. Furthermore, dementia cases show a assess independent processes.
large range of issues: impaired performance in both recol- Moulin, James, Freeman, and Jones (2004) sought to
lection (e.g., free recall) and familiarity (e.g., old/new and replicate and extend Woodard et al.’s findings by utilising
forced-choice recognition) tasks, a lack of learning over a larger sample, comparing controls with a MCI group
repeated exposure, impaired recall consistency, reduced and using a different test (CERAD word learning list;
subjective organisation, increased sensitivity to proactive Welsh, Butters, Hughes, Mohs, & Heyman, 1991). Like
interference, higher rates of forgetting, greater suscepti- Woodard et al. they found significantly lower levels of
bility to intrusions, higher rates of false alarms, greater gained items and higher levels of lost (forgotten) items
number of perseverative errors, and impaired retention for dementia cases, and no correlation between the two
over time (e.g., Algarabel et al., 2009, 2012; Ally, Gold, & measures. In addition they also found that dementia
Budson, 2009; Butters et al., 1983; Butters, Granholm, cases were more impaired on both acquisition and conso-
Salmon, Grant, & Wolfe, 1987; Dallabarba & Wong, 1995; lidation measures compared to MCI cases, indicating that
Dannenbaum, Parkinson, & Inman, 1988; Delis et al., these both decline as dementia severity progresses.
1991; Granholm & Butters, 1988; Grober & Buschke, 1987; Additionally, Genon et al. (2013) utilised this gained and
Koen & Yonelinas, 2014; Kopelman, 1991; Martin, Brouwers, lost access criteria within another test (CVLT), and also con-
Cox, & Fedio, 1985; Ober, Koss, Friedland, & Delis, 1985; firmed deficient acquisition and consolidation levels with
Perri et al., 2005; Weingartner et al., 1981; Weingartner, dementia cases whilst also providing the neural correlates.
Grafman, Boutelle, Kaye, & Martin, 1983; Wolk, Dunfee, We are the first to examine the levels of re-remembered
Dickerson, Aizenstein, & DeKosky, 2011). These findings (recalled in this trial and previously), new (recalled in this
indicate widespread issues with encoding, storage, and trial but not previously, akin to gained access), forgotten
retrieval. (not recalled in this trial but recalled previously, akin to
However, in most cases the issues researched above lost access) and never recalled (not recalled in this trial or
do not examine performance from trial to trial (except previously) words within the HVLT. We also operationalise
for recall consistency and subjective organisation). Rep- these calculations in a different way. Both Woodard et al.
etition of the same words across three separate trials (1999) and Moulin et al. (2004) based their measures on
within the HVLT allows examination of retrieval dynamics consecutive trials. This means that a word recalled in trial
is possible via the tracking of each individual word from 1 and 3 but not 2 would be seen as a gained item (new)
trial to trial. One can investigate which words are learnt, in trial 3 rather than a re-remembered item. Therefore we
retained, and forgotten from trial to trial. Typically the sought to eliminate this by amending our calculation to
learning research above uses a calculation of the differ- take into account recall of specific words from all previous
ence between the number of words recalled in the last trials.
immediate trial compared to the first immediate trial The first author’s previous work, looking at the recall
and the forgetting rates are calculated as the difference patterns in two other immediate memory tasks (immediate
between the number of words recalled in a delayed free recall and immediate serial recall), has shown how
recall trial compared to the final immediate recall trial. important in-depth analysis can be to advance a field
Immediate forgetting from trial to trial is synonymous (Grenfell-Essam & Ward, 2012, 2015; Grenfell-Essam,
with failure to benefit from repeated exposure. Ward, & Tan, 2013, 2017; Ward, Tan, & Grenfell-Essam,
However, these calculations are not sensitive as to 2010). We hope that a deeper exploration of the recall
patterns of controls and dementia cases within the HVLT hereafter as stones), and human dwellings (referred here-
may help improve its diagnostic efficacy. We are the first after as shelter). The words are read out loud by the exper-
to investigate whether there are differences between imenter at a rate of around one word every two seconds. At
word frequency, word length, age-of-acquisition and the end of each trial participants’ immediate free recall of
orthographic neighbourhood size within the 12 words of the words is recorded. A total immediate recall measure
the HVLT for both English and Indonesian languages3 is obtained by summing the total number of words cor-
and the first to compare the two languages. We also inves- rectly recalled from each trial. The maximum total immedi-
tigated whether there were differences between controls ate recall score possible is 36. An Indonesian version of the
and dementia cases on the total HVLT score, the overall HVLT was used (Hogervorst et al., 2011) and as such all ana-
proportion of words recalled (allowing assessment of learn- lyses unless otherwise stated are based on this Indonesian
ing), category recall and accessibility, the effects of word version. Back-translation and focus groups were used to
frequency, word length, and orthographic neighbourhood further validate the test items for local use. Note that
size on recall,4 the levels of primacy and recency (via serial Hogervorst et al. (2011) modified some of the words
position curves), and the retrieval dynamics across trials from the “precious stones” category to suit local knowl-
(via the levels of re-remembered, new, forgotten and edge following a pilot study.5
never recalled words). The majority of these recall patterns MMSE: This test consists of 11 questions that can be
have not been previously investigated within the HVLT. We used to assess mental status across a range of five
build upon a small number of studies that have investi- aspects of cognitive ability (orientation, registration, atten-
gated these areas by providing a more systematic exploration and calculation, recall, and language). Note that
tion of recall within all trials and by utilising a larger sample. Hogervorst et al. (2011) adapted the questions for local cir-
The main aim of this paper is to improve the efficacy of cumstances similar to Ganguli et al.’s (1995) Hindi version.
the HVLT by discovering whether, over a range of perform- The maximum score possible is 30 and a cut-off score of 24
ance measures, there are differences between control and or less was adopted for dementia cases.
dementia cases. These more subtle differences between IADL: This test consists of eight activities, such as ability
control and dementia cases, over and above the current to use a telephone, housekeeping and responsibility for
overall recall measure currently used, may help screen for own medications, and scores their functional ability level
earlier signs of dementia. for each activity. Note that Hogervorst et al. (2011) used
a modified version that can be found within their article
where the ability to perform each activity was scored
Method from 0 (unable to do) to 2 (independent ability). The
Sample maximum score possible for this modified version is 16
and a cut-off score of 9 or less was adopted for dementia
Data were taken from Hogervorst et al. (2011), which was cases.
interested in investigating the validation of the HVLT and
MMSE dementia screening tests in Indonesia. The data
was from a cross-sectional study involving 719 elderly Statistical analyses
Indonesian participants from West Java, Central Java and We classified participants in the data set into control or
Jakarta (see Hogervorst et al., 2011; Yesufu, Bandelow, dementia cases based on their scores on the MMSE and
Rahardjo, & Hogervorst, 2009 for specific details of the IADL. From the data set of 719 participants, dementia
study design). Informed consent had been obtained cases were selected if they met all of the following three
before study onset and ethical approval had been obtained criteria: they had an MMSE score of 24 or less, an IADL
from a local Indonesian ethical committee, as well as from score of 9 or less and had a recorded value for the Indo-
Loughborough University in the United Kingdom. nesian version of the HVLT (i.e., no missing data).
Control cases were selected if they met all the following
three criteria: they had an MMSE score of 25 or more, an
Cognitive measures
IADL score of 10 or more and had a recorded value for
Three cognitive measures taken from Hogervorst et al. the Indonesian version of the HVLT. Using these criteria
(2011) were used in this study: HVLT (Brandt, 1991), 216 participants were excluded because they did not
MMSE (Folstein, Folstein, & McHugh, 1975), and Instrumen- match the cut-off criteria for either dementia or control
tal Activities of Daily Living (IADL, Lawton & Brody, 1969). based on their MMSE and/or IADL values and a further
The latter two measures were used to classify control and 31 were excluded because they had missing values for
dementia cases. either HVLT, MMSE or IADL. Differences in demographic
HVLT: This test consists of three trials; each trial contains characteristics (gender, age and education level) of the
the same 12 words taken from three semantic categories control and dementia cases were assessed via Chi-
(four words from each category). There are six forms to square and Mann–Whitney U tests. Due to significant
the test; form 1 was used containing four-legged animals differences between age and education level in the two
(referred hereafter as animals), precious stones (referred groups a matched samples approach was adopted.
MEMORY 389
Participants were matched by their gender, education significant difference for gender between the dementia
level (exact match) and age (within 5 years). A Mann– group (Males = 32, Females = 44) and control group
Whitney U test confirmed non-significant differences (Males = 140, Females = 256), χ 2 (1) = 1.26, p = 0.263.
between the two groups in age. However, Mann–Whitney U tests indicated that the
A set of analyses were performed to examine the word dementia group (mean = 76.5, SD = 8.11) was significantly
frequency, word length, and orthographic neighbourhood older than the control group (mean = 67.2, SD = 6.34),
size characteristics mentioned in the introduction and Z = −8.56, p < 0.001, and that the dementia group had a
methods section for both English and Indonesian significantly lower education level than the control
languages (see Table 1).6 This included a series of one- group, Z = −7.82, p < 0.001.
way analysis of variances (ANOVAs) with category (3 Due to the significant differences in age and education
levels: animals, stones and shelter) as the independent vari- level between the control and dementia cases, a matched
able and also a series of correlations to see whether any of samples approach was adopted. Control and dementia par-
the word characteristics were associated with each other. ticipants were matched on gender (exact matches only),
An independent sample t-test examined whether the education level (exact matches only) and age (matches
HVLT total recall score differed between dementia cases exact where possible but no further than 5 years apart).
and controls. The overall proportion of words recalled This resulted in 62 matched participants in each group
across the three trials of the HVLT was examined via a (Males = 23, Females = 39). A Mann–Whitney U test indi-
two-way mixed ANOVA. A set of analyses were performed cated that there was no significant difference in age
to examine category recall and accessibility; this included a between the dementia group (mean = 74.6, SD = 7.38)
series of two-way mixed ANOVAs and also a series of chi- and the control group (mean = 73.5, SD = 6.47), Z = −.702,
squares and independent samples t-tests. The relationship p = 0.483. Full details of the demographic information of
of word characteristics and recall was then assessed via a the participants can be found in Appendix 1.
series of correlations. Serial positions curves, that allow
assessment of primacy and recency within trials, were
Word characteristics
examined via a series of two-way mixed ANOVAs. Finally,
the levels of learning and forgetting were investigated This analysis explored whether the characteristics dis-
via a series of two-way mixed ANOVAs. cussed in the introduction (word frequency, word length,
For all two-way and three-way ANOVA analyses one of age-of-acquisition, and orthographic neighbourhood size)
the independent variables was always group (dementia vary within Form 1 of the HVLT in both English and Indone-
or control) and the results are reported within tables for sian. Table 1 shows the word frequency, word length
clarity. Mauchly’s Test of Sphericity was checked for all (number of syllables), age-of-acquisition,7 and ortho-
ANOVA analyses containing within-subject independent graphic neighbourhood size (based on Coltheart’s N;
variables with more than two levels. Where Sphericity Coltheart, Davelaar, Jonasson, & Besner, 1977) values for
was violated the Greenhouse–Geisser correction was the words and categories in Form 1. For the English
applied. Where significant main effects for within-subject version: word frequency values represent number of occur-
variables occurred these were followed up via Bonferroni rences per million words taken from Brysbaert and New
corrected pairwise comparisons. Where significant two- (2009) which are based on the SUBTL database; age-of-
way interactions occurred these were followed up via acquisition values were taken from Kuperman, Stadtha-
simple effects using a Bonferroni correction. An alpha gen-Gonzalez, and Brysbaert (2012); and orthographic
value of 0.05 was used for all statistical analyses unless neighbourhood size were based on Coltheart’s N values
otherwise stated and SPSS 23.0 was used. from the CLEARPOND database (Marian, Bartolotti,
Chabal, & Shook, 2012). For the Indonesian version: word
frequency values were taken from Hermit Dave (2016)
Results
and are based on the OpenSubtitles2016 data for Indone-
The data was considered in eight different analyses: partici- sian subtitles.8 Orthographic neighbourhood size values
pant demographics, word characteristics, HVLT total recall were calculated by the first author by systematically substi-
score, overall proportion of words recalled, category tuting each letter individually with all possible letters of the
recall and accessibility, relationship of word characteristics alphabet to make all possible combinations of ortho-
and recall, serial position curves, and retrieval dynamics graphic neighbours. These words were then verified as
across trials. real Indonesian words via at least three separate sources
(a translation tool, an Indonesian dictionary and an Indone-
sian thesaurus).
Participant demographics
Table 1 shows that the individual words display vari-
The first analysis investigated the demographic character- ation within the characteristics for both the English and
istics of the dementia and control groups. The criteria out- Indonesian languages. In particular the word frequency
lined in the method resulted in 76 dementia cases and 396 values show a large range in both languages. These
control cases. A Chi-square indicated that there was a non- values cannot be directly numerically compared due to
differences in measurement but they appear to follow
Table 1. Characteristics of Form 1 of the Hopkins Verbal Learning Test: word frequency, word length (number of syllables), age-of-acquisition, and orthographic neighbourhood size for both English and Indonesian languages.
neighbourhood size
Orthographic
similar trends. This was confirmed by a significant corre-
3.08 (2.98)
3.20 (4.30)
1.40 (1.74)
2.80 (1.50)
lation, r (12) = 0.941, p < 0.001. Word length shows much
1
3
4
5
4
1
4
0
0
0
11
4
lower variability, but more similarity across the two
languages (English 1–3 syllables; Indonesian 2–4 syllables)
with only 1 word in both languages having the highest
number of syllables. However, there was a non-significant
Word length (syllables)
correlation between the two languages for word length, r

(12) < 0.001, p = 1.000. Age-of-acquisition values were
2.33 (0.62)
2.25 (0.84)
2.75 (0.83)
2.00 (0.00)
only available for English but these displayed a wide
Indonesian version
2
2
2
2
2
2
2
3
3
4
2
2
range of 5.34 years. Orthographic neighbourhood size
shows a wider range for English (English 0–24 neighbour-
ing words; Indonesian 0–11 neighbouring words) but
they appear to follow similar trends as confirmed by a sig-
nificant correlation, r (12) = 0.796, p = 0.002.
1010.25 (1168.21)
1622.75 (1169.02)
1325.50 (1172.92)
Word frequency
82.50 (128.70)
To investigate whether each characteristic varied at the
category level, a series of one-way ANOVAs were con-
923
21
3584
485
1
3160
1513
3
583
305
1401
144
ducted on each characteristic with category (3 levels:

animals, stones and shelter) as the independent variable.
For the English language these ANOVAs revealed non-sig-
nificant differences between: frequency of occurrences per
kecubunga
million, F (2, 9) = 1.25, MSE = 1119.0, p = 0.331; word length,

Words
harimau
mutiara
gubuk
F (2, 9) = 1.93, MSE = 0.389, p = 0.201; and orthographic

tenda
singa
hotel
intan
kuda
akika
sapi
gua
neighbourhood size values, F (2, 9) = 3.16, MSE = 49.4, p =

0.091. However, there was a significant difference
between the age-of-acquisition, F (2, 9) = 14.6, MSE = 1.18,
neighbourhood size
p = 0.002. Bonferroni corrected pairwise comparisons

Orthographic
7.75 (7.94)
9.75 (8.44)
0.75 (0.83)
12.75 (6.26)
showed that animals were acquired significantly earlier

4
0
8
15
0
2
17
2
3
1
24
17
than both stones (p = 0.001) and shelter (p = 0.038), but

that stones and shelter were not different (p = 0.147).
Additionally only two of the characteristics significantly
correlated with each other: word length and orthographic
neighbourhood size, r (12) = −0.642, p = 0.024. All other
Age of acquisition
correlations were not significant; word frequency and

6.30 (1.94)
4.13 (0.19)
8.26 (1.21)
6.51 (1.07)
(years)
word length, r (12) = −0.123, p = 0.703, word frequency

4.42
8.26
4.15
5.16
9.22
6.05
6.74
9.28
4.00
6.28
3.94
8.10
and age-of-acquisition, r (12) = −0.441, p = 0.151, word fre-

quency and orthographic neighbourhood size, r (12) =
English version
−0.020, p = 0.950, word length and age-of-acquisition, r

Note: aDenotes non-direct translations: akik = agate, kecubung = amethyst.
(12) = 0.484, p = 0.111, and age-of-acquisition and ortho-

Word length
1.50 (0.65)
1.25 (0.43)
2.00 (0.71)
1.25 (0.43)
graphic neighbourhood size, r (12) = −0.335, p = 0.287.

(syllables)
For the Indonesian language these ANOVAs exploring

1
3
1
1
2
2
1
2
2
1
1
1
category differences also revealed non-significant differ-

ences between: word frequency, F (2, 9) = 2.18, MSE =
1226176.1, p = 0.169; word length, F (2, 9) = 1.50, MSE =
Word frequency
26.72 (32.75)
38.07 (31.86)
36.98 (38.28)
(per million)
0.389, p = 0.274; and orthographic neighbourhood size

5.12 (6.12)
values, F (2, 9) = 0.525, MSE = 10.64, p = 0.609. Additionally

15.35
2.57
92.88
17.49
1.20
103.22
13.98
1.02
18.53
15.67
25.51
13.22
none of the characteristics significantly correlated with

each other: word frequency and word length, r (12) =
−0.325, p = 0.302, word frequency and orthographic neigh-
bourhood size, r (12) = 0.133, p = 0.680, and word length and
sapphire
emerald
Animals
Words
Overall
Shelter
Stones
orthographic neighbourhood size, r (12) = −0.552, p = 0.063.

horse
hotel
pearl
tiger
cave
opal
tent
cow
lion
hut
HVLT total recall score

Average (SD)
The HVLT is usually scored using a total recall score. This is

the sum of all words correctly recalled, including any re-
10
11
12
remembered words, across the three immediate trials.

1
2
3
4
5
6
7
8
9
MEMORY 391
Figure 1. Mean proportion of words recalled overall for trials 1–3 in the HVLT for control and dementia cases.
The total recall score was 14.9 words (SD = 5.65) for the recall performance. There was a significant main effect of
control cases and 6.6 words (SD = 5.39) for the dementia group; controls had significantly higher recall than demen-
cases. This difference was statistically significant, t (122) = tia cases. There was a significant main effect of trial; pair-
8.36, p < 0.001, with a large effect size (Cohen’s d = 1.5). wise comparisons using a Bonferroni correction indicated
that recall significantly increased between each trial (trial
1 < trial 2 < trial 3). Finally, there was a significant inter-
Overall proportion of words recalled action. Simple effects using a Bonferroni correction (p =
Figure 1 shows the overall proportion of words recalled for 0.00556) comparing control and dementia cases within
the dementia and control groups for the three immediate each trial were all significantly different (all p’s < 0.001).
trials of the HVLT. It is evident that recall for all trials is sub- Looking within the control cases only across trials, all com-
stantially higher for the control group compared to demen- parisons were significantly different (all p’s < 0.001).
tia cases. There is a greater increase in recall occurring However, looking within the dementia cases only across
across trials for controls (14%) but a smaller increase in trials, there were no significant comparisons (all p’s >
recall for dementia cases (3%). 0.100). Therefore the interaction was driven by the signifi-
Table 2 summarises a 2 (group: dementia or control) × 3 cantly increasing performance of controls across trials 1–3,
(trial: 1–3) mixed ANOVA performed to examine overall compared to the consistent and poor performance of
dementia cases across trials 1–3.
Table 2. Summary of the ANOVA analyses conducted on the overall
proportion of words recalled and the proportion of words recalled by
category for trial 1, trial 2, trial 3 and overall for trials 1–3.
Category recall and accessibility
df MSE F p
Overall proportion of words recalled a Figure 2 shows the proportion of words recalled for each
Group 1, 122 10.2 69.9 <.001 category for the dementia and control groups for each of
Trial 1.785, 217.7 1.46 25.8 <.001 the three trials (Panel A – trial 1, Panel B – trial 2, and
Group × trial 1.785, 217.7 1.46 11.2 <.001
Proportion of words recalled by category for trial 1 a Panel C – trial 3). Panel D shows the overall proportion of
Group 1, 122 0.073 37.5 <.001 words recalled from each category at any point during
Category 1.904, 232.3 0.046 49.7 <.001 the three trials. In all four panels it is evident that recall
Group × category 1.904, 232.3 0.046 6.01 0.003
Proportion of words recalled by category for trial 2 for all categories is higher for the control group compared
Group 1, 122 0.082 61.0 <.001 to dementia cases. Both control and dementia cases
Category 2, 244 0.044 31.0 <.001 recalled the animal category the best, and the shelter cat-
Group × category 2, 244 0.044 1.43 0.243
Proportion of words recalled by category for trial 3 a egory the worst.
Group 1, 122 0.110 69.3 <.001 Table 2 summarises a series of four 2 (group: dementia
Category 1.897, 231.4 0.044 17.9 <.001 or control) × 3 (category: animals, stones, and shelter)
Group × category 1.897, 231.4 0.044 0.593 0.554
Proportion of words recalled by category for trials 1–3 mixed ANOVAs performed to examine category recall per-
Group 1, 122 0.154 66.0 <.001 formance in each of the four panels of Figure 2. For all
Category 2, 244 0.049 43.1 <.001 ANOVAs the main effect of group was significant; controls
Group × category 2, 244 0.049 0.807 0.447
had significantly higher recall of the categories than
Note: aIndicates the initial assumption checks indicated that Mauchly’s Test
of Sphericity was violated therefore degrees of freedom were corrected dementia cases. For all ANOVAs the main effect of category
using the Greenhouse–Geisser correction. was significant. Bonferroni corrected pairwise comparisons
Figure 2. Mean proportion of words recalled within each category of the HVLT for control and dementia cases for trial 1 (A), trial 2 (B), and trial 3 (C). Panel D
shows the overall likelihood of recalling the words within the category across the three trials.
indicated that for trial 2, trial 3 and overall trials 1–3 ana- The following analysis looks at category accessibility
lyses all categories were recalled significantly differently and subsequent number of words recalled to determine
(all p’s < 0.001; shelter < stones < animals). However, for whether the overall category recall findings reflect differ-
trial 1 animals were recalled significantly better than ent accessibility abilities for control and dementia cases.
stones and shelter (both p’s < 0.001), but stones and Figure 3 shows the percentage of participants who
shelter were not different (p = 0.100). The interaction was accessed each category within the three trials giving a
not significant for trial 2, trial 3 and overall trials 1–3 ana- measure of category accessibility (Panel A = trial 1, Panel
lyses. However, the interaction was significant for trial B = trial 2, and Panel C = trial 3) and displays the average
1. Simple effects using a Bonferroni correction (p = number of words recalled given that the category was
0.00556) comparing control and dementia cases within accessed (Panel D = trial 1, Panel E = trial 2, and Panel F =
each category were all significantly different (all p’s < trial 3). It appears that for all categories across all trials con-
0.004). Looking across categories within the control cases trols have a greater ability to access each category com-
only, animals were recalled significantly better than both pared to controls. However, when looking at the
stones and shelter (both p’s < 0.001) but there was a non- subsequent number of words recalled there is a more con-
significant difference between stones and shelter (p = sistent approximate 0.5 word advantage for control over
0.036). Looking across categories within the dementia dementia cases. It also appears that in trial 1 there is a
cases only, animals were recalled significantly better than smaller difference between control and dementia cases
both stones and shelter (both p’s < 0.002) but there was a in both the stones and shelter categories.
non-significant difference between stones and shelter (p Table 3 summarises a series of nine Chi-square analyses
= 0.397). Therefore the interaction for trial 1 was driven performed to examine category accessibility for control
by the difference in recall of stones compared to shelter and dementia cases for each category within each trial
between control and dementia cases – recall of these (Figure 3, Panels A–C). All Chi-squares were significant
two categories was deemed non-significant for controls indicating that for all categories across all trials the
due to the Bonferroni correction but was highly non-signifi- dementia cases had impaired accessibility compared to
cant for dementia cases. controls.
MEMORY 393
Figure 3. Percentage of participants accessing each category for trial 1 (A) trial 2 (B), and trial 3 (C) and average number of words recalled given the category
was accessed for trial 1 (D), trial 2 (E), and trial 3 (F) for control and dementia cases.
Table 3 also summarises a series of nine independent performance (the average recall over trials 1–3 of the
samples t-tests performed to compare the number of HVLT) was related to the characteristics of word fre-
words recalled given the category was accessed for quency, word length or orthographic neighbourhood
control and dementia cases for each category and each size. None of the correlations were significant indicating
trial (Figure 3, Panels D–F). Note that mixed ANOVAs that there is no relationship between these character-
were not performed due to the low number of participants istics and recall performance. Control group: word fre-
who accessed every category within a trial. Controls quency, r (12) 0.131, p = 0.685, word length, r (12)
recalled significantly more words than dementia cases for 0.065, p = 0.841, and orthographic neighbourhood size,
the majority of categories and trials, apart from trial 1 r (12) 0.028, p = 0.932; Dementia group: word frequency,
stones and shelter. r (12) 0.256, p = 0.422, word length, r (12) −0.041, p =
0.900, and orthographic neighbourhood size, r (12)
0.197, p = 0.539.
Relationship of word characteristics and recall
For each characteristic within the Indonesian version
A series of six correlations were performed to deter- the 12 words were median split into high and low fre-
mine whether control and dementia cases recall quency, short and long words, and small and large
Table 3. The likelihood of accessing each category across trials 1–3 for the Serial position curves
Control and Dementia cases.
Chi-square t-test Figure 5 shows the serial position curves for the dementia
χ2 p df t p and control groups for each of the three trials (Panel A –
Trial 1 trial 1, Panel B – trial 2, and Panel C – trial 3). Panel D
Animal 7.49 0.006 93 2.96 0.004 shows how likely each word was to be recalled across
Stones 7.64 0.006 59 1.32 0.191 the three trials. Recall in all trials is higher for the control
Shelter 8.18 0.004 55 0.34 0.739
Trial 2 group compared to dementia cases. Both controls and
Animal 6.86 0.009 92 2.78 0.007 dementia cases show increased recall for the start and
Stones 9.76 0.002 76.7 4.02a <0.001 end of the list, indicating primacy and recency respectively.
Shelter 11.48 <0.001 54.7 2.79a 0.007
Trial 3 However, for dementia cases in particular, this may well be
Animal 8.49 0.004 90 3.30 0.001 a consequence of the placement of the words; serial pos-
Stones 12.68 <0.001 84 3.37 0.001 itions 1, 3 and 11 are words from the highly recalled
Shelter 11.69 <0.001 71 3.45 0.001
animal category.
Note: aIndicates the initial assumption checks indicated that Levene’s Test for
Equality of Variances was violated therefore the equal variance not Table 5 summarises a series of four 2 (group: dementia
assumed values were reported. or control) × 12 (serial position: 1–12) mixed ANOVAs per-
formed to examine serial position performance in each of
orthographic neighbourhood size.9 Figure 4 shows the the four panels of Figure 5. For all ANOVAs the main
values for high and low word frequency (Panel A) and effect of group was significant; controls had significantly
large and small orthographic neighbourhood size (Panel higher recall than dementia cases. In addition, the main
B) for both dementia and control groups. effect of serial position was significant; Bonferroni cor-
Table 4 summarises two 2 (group: dementia or rected pairwise comparisons indicated primacy and
control) × 2 (word characteristic: low or high) × 3 (trial: recency (the exact statistics can be found in Appendix 2
1–3) mixed ANOVAs performed separately on the word for trial 1, Appendix 3 for trial 2, Appendix 4 for trial 3,
frequency and orthographic neighbourhood size word and Appendix 5 for trials 1–3). The interaction was not sig-
characteristics to examine whether they have an effect nificant for trial 2, trial 3 and overall trials 1–3 analyses.
on recall. The overall proportion of words recalled analysis However, the interaction was significant for trial 1. Simple
reported earlier summarises the findings for the main effects compared performance between the groups at
effects of group and trial and the two-way interaction each serial position only, as this was the main comparison
between them. Within the current analysis we are inter- of interest. Using a Bonferroni correction (p = 0.00417),
ested in the effects of word frequency and orthographic recall for dementia cases was significantly lower than con-
neighbourhood size. For word frequency the main effect trols at serial positions 2, 3, and 9 (all p’s < 0.002). Therefore
and two-way interaction with trial were significant; high the interaction was driven by similar recall for dementia
frequency words were recalled better than low frequency cases and controls except for serial positions 2, 3, and 9.
words. Simple effects using a Bonferroni correction (p =
0.00556) comparing high and low frequency within
each trial indicated that low frequency words were
Retrieval dynamics across trials
remembered significantly worse than high frequency Figures 2 and 5 hint at the learning and forgetting that
words for trial 1 (p < 0.001) but there were no significant occurred across the trials. Within Figure 5 the dementia
differences for trials 2 and 3 (both p’s < 0.018). Looking cases appeared to demonstrate low levels of learning
across trials within the high frequency words only, across trials (i.e., adding new words), whereas the control
recall stayed the same across trials (trial 1 = trial 2 = trial group demonstrated higher levels particularly for the
3). Looking across trials within the low frequency words asymptote (middle) serial positions. However, Figure 5
only, recall significantly increased between each trial only indicates overall increases or decreases in specific
(trial 1 < trial 2 < trial 3). Therefore the interaction was words over trials. It does not allow for a detailed under-
driven by improvement in recall for the low frequency standing of the learning, retention and forgetting across
words as the test progressed. However, the two-way trials for individual participants. Therefore for each partici-
interaction between word frequency and group and the pant, for each word, and across all trials, the number of
three-way interaction between word frequency, group times they retained, learnt or forgot a word was calculated
and trial were not significant. For orthographic neigh- in order to assess retrieval dynamics.
bourhood size the main effect was significant; words Table 6 shows the proportion of learning, retention and
with fewer orthographic neighbours were recalled forgetting occurring across trials 2 and 3 for control and
better than words with more orthographic neighbours. dementia cases. The data was divided into four categories:
However, all interactions were non-significant. Therefore (1) Re-remembered means that a word was recalled on the
both word frequency and orthographic neighbourhood current trial and had been recalled on previous trial(s), that
size did not affect dementia cases and controls is, retention, (2) New means that it was recalled on the
differently. current trial but had not been recalled on any previous
MEMORY 395
Figure 4. Mean proportion of words recalled for trials 1–3 in the HVLT for control and dementia cases for high and low frequency words (A) and large and
small orthographic neighbourhood size words (B).
trial(s), that is, learning, (3) Forgotten means that it was not remembered and forgotten, and more words were
recalled on the current trial but had been recalled on pre- acquired and never recalled in trial 2 compared to trial
vious trial(s), (4) Never recalled means that it was not 3. The interaction was not significant for the new or forgot-
recalled on the current trial and had not been recalled on ten words analyses. However, the interactions were signifi-
any previous trial(s). Control cases showed much greater cant for the re-remembered and never recalled words
levels of re-remembering and lower levels of words never analyses. Looking first at the re-remembered interaction
recalled for trials 2 and 3 compared to dementia cases. simple effects, using a Bonferroni correction (p = 0.0125),
However, the rates of new words learnt and words forgot- indicated that controls re-remembered significantly more
ten for trials 2 and 3 appeared to be more similar for both than dementia cases for both trial 2 and 3 (both p’s <
control and dementia cases. 0.001) and the amount of re-remembered words signifi-
Table 7 summarises a series of four 2 (group: dementia cantly increased between trial 2 and 3 for both control
or control) × 2 (trial: 2–3) mixed ANOVAs performed to and dementia cases (both p’s < 0.002). Therefore, this inter-
examine the levels of re-remembered, new, forgotten, action was driven by dementia cases showing significantly
and never recalled words for controls and dementia lower levels of re-remembering than controls for both trials
cases shown in Table 7. For all ANOVAs the main effect 2 and 3. Looking now at the never recalled interaction
of group was significant; compared to controls, dementia simple effects, using a Bonferroni correction (p = 0.0125),
cases re-remembered, acquired, and forgot fewer words indicated that the amount of words never recalled was sig-
and never recalled more words. For all ANOVAs the main nificantly higher for dementia cases compared to controls
effect of trial was significant; fewer words were re- for both trial 2 and 3 (both p’s < 0.001) and the amount of
words never recalled significantly decreased between trial
2 and 3 for both control and dementia cases (both p’s <
Table 4. Summary of the ANOVA analyses conducted on the proportion of 0.001). Therefore, the interaction was driven by dementia
words recalled as a function of word frequency and orthographic
neighbourhood size for trials 1–3. cases showing significantly higher levels of words never
df MSE F p recalled than controls for both trials 2 and 3.
Word frequency Due to the difference between the number of words of
Group 1, 122 0. 141 69.91 <.001 the HVLT list remembered by controls and dementia cases,
Word frequency 1, 122 0.054 11.77 .001 perhaps a more accurate comparison to assess retrieval
Triala 1.785, 217.7 0.018 25.84 <.001
Group × word frequency 1, 122 0.054 0.07 .792 dynamics would be to calculate the proportion of words
Group × trial 2, 244 0.018 11.22 <.001 acquired and forgotten relative to the proportion of the
Word frequency × trial 2, 244 0.017 5.59 .004 list previously recalled. For example, dementia cases
Group × word frequency × trial 2, 244 0.017 2.20 .114
Orthographic neighbourhood size appear to have forgotten more words than controls in
Group 1, 122 0. 141 69.91 <.001 the above analysis but this is because they remember
Orthographic N 1, 122 0.046 5.97 .016 less than controls to start with. Looking at the words for-
Triala 1.785, 217.7 0.018 25.84 <.001
Group × orthographic N 1, 122 0.046 2.72 .102 gotten relative to the proportion of the list previously
Group × trial 2, 244 0.018 11.22 <.001 recalled, controls show the same proportion of words
Orthographic N × trial 2, 244 0.023 0.36 .701 that they forgot across the two trials (trial 2 = 29% of the
Group × orthographic N × trial 2, 244 0.023 0.37 .694
list previously remembered, trial 3 = 29% of the list pre-
of Sphericity was violated therefore degrees of freedom were corrected viously remembered). Dementia cases also show a very
using the Greenhouse–Geisser correction. similar, albeit higher than controls, proportion of words
Figure 5. Overall serial position curves for control and dementia cases for trial 1 (A), trial 2 (B), and trial 3 (C). Panel D shows the overall likelihood of recalling
each word across the three trials.
that they forgot across the two trials (trial 2 = 41% of the list However, dementia cases show a lower- but greater- differ-
previously remembered, trial 3 = 42% of the list previously ence in the proportion of the remaining list that they
remembered). Looking at the words newly acquired rela- acquire across the two trials (trial 2 = 11% of the list yet
tive to the proportion of the list previously recalled, con- to be recalled, trial 3 = 7% of the list yet to be recalled).
trols show a very similar proportion of newly acquired
words across the two trials (trial 2 = 26% of the list yet to
be recalled, trial 3 = 27% of the list yet to be recalled). Discussion
The aim of this paper was to investigate in-depth recall pat-
terns within the HVLT to determine whether there are
Table 5. Summary of the ANOVA analyses conducted on the serial position
curves for trial 1, trial 2, trial 3, and overall for trials 1–3. important subtle differences between control and demen-
df MSE F p tia cases recall that are not currently being captured using
Serial position curve for trial 1 a the total immediate recall score. We hoped that any novel
Group 1, 122 0. 293 37.5 <.001 findings might lead to improvement in the efficacy of the
Serial position 9.214, 1124.2 0.187 12.8 <.001 HVLT.
Group × serial position 9.214, 1124.2 0.187 1.90 0.036
Serial position curve for trial 2 a As expected the dementia cases had a much lower total
Group 1, 122 0.330 61.0 <.001 immediate recall score on the HVLT compared to controls.
Serial position 9.694, 1182.7 0.194 10.8 <.001 However, we found dementia cases exhibited widespread
Serial position curve for trial 3 a impairments across all analyses. The dementia cases
Group 1, 122 0.440 69.3 <.001 showed no improvement in recall across the three trials
Serial position 9.493, 1158.2 0.200 8.53 <.001 in agreement with previous research of diminished learn-
Serial position curve for trials 1–3 a ing rates (Delis et al., 1991; Ober et al., 1985; Perri et al.,
Group 1, 122 0.616 66.0 <.001 2005; Weingartner et al., 1981, 1983). Focusing on recall
Serial position 9.581, 1168.9 0.191 16.3 <.001 within the three categories of the HVLT, dementia cases
showed poorest recall for the shelter category and best
of Sphericity was violated therefore degrees of freedom were corrected recall for the animals category; although for trial 1 recall
using the Greenhouse–Geisser correction. of stones and shelter was not different. These trends
MEMORY 397
Table 6. The learning and retention percentages across trials 2 and 3 for the Control and Dementia cases.
SP 1 2 3 4 5 6 7 8 9 10 11 12
Word Lion Emerald Horse Tent Sapphire Hotel Cave Opal Tiger Pearl Cow Hut Total
CONTROL GROUP
Trial 2 Re-remembered 0.37 0.26 0.44 0.11 0.13 0.13 0.06 0.19 0.39 0.24 0.35 0.24 0.24
New 0.21 0.23 0.21 0.13 0.10 0.13 0.11 0.26 0.10 0.23 0.11 0.26 0.17
Forgotten 0.16 0.08 0.13 0.08 0.05 0.05 0.03 0.13 0.13 0.03 0.19 0.10 0.10
Never recalled 0.26 0.44 0.23 0.68 0.73 0.69 0.79 0.42 0.39 0.50 0.34 0.40 0.49
New 0.11 0.15 0.10 0.10 0.19 0.24 0.11 0.10 0.08 0.15 0.08 0.10 0.13
Forgotten 0.24 0.08 0.24 0.15 0.10 0.10 0.11 0.23 0.21 0.15 0.18 0.06 0.15
Never recalled 0.15 0.29 0.13 0.58 0.53 0.45 0.68 0.32 0.31 0.35 0.26 0.31 0.36
DEMENTIA GROUP
New 0.13 0.08 0.19 0.03 0.05 0.06 0.02 0.08 0.05 0.16 0.10 0.08 0.09
Forgotten 0.13 0.05 0.11 0.05 0.02 0.02 0.03 0.11 0.05 0.10 0.13 0.06 0.07
Never recalled 0.52 0.81 0.56 0.90 0.87 0.84 0.94 0.74 0.81 0.68 0.58 0.71 0.75
New 0.05 0.11 0.05 0.08 0.06 0.03 0.03 0.00 0.06 0.05 0.03 0.06 0.05
Forgotten 0.26 0.06 0.11 0.05 0.06 0.05 0.03 0.10 0.11 0.15 0.18 0.15 0.11
Never recalled 0.47 0.69 0.52 0.82 0.81 0.81 0.90 0.74 0.74 0.63 0.55 0.65 0.69
Notes: Guide to categories: Re-remembered means that it was recalled on the current trial and had been recalled on previous trial(s), New means that it was
recalled on the current trial but had not been recalled on any previous trial(s), Forgotten means that it was not recalled on the current trial but it had been
recalled on previous trial(s), Never recalled means that it was not recalled on the current trial and had not been recalled on any previous trial(s).
were mirrored in the control cases, albeit at higher recall significantly earlier than both stones and shelter with
values. This is in contrast to Schrijnemaekers et al. (2006) Schrijnemaekers et al. (2006) non-significant findings
who found no significant difference between recall of the exactly mirroring these trends. Dementia cases show a
three categories. Although their trends revealed animals robust advantage for early acquired words over a range
were recalled best, stones and shelter were equally of tasks (Cuetos et al., 2010; Forbes-McKay et al., 2005;
poorly recalled. However, it is important to note that Schrij- Kremin et al., 2000; Ralph et al., 1998; Silveri et al., 2002).
nemaekers et al. study utilised an English sample and the We also found dementia cases had reduced accessibility
English version of the HVLT whereas we utilised an Indone- to all categories within all trials compared to controls.
sian sample and an Indonesian version of the HVLT. Cul- Given that a category was accessed, dementia cases sub-
tural differences can impact on category recall, possibly sequent recall was also impaired relative to controls in
related to the following linguistic aspects. accord with previous research (Martin & Fedio, 1983;
The linguistic characteristics we were able to analyse did Ober, Dronkers, Koss, Delis, & Friedland, 1986).
not reveal significant differences on any of our measures Dementia cases often display a reduced primacy effect,
between the categories. Therefore another linguistic but a relatively preserved recency effect relative to controls
feature that we could not measure may be driving these (e.g., Bayley et al., 2000; Carlesimo et al., 1996; Foldi et al.,
differences. Age-of-acquisition is a potential characteristic 2003; Gainotti & Marra, 1994; Howieson et al., 2011;
that could be causing these differences. The English age- Massman et al., 1993; Tierney et al., 1994). However, we
of-acquisition values showed that animals were acquired found reduced levels of both primacy and recency for
dementia cases compared to controls. This discrepant
finding may be due to the recency effect declining with
Table 7. Summary of the ANOVA analyses conducted on the proportion of the severity of dementia (Pepin & Eslinger, 1989). Whilst
re-remembered, new, forgotten, and never recalled words.
these levels were reduced, dementia cases did show sig-
df MSE F p
nificant levels of primacy and recency within their recall.
Proportion of re-remembered words
Group 1, 122 0.421 57.3 <.001 However, we are concerned that the higher proportion of
Trial 1, 122 0.101 49.5 <.001 the well-remembered animals category at the start and
Group × trial 1, 122 0.101 8.32 0.005 end of the list (serial positions 1, 3 and 11) may be inflating
Proportion of new words
Group 1, 122 1.51 37.3 <.001 primacy and recency levels. Due to this confound, it is
Trial 1, 122 1.20 12.1 0.001 unclear whether the improved performance is due to one
Group × trial 1, 122 1.20 0.337 0.563 or both of these factors. In addition three of the words
Proportion of forgotten words
Group 1, 122 2.49 4.39 0.038 from the shelter category are in the middle serial positions
Trial 1, 122 0.709 27.9 <.001 (4, 6 and 7) which may contribute to their poorer recall.
Group × trial 1, 122 0.709 1.12 0.293 Other memory tests used to screen for dementia present
Proportion of never recalled words
Group 1, 122 12.7 60.7 <.001 the words in a random order each time (e.g., CVLT, Delis
Trial 1, 122 0.500 14.1 <.001 et al., 1987; CERAD word learning list, Welsh et al., 1991)
Group × trial 1, 122 0.500 23.5 <.001 which eliminates these confounds. However, this may be
one of the reasons why floor effects are not usually that the two languages provided broadly similar findings.
reported on the HVLT. To help disentangle these issues The English and Indonesian languages were strongly corre-
future work could manipulate the presentation order of lated for both word frequency and orthographic neigh-
the words by either presenting the words in a random bourhood size characteristics. Within each language, all
order for each trial or, perhaps more interestingly, by sys- but one characteristic did not correlate with each other,
tematically varying the location of the categories within indicating that these linguistic measures are relatively inde-
the list such that that one word of each category occurs pendent. For both languages word frequency, word length,
in all four sections of the list. and orthographic neighbourhood size did not vary across
Looking in detail at the intertrial retrieval dynamics the three categories of Form 1 of the HVLT. However,
revealed that dementia cases did recall new words in there were two differences between the two languages.
trials 2 and 3 despite not showing a significant increase Firstly, there was no correlation between the English and
in learning from trial 1 to 3 overall. However, dementia Indonesian languages for word length, although this is
cases did learn significantly less than controls. Dementia most likely due to the lack of variation within the data. Sec-
cases also recall significantly less of the list and, relative ondly, within the English language there was a relationship
to the proportion of the list already recalled, also forgot between word length and orthographic neighbourhood
more words, mirroring the findings for gained and lost size which was non-significant within the Indonesian
access (Genon et al., 2013; Moulin et al., 2004; Woodard language. Therefore the use of Indonesian language may
et al., 1999). Dementia cases appear to have deficits in have partially affected the results we may have expected
both learning and consolidation indicating issues with with an English speaking sample. We also found age-of-
encoding and storage, respectively. It would appear from acquisition did vary across the categories within the
these results and previous findings (Algarabel et al., 2009, English version, but no comparison with Indonesian was
2012; Ally et al., 2009; Butters et al., 1983, 1987; Carlesimo possible due to the lack of values for age-of-acquisition.
et al., 1998; Dallabarba & Wong, 1995; Dannenbaum Therefore, we were also unable to determine the possible
et al., 1988; Delis et al., 1991; Gaines et al., 2006; Genon effect of age-of-acquisition on recall within the current
et al., 2013; Glosser et al., 2002; Granholm & Butters, data. Previous research has not looked at recall (Cuetos
1988; Grober & Buschke, 1987; Hodges et al., 1999; Koen et al., 2010; Forbes-McKay et al., 2005; Kremin et al., 2000;
& Yonelinas, 2014; Kopelman, 1991; Martin et al., 1985; Ralph et al., 1998; Silveri et al., 2002). We would be keen
Moulin et al., 2004; Ober et al., 1985; Perri et al., 2005; Wein- to address this within future research.
gartner et al., 1981, 1983; Wolk et al., 2011; Woodard et al., We also investigated the effect of these characteristics
1999) that dementia cases have impairment at all stages of on recall. We found that neither word frequency nor ortho-
memory (encoding, storage, and retrieval). graphic neighbourhood size had a differential effect on
What was interesting were the relative rates across trials recall in control and dementia cases. However, we did
2 and 3 of the proportion of newly acquired words and the find an overall word frequency effect; high frequency
loss of previously recalled words. With regards to immedi- words were recalled better than low frequency words, for
ate forgetting rates, the proportion of forgotten words both control and dementia cases. This is in line with pre-
changed by less than 1% across trials 2 and 3 for both vious findings from recall tasks for controls (Deese, 1960;
control and dementia cases, showing that whilst immedi- Postman, 1970; Sumby, 1963) and the only study to inves-
ate forgetting rates are higher in dementia cases, they tigate dementia cases with free recall (Hessler et al., 2016).
follow a similar trend. Acquisition rates are slightly more We also found an overall effect of orthographic neighbour-
divergent between dementia cases and controls. Controls hood size; where words with fewer orthographic neigh-
show a stable rate of acquisition across trials 2 and 3, bours produced better recall for both control and
however, dementia cases show a decrease in acquisition dementia cases. Hessler et al. (2016) also found better
rates, even though in absolute terms they have a much recall with fewer orthographic neighbours for dementia
larger number of never recalled words to draw from. cases. However, they found the opposite pattern in
These rates of immediate forgetting and, in particular, control cases. Previous research using serial recall also sup-
acquisition might be a useful diagnostic tool alongside ports Hessler et al.’s finding for controls (Allen & Hulme,
the total recall score to be able to differentiate between 2006; Glanc & Greene, 2012; Jalbert, Neath, & Surprenant,
dementia and other memory disorders, such as aMCI, 2011; Jalbert, Neath, Bireta, et al., 2011; Roodenrys et al.,
which often displays results in-between dementia cases 2002). We are not certain why our results differ, but one
and controls. possibility may be due to the much smaller vocabulary of
Additionally we also looked at several linguistic charac- Indonesian compared to German and English affording
teristics of the words within the HVLT. This data set was col- the possibility of fewer orthographic neighbours. Further
lected with Indonesian speaking participants but we research investigating the effect of orthographic neigh-
analysed the linguistic features of the 12 words of Form bourhood size within free recall would address this
1 of the HVLT for both English and Indonesian languages current discrepancy.
as these characteristics have never been researched These linguistic findings reveal the importance of the
before, in either language or within the HVLT. We found appropriate and careful selection of words for verbal
MEMORY 399
recall tests and have an impact on the construction of collect data that would allow us to extend our novel set
future word-based dementia screening tests and the of analyses to determine whether the six forms are actu-
potential updating of current tests. Much research in ally equivalent based on the findings from Form 1 in this
dementia uses the levels of primacy and recency as a paper. Secondly, it was unknown at the time of data col-
marker for diagnosis. However, the tests that they utilise lection that the data would be used for in-depth analysis
do not look at the linguistic characteristics of each individ- and therefore output order was not recorded. Output
ual word. The effects of these four characteristics may not order data would have enabled recall consistency and
be uniform across the list and therefore be producing a semantic clustering analyses to be performed that are
confound between serial position and these characteristics. known to be reduced in dementia recall (e.g., Carlesimo
For example, word frequency appears to affect pre-recency et al., 1998; Delis et al., 1991; Gaines et al., 2006; Glosser
locations (Raymond, 1969; Sumby, 1963). However, more et al., 2002; Hodges et al., 1999; Perri et al., 2005; Wein-
recently word frequency has been found to affect gartner et al., 1981). Recording output order would also
recency positions (Van Overschelde, 2002) but these dis- have allowed for analysis of where in the list participants
cordant findings may be due to the use of pure and initiate their recall. There may be differences in the start-
mixed frequency lists. Word frequency may also affect ing point of recall between controls and dementia
where recall is initiated within the list which then has a patients, and also differences in whether the same word
knock-on effect in recall. Recall is more likely to be initiated is chosen as the first word in trials 1, 2 and 3. Output
with the first word in the list for high frequency words, but order also allows the full sequence to be investigated
towards the end of the list for low frequency words allowing for assessment, for each participant, of how
(Sumby, 1963). Grenfell-Essam and colleagues have similar recall is from trial to trial. Thirdly, we would also
shown that where recall is initiated has a large effect on like to reproduce this work, along with the added
primacy and recency (Grenfell-Essam & Ward, 2012, 2015; output order analyses, in an English sample as changes
Grenfell-Essam et al., 2013, 2017; Ward et al., 2010). to some of the stones were necessary to accommodate
We have shown that both word frequency and ortho- local knowledge and we were also unable to examine
graphic neighbourhood size affect recall, with the effect the possible effect of age-of-acquisition. The linguistic
of age-of-acquisition unknown. As the HVLT repeats the characteristics may have a large effect on primacy and
words in the same order in each list this makes an in- recency. Finally, we would also be keen to extend this
depth analyses of the linguistic characteristics of the five in-depth analysis of HVLT with other populations, such
other forms of the HVLT all the more important. In the as vascular dementia and very early dementia cases
future to avoid either floor or ceiling effects, the difficulty (e.g., possible Alzheimer’s disease or aMCI), to see
of tests can be altered by adjusting the word frequency, whether there are distinct patterns of deficits that can
word length, age-of-acquisition and/or orthographic be identified to once again help improve the efficacy of
neighbourhood size. Indeed the CVLT-II utilises higher fre- this test. Indeed, due to the widespread impairments of
quency words specifically to avoid floor effects (Delis, dementia cases across all performance measures in this
Kramer, Kaplan, & Ober, 2000). Researchers need to also study, it might be more useful to profile earlier deficits
carefully consider the location of these items within the in patients with aMCI. aMCI patients may show less wide-
list and whether the words are presented in the same, or spread patterns of impairment that when tracked over
random, order. time might be shown to be more likely to lead to
This study does have some limitations. Our algorithm dementia.
for dementia was validated in several studies in Indonesia In conclusion, this paper has demonstrated that there
(see Hogervorst et al., 2011), However, future work needs are important differences between control and dementia
to further validate this work in clinically established case recall that are not captured in the total recall score.
dementia cases and controls. Another limitation is the We have demonstrated that, within the HVLT, dementia
translation of the test which does not allow transference cases exhibit: consistent and poor recall across the three
of results to other cultures. Future work should look at a trials, poorest recall for shelter, but best for animals,
range of cultures to investigate the generalisability of our impaired accessibility of all categories and reduced sub-
results. The translation of this test into Indonesian also sequent recall from accessed categories, reduced recall at
did not allow for the characteristic of age-of-acquisition the start and end of the list, lower levels of re-remembering
to be assessed or use of established word frequency and and recalling new words; and higher levels of forgetting
orthographic neighbourhood size corpora. We see this and never recalling words. These differences help charac-
research as the first step in developing a validated terise in more detail the exact nature of the reduced
version of the HVLT. recall performance of dementia cases compared to con-
There are several ways in which this research can be trols, and when used alongside the total recall score, may
pursued in the future. Firstly, the fact that the HVLT has help improve the efficacy of the HVLT. This research may
six different forms is a strength of the test, as it allows help improve the diagnostic ability and accuracy of the
for repeated testing (Benedict et al., 1998; Krebs, 1994; HVLT to detect earlier stages of dementia than currently
Rasmusson et al., 1995; Woods et al., 2005). We wish to possible. It could also be valuable when monitoring
individuals over time to identify subtle changes in HVLT recognition performance. Journal of Memory and Language, 46(1),
performance not captured with the total recall score. 199–226.
Barry, C., Morrison, C. M., & Ellis, A. W. (1997). Naming the Snodgrass
Finally, it might prove beneficial in assessment of treat-
and Vanderwart pictures: Effects of age of acquisition, frequency
ment efficacy. Therefore, going forward this research may and name agreement. Quarterly Journal of Experimental
have wide reaching implications. Psychology, 50(A), 560–585.
Bayley, P. J., Salmon, D. P., Bondi, M. W., Bui, B. K., Olichney, J., Delis,
D. C., … Thal, L. J. (2000). Comparison of the serial position effect
Notes in very mild Alzheimer’s disease, mild Alzheimer’s disease, and
amnesia associated with electroconvulsive therapy. Journal of the
1. The data were collected from Indonesian speaking participants.
International Neuropsychological Society, 6(03), 290–298.
2. Indonesian uses the Latin alphabet making direct comparison
Belleville, S., Peretz, I., & Malenfant, D. (1996). Examination of the
easier.
working memory components in normal aging and in dementia
3. Age-of-acquisition values were not available for the Indonesian
of the Alzheimer type. Neuropsychologia, 34(3), 195–207.
version.
Benedict, R. H., Schretlen, D., Groninger, L., & Brandt, J. (1998). Hopkins
4. Using the Indonesian values.
Verbal Learning Test – revised: Normative data and analysis of inter-
5. Please see Table 1 for details.
form and test-retest reliability. The Clinical Neuropsychologist, 12(1),
6. This is the only set of analyses which investigate the English
43–55.
version of the HVLT.
Benedict, R. H., Schretlen, D., Groninger, L., Dobraski, M., & Shpritz, B.
7. No age-of-acquisition values within the Indonesian language
(1996). Revision of the brief visuospatial memory test: Studies of
were available.
normal performance, reliability, and validity. Psychological
8. These provide raw scores rather than values per million. This
Assessment, 8(2), 145–153.
word frequency source was the only data set which provided
Brandt, J. (1991). The Hopkins Verbal Learning Test: Development of a
values for all words within the Indonesian version of the HVLT.
new memory test with six equivalent forms. Clinical
9. Word length was not analysed further due to the unequal
Neuropsychologist, 5(2), 125–142.
median split (low = 9 words, high = 3 words).
Brysbaert, M., & New, B. (2009). Moving beyond Kucera and Francis: A
critical evaluation of current word frequency norms and the intro-
duction of a new and improved word frequency measure for
Acknowledgements American English. Behavior Research Methods, 41(4), 977–990.
We would like to thank all staff (in particular Tanto and Yono from Uni- Butters, N., Albert, M. S., Sax, D. S., Miliotis, P., Nagode, J., & Sterste, A.
versity of Respati Yogyakarta) and all participants from SEMAR for their (1983). The effect of verbal mediators on the pictorial memory of
help, efforts and support. brain-damaged patients. Neuropsychologia, 21(4), 307–323.
Butters, N., Granholm, E., Salmon, D. P., Grant, I., & Wolfe, J. (1987).
Episodic and semantic memory: A comparison of amnesic and
Disclosure statement demented patients. Journal of Clinical and Experimental
Neuropsychology, 9(5), 479–497.
No potential conflict of interest was reported by the authors. Carlesimo, G. A., Fadda, L., Sabbadini, M., & Caltagirone, C. (1996).
Recency effect in Alzheimer’s disease: A reappraisal. Quarterly
Journal of Experimental Psychology A, Human Experimental
Funding Psychology, 49, 315–325.
Carlesimo, G. A., Mauri, M., Graceffa, A. M., Fadda, L., Loasses, A.,
Data collection was supported by an Alzheimer’s Research UK grant Lorusso, S., & Caltagirone, C. (1998). Memory performances in
(Relationship of salivary phytoestrogens to age-related cognitive young, elderly, and very old healthy individuals versus patients
decline and possible dementia in Indonesia, ART/PPG2006A/2). We with Alzheimer’s disease: Evidence for discontinuity between
would also like to thank Loughborough University UK and Universitas normal and pathological aging. Journal of Clinical and
of Diponegoro, Respati and Indonesia for their generous support of Experimental Neuropsychology, 20(1), 14–29.
this study and without whom this study would not have been possible. Carroll, J. B., & White, M. N. (1973). Word frequency and age-of acqui-
sition as determiners of picture-naming latency. Quarterly Journal of
Experimental Psychology, 25, 85–95.
References Cherner, M., Suarez, P., Lazzaretto, D., Artiola i Fortuny, L., Mindt, M. R.,
Algarabel, S., Escudero, J., Mazón, J. F., Pitarque, A., Fuentes, M., Peset, Dawes, S., … Heaton, R. (2007). Demographically corrected norms
V., & Lacruz, L. (2009). Familiarity-based recognition in the young, for the brief visuospatial memory test-revised and Hopkins Verbal
healthy elderly, mild cognitive impaired and Alzheimer’s patients. Learning Test-revised in monolingual Spanish speakers from the
Neuropsychologia, 47(10), 2056–2064. US–Mexico border region. Archives of Clinical Neuropsychology, 22
Algarabel, S., Fuentes, M., Escudero, J., Pitarque, A., Peset, V., Mazón, (3), 343–353.
J. F., & Meléndez, J. C. (2012). Recognition memory deficits in mild Christian, J., Bickley, W., Tarka, M., & Clayton, K. (1978). Measures of free
cognitive impairment. Aging, Neuropsychology, and Cognition, 19 recall of 900 English nouns: Correlations with imagery, concrete-
(5), 608–619. ness, meaningfulness, and frequency. Memory & Cognition, 6(4),
Allen, R., & Hulme, C. (2006). Speech and language processing mech- 379–390.
anisms in verbal serial recall. Journal of Memory and Language, 55, Coltheart, M., Davelaar, E., Jonasson, J. T., & Besner, D. (1977). Access to
64–88. the internal lexicon. In S. Dornic (Ed.), Attention and performance VI
Ally, B. A., Gold, C. A., & Budson, A. E. (2009). An evaluation of recollec- (pp. 535–555). Hillsdale, NJ: Erlbaum.
tion and familiarity in Alzheimer’s disease and mild cognitive Coltheart, V., & Winograd, E. (1986). Word imagery but not age of
impairment using receiver operating characteristics. Brain and acquisition affects episodic memory. Memory & Cognition, 14(2),
Cognition, 69(3), 504–513. 174–180.
Balota, D. A., Burgess, G. C., Cortese, M. J., & Adams, D. R. (2002). The Cuetos, F., Herrera, E., & Ellis, A. W. (2010). Impaired word recognition in
word-frequency mirror effect in young, old, and early-stage Alzheimer’s disease: The role of age of acquisition.
Alzheimer’s disease: Evidence for two processes in episodic Neuropsychologia, 48(11), 3329–3334.
MEMORY 401
Dallabarba, G., & Wong, C. (1995). Encoding specificity and intrusion in Experimental Psychology: Learning, Memory, and Cognition, 24,
Alzheimers-disease and amnesia. Brain and Cognition, 27(1), 1–16. 267–283.
Dannenbaum, S. E., Parkinson, S. R., & Inman, V. W. (1988). Short-term Gilhooly, K. J., & Gilhooly, M. L. (1979). Age-of-acquisition effects in
forgetting: Comparisons between patients with dementia of the lexical and episodic memory tasks. Memory & Cognition, 7(3),
Alzheimer type, depressed, and normal elderly. Cognitive 214–223.
Neuropsychology, 5(2), 213–233. Glanc, G., & Greene, R. (2012). Orthographic distinctiveness and
de Jager, C. A., Schrijnemaekers, A. M. C., Honey, T. E. M., & Budge, M. M. memory for order. Memory, 20(8), 865–871.
(2009). Detection of MCI in the clinic: Evaluation of the sensitivity Glosser, G., Gallo, J. L., Clark, C. M., & Grossman, M. (2002). Memory
and specificity of a computerised test battery, the Hopkins Verbal encoding and retrieval in frontotemporal dementia and
Learning Test and the MMSE. Age and Ageing, 4, 1–6. Alzheimer’s disease. Neuropsychology, 16(2), 190–196.
Deese, J. (1960). Frequency of usage and number of words in free Granholm, E., & Butters, N. (1988). Associative encoding and retrieval in
recall: The role of association. Psychological Reports, 7, 337–344. Alzheimer’s and Huntington’s disease. Brain and Cognition, 7(3),
Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (1987). CVLT: California 335–347.
verbal learning test-adult version: Manual. San Antonio, TX: Grenfell-Essam, R., & Ward, G. (2012). Examining the relationship
Psychological Corporation. between free recall and immediate serial recall: The role of list
Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000). CVLT-II: length, strategy use, and test expectancy. Journal of Memory and
California Verbal Learning Test: Adult version. San Antonio, TX: Language, 67, 106–148.
Psychological Corporation. Grenfell-Essam, R., & Ward, G. (2015). The effect of selective attention
Delis, D. C., Massman, P. J., Butters, N., Salmon, D. P., Cermak, L. S., & and a stimulus prefix on the output order of immediate free
Kramer, J. H. (1991). Profiles of demented and amnesic patients recall of short and long lists. Canadian Journal of Experimental
on the California verbal learning test: Implications for the assess- Psychology, 69(1), 1–16.
ment of memory disorders. Psychological Assessment: A Journal of Grenfell-Essam, R., Ward, G., & Tan, L. (2013). The role of rehearsal on
Consulting and Clinical Psychology, 3(1), 19–26. the output order of immediate free recall of short and long lists.
Dewhurst, S. A., Hitch, G. J., & Barry, C. (1998). Separate effects of word Journal of Experimental Psychology: Learning, Memory, and
frequency and age of acquisition in recognition and recall. Journal Cognition, 39, 317–347.
of Experimental Psychology: Learning, Memory, and Cognition, 24(2), Grenfell-Essam, R., Ward, G., & Tan, L. (2017). Common modality effects
284–298. in immediate free recall and immediate serial recall. Journal of
Dunabeitia, J. A., Marín, A., & Carreiras, M. (2009). Associative and Experimental Psychology: Learning, Memory, and Cognition.
orthographic neighborhood density effects in normal aging and Advance online publication. doi:10.1037/xlm0000430
Alzheimer’s disease. Neuropsychology, 23(6), 759–764. Grober, E., & Buschke, H. (1987). Genuine memory deficits in dementia.
Foldi, N. S., Brickman, A. M., Schaefer, L. A., & Knutelska, M. E. (2003). Developmental Neuropsychology, 3(1), 13–36.
Distinct serial position profiles and neuropsychological measures Hermit Dave. (2016). 2016 Frequency list files generated from http://opus.
differentiate late life depression from normal aging and lingfil.uu.se/OpenSubtitles2016.php. Retrieved from https://github.
Alzheimer’s disease. Psychiatry Research, 120(1), 71–84. com/hermitdave/FrequencyWords/blob/master/content/2016/id/
Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). “Mini-mental id_full.txt
state”: A practical method for grading the cognitive state of Hessler, J. B., Fischer, A. M., & Jahn, T. (2016). Differential linguistic recall
patients for the clinician. Journal of Psychiatric Research, 12(3), effects in the California verbal learning test in healthy aging and
189–198. Alzheimer’s dementia: Analysis of routine diagnostic data.
Forbes-McKay, K. E., Ellis, A. W., Shanks, M. F., & Venneri, A. (2005). The Archives of Clinical Neuropsychology, 31(7), 689–699.
age of acquisition of words produced in a semantic fluency task can Hester, R. L., Kinsella, G. J., Ong, B., & Turner, M. (2004). Hopkins Verbal
reliably differentiate normal from pathological age related cogni- Learning Test: Normative data for older Australian adults. Australian
tive decline. Neuropsychologia, 43(11), 1625–1632. Psychologist, 39(3), 251–255.
Frank, R. M., & Byrne, G. J. (2000). The clinical utility of the Hopkins Hodges, J. R., Patterson, K., Ward, R., Garrard, P., Bak, T., Perry, R., &
Verbal Learning Test as a screening test for mild dementia. Gregory, C. (1999). The differentiation of semantic dementia and
International Journal of Geriatric Psychiatry, 15(4), 317–324. frontal lobe dementia (temporal and frontal variants of frontotem-
Friedman, M. A., Schinka, J. A., Mortimer, J. A., & Graves, A. B. (2002). poral dementia) from early Alzheimer’s disease: A comparative neu-
Hopkins Verbal Learning Test – revised: Norms for elderly African ropsychological study. Neuropsychology, 13(1), 31–40.
Americans. The Clinical Neuropsychologist, 16(3), 356–372. Hogervorst, E., Combrinck, M., Lapuerta, P., Rue, J., Swales, K., & Budge,
Gaines, J. J., Shapiro, A., Alt, M., & Benedict, R. H. B. (2006). Semantic M. (2002). The Hopkins Verbal Learning Test and screening for
clustering indexes for the Hopkins Verbal Learning Test revised: dementia. Dementia and Geriatric Cognitive Disorders, 13(1), 13–20.
Initial exploration in elder control and dementia groups. Applied Hogervorst, E., Mursjid, F., Ismail, R. I., Prasetyo, S., Nasrun, M., Mochtar,
Neuropsychology, 13(4), 213–222. T. N. … Rahardjo, T. B. W. (2011). Validation of two short dementia
Gainotti, G., & Marra, C. (1994). Some aspects of memory disorders screening tests in Indonesia. In S. R. Jacobsen (Ed.), Vascular demen-
clearly distinguish dementia of the Alzheimer’s type from depress- tia: Risk factors, diagnosis and treatment (pp. 121–132). New York,
ive pseudo-dementia. Journal of Clinical and Experimental NY: Nova Sci.
Neuropsychology, 16(1), 65–78. Howieson, D. B., Mattek, N., Seeyle, A. M., Dodge, H. H., Wasserman, D.,
Ganguli, M., Ratcliff, G., Chandra, V., Sharma, S., Gilby, J., Pandav, R., … Zitzelberger, T., & Jeffrey, K. A. (2011). Serial position effects in mild
Dekosky, S. (1995). A Hindi version of the MMSE: The development cognitive impairment. Journal of Clinical and Experimental
of a cognitive screening instrument for a largely illiterate rural Neuropsychology, 33(3), 292–299.
elderly population in India. International Journal of Geriatric Jahnke, J. C. (1965). Primacy and recency effects in serial-position
Psychiatry, 10(5), 367–377. curves of immediate recall. Journal of Experimental Psychology, 70,
Genon, S., Collette, F., Moulin, C. J., Lekeu, F., Bahri, M. A., Salmon, E., & 130–132.
Bastin, C. (2013). Verbal learning in Alzheimer’s disease and mild Jalbert, A., Neath, I., Bireta, T. J., & Surprenant, A. M. (2011). When does
cognitive impairment: Fine-grained acquisition and short-delay length cause the word length effect? Journal of Experimental
consolidation performance and neural correlates. Neurobiology of Psychology: Learning, Memory, and Cognition, 37(2), 338–353.
Aging, 34(2), 361–373. Jalbert, A., Neath, I., & Surprenant, A. M. (2011). Does length or neigh-
Gerhand, S., & Barry, C. (1998). Word frequency effects in oral reading borhood size cause the word length effect? Memory & Cognition, 39
are not merely age-of-acquisition effects in disguise. Journal of (7), 1198–1210.
Koen, J. D., & Yonelinas, A. P. (2014). The effects of healthy aging, Ober, B. A., Koss, E., Friedland, R. P., & Delis, D. C. (1985). Processes of
amnestic mild cognitive impairment, and Alzheimer’s disease on verbal memory failure in Alzheimer-type dementia. Brain and
recollection and familiarity: A meta-analytic review. Cognition, 4(1), 90–103.
Neuropsychology Review, 24(3), 332–354. Pepin, E. P., & Eslinger, P. J. (1989). Verbal memory decline in
Kopelman, M. D. (1991). Non-verbal, short-term forgetting in the alco- Alzheimer’s disease A multiple-processes deficit. Neurology, 39
holic Korsakoff syndrome and Alzheimer-type dementia. (11), 1477–1477.
Neuropsychologia, 29(8), 737–747. Perri, R., Carlesimo, G. A., Serra, L., Caltagirone, C., & the Early Diagnosis
Krebs, R. (1994). The Hopkins Verbal Learning Test: An alternative to Group of the Italian Interdisciplinary Network on Alzheimer’s
the MMSE? The Gerontologist, 34, 692. Disease. (2005). Characterization of memory profile in subjects
Kremin, H., Hamerel, M., Dordain, M., De Wilde, M., & Perrier, D. (2000). with amnestic mild cognitive impairment. Journal of Clinical and
Age of acquisition and name agreement as predictors of mean Experimental Neuropsychology, 27(8), 1033–1055.
response latencies in picture naming of French adults. Brain and Postman, L. (1970). Effects of word frequency on acquisition and reten-
Cognition, 43, 286–291. tion under conditions of free-recall learning. The Quarterly Journal of
Kuperman, V., Stadthagen-Gonzalez, H., & Brysbaert, M. (2012). Age-of- Experimental Psychology, 22, 185–195.
acquisition ratings for 30,000 English words. Behavior Research Prince, M., Wimo, A., Guerchet, M., Ali, G. C., Wu, Y. T., & Prina, M.
Methods, 44(4), 978–990. (2015). The Global Impact of Dementia: An analysis of prevalence,
Kuslansky, G., Katz, M., Verghese, J., Hall, C. B., Lapuerta, P., LaRuffa, G., incidence, cost and trends. World Alzheimer Report. Retrieved
& Lipton, R. B. (2004). Detecting dementia with the Hopkins Verbal from https://www.alz.co.uk/research/WorldAlzheimerReport2015.
Learning Test and the mini-mental state examination. Archives of pdf
Clinical Neuropsychology, 19(1), 89–104. Ralph, M. A. L., Graham, K. S., Ellis, A. W., & Hodges, J. R. (1998). Naming
Lacritz, L. H., & Cullum, C. M. (1998). The Hopkins verbal learning test in semantic dementia – what matters? Neuropsychologia, 36(8),
and CVLT: A preliminary comparison. Archives of Clinical 775–784.
Neuropsychology, 13(7), 623–628. Rasmusson, D. X., Bylsma, F. W., & Brandt, J. (1995). Stability of perform-
Lacritz, L. H., Cullum, C. M., Weiner, M. F., & Rosenberg, R. N. (2001). ance on the Hopkins verbal learning test. Archives of Clinical
Comparison of the Hopkins verbal learning test-revised to the Neuropsychology, 10(1), 21–26.
California verbal learning test in Alzheimer’s disease. Applied Raymond, B. (1969). Short-term storage and long-term storage in free
Neuropsychology, 8(3), 180–184. recall. Journal of Verbal Learning and Verbal Behavior, 8(5), 567–574.
Lawton, M. P., & Brody, E. M. (1969). Assessment of older people: Self- Rey, A. (1964). L’examen clinique en psychologie. Paris: Presses
maintaining and instrumental activities of daily living. Gerontologist, Universitaire de France.
9(3), 179–186. Rieu, D., Bachoud-Lévi, A. C., Laurent, A., Jurion, E., & Dallabarba, G.
Malek-Ahmadi, M., Raj, A., & Small, B. J. (2011). Semantic clustering as a (2006). Adaptation française du «Hopkins Verbal Learning Test».
neuropsychological predictor for amnestic-MCI. Aging, Revue Neurologique, 162(6), 721–728.
Neuropsychology, and Cognition, 18(3), 280–292. Roberts, W. A. (1972). Free recall of word lists varying in length and rate
Marian, V., Bartolotti, J., Chabal, S., & Shook, A. (2012). CLEARPOND: of presentation: A test of total-time hypotheses. Journal of
Cross-linguistic easy-access resource for phonological and ortho- Experimental Psychology, 92, 365–372.
graphic neighborhood densities. PLoS ONE, 7(8), e43230. doi:10. Roodenrys, S., Hulme, C., Lethbridge, A., Hinton, M., & Nimmo, L. M.
1371/journal.pone.0043230 (2002). Word-frequency and phonological-neighborhood effects
Martin, A., Brouwers, P., Cox, C., & Fedio, P. (1985). On the nature of the on verbal short-term memory. Journal of Experimental Psychology:
verbal memory deficit in Alzheimer’s disease. Brain and Language, Learning, Memory, and Cognition, 28, 1019–1034.
25(2), 323–341. Rubin, D. C. (1980). 51 properties of 125 words: A unit analysis of verbal
Martin, A., & Fedio, P. (1983). Word production and comprehension in behavior. Journal of Verbal Learning and Verbal Behavior, 19(6), 736–
Alzheimer’s disease: The breakdown of semantic knowledge. Brain 755.
and Language, 19, 124–141. Schrijnemaekers, A. M. C., de Jager, C. A., Hogervorst, E., & Budge, M. M.
Massman, P. J., Delis, D. C., & Butters, N. (1993). Does impaired primacy (2006). Cases with mild cognitive impairment and Alzheimer’s
recall equal impaired long-term storage?: Serial position effects in disease fail to benefit from repeated exposure to episodic
Huntington’s disease and Alzheimer’s disease. Developmental memory tests as compared with controls. Journal of Clinical and
Neuropsychology, 9(1), 1–15. Experimental Neuropsychology, 28(3), 438–455.
Morris, P. E. (1981). Age of acquisition, imagery, recall, and the limit- Shapiro, A. M., Benedict, R. H. B., Schretlen, D., & Brandt, J. (1999).
ations of multiple-regression analysis. Memory & Cognition, 9(3), Construct and concurrent validity of the Hopkins Verbal Learning
277–282. Test – revised. The Clinical Neuropsychologist, 13(3), 348–358.
Morris, R. G. (1984). Dementia and the functioning of the articulatory Shi, J., Tian, J., Wei, M., Miao, Y., & Wang, Y. (2012). The utility of the
loop system. Cognitive Neuropsychology, 1(2), 143–157. Hopkins verbal learning test (Chinese version) for screening
Morrison, C. M., & Ellis, A. W. (1995). Roles of word frequency and age of dementia and mild cognitive impairment in a Chinese population.
acquisition in word naming and lexical decision. Journal of BMC Neurology, 12, 136.
Experimental Psychology: Learning, Memory, and Cognition, 21, Silveri, M. C., Cappa, A., Mariotti, P., & Puopolo, M. (2002). Naming in
116–133. patients with Alzheimer’s disease: Influence of age of acquisition
Moulin, C. J., James, N., Freeman, J. E., & Jones, R. W. (2004). and categorical effects. Journal of Clinical and Experimental
Deficient acquisition and consolidation: Intertrial free recall per- Neuropsychology, 24(6), 755–764.
formance in Alzheimer’s disease and mild cognitive impair- Sumby, W. H. (1963). Word frequency and serial position effects.
ment. Journal of Clinical and Experimental Neuropsychology, 26 Journal of Verbal Learning and Verbal Behavior, 1, 443–450.
(1), 1–10. Tierney, M. C., Nores, A., Snow, W. G., Fisher, R. H., Zorzitto, M. L., & Reid,
Murdock, B. B., Jr. (1962). The serial position effect of free recall. Journal D. W. (1994). Use of the Rey auditory verbal learning test in differ-
of Experimental Psychology, 64, 482–488. entiating normal aging from Alzheimer’s and Parkinson’s dementia.
Ober, B. A., Dronkers, N. F., Koss, E., Delis, D. C., & Friedland, R. P. (1986). Psychological Assessment, 6(2), 129–134.
Retrieval from semantic memory in Alzheimer-type Van Overschelde, J. P. (2002). The influence of word frequency on
dementia. Journal of Clinical and Experimental Neuropsychology, 8 recency effects in directed free recall. Journal of Experimental
(1), 75–92. Psychology: Learning, Memory, and Cognition, 28(4), 611.
MEMORY 403
Ward, G., Tan, L., & Grenfell-Essam, R. (2010). Examining the relation- memory in aging and early Alzheimer disease. Hippocampus, 21
ship between free recall and immediate serial recall: The effects (5), 461–466.
of list length and output order. Journal of Experimental Woodard, J. L., Dunlosky, J., & Salthouse, T. A. (1999). Task decompo-
Psychology: Learning, Memory, and Cognition, 36, 1207–1241. sition analysis of intertrial free recall performance on the Rey
Weingartner, H., Grafman, J., Boutelle, W., Kaye, W., & Martin, P. R. Auditory Verbal Learning Test in normal aging and Alzheimer’s
(1983). Forms of memory failure. Science, 221, 380–383. disease. Journal of Clinical and Experimental Neuropsychology, 21
Weingartner, H., Kaye, W., Smallberg, S. A., Ebert, M. H., Gillin, J. C., & (5), 666–676.
Sitaram, N. (1981). Memory failures in progressive idiopathic Woods, S. P., Scott, J. C., Conover, E., Marcotte, T. D., Heaton, R. K., Grant, I.,
dementia. Journal of Abnormal Psychology, 90(3), 187–196. & HIV Neurobehavioral Research Center (HNRC) Group. (2005). Test-
Welsh, K., Butters, N., Hughes, J., Mohs, R., & Heyman, A. (1991). retest reliability of component process variables within the Hopkins
Detection of abnormal memory decline in mild cases of Verbal Learning Test – revised. Assessment, 12(1), 96–100.
Alzheimer’s disease using CERAD neuropsychological measures. Xu, X., Rahardjo, T. B., Xiao, S., & Hogervorst, E. (2014). The Hopkins
Archives of Neurology, 48(3), 278–281. Verbal Learning Test and detection of MCI and mild dementia: A lit-
Wilson, R. S., Bacon, L. D., Fox, J. H., Kramer, R. L., & Kaszniak, A. W. erature review. Journal of Alzheimer Disease and Parkinsonism, 4,
(1983). Word frequency effect and recognition memory in dementia 166.
of the Alzheimer type. Journal of Clinical and Experimental Yesufu, A., Bandelow, S., Rahardjo, T. B., & Hogervorst, E. (2009).
Neuropsychology, 5(2), 97–104. Chapter 8: Dementia diagnoses. In Quentin Gariépy & Raphaël
Wolk, D. A., Dunfee, K. L., Dickerson, B. C., Aizenstein, H. J., & DeKosky, S. Ménard (Eds.), The handbook of cognitive aging: Causes, processes
T. (2011). A medial temporal lobe division of labor: Insights from and mechanisms. New York, NY: Nova Sciences. Invited submission.
Appendix 1. Full demographic information for the age, gender and educational level matched
participants in the Dementia and Control groups.
Age (in years)

Gender Education level Control cases Dementia cases
Female Primary school 81 85
Female Primary school 83 88
Female Elementary school (unfinished) 60 60
Female Elementary school 62 62
Female High school 74 75
Female No formal education 60 60
Male Elementary school (unfinished) 60 60
(Continued)
Continued.
Age (in years)
Gender Education level Control cases Dementia cases
Male Elementary school 70 70
Male No formal education 75 75
Note: participants on the same row were matched.
Appendix 2. Pairwise comparisons, with a Bonferroni correction, to follow up the significant main
effect of serial position for trial 1 (Figure 5, Panel A).
SP 2 3 4 5 6 7 8 9 10 11 12
1 0.004 1.000 <0.001 <0.001 <0.001 <0.001 0.015 1.000 0.006 1.000 0.241
2 0.133 1.000 1.000 1.000 0.065 1.000 1.000 1.000 0.021 1.000
3 <0.001 <0.001 <0.001 <0.001 0.459 1.000 0.099 1.000 1.000
4 1.000 1.000 1.000 1.000 0.004 1.000 <0.001 0.215
5 1.000 1.000 0.573 0.003 1.000 <0.001 0.163
6 1.000 0.671 0.014 1.000 <0.001 0.322
7 0.007 <0.001 0.036 <0.001 0.002
8 1.000 1.000 0.091 1.000
9 1.000 1.000 1.000
10 0.010 1.000
11 0.189
Note: The values show the significance level; significant values are indicated in bold.
effect of serial position for trial 2 (Figure 5, Panel B).
SP 2 3 4 5 6 7 8 9 10 11 12
1 0.338 1.000 <0.001 <0.001 <0.001 <0.001 0.238 0.747 1.000 1.000 1.000
2 0.164 0.153 0.227 1.000 <0.001 1.000 1.000 1.000 1.000 1.000
3 <0.001 <0.001 <0.001 <0.001 0.074 0.103 1.000 1.000 1.000
4 1.000 1.000 1.000 0.072 0.054 0.009 0.001 0.002
5 1.000 1.000 0.796 0.360 0.082 0.012 0.019
6 1.000 1.000 1.000 0.259 0.109 0.325
7 0.006 0.001 <0.001 <0.001 <0.001
8 1.000 1.000 1.000 1.000
9 1.000 1.000 1.000
10 1.000 1.000
11 1.000
MEMORY 405
effect of serial position for trial 3 (Figure 5, Panel C).
P 2 3 4 5 6 7 8 9 10 11 12
1 1.000 1.000 <0.001 0.015 0.702 <0.001 0.723 1.000 1.000 1.000 1.000
2 1.000 0.001 0.086 0.245 <0.001 0.859 0.881 1.000 1.000 1.000
3 <0.001 0.001 0.032 <0.001 0.078 0.043 1.000 1.000 1.000
4 1.000 1.000 1.000 1.000 1.000 0.146 0.007 0.005
5 1.000 1.000 1.000 1.000 1.000 0.241 0.117
6 0.059 1.000 1.000 1.000 1.000 1.000
7 0.054 0.022 0.002 <0.001 <0.001
8 1.000 1.000 1.000 1.000
9 1.000 1.000 1.000
10 1.000 1.000
11 1.000
effect of serial position for trials 1–3 (Figure 5, Panel D).
SP 2 3 4 5 6 7 8 9 10 11 12
1 0.007 1.000 <0.001 <0.001 <0.001 <0.001 0.001 0.005 0.044 1.000 0.103
2 0.030 0.010 0.082 0.407 <0.001 1.000 1.000 1.000 1.000 1.000
3 <0.001 <0.001 <0.001 <0.001 0.003 <0.001 0.125 1.000 0.286
4 1.000 1.000 1.000 0.069 0.121 0.012 <0.001 0.005
5 1.000 1.000 0.230 0.389 0.239 <0.001 0.048
6 0.146 1.000 1.000 0.715 0.006 0.106
7 <0.001 <0.001 <0.001 <0.001 <0.001
8 1.000 1.000 0.842 1.000
9 1.000 1.000 1.000
10 1.000 1.000
11 1.000

The Hopkins Verbal Learning Test An in Depth Analysis of Recall Patterns

Uploaded by

Copyright:

Available Formats

The Hopkins Verbal Learning Test An in Depth Analysis of Recall Patterns

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

The Hopkins Verbal Learning Test An in Depth Analysis of Recall Patterns

Uploaded by

Copyright:

Available Formats

What is the Hopkins Verbal Learning Test and what is it used for?

What is the Hopkins Verbal Learning Test and what is it used for?

What differences were found in recall patterns between dementia cases and controls?

What differences were found in recall patterns between dementia cases and controls?

Memory

ISSN: 0965-8211 (Print) 1464-0686 (Online) Journal homepage: https://www.tandfonline.com/loi/pmem20

The Hopkins Verbal Learning Test: an in-depth

Rachel Grenfell-Essam, Eef Hogervorst & Tri Budi W. Rahardjo

To link to this article: https://doi.org/10.1080/09658211.2017.1349804

Published online: 11 Jul 2017.

Submit your article to this journal

Article views: 310

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

The Hopkins Verbal Learning Test: an in-depth analysis of recall patterns

ABSTRACT ARTICLE HISTORY

differences in measurement but they appear to follow

correlation between the two languages for word length, r

ducted on each characteristic with category (3 levels:

million, F (2, 9) = 1.25, MSE = 1119.0, p = 0.331; word length,

F (2, 9) = 1.93, MSE = 0.389, p = 0.201; and orthographic

neighbourhood size values, F (2, 9) = 3.16, MSE = 49.4, p =

p = 0.002. Bonferroni corrected pairwise comparisons

showed that animals were acquired significantly earlier

than both stones (p = 0.001) and shelter (p = 0.038), but

correlations were not significant; word frequency and

word length, r (12) = −0.123, p = 0.703, word frequency

and age-of-acquisition, r (12) = −0.441, p = 0.151, word fre-

−0.020, p = 0.950, word length and age-of-acquisition, r

(12) = 0.484, p = 0.111, and age-of-acquisition and ortho-

graphic neighbourhood size, r (12) = −0.335, p = 0.287.

For the Indonesian language these ANOVAs exploring

category differences also revealed non-significant differ-

0.389, p = 0.274; and orthographic neighbourhood size

values, F (2, 9) = 0.525, MSE = 10.64, p = 0.609. Additionally

none of the characteristics significantly correlated with

orthographic neighbourhood size, r (12) = −0.552, p = 0.063.

HVLT total recall score

The HVLT is usually scored using a total recall score. This is

remembered words, across the three immediate trials.

Age (in years)

You might also like