Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
RESEARCH PAPER

Attention! A good bedside test for delirium?

Niamh A O’Regan,1 Daniel J Ryan,1 Eve Boland,2 Warren Connolly,2 Ciara McGlade,1
Maeve Leonard,3 Josie Clare,4 Joseph A Eustace,5 David Meagher,6,7
Suzanne Timmons1

▸ Additional material is ABSTRACT by the referring team.3 Detection rates are lower in
published online only. To view Background Routine delirium screening could improve older patients,4 those with premorbid dementia5
please visit the journal online
(http://dx.doi.org/10.1136/
delirium detection, but it remains unclear as to which and in hypoactive cases.6 7 Collins et al found recog-
jnnp-2013-307053). screening tool is most suitable. We tested the diagnostic nition rates to be as low as 28% in older medical
accuracy of the following screening methods (either inpatients8 and studies in the emergency department
For numbered affiliations see
end of article individually or in combination) in the detection of delirium: (ED) show similar rates of underdetection.9 10 The
MOTYB (months of the year backwards); SSF (Spatial Span reasons for poor recognition are multifactorial.
Correspondence to Forwards); evidence of subjective or objective ‘confusion’. ‘Confusion’ is commonly considered normal in
Dr Niamh O’Regan, Centre for Methods We performed a cross-sectional study of older patients, who are most at risk. The symptom
Gerontology and
Rehabilitation, School of general hospital adult inpatients in a large tertiary referral profile varies greatly from patient to patient, and the
Medicine, University College hospital. Screening tests were performed by junior medical prevailing stereotype of hyperactive delirium (‘delir-
Cork, St Finbarr’s Hospital, trainees. Subsequently, two independent formal delirium ium tremens’) is misleading, as delirium most com-
Douglas Road, Cork, Ireland; assessments were performed: first, the Confusion monly presents in its less obvious and more serious
[email protected]
Assessment Method (CAM) followed by the Delirium hypoactive form. Clinicians may also be deceived by
Received 18 October 2013 Rating Scale-Revised 98 (DRS-R98). DSM-IV (Diagnostic patients during periods of lucidity, as the symptoms
Revised 18 January 2014 and Statistical Manual of Mental Disorders, fourth edition) characteristically fluctuate over the course of the
Accepted 22 January 2014 criteria were used to assign delirium diagnosis. Sensitivity day. Studies have shown the importance of early rec-
Published Online First and specificity ratios with 95% CIs were calculated for ognition and intervention in reducing the severity
25 February 2014
each screening method. and duration of delirium.11 Although studies investi-

copyright.
Results 265 patients were included. The most precise gating impact of early intervention on long-term
screening method overall was achieved by simultaneously outcomes have been inconsistent, Gonzalez and col-
performing MOTYB and assessing for subjective/objective leagues found an 11% increase in mortality with
confusion (sensitivity 93.8%, 95% CI 82.8 to 98.6; every additional 48 h of delirium,12 and Kakuma
specificity 84.7%, 95% CI 79.2 to 89.2). In older et al13 reported a significant increase in mortality at
patients, MOTYB alone was most accurate, whereas in 6 months in older patients with undetected delirium
younger patients, a simultaneous combination of SSF (cut- discharged from the ED, compared to those with
off 4) with either MOTYB or assessment of subjective/ delirium who had been appropriately diagnosed.
objective confusion was best. In every case, addition of the Ideally, all patients at high risk of delirium
CAM as a second-line screening step to improve specificity should be assessed regularly using systematic appli-
resulted in considerable loss in sensitivity. cation of sensitive tools. However, formal delirium
Conclusions Our results suggest that simple attention diagnosis is based on thorough, and often lengthy,
tests may be useful in delirium screening. MOTYB used assessment by a trained and experienced clinician.
alone was the most accurate screening test in older Hence, a two-phase approach to detection is most
people. efficient: first, screening for key delirium features
using a simple, short test, followed by formal
assessment in those who screen positive. The recent
INTRODUCTION National Institute of Health and Care Excellence
Delirium is a serious neuropsychiatric condition (NICE) guidelines advocate this approach,14 and
which occurs in the setting of acute illness. It is ubi- recommend daily screening for all those at risk.
quitous in the acute hospital setting, having a point Currently, consensus is lacking as to which screen-
Open Access
Scan to access more
prevalence of almost 20%,1 with higher rates in ing method is best, but it is clear that test sensitivity
free content older patients. It is independently associated with must be emphasised over specificity to minimise the
adverse outcomes,2 including increased length of dangers of missed cases. The NICE guidelines
stay, increased mortality and accelerated cognitive screening approach is based on monitoring for a
and functional decline. list of specific delirium indicators, however, this
A major challenge in delirium care is that, despite method has yet to be validated and likely requires
its significance, delirium is commonly missed or mis- some understanding of delirium in order to ensure
taken for other conditions across treatment settings. accurate application. The most widely used screen-
In a point-prevalence study of delirium in 311 ing test is the Confusion Assessment Method
To cite: O’Regan NA,
Ryan DJ, Boland E, et al. patients across a general hospital, doctors missed (CAM)15 16 which has been validated in several lan-
J Neurol Neurosurg over half the delirium cases.1 In a study of general guages and settings.17 This tool, designed for delir-
Psychiatry 2014;85: hospital referrals to liaison psychiatry, Kishi et al ium diagnosis, was based on DSM-IIIR (Diagnostic
1122–1131. showed that 46% of delirium diagnoses were missed and Statistical Manual of Mental Disorders, third

1122 O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053

 Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
edition) criteria. It requires training to ensure accuracy18 and METHODS
lacks the brevity desired for routine general use. This study was part of a large point-prevalence study of delir-
The prevailing gold standard for delirium diagnosis at the ium, which was conducted at Cork University Hospital (CUH)
time of the study was DSM-IV19 criteria applied by an experi- on 15th May 2010. The details of this study’s methodology and
enced clinician following standardised testing. In order to be ethical procedures have been published elsewhere.1 All adult
diagnosed with DSM-IV delirium, a patient must present with inpatients were eligible for inclusion in the study, excluding
the following features: a disturbance of consciousness with those in the ED, intensive care unit and haematology/burns iso-
reduced ability to focus, shift or sustain attention; a change in lation unit. Patients were also excluded if they refused participa-
cognition or perception that is not explained by a pre-existing, tion, or were severely aphasic; comatose; dying; or considered
established or evolving dementia; acute onset and fluctuating too unwell for interview by nursing staff.
course; and evidence of an underlying general medical cause. The study involved three stages of assessment: attention
Hence, inattention is a core delirium feature, mandatory to testing/screening for subjective or objective ‘confusion’; CAM
DSM-IV criteria diagnosis (reflected also in the recently pub- assessment; and formal evaluation by experienced psychiatrists.
lished DSM-5 criteria).20 Attention is a basic component of cog- Trained junior medical staff first screened every patient for
nitive function, and can affect performance in many other inattention using the SSF and MOTYB. The SSF was performed
cognitive domains. It is affected in many disorders other than using an A5-sized piece of white card with eight red squares
delirium, including dementia, depression and developmental (each measuring 1.5 cm2) evenly spaced over three rows (config-
conditions, such as attention-deficit hyperactivity disorder. uration three, two, three; landscape; see online supplementary
Other factors which may impact negatively on measures of files). The investigators tapped out predetermined sequences for
attention are level of education, female sex and increasing the patients to replicate. The test began with a sequence of two
age,21–23 however reports are conflicting.22 24 25 Attention is squares and increased in number with each correct iteration, up
particularly affected in Dementia with Lewy Bodies (DLB), to a maximum sequence of seven. Two attempts were allowed at
which is phenomenologically and neurochemically more similar each level using different predetermined sequences. Patients
to delirium than other dementias.26 In Alzheimer’s dementia who were unable to correctly repeat a sequence of five were
(AD), complex attentional functions are affected early on in the considered to have failed the test. For the MOTYB, the patients
disease with performance on more basic attention tasks rela- were first requested to say the months of the year forward from
tively preserved until the advanced stages of the condition,27 January to December. They were then asked to recite the
however, it is thought that the deficits may be reflective of more months in reverse order from December back to January.
primary memory problems.28 Contrastingly, in delirium, the Patients were considered to have passed this test on reaching
attentional deficit is more global, more marked, and being a July without error. Hence, inattention was deemed present in

copyright.
mandatory feature of delirium, occurs with much higher fre- those who scored less than five on SSF, or were unable to cor-
quency than other cognitive deficits.29 Bedside tests to capture rectly recite the MOTYB as far as July. Additionally, the patient
inattention are simple and quick to perform, and include was screened for subjective confusion, and objective reports of
‘WORLD backwards’ and ‘serial 7s’ from Folstein’s Mini confusion (or proxy terms) by nursing staff interview and
Mental State Examination (MMSE),30 and reciting the months inspection of the medical notes. The patients were asked the fol-
of the year or the days of the week backwards.31 32 The former lowing question: ‘Have you felt muddled in your thinking, or
two tests from the MMSE are well recognised to be particularly confused, since you came into hospital?’ to determine the pres-
sensitive to educational level.33 Other examples, such as the ence of subjective confusion. The nurses were interviewed using
Digit Span Test, the Vigilance ‘A’ Test, and the Digit a standardised set of questions (available as an online supple-
Cancellation Test have been shown to aid delirium detec- mentary file and published elsewhere1), used to investigate the
tion.34 35 The visual Spatial Span Forwards (SSF), a pattern rec- presence or absence of delirium features. The medical notes
ognition test based on the digit span forwards,36 has recently were also searched for any documentation of delirium or the
shown to be of some use in identifying inattention in patients proxy terms ‘confusion’ or ‘agitation’ during the admission. The
with delirium versus those with dementia.37 These bedside tests patient was considered to have objective evidence of confusion
are somewhat observer-dependent, and may be affected by defi- if nursing staff responded positively to any of the nursing ques-
cits in other domains, such as visual or auditory processing tions relating to delirium, and/or if there was reference to delir-
speed and motor execution. The Edinburgh Delirium Test Box ium or proxy term in the medical notes.
is a device which was developed specifically to objectively Patients who failed at least one of the attention tests, or who
measure performance on tasks of sustained visual attention only. had subjective or objective reports of confusion, were then inde-
Using this device, Brown and colleagues showed that patients pendently assessed using the sensitive, short form of the CAM,
with delirium performed much more poorly on sustained atten- by a team of eight trained Geriatric Medicine registrars and con-
tion tasks than those with AD or healthy controls. Additionally, sultants,39 and deemed either CAM-positive or CAM-negative.
the device showed excellent accuracy for discriminating delirium These doctors had undergone 8 h of CAM training over a
from dementia and cognitively intact controls.38 Using devices 3-month period, including instruction, discussion, simulated
such as this, however, is cumbersome and, hence, we considered cases, interval online self-assessment and, finally, trainer-
simple bedside tests, which require minimal training, to be more observed CAM performance and scoring with feedback. Table 1
feasible for routine and repeated use in a busy clinical setting. illustrates how each item on the CAM was scored. Of note, the
Thus, the aim of our study was to determine if the simple SSF and MOTYB were repeated by the CAM assessors for the
bedside attention tests, Months of the Year Backwards purposes of scoring the CAM.
(MOTYB) and SSF, or reports of confusion (either subjective or Patients who underwent CAM then proceeded to be assessed
objective) were predictive of the presence of delirium. We also by a team of four experienced psychiatrists, with specific expert-
aimed to assess the usefulness of the CAM, as a second-line ise in delirium detection, using the Delirium Rating Scale–
screening step following initial testing. Revised ’98 (DRS-R98)40 (time to psychiatry assessment

O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053 1123

 Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
Table 1 Details of how each item on the CAM was scored by the CAM assessors
Example of how the CAM was performed

1. Introduction: “Hello, Mr…. My name is …. As you have been informed, a study is ongoing in the hospital today and earlier you agreed to participate. You have already
answered some questions for us. Are you happy for me to ask you some more questions as part of the same study? It should not take longer than five minutes.”
2. General assessment: General conversation questions to assess if any obvious distractibility or disorganised thinking (eg, how are you feeling today?; how long have you
been in hospital?; etc.)
3. Formal assessment: Testing of attention and thought process using methods described below.
4. Nursing questionnaire: Responses from screening nursing questionnaire viewed for information relating to temporal onset and fluctuations
CAM item How each item was scored
1A: Acute onset This was scored using answers from the standardised nursing interview.
1B: Fluctuating course The raters observed for evidence of fluctuations during the CAM interview. The standardised nursing interview was also used to
assess for presence of fluctuations.
2: Inattention The SSF and MOTYB were repeated by the CAM assessors. This item was positive if either one of the tests was failed on this
occasion, or if there was evidence of inattention or distractibility during the interview. The results from the initial screening attention
tests were not used.
3: Disorganised thinking The patient was asked the following questions:
▸ Can you tell me what this proverb means? ‘Every cloud has a silver lining’ (example)
▸ Abstract questions*:
1. Would a stone float on water?
2. Would two pounds of flour weigh more than one pound?
Disorganised thinking was considered present if the patient was unable to correctly interpret the proverb or answered either of
the abstract questions incorrectly, or if the patient demonstrated obvious evidence of disorganised thinking during the
interview.
4: Altered level of consciousness This item was considered positive if there was any evidence of drowsiness/hyperalertness during the interview.
*From the CAM-ICU.17
CAM, Confusion Assessment Method; MOTYB, Months of the Year backwards; SSF, Spatial Span Forwards.

<24 h). This is a well-validated,40–42 diagnostically precise tool Statistical analyses

which displays high inter-rater reliability, validity, sensitivity and Statistical analysis was performed using SPSS V.18. To compare

copyright.
specificity for differentiating delirium from mixed neuropsychi- baseline characteristics between patients with and without delir-
atric conditions including dementia and depression.40 42 43 This ium, Mann–Whitney U-test was used. Sensitivities and specifici-
assessment was performed completely independently of the two ties were calculated for each screening test/combination of tests
previous stages, and all items were scored according to the from 2×2 tables, with CIs testing significance at 95%. Each
DRS-R98 training manual.44The presence of delirium was screening method was assessed in this way initially in isolation,
ultimately determined according to DSM-IV criteria (reference and then subsequently in combination with other screening
standard). methods. Hence, we examined the performance of a variety of
test combinations. This included applying test combinations sim-
ultaneously or sequentially, and then with or without the CAM
as a further screening step. There are two potential simultan-
Assessment of previous cognitive status eous test combination scenarios: (1) screen positive if at least
In all patients with delirium, the medical case-notes were one test failed (figure 1A) and (2) screen positive if both tests
reviewed for a diagnosis of pre-existing dementia made by a failed (figure 1B). These two approaches were analysed separ-
suitably trained physician. Where this was not available, premor- ately. In sequential test combinations, the second test is only
bid cognition was determined by telephone interview using the applied in the setting of a failed first test (figure 1C).
Informant Questionnaire on Cognitive Decline in the Simultaneous testing in which a patient screens positive if either
Elderly-Short Form (IQCODE-SF), a validated screening tool or both screening methods are failed (scenario 1) will always
for detecting dementia.45 46 Patients without delirium who were have a higher net sensitivity than either sequential tests or simul-
less than 65 years of age were presumed not to have dementia taneous tests where both tests must be failed to screen positive
unless it was documented in the case notes. As dementia is more (scenario 2); however, there is a net loss in specificity.
prevalent and is known to be underdetected in older people, Sequential testing results in a net loss in sensitivity, but a net
depending on medical chart documentation for diagnosis was gain in specificity.48
likely to be highly undersensitive. Hence, a random sample of A modified version of the Forest Plot viewer programme was
40 older non-delirious patients also had baseline preadmission used to create graphical representation of the data.49 As advan-
cognition assessed using the IQCODE-SF. cing age and prior history of dementia have previously been
shown to be independent predictors of delirium in this cohort,1
we repeated all calculations in those older than and younger
Other data collected than the median age of 69 years, and in those with and without
Information relating to medication use was documented, and a previous history of dementia. In total, over 70 test combina-
laboratory results (sodium, glucose, thyroid stimulating tions were analysed.
hormone, calcium, urea, C-reactive protein, white cell count
and albumin level) were also collected. Current and previous RESULTS
alcohol history was recorded where available, and the Charlson There were 311 patients included in the delirium point-
comorbidity index47 was calculated. prevalence study and 280 had full data available, 55 of whom

1124 O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053

 Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
copyright.

Figure 1 Testing processes using different test combination models a) Example of simultaneous testing (a): Two screening tests administered
simultaneously. Further assessment required if either test failed. b) Example of simultaneous testing (b): Two screening tests administered
simultaneously. Further assessment required only if both tests failed c) Example of sequential testing: First screening test is performed. Proceed to
second screening test only if first test failed. Then proceed to further assessment only if second test is also failed. (MOTYB=months of the year
backwards test; SSF5=spatial span forwards test with a cutoff of 5).

were diagnosed with DSM-IV delirium (19.6%).1 For this ana- and/or had documentation of confusion in the medical notes.
lysis, 265 patients were included, 48 (18.1%) of whom had Of the included patients, 23 had no data pertaining to medical
DSM-IV delirium. We excluded those who had not completed documentation of delirium and four had missing data in relation
SSF and MOTYB testing (hence, excluding patients with severe to nursing opinion. Patient demographics for this cohort are
visual difficulties and aphasia), and those who had not had sub- outlined in table 2. The median age of the cohort was 69 years
jective and/or objective ‘confusion’ status recorded (see figure 2). (range 17–95) and 51.1% were men. Reason for admission was
Patients who were deemed to have objective evidence of confu- recorded in 84.9% (n=225) included patients. The most com-
sion are those who were reported as confused by nursing staff monly documented reasons for admission were, first, to

O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053 1125

 Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
MOTYB and proceeded to CAM assessment. Of these, 40
patients were subsequently diagnosed with DSM-IV delirium,
10 of whom were missed by the CAM, hence a sensitivity of
75%. Of the 20 patients without DSM-IV delirium, only three
had been CAM-positive, yielding a specificity of 85%.
Following assessment of each individual test, we analysed the
performance of a variety of test combinations, as described
earlier. Figure 3B illustrates the most efficient test combinations
for the prediction of delirium for the overall group (additional
data in online supplementary files). Using simultaneous
approach (scenario 1) appears most effective, in particular the
combination labelled ‘CONF/MOTYB’, which means that the
patient failed MOTYB, or was confused subjectively/objectively
(CONF). This combination had a sensitivity of 93.8% (95% CI
82.8 to 98.6) and a specificity of 84.7% (95% CI 79.2 to 89.2)
and was marginally more accurate than the simultaneous appli-
cation (scenario 1) of MOTYB and SSF4, sensitivity of 93.8%
(95% CI 82.8 to 98.6); specificity 81.1% (95% CI 75.2 to
86.1).
As mentioned earlier, analysis of test combinations was also
performed in older and younger patients. Figures representing
this data are available as supplementary material. The MOTYB
as a single test was best for those over the median age of
69 years, having high sensitivity and specificity for prediction of
delirium, 83.8% (95% CI 68 to 93.8) and 89.6% (95% CI 81.7
to 94.9). Contrastingly, for younger patients, many test options
Figure 2 Flow of patients through the study. were highly accurate, the most precise being a simultaneous
(scenario 1) combination of SSF4 and subjective/objective confu-
sion (sensitivity 100%, 95% CI 73.3 to 100; specificity 87.5%,
undergo procedure/surgery (n=39, 14.7%), followed by neuro- 95% CI 80.2 to 92.8) or SSF (cut-off 4) and MOTYB (sensitiv-

copyright.
logical causes (n=33, 12.5%), respiratory causes (n=31, ity 100%, 95% CI 71.3 to 100; specificity 86.8%, 95% CI 79.4
11.7%), cardiac causes (n=30, 11.3%) and malignancy/tumour to 92.2). In this younger subgroup, evidence of confusion alone,
(n=25, 9.4%). without the use of any attention test, was also highly predictive
Figure 3A illustrates the sensitivity, specificity and 95% CIs of delirium (sensitivity 90.9%, 95% CI 58.7 to 98.5; specificity
for each screening test individually, and additionally with a sub- 92.5%, 95% CI 86.2 to 96.5).
sequent CAM assessment, in the prediction of delirium for the Pre-existing cognitive status, as outlined above, was ascer-
whole group. The most accurate single test was MOTYB, with a tained in 194 patients. Four patients had a documented history
sensitivity of 83.3% (95% CI 69.8 to 92.5) and specificity of of dementia in the medical notes, all of whom had delirium. A
90.8% (95% CI 86.1 to 94.3). SSF5 (SSF using a cutoff of 5: ie, further 27 were considered to have pre-existing dementia based
a patient who can repeat a maximum sequence of four squares on an IQCODE-SF of ≥3.5. Prior cognitive impairment inde-
has inattention) was highly sensitive (91.7%, 95% CI 80 to pendently predicted a diagnosis of delirium (adjusted OR 15.3;
97.6) but lacked specificity (69.12%, 95% CI 62.5 to 75.2). CI 5.2 to 45.4, p<0.001).1 In patients with dementia, MOTYB
However, SSF4 (using a cutoff of 4) missed almost one-quarter was again the most sensitive single test (87.5%, 95% CI 67.6 to
of delirium cases (sensitivity 77.1%, 95% CI 62.7 to 87.9). 97.2), however the specificity was low and did not reach statis-
Figure 3A also illustrates that although the addition of the CAM tical significance (71.4%, 95% CI 29.3 to 95.5). In patients
as a second-line screening step predictably increased net specifi- without dementia, three screening combinations had almost
city of each method, it led to considerable net loss in sensitivity. equivalent accuracy. Using MOTYB and SSF4, scenario (a),
For example, of 265 included patients, 60 patients failed the picked up 87.5% (95% CI 67.6 to 97.2) of delirium cases, and
was 86.3% (95% CI 79.5 to 91.6) specific. Very similar results
were achieved using either one of these tests simultaneously
with seeking for evidence of ‘confusion’ (scenario 1). Table 3
Table 2 Patient demographics displays a summary of the most diagnostically accurate
No approaches.
Total Delirium delirium
(n=265) (n=48) (n=217) Sig. DISCUSSION
Delirium is vastly underdetected, a factor which contributes
Age (years), median (IQR) 69 (27) 78 (15.25) 66 (29.5) p<0.001*
greatly to its long-term personal, social and economic burden.
Sex (% male) 51.1 52.1 50.7 p=0.872†
Most clinicians do not routinely screen for delirium in practice,
Dementia status (n=194)
possibly in part due to an underappreciation of its impact, but
Dementia, n (%)‡ 31 (16.0) 24 (50.0) 7 (4.8) p<0.001‡
also due to a paucity of brief screening assessments.9 50 Most
*Independent Samples Mann–Whitney U test. of the currently employed assessment techniques take longer
†Fisher’s Exact test.
‡Number of patients with dementia in each group is presented as a percentage of than 5 min to perform,51 for example, the CAM, which in
total number of patients in each group in whom dementia status was known: total addition, requires training for accurate use.18 Recently, Han
(n=194); delirium (n=48); no delirium (n=146).
et al found that using a two-step screening process based on a

1126 O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053

 Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
copyright.
Figure 3 Forest plots depicting performance of tests individually and in combination: sensitivity and specificity plotted with 95% confidence
intervals 3a) Sensitivity of each individual test, with and without the CAM as a second–line test 3b) Specificity of each individual test, with and
without the CAM as a second–line test 3c) Most efficient test combinations, sensitivity 3d) Most efficient test combinations, specificity
(MOTYB=Months of the year backwards; SSF5=SSF with cutoff of 5; SSF4= SSF with cutoff of 4; Pt pos=Subjective confusion ( patient felt
subjectively confused when questioned); Nurse pos=nurse thought patient was confused when questioned; Med pos=‘confusion’ or proxy term
documented in the patient’s medical notes; Nurse or med pos=Objective confusion (either nurse felt patient was confused or ‘confusion’ or proxy
term was documented in the medical notes); Nurse/med pos=objective confusion (by nurse report and/or medical documentation); CONF/MOTYB
pos=any evidence of confusion and/or MOTYB failed; CONF/SSF5 pos=any evidence of confusion and/or SSF failed with a cutoff of 5; CONF/SSF4
pos=Any evidence of confusion and/or SSF failed with a cutoff of 4; MOTYB/SSF5 pos=MOTYB failed and/or SSF5 failed with a cutoff of 5; MOTYB/
SSF4 pos=MOTYB failed and/or SSF failed with a cutoff of 4]

brief operationalised version of the CAM (b-CAM) in the ED, examined the predictive potential of each screening method in
had a sensitivity of 70.0–84.0% and a specificity of 95.8– isolation, followed by various screening combinations.
97.2% when tested against DSM-IV diagnosis.52 The authors Additionally, for each method, we assessed whether or not
describe the b-CAM taking <1 min to perform, however, given using the CAM as a second-line screening step improved accur-
that collateral history is required to assess temporal onset and acy. We also investigated if tailoring the approach depending
fluctuations, it is likely that the process could often be more on patient age, or prior cognitive status could improve
lengthy. In a similar vein, in this report, we describe the per- efficiency.
formance of a number of screening methods in the prediction The best overall individual screening test was MOTYB, with a
of DSM-IV delirium across a large general hospital. We sensitivity of 83.3% and a specificity of 90.8% for the entire
assessed three attention tests (MOTYB, SSF5, SSF4), as well as group. The simultaneous addition of another screening method
staff and patient’s own impression of cognitive status. We (either SSF4 or evidence of subjective/objective confusion),

O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053 1127

 Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
copyright.
Figure 3 (Continued)

where any result being abnormal denoted a positive screen, (nursing staff missed one-third of cases, and medical staff failed
increased net sensitivity to over 90%. Subanalysis in patients to recognise one-half).1 These results are also consistent with
over and under the median age of 69 years, and in patients with previous work which illustrated that lower scores on the SSF are
and without dementia, gave similar results. In all subgroups, highly predictive of delirium, but higher scores are less useful in
MOTYB performed well as an individual screening test, and in outruling its presence.37 Our study shows that the SSF appears
older patients or those with dementia, this approach was best. to be particularly useful as a single test in patients with no prior
In younger patients or those without dementia, using MOTYB history of dementia, when a cut-off of 5 yields a sensitivity of
as part of a two-pronged screening approach, rather than alone, 91.7% and specificity of 75.5%.
increased sensitivity without compromising specificity. It is also As mentioned earlier, deficits in sustained attention occur in
interesting to note that in younger patients, any evidence of the late stages of AD, but performance on more complex atten-
confusion was highly predictive of delirium. This implies that tional functions is impaired earlier in the disease.27 Some
formal attention testing may be less crucial in this group, as any studies have used the MOTYB to distinguish stages of AD,53 54
evidence of confusion raises a strong suspicion of delirium. The including one small study which showed that the predictive
SSF is a quick, simple attention test, which can be used in value of the MMSE for dementia could be augmented by the
patients with expressive language difficulties. Using a cutoff of addition of the MOTYB.53 In a study of multiple cognitive
5, it is a highly sensitive test for the presence of delirium, domains in patients with AD and fronto-temporal dementia
however, specificity is low at 69%. When the cutoff is lowered (FTD), performance on Digit Span Forwards and Backwards
to 4, almost one-quarter of patients with delirium are missed. and on Vigilance ‘A’ and ‘B’ tests was impaired to varying
Nonetheless, this remains more accurate than staff detection degrees in typical AD and FTD when compared with normal

1128 O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053

 Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
Table 3 Most accurate screening methods from our study, overall group and subgroups based on age and cognitive status. Our preferred
screening approaches highlighted in bold
Sensitivity* Specificity*
Screening method (95% CI) (95% CI)

General hospital inpatients (n=265) Single test MOTYB 83.3% (69.8–92.5) 90.8% (86.1–94.3)
Simultaneous tests MOTYB/evidence of confusion (either positive = positive) 93.8% (82.8–98.6) 84.7% (79.2–89.2)
MOTYB/SSF4 (either failed = positive) 93.8% (82.8–98.6) 81.1% (75.2–86.1)
Older inpatients, ≥69 years (n=133) Single test MOTYB 83.8% (68–93.8) 89.6% (81.7–94.9)
Younger inpatients, ≤ 69 years (n=132) Single test Evidence of confusion 90.9% (58.7–98.5) 92.5% (86.2–96.5)
Simultaneous tests SSF4/evidence of confusion (either positive = positive) 100% (73.3–100) 87.5% (80.2–92.8)
SSF4/MOTYB (either positive = positive) 100% (73.3–100) 86.8% (79.4–92.2)
Patients with known dementia (n=31) Single test MOTYB 87.5% (67.6–97.2) 71.4% (29.3–95.5)
Patients with no history of dementia (n=154) Simultaneous tests MOTYB/SSF4 (either positive = positive) 87.5% (67.6–97.2) 86.3% (79.5–91.6)
*Sensitivities and specificities with 95% CIs based only on results from our study.
MOTYB, months of the year backwards test; ssf4, spatial span forwards with a cut-off of 4.

controls, whereas patients with amnestic AD were less different screening approaches depending on prior cognitive
affected.55 Interestingly, studies using more objective compu- status may not be as easily applicable as, for example, varying
terised attention tests have found that AD patients do not dem- the approach based on age, and supports the use of more versa-
onstrate impairments in focusing56 or sustained attention,38 tile tools, such as MOTYB.
whereas delirious patients are significantly impaired in this Our study has some limitations. Ideally, formal delirium
domain.38 testing should have been performed on all included patients,
In our study, although the SSF missed very few delirium cases however, due to the time-consuming nature of thorough delir-
in patients with dementia, even with a cut-off of 4, its specificity ium assessment, it would not have been feasible to perform 265
was below 50%. This indicates that the test is sensitive to cogni- such assessments over the course of 1 day. Hence, only those
tive impairment in general, and test failure may, in fact, be who had an indication of possible delirium, as detailed earlier,
reflective of attentional deficits related to underlying dementia were formally assessed. It is possible that a patient with delirium
rather than delirium. MOTYB seemed more accurate at predict- may have passed both attention tests during a lucid interval, and

copyright.
ing delirium in the dementia group, however, results did not had no other suggestion of possible delirium (ie, no subjective
reach statistical significance, possibly due to small patient confusion or recollection of confusion during the lucid period,
number (total n=31). Additionally, the low number of dementia and no staff reports of potential delirium features). This is an
patients who did not have delirium was very small (n=7) and, unlikely scenario, thus, it can be assumed that those patients
hence, interpretation of specificity in this subgroup is difficult. without formal delirium assessment did not have delirium. To
In practice, it is often challenging to ascertain premorbid cogni- manage time constraints and to ensure feasibility, we were
tion in patients who are acutely unwell, and who may have asso- unable to examine all existing bedside attention tests. Some
ciated delirium or subsyndromal delirium, especially without a other well-known tests of attention were not tested, for
readily available, accurate collateral history. Hence, using example, spelling the word WORLD backwards; subtracting

Figure 4 A suggested approach to delirium screening in the acute hospital setting (MOTYB;months of the year backwards test; SSF4;spatial span
forwards test with a cutoff of 4).

O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053 1129

 Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
6
serial sevens from 100; and counting backwards from 20 to 1. Department of Psychiatry, University of Limerick, Limerick, Ireland
7
The former two, taken from the MMSE (Mini-Mental State Cognitive Impairment Research Group, Centre for Interventions in Infection,
Inflammation & Immunity (4i), Graduate Entry Medical School, University of Limerick,
Examination),30 rely heavily on level of education and mathem- Limerick, Ireland
atical ability57 and, hence, were thought not to be suitable in
this cohort. Counting backwards from 20 to 1 is taken from the Acknowledgements We wish to thank the nursing and junior medical staff of
Abbreviated Mental Test Score developed by Hodkinson.58 This Cork University Hospital who assisted with this study.
test seemed too straightforward for inclusion and likely to have Contributors All authors meet ICJME criteria for authorship, and had full access to
a considerable ceiling effect. Therefore, it seemed more logical the data: All authors have substantial contributions to conception and design,
to us to use a more challenging test, such as the SSF. As men- acquisition of data, or analysis and interpretation of data; drafting the article or
revising it critically for important intellectual content; and final approval of the
tioned above, due to feasibility, we did not use objective compu- version to be published. NAO, DJR, JC, DM and ST: all substantially contributed to
terised tests of attention. study conception and design, data collection, statistical analysis and interpretation,
In all test scenarios, the addition of the CAM as a second-line drafting the initial manuscript, and subsequent critical revision of the manuscript
screening step, before proceeding to full delirium assessment, and final approval. JAE: assisted with statistical analysis, contributed to drafting the
original manuscript, performed critical revisions and has approved the final
resulted in reduced net sensitivity. Other studies have shown manuscript. EB, WC, CMG and ML: all majorly involved in data acquisition, critical
low CAM sensitivity when used by inexperienced and minimally revisions and final approval of the manuscript. The lead author, Dr Niamh O’Regan,
trained raters.18 In this study, all those performing CAM ratings affirms that the manuscript is an honest, accurate and transparent account of the
underwent rigorous training in its use, based on the CAM train- study, and that no important aspects or discrepancies have been omitted.
ing manual.39 One of the cornerstones of CAM assessment is Competing interests All authors have completed the ICJME uniform disclosure
accurate, dependable collateral history and, in this study, the form at http://www.icjme.org/coi_disclosure.pdf and declare: no support from any
temporal nature and evidence of fluctuations for the purposes organisation for the submitted work; ST has received educational grants and
honoraria from Janssen, Pfizer, Sanofi-Aventis, UCB, Astra-Zeneca, Orion, Lundbeck,
of CAM scoring was based mainly on collateral history from Novartis, Boehinger, MSD pharmaceutical companies; no other relationships or
on-duty nursing staff. Poor awareness of delirium features activities that could appear to have influenced the submitted work.
among staff, or a lack of emphasis during staff handover, can Ethics approval This study was granted ethical approval by Cork Research Ethics
hinder attempts to ascertain their presence or absence during Committee, Cork, Ireland.
CAM assessment and make it extremely challenging to pinpoint Provenance and peer review Not commissioned; externally peer reviewed.
acuity of onset or degree of fluctuations. We believe that this
Data sharing statement No additional data is available
issue was a major contributor to the reduced sensitivity of the
CAM in this study. The difficulty in obtaining accurate collateral Open Access This is an Open Access article distributed in accordance with the
Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which
history is a well-recognised barrier to delirium recognition in permits others to distribute, remix, adapt, build upon this work non-commercially,
practice.

copyright.
and license their derivative works on different terms, provided the original work is
Our study involved a single sample of hospital inpatients, and properly cited and the use is non-commercial. See: http://creativecommons.org/
further studies are required to extrapolate our findings to other licenses/by-nc/3.0/
samples and populations. Nonetheless, we conclude that simple
tests of attention may provide an efficient method of screening
for delirium in the acute hospital setting. Our suggested REFERENCES
approach towards screening for each patient subgroup is illu- 1 Ryan DJ, O’Regan N, Caoimh RO, et al. Delirium in an adult acute hospital
strated in figure 4. It is well established that the hospitalised population: predictors, prevalence and detection. BMJ Open 2013;3:e001772.
elderly are very delirium prone, and our study suggests that 2 Trzepacz P, Meagher D, Leonard M. Delirium. In: Levenson J. ed. Textbook of
Psychosomatic Medicine. 2nd edn. Washington, DC: American Psychiatric Publishing
using MOTYB may be particularly useful for screening in this
Press, 2010. Chapter 5, 71–114.
population. In younger people at risk of delirium, using a com- 3 Kishi Y, Kato M, Okuyama T, et al. Delirium: patient characteristics that predict a
bination of MOTYB and SSF4, where failing either test is a missed diagnosis at psychiatric consultation. Gen Hosp Psychiatry 2007;29:442–5.
positive screen, is more sensitive. The SSF, being a non-verbal 4 Cole MG. Delirium in elderly patients. Am J Geriatr Psychiatry 2004;12:7–21.
attention test, has greater coverage in populations with speech 5 Inouye SK, Foreman MD, Mion LC, et al. Nurses’ recognition of delirium and its
symptoms: comparison of nurse and researcher ratings. Arch Intern Med
or communication deficits, for example, stroke patients with 2001;161:2467–73.
dysphasia (also at high risk of delirium). In patients with known 6 Fang CK, Chen HW, Liu SI, et al. Prevalence, detection and treatment of delirium in
cognitive impairment, screening using the methods outlined terminal cancer inpatients: a prospective survey. Jpn J Clin Oncol 2008;38:56–63.
may not be as effective, but this group is especially at risk (77% 7 Voyer P, McCusker J, Cole MG, et al. Factors associated with delirium severity
among older patients. J Clin Nurs 2007;16:819–31.
in this study, but up to 89% in others59) and, hence, should
8 Collins N, Blanchard MR, Tookman A, et al. Detection of delirium in the acute
undergo regular formal delirium testing during hospitalisation. hospital. Age Ageing 2010;39:131–5.
In conclusion, we have described a novel, brief and straightfor- 9 Han JH, Zimmerman EE, Cutler N, et al. Delirium in older emergency department
ward screening approach for delirium detection across the patients: recognition, risk factors, and psychomotor subtypes. Acad Emerg Med
general hospital. Our method requires minimal training and can 2009;16:193–200.
10 Elie M, Rousseau F, Cole M, et al. Prevalence and detection of delirium in elderly
be used by all healthcare professionals, including those with a emergency department patients. CMAJ 2000;163:977–81.
limited understanding of delirium features. This brief approach 11 Milisen K, Lemiengre J, Braes T, et al. Multicomponent intervention strategies for
to screening has the potential to greatly increase delirium recog- managing delirium in hospitalized older people: systematic review. J Adv Nurs
nition across the hospital, however, further prospective studies 2005;52:79–90.
12 Gonzalez M, Martinez G, Calderon J, et al. Impact of delirium on short-term
are needed to confirm our findings.
mortality in elderly inpatients: a prospective cohort study. Psychosomatics
2009;50:234–8.
Author affiliations 13 Kakuma R, du Fort GG, Arsenault L, et al. Delirium in older emergency department
1
Centre for Gerontology and Rehabilitation, School of Medicine, University College patients discharged home: effect on survival. J Am Geriatr Soc 2003;51:443–50.
Cork, Cork, Ireland 14 Young J, Murthy L, Westby M, et al. Diagnosis, prevention, and management of
2
Cork University Hospital, Cork, Ireland delirium: summary of NICE guidance. BMJ 2010;341:c3704.
3
Department of Psychiatry, University of Limerick, Limerick, Ireland 15 Inouye SK, van Dyck CH, Alessi CA, et al. Clarifying confusion: the confusion
4
Department of Geriatric Medicine, Waterford Regional Hospital, Waterford, Ireland assessment method. A new method for detection of delirium. Ann Intern Med
5
HRB Clinical Research Facility at UCC, University College Cork, Cork, Ireland 1990;113:941–8.

1130 O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053

 Neuropsychiatry

J Neurol Neurosurg Psychiatry: first published as 10.1136/jnnp-2013-307053 on 25 February 2014. Downloaded from http://jnnp.bmj.com/ on September 12, 2022 by guest. Protected by
16 Wei LA, Fearing MA, Sternberg EJ, et al. The Confusion Assessment Method: a 39 Inouye SK. The confusion assessment (CAM): training manual and coding guide.
systematic review of current usage. J Am Geriatr Soc 2008;56:823–30. New Haven, CT: Yale University School of Medicine, 1991.
17 Ely EW, Margolin R, Francis J, et al. Evaluation of delirium in critically ill patients: 40 Trzepacz PT, Mittal D, Torres R, et al. Validation of the Delirium Rating
validation of the Confusion Assessment Method for the Intensive Care Unit Scale-revised-98: comparison with the delirium rating scale and the cognitive test
(CAM-ICU). Crit Care Med 2001;29:1370–9. for delirium. J Neuropsychiatry Clin Neurosci 2001;13:229–42.
18 Ryan K, Leonard M, Guerin S, et al. Validation of the confusion assessment method 41 Trzepacz PT, Dew MA. Further analyses of the delirium rating scale. Gen Hosp
in the palliative care setting. Palliat Med 2009;23:40–5. Psychiatry 1995;17:75–9.
19 American Psychiatric Association. Diagnostic and statistical manual of mental 42 Adamis D, Treloar A, MacDonald AJ, et al. Concurrent validity of two instruments
disorders—text revision. 4th edn. Washington, DC: American Psychiatric Publishing (the Confusion Assessment Method and the Delirium Rating Scale) in the detection
Inc, 1994. of delirium among older medical inpatients. Age Ageing 2005;34:72–5.
20 American Psychiatric Association. Diagnostic and statistical manual of mental 43 Trzepacz PT, Mulsant BH, Dew MA, et al. Is delirium different when it occurs in dementia?
disorders. 5th edn. Arlinglton, VA: American Psychiatric Publishing Inc, 2013. A study using the delirium rating scale. J Neuropsychiatry Clin Neurosci 1998;
21 Tun PA, Lachman ME. Age differences in reaction time and attention in a national 10:199–204.
telephone sample of adults: education, sex, and task complexity matter. Dev 44 Trzepacz PT, Maldonado JR, Kean J, et al. Delirium Rating Scale-Revised-98
Psychol 2008;44:1421–9. (DRS-R98) Administration Manual. A guide to increase understanding of how to
22 Gomez-Perez E, Ostrosky-Solis F. Attention and memory evaluation across the life solicit delirium symptoms when administering the DRS-R98., 2010.
span: heterogeneous effects of age and education. J Clin Exp Neuropsychol 45 Jorm AF. A short form of the Informant Questionnaire on Cognitive Decline in
2006;28:477–94. the Elderly (IQCODE): development and cross-validation. Psychol Med 1994;24:
23 Mazaux JM, Dartigues JF, Letenneur L, et al. Visuo-spatial attention and 145–53.
psychomotor performance in elderly community residents: effects of age, gender, 46 Jorm AF. The Informant Questionnaire on cognitive decline in the elderly (IQCODE):
and education. J Clin Exp Neuropsychol 1995;17:71–81. a review. Int Psychogeriatr 2004;16:275–93.
24 Van Gerven PW, Meijer WA, Jolles J. Education does not protect against age-related 47 Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic
decline of switching focal attention in working memory. Brain Cogn 2007;64:158–63. comorbidity in longitudinal studies: development and validation. J Chronic Dis
25 Chan RC. A further study on the sustained attention response to task (SART): the 1987;40:373–83.
effect of age, gender and education. Brain Inj 2001;15:819–29. 48 Gordis L. Epidemiology. 3rd edn. Philadelphia, PA: Elsevier, 2004:71–94.
26 Metzler-Baddeley C. A review of cognitive impairments in dementia with Lewy 49 Boyles AL, Harris SF, Rooney AA, et al. Forest Plot Viewer: a new graphing tool.
bodies relative to Alzheimer’s disease and Parkinson’s disease with dementia. Epidemiology 2011;22:746–7.
Cortex 2007;43:583–600. 50 Press Y, Margulin T, Grinshpun Y, et al. The diagnosis of delirium among elderly
27 Kolanowski AM, Fick DM, Yevchak AM, et al. Pay attention! The critical importance of patients presenting to the emergency department of an acute hospital. Arch
assessing attention in older adults with dementia. J Gerontol Nurs 2012;38:23–7. Gerontol Geriatr 2009;48:201–4.
28 Exton C, Leonard M. Eye tracking technology: a fresh approach in delirium 51 Wong CL, Holroyd-Leduc J, Simel DL, et al. Does this patient have delirium?: value
assessment? Int Rev Psychiatry 2009;21:8–14. of bedside instruments. JAMA 2010;304:779–86.
29 Meagher DJ, Moran M, Raju B, et al. Phenomenology of delirium. Assessment of 52 Han JH, Wilson A, Vasilevskis EE, et al. Diagnosing delirium in older emergency
100 adult cases using standardised measures. Br J Psychiatry 2007;190:135–41. department patients: validity and reliability of the delirium triage screen and the
30 Folstein MF, Robins LN, Helzer JE. The mini-mental state examination. Arch Gen brief confusion assessment method. Ann Emerg Med 2013;62:457–65.
Psychiatry 1983;40:812. 53 Tardiff M, Roy M, Bouchard R, et al. Months backwards test as a reliable predictor

copyright.
31 Katzman R, Brown T, Fuld P, et al. Validation of a short Orientation-Memory-Concentration of cognitive decline in mild Alzheimer’s disease. Alzheimers Demen 2013;9(4
Test of cognitive impairment. Am J Psychiatry 1983;140:734–9. supplement):P741–P42.
32 Hall RJ, Meagher DJ, MacLullich AM. Delirium detection and monitoring outside the 54 Ostberg P, Fernaeus SE, Bogdanovic N, et al. Word sequence production in
ICU. Best Pract Res Clin Anaesthesiol 2012;26:367–83. cognitive decline: forward ever, backward never. Logoped Phoniatr Vocol 2008;33:
33 Rosselli M, Tappen R, Williams C, et al. The relation of education and gender on 126–35.
the attention items of the Mini-Mental State Examination in Spanish speaking 55 Stopford CL, Thompson JC, Neary D, et al. Working memory, attention, and
Hispanic elders. Arch Clin Neuropsychol 2006;21:677–86. executive function in Alzheimer’s disease and frontotemporal dementia. Cortex
34 Pompei P, Foreman M, Cassel CK, et al. Detecting delirium among hospitalized 2012;48:429–46.
older patients. Arch Intern Med 1995;155:301–7. 56 Ishizaki J, Meguro K, Nara N, et al. Impaired shifting of visuospatial attention in
35 O’Keeffe ST, Gosney MA. Assessing attentiveness in older hospital patients: global Alzheimer’s disease as shown by the covert orienting paradigm: implications for
assessment versus tests of attention. J Am Geriatr Soc 1997;45:470–3. visual construction disability. Behav Neurol 2013;26:121–9.
36 Wilde NJ, Strauss E, Tulsky DS. Memory span on the Wechsler Scales. J Clin Exp 57 Ganguli M, Ratcliff G, Huff FJ, et al. Serial sevens versus world backwards: a
Neuropsychol 2004;26:539–49. comparison of the two measures of attention from the MMSE. J Geriatr Psychiatry
37 Meagher DJ, Leonard M, Donnelly S, et al. A comparison of neuropsychiatric and Neurol 1990;3:203–7.
cognitive profiles in delirium, dementia, comorbid delirium-dementia and cognitively 58 Hodkinson HM. Evaluation of a mental test score for assessment of mental
intact controls. J Neurol Neurosurg Psychiatry 2010;81:876–81. impairment in the elderly. Age Ageing 1972;1:233–8.
38 Brown LJ, Fordyce C, Zaghdani H, et al. Detecting deficits of sustained visual 59 Fick DM, Agostini JV, Inouye SK. Delirium superimposed on dementia: a systematic
attention in delirium. J Neurol Neurosurg Psychiatry 2011;82:1334–40. review. J Am Geriatr Soc 2002;50:1723–32.

O’Regan NA, et al. J Neurol Neurosurg Psychiatry 2014;85:1122–1131. doi:10.1136/jnnp-2013-307053 1131