Intensive and Critical Care Nursing (2005) 21, 284—301

ORIGINAL ARTICLE

A human factors engineering conceptual

framework of nursing workload and patient safety
in intensive care units
Pascale Carayon ∗, Ayşe P. Gürses

Systems Engineering Initiative for Patient Safety, Center for Quality and Productivity Improvement and
Department of Industrial and Systems Engineering, University of Wisconsin-Madison, 610 Walnut Street
575 WARF, Madison, WI 53726, USA

Received 26 August 2004 ; received in revised form 10 December 2004; accepted 17 December 2004

KEYWORDS Summary In this paper, we review the literature on nursing workload in inten-
Performance obstacles; sive care units (ICUs) and its impact on patient safety and quality of working life of
Nursing workload; nurses. We then propose a conceptual framework of ICU nursing workload that de-
Quality of working life; fines causes, consequences and outcomes of workload. We identified four levels of
Patient safety;
nursing workload (ICU/unit level, job level, patient level, and situation level), and
discuss measures associated with each of the four levels. A micro-level approach to
Intensive care units
ICU nursing workload at the situation level is proposed and recommended in order
to reduce workload and mitigate its negative impact. Performance obstacles are
conceptualized as causes of ICU nursing workload at the situation level.
© 2005 Elsevier Ltd. All rights reserved.

Introduction by over 250% and now accounts for approximately

10% of all hospital beds (American Hospital Associ-
Intensive care constitutes a significant portion of ation, 1995; Lustbader and Fein, 2000). This per-
the American health care system. The number of centage is expected to increase as the population
intensive care unit (ICUs) in the US is approximately ages (Groeger et al., 1992). It has been estimated
6000 (Angus et al., 2000). Every day, 55,000 pa- that ICUs account for between 20 and 34% of acute
tients are cared for in these 6000 ICUs (Halpern care hospitals in the US; this is double that of
et al., 1994). With a declining number of overall other developed countries (Office of Technology As-
acute care hospital beds over the past 20 years, sessment, 1984; Jacobs and Noseworthy, 1990; So-
the number of intensive care beds has increased ciety of Critical Care Medicine, 1992; Cohen and
Chalfin, 1994; Chalfin et al., 1995). The cumula-
∗ Corresponding author. Tel.: +1 608 2650503;
tive effect of this costly resource results in 1—1.5%
fax: +1 608 2631425.
of the gross national product devoted to intensive
E-mail addresses: [email protected] (P. Carayon), care medicine (Wagner et al., 1983). According to
[email protected] (A.P. Gürses). Miranda et al. (2003), nursing staff budget consti-

0964-3397/$ — see front matter © 2005 Elsevier Ltd. All rights reserved.
doi:10.1016/j.iccn.2004.12.003
Human factors engineering conceptual framework 285

tutes approximately 50% of the total costs in an ing workload, patient safety, quality of working life,
ICU. job stressors, ICU nurse, intensive care unit, and
Every year, approximately 400,000 to 500,000 nurse/patient ratio. Based on the literature review,
patients die in ICUs in the US (Angus et al., 1996). we then propose a conceptualization of nursing
Morbidity and mortality rates in ICUs vary widely workload that includes various levels and measures.
between institutions (Pronovost et al., 1999). Based Finally, we discuss possible causes of nursing work-
on data from England, Wales, and Northern Ireland, load in ICUs (performance obstacles) that can be
the overall hospital mortality after adult intensive identified by studying workload at the situation
care was estimated as 30.8%, ranging from 17.7 to level.
48.7% across hospitals (Intensive Care National Au-
dit and Research Center, 2001). Although some of
this variation is due to case mix (i.e. age, acute Patient safety in ICUs
severity, co morbidity, surgical status and reason for
admission), a significant variation in hospital mor- Research on patient safety and medical errors
tality is observed even after adjusting for differ- shows that errors and the adverse events that
ences in case mix (Knaus et al., 1986; Rowan et al., may result from those errors are frequent in ICUs
1993; Carmel and Rowan, 2001). This wide variation (Abramson et al., 1980; Merill and Boisaubin, 1981;
in the organization and delivery of intensive care Donchin et al., 1995). Donchin et al. (1995) con-
makes it important to understand the impact of or- ducted a study to evaluate human errors in a
ganizational factors on ICU outcomes. One cause of medical—surgical ICU by using self-reports and di-
this variation may be related to the ‘micro’ system rect observations. They found that an estimated
of the ICUs (Nelson et al., 2002). Nursing workload number of 1.7 errors per patient per day occur in
is one important aspect of the design of a clinical ICUs. In this study, a human error was defined as a
micro system. deviation from standard conduct, as well as addi-
There is substantial interest on the part of health tion or omission of actions related to standard op-
care purchasers for quality and safety in ICUs. erational instructions or routines of the unit. Ac-
This is best illustrated by the Leapfrog Group’s cording to Donchin et al. (1995), patients in an ICU
initiative. The Leapfrog Group is a coalition of receive on average 178 activities per day, and 1% of
more than 140 public and private organizations these activities involve errors. Twenty-nine percent
that provide health care benefits to approximately of these errors have the potential to cause signifi-
34 million health consumers in all 50 states. It cant deterioration in patient’s condition or death.
was founded by the Business Roundtable, a na- Bracco et al. (2000) conducted a prospective ob-
tional association of Fortune 500 CEOs, ‘‘in or- servational study of consecutive patients admitted
der to help save lives and reduce preventable over 1 year to an 11-bed multidisciplinary ICU in a
medical mistakes by mobilizing employer purchas- non-university teaching hospital in order to identify
ing power to initiate breakthrough improvements critical incidents and associated risk factors. A total
in the safety of health care and by giving con- number of 1024 patients were included in the study.
sumers information to make more informed hos- These patients required 2801 days of treatment in
pital choices’’ (http://www.leapfroggroup.com). the ICU. The authors developed a list of 105 critical
The Leapfrog Group has identified three initia- incidents, based on published studies (Vitez, 1990;
tives that are assumed to improve patient safety. Brennan et al., 1991; Giraud et al., 1993; Fasting
One of these initiatives is specifically related and Gisvold, 1996). After a critical incident was
to intensive care: ‘‘Hospital ICU care should be identified, it was analyzed and its cause was classi-
managed or co-managed by a physician certi- fied as equipment-, patient- or human-related. If
fied (or eligible for certification) in critical care the cause was related to a technical equipment
medicine’’ (http://www.leapfroggroup.com). This failure, it was classified as equipment-related. If
explicit focus of the Leapfrog Group on inten- the cause was related to a human error, such as a
sive care demonstrates the importance given by health care provider’s actions not going as intended
American businesses on improving ICU patient or the intended action not being appropriate, the
care. critical incident was identified as human-related. If
In this paper, we examine studies of patient none of the above was identified, the critical inci-
safety in ICUs and conclude that nursing work- dent was classified as patient-related. The number
load is a major contributor to patient safety in of critical incidents detected during the study pe-
ICUs. We conducted a literature search of stud- riod (1 year) was 777. Thirty-one percent of these
ies published after 1970 in MEDLINE and CINAHL incidents were human-related, 2% were equipment-
databases. We used the following keywords: nurs- related, and 67% were patient-related. The authors
286 P. Carayon, A.P. Gürses

also found that human errors prolonged ICU stay by tal of 2009 observed events (6.6%) in a medi-
425 patient days over a 1-year period. cal ICU. Tissot and colleagues used an adaptation
Several studies have examined various types of of the ‘‘disguised-observation technique’’ devel-
errors in ICUs. Giraud et al. (1993) conducted a oped by Barker and McConnell (1962). For 6 h ev-
prospective, observational study to examine ia- ery day, two observers (pharmacy residents) ob-
trogenic complications and to identify the factors served two randomly selected nurses giving med-
that favor the occurrence of these complications. ication to patients. Based on these observations,
The authors define an iatrogenic complication the researchers classified errors into 6 categories:
as ‘‘an adverse event that was independent of wrong drug preparation, dose error, wrong adminis-
the patient’s underlying disease’’. A total of 382 tration technique, physicochemical incompatibility
patients for 400 consecutive admissions provided errors occurring at simultaneous administration of
data. Thirty-one percent of the admissions had two or more medicines via the same route, wrong
iatrogenic complications. The risk of mortality rate, and wrong time. They identified 10 possible
among patients with iatrogenic complications causes of medication errors, including high nursing
was significantly higher than the risk of mortality workload and fatigue.
among patients without iatrogenic complications. Research on patient safety and medical errors
Forty-four percent of all iatrogenic complications in ICUs shows that errors are frequent in ICUs and
were associated with either human errors (insuf- that they can result in negative patient outcomes.
ficient surveillance, inadequate experience) or System-related human errors seem to be particu-
equipment-related problems (equipment failure, larly prevalent in ICUs. Several factors may lead to
inadequate equipment). The relationship of the poor outcomes in ICUs, such as inadequate commu-
occurrence of iatrogenic complications with prog- nication between nurses and physicians (Donchin et
nostic indices, nursing workload, and length of stay al., 1995), impaired access to information (Leape
was examined. Patients with two or more organ et al., 1995), and high nursing workload (Giraud et
failures and patients older than 65 years were found al., 1993; Tissot et al., 1999). This paper focuses
to have a higher risk of developing major iatrogenic on nursing workload and its impact on quality of
complications. Nursing workload was measured working life of nurses and quality and safety of care
with the OMEGA system. The OMEGA system mea- provided to ICU patients.
sures nursing workload based on 47 diagnostic and
therapeutic items (Le Gall et al., 1990). In addition
to the OMEGA system, the authors measured nursing Workload in ICUs
workload subjectively in order to take into account
non-objectively measurable factors, such as expe- ICU practice is filled with high workload situations
rience, fatigue, and availability of help from others (Crickmore, 1987; Malacrida et al., 1991; Oates and
in the unit. At the end of each 12-h shift, nurses Oates, 1996). Nurses must continuously respond to
were asked to rate the level of their workload using the needs of patients and families, and routinely
the following four categories: 1 = minor, 2 = normal, interact with the most intense emotional aspects of
3 = heavy, 4 = excessive. Results showed that nurses life. Research shows that nursing workload is one of
rated workload as ‘‘excessive’’ 302 times (4.7%), the most important determinants of patient safety
‘‘heavy’’ 1494 times (23.3%), ‘‘normal’’ 4263 times and quality of care in ICUs.
(66.5%) and ‘‘minor’’ 349 times (5.4%). The non- An analysis of the Australian Incident Monitor-
parametric Kruskal—Wallis test was used to test the ing Study data for ICUs shows that a nursing staff
association between the occurrence of iatrogenic shortage may lead to compromised quality of care
complications, and nursing workload and length of (Beckmann et al., 1998). The two main causes of
stay. Results revealed that the frequency of iatro- nursing staff shortage were inappropriate staffing
genic complications was positively associated with for current patient load (81% of the incidents)
both the subjectively rated nursing workload and and inability to respond to increased unit activ-
the OMEGA scores. Length of stay was also found to ity (19% of the incidents). Insufficient nursing staff
be significantly higher for patients who developed was found to be associated with the occurrence
iatrogenic complications than for patients who did of the following incidents: drug administration or
not. Based on their findings, the authors recom- documentation problems, inadequate patient su-
mended improving the organization of workload pervision, incorrect ventilator or equipment setup,
and training, and increasing the use of noninvasive and self-extubation. Undesirable patient outcomes
monitoring. associated with insufficient nursing staff include
Tissot et al. (1999) detected 132 medication major physiological change, patient or relative dis-
preparation and administration errors in a to- satisfaction, and physical injury. An undesirable pa-
Human factors engineering conceptual framework 287

tient outcome was reported in 37% of the incidents method developed by the authors for this study.
associated with insufficient nursing staff. In the computer-aided self-observation method, 16
Tarnow-Mordi et al. (2000) examined the rela- ICU nurses were asked to record their experiences
tionship between mortality rates and the workload on a computer placed in the ICU, immediately af-
of hospital staff in one adult ICU in the United ter having experienced a stressful event. These
Kingdom. Measures of workload for a specific pa- same 16 ICU nurses and 31 other nurses working
tient’s stay included occupancy per shift, peak oc- at the same hospital were then asked to fill out
cupancy, ICU nursing requirement per shift (i.e. the the NSS questionnaire. Results obtained by both
highest number of nurses required for the ICU ac- data collection methods showed that the most im-
cording to the recommendations of the UK Inten- portant stressors for nurses were related to deal-
sive Care Society; i.e. 0.5 nurse per patient per ing with death and high workload. In this study,
shift for patients who are spontaneously breath- staffing and scheduling problems, not having suf-
ing and need simple monitoring only), ICU nursing ficient time to complete nursing tasks and not hav-
requirement during patient’s first shift, ICU occu- ing sufficient time to provide emotional support to
pancy during patient’s first shift, the ratio of oc- patients were all considered as workload-related
cupied to appropriately staffed beds per shift, and problems.
the ICU nursing requirement per occupied bed per In addition to research on working conditions and
shift. Results demonstrated that patients exposed stress, there is considerable evidence for the re-
to high ICU workload were more likely to die than lationship between nursing working conditions and
those exposed to low workload. The three measures job satisfaction (Bratt et al., 2000; Darvas and
of workload most strongly associated with mortality Hawkins, 2002). Behavioral consequences of job
were peak occupancy, average nursing requirement dissatisfaction in nursing, such as low morale, ab-
per occupied bed per shift, and the ratio of occu- senteeism, turnover, and poor job performance,
pied to appropriately staffed beds. Explanations for can potentially threaten patient care quality and
the association between high workload and mor- organizational effectiveness (Cavanagh, 1992). Sev-
tality include insufficient time for clinical proce- eral nursing studies have examined the outcomes
dures to be done appropriately, inadequate training or consequences of job satisfaction. Studies have
or supervision, errors, overcrowding and conse- found positive associations between job satisfac-
quently nosocomial infections, limited availability tion and job performance (McCloskey and Mc-
of equipment, and premature discharge from the Cain, 1987), and patient satisfaction and quality
ICU. of care (Tarnowski-Goodell and Van Ess Coeling,
The UK Neonatal Staffing Study Group (2002) 1994).
conducted a study to determine whether work- High workload seems to be related to sub-
load was associated with risk-adjusted outcomes in optimal patient care (Keijsers et al., 1995; Aiken
neonatal intensive care units (NICUs). They studied et al., 2002), seems to influence care provider’s
13,515 infants consecutively admitted to 54 ran- decision to perform various procedures (Griffith et
domly selected NICUs. Results showed that mor- al., 1999) and may lead to reduced patient satis-
tality was significantly positively associated with faction (Anderson and Maloney, 1998). High work-
the percentage maximum occupancy on admission load may also lead to poor nurse-patient com-
and negatively associated with the nurse-to-infant munication (Llenore and Ogle, 1999), impaired
ratio in all types of NICUs. As the maximum oc- nurse—physician collaboration (Baggs et al., 1999),
cupancy in the unit increases, nurse staffing be- nurse burnout and job dissatisfaction (Aiken et al.,
comes more inadequate, which, in turn, results 2002).
in an increase in the risk of mortality. Based on Evans and Carlson (1992) suggest strate-
this study, the authors suggest that service in gies such as improving working conditions and
NICUs should be improved by reducing nursing work- nurse—physician collaboration to reduce workload
load. and to deal with the nursing shortage crisis. In this
Research also shows that workload is one of paper, we present a framework for conceptualizing
the most important job stressors among ICU nurses nursing workload, its work-related causes, its
(Crickmore, 1987; Malacrida et al., 1991; Oates and effects on the quality of working life of nurses,
Oates, 1996). The practice of medicine is inherently the quality and safety of care they provide, and
stressful. For example, Malacrida et al. (1991) stud- associated economic outcomes. Our ultimate
ied the quality and frequencies of stressors in an goal is to provide a human factors engineering
ICU environment by using two different data collec- framework for identifying work system factors that
tion methods: the Nursing Stress Scale (NSS) ques- can be modified to reduce workload in order to
tionnaire, and a computer-aided self-observation improve quality of working life of ICU nurses and
288 P. Carayon, A.P. Gürses

improve the quality and safety of care provided to nurses encounter various situations and various pa-
ICU patients. tients. Therefore, the situation- and patient-levels
of workload are embedded in the job-level of work-
load, and the job-level of workload is embedded in
Conceptual framework of ICU nursing the unit-level of workload. In addition, measures
at a lower level, e.g., patient level, can be aggre-
workload
gated to measure workload at a higher level, e.g.,
unit level. We provide more information about each
Fig. 1 shows the proposed conceptual framework
of these four workload measures below.
of ICU nursing workload. Given our human fac-
tors engineering approach, one can define vari-
ous work-related factors that contribute to nursing
workload (i.e. causes). We also define different lev- Workload at the ICU (unit) level
els of nursing workload (see next section). Nurs-
The most widely used measure to assess work-
ing workload, in turn, can have a number of
load at the ICU level is the nurse/patient ratio.
intermediate consequences (such as insufficient
Studies have examined the relationship between
compliance of nurses with hand hygiene poli-
nurse/patient ratio and clinical and economic out-
cies or inadequate patient supervision), which can
comes of abdominal aortic surgery (Pronovost et
lead to various outcomes. The consequences of
al., 1999), complications and resource use after
nursing workload provide mechanisms for under-
hepatectomy (Dimick et al., 2001), complications
standing and specifying how nursing workload can
and resource use among patients who had oe-
contribute to various outcomes. The consequences
sophagectomy (Amaravadi et al., 2000), and infec-
mediate the relationship between nursing work-
tion rates (Archibald et al., 1997; Grundmann et
load and outcomes. Outcomes include patient out-
al., 2002).
comes (quality of care and patient safety), family
ICU-level measures of workload include a modi-
outcomes (satisfaction with the care provided),
fied nurse/patient ratio measure that considers ed-
nurse outcomes (quality of working life), and eco-
ucation and tenure of nurses. For example, Robert
nomic outcomes. The conceptual framework de-
et al. (2000) conducted a nested case-control study
fines a ‘chain of events’ from causes to workload,
to identify the risk factors associated with the ac-
to consequences and outcomes. Table 1 provides
quisition of nosocomial primary bloodstream in-
a summary of the research evidence that pro-
fections (BSIs) in a surgical intensive care unit
vides support for various aspects of the conceptual
(SICU). The authors compared 28 case-patients (pa-
framework.
tients with BSI) with 99 randomly selected control-
patients who stayed in the SICU for at least 3 days.
The study failed to show any significant associ-
Levels and measures of nursing ation between the overall nurse/patient ratio in
workload in ICUs the unit and the acquisition of BSIs. The authors
also studied nursing staff composition as a possible
Studies have conceptualized nursing workload in risk factor for the acquisition of BSIs. They defined
ICUs differently and have used various instruments ‘‘regular staff’’ as nurses permanently assigned to
to measure nursing workload. We can categorize the SICU, and ‘‘pool staff’’ as agency nurses or
the workload measures into four groups: (1) work- nurses from the hospital pool service. The authors
load measures at the ICU (unit) level, (2) workload then compared nursing staff composition (regular
at the job level, (3) workload at the patient level, nurse/patient ratio versus pool nurse/patient ratio)
and (4) workload at the situation level (see Table 2). between the case- and control-patients. Results in-
These measures can be organized into a hierar- dicated that having a low regular nurse/patient ra-
chy of measures. In an ICU (unit level), there are tio and a high pool nurse/patient ratio significantly
various nursing jobs. When performing their job, increase the risk of BSIs.

Figure 1 Conceptual framework of ICU nursing workload.

 Human factors engineering conceptual framework
Table 1 Causes, consequences and outcomes of nursing workload in ICUs
Author Title Causes Workload Consequences Outcomes
Amaravadi ICU nurse/patient ratio is Staffing level of nurses Workload at the unit A nurse/patient ratio of less than 1:2
et al. associated with level (nurse/patient was found to be related to an
(2000) complications and resource ratio) increased risk of several postoperative
use after esophagectomy pulmonary and infectious
complications, a 39% increase in
length of stay, and a 32% increase in
direct hospital costs ($4,810) among
patients who had esophagectomy.
Anderson A descriptive, correlational Not investigated Nursing workload at the Patient satisfaction was found to be
and Maloney study of patient satisfaction, unit level significantly higher in ICUs than in
(1998) provider satisfaction, and (nurse/patient ratio) general care units because of the high
provider workload at the nurse/patient ratio in ICUs.
army medical center
Archibald Patient density, Staffing level of nurses Nursing workload at the The nosocomial infection rate in the
et al. (1997) nurse/patient ratio and unit level (nursing pediatric cardiac intensive care unit
nosocomial infection risk in a hours /patient day was negatively correlated with the
pediatric cardiac intensive ratio) nursing hours/patient day ratio.
care unit
Ball and Realising the potential of Geography (layout of Changes in nursing • Decreased vs. increased risk to
McElligot critical care nurses: an the ICU) attributes: patients
(2003) exploratory study of the Activity level in the • patient centered • Timely vs. delayed patient
factors that affect and unit (e.g., admissions, versus task-orientated progression
comprise the nursing transfers) care
contribution to the recovery Skill-mix • proactive care vs. • Potential for recovery vs.
of critically ill patients Patient dependency progress hindered deterioration of the patient
• vigilance vs. failure • Family satisfaction with the care
to appreciate cues provided
• coping with
unpredictable vs.
ineffective coping
• emotional support vs.
emotional tension

289
 290
Table 1 (Continued )
Author Title Causes Workload Consequences Outcomes
Beckmann Problems associated with Staffing level of nurses Nursing workload at the Incidents that were Nursing staffing level in the ICU was
et al. nursing staff shortage: An Inability to respond to unit level (staffing level most frequently found to be negatively associated with
(1998) analysis of the 3600 incident increased unit activity in the unit was associated with the possibility of a major physiological
reports submitted to the subjectively assessed insufficient nursing change in patient’s condition and the
Australian Incident by nurses as sufficient staff: possibility of physical injury of a
Monitoring Study (AIMS-ICU) or insufficient) • drug administration patient and positively associated with
or documentation patient or family satisfaction.
problems
• inadequate patient
supervision
• incorrect set up of
ventilator or equipment
• self-extubation
Dang et al. Postoperative complications: Staffing level of nurses Workload at the unit The level of nurse staffing was
(2002) Does intensive care unit staff level: negatively associated with the risk of
nursing make a difference? • Low-intensity complications in patients undergoing
staffing: 1:3 or greater abdominal aortic surgery.
on the day and night
shifts
• Medium-intensity
staffing: 1:3 or greater
on either the day or
the night shift, but not
both
• High-intensity
staffing: 1:2 or fewer
on the day and night
shifts
Dimick et Effect of nurse/patient ratio Staffing level of nurses Nursing workload at the As nurse/patient ratio decreases,

P. Carayon, A.P. Gürses

al. (2001) in the intensive care unit on unit level postoperative pulmonary
pulmonary complications and (nurse/patient ratio) complications and use of resources
resource use after increase. A nurse/patient ratio less
hepatectomy than 1/2 was related to a 14% increase
in direct hospital costs.
 Human factors engineering conceptual framework
Giraud et Iatrogenic complications in Nursing workload at the As the level of nursing The occurrence of major iatrogenic
al. (1993) adult intensive care units: A situation level workload (both at the complications is positively associated
prospective two-center study (subjectively situation level and the with increased morbidity and
measured) patient level) mortality.
Nursing workload at the increases, the risk of
patient level (OMEGA) major iatrogenic
complications
increases.
Grundmann Risk factors for the Nursing workload at the Insufficient compliance Exposure to relative nursing staff
et al. transmission of unit level (daily nurse with hand hygiene deficit was the only factor associated
(2002) methicillin-resistant staffing level/daily bed policies with the potential transmission of
staphylococcus aureus in an occupancy) methicillin-resistant staphylococcus
adult intensive care unit: aureus.
Fitting a model to the data
Gunnarsson Mobile computerized Use of mobile Nursing workload at the A reduction of 60% to 75% in staff
et al. tomography scanning in the computerized situation level (time workload is estimated when mobile CT
(2000) neurosurgery intensive care tomography (CT) head spent by nurses on a is used as opposed to conventional CT.
unit: Increase in patient scanning versus single scanning
safety and reduction of staff conventional head CT procedure = Mean
workload scanning in a number of
neurosurgery ICU nurses × Mean time for
transportation and
scanning)
Jakob and Intensive care 1980—1995: • Increase in the Nursing workload at the Although there has been a significant
Rothen Change in patient number of patients unit level (Swiss increase in nursing workload during
(1997) characteristics, nursing • Number of severely Society of Intensive the last 16 years, ICU mortality rate
workload and outcome sick patients Care Medicine grading and length of stay decreased. This
• Age of patients system) Nursing may be due to improved patient
admitted to this ICU workload at the patient treatment or care.
during the last 16 years level (modified version
of TISS)

291
Table 1 (Continued )

292
Author Title Causes Workload Consequences Outcomes
Malacrida Computer-aided Nursing workload at the High workload is one of the most
et al. self-observation job level (as measured important stressors among ICU nurses.
(1991) psychological stressors in an by Nursing Stress Scale
ICU and a computer-aided
self-observation
method)
Morales et Hospital mortality rate and Admissions to the ICU Nursing workload at the No significant association was found
al. (2003) length of stay in patients situation level between workload and the mortality
admitted at night to the (situation = patient rate, and between workload and
intensive care unit admission) length of stay.
Workload rated as
‘‘heavy’’ if 3 or more
admissions during night
shifts
Workload rated as
‘‘regular’’ if less than 3
admissions during night
shifts
Moreno et Mortality after discharge Organ dysfunction or Workload at the patient Patients who have a high degree of
al. (2001) from intensive care: The failure level (NEMS) organ dysfunction or failure before
impact of organ system being discharged from ICU require
failure and nursing workload more nursing workload and have a
use at discharge higher probability of dying after being
discharged from ICU.
Pronovost Organizational Staffing level of nurses Workload at the unit A nurse/patient ratio less than 1/2
et al. characteristics of intensive level (nurse/patient was related to increased ICU days and
(1999) care units related to ratio) hospital length of stay.
outcomes of abdominal aortic
surgery

P. Carayon, A.P. Gürses

Robert et The influence of the Staffing level of nurses Workload at the unit No significant association exists
al. (2000) composition of the nursing Regular nursing staff level (nurse/patient between the overall nurse/patient
staff on primary bloodstream versus pool nursing ratio, regular ratio in the unit and acquisition of
infection rates in a surgical staff nurse/patient ratio, blood stream infections. A low regular
intensive care unit pool nurse/patient nurse/patient ratio and a high pool
ratio) nurse/patient ratio significantly
increase the risk of blood stream
infections.
 Human factors engineering conceptual framework
Schaufeli Personnel Workload at the job level Perceived workload was positively
and Le (scale derived from related with burnout and negatively
Blanc Karasek’s job content related with perceived unit
(1998) instrument (1985) performance. Burnout was a mediator
Ipatients /Opatients , between workload and unit
Iteam /Oteam ) performance. An objective measure of
Workload at the patient workload was positively related to
level (NEMS) burnout, but burnout was not related
to objective unit performance.
Stillwell et The impact of do-not Do-not-resuscitate Workload at the patient This study did not find any difference
al. (1997) resuscitate orders on nursing (DNR) order level (Medicus Systems in the number of hours of nursing care
workload in an ICU Corporation Interact 2000 required for a patient in the ICU
Workload and Productivity before and after a DNR order is
System) written. This finding is in contradiction
with the belief that patients with DNR
orders require less nursing care.
Tarnow- Hospital mortality in relation Staffing level of nurses Workload at the unit level Consequences of As ICU workload increases, the risk of
Mordi et al. to staff workload: A 4-year High patient volume (ICU occupancy per shift, excessive workload: mortality increases. Peak occupancy,
(2000) study in an adult intensive (number of beds ICU occupancy during a • insufficient time for average nursing requirement per
care unit occupied exceeds the patient’s first shift, peak clinical procedures to occupied bed per shift, and the ratio
number appropriately occupancy during a be done appropriately of occupied to appropriately staffed
staffed) patient’s stay, average • inadequate training beds are related to increased
occupancy per shift, ICU or supervision mortality risk.
nursing requirement per • errors
shift, ICU nursing • nosocomial infections
requirement during as a result of
patient’s first shift, ratio overcrowding
of occupied to • limited availability of
appropriately staffed equipment
beds) • premature discharge
from the ICU

293
 294
Table 1 (Continued )
Author Title Causes Workload Consequences Outcomes
Workload at the patient
level (ICU nursing
requirement per
occupied bed per shift)
Thorens et Influence of the quality of Staffing level of nurses Workload at the unit The duration of mechanical
al. (1995) nursing on the duration of Education level of ICU level: index of nursing ventilation is negatively correlated
weaning from mechanical nurses (certified versus is a ratio based on the with the nursing index.
ventilation in patients with noncertified) number and the
chronic obstructive qualifications of the
pulmonary disease nurses working in the
unit.
Tissot et al. Medication errors at the Work overload The authors identified ten possible
(1999) administration stage in an causes of medication errors, including
intensive care unit high nursing workload fatigue.
UK Patient volume, staffing, and Staffing level of nurses Percentage maximum Risk adjusted mortality increases as
Neonatal workload in relation to occupancy on the maximum occupancy on admission
Staffing risk-adjusted outcomes in a admission increases and the nurse/infant ratio
Study random stratified sample of Nurse/infant ratio decreases.
Group UK neonatal intensive care
(2002) units: a prospective
evaluation

P. Carayon, A.P. Gürses

Human factors engineering conceptual framework 295

Nurse/patient ratio and other ICU-level work- This approach can provide information on work-
load measures can be used to measure the overall load in ICU nursing as compared to other jobs.
nursing workload in an ICU. This information can When using this approach to reduce nursing work-
be used to compare ICUs and their patient out- load, the main improvement strategy consists of
comes. Important research using ICU-level work- changing the work organization, which is difficult
load measures has demonstrated strong evidence of to accomplish in ICUs. In addition, some of the
the relationship between high nursing workload and key elements of an ICU nursing job, e.g. patient
negative patient outcomes. This type of workload requirements and patient care tasks, cannot be
measure is at a macro-level, and therefore does not changed.
consider contextual and organizational character-
istics of a particular ICU, such as the type of work Workload at the patient level
performed by the nurses and other staff involved
in patient care as well as ancillary tasks. Improve- Several measures have been developed to deter-
ment solutions that can be derived from studies mine nursing requirements and resource allocation
using ICU-level workload measures would typically in ICUs. These measures estimate nursing workload
involve either increasing the number of nurses or based on the condition of the patient. The most
decreasing the number of patients. This type of im- widely used patient-level workload measure is the
provement strategy is probably not feasible to im- Therapeutic Intervention Scoring System-76 (TISS-
plement given the current nursing shortage and the 76) developed originally by Cullen et al. (1974)
ever-expanding ICU patient population. Therefore, and expanded by Keene and Cullen (1983). TISS-76
we need another type of approach for examining measures nursing workload based on several thera-
and dealing with workload. peutic variables related to patient’s condition. The
original TISS-76 has been criticized for the following
Workload at the job level reasons (Miranda et al., 1996, 1997):
1- Scoring with TISS-76 is time consuming.
Workload can be conceptualized as a working con-
2- Users find filling out TISS-76 instrument cumber-
dition that is a stable characteristic of the job.
some and boring.
One example of a study using a job-level mea-
3- TISS-76 does not have items related to indirect
sure of workload is that conducted by Schaufeli
patient care activities of ICU nurses such as con-
and Le Blanc (1998). They examined the impact
tact with family or maintaining supplies.
of workload on burnout and performance among
4- TISS does not have items related to activities
ICU nurses. They used three different subjective
of the nursing staff that are not related to pa-
measures and one objective measure to measure
tient care (e.g., meetings, trainee supervision,
nursing workload. One of the subjective mea-
research).
sures was a scale derived from Karasek’s Job Con-
5- TISS-76 does not have high inter-rater reliability.
tent instrument (Karasek, 1985). Results of the
study showed that perceived workload was pos- A simplified version of TISS with 28 items (TISS-
itively related with burnout and negatively re- 28) was created to address the first two criticisms.
lated with unit performance evaluated by ICU TISS-28 explains 86% of the variation in TISS-76
nurses. (Miranda et al., 1996). In 1994, the Foundation for
There is a large body of research that links Research on Intensive Care in Europe (FRICE) began
job-level workload (a working condition) to vari- a large prospective survey of ICUs in Europe. TISS-
ous nursing outcomes, such as stress (Crickmore, 28 was found to be too long and time consuming
1987; Malacrida et al., 1991) and job dissatisfac- for use in this multi-center multinational epidemi-
tion (Freeman and O’Brien-Pallas, 1998). ological study (Miranda et al., 1997). As a result,
Job-level workload is one of many working condi- FRICE developed and validated a new therapeutic
tions that characterize a job. Therefore, job-level index called the Nine Equivalents of Nursing Man-
workload is a stable job characteristic that does power Use Score (NEMS). NEMS was derived from
not vary much over time unless some change is the TISS-28 and includes 9 items. NEMS was com-
introduced in the work organization that will af- pared with TISS-28 in several studies and found to
fect the characteristics of the job (Carayon et al., be highly associated with TISS-28. However, NEMS
2001). Measures of job-level workload can be use- was found to have a poor discriminative power of
ful to compare and contrast ICU nursing workload workload At the individual patient level (Miranda et
to workload in other nursing jobs and to examine al., 1997; Rothen et al., 1999).
how ICU nursing workload affects quality of work- Although TISS measures nursing workload based
ing life of nurses and quality and safety of care. on therapeutic interventions related to severity of
296 P. Carayon, A.P. Gürses

Table 2 Levels and Measures of Nursing Workload in ICUs

Levels of Workload Measures of Workload
ICU/Unit Level Nurse/patient ratio (Dimick et al., 2001; Pronovost et al., 1999)
Nursing hours/patient day ratio (Archibald et al., 1997)
Regular nurse/patient ratio (Robert et al., 2000)
Pool nurse/patient ratio (Robert et al., 2000)
Index of nursing = ratio calculated based on both the number and the qualifications
of the nurses working in the unit (Thorens et al., 1995)
Patient census (Anderson and Maloney, 1998)
Daily bed occupancy = measure of the entire patient load passing through the ICU
during a single day (Grundmann et al., 2002)
Daily nurse staffing level/daily bed occupancy = ratio of less than 1 is considered as
a relative staff deficit (Grundmann et al., 2002).
Occupancy per shift = highest number of ICU beds occupied each shift during a pa-
tient’s stay (Tarnow-Mordi et al., 2000)
Peak occupancy = highest occupancy per shift during a patient’s stay (Tarnow-Mordi
et al., 2000)
Ratio of occupied /appropriately staffed beds per shift (Tarnow-Mordi et al., 2000)
ICU nursing requirement per shift = total number of ICU nurses required in an ICU
per shift (Tarnow-Mordi et al., 2000)
Number of nursing days per year (Jakob and Rothen, 1997)
Number of patients per nurse per year (Jakob and Rothen, 1997)
Work Utilization ratio (WUR) = ratio between the produced workload and the avail-
able workload which can be calculated by dividing total NEMS points used during
1 year by the number of nurses × 200 × 46/3 where 200 is the annual number of
working days for each nurse, 46 is the maximal number of NEMS points a nurse can
perform in a day, and 3 is the number of 8-h nursing shifts (Moreno and Miranda,
1998).
Job Level Overall perceived workload associated with a job - Karasek’s Job Content Instrument
(Karasek, 1985)
Ipatients /Opatients = ratio of investments to outcomes evaluated by ICU nurses, in the
relationship with their patients (Schaufeli and Le Blanc, 1998)
Iteam /Oteam = ratio of investments to outcomes evaluated by ICU nurses, in the rela-
tionship with their colleagues (Schaufeli and Le Blanc, 1998)
Perceived quantitative workload - Caplan et al. (1975)
Patient Level Time spent by nurses on a single scanning procedure = mean number of
nurses × mean time for transportation and scanning (Gunnarsson et al., 2000)
Therapeutic Intervention Scoring System-28 (TISS-28) = simplified version of TISS-76
with 28 items (Miranda et al., 1996).
Nine Equivalents of Nursing Manpower Use Score (NEMS) = NEMS is the simplified
version of TISS-28. It quantifies the nursing manpower use in ICUs based on nine
items (Miranda et al., 1997).
Nursing Activities Score (NAS) = modified version of TISS-28 with five additional items
and 14 sub-items describing nursing activities in an ICU (e.g., monitoring, care of rel-
atives, administrative tasks). As opposed to the weights in TISS-28, which represent
the severity of the patient’s condition, the weights of NAS represent the calculated
percentage of nursing time dedicated to the performance of the activities listed
(Miranda et al., 2003).
PRN 80 = Canadian information system for the management of nursing staff in hos-
pitals and homes for the elderly. This instrument lists 214 indicators or tasks that
nurses complete on behalf of patients during a 24-h period. Each indicator or task
is assigned a point value that represents the time to complete a specific nursing
intervention as well as the number of times the task will be completed on a shift
(EROS Equipe de Recherche Operationnelle en Sante, 1981).
Time Oriented Score System (TOSS) = method for direct quantification of nurses’
workload required by ICU patients (Italian Multicenter Group of ICU Research, 1991)
OMEGA = ICU-specific activity scoring system used in the French ICUs since 1990. The
Omega score is calculated during each patient’s ICU stay and scored from 1 to 10
(Le Gall et al., 1990).
Human factors engineering conceptual framework 297

Table 2 (Continued )
Levels of Workload Measures of Workload
UK Intensive Care Society Nursing Dependency Schedule = quantifies the qualitative
description of nurse dependency based on the guidelines of the UK Intensive Care
Society. The number of nurses required to take care of a patient is between 0.5 and
2 (Standards Subcommittee Intensive Care Society, 1983).
Patient Intensity for Nursing Index (PINI) = A multidimensional measurement instru-
ment for measuring nursing intensity in ICUs. It consists of three dimensions; depen-
dency, severity, and complexity with time (hours of nursing care) (Prescott et al.,
1989; Soeken and Prescott, 1991).
Patient nursing condition as measured by the number and types of nursing diag-
noses: Nursing diagnoses are determined by the National Conferences on the Clas-
sification of Nursing Diagnosis to describe patients’ nursing conditions (Gebbie and
Levin, 1975). Nursing diagnosis has the advantage of being related to nursing theory
as opposed to medical diagnosis generated by physicians (Halloran, 1985).
Swiss Society of Intensive Care Medicine (SGI) Grading System = category I = one
nurse/patient per shift, category II = one nurse/two patients/shift, category
III = one nurse/three patients/shift, category Ia = 1.3 nurses/patient/shift (Jakob and
Rothen, 1997)
Medicus Systems Corporation InterAct 2000 Workload and Productivity System
(Interact, 2000)
Situation level Subjective workload scales, such as NASA-TLX (Hart and Staveland, 1988)
Time spent by nurses on a single scanning procedure = mean number of
nurses × mean time for transportation and scanning (Gunnarsson et al., 2000)

illness, many nursing activities are not necessar- Workload at the situation level
ily related to illness severity. In response to this
criticism of TISS, Miranda et al. (2003) recently de- Situation-level workload is that workload explained
veloped a new instrument called the Nursing Ac- by what happens in a clinical microsystem and is
tivities Score (NAS). NAS is a modified version of temporally bound. First, the characteristics of the
TISS-28 with an additional five new items and 14 clinical microsystem can contribute to situation-
sub-items. The weights of NAS measure time con- level workload. For example, a nurse who goes into
sumed by nursing activities at the patient level. an isolation patient’s room to perform some nursing
As opposed to the weights in TISS-28, which rep- activities discovers unexpectedly that one of the
resent the severity of the patient’s condition, the supplies s/he needs is not available in the room.
weights of NAS represent the calculated percent- The nurse, then, leaves the room, removes his/her
age of nursing staff’s time (one period of 24 h) isolation gown, goes to the supply room, finds the
dedicated to the performance of the activities in- right supply, takes it back to the unit, puts the iso-
cluded in the instrument. The sum of the weights lation gown on, and enters the patient’s room with
of the individual items scored reflects the amount the supply necessary to perform the activities. In
of time that the nursing staff in an ICU spends this example, additional workload in the form of
on performing tasks during a particular day. NAS extra tasks and increased time is related to inade-
was validated in a study of 99 ICUs in 15 coun- quate stocking of isolation rooms. This micro-level
tries. Results indicate that NAS explains 81% of the characteristic is related to the design of the clini-
nursing time, whereas TISS-28 explains only 43% cal microsystem. A second important characteristic
(Miranda et al., 2003). of situation-level workload is its narrow temporal
The nursing workload measures in the patient- boundaries. Situation-level workload is evaluated
level category are based on a patient’s condition. either for a specific event, such as intrahospital pa-
They are not helpful to understand the impact of tient transport, or over a short time period, such as
different ‘‘situations’’ that are likely to occur in an a shift.
ICU on nursing workload. We, therefore, introduce We propose that workload measures developed
the situation level of workload measures to under- in the human factors engineering literature can
stand the impact of various situations on nursing be used to assess situation-level workload. Very
workload. few studies conducted in ICUs have used the work-
298 P. Carayon, A.P. Gürses

load measurement instruments developed by hu- These subjective workload measures can be used
man factors engineers. Among the many tools de- to evaluate ICU nursing workload with regard to a
veloped by human factors engineers for measur- specific event, such as intrahospital patient trans-
ing workload, subjective workload rating scales port, or with regard to a short time period, such as
have emerged as a primary source of informa- after a shift. We argue that the strength of study-
tion about situational loading (Casali and Wier- ing workload at the situation level is the ability to
wille, 1983). Tsang (2001) identify three different identify specific characteristics of the microsystem
ways to measure workload subjectively: (1) using that contribute to workload. This information can
a single one-dimensional rating scale for measur- then be used in redesign and improvement efforts.
ing the overall workload level versus using a multi-
dimensional rating scale to measure different di-
mensions of workload; (2) subjects’ rating of work- Conclusion
load experienced immediately after they complete
the task or retrospectively after experiencing all We suggest that focusing on ICU nursing workload
the task conditions; and (3) absolute or relative at the situation level can provide the information
ratings. An absolute rating is based on the task necessary to identify specific causes of workload.
by itself. For relative ratings, subjects compare Using human factors engineering (HFE) knowledge,
the task to a single standard or to multiple task and HFE approaches and measurement methods of
conditions. workload can provide direction for specific inter-
Subjective measures have high acceptance be- ventions that can be implemented in ICUs in order
cause subjects can express their opinions. Further- to improve nursing quality of working life, and qual-
more, they are easy to use. They also have high ity and safety of patient care.
transferability to new system or new task condi- According to this ‘‘micro’’ approach, one needs
tions because their unit of measurement is not task to examine the factors in a nurse’s immediate work
dependent (Nisbett and Wilson, 1977). Subjective system and her/his clinical microsystem that add
measures are reliable and have concurrent validity unnecessary workload, increase the stress level,
with performance measures (Gopher and Braune, and hinder performance. These factors are known
1984). One weakness of subjective measures is that as performance obstacles (Peters et al., 1980;
subjects may not remember everything accurately Brown and Mitchell, 1988; Park and Han, 2002).
if they rate their workload level long after complet- There may also be work system factors that can
ing the tasks. Subject’s bias and past experience facilitate the ICU nurse in performing her job;
may also affect subjective workload ratings (Tsang, these factors are called performance facilitators
2001). (Carayon et al., 2005). We propose that the con-
Two of the most widely subjective workload cept of performance obstacles and facilitators can
measures are the NASA Task Loading Index (NASA- be used to identify the sources of nursing workload,
TLX) (Hart and Staveland, 1988) and the Subjective and their impact on quality of working life, and
Workload Assessment Technique (SWAT) developed quality and safety of care in ICUs (Carayon et al.,
by the USAF Armstrong Aeromedical Research Lab- 2005). The performance obstacles and facilitators
oratories (Reid et al., 1981). Both scales are mul- are the causes of workload (see Fig. 1). In a man-
tidimensional. The scales differ from each other in ner inspired by the Balance Theory of Job Design
many respects such as the way they were derived, of Smith and Carayon (1989, 2000), one can iden-
the number and type of dimensions, the degree to tify the performance obstacles that significantly in-
which individual differences are represented, and crease unnecessary workload in an ICU and build
the scaling method used to combine the individ- up the performance facilitators, and therefore re-
ual scale ratings into an overall value of workload design some aspects of the ICU work system.
(Hendy et al., 1989). A similar approach has been proposed by Ball and
The NASA-TLX was developed from the NASA McElligot (2003). They conducted an exploratory
Bipolar Scale (Hart and Staveland, 1988), and con- study to identify the factors that affect and com-
sists of six scales: mental demand, physical de- prise the contribution of ICU nurses to the recovery
mand, temporal demand, performance, effort and of critically ill patients. Based on data collected via
frustration level. Each scale is divided into 20 equal various methods (e.g., interviews, observations),
intervals with the verbal anchors Low and High they developed a conceptual model of the con-
(Hendy et al., 1989). SWAT consists of three scales, textual factors in an ICU that affect individual at-
i.e. mental effort load, time load, and psychologi- tributes of ICU nurses and consequently the patient
cal stress load, with three levels for each scale ([1]- outcomes. Contextual factors in an ICU belonged
low, [2]-medium, [3]-high) (Reid et al., 1981). to four categories: geography (layout of the unit),
Human factors engineering conceptual framework 299

activity (the overall activity level in the unit such Amaravadi RK, Dimick JB, et al. ICU nurse-to-patient ratio
as the number of admissions and transfers), skill is associated with complications and resource use after
esophagectomy. Intensive Care Med 2000;26:1857—62.
mix (permanent versus agency nursing staff expe-
American Hospital Association. American Hospital Association
rience level), and patient dependency (e.g., the Statistics-1973—1994. Chicago, Illinois: AHA; 1995.
degree of confusion and agitation of the patient, Anderson FD, Maloney JP. A descriptive, correlational study
and number of turns required). Positive individual of patient satisfaction, provider satisfaction, and provider
attributes included patient centered care, proac- workload at an Army medical center. Mil Med 1998;163:90—4.
Angus DC, Kelly MA, et al. Current and projected work-
tive management, vigilance, coping with unpre-
force requirements for the care of the critically ill pa-
dictable situations, and providing emotional sup- tient and patients with pulmonary disease; can we meet
port to patients and families. Negative individual the requirements of an aging population. J Am Med Assoc
attributes were task orientated care, failure to ap- 2000;284:2762—70.
preciate cues about the condition of the patient, in- Angus DC, Linde-Zwirble WT, et al. The effect of managed care
on ICU length of stay. J Am Med Assoc 1996;276:1075—82.
effective coping with unpredictable situations, and
Archibald LK, Manning ML, et al. Patient density, nurse-
increased emotional tension. Therefore, it is im- to-patient ratio and nosocomial infection risk in a pe-
portant to take into account both the contextual diatric cardiac intensive care unit. Pediatr Infect Dis J
factors and individual attributes of nurses when ex- 1997;16(11):1045—8.
amining workload of nurse staffing in an ICU. Baggs JD, Schmitt MH, et al. Association between nurse-
physician collaboration and patient outcomes in three inten-
In this paper, we have presented an overview of
sive care units. Crit Care Med 1999;27(9):1991—8.
nursing workload in ICUs, its causes, consequences Ball C, McElligot M. Realising the potential of critical care
and outcomes, in particular with regard to patient nurses’: an exploratory study of the factors that affect and
safety. We have proposed a conceptual framework comprise the nursing contribution to the recovery of critically
of ICU nursing workload that defines levels of work- ill patients. Intensive Crit Care Nurs 2003;19:226—38.
Barker K, McConnell W. The problems of detecting medication
load (unit, job, patient, and situation). Much re-
errors in hospitals. Am J Hosp Pharm 1962;19:360—9.
search has been conducted at the ICU/unit level Beckmann U, Baldwin I, et al. Problems associated with nursing
and at the patient level; some research has been staff shortage: an analysis of the first 3600 incident reports
conducted at the job level; but little is known about submitted to the Australian Incident Monitoring Study (AIMS-
workload at the situation-level. There is a need ICU). Anaesth Intensive Care 1998;26:396—400.
Bracco D, Favre JB, et al. Human errors in a multidisciplinary in-
to focus on situation-level workload and identify
tensive care unit: A 1-year prospective study. Intensive Care
those performance obstacles and facilitators that Med 2000;27(1):137—45.
contribute to workload. This information can help Bratt MM, Broome M, et al. Influence of stress and nursing lead-
us to design and target interventions aimed at re- ership on job satisfaction of pediatric intensive care unit
ducing or eliminating specific causes of workload. nurses. Am J Crit Care 2000;9(5):307—17.
Brennan TA, Leape LL, et al. Incidence of adverse events and
This approach can be expanded to address nurs-
negligence in hospitalized patients. Results of the Harvard
ing workload in other care settings (Carayon et al., medical practice study I. N Eng J Med 1991;324(6):370—6.
2003; Institute of Medicine, 2004). We would like to Brown KA, Mitchell TR. Performance obstacles for direct and
encourage researchers and practitioners to further indirect labour in high technology manufacturing. Int J Prod
investigate the various levels of nursing workload Res 1988;26(11):1819—32.
Caplan RD, Cobb S, et al. Job Demands and Worker Health: Main
and outcomes, in particular situation-level work-
Effects and Occupational Differences. Ann Arbor, MI: Institute
load, and the means of reducing ICU nursing work- for Social Research; 1975.
load. Carayon P, Alvarado C. et al., Reducing workload and increas-
ing patient safety through work and workspace design. Pa-
per commissioned by the Institute of Medicine Committee
Acknowledgements on the Work Environment for Nurses and Patient Safety,
Center for Quality and Productivity Improvement (CQPI),
University of Wisconsin-Madison; 2003. http://www.engr.
Funding was provided by the Agency for Healthcare wisc.edu/centers/cqpi/reports/pdfs/r185.pdf.
Research and Quality (Grant # P20 HS11561-01; PI: Carayon P, Gürses AP, et al. Performance obstacles of health care
Pascale Carayon, and Dissertation Grant # 1 R36 providers. In: Korunka C, Hoffmann P, editors. To be published
HS014517). in the Organizational Psychology and Health Care, 4. Munich:
Hampp Publishers; 2005.
Carayon P, Haims MC, et al. Psychosocial work factors and work
organization. In: Karwowski W, editor. The International En-
References cyclopedia of Ergonomics and Human Factors. London: Taylor
and Francis; 2001. p. 111—21.
Abramson NA, Wald KS, et al. Adverse occurrences in intensive Carayon P, Smith MJ. Work organization and ergonomics. Appl
care units. J Am Med Assoc 1980;244:1582—4. Ergonom 2000;31:649—62.
Aiken LH, Clarke SP, et al. Hospital nurse staffing and patient Carmel S, Rowan K. Variation in intensive care unit outcomes: A
mortality, nurse burnout, and job dissatisfaction. J Am Med search for the evidence on organizational factors. Curr Opin
Assoc 2002;288(16):1987—93. Crit Care 2001;7:284—96.
300 P. Carayon, A.P. Gürses

Casali JG, Wierwille WW. A comparison of rating scale, Hart SG, Staveland LE. Development of NASA-TLX (Task Load In-
secondary-task, physiological, and primary task workload es- dex): Results of emprical and theoretical research. In: Han-
timation techniques in a simulated flight task emphasizing cock PA, Meshkati N, editors. Human Mental Workload. North-
communications load. Human Factor 1983;25:623—41. Holland: Elsevier Science Publishers B.V.; 1988. p. 139—83.
Cavanagh SJ. Job satisfaction of nursing staff working in hospi- Hendy KC, Campbell EL, et al. Review of workload: Measure-
tals. J Adv Nurs 1992;17:704—11. ment and prediction. In: Proceedings of the Human Factors
Chalfin DB, Cohen IL, et al. The economics and cost- Association of Canada 22nd Annual Conference; 1989.
effectiveness of critical care medicine. Intensive Care Med Institute of Medicine. Keeping Patients Safe: Transforming the
1995;21:952—61. Work Environment of Nurses. Washington, DC: The National
Cohen IL, Chalfin DB. Economics of mechanical ventilation: Sur- Academies Press; 2004.
viving the 90’s. Clin Pulm Med 1994;1:100—7. Intensive Care National Audit and Research Center, Intensive
Crickmore R. A review of stress in the intensive care unit. Inten- Care National Audit and Research Center: Annual Report from
sive Care Nurs 1987;3:19—27. the Case Mix Program Database. London; 2001.
Cullen DJ, Civetta JM, et al. Therapeutic intervention scoring Interact. Workload and Productivity System. Evanston, Ilinois:
system: A method for quantitative comparison of patient Medicus Systems Corporation; 2000.
care. Crit Care Med 1974;2(2):57—60. Italian Multicenter Group of ICU Research. Time oriented score
Dang D, Johantgen ME, et al. Postoperative complications: Does system (TOSS): A method for direct and quantitative assess-
intensive care unit staff nursing make a difference? Heart ment of nursing workload for ICU patients. Intensive Care
Lung 2002;31(3):219—28. Med 1991;17:340—5.
Darvas JA, Hawkins LG. What makes a good intensive care unit: Jacobs P, Noseworthy TW. National estimates of intensive care
a nursing perspective. Aust Crit Care 2002;15(2):77—82. utilization and costs: Canada and the United States. Crit Care
Dimick JB, Swoboda SM, et al. Effect of nurse-to-patient ra- Med 1990;18:1282—6.
tio in the intensive care unit on pulmonary complica- Jakob SM, Rothen HU. Intensive care 1980—1995: Change in pa-
tions and resource use after hepatectomy. Am J Crit Care tient characteristics, nursing workload and outcome. Inten-
2001;10(6):376—82. sive Care Med 1980-;23:1165—70.
Donchin Y, Gopher D, et al. A look into the nature and causes Karasek RA. Job content instrument: questionnaire and users
of human errors in the intensive care unit. Crit Care Med guide, Revision 1.1. LA. University of Southern California;
1995;23:294—300. 1985.
EROS Equipe de Recherche Operationnelle en Sante. PRN 80: Keene AR, Cullen DJ. Therapeutic intervention scoring system:
La mesure du niveau des soins infirmiers requis. Montreal: Update 1983. Crit Care Med 1983;11:1—3.
Bibliotheque Nationale du Quebec et du Canada; 1981. Keijsers GJ, Schaufeli WB, et al. Performance and burnout in
Evans SA, Carlson R. Nurse-physician collaboration: Solv- intensive care units. Work and Stress 1995;9(4):513—27.
ing the nursing shortage crisis. J Am Coll Cardiol Knaus WA, Draper EA, et al. An evaluation of outcome from
1992;20(7):1669—73. intensive care in major medical centers. Ann Internal Med
Fasting S, Gisvold SE. Data recording of problems during anaes- 1986;104:410—8.
thesia: Presentation of a well-functioning simple system. Le Gall JR, Loirat P, et al. The patients in management of in-
Acta Anaesthesiol Scand 1996;40:1173—83. tensive care. In: Miranda DR, Williams A, Loirat P, editors.
Freeman T, O’Brien-Pallas LL. Factors influencing job sat- Guidelines for Better Use of Resources. Dordrecht: Kluwer;
isfaction on specialty nursing units. Can J Nurs Admin 1990. p. 11—53.
1998;11(3):25—51. Leape LL, Bates DW, et al. Systems analysis of adverse drug
Gebbie K, Levin MA. In: Proceedings of the first national confer- events. J Am Med Assoc 1995;274(1):35—43.
ence on the classification of nursing diagnoses; 1975. Llenore E, Ogle KR. Nurse-patient communication in the inten-
Giraud T, Dhainaut JF, et al. Iatrogenic complications in adult in- sive care unit: a review of the literature. Aust Crit Care
tensive care units: A prospective two-center study. Crit Care 1999;12(4):142—5.
Med 1993;21(1):40—51. Lustbader D, Fein A. Emerging trends in ICU management and
Gopher D, Braune R. On the psychophysics of workload: staffing. Crit Care Clin 2000;16(4).
Why bother with subjective measures? Human Factor Malacrida R, Bomio D, et al. Computer-aided self-observation
1984;26:519—32. psychological stressors in an ICU. Int J Clin Monitor Comput
Griffith CH, Wilson JF, et al. Housestaff workload and procedure 1991;8:201—5.
frequency in the neonatal intensive care unit. Crit Care Med McCloskey JC, McCain BE. Satisfaction, commitment and profes-
1999;27(4):815—20. sionalism of newly employed nurses. Image: J Nurs Scholar-
Groeger JS, Strosberg MA, et al. Descriptive analysis of ship 1987;19(1):20—4.
critical care units in the United States. Crit Care Med Merill JM, Boisaubin EV. Adverse occurences in the intensive care
1992;20:846—83. unit. J Am Med Assoc 1981;245:1214.
Grundmann H, Hori S, et al. Risk factors for the transmission Miranda R, Nap, et al. Nursing Activities Score. Crit Care Med
of methicilin-resistant staphylococcus aureus in an adult in- 2003;31(2):374—82.
tensive care unit: Fitting a model to the data. J Infect Dis Miranda DR, de Rijk A, et al. Simplified therapeutic intervention
2002;185:481—8. scoring system: The TISS-28 items- results from a multicenter
Gunnarsson T, Theodorsson A, et al. Mobile computerized to- study. Crit Care Med 1996;24:64—73.
mography scanning in the neurosurgery intensive care unit: Miranda DR, Moreno R, et al. Nine equivalents of nursing man-
Increase in patient safety and reduction of staff workload. J power use score (NEMS). Intensive Care Med 1997;23:760—5.
Neurosurg 2000;93:432—6. Morales IJ, Peters SG, et al. Hospital mortality rate and length of
Halloran EJ. Nursing workload, medical diagnosis related groups, stay in patients admitted at night to the intensive care unit.
and nursing diagnoses. Res Nurs Health 1985;8:421—33. Crit Care Med 2003;31(3):858—63.
Halpern NA, Bettes L, et al. Federal and nationwide intensive Moreno R, Miranda DR. Nursing staff in intensive care in Eu-
care units and healthcare costs: 1986—1992. Crit Care Med rope: the mismatch between planning and practice. Chest
1994;22:2001—7. 1998;113(3):752—8.
Human factors engineering conceptual framework 301

Moreno R, Miranda DR, et al. Mortality after discharge from Schaufeli W, Le Blanc P. Personnel. In: Miranda DR, Ryan DW,
intensive care: the impact of organ system failure and Schaufeli WB, Fidler V, editors. Organisation and Manage-
nursing workload use at discharge. Intensive Care Med ment of Intensive Care: A Prospective Study in 12 European
2001;2001(27):999—1004. Countries. Springer; 1998. p. 169—205.
Nelson EC, Batalden PB, et al. Microsystems in health care: Part Smith MJ, Carayon-Sainfort P. A balance theory of job design for
1. learning from high-performing front-line clinical units. stress reduction. Int J Industr Ergonom 1989;4:67—79.
Joint Comm J Qual Safety 2002;28(9):472—93. Society of Critical Care Medicine. Critical Care in the United
Nisbett RE, Wilson TD. Telling more than we can know: Valid States. Anaheim, CA: Society of Critical Care Medicine;
reports on mental processes. Psychol Rev 1977;84:231—59. 1992.
Oates PR, Oates RK. Stress and work relationships in the neona- Soeken KL, Prescott PA. Patient intensity for nursing index: The
tal intensive care unit: Are they worse than in the wards? J Measurement model. Res Nurs Health 1991;14:297—304.
Paediatr Child Health 1996;32:57—9. Standards Subcommittee Intensive Care Society. Standards for
Office of Technology Assessment. Health technology case study Intensive Care Units. London: Biomedica Ltd; 1983.
28: Intensive care units (ICUs), clinical outcomes, costs and Stillwell SB, Woletz J, et al. The impact of do-not-resuscitate
decision making. Washington DC: U.S. Congress, Office of orders on nursing workload in an ICU. Am J of Crit Care
Technology Assessment; 1984. 1997;6(5):400—5.
Park KS, Han SW. Performance obstacles in cellular manufactur- Tarnow-Mordi W, Hau C, et al. Hospital mortality in relation to
ing implementation - Empirical investigation. Human Factors staff workload: A 4-year study in an adult intensive-care unit.
Ergonom Manufact 2002;12(1):17—29. Lancet 2000;356:185—9.
Peters LH, O’Connor EJ, et al. The behavioral and affective con- Tarnowski-Goodell T, Van Ess Coeling H. Outcomes of nurses’ job
sequences of performance-relevant situational variables. Or- satisfaction. J Nurs Admin 1994;24(11):36—41.
ganizat Behav Human Perf 1980;25:79—96. Thorens J, Kaelin RM, et al. Influence of the quality of nursing on
Prescott PA, Soeken KL, et al. Measuring patient intensity: a re- the duration of weaning from mechanical ventilation in pa-
liability study. Evaluat Health Profess 1989;12(3):255—69. tients with chronic obstructive pulmonary disease. Crit Care
Pronovost PJ, Jenckes MW, et al. Organizational characteristics Med 1995;23(11):1807—15.
of intensive care units related to outcomes of abdominal aor- Tissot E, Cornette C, et al. Medication errors at the adminis-
tic surgery. J Am Med Assoc 1999;281(14):1310—7. tration stage in an intensive care unit. Intensive Care Med
Reid GB, Shingledecker CA, et al. Development of multidimen- 1999;25:353—9.
sional subjective measures of workload. In: Proceedings of Tsang PS. Mental workload. In: Karwowski W, editor. Interna-
the IEEE International Conference on Cybernetics and Soci- tional Encyclopedia of Ergonomics and Human Factors, 1.
ety; 1981. New York: Taylor & Francis; 2001. p. 500—3.
Robert J, Fridkin S, et al. The influence of the composition of UK Neonatal Staffing Atudy Group. Patient volume, staffing,
the nursing staff on primary bloodstream infection rates in and workload in relation to risk-adjusted outcomes in
a surgical intensive care unit. Infect Control Hosp Epidemiol a random stratified sample of UK neonatal intensive
2000;21(1):12—7. care units: a prospective evaluation. Lancet 2002;359:99—
Rothen HU, Kung V, et al. Validation of ‘‘nine equivalents of nurs- 107.
ing manpower use score’’ on an independent data sample. Vitez TS. A model for quality assurance in anesthesiology. J Clin
Intensive Care Med 1999;25:606—11. Anesthesiol 1990;2:280—7.
Rowan KM, Kerr JH, et al. Intensive Care Society’s APACHE II Wagner DP, Wineland TD, et al. The hidden costs of treating
study in Britain and Ireland-II: Outcome comparisons of inten- severely ill patients: Charges and resource consumption in
sive care units after adjustment for case mix by the APACHE an intensive care unit. Health Care Financ Rev 1983;5:81—
II method. Br Med J 1993;307:977—81. 6.
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.