Quality management systems and clinical outcomes in Dutch nursing homes

Wagner, Cordula; Klein Ikkink, Karen; Wal, Gerrit van der; Spreeuwenberg, Peter; Bakker,
Dinny Herman de; Groenewegen, Peter Paulus
Published in:
Health Policy

DOI:
10.1016/j.healthpol.2005.03.010

IMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to
cite from it. Please check the document version below.

Document Version
Publisher's PDF, also known as Version of record

Publication date:
2006

Link to publication in University of Groningen/UMCG research database

Citation for published version (APA):

Wagner, C., Klein Ikkink, K., Wal, G. V. D., Spreeuwenberg, P., Bakker, D. H. D., & Groenewegen, P. P.
(2006). Quality management systems and clinical outcomes in Dutch nursing homes. Health Policy, 75(2),
230. DOI: 10.1016/j.healthpol.2005.03.010

Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the
author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).

Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately
and investigate your claim.

Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the
number of authors shown on this cover page is limited to 10 maximum.

Download date: 26-04-2017

 Health Policy 75 (2006) 230–240

Quality management systems and clinical outcomes in

Dutch nursing homes
Cordula Wagner a,∗ , Karen Klein Ikkink a , Gerrit van der Wal b ,
Peter Spreeuwenberg a , Dinny Herman de Bakker a , Peter Paulus Groenewegen a
a NIVEL, Netherlands Institute of Primary Health Care, Utrecht, P.O. Box 1568, 3500 BN Utrecht, The Netherlands
b EMGO, Institute for Research in Extramural Medicine and Department of Social Medicine, Vrije Universiteit,

Van der Boechorststraat 7, 1081 BT Amsterdam, The Netherlands

Abstract

The objective of the article is to explore the impact quality management systems and quality assurance activities in nursing
homes have on clinical outcomes. The results are based on a cross-sectional study in 65 Dutch nursing homes. The management
of the nursing homes as well as the residents (N = 12,377) participated in the study. Primary survey-data about the implementation
of quality management systems and quality assurance activities were collected in 1994/1995 and in 1998, and were combined
with information on resident characteristics and the prevalence of undesirable clinical outcomes. The results demonstrate that
there are differences between nursing homes in the prevalence of undesirable clinical outcomes. In the nursing homes with the
lowest scores, undesirable outcomes occur approximately 10 times less often than in nursing homes with the highest scores. The
multi-level analysis has demonstrated that the differences in outcomes are mainly caused by differences between residents and,
to some extent, also by differences between nursing homes. Resident characteristics explain 48% of the differences between
residents and 72% of the differences between nursing homes. The size of the nursing home, the involvement of a client council
and the implementation of a quality management system could explain a small part of the remaining variation in the number of
undesirable outcomes. It seems that the implementation of a quality management system and the involvement of a client council
had significant influence on the number of undesirable outcomes. Approximately 50% of the undesirable outcomes cannot be
explained by the selected resident characteristics, the size of the nursing home and the implementation of quality management
systems or quality assurance activities.
© 2005 Elsevier Ireland Ltd. All rights reserved.

Keywords: Quality management systems; Effectiveness; Clinical outcomes; Nursing homes

1. Introduction

This article explores the impact quality management

∗ Corresponding author. Tel.: +31 30 2729712; systems (QMS) and quality assurance (QA) activities
fax: +31 30 2729729. in nursing homes have on clinical outcomes. During
E-mail address: [email protected] (C. Wagner). the past 10 years, nursing homes in various countries

0168-8510/$ – see front matter © 2005 Elsevier Ireland Ltd. All rights reserved.
doi:10.1016/j.healthpol.2005.03.010
 C. Wagner et al. / Health Policy 75 (2006) 230–240 231

have made a start with the systematic implementation earlier research undesirable clinical outcomes, such as
and evaluation of care planning [1–4], practice guide- mortality, pressure sores, incontinence or indwelling
lines [5–9], quality indicators [10,11], client councils catheters, have been used to assess the quality of
and quality management systems [12–15] to improve nursing home care [10,11,23–27]. These clinical
the quality of care provided [16,17] and adapt to the outcomes are undesirable because of their negative
changing needs of nursing home residents [18]. In sev- influence on the health status of residents. In other
eral countries, the government has also imposed legal research, the relationship between quality of care and
requirements to improve the quality of nursing home organizational determinants have been investigated
care. The American Congress, for example, introduced [28–34]. Over the years, research results have shown
the Omnibus Budget Reconciliation Act (OBRA) in that differences in clinical outcomes could partly be
1987. This contains specific guidelines regulating the explained by organizational determinants, such as
use of psychotropic drugs and physical restraints in economic status, size, bed-occupancy, nursing home
long-term care facilities [19,20], and mandates the staffing and environmental determinants, such as per
use of the Resident Assessment Instrument (RAI) for capita income, bed supply and competition [28,33–37].
care planning. Moreover, nursing homes are obliged Quality management systems and QA-activities are
by this act to create a quality improvement team. In designed to improve clinical outcomes for residents by
Canada and Iceland, for instance, the government has improving the process of health care provision. Studies
also mandated the use of the RAI to improve the pro- which have examined the impact of the implementation
cess of care planning and to monitor the quality of of specific QA-activities on the process and outcomes
care [11]. In The Netherlands, the Dutch parliament of care have found some evidence that specific further
introduced the Care Institutions Quality Act in 1996 training of professionals, practice guidelines and
(http://www.minvws.nl). The aim of this Quality Act individual care planning have a positive impact on
is to ensure that the care provided is of a high standard. resident-related outcomes [1,3,38] or staff work satis-
Nursing homes and other health care organizations faction [39]. To date, however there is scarce evidence
must therefore, develop a quality management system that quality management system improve clinical out-
and implement QA-activities. A quality management comes in nursing home residents to any great extent.
system comprises the entire process of formulat- In this article is studied the relationship between
ing requirements, collecting information, assessing quality management systems and the prevalence of un-
outcomes and adjusting policies at all levels of an desirable clinical outcomes. The central research ques-
organization [15]. In The Netherlands, the first nursing tion was: Do nursing homes with a quality management
homes started in 1990 with the development and im- system have less undesirable clinical outcomes than
plementation of more systematic QA-activities, such nursing homes without a quality management system?
as systematic care planning, practice guidelines and a
quality policy for the entire nursing home. In 1995 and
2000 still only 5% of the health care organizations had 2. Method
fully implemented a quality management systems, oth-
ers were still working on the implementation and the 2.1. Sample
systematic improvement of their care processes [21].
The basic assumption underlying the implementa- Sixty-eight nursing homes, representing 20% of all
tion of quality management systems is that effective nursing homes in The Netherlands, participated in the
and efficient care processes will lead to appropriate study. The sampling process consisted of two steps. In
care and positive outcomes. However, in nursing 1994/1995, we took a random sample of 50% (N = 159)
homes it is not always possible to improve or maintain of all Dutch nursing homes to investigate the develop-
the health status of elderly residents. Therefore, the ment of quality management systems in nursing homes;
quality of life of nursing home residents could be the response rate was 75% (N = 120). For the study in
an important outcome measure, but, until now there 1998, we started with the respondents of the 1994/1995
have been no relevant health-related quality of life sample, which would allow us to measure changes in
measures that could indicate appropriate care [22]. In the development of quality management systems and
232 C. Wagner et al. / Health Policy 75 (2006) 230–240

quality assurance activities over time. An additional is a national registry that registers resident characteris-
selection criterion was the availability of standardized tics and some clinical outcomes. The sample included
patient data from the SIG Nursing Home Information 12,377 residents. Resident characteristics included
System (SIVIS). Out of the 120 nursing homes 101 met age, sex, somatic or psychogeriatric diagnosis and
the additional criterion; they received a postal question- functional status. The functional status was obtained
naire on quality management systems and were asked by creating a severity-index by summing an ADL-
to give permission to use anonymous patient data from score and active communication, bed transferring,
SIVIS. Finally, full data were obtained from 65 nurs- bladder and bowel continence, walking and wheelchair
ing homes (response 64%). There were no differences dependency. A score of 0 indicated no functional
found in the average amount of QA-activities that had disabilities and a score of 12 indicated great functional
been implemented in 1994/1995 between respondents disabilities. The 12 items of the severity-index form
and non-respondents. a strong hierarchical scale [41]. The ADL-score was
obtained by summing the amount of help (0=can do
2.2. Quality management systems alone or can do with help; 1=must be done for) required
in five areas (eating, bathing upper or lower part of the
To measure the implementation of quality manage- body, dressing and toileting). A score of 0 indicated
ment systems and the amount of QA-activities a postal independent performance in these areas, and a score
questionnaire was sent to the medical director of the of 5 indicated total dependence. The resident charac-
nursing home, asking for the implementation of QA- teristics and the functional status have been used for
activities (Appendix A). These activities are indicators case-mix adjustment. Earlier research has shown that
of a quality management system. All activities together case-mix adjustment is sometimes necessary to better
represent a quality management system. The QA- understand the results of quality indicators [42–45].
activities were measured in 1994/1995 and 1998. For
this study, we have used two measures: the total amount 2.5. Undesirable outcomes
of QA-activities of an organisation in 1994/1995 and
In this study, five undesirable clinical outcome
the increase of QA-activities between 1994/1995 and
measures, e.g. the prevalence of bladder incontinence,
1998. The Pearson correlation coefficient between
pressure ulcers, urethral catheterization, restricted mo-
these two measures is 0.58 (p = 0.00).
bility and behavioural problems were used. These out-
come measures have been selected in earlier research
2.3. Nursing home characteristics
as quality indicators for nursing homes [10,42,45–47].
In The Netherlands no standardized registration of
A nursing home or a long-term care facility is an in-
health outcomes is obliged, therefore only some
stitution providing nursing care 24 h a day, assistance
outcome measures can be used for research purposes.
with activities of daily living and mobility, psychoso-
In addition, the outcome variable ‘combination of
cial and personal care, paramedical care, as well as
undesirable outcomes’ was constructed by summing up
room and board [40].
the five separate clinical outcome measures. Thereby,
To control for nursing home characteristics that
we counted the prevalence of bladder incontinence or
could influence residents’ outcomes (besides the qual-
the prevalence of indwelling catheterization. From a
ity management system), we have included organi-
care perspective it is easier to treat incontinence by
sation size (total bed capacity) and occupancy rate
catheterization than by a toileting plan, but from a qual-
[18,19].
ity and client-centred perspective catheterization is less
desirable. Therefore, catheterization was weighted for
2.4. Resident characteristics and case mix
two points, the other outcomes for one point.
Data were obtained on all permanent residents liv- 2.6. Data analyses
ing in the nursing homes between September 1997 and
February 1998 from the SIG Nursing Homes Informa- We have used descriptive statistics and multi-level-
tion System. The Nursing Homes Information System analysis to describe the data. The relationship between
 C. Wagner et al. / Health Policy 75 (2006) 230–240 233

quality management systems and clinical outcomes of the residents was incontinent, 27% was restricted
have been examined by multi-level analyses with two in their mobility, 13% showed disturbing behavioural
levels: nursing home level and resident level [48]. Sep- problems and 10% had pressure ulcers or indwelling
arate logistic multi-level analyses were run on the in- catheters. Of the 12,377 residents 38% had none of
dividual (dichotomous) outcome variables to examine the selected undesirable outcomes, 30% had one un-
the predictive effects of nursing homes that have devel- desirable outcome, 22% had two and 10% of the resi-
oped a quality management system, while controlling dents suffered from three or four undesirable outcomes
for differences in case mix. Linear regression multi- (Table 2).
level analyses were run on the variable ‘combination
of undesirable outcomes’, based on the linear associa- 3.2. Implementation of quality management
tion between the number of quality assurance activities systems and QA-activities
and the number of undesirable outcomes. For all tests
the significance level was set to 0.05. Analyses were In 1994/1995 the average amount of QA-activities
performed using spss-X and MLn. in nursing homes was 21 out of 52 (S.D. = 8). Three
years later, the participating nursing homes had imple-
mented 27 QA-activities (S.D. = 8). None of the nurs-
3. Results ing homes had yet implemented all QA-activities that
are conditional for a quality management system. The
3.1. Resident characteristics and clinical number of QA-activities has increased over the past 3
outcomes years with an average of six activities.
The most common QA-activity in nursing homes
Approximately 74% of all 12,377 residents were was the systematic use of care planning (91%). Nearly
women. The average age of residents was 81 years. Of two-third of the nursing homes had implemented qual-
all residents 56% had a psychogeriatric diagnosis such ity documents, such as a quality policy, quality action
as dementia. The overall level of dependency was 7.5 plans, an annual quality report and a quality handbook
on a 12-point-scale. There were only small differences (62%). A client council was active in 63% of the nurs-
in the demographic characteristics and the severity in- ing homes. Less often, nursing homes had implemented
dex of residents in nursing homes that have partici- continuous education for professionals and systematic
pated in this study and other Dutch nursing homes [49] feedback of results as means for quality improvement
(Table 1). (37%). Finally, 37% of the nursing homes had imple-
Most of the residents in the study homes received mented several practice guidelines, such as guidelines
nursing care, 20% received treatment from an allied for specific diagnostic groups, guidelines for medical
health care professional and 7% received specific at- interventions by nurses and the utilisation of medical
tention because of their dementia. Furthermore, 45% equipment.

3.3. Differences in undesirable outcomes between

Table 1 nursing homes
Resident characteristics of participating nursing homes and overall
Dutch nursing homes
Table 3 displays the percentile scores for five
Resident characteristics Residents Residents
undesirable outcomes. The score of a nursing home
participating Dutch nursing
nursing homes, is the percentage of residents that had, for example,
homes, N = 45,645 bladder incontinence. The first row shows that 13% of
n = 12,377 the residents in the best nursing homes (0th percentile)
Diagnosis, % psychogeriatric 56 54 had bladder incontinence. A nursing home with a
Sex, % female 74 73 score of 27% would rank in the top 10%, and a nursing
Age female, mean years 82 82 home with 62% would rank at the 90th percentile and
Age male residents, mean years 77 77
belongs to the 10% nursing homes with a poor score. A
Severity index (scale 0–12), mean 7.5 8
nursing home may have scores at different percentiles
234 C. Wagner et al. / Health Policy 75 (2006) 230–240

Table 2
Description of the dependent and independent variables used in1998 to describe the implementation of quality systems and QA-activities (N = 65
nursing homes), and undesirable resident outcomes (N = 12,377)
Description Characteristics
Quality system
Amount of QA-activities in 1994/1995; (maximum 52) Mean = 21; S.D. = 8
Increase of QA-activities between 1994/1995 and 1998; Mean = 6; S.D. = 8
QA-activities
Systematic use of care planning; dummy (%) 91
Systematic involvement of client council; dummy (%) 63
Quality policy; dummy (%) 62
Further education and feedback; dummy (%) 37
Use of practice guidelines; dummy (%) 37
Organization characteristics
Number of beds Mean = 193; S.D. = 77
Percentage full beds (occupation rate) 99%; S.D. = 2.4
Undesirable outcomes: prevalence of:
Bladder incontinence, n = 5551 (%) 45
Restricted mobility, n = 3302 (%) 27
Behavioural problems, n = 1579 (%) 13
Indwelling catheter, n = 1233 (%) 10
Pressure ulcers, n = 1229 (%) 10
Sum of five undesirable outcomes; scale 0–5
0 undesirable outcomes, n = 4695 (%) 38
1 undesirable outcome, n = 3717 (%) 30
2 undesirable outcomes, n = 2730 (%) 22
3 undesirable outcomes, n = 955 (%) 8
4 undesirable outcomes, n = 169 (%) 2
5 undesirable outcomes, n = 9 (%) 0

for different outcomes. Table 3 illustrates that, in those scoring most poorly on this outcome, reported
some nursing homes 2% of the residents had pressure indwelling catheters for 38% of their residents.
ulcers, whereas in nursing homes scoring at the 90th Overall, for each of these outcomes, there are
percentile 17% of the residents reported pressure nursing homes doing very well and others doing very
ulcers. Similarly, for catheterization, in homes scoring poorly. In the next section, we will try to explain the
at the 10th percentile 3% of residents had an indwelling difference at resident level between nursing homes
catheter, whereas in homes at the 90th percentile, 20% by accounting for variation in resident populations.
had. Nursing homes scoring at the 100th percentile, In addition, we will examine the relationship between

Table 3
Percentile scores for selected undesirable outcomes of 65 nursing homes
Clinical outcomes 0th percentile (minimum 10th percentile 50th percentile 90th percentile 100th percentile (maximum
or best score) (%) (good score) (%) (average score) (%) (poor score) (%) or worst score) (%)
Bladder incontinence 13 27 46 62 65
Restricted mobility 5 17 28 39 51
Behaviour problems 2 5 12 21 29
Pressure ulcers 2 5 10 17 22
Indwelling catheter 1 3 9 20 38
 C. Wagner et al. / Health Policy 75 (2006) 230–240 235

Table 4
The impact of resident characteristics and quality systems on various undesirable outcomes: regression and variance coefficients of logistic
multi-level analyses
Characteristics Bladder Restricted Behaviour Pressure Indwelling
in-continence mobility problems ulcers catheter
Resident characteristics
Age 0.01* (0.00) −0.01* (0.00) −0.01* (0.00) 0.01* (0.00) −0.01* (0.00)
Female 0.06 (0.07) 0.52* (0.09) −0.01 (0.06) −0.08 (0.07) −0.26* (0.07)
Psychogeriatric diagnoses 1.00* (0.07) −1.67* (0.09) 0.70* (0.07) −0.82* (0.07) −1.81* (0.08)
Severity-index 1.01* (0.02) 1.02* (0.03) 0.18* (0.01) 0.26* (0.01) 0.25* (0.01)
Organization characteristics
Number of beds −0.00 (0.00) −0.01 (0.01) −0.00 (0.00) −0.01* (0.001) −0.00 (0.00)
Implementation quality system
Amount of QA-activities 1994/1995 −0.00 (0.01) −0.01 (0.01) −0.02 (0.01) −0.001 (0.008) −0.003 (0.01)
Increase of QA-activities 1994/1995–1998 −0.00 (0.01) −0.01 (0.01) 0.01 (0.01) 0.000 (0.008) −0.001 (0.01)
Variance coefficients
Nursing home level 0.10* (0.03) 0.04* (0.02) 0.23* (0.05) 0.27* (0.06) 0.30* (0.07)
Resident level 0.96 (0.01) 1.88 (0.02) 0.97 (0.01) 1.37 (0.02) 0.93 (0.01)
* p < 0.05.

quality management systems c.q. QA-activities and included in the models used to explain differences
undesirable outcomes. in undesirable outcomes have an independent effect
that exceeds the 0.05 significance level. An excep-
3.4. Multi-level analyses tion can be made for gender; no relationships were
found between gender and incontinence, behavioural
In Table 4, regression coefficients are presented for problems and pressure ulcers. Of the nursing home
each of the outcome variables included in the analyses. characteristics the number of beds has a relationship
By including resident characteristics we can establish with the prevalence of pressure ulcers. Residents in
whether there are differences in the resident population larger nursing homes have less often pressure ulcers.
with respect to relevant characteristics, which possibly The implementation of quality management systems in
influence the nursing home scores on undesirable 1994/1995 and the increase of QA-activities over the
outcomes. Also included in the table are characteristics last 3 years had no independent effect on the outcomes.
of nursing homes, such as the total bed capacity and The variance coefficients illustrates that after
the amount of QA-activities in 1994/1995 and the including the independent variables, there remain
increase of QA-activities over the period 1994/1995 significant differences between residents and nursing
and 1998, representing the implementation of quality homes. These differences cannot be explained by the
management systems The occupancy rate has not been used resident and organisation characteristics.
included in the analyses because of the little difference Table 5 presents the effect of resident characteris-
in occupancy between nursing homes (mean = 99%, tics, organisation characteristics, the implementation
S.D. = 2.4). By including variables at the organisation- of quality management systems and separate QA-
level we can establish whether some of the differences activities on the amount of undesirable outcomes.
found between residents and nursing homes can be ex- Furthermore, Table 5 presents the estimated vari-
plained by the implementation of quality management ances for the ‘null’ (no predictor variable) model
systems or the size of the nursing home. and the applied (with stepwise inclusion of all
Most of the variances (95%, not in Table 4) in all predictor variables) models. If the variances of the
analysis have been found between residents. However, null-model are entered in the formula for calculating
there were also significant differences between nursing the intraclass-correlation, the percentage of variance
homes (5%, not in Table 4). The resident characteristics at level 2 (nursing home) is 4.7% of the total variance
236 C. Wagner et al. / Health Policy 75 (2006) 230–240

Table 5
The effect of resident characteristics and quality systems on the amount of undesirable outcomes: regression and variance coefficients and
explained variance of linear multi-level analyses
Risk factor 0 model A model B model C model D model
Intercept 1.06(0.08) −0.49 (0.06) −0.38 (0.07) −0.31 (0.08) −0.32 (0.08)
Resident characteristics
Age −0.003* (0.000) −0.003* (0.001) −0.003* (0.001) −0.003* (0.001)
Female 0.03* (0.01) 0.03* (0.01) 0.03* (0.01) 0.03* (0.01)
Psychogeriatric care −0.13* (0.02) −0.13* (0.02) −0.13* (0.02) −0.13* (0.02)
Severity-index 0.24* (0.00) 0.24* (0.00) 0.24* (0.00) 0.24* (0.00)
Organization characteristics
Number of beds −0.0005* (0.0002) −0.0004* (0.0002) −0.0004* (0.0002)
Implementation quality system
Amount of QA-activities 1994/1995 −0.005* (0.002)
Increase QA-activities 1994/1995–1998 −0.002 (0.002)
QA-activities
Systematic use of care planning −0.06 (0.05)
Quality policy −0.02 (0.03)
Further education and feedback −0.02 (0.03)
Use of practice guidelines 0.05 (0.04)
Involvement of client council −0.06* (0.03)
Variance coefficients
Nursing home 0.05* (0.01) 0.012* (0.00) 0.0108* (0.00) 0.0102* (0.00) 0.009* (0.00)
Resident 1.01(0.01) 0.54(0.01) 0.54 (0.01) 0.54 (0.01) 0.54 (0.01)
Reduction of variance
Nursing home level (%) 72 9.5 5 12.5
Resident level (%) 48 0.3 0.1 0.3
* p < 0.05.

((0.05/0.05 + 1.01) × 100) [50]. So, 95.3% can be D), such as systematic use of care planning, a quality
labelled as variance on level 1 (resident). policy, further education of professionals and the use
On the resident level, the applied A-model for of practice guidelines, had no effect on the amount of
resident characteristics, explained 48% of the 95% undesirable outcomes. However, a relation was found
variance due to between resident differences. The 5% between the involvement of client councils and the
variance between nursing homes could, to a great extent amount of undesirable outcomes. Thus, after including
(72%), be explained by differences between resident the independent variables there remain significant
characteristics. In model B–D, the independent vari- differences in the amount of undesirable outcomes
ables of the nursing home level are included. In model between residents. The remaining differences between
B, the size (number of beds) of the nursing homes is in- nursing homes are significant, but relative small.
cluded. Size has an effect on the amount of undesirable
outcomes. The reduction in variance compared to the
A model is 0.3% at resident level and 9.5% at nursing 4. Discussion
home level. In model C, the implementation of quality
management systems have an independent effect on the The objective of this study was to determine the ex-
amount of undesirable outcomes. The variance at resi- tent to which the differences found in outcomes can
dent level was, additionally, reduced by 0.1%, whereby be explained by the existence of a quality management
the variance at nursing home level was reduced by 5%. system in the nursing homes. Quality management sys-
The implementation of separate QA-activities (model tems were chosen as determinant because the objective
 C. Wagner et al. / Health Policy 75 (2006) 230–240 237

of these systems is to systematically attune the policy of management system have better clinical outcomes
the organization and the care process to the needs of the than nursing homes without a quality management
residents. This implies that effective and efficient care system. The results of the multi-level analysis have
processes should result in optimal care for residents, demonstrated that the differences in outcomes are
and thus, the best clinical outcomes as possible. mainly caused by differences between residents and,
From the results of the study it is apparent that, of the to some extent, also by differences between nursing
52 quality assurance activities studied, nursing homes homes. At resident level, characteristics such as
had implemented an average of 21 in 1994/1995 and gender, psychogeriatric diagnoses and the dependence
27 in 1998. The number of activities per nursing home of the resident explain 48% of the differences between
in 1998 varied from 11 to 48, indicating that only a residents and 72% of the differences between nursing
few of the nursing homes had implemented an inte- homes. On the other hand, this implies that approxi-
gral quality management system. This implies that the mately 50% of the differences in undesirable outcomes
results of the study must be interpreted with caution, cannot be explained by the selected case-mix variables,
because a quality management system which has not and that there are therefore other underlying causes.
been fully implemented could have less effect on the The size of the nursing home can explain a small
results. Moreover, there are certain shortcomings in us- part of the remaining variation in some undesirable
ing the number of activities as a measure for the im- outcomes. Residents in larger nursing homes have
plementation of a quality management system, because less undesirable outcomes. Finally, it seems that the
the existence, for instance, of a quality policy, a quality implementation of a quality management system in
manual or practice guidelines is less important in the 1994/1995 and the involvement of a client council had
achievement of positive results than the implementa- significant influence on the number of undesirable out-
tion of the activities at all levels in the nursing home. comes in 1998. There was a small reduction in the
In the present study, these limitations have been ac- variance at resident and nursing home level. On the
counted for by asking about quality activities, which other hand, the increase in the number of activities
apply to various aspects of a quality management sys- themselves and individual quality assurance activities
tem (i.e. policy and strategy, personnel management, (i.e. care planning, quality policy, further education and
process management and client involvement). These practice guidelines) appeared to have no independent
aspects have been derived from the Baldrige Quality influence on the outcomes. This could indicate that
Award and the European Quality Award [15]. quality management systems are not effective or that
The results of this study clearly demonstrate that the results of quality management systems only be-
there are differences between nursing homes in the come apparent in the long term, and that it takes some
prevalence of undesirable outcomes. The differences time before a quality management system influences
can be seen in the five outcome measures studied. In the care process, the behaviour of the carers and the
the nursing homes with the lowest scores, undesir- clinical outcomes. This could also imply that individ-
able outcomes occur approximately 10 times less often ual quality assurance activities have no influence on
than in the nursing homes with the highest scores. In undesirable outcomes, whereas a ‘system’ is more ad-
an American study, among 352 nursing homes even vantageous for the quality of care provided for the resi-
greater differences in the five undesirable outcomes dents. It is not the existence of quality activities, which
have been found [24]. Nevertheless, based on these leads to improvements, but the systematic application
data, no foregone conclusion can be drawn that nursing of these activities. A third possible explanation is that
homes with high scores provide sub-optimal care. Dif- nursing homes, which already have effective care pro-
ferences between residents, with regard to the intensity cesses implement a quality management system to be-
of care needed, could explain the differences in out- come even better. In this case, not the implementation
comes. These outcomes clearly demonstrate that it is of a quality management system determines whether
important to carry out further research on the possible nursing homes have less undesirable outcomes, but the
reasons for these differences. already existing level of care provided. A longitudinal
The central research question addressed in this research design could give more insight, but is more ex-
article was whether nursing homes with a quality pensive and time consuming. From the results of this
238 C. Wagner et al. / Health Policy 75 (2006) 230–240

study, it can be concluded that approximately 50% of

assessment of the targets

Systematic participation
in commit-tees, projects
development of criteria/
committees projects the

and in the setting up of

the differences in outcomes between residents cannot

Patient is not involved

Discussions of results

be explained by differences in the resident population,

criteria/ protocols
Participation in:
Participation of
clients/patients

the size of the nursing home, or the existence of a qual-

ity management system or individual quality assurance
protocols
achieved

activities. Future research to investigate other possible

explanations, possibly involving a more specific and
comprehensive study of nursing home policies and the
Committees record of complaints

Management information system

existing care processes, is of vital importance in order

Testing single/multidisciplinary
Process control by sub-systems

internal audit interinstitutional

to identify the basic elements, which are of influence in

client council job assessment

Satisfaction research needs

preventing undesirable resident outcomes and improv-

ing the quality of care.
Quality assurance activities as indicators for the development stage of a quality management system.

interviews
care plans

analysis

testing

References

[1] Phillips CD, Zimmerman D, Bernabei R, Jonsson PV. Using

the Resident Assessment Instrument for quality enhancement
cooperation with other
Protocols for specific

in nursing homes. Age and Ageing 1997;26-S2:77–81.

Information specific

unforeseen activity
Process control by

[2] Hawes C, Mor V, Phillips CD, Fries BE, Morris JN, Stelle-
Critical moments

Routing patient
standardization

Friedlob E, et al. The OBRA-87 nursing home regulations and

target groups,

medical aids

implementation of the Resident Assessment Instrument: ef-

institutions
treatment

fects on process quality. Journal of American Geriatrics Society

1997;45:977–85.
[3] Fries BE, Hawes C, Morris JN, Phillips CD, Mor V, Park PS. Ef-
fect of the national Resident Assessment Instrument on selected
training staff participation

indicated by management
Encouraging professional

health conditions and problems. Journal of American Geriatrics

Human resources policy

Training group leaders

management monitors

quality policy training

Society 1997;45:994–1001.
during working hours

selection of new staff

Systematic feedback
priorities relating to

[4] Mor V, Intrator O, Fries BE, Phillips C, Teno J, Hiris J, et al.

Management tests

Changes in hospitalization associated with introducing the Res-

development

ident Assessment Instrument. Journal of American Geriatrics

Society 1997;45:1002–10.
new staff

[5] Levine JM, Totolos E. A quality-oriented approach to pres-

sure ulcer management in a nursing facility. The Gerontologist
1994;34(3):413–7.
[6] Suntken G, Starr B, Ermer-Seltun J, Hopkins L, Preftakes D.
for some departments

Annual quality report

Quality working plan

Implementation of a comprehensive skin care program across

working plan quality

quality working plan

institutional quality
Policy and strategy

for all departments

care settings using the AHCPR pressure ulcer prevention and

Mission product

quality manual

treatment guidelines. Ost/Wound Man 1996;42(2):20–32.

Quality policy

[7] Moseley CB. The impact of federal regulations on urethral

description

catheterization in Virginia nursing homes. American Journal

profiles

of Medical Quality 1996;11(4):222–6.

[8] Semla TP, Palla K, Poddig B, Brauner DJ. Effect of the Om-
nibus Reconciliation Act 1987 on antipsychotic prescribing in
nursing home residents. Journal of American Geriatrics Society
Orientation/consciousness

approach/establishment

1994;42:648–52.
Implementation stage

[9] Shorr RI, Fought RL, Ray WA. Changes in antipsychotic drug
use in nursing homes during impelementation of the OBRA-
Preparation stage
Appendix A

87 regulations. Journal of the American Medical Association

1994;271(5):358–62.
Systematic

[10] Zimmerman DR. Improving nursing home quality of care

Stages

through outcomes data: the MDS quality indicators. Interna-

tional Journal of Geriatrics Psychiatry 2003;18(3):250–7.
 C. Wagner et al. / Health Policy 75 (2006) 230–240 239

[11] Jensdottir AB, Rantz M, Hjaltadottir I, Gudmundsdottir H, nursing and long-term care settings: evolving targets for qual-
Rook M, Grando V. International comparison of quality indica- ity improvement. American Journal of Geriatric Psychiatry
tors in United States. Icelandic and Canadian nursing facilities. 2004;12(3):288–95.
International Nursing Review 2003;50(2):79–84. [28] Davis MA. On nursing home quality: a review and analysis.
[12] Wagner C, de Bakker DH, Sluijs EM. Kwaliteitssystemen in Medical Care Review 1991;48(2):129–66.
instellingen: de stand van zaken in 1995 (Quality management [29] Flood AB. The impact of organizational and managerial factors
systems in health care organizations: state of the art 1995). on the quality of care in health care organizations. Medical Care
Utrecht: NIVEL; 1995. Review 1994;51(4):381–428.
[13] Casparie AF, Sluijs EM, Wagner C, de Bakker DH. Quality [30] Steffen TM. Organizational determinants of service quality in
management systems in Dutch health care institutions. Health nursing homes. Hospital and Health Services Administration
Policy 1997;42:255–67. 1997;42(2):179–91.
[14] Zinn JS, Weech RJ, Brannon D. Resource dependence and in- [31] Zinn JS, Mor V, Castle N, Intrator O, Brannon D. Organi-
stitutional elements in nursing home TQM adoption. Health zational and environmental factors associated with nursing
Services Research 1998;33(2):261–73. home participation in managed care. Health Services Research
[15] Wagner C, de Bakker DH, Groenewegen PP. A measuring in- 1999;33(6):1753–67.
strument for evaluation of quality systems. International Journal [32] Hughes CM, Lapane KL, Mor VM. Influence of facility charac-
for Quality in Health Care 1999;11(2):119–30. teristics on use of antipsychotic medications in nursing homes.
[16] Rantz MJ, Zwygart-Stauffacher M. Back to the fundamentals of Medical Care 2000;38(12):1164–73.
care: a roadmap to improve nursing home care quality. Journal [33] Zinn JS, Mor V, Intrator O, Feng Z, Angelelli J, Davis JA.
of Nursing Care Quality 2004;19(2):92–4. The impact of the prospective payment system for skilled
[17] Brazil K, Royle JA, Montemuro M, Blythe J, Church A. Moving nursing facilities on therapy service provision: a transaction
to evidence-based practice in long-term care: the role of a Best cost approach. Health Services Research 2003;38(6):1467–
Practise Resource Centre in two long-term care settings. Journal 85.
of Gerontological Nursing 2004;30(3):14–9. [34] Bostick JE. Relationship of nursing personnel and nursing home
[18] Bishop CE. Where are the missing elders? The decline care quality. Journal of Nursing Care Quality 2004;19(2):130–6.
in nursing home use, 1985 and 1995. Health Affaires [35] Zinn JS, Aaronson WE, Rosko MD. Variations in the outcomes
1999;18(4):146–55. of care provided in Pennsylvania nursing homes. Medical Care
[19] Castle NG. Differences in nursing homes with increasing 1993;31(6):475–87.
and decreasing use of physical restraints. Medical Care [36] Brannon D, Zinn JS, Mor V, Davis J. An exploration
2000;38(12):1154–63. of job, organizational, and environmental factors associated
[20] Castle NG. Nursing homes with persistent deficiency citations with high and low nursing assistant turnover. Gerontologist
for physical restraint use. Medical Care 2002;40(10):868–78. 2002;42(2):159–68.
[21] Sluijs EM, Wagner C. Progress in the implementation of quality [37] Schnelle JF, Simmons SF, Harrington C, Cadogan M, Gar-
management systems in health care: 1995–2000. International cia E, Bates-Jensen B. Relationship of nursing home staffing
Journal for Quality in Health Care 2003;15(3):223–34. to quality of care. Health Services Research 2004;39(2):225–
[22] Treurniet HF, Essink-Bot ML, Mackenbach JP, vander Maas PJ. 50.
Health-related quality of life: an indicator of quality of care? [38] Fitzgerald RP, Shiverick BN, Zimmerman D. Applying perfor-
Quality of Life Research 1997;6:363–9. mance measures to long term care. Journal on Quality Improve-
[23] Zinn JS, Aaronson WE, Rosko MD. The use of standardized ment 1996;22(7):505–17.
indicators as quality improvement tools: an application in Penn- [39] van Harten WH, Casparie AF, Fisscher OAM. The evaluation
sylvania nursing homes. American College of Medical Quality of the introduction of a quality management system: a process-
1993;8(2):72–8. oriented case study in a large rehabilitation hospital. Health
[24] Rantz MJ, Mehr DR, Conn VS, Hicks LL, Porter R, Madson Policy 2002;60:17–37.
RW, et al. Assessing quality of nursing home care: the founda- [40] Ribbe MW, Ljunggren G, Steel K, Topinkova E, Hawes C,
tion for improving resident ourcomes. Journal of Nursing Care Ikegami N, et al. Nursing homes in 10 nations: a compari-
Quality 1996;10(4):1–9. son between countries and settings. Age and Ageing 1997;26-
[25] Bates-Jenson BM, Cadogan M, Osterweil D, Levy-Storms L, S2:3–12.
Jorge J, Al-Samarrai N, et al. The minimum data set pressure ul- [41] Van Drunen PHC, Van Montfort APAW. Een analyse van de
cer indicator: does it reflect differences in care processes related kostenstructuur van Nederlandse Verpleeghuizen (An analyses
to pressure ulcer prevention and treatment in nursing homes? on the coststructure of nursing homes in The Netherlands). G&S
Journal of American Geriatric Society 2003;51(9):1203–12. 1981;2(4):246–61.
[26] Wipke-Tevis DD, Williams DA, Rantz MJ, Popejoy LL, Mad- [42] Berlowitz DR, Bandeis GH, Morris JN, Ash AS, Anderson JJ,
son RW, Petroski GF, et al. Nursing home quality and pressure Kader B, et al. Deriving a risk-adjustment model for pressure
ulcer prevention and management practices. Journal of Ameri- ulcer development using the Minimum Data Set. Journal of
can Geriatric Society 2004;52(4):583–8. American Geriatric Society 2001;49(7):866–71.
[27] Boyle VL, Roychoudhury C, Beniak R, Cohn L, Bayer A, [43] Beck K, Spycher S, Holly A, Gardiol L. Risk adjustment in
Katz I. Recognition and management of depression in skilled- Switzerland. Health Policy 2003;65(1):63–74.
240 C. Wagner et al. / Health Policy 75 (2006) 230–240

[44] Mukamel DB, Watson NM, Meng H, Spector WD. Develop- [47] Bula CJ, Ghilardi G, Wietlisbach V, Petignat C, Francioli P.
ment of a risk-adjusted urinary incontinence outcome measure Infections and functional impairment in nursing home residents:
of quality for nursing homes. Medical Care 2003;41(4):467– a reciprocal relationship. Journal of American Geriatric Society
78. 2004;52(5):700–6.
[45] Bours GJ, Halfens RJ, Berger MP, Huijer Abu-Saad H, Grol RT. [48] Goldstein H. Multilevel statistical models. 2nd ed. New York:
Development of a model for case-mix adjustment of pressure Haltsted Press; 1995.
ulcer prevalence rates. Medical Care 2003;41(1):45–55. [49] SIG Jaarboek verpleeghuizen (SIG Yearly Review of Nursing
[46] Holroyd-Leduc JM, Mehta KM, Covinsky KE. Urinary incon- Homes). Utrecht: SIG: Dutch Center of Health Care Informa-
tinence and its association with death, nursing home admission, tion; 1998.
and functional decline. Journal of American Geriatric Society [50] Snijders TAB, Bosker RJ. Modelled Variance in two-level mod-
2004;52(5):712–8. els. Sociological Methods & Research 1994;22(3):342–63.