WHO/RHR/05.

14

Reproductive Health and Research Effective Health Care Alliance Programme Population Council

Evidence-led
obstetric care
Report of a WHO meeting

Geneva, Switzerland, 28-30 January 2004

Contributors William Adu-Krow, Heather Brown, Guillermo Carroli, Paul Garner, Jeremy Grimshaw,
Metin Gülmezoglu, Justus Hofmeyr, Ardi Kaptiningsih, Gunta Lazdane, Ana Langer, Therese Lesikel,
Helen Lugina, Pisake Lumbiganon, Hatem El-Din N Hassan Mohamed, Andy Oxman, Helen Smith,
Yvonne Thomas, Qian Xu.

Writing Group Helen Smith, Metin Gülmezoglu, Paul Garner

Funding The meeting was funded by the Department of Reproductive Health and Research, WHO, and the
Department for International Development, UK (through the Effective Health Care Alliance Programme).

© World Health Organization 2005

All rights reserved. Publications of the World Health Organization can be obtained from Marketing and Dissemination,
World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel: +41 22 791 2476; fax: +41 22 791 4857;
email: [email protected]). Requests for permission to reproduce or translate WHO publications – whether for sale or
for noncommercial distribution – should be addressed to Publications, at the above address (fax: +41 22 791 4806; email:
[email protected]).

The designations employed and the presentation of the material in this publication do not imply the expression of any
opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city
or area or of its authorities, or concerning the delimitation of its frontiers or boundaries.
The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or
recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors
and omissions excepted, the names of proprietary products are distinguished by initial capital letters.

The World Health Organization does not warrant that the information contained in this publication is complete and
correct and shall not be liable for any damages incurred as a result of its use.
 Contents

Summary 1

1. Introduction 3

1.1 Objectives 3

1.2 Evidence-based reproductive health 3

1.3 Methods for increasing evidence-led obstetric care 6

Delineating the gaps 6

Changing practice 8

Influencing policy 9

2. Changing practice: lessons from 4 projects 13

2.1 The projects 13

RHL trial 14
Labour support study 14

Better Births Initiative pilot project 14

Position during labour study 14

2.2 Lessons learnt and implications for research 15

Lessons learnt 15

Implications for research 16

Implications for practice 17

3. Changing policy: lessons from 3 initiatives 19

3.1 The initiatives 19

China dissemination programme 20

Promoting the new WHO antenatal care model in Thailand 20

Promoting the new WHO antenatal care model in Argentina 21

3.2 Lessons learnt 21

4. Strategic options 23

4.1 Using accreditation 23

4.2 Formal linear approaches 24

4.3 Training 24

4.4 Test evidence-based medicine teaching methods 24

4.5 Informal approaches 26

4.6 Increase political commitment 26

4.7 Involve international donors 27

Annex 1 Evidence-based health worker behaviour change 29

Annex 2 Abstracts of RHL-related projects 33

Annex 3 Meeting participants 39

References 43
iv Evidence-led obstetric care: strategies to change practice and policy
 Summary 1

Introduction
The present report is based on a meeting entitled “The RHL and beyond: influencing
policy and changing practice”, held in Geneva on 28–30 January 2004 under the
auspices of the Department of Reproductive Health and Research of the World Health
Organization. The meeting focused on ways of promoting evidence-based care in the
area of pregnancy and childbirth, using the WHO Reproductive Health Library (RHL)
as a source of systematic review evidence. Researchers and clinicians attending this
meeting analysed various approaches and projects which had been implemented
to promote the RHL as a tool for policy-making and practice change. The meeting
reviewed the lessons learnt from such experiences and strove to determine how they
could be used to inform future strategy.

Meeting objectives
The meeting had three objectives: (i) to draw lessons from initiatives promoting
evidence-based reproductive health in low- and middle-income countries; (ii) to
combine these lessons with existing knowledge on effective behaviour change to
guide future implementation initiatives; and (iii) to recommend strategies and tools for
clinicians, administrators and policy-makers for introducing and sustaining evidence-
based approaches.

Lessons about changing practice

Four dissemination and good-practice-promotion projects were discussed: (i) a multi-
centred trial to evaluate teaching RHL using interactive workshops and training
materials; (ii) a before-and-after evaluation of RHL training combined with an
educational intervention in South Africa; and (iii) two before-and-after evaluations
using a combination of audit and educational outreach focused on specific
interventions.
The analysis revealed that there is insufficient evidence to recommend that the specific
strategies used in the four projects be implemented at a national scale. Initiatives
to change practice and improve health care outcomes often require integrated
programmes designed to change both the health care system and organization of
services, in addition to interventions targeting individual clinicians.

Lessons about changing policy

Three policy initiatives were also presented and analysed at the meeting: one
sought to increase commitment to evidence-based approaches in China, and two
involved promoting adoption of the new WHO antenatal care model in Thailand and
Argentina. Meeting participants concluded that it is difficult to systematically evaluate
the promotion of policy change due to its complexity, but that clear messages and
effective dialogue can facilitate the translation of evidence into policy.
2 Evidence-led obstetric care: strategies to change practice and policy

Recommendations and conclusions

To develop further and sustain the achievements already made in evidence-based
reproductive health, providers should focus on behaviour-change strategies relevant
to local settings. For example, accreditation may be an option in some contexts (South
Africa), while reliance on formal, linear strategies to evaluate interventions aimed at
changing clinician behaviour may be better in others. Some participants considered
continuous, multi-faceted promotion of evidence-based care in undergraduate and
continuing education programmes as an appropriate strategy (China, Thailand, the United
Republic of Tanzania). Suggestions were made on how to increase political commitment
and enlist the involvement of international donors.
 1. Introduction 3

1.1 Objectives
Research synthesis which provides a critical evaluation and summary of reliable
research on the benefits and harms of health care interventions, constitutes a vital
source of evidence-based knowledge.

The World Health Organization strives to promote evidence-based practices,

particularly in the area of pregnancy and childbirth. As part of this effort, the WHO
Department of Reproductive Health and Research has, in collaboration with the
Cochrane Collaboration, developed the WHO Reproductive Health Library (RHL), and
is working closely with institutions and collaborators throughout the world to promote
the use of the findings of systematic reviews in clinical practice and policy.

In the light of the uncertainties around how best to introduce and mainstream
evidence-based approaches at local and national levels, WHO held a meeting
to consider and review some of the training and dissemination projects involving
RHL. The meeting brought together people in the field working in these areas and
evaluation specialists to analyse the activities to date.

The meeting had three objectives:

To draw lessons from initiatives designed to promote evidence-based
reproductive health in low- and middle-income countries.

To combine these lessons with existing knowledge on effective behaviour

change in order to guide future implementation initiatives.

To recommend strategies and tools for clinicians, health administrators and

policy-makers for introducing and sustaining evidence-based approaches.

1.2 Evidence-based reproductive health

Systematic reviews: The first systematic summaries of interventions tested in
randomized controlled trials were in the area of pregnancy and childbirth. These
summaries were published as the Oxford Database of Perinatal Trials, and served as
the basis for Effective Care in Pregnancy and Childbirth,1 the first textbook to draw
almost exclusively on systematic reviews. The database and the book were very
influential in high-income countries; for example, they were instrumental in the
Government of the United Kingdom’s decision to institute an evidence-based health
service. This led to the creation of the Cochrane Centre, to the extension of the Oxford
Database to other medical specializations, and, finally, to the establishment of the
Cochrane Collaboration.

Changing practice: The principle of evidence-based health care policy and practice
is disarmingly simple, but the reality of interpreting the evidence and using it to
improve policy and practice is more complicated. Programmes to improve quality
4 Evidence-led obstetric care: strategies to change practice and policy

Framework for dissemination and implementation of evidence-based medicine

Figure 1.1

Level 1 Awareness raising

Purpose Increase awareness about effective interventions and the potential gains from
using research based knowledge in policy and practice
Activities Produce and publish relevant systematic reviews in a variety of professional
and consumer publications
Communicate potential relevance of systematic reviews to current practice,
with examples through commentaries
Level 2 Targeting groups and individuals responsible for implementation
Purpose Identify target groups and individuals with specific roles in implementing
research-based knowledge in practice
Activities Identify target groups, such as health ministry policy-makers, donor aid
advisers, professional groups, managers with responsibility for clinical and
public health policy
Communicate results from systematic reviews and their implications for
practice face-to-face and through short summaries
Give examples of how others have used systematic reviews combined with
audit to change practice for the better in their own hospital or practice
Make people aware of the evidence base for effective practice change

Level 3 Pilot and innovation projects

Purpose Support individuals in specific pilot projects to evaluate potential ways to
implement change in practices that seem to run contrary to current, available
evidence
Activities Identify collaborators engaged or interested in developing pilot projects to
implement research findings and opportunities to make care more evidence
based
Help them to stimulate change (such as by audit and feedback or by means
of opinion leaders) to practices for which systematic reviews of effectiveness
provide reliable evidence
Ensure collaborators monitor change in policy and practice
Level 4 National or institutional policies for evidence-based decisions
Purpose Encourage national governments, institutions, or donors to commit to evidence-
based approaches, with effective implementation and monitoring systems
Activities Work with government and donors in establishing or strengthening health
technology assessment offices or similar bodies at national level
Encourage national policies for evidence-based guidelines, with management
systems to ensure that guidelines are implemented and monitored
Help institutions to train doctors, nurses, and other health staff to deliver
training in evidence-based approaches

Adapted from York Centre for Reviews and Dissemination

 Introduction 5

of care, clinical care guidelines and medical education curricula are now drawing on
systematic summaries of reliable research about the benefits and harmful effects of
interventions, yet the most effective way of bringing this information to bear is still being
debated.2

Dissemination: For many health professionals in low- and middle-income countries,

evidence-based approaches are something relatively new. Some Universities and leaders
in their fields are avidly promoting evidence-based health care, but precisely how the
concepts and information can be woven into day-to-day decision-making remains unclear.
The dissemination process has several levels—simple, passive dissemination, targeted
dissemination, and active demonstration projects to show change is possible. Similarly,
the dissemination process can involve persuading individuals and institutions to adopt
evidence-based approaches, leading to the embedding of this knowledge and its use for
decision-making in mainstream medicine. All of this can be viewed as dissemination
(Figure 1.1). However, there is no hierarchy to these levels, and highly motivated
individuals or institutions often begin with projects aimed at developing capacity for
changing practice.

The WHO Reproductive Health Library (RHL): The World Health Organization
publishes and disseminates RHL, an electronic package of Cochrane Systematic Reviews
in reproductive health of interest to low-income countries that includes commentaries by
professionals working in low-income settings on the relevance of review findings, as well
as training materials (Box 1). Its aim is to increase access to evidence and help promote
change towards evidence-based reproductive health; over 32 000 copies are distributed
annually.

Box 1
The WHO Reproductive Health Library
The Reproductive Health Library (RHL) is an annual electronic review journal that focuses
on evidence-based solutions to reproductive health problems in low- and middle-income
countries. RHL includes Cochrane reviews and corresponding commentaries with practical
recommendations.
Editorials Provide information and expert views on reproductive health and evidence-
based decision-making in developing countries
Beneficial and Categorizes interventions according to the level of evidence, and helps
harmful care policy-makers and clinicians make sound practical decisions

Expert Comment on the relevance of systematic review findings, and practical

commentaries aspects with management recommendations

Implementation Materials designed to help health professionals use research evidence in

aids practice

Research Scientific articles on research synthesis methodology, which facilitate the

methodology interpretation of the results of systematic reviews and randomized controlled
series trials

Internet links Links to evidence-based medicine web sites and a register of funding
agencies, NGOs and other organizations in the reproductive health field
6 Evidence-led obstetric care: strategies to change practice and policy

The Department of Reproductive Health and Research at WHO has promoted a variety of
approaches to encourage people to use the RHL as a driver for change in practice. These
include formal multicentre trials to measure the impact of RHL training packages intended
to support health professionals, small-scale demonstration projects to engage individuals
on the potential value of systematic reviews to improve care, and simple observational
studies documenting how obstetric practice in a particular setting may vary from what
research evidence suggests is best practice.

1.3 Methods for increasing evidence-led obstetric care

Delineating the gaps

Sometimes clinical or public health practice is consistent with best available evidence.
For example, a national caesarean section audit in England and Wales found prophylactic
antibiotics were administered to 87% of women who delivered by emergency caesarean
section.3

Clinical or public health practices which are not consistent with best available evidence
do patients a disservice by providing care that may cause harm. For example, the
systematic review of episiotomies suggests that women randomized to a policy of
performing an episiotomy routinely have worse outcomes than those randomized to a
policy of avoiding episiotomies when possible.4

Harmful practices also include giving patients a drug that has some benefit, but is not as
effective as another treatment. For example, diazepam may be used to treat women with
eclampsia, but is not as effective as magnesium sulfate in preventing further eclamptic fits5;
clinicians giving diazepam but withholding a known, better treatment are actually harming
women.

Interventions involving expenditure of considerable resources, but which are of little

benefit, reduce the amount of resources available for other activities, which, from a
public health viewpoint, is the equivalent of doing harm. This includes high technology
interventions, for example, electronic fetal heart rate monitoring8 and frequent antenatal
care visits. Research evidence shows that small reductions in the number of antenatal visits
in favour of goal-oriented, evidence-based activities are compatible with similar good
perinatal outcomes as in the standard model; services that routinely provide the higher
number of visits for low-risk women are wasting resources that could be used elsewhere.9
Table 1.1 summarizes the systematic review findings for these interventions.
Identifying practices which are inconsistent with available evidence can help pinpoint
areas in which professional changes are needed. Changing practices will benefit mothers
and their infants, qualitatively in terms of better health and quantitatively, in terms of the
number of women who had not been receiving optimal care.
 Introduction 7

Systematic review findings for selected obstetric interventions

Table 1.1
Intervention Question Systematic review summary Conclusions

Episiotomy What are the benefits Six trials (n=5000 women) There is clear evidence
for vaginal and risks of restrictive Restrictive policy resulted in to recommend restrictive
birth versus routine less perineal trauma (RR 0.88; use of episiotomy.
episiotomy during 95%CI 0.84–0.92), less
vaginal birth? suturing (RR 0.74; 95%CI
0.71–0.77), and fewer healing
complications (RR 0.69; 95%CI
0.56–0.85).
Magnesium How effective is Five trials (n= 1236 women) For women with
sulfate for magnesium sulfate Magnesium sulfate was eclampsia magnesium
eclampsia compared to diazepam associated with a substantial sulfate, rather than
or phenytoin for women reduction in recurrence of diazepam or phenytoin
with eclampsia? convulsions (RR 0.45; 95%CI appears more effective.
0.35–0.58).

Maternal mortality was also

reduced (RR 0.60; 95%CI
0.36–1.00).
Routine What are the effects Two trials (n= 539 women) There is insufficient
perineal of routine shaving No difference in maternal evidence to recommend
shaving compared to no shaving febrile morbidity was detected routine perineal shaving
on admission in labour? (OR 1.26; 95%CI 0.75–2.12) on admission in labour.

In the smaller trial, fewer

women who had not been
shaved had Gram-negative
bacterial colonisation (OR 0.43;
95% CI 0.20–0.92).
Enemas What are the effects Two trials (n= 665 women) There is insufficient
during labour of enemas during No clear difference in infection evidence to recommend
first stage labour on rates for puerperal mothers (OR the use of enemas during
maternal and neonatal 0.61, 95% CI 0.36–1.04) or labour.
outcomes? newborn children.
Routine What are the effects of Ten trials (n=over 60 000 Small reductions in the
antenatal care antenatal care visits for women) number of prenatal visits
in low risk low-risk women? A reduction in the number (two visits or fewer) are
pregnancy of antenatal visits was not compatible with similar
associated with an increase in good perinatal outcomes
any of the negative maternal as in the standard model.
and perinatal outcomes
reviewed.
8 Evidence-led obstetric care: strategies to change practice and policy

Changing practice
Changing the behaviour of health professionals is not easy, and a variety of strategies, such
as continuing medical education or dissemination of printed materials, have been tested. It
is important that clinical practice is based on the best available evidence, but it is equally
important that interventions to change practice are based on evidence as to whether they
are effective or not.

Systematic reviews available in the Cochrane Library summarize the effects of various
interventions to change professional behaviour. In this section, interventions are
categorized as likely to be beneficial, of unknown effectiveness, and as likely to be
ineffective in changing health professional behaviour (see Boxes 2–4). Annex 1 contains
summaries of the systematic reviews, with full references.

Clinical care decisions in obstetrics often involve an individual health professional making
a personal decision about whether or not to do something, or a small team working within
a health facility determining how they will manage care. However, implementing some
interventions requires decisions to be made about how care is organized, and thus requires
introducing changes in policy.
Box 2

Interventions likely to be beneficial

Interactive workshops A systematic review of the effects of continuing educational
meetings (32 studies, 2995 health professionals)10 found moderate
or moderately large effects in six studies comparing interactive
workshops with no intervention or control (all were statistically
significant) and small effects in four (one was statistically
significant).

Workshops combined The same systematic review10 showed interventions that combined
with didactic workshops and didactic presentations had moderate or moderately
presentations large effects in 12 comparisons (11 were statistically significant)
and small effects in seven comparisons (one was statistically
significant).

Opinion leaders A systematic review (8 studies, 296 health professionals) of the

effects of using health professionals nominated by their colleagues
as “educationally influential” (local opinion leaders)11 found some
improvement for at least one practice outcome variable (6/7 trials);
in two of the trials, the results were statistically significant and
clinically important.

In three trials that measured patient outcomes, only one achieved

a practically important impact on practice.

Audit and feedback A systematic review of feedback to health professionals on

their performance relative to their peers or accepted guidelines
(85 studies, 3500 health professionals)12 found an absolute
improvement in practice of -9%–70% (52 studies comparing audit
and feedback with no intervention).
 Introduction 9

Box 3
Interventions with unknown effectiveness
Dissemination of A systematic review of the effects of printed educational materials
printed materials on health professional practice13 needs updating and has been
withdrawn from the Cochrane Library. The original review found
only a small effect on practice in nine studies (mainly interrupted
time-series) comparing the distribution of printed educational
materials with no intervention.

Dissemination of In a systematic review of 18 studies, 4 cluster-randomized trials

practice guidelines showed improvements but with unit-of-analysis errors. The
available evidence is sparse and generally poor quality.

Educational outreach A systematic review of the effects of a personal visit by a trained

person to a health-care provider in his or her own setting (18
trials, 1896 health professionals).14 Targeted behaviours included
prescribing (13 studies); preventive services (3 studies); general
management of common problems (2 studies).
Small-to-moderate effects on practice were observed in all the
studies. Educational outreach is more likely to have an effect on
prescribing practices; the effect on other aspects of practice is
uncertain.

Box 4
Interventions likely to be ineffective
Didactic lectures A systematic review of the effects of continuing educational
meetings (32 studies, 2995 health professionals)10 found didactic
lectures had no effect on practice (7 studies).

Continuous Quality A systematic review of continuous quality improvement efforts in

Improvement (CQI) in clinical settings15 found a positive effect in 41/43 before-and-after
clinical settings studies, but three randomized controlled trials showed no effect on
clinical outcomes or organization-wide improvement in clinical
performance.

Influencing policy
Policy can be defined as “laws, rules, financial and administrative orders made by
governments, nongovernmental organizations or private insurers that are intended to
directly affect the use of a practice or form of care”.16 For example, to change from
diazepam to magnesium sulfate for eclampsia treatment not only needs a clinician to
be aware of its value and decide on its use, but requires, in the public sector, a change
in policy so that it is included in the national essential drug list, and is purchased and
distributed alongside other medical supplies.17 However, some decisions made by
governments or other organizations involved in setting norms represent “non-binding
recommendations” rather than specific policy changes. For example, government
endorsement of a baby-friendly hospital initiative, without resource commitment, represents
a decision to support an effective model of care, but does not constitute a policy change.
10 Evidence-led obstetric care: strategies to change practice and policy

Policy also determines how care is organized in its own right. For example, WHO
recently considered whether reproductive health services should be integrated, meaning
contraceptive services, sexually transmitted disease care, antenatal and postnatal care
could be provided together by the same health care worker at the same time. Research
evidence around this is not strong, but demonstrates that there are both advantages and
disadvantages to integration at the point of delivery.18

Influencing policy requires commitment on the part of national governments, institutions

and donors to evidence-based approaches, through policy changes or “non-binding
recommendations”. This can take the form of national policies for developing evidence-
based guidelines, and management systems that ensure these guidelines are implemented
and monitored. Sometimes an experimental design can be used to formally evaluate the
way policies are implemented. In Nigeria, clinicians have teamed up with policy-makers
in Cross River state to develop evidence-based guidelines using a multidisciplinary team
approach; if the state-level evaluation demonstrates impact on practice, the strategy
will be scaled up for use in other states. The decision to change policy or to make
recommendations for the use of national evidence-based guidelines lies with the policy-
makers involved.
Box 5

Health policy-makers perceptions of their use of evidence

Systematic review; last updated 2002
Question What are the barriers and facilitators to the use of evidence by health
policy decision-makers?

Summary of 24 studies comprising 2041 interviews with policy-makers were

review findings included.
The studies were qualitative assessments that focused on hypothetical
scenarios or retrospective perceptions.
Personal contact (13/24), clear recommendations from research
(10/24), and timeliness and relevance of the research (10/24) were the
three most frequently mentioned factors that facilitated use of research
in policy.
Mutual mistrust (16/24), lack of timeliness or relevance (10/24), and
power and budget struggles (8/24) were the three most frequently
mentioned barriers to the use of research in policy decisions.
Reviewers Strategies researchers can use that might increase the selective use of their
conclusions research include: using personal and close two-way communication;
writing brief summaries that include effectiveness data; and ensuring
research is timely, relevant and of high quality. These strategies will often
not be effective in increasing appropriate use of research evidence, and
are difficult to implement and evaluate.

Reference Innvaer S, Vist G, Trommald M, Oxman A. Health policy-makers’s

perceptions of their use of evidence: a systematic review. Journal of
Health Services Research and Policy, 2002, 7(4):239–44.
 Introduction 11

A systematic review of observational studies examined the barriers to and facilitators of the
use of research evidence by health policy-makers and identified strategies researchers can
rely on to increase use of their research (see Box 5).

A number of observations about effective policy change, based on common sense and
personal experience, were highlighted by meeting participants:

Researchers could provide clear recommendations in brief summaries that include

effectiveness data.

Personal contact between researchers and policy-makers is important.

The timeliness, relevance and quality of research outputs are important if research is to
have an effect on policy decisions. Policy-makers often consult researchers in times of
health crises; it is frequently important for researchers to make use of these “windows
of opportunity” and to reflect candidly without risking premature public disclosures.

Researchers are accountable in different ways to policy-makers; they are not

responsible for policy-making, but can inform the judgements of policy-makers. It is
important to be explicit about these distinct roles. Formal or informal contracts might
be a useful way to define these roles, for example, to set mutually agreed-upon rules
about such matters as confidentiality and communication; “The clearer the rules, the
better”, and decision-making processes should be transparent to everyone.

Even institutionalized decision-making processes established by legislation can be

threatened by changes in government or by groups with vested interests; senior policy-
makers are rarely in post for very long and researchers frequently move on.

No systematic reviews or trials were quoted or identified, but these were informed opinions
made at the meeting from the experienced team present.
12 Evidence-led obstetric care: strategies to change practice and policy

Summary
In changing the behaviour of individual practitioners, research evidence suggests:
passive dissemination of evidence alone is often not effective in changing provider
practice;

compared with passive dissemination, interventions that promote active engagement

of participants, such as through interactive workshops, audit and feedback, appear
more likely to be effective, as does using opinion leaders to promote change. However,
the magnitude of the effect depends on the clinical significance of the practice and its
relevance to the specific setting.

In relation to policy, research evidence on effective practices is relatively weak; however the
following factors were thought to facilitate uptake: a) providing clear, brief summaries with
recommendations; b) personal contact between researchers and policy-makers; c) timeliness,
with particular policy opportunity windows.

There is sparse evidence on the effect of teaching evidence-based medicine; training might
influence knowledge, but the effect on practice is unclear.
 2. Changing practice: lessons from 4 projects 13

2.1 The projects

Four projects that aimed to promote dissemination and use of systematic review
findings from the RHL in individual health facilities were presented at the meeting;
Table 2.1 summarizes the objectives, design, interventions and primary outcomes of
each project.

Projects to change practice

Table 2.1
Country Objectives Design Units Interventions Primary
outcomes
RHL trial Thailand and To evaluate Cluster- 40 hospitals Three Clinical
Mexico the impact randomized interactive practice; user
of teaching controlled workshops experiences
RHL to health trial over 6 months;
workers on RHL training
obstetric materials
outcomes
Labour Johannesburg To evaluate Before-and- 10 hospitals Training in the Change in
support South Africa the impact of after study use of RHL (10 4 obstetric
study a staff directed sites), and an indicators;
intervention intervention number
promoting promoting of women
childbirth childbirth accompanied
companions companions during labour
(5 sites)
Better Johannesburg To test a set Before-and- 10 hospitals Educational Change in
Births South Africa of training after study workshop 7 marker
Initiative materials (10 sites) and practices;
pilot and explore self-audit qualitative
project change mechanism analysis of
processes in (5 sites) the change
ten hospitals process

Position Dar es Salaam To evaluate Before-and- 4 hospitals Audit and Change in

during United a training after study feedback practice rates
labour Republic of package to followed by for mobility
study Tanzania influence an educational and supine
position workshop position
during labour

See Annex 2 for project abstracts.

14 Evidence-led obstetric care: strategies to change practice and policy

Formal evaluation of all studies is currently in progress or being written up (Annex 2). A
summary of comments by investigators and meeting participants concerning those studies
is given here.

RHL trial
The RHL could bridge the information gap for clinicians in low- and middle-income
countries, but its dissemination alone is unlikely to lead to significant practice change.
Behaviour change often requires managerial and organizational restructuring as well
as interventions to motivate staff. This cluster-randomized trial evaluated a programme
supporting health professionals in accessing and using evidence from the RHL. The
programme consisted of a multifaceted intervention including three interactive workshops
delivered by a specialist over six months, meetings with hospital directors, computer
hardware placed in labour wards with an RHL resource person identified within the staff,
and materials to promote awareness and use of the RHL in Thailand and Mexico.

Labour support study

The investigators of this cluster-randomized trial hypothesized that providing women
with childbirth companions would improve the quality of care for women during labour,
and providing staff with access to evidence would encourage them to implement best
practice. The results showed no impact on the number of women accompanied during
labour, and only one of four indicator practices (episiotomy) showed a (non-significant)
change. The findings demonstrate that introducing childbirth companions is complicated
due to infrastructure and organizational factors as well as barriers to changing individual
practitioner behaviour; alternative strategies to help clinicians implement companionship
as well as make their practice more evidence-based need to be tested. Participants
found the RHL difficult to use and although prior computer training makes the database
more accessible, merely providing access to this information does not directly influence
practice.

Better Births Initiative pilot project

Building on the findings of the labour companions study, an international network of
researchers developed a focused educational programme to communicate evidence-
based approaches and disseminate best practice to obstetric staff in Johannesburg, South
Africa. The Better Births Initiative (BBI) used printed and interactive materials to help
clinicians compare current practice to evidence-based standards and identify ways to
change practice. With regard to enemas and perineal shaving, this one-group, pre–post
study showed some evidence of improved practice. Meanwhile, supine position, the use of
oral fluids, and companionship during labour were less likely to have changed at follow-
up. Qualitative findings reveal that behaviour change was more likely at hospitals where
motivation among staff was high and social structures existed to support and maintain
changes to practice. The study illustrates that clinician behaviour change happens
incrementally; implementation trials with short follow-up for primary outcomes are
unlikely to achieve the expected impact given the complexity of the change process.

Position during labour study

The Africa Midwives Research Network promotes evidence-based midwifery practice
through regional and in-country workshops. Midwives in the United Republic of Tanzania
identified a need to encourage mobility during labour, and delivery positions other than
supine. The study used a one-group before-and-after design to evaluate the impact of an
audit and feedback intervention combined with an educational workshop to promote
 2: Changing practice: lessons from 4 projects 15

evidence-based practice standards in four Tanzanian hospitals. Following implementation

of the workshops, practice changed significantly for mobility during labour at two
hospitals; but supine delivery position remained routine practice at all hospitals. The
study concludes that barriers to change are complicated and require providers to want to
change, and women to be informed of alternative positions; practice change often involves
additional resources.

2.2 Lessons learnt and implications for research

Lessons learnt
Workshop participants analysed the four projects and identified key lessons from each; these
are presented below in terms of the context in which interventions were implemented, the
design and the evaluation of interventions. Learning points are summarized at the end of the
section.
It is not realistic to expect health professionals to make decisions about changing
clinical care on their own. The labour support and RHL trials found professionals
often want recommendations to guide decision-making, and support from hospital
management for practice changes.

Attributes of the proposed practice changes determine the extent of clinician behaviour
change. The labour support trial and position during labour study found that practices
that have resource or infrastructure implications are less likely to change. Conversely,
the Better Births Initiative study found that best practices that save time (for example
avoiding enemas, episiotomy and shaving) were more readily implemented.

Medical regulations in some countries are a constraint to implementing best practice in

obstetric care. In China, under the recently introduced regulations penalizing medical
accidents, third-degree perineal tears are considered a medical accident, and in South
Africa, medical students are required to conduct several supervised episiotomies as part
of their training. Such protocols may preclude the implementation of a restrictive policy
for episiotomy.

Overcrowding, poor working conditions and staff shortages often mean staff are
demoralized and lack motivation to adopt new initiatives. The South African and
Tanzanian case studies particularly highlighted the impact of human resource
constraints on efforts to influence health-provider practice.

Local ownership and demand creation (or provider buy-in) are important to the success
of dissemination and implementation projects. A perceived lack of local “ownership”
of the Better Births Initiative influenced providers’ willingness to participate in the
programme. The Tanzanian study involved key hospital personnel and clinicians in
the planning and facilitation of the intervention, and in the RHL trial, hospital directors
were consulted and local clinicians trained to facilitate workshops; explicit local
involvement can increase the likelihood of interventions being implemented.

Staff turnover is a constraint to implementing initiatives to change clinical practice.

The South African case studies found that very high staff turnover and frequent staff
rotations have implications for the intensity of implementation, for example, the
number of outreach visits conducted. Unless change interventions are repeated over
time, as in the RHL trial, uptake of best practice by all health professionals in a given
setting is unlikely.
16 Evidence-led obstetric care: strategies to change practice and policy

Providing practitioner access to evidence has limited effect on practice. The RHL
trial trained practitioners to access and use the Reproductive Health Library; this
alone seems unlikely to improve practice in the absence of specific management and
organizational interventions to influence practitioners’ decisions and mainstream
evidence-based standards, and to monitor change over time.

Implications for research

Interventions should be tailored to specific barriers or problems. Some interventions

may strive to influence provider practice, while didactic lectures may have an
impact on provider knowledge, but not necessarily on practice. The Tanzanian study
consulted key stakeholders about the potential barriers to changing practice, but a
more systematic assessment of barriers or constraints (for example clinician knowledge,
organizational constraints or information access) would help to identify appropriate
interventions to address them.

There is sparse evidence about which interventions are most effective for which
behaviours. The impact of interventions to change professional practice is usually small
or moderate, and it is sensible to acknowledge the limitations of interventions when
designing and implementing initiatives. Three of the projects to change practice (labour
support, BBI and position during labour) relied on interventions likely to influence
behaviour change including audit and feedback, the involvement of opinion leaders,
and interactive workshops.

Availability of materials in local languages helps. The RHL was used with more
difficulty in the trial in Thailand due to the language barrier, but in some hospitals
English-speaking staff translated key documents in order to facilitate dissemination of
evidence.

Change takes time, effort and resources. The BBI and labour support study findings
suggest that implementation trials with short follow-up are unlikely to achieve
the anticipated impact on practice, because behaviour change usually happens
incrementally. Enthusiasts or local opinion leaders who maintain the momentum for
change can be central to the success of interventions to change provider practice.

Collaboration and a multidisciplinary approach are essential. None of the projects

adequately involved consumers, for example, pregnant women or local women’s
groups. Similarly, hospital staff were not extensively involved in the design of
interventions to change their practice, which led to a level of disconnect between the
research team and participating staff.

Model potential impact to improve “buy-in’. One way to improve government

and donor “buy-in” to initiatives for change is to model the potential impact of
implementing the practice changes being promoted. By illustrating the impact on
costs, the number of deaths, morbidity outcomes, and the number of near misses—for
example, reducing the enema rate by xx% resulted in a saving of xxUS$—would
demonstrate potential and encourage support at the national level.

Be realistic in terms of expectations of impact of interventions. The BBI pilot study

aimed to test a programme to engage providers in evidence-based approaches and
methods to implement changes; the intention was not to impact directly on practice.
 2: Changing practice: lessons from 4 projects 17

Conversely, the RHL trial aimed specifically to evaluate the impact of an intervention
on obstetric outcomes. Despite the rigorous design, however, it was difficult to detect
any change in practice, and the intervention’s impact was rather in terms of improved
knowledge.

Evaluation design depends on the effect sought. Before-and-after designs are useful for
flag waving and engaging people, but they tend to overestimate the size of the effect
on practice. The lack of a control arm also means any practice change detected cannot
be attributed to the interventions used. The RHL trial was commended for its rigorous
design.

Two of the before-and-after studies provided good anecdotal evidence of impact on

engaging people and on practice (the United Republic of Tanzania and BBI South
Africa); the labour support trial showed evidence of no change. These three projects
were initially to develop intervention approaches that could be used on a wider
scale; the intention was not that they should be studies of the generalizability of these
interventions to induce change nationally or more widely.

Process evaluation is important. Variation in effects within and between study sites is
common. Process evaluation, using qualitative research methods such as focus group
discussions and in-depth interviews (as in the BBI study), can help to disentangle the
effect of an intervention from the influence of contextual factors or existing individual
and social processes of change. Without attention to these factors, it is easy to
hypothesize about what is different, but it is difficult to tease out the factors responsible
for the success, or failure, of interventions.

Systematic approach to evaluation. Before scaling up to national level in any country,

initiatives like the BBI programme should be implemented and evaluated at a larger
scale, possibly as a cluster-randomized controlled trial or an interrupted time-series
design (and outside of the South African setting). The design should incorporate careful
selection of clinical outcomes, process evaluation, and cost–effectiveness.

Implementation research requires long-term planning and design. Most of the case
studies presented were conducted with limited resources over relatively short time
periods. Long-term funding is necessary, especially in the planning and intervention
design stages, to avoid the tendency to rush to demonstrate impact. Evidence of impact
from large cluster-randomized trials or interrupted time-series designs will indicate
whether a scale-up will work, but such projects require funding.

Implications for practice

There is insufficient evidence that the specific strategies used in the four projects are
effective in inducing clinician behaviour change to recommend their implementation
by countries through national initiatives.

Initiatives to change practice and improve health care outcomes often require more
than interventions (single or combined) targeting individual clinicians. Integrated
programmes aimed at both system and organizational change are more likely to
achieve consensus on practice changes and consistent implementation of these
changes. They are also more likely to generate long-term organizational support (in
terms of resources, policies, commitment).
18 Evidence-led obstetric care: strategies to change practice and policy
 3. Changing policy: lessons from 3 initiatives 19

3.1 The initiatives

Three policy initiatives were presented that aimed to influence policy in relation to
obstetrics and the adoption of specific evidence-based policies. Table 3.1 outlines the
characteristics of each initiative.

Characteristics of initiatives to influence policy

Table 3.1
Objectives Activities

China To increase commitment to Presentations at national conferences and

dissemination evidence-led decisions by academic meetings on women’s health
programme national institutions
Evidence-based medicine incorporated into
post graduate medical curricula (Shanghai)
Evidence-based medicine introduced to
health care managers (Shanghai)
To communicate findings from Translation of “Evidence Update”
selected systematic reviews (summaries of Cochrane review evidence)
into Chinese and dissemination at
academic meetings
To demonstrate potential Observational study of practices in major
impact on current obstetric Shanghai hospitals (1999 and 2003)
practice
Promoting the To promote national adoption Engaged key stakeholders prior to
new WHO of new guidelines for completion of ANC trial
antenatal antenatal care visits
care model in
Thailand
Trial results approved by Royal Thai
College of Obstetrics & Gynaecology and
Ministry of Public Health
Implementation of the new WHO ANC
model in Khon Kaen province; including
workshops for health professionals
Press conference to publicize the new ANC
model
20 Evidence-led obstetric care: strategies to change practice and policy

Objectives Activities

Promoting the To promote national adoption Publication of trial results in Spanish

new WHO of new guidelines for language medical journals; publicized in
antenatal antenatal care visits national press and media
care model in
Argentina
Presentation of new model at national and
international scientific meetings
Modified existing national ANC guidelines
Implementation of the new model in
Corrientes province, including workshops
and information dissemination

China dissemination programme

The comprehensive dissemination programme in China demonstrated impact on childbirth
policy both nationally and institutionally. At the national and provincial levels, the
Women’s Health Division of the Chinese Preventive Medicine Association now promotes
evidence-based practice countrywide; several provinces have pioneered initiatives to
reduce interventions during childbirth; and evidence-based medicine is the main topic of
the next annual academic meeting of the Women’s Health Care Association of China. At
the institutional level, the observational study of obstetric practice in Shanghai showed
some significant changes to obstetric indicators in the period 1999–2003. Although these
changes cannot be attributed to the dissemination activities alone, the study demonstrates
that the health system is responsive to evidence-based approaches. On the basis of these
findings, WHO funded a study to promote evidence-based obstetric care in five cities in
China*. The dissemination programme has also started to influence policy for medical
education; the School of Public Health at Fudan University is running a postgraduate
course in evidence-based health care (including critical appraisal and evidence-
based decision-making) and a continuing medical education programme for practising
professionals, conducted jointly with an affiliated hospital. In addition, translated materials
such as Evidence Update, and discussion of evidence-based approaches at national
academic meetings have improved access by health professionals to scientific evidence on
obstetric practices.

Promoting the new WHO antenatal care model in Thailand

A systematic review published in the RHL shows that the number of antenatal care visits
could be reduced without any increase in adverse maternal and perinatal outcomes;
this new antenatal care model costs less to mothers and health services. In Thailand,
researchers and clinicians experienced in evidence-based obstetrics used various activities
to promote the new model in Khon Kaen province. Involving key policy-makers at an early
stage helped to influence policy decisions at the provincial level; chief provincial medical
officers and national policy-makers from the Ministry of Public Health were made aware of
the new model and approved the implementation strategy. A high-profile press conference

*
Strengthening maternal safety. Grant ID CHN/RPH/001 (2000-2002)
 3: Changing policy: lessons from 3 initiatives 21

chaired by the chief provincial medical officer was used to inform the public about the
new antenatal care model. The appearance of medical experts together with policy-makers
helped reinforce the scientific message underpinning the new model and garner public
support.

Promoting the new WHO antenatal care model in Argentina

A similar approach was used to promote the new antenatal care model in Argentina. Local
publications and dissemination at scientific meetings helped to impress upon clinicians
and policy-makers the evidence supporting reduced antenatal care visits. A key part
of the strategy was modification and implementation of existing national guidelines in
one province; this involved key representatives from the national health authorities. The
opportunistic nature of the strategy was emphasized, and it was concluded that policy
change and implementation at the local level is a complex process that has substantial time
and funding implications.

3.2 Lessons learnt

Workshop participants analysed the strengths and weaknesses of each approach to
influence policy; key learning points are outlined below:
Influencing policy is opportunistic. The three initiatives show that it is difficult to plan
a systematic approach to influencing policy, and that the process of engaging policy-
makers is often opportunistic because research findings may or may not fit into the
political agenda. The Thailand case study acknowledges this, and a key lesson is,
“think big, start small, but start now!”. If researchers can demonstrate an intervention or
practice change can be implemented easily and has clinical impact, it is more likely to
be acknowledged and placed on the policy agenda.

The relationship between researchers and policy-makers is important. Personal

communication and good relations between them is a key part of initiatives to
influence policy. In the Thai example, researchers had built up a relationship with
provincial policy-makers over time, which facilitated the inclusion of the new model in
policy. In China, the success of the dissemination programme was partly due to good
relations between influential researchers and clinicians and policy-makers at provincial
and national levels.

Mutual understanding and acknowledgement of the respective roles and responsibilities

of researchers and policy-makers for the consequences of the changes is important
for policy change. The Thai example showed that the presence of researchers and
politicians in support of the new antenatal care model at a press conference helped
convey to the public the partnership between research and policy.

Clear research messages are important. Translation into the local language of key
messages and local dissemination of research findings can help convince policy-
makers of the relevance of an intervention or practice locally. In China, translations
of evidence summaries and materials promoting evidence-based obstetrics helped to
improve the accessibility of scientific evidence underpinning proposed policy changes.
In Argentina, local publication of the antenatal care trial results helped to raise political
support for the adoption of the new model locally.
22 Evidence-led obstetric care: strategies to change practice and policy

The policy implications of research need to be considered. Dialoguing with policy-

makers about a research project from the outset, and consideration of the policy
implications up front can facilitate the translation of findings into policy and practice.
The Thai and Argentinian projects are examples of how key decision-makers were
consulted and involved in the planning of the trial, which inevitably influenced their
decisions to approve the new model for provincial implementation.
 4. Strategic options 23

The need for more rigorous evaluation of interventions to change professional

behaviour should not deter individuals from promoting evidence-based reproductive
health using existing knowledge of effective approaches. Some meeting participants
were quite focused on behaviour change in localities, and others were concerned with
wider promotion of evidence-based obstetric care.

4.1 Using accreditation

Accreditation systems, where institutions are rewarded for achieving certain levels of
“good practice” (defined according to available evidence), appear to motivate health
professionals to change their practice, especially when endorsed by international
organizations.

The meeting participants highlighted the need for systematic review of accreditation
as an approach for promoting evidence-based care. A logical starting point would be
trials evaluating the Baby Friendly Hospital Initiative; there are at least three existing
trials in this area.19,20,21 It was agreed that WHO and EHCAP would jointly identify
and support someone to complete a Cochrane Review in this area in collaboration
with the EHCAP group.

Accreditation may have varying effects, depending on the practices it is used to

promote. To determine if accreditation has potential as a strategy for promoting
evidence-based reproductive health, meeting participants proposed developing simple
tools and piloting them at the provincial level in South Africa. If the approach and
tools seem to work, the next step would be a more rigorous design with adequate
comparisons for evaluation at the national level.

A plan was suggested for testing the approach in Eastern Cape Province, South Africa:
Work with the provincial Department of Health to develop a set of verifiable
standards for evidence-based obstetric care. For example: availability of
magnesium sulfate and administration to >95% women with eclampsia;
availability of oxytocin and administration to >95% women at delivery;
episiotomy in <25% nulliparous women.

Establish a provincial government “BBI Award” for achievement, or partial

achievement of these standards. Design an evaluation tool to measure adherence.

Continue running BBI workshops with achievement of the award as a motivation.

Practice at each site could be evaluated before the workshop, so that they have a
baseline from which to work.

Define the success of the programme simply in terms of sites achieving and
maintaining accreditation (acknowledging that this model will not provide a
cause-and-effect relationship); WHO could facilitate this approach by endorsing
the BBI Award.
24 Evidence-led obstetric care: strategies to change practice and policy

4.2 Formal linear approaches

Most of the projects discussed in the workshop did not adequately assess barriers and
opportunities for changing practice at the outset. This assessment is an important stage
in the development of any strategy to influence provider practice, and frameworks exist
to facilitate this process. The participants discussed the limitations of existing initiatives
and, based on this, suggested the linear approach to implementation below, comprising
several key stages (see Box 6). There was debate as to whether formal linear approaches
with rigorous research evaluation are necessary before promoting an approach to change
practice (such as continuing medical education). It was agreed that reliable research
was extremely limited in these areas, and that it would be inappropriate to promote
any proposal at national or regional levels that involved a major investment of staff and
resources unless direct evidence that it worked was available.

4.3 Training
Priming health professionals with the principles of evidence-based medicine through
undergraduate training and dissemination of training materials in local languages should
be a long-term goal in all countries where this is not already in place. Examples of where
training has been introduced and materials developed for local dissemination include:
Training on evidence-based medicine:
In China, postgraduate courses at the School of Public Health at Fudan University
in Shanghai now include critical appraisal training and an introduction to evidence-
based approaches.

Principles of evidence-based medicine are now taught in undergraduate and

postgraduate curricula nationwide in Thailand.

A module on the principles of critical appraisal is being developed for use in nursing
and midwifery training in Dar Es Salaam, the United Republic of Tanzania.

In South Africa, the University of Pretoria includes RHL training in obstetric modules.
Materials in local languages:
In China, summaries of systematic reviews (Evidence Update) have been translated and
posted on the Fudan University web site, and printed versions disseminated to hospital
managers nationwide.

The RHL is available in Spanish and is widely circulated in Latin America.

The training manual for implementing the WHO antenatal care model is available in
Thai and Spanish.

The Better Births Initiative training materials are available in Thai.

4.4 Test evidence-based medicine teaching methods

Few trials provide evidence of the effect of teaching evidence-based medicine skills in
health care settings. A systematic review that included one trial suggests teaching critical
appraisal has positive effects on knowledge, but the effect on decision-making and patient
outcomes is uncertain. The group discussed the possibility of developing a multicentre
 4: Strategic options 25

Box 6
Formal approach to changing practice
1 Set priorities
Activities Obtain consensus on priority areas in reproductive health and involve consumer
and professional groups in priority setting.
Conduct a comprehensive needs assessment, including a situation analysis of the
context of change, the resources, cost, time, staff involved, and the feasibility of
implementation.
Assess barriers to, and opportunities for changing practice, using systematic
methods to consider the practice environment, social context, attitudes and
knowledge.22 Methods used to identify barriers include conducting informal
meetings, consulting opinion leaders, and observation.
Relate barriers to individual practices and discuss problems and solutions.
2 Measure current practice
Activities Measure practice rates and variation in hospitals where relevant.
Identify practice areas with good evidence, and assess gaps between evidence
and practice.
Develop good evidence-based messages that target the right level—this may be
the health care system, an organization, teams or individuals.
Consider the packaging of practice changes—where a “menu of change” is
provided it may be useful to help health professionals pick and choose according
to their needs and priorities of their setting.

3 Selected intervention based on priorities and barriers

Activities Assess the need for guidelines. Can existing guidelines be adapted for local use?
Challenge guidelines that are not evidence-based.
Tailor interventions according to barriers.
Is there a role for interventions based on regulation or incentives?
Empower women with information about best practice and informed choices.
Consider the role of the media or women’s groups.
4 Evaluate interventions systematically
Activities Decide on measurable outcomes.
Uncontrolled before-and-after studies, with qualitative components, may be
useful for engaging staff in the change process.
Implement evidence using effective behaviour change interventions.
Evaluate impact on practice.
Controlled before-and-after, interrupted time-series, and cluster-randomized
designs can be used to evaluate the effect on practice.
Include process evaluation and cost–effectiveness estimates.
Have realistic expectations of what can be achieved with the time and resources
available.
Audit and feedback can be useful to engage practitioners in the quality
improvement process.
26 Evidence-led obstetric care: strategies to change practice and policy

randomized trial to assess the effects of teaching evidence-based medicine skills using
existing tools developed by the group (Evidence Update, critical appraisal module). One
hypothesis is that combining teaching of evidence-based medicine skills with training on
identifying organizational constraints and establishing quality assurance would strengthen
the likelihood of behaviour change.

A large multicentre trial to test these approaches could be developed within an existing
programme in West Africa to promote access to, and use of, evidence: the Use of Scientific
Evidence Initiative (USE-It), funded by the Department for International Development,
United Kingdom. The USE-It network includes national and regional institutions with
commitment to evidence-based approaches, many of whom would support further research
into teaching critical appraisal and evidence-based medicine skills (West African Health
Organization, West African College of Physicians, and the National Institute for Medical
Research Nigeria).

4.5 Informal approaches

Where it is not possible to develop formal approaches to implement change interventions,
persons working in institutions should be encouraged to develop ways to influence practice,
starting with highly motivated individuals and advocates at the local level. Small-scale
quality improvement projects, using the “plan-do-study-act” scheme, can test the effects of
small changes by means of cycles of action and reflection.23 First, changes are planned and
agreed upon by all participants and baseline data are collected (plan), then small changes
are carried out, the effects are noted and data collection is repeated (do). Lastly, the results
of the changes and the change process itself are analysed (study), and changes that lead
to improvement are implemented on a larger scale (act). The strength of the model lies in
the reflection phase—learning from the process of implementing change. As with other
quality improvement cycles, obstacles to the implementation of this model include lack
of time, resources, commitment, and strong leadership to initiate programmes. Systematic
review evidence suggests that clinician involvement, feedback from individual practitioners
and a supportive organizational culture enhance the effectiveness of quality improvement
programmes.15

4.6 Increase political commitment

Researchers need to interact more with policy-makers and understand the way they
function.

National governments need to demonstrate their commitment to evidence-based health

care by allocating funds for the training of young researchers and clinicians. For example,
the Thai government has sponsored three PhD students to undertake training in evidence-
based medicine at the Liverpool School of Tropical Medicine.

National institutes need to be encouraged to give priority to research in evidence-based

practice. For example, in Nigeria, the government-commissioned National Tropical
Diseases Research Institute in Calabar has requested an evidence-based medicine office
be located within the Institute.
 4: Strategic options 27

4.7 Involve international donors

Make the inclusion of policy and practice implications in funding applications mandatory.
Researchers should make clear to what extent policy-makers will be involved with
planning the research and propose methods to ensure the findings reach policy-makers.

Make evidence of the impact of completed research on policy and practice a

requirement in annual reports of major research programmes. However, dissemination
of findings alone is insufficient to influence policy, and donors must insist on details of
how researchers intend to interact with policy-makers. Target audiences for research
findings should be identified before and during the research process, and dissemination
outputs should reflect their needs.

Only grant funds for scaling up implementation of interventions with good evidence of
effect; establish a funding mechanism for implementation of practices that are shown to
be evidence-based.

Develop effective methods to keep abreast of knowledge programme outputs,

particularly those with high impact, likely to have important policy implications.
Researcher-produced policy briefings and fact sheets for decision makers, and “sound
bites” for the media could facilitate this.
28 Evidence-led obstetric care: strategies to change practice and policy
 Annex 1 29

Evidence-based health worker behaviour change

Summaries of systematic reviews

Dissemination of printed educational materials

Review withdrawn from the Cochrane Library; an updated version is in progress

Question Can dissemination of printed educational materials change health

professional practice?
Systematic 9 studies compared the distribution of printed educational materials
review summary with no intervention; mainly interrupted time-series
No difference between academic or glossy materials
Can affect knowledge and/or attitude

Reviewers At best only likely to be a small impact on practice

conclusions Improvement more likely in areas of low preference for mailed CME
materials
Reference Freemantle N, Harvey EL, Wolf F, Grimshaw JM, Grilli R, Bero LA.
Printed educational materials: effects on professional practice and health
care outcomes. Cochrane Database of Systematic Reviews, 2001, (2):
CD000172.

Dissemination of practice guidelines

Non-Cochrane systematic review; last updated 2004

Question How effective are different strategies for disseminating and implementing
guidelines?
Systematic 235 studies (73% of comparisons evaluated multi-faceted
review summary interventions).
Single interventions included reminders, educational materials, audit
and feedback, and 23 comparisons of educational outreach.
The majority showed modest to moderate improvements in care.

Reviewers Reviewers conclusions The evidence is sparse and generally poor

conclusions quality. Educational materials may have a modest effect on guidelines
implementation that is short lived.
Reference Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay C, Vale
L et al. Effectiveness and efficiency of guideline dissemination and
implementation strategies. Health Technology Assessment, 2004, 8(6).
30 Evidence-led obstetric care: strategies to change practice and policy

Continuing medical education

Cochrane Library, Issue 4, 2003; last updated 26 February 2001

Question What are the effects of educational meetings (lectures or workshops) on

professional practice and health care outcomes?
32 studies involving 2995 health professionals were included
Systematic Interactive workshops (10 comparisons) showed moderate or
review summary moderately large effects in six studies (all were statistically significant)
and small effects in four (one was statistically significant).
Interventions that combined workshops and didactic presentations
showed moderate or moderately large effects in 12 comparisons (11
were statistically significant) and small effects in seven comparisons
(one was statistically significant).
Didactic lectures (7 studies) showed no effect on practice.
Reviewers Interactive workshops can result in moderately large changes in
conclusions professional practice. Didactic sessions alone are unlikely to change
practice. There is a need to determine which specific attributes of
interventions (message, audience, size of group) contribute to effectiveness.

Reference Thomson O’Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA,
Herrin J Continuing education meetings and workshops: effects on
professional practice and health care outcomes. Cochrane Database of
Systematic Reviews, 2001, (2):CD003030.

Local opinion leaders

Cochrane Library Issue 4, 2003; last updated 24 November 1998
Question What are the effects of using health professionals nominated by their
colleagues as “educationally influential” on professional practice and
patient outcomes?
Eight studies involving 296 health professionals were included.
Systematic Six of seven trials that measured health professional practice
review summary demonstrated some improvement for at least one outcome variable,
and in two trials, the results were statistically significant and clinically
important.
In three trials that measured patient outcomes, only one achieved an
impact on practice that was of practical importance.
Reviewers Using local opinion leaders results in mixed effects on professional
conclusions practice. It is not always clear what local opinion leaders do and replicable
descriptions are needed. Further research is required to determine if
opinion leaders can be identified and in which circumstances they are
likely to influence the practice of their peers.
Reference Thomson O’Brien MA, Oxman AD, Haynes RB, Davis DA, Freemantle
N, Harvey EL Local opinion leaders: effects on professional practice and
health care outcomes. Cochrane Database of Systematic Reviews, 1999,
(1):CD000125.
 Annex 1 : E v i d e n c e - b a s e d h e a l t h w o r k e r s b e h a v i o u r c h a n g e 31

Audit and feedback

Cochrane Library Issue 4, 2003; last updated 25 August 2003
Question Does feedback to health professionals on their performance relative to
their peers or accepted guidelines improve professional practice or health
care outcomes?
85 studies involving over 3500 health professionals were included.
Systematic 52 studies compared audit and feedback (+ or – printed educational
review summary materials) to no intervention showed an absolute improvement in
practice of -9%–70% (median = 0.07, inter-quartile range = 0.02–
0.11).
Reviewers Audit and feedback can be effective in improving professional practice.
conclusions When it is effective, the effects are generally small to moderate. The
absolute effects of audit and feedback are more likely to be larger when
baseline adherence to recommended practice is low. The findings
do not support the conclusions of previous reviews that multifaceted
interventions are more likely to be effective than single interventions.
Reference Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman
AD Audit and feedback: effects on professional practice and health
care outcomes. Cochrane Database of Systematic Reviews, 2003, (3):
CD000259.

Educational outreach visits

Cochrane Library Issue 4, 2003; last updated 1 September 1997

Question Can a personal visit by a trained person to a health-care provider in his or
her own setting improve professional practice or patient outcomes?
18 trials involving 1896 health professionals were included.
Systematic Targeted behaviours included prescribing (13 studies); preventive
review summary services (3 studies); general management of common problems
(2 studies).
Small to moderate effects on practice were observed in all the studies.
Reviewers Educational outreach visits can change health professional behaviour,
conclusions especially prescribing; the effect on other aspects of practice is uncertain.
Research is needed to identify key characteristics of outreach visits that
are important to its success. The cost–effectiveness of outreach visits is
not well evaluated.
Reference Thomson O’Brien MA, Oxman AD, Davis DA, Haynes RB, Freemantle
N, Harvey EL. Educational outreach visits: effects on professional practice
and health care outcomes. Cochrane Database of Systematic Reviews,
2000, (2):CD000409.
32 Evidence-led obstetric care: strategies to change practice and policy

Continuous quality improvement (CQI)

Non-Cochrane systematic review; last updated 1998
Question Do continuous quality improvement efforts improve outcomes of care and
reduce costs in clinical settings?
Systematic 41/43 single-site before-and-after studies showed a positive effect.
review Three randomized controlled trials showed no effect on clinical outcomes
summary or organization-wide improvement in clinical performance.

Reviewers Effectiveness of CQI is enhanced by clinician involvement, individual

conclusions practitioner feedback and a supportive organizational culture
Reference Shortell SM, Bennett CL, Byck GR. Assessing the impact of continuous quality
improvement on clinical practice: what will it take to accelerate progress.
Milbank Quarterly 1998, 76, (4):593–624.
 Annex 2 33

Abstracts of RHL-related projects

Practice audit and national dissemination in China

Abstract from: Qian X, Smith H, Zhou L, Liang J, Garner P. Evidence-based
obstetrics in four hospitals in China: An observational study to explore clinical
practice, women’s preferences and provider’s views. BMC Pregnancy and
Childbirth, 2001, 1:1 (16 May 2001).

Background: Evidence-based obstetric care is widely promoted in developing

countries, but the success of implementation is not known. Using selected
childbirth care procedures in four hospitals in Shanghai, we compared practice
against evidence-based information, and explored user and provider views
about each procedure.

Methods: Observational study. Using the Cochrane Library, we identified

six procedures that should be avoided as routine and two that should be
encouraged. Procedure rate was determined by exit interviews with women,
verified using hospital notes. Views of women and providers were explored
with in-depth interviews. The study sites were three hospitals in Shanghai and
one in neighbouring province of Jiangsu. The study comprised 150 women
at each centre for procedure rate, and 48 women and providers had in-depth
interviews.

Results: Vaginal births were 50% (303/599) of the total. Of the six practices
where evidence suggests they should be avoided as routine, three were
performed with rates above 70%: pubic shaving (3 hospitals), rectal
examination (3 hospitals), and episiotomy (3 hospitals). Most women delivered
lying down, pain relief was rarely given, and only in the urban district hospital
did women routinely have a companion. Most women wanted support or
companionship during labour and to be given pain relief; but current practice is
insufficient to meet women’s needs.

Conclusion: Obstetric practice in the hospitals studied is not following best

available evidence. There is a need to adjust hospital policy to support the
use of interventions proven to be of benefit to women during childbirth, and
develop approaches that ensure clinical practice changes.

Practice audit and educational intervention to influence

practice in the United Republic of Tanzania
Abstract from: Smith H, Lugina H, Mlay R. Using an educational workshop to
promote mobility during childbirth in government hospitals in Dar Es Salaam
and the Coast region, Tanzania. Submitted to East African Medical Journal,
2004, 81(11): 603-608.
34 Evidence-led obstetric care: strategies to change practice and policy

Introduction: “Evidence-based practice” is a term now frequently used in all health-

related disciplines. Providing care according to the principles of evidence-based practice
assumes ready access to relevant information sources, and an understanding of the
approach, and skills to use the information. Access to up-to-date health information is
improving, especially given the efforts of free-access initiatives, but acquiring the skills to
understand and use evidence in practice is not straightforward.

Objectives: To evaluate the impact of an educational intervention to introduce the

principles of evidence-based obstetric care and encourage mobility during labour in
government hospitals in Dar es Salaam, and to explore the barriers and opportunities to
implementing evidence-based practice.

Methods: The study used a before-and-after design with quantitative and qualitative
methods. We documented practice rates for mobility during labour at baseline, conducted
educational workshops with labour ward staff from four hospitals, and followed-up
practice rates six to nine months after the workshop.

Results: Following implementation of the workshops, practice changed significantly for

mobility during labour at the district and regional hospitals; midwives at these hospitals
appeared to be aware of the benefits and were willing to encourage women to be mobile
during labour. Practice changed little at the referral hospital, but most women delivering
at this hospital are likely to have been transferred from regional or district hospitals with
complications, which could explain why most women were confined to bed rather than
ambulant.

Discussion: This small study demonstrates the potential of using educational workshops
to change childbirth practice at the district and regional levels in the United Republic
of Tanzania. Institutionalization of practice changes involves influencing policy-makers,
engaging opinion leaders to disseminate evidence and advocate for change among their
peers, as well as developing mechanisms to overcome barriers to practice change at
the hospital level. Results from this study will feed into a project to develop evidence-
based guidelines for childbirth care, involving policy-makers, practitioners and hospital
managers.

Using a focused change programme (the Better Births Initiative)

to influence practice in South Africa
Abstract from: Helen Smith, Heather Brown, G Justus Hofmeyr, Paul Garner. Evidence-
based obstetric care in South Africa: influencing practice through the “Better Births
Initiative”. South African Medical Journal, 2004, 94: 117–120.

Ensuring health professionals practice according to evidence-based standards is important,

since it affects the quality and cost of care patients receive. The purpose of this research
was to use a focused change programme (the Better Births Initiative) to influence obstetric
practice at ten hospitals in Gauteng Province, South Africa. The findings show some
important improvements in practice following the implementation of the BBI; providers
at some sites reduced the use of enemas, shaving and episiotomy, and increased use of
oral fluids and companionship during labour. Qualitative data suggest that an interactive
approach to implementing evidence-based practice can influence health professionals”
decisions to change practice, and that good working relationships and enthusiastic staff are
central to effective change.
 Annex 2: Abstracts from RHL related projects 35

Trial to evaluate a programme promoting evidence on the RHL in

Mexico and Thailand
Abstract from: Gülmezoglu AM, Villar J, Grimshaw J, Piaggio G, Lumbiganon P, Langer A.
Cluster-randomized trial of an active, multifaceted information dissemination intervention
based on The WHO Reproductive health library to change obstetric practices: methods
and design issues [ISRCTN14055385]. BMC Medical Research Methodology, 2004, 4:2 (15
January 2004).

Background: Effective strategies for implementing best practices in low- and middle-
income countries are needed. RHL is an annually updated electronic publication
containing Cochrane systematic reviews, commentaries and practical recommendations
on how to implement evidence-based practices. We are conducting a trial to evaluate
the improvement in obstetric practices using an active dissemination strategy to promote
uptake of recommendations in the WHO Reproductive Health Library (RHL).

Methods: A cluster-randomized trial to improve obstetric practices in 40 hospitals in

Mexico and Thailand is conducted. The trial uses a stratified random allocation based on
country and size and type of hospitals. The core intervention consists of three interactive
workshops delivered over a period of six months. The main outcome measures are changes
in clinical practices that are recommended in RHL measured approximately a year after the
first workshop.

Results: The design and implementation of a complex intervention using a cluster-

randomized trial design are presented.

Conclusion: Designing the intervention, choosing outcome variables and implementing

the protocol in two diverse settings has been a time-consuming and challenging process.
We hope that sharing this experience will help others planning similar projects and
improve our ability to implement change.

Trial to evaluate a programme to promote support for women

during labour in South Africa
Abstract from: Brown H, Hofmeyr GJ, Nikodem VC, Smith H, Garner P. Survey of
childbirth practices in South African state maternity services and a randomized trial of an
intervention to promote childbirth companions. Paper in progress.

Background: Care for women during childbirth in South African state maternity units is
not evidence-based and instances of poor quality of care such as shouting, slapping or
striking of women occur. We hypothesized that by providing maternity staff with access
to evidence-based information and an encouraging the use of childbirth companions the
quality of care for women during childbirth would improve.

Methods: A cluster-randomized trial nested within a before-and-after design at ten

maternity units in South Africa.

After carrying out a baseline survey of practice we provided access to evidence-based

information for maternity staff in the form of the World Health Organization Reproductive
36 Evidence-led obstetric care: strategies to change practice and policy

Health Library (RHL), computer hardware and training to all ten sites. Five hospitals were
then randomly selected to receive an additional educational intervention to promote the
implementation of childbirth companions.

Outcomes were change in the proportion of women who had a companion during
childbirth (and other obstetric practices) and instances of poor quality of care measured by
observations during the study and exit interviews with 2058 postnatal women at follow-up
eight months after the intervention and compared to baseline results. The effect of access
to the WHO RHL is reported separately.

Results: Despite an initial positive response from staff to the childbirth companion
intervention, no difference on the proportion of women who had a childbirth companion,
or on the quality of care was demonstrated. There was a reduction in episiotomy and
women moving around during the second stage of labour at the intervention sites
compared with control sites.

Conclusion: Introducing support for women during labour is difficult, particularly in

under-resourced health care systems. Lessons learnt from this study contributed to the
development of the Better Births Initiative, an international initiative aimed at providing
humane and evidence-based care for women during childbirth.

Antenatal care policy change in Thailand and Argentina

Abstract from: Villar J, Ba’aqeel H, Piaggio G, Lumbiganon P, Miguel Belizan J, Farnot U,
Al-Mazrou Y, Carroli G, Pinol A, Donner A, Langer A, Nigenda G, Mugford M, Fox-Rushby
J, Hutton G, Bergsjo P, Bakketeig L, Berendes H, Garcia J; WHO Antenatal Care Trial
Research Group. WHO antenatal care randomized trial for the evaluation of a new model
of routine antenatal care. Lancet, 2001,19;357(9268):1551–1564.

Background: We undertook a multicentre randomized controlled trial that compared the

standard model of antenatal care with a new model that emphasizes actions known to be
effective in improving maternal or neonatal outcomes and has fewer clinic visits.

Methods: Clinics in Argentina, Cuba, Saudi Arabia, and Thailand were randomly allocated
to provide either the new model (27 clinics) or the standard model currently in use (26
clinics). All women presenting for antenatal care at these clinics over an average of 18
months were enrolled. Women enrolled in clinics offering the new model were classified
on the basis of history of obstetric and clinical conditions. Those who did not require
further specific assessment or treatment were offered the basic component of the new
model, and those deemed at higher risk received the usual care for their conditions;
however, all were included in the new-model group for the analyses, which were by
intention to treat. The primary outcomes were low birthweight (<2500 g), pre-eclampsia/
eclampsia, severe postpartum anaemia (<90 g/L haemoglobin), and treated urinary-tract
infection. There was an assessment of quality of care and an economic evaluation.

Findings: Women attending clinics assigned the new model (n=12 568) had a median
of five visits compared with eight within the standard model (n=11 958). More women in
the new model than in the standard model were referred to higher levels of care (13.4%
 Annex 2: Abstracts from RHL related projects 37

vs 7.3%), but rates of hospital admission, diagnosis, and length of stay were similar. The
groups had similar rates of low birthweight (new model 7.68% vs standard model 7.14%;
stratified rate difference 0.96 [95% CI −0.01 to 1.92]), postpartum anaemia (7.59% vs
8.67%; 0.32), and urinary-tract infection (5.95% vs 7.41%; −0.42 [−1.65 to 0.80]). For pre-
eclampsia/eclampsia the rate was slightly higher in the new model (1.69% vs 1.38%; 0.21
[0.25 to 0.67]). Adjustment by several confounding variables did not modify this pattern.
There were negligible differences between groups for several secondary outcomes. Women
and providers in both groups were, in general, satisfied with the care received, although
some women assigned the new model expressed concern about the timing of visits. There
was no cost increase, and in some settings the new model decreased cost.

Interpretations: Provision of routine antenatal care by the new model seems not to affect
maternal and perinatal outcomes. It could be implemented without major resistance from
women and providers and may reduce cost.
38 Evidence-led obstetric care: strategies to change practice and policy
 Annex 3 39

Meeting participants

Dr Heather Brown Dr Ana Langer (Chair)

Specialist Registrar Population Council
Obstetrics and Gynaecology Panzacola 62
Princess Anne Hospital Int. 102, Colonia Villa Coyoacán
Southampton, UK CP 04000
Email: [email protected] Mexico DF, Mexico
Tel: 52 55 5999 8630
Email: [email protected]

Dr Guillermo Carroli Dr Helen Lugina

Centro Rosarino de Estudios Perinatales Africa Midwives Research Network
San Luis 2493 C/o Faculty of Nursing
2000 San Rosario Muhimbili Univesity College of Health
Argentina Sciences
Tel: 54 341 447 2625 PO Box 65004
Fax: 54 341 448 3887 Dar es Salaam, Tanzania
Email: [email protected] Tel/Fax: 255 51 153 506
Email: [email protected]

Prof. Paul Garner Dr Pisake Lumbiganon

International Health Research Group Khon Kaen University
Liverpool School of Tropical Medicine Dept. of Obstetrics & Gynaecology
Pembroke place Faculty of Medicine
Liverpool L3 5QA Khon Kaen
UK Thailand
Tel: 44 151 795 3201 Tel: 66 1 871 9039
Fax: 44 151 705 3364 Fax: 66 43 348 395
Email: [email protected] Email: [email protected]

Prof. Jeremy Grimshaw Dr Andy Oxman

Ottawa Health Research Institute Department of Health Services Research
Clinical Epidemiology Program Norwegian Directorate for Health and
1053 Carling Avenue C-403 Social Welfare
Ottawa ON Pb 8054 Dep
K1Y 4E9 0031 Oslo
Canada Norway
Tel: 613 761 5231 Tel: 47 24 16 32 90
Fax: 613 761 5492 Fax: 47 24 16 30 11
Email: [email protected] Email: [email protected]
40 Evidence-led obstetric care: strategies to change practice and policy

Prof. Justus Hofmeyr Dr Helen Smith

Frere/Cecilia Makiwane Hospitals International Health Research Group
P Bag 9047 Liverpool School of Tropical Medicine
East London 5200
Pembroke place
Eastern Cape
South Africa Liverpool L3 5QA
Tel/Fax: 27 43 709 2483 UK
Email: [email protected] Tel: 44 151 795 3201
Fax: 44 151 705 3364
Email: [email protected]

Ms Yvonne Thomas Dr Qian Xu

Policy Division School of Public Health
Department for International Development Fudan University
1 Palace Street 138 Yi Xue Yuan Road
London SW1E 5HE Shanghai 200032
UK P R China
Tel: 44 20 7023 0557 Tel/Fax: 86 21 641 741 72
Fax: 44 20 7023 0105 Email: [email protected]
Email: [email protected]

WHO Regional Office representatives

Dr William Adu-Krow Dr Therese Lesikel
WHO Office of Caribbean Program WHO Regional Office for Africa (AFRO)
Coordination (PAHO) BP6
PO Box 508 Bridgetown Brazzaville
Dayrells Road & Navy Gardens Republic of Congo
Christchurch, Barbados Tel: 47 241 391 44
Tel: 246 426 3860 ext: 5019 Fax: 47 241 395 17
Fax: 246 436 9779 Email: [email protected]
Email: [email protected]

Dr Ardi Kaptiningsih Dr Hatem El-Din N Hassan Mohamed

WHO Regional Office for the South-East WHO Regional Office for the Eastern
Pacific (SEARO) Mediterranean (EMRO)
World Health House Nasser City
Indeaprastha Estate Cairo 11371
Mahatma Ghandi Road Egypt
New Delhi 11000 Tel: 20 2 670 2535
India Fax: 20 2 670 2492
Tel: 91 112 337 0804 Email: [email protected]
Fax: 91 112 337 0197
Email: [email protected]
 Meeting participants 41

Dr Gunta Lazdane
WHO Regional Office for Europe (EURO)
8 Scherfigsvej
DK-2100 Copenhagen
Denmark
Tel: 45 3917 1426
Fax: 45 39171850
Email: [email protected]

Secretariat WHO-HQ, FCH/RHR

Dr Metin Gülmezoglu Dr Enrique Ezqurra
Mr Raju Khanna Mrs Åsa Cuzin
Dr Lale Say Ms Harriet Kabagenyi
Dr Davy Chikamata Mr Paulo Dos Santos
Dr Katherine Ba-Thike
42 Evidence-led obstetric care: strategies to change practice and policy
 References 43

1 Chalmers I, Enkin M, Keirse MJNC, eds. Effective care in pregnancy and childbirth.
Oxford, Oxford University Press, 1989.

2
Grol R. Improving the quality of medical care: building bridges among professional pride,
payer profit and patient satisfaction. JAMA, 2001, 286(20):2578–2585.

3 Thomas J, S Paranjothy. Royal College of Obstetricians and Gynaecologists (RCOG).

National Sentinel Caesarean Section Audit Report. London, RCOG Press, 2001.

4 Carroli G, Belizan J. Episiotomy for vaginal birth. Cochrane Database of Systematic

Reviews,1999, (2):CD000081.

5 Duley L, Henderson-Smart D. Magnesium sulphate versus diazepam for eclampsia.

Cochrane Database of Systematic Reviews, 2003, (3):CD000127.

6 Basevi V, Lavender T. Routine perineal shaving on admission in labour. Cochrane

Database of Systematic Reviews, 2000, (4):CD001236.

7 Cuervo LG, Rodríguez MN, Delgado MB. Enemas during labour. Cochrane Database of
Systematic Reviews, 1999, (3):CD000330.

8 Thacker SB, Stroup D, Chang M. Continuous electronic heart rate monitoring for fetal
assessment during labor. Cochrane Database of Systematic Reviews, 2001, (2):CD000063.

9 Villar J, Carroli G, Khan-Neelofur D, Piaggio G, Gülmezoglu M. Patterns of routine

antenatal care for low-risk pregnancy. Cochrane Database of Systematic Reviews, 2001,
(4):CD000934.

10 Thomson O’Brien MA, Freemantle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing
education meetings and workshops: effects on professional practice and health care
outcomes. Cochrane Database of Systematic Reviews, 2001, (1):CD003030.

11 Thomson O’Brien MA, Oxman AD, Haynes RB, Davis DA, Freemantle N, Harvey EL.
Local opinion leaders: effects on professional practice and health care outcomes.
Cochrane Database of Systematic Reviews, 1999, (1):CD000125.

12 Jamtvedt G, Young JM, Kristoffersen DT, Thomson O’Brien MA, Oxman AD. Audit and
feedback: effects on professional practice and health care outcomes Cochrane Database of
Systematic Reviews, 2003, (3):CD000259.

13 Freemantle N, Harvey EL, Wolf F, Grimshaw JM, Grilli R, Bero LA. Printed educational
materials: effects on professional practice and health care outcomes Cochrane Database of
Systematic Reviews, 2001, (2):CD000172.

14 Thomson O’Brien MA, Oxman AD, Davis DA, Haynes RB, Freemantle N, Harvey EL.
Educational outreach visits: effects on professional practice and health care outcomes
Cochrane Database of Systematic Reviews, 2000, (2):CD000409.

15 Shortell SM, Bennett CL, Byck GR. Assessing the impact of continuous quality
improvement on clinical practice: what will it take to accelerate progress. Milbank
Quarterly, 1998, 76, (4):593–624.
44 Evidence-led obstetric care: strategies to change practice and policy

16 Aaserud M, Dahlgren AT, Sturm H, Kösters JP, Hill S, Furberg CD, Grilli R, Henry DA, Oxman AD,
Ramsay C, Ross-Degnan D, Soumerai SB. Pharmaceutical policies: effects on rational drug use.
Cochrane Database of Systematic Reviews, 2003, (3):CD004397.

17 Garner P, Smith H, Kale R, Dickson R, Dans T, Salinas R. Implementing research findings in developing
countries. In : Haines A, Donald A, eds. Getting research findings into practice. London, BMJ Books,
2002.

18 Briggs CJ, Capdegelle P, Garner P. Strategies for integrating primary health services in middle- and
low-income countries: effects on performance, costs and patient outcomes. Cochrane Database of
Systematic Reviews, 2001, (4):CD003318.

19 Cattaneo A, Buzzetti R. Effect on rates of breast feeding of training for the baby friendly hospital
initiative. British Medical Journal, 2001,323 (7325):1358–1362.

20 Kramer MS, Chalmers B, Hodnett ED, Sevkovskaya Z, Dzikovich I, Shapiro S. Promotion of

Breastfeeding Intervention Trial (PROBIT): a randomized trial in the Republic of Belarus. JAMA, 2001,
285(4):413–420.

21 Fairbank L, O’Meara S, Renfrew MJ, Woolridge M, Sowden AJ, Lister-Sharp D. A systematic review to
evaluate the effectiveness of interventions to promote the initiation of breastfeeding. Health Technology
Assessment, 2000, 4(25):1-171.

22 Grol R. Beliefs and evidence in changing clinical practice. British Medical Journal, 1997,
315(7105):418–421.

23 Berwick D. A primer on leading the improvement of systems. British Medical Journal, 1996,
312(7031):619–622.
45