Dr Yaser Adi MD, MPH, MSc HTA

Scientific Advisor for the Saudi Centre

for Evidence Based Health Care (EBHC)
Madeenah 31st Mar 2015

Aim

To introduce you to the

EBM/EBHC concept and & its applications

Objectives
How to be successful in you career
Explain the components of EBM
Benefits of EBM
Hierarchy of evidence
An example to illustrate the EBHC use
Clinical practice guidelines
Mobile App at EBHC

To be successful in your job

you do need
Two words that describe two things...
What are they?

Competence

Is the ability to perform a specific

task successfully.
Incompetence is the (opposite)

Confidence

A belief in yourself that you

can do this job and do it well

Competence & Confidence

High

Confidence

DESPERATELY
DELUDED

YOURE
DESTINED
FOR SUCCESS

YOU NEED
HELP!
Low

YOURE NOT
ACHIEVING
YOUR
POTENTIAL
High

Competence

EBM/EBHC

What is Evidence Based Medicine?

Evidence:
A thing or things helpful in forming a conclusion or
judgment

Medicine:
The art and science of the diagnosis, treatment, and
prevention of disease and the maintenance of good
health

EBM/EBHC

Non evidence based medicine

Any examples that you may think of?

Babies to sleep on their tummies?

Rosiglitazone 1999-2010
In Europe, the European Medicines Agency
(EMA) recommended in September 2010 that
the drug be suspended from the European
market because the benefits of rosiglitazone no
longer outweighed the risks.

HRT
In observational studies show positive
effect on heart for postmenopausal ladies
Later on, after a large RCT, it was proved
the opposite!

Benefits of EBHC
Daily need for up-to-date reliable information
Inadequacy of traditional sources of
information
Develop skills for life long learning

Benefits of EBHC

Patient satisfaction
Clinicians fulfillment
Healthier nation

Benefit of EBHC

Textbooks - out of date

Experts - frequently wrong
Didactic CME - ineffective
Medical journals
Overwhelming in volume
Variable in validity

EBHC is then simple

But
how do we do it ?

EBHC is simple! The 5 As

1. Asking an
answerable question

P
I/E
C
O

1. Asking an answerable question

Population (P)
Intervention /exposure (I)
Comparator (C)
Outcome (O)
Using the paper that you have, what is
the question you want to answer?

1. Asking an answerable question

In people aged 66 years or older,

who are receiving ACE inhibitors,
would exposure to co-trimoxazol
compared with other antibiotics
cause sudden death?

EBHC is simple! The 5 As

1. Asking
answerable question

P
I/E
C
O

2. Accessing
the best evidence

Secondary source
Primary source

2.Sources
Searching
for evidence
of medical
info.

Sources of information

2. Searching for evidence in pubmed

Search Item

Identified

inhibitors of renin-angiotensin

8894

inhibitors of renin angiotensin

11349

Cotrimoxazole

8178

Co-trimozazole

8008

Sudden death

47621

((inhibitors of renin angiotensin) OR (inhibitors of reninangiotensin)) AND (Cotrimoxazole OR Co-trimozazole)

AND (Sudden death)

Tips for searching

Results in Pubmed

EBHC is simple! The 5 As

1. Asking
answerable question

P
I
C
O

2. Accessing
the best evidence

Secondary source
Primary source

3. Appraising
the evidence

Valid?
Important?
Can it help?

http://www.casp-uk.net/#!casp-tools-checklists/c18f8

3. Appraising the evidence

Please have a look at the paper

& the CASP questions

EBHC is simple! The 5 As

1. Asking
answerable question

P
I
C
O

2. Accessing
the best evidence

Secondary source
Primary source

3. Appraising
the evidence

Valid?
Important?
Can it help?

4. Applying the evidence

How much will it help?

Patients value
Cost-effective

EBHC is simple! The 5 As

1. Asking
answerable question

P
I
C
O

2. Accessing
the best evidence

Secondary source
Primary source

3. Appraising
the evidence

Valid?
Important?
Can it help?

4. Applying the evidence

How much will it help?

Patients value
Cost-effective

5. Assessing the
performance

How could you do it better

next time

Clinical questions &study design to

answer them
You accompany a relative to the clinic
suspected to have DM.
List at least three questions that your relative
think important to inquire about
Q.1
Q.2
Q.3

Typology for question building

Type of question

Meaning

Study design

Aetiology/Harm

The causes of a disease

Case control or
Cohort study

Diagnosis

Signs , symptoms or tests for

diagnosing a disorder. Comparison to
goal standard

Diagnostic validation
study

Prognosis

The probable course of disease over

time

Inception cohort

Therapy/Prevention

Effective treatments which meet your

patient's values

Systematic review,
Randomized control
trial

Cost-effectiveness

Is one intervention more cost-effective

than others?

Economic evaluation

Quality of life

What will be the quality of life of the

patient?

Qualitative study

Hierarchy of Evidence

A question about effectiveness

Where do you look for information first?

A question about effectiveness

Type of question

Meaning

Study design

Aetiology/Harm

The causes of a disease

Case control or
Cohort study

Diagnosis

Signs , symptoms or tests for

diagnosing a disorder. Comparison to
goal standard

Diagnostic validation
study

Prognosis

The probable course of disease over

time

Inception cohort

Therapy/Prevention Effective treatments which meet your

patient's values

Systematic review,
Randomized control
trial

Cost-effectiveness

Is one intervention more cost-effective

than others?

Economic evaluation

Quality of life

What will be the quality of life of the

patient?

Qualitative study

Cochrane Library

CRD

What is a Systematic Review?

More than one study addressing a particular
health question. It is logical to collect all these
studies together and base conclusions on the
cumulated results.
The most obvious sign that a review is
systematic will be the presence of a methods
section. Meta-analysis is the statistical process
of combining the results from several studies
that is often part of a systematic review.

What is a Randomised Controlled

Trial (RCT)?
An RCT is a type of interventional or experimental
study design. Participants (individuals or groups)
are randomly allocated to receive either the new
intervention or a control treatment (usually the
standard treatment or a placebo).
Each arm of the study is then followed up and the
amount or severity of the disease measured in
the intervention group and compared with the
control group.
RCTs are by definition prospective.

What is a Cohort study?

Cohort study, also known as a follow-up or longitudinal
study, is another observational study design. In this
study a population who do not have the health
outcome or disease of interest
Are first divided into those who are exposed to a risk
factor, often over long periods of time.
At the end of the period of observation the incidence
of disease or frequency of health outcome in the
exposed group is compared to that in the unexposed
group. The study is generally prospective as it looks
forward from potential cause to consequence.

What is a Case-Control study?

A case-control study belongs to the observational
group of studies. It begins by choosing individuals who
have a health outcome or disease whose cause you
want to investigate. These are the cases.
Controls without the health outcome are then chosen.
You then determine the proportion of cases who were
exposed to any risk factor of interest in the past, and
compare this with the proportion exposed in the
control group.
The study is generally retrospective because it looks
backwards in time to the earlier exposures of
individuals.

Summary
Evidence-based medicine is a systematic approach to care for
your patients

EBM requires Skills that are needed to integrate the available

evidence with clinical experience and patient concern
EBM skills will provide a frame work for life-long learning.

A simple & easy to read EBM book!

Clinical Practice
Guideline (CPG)

What is a Clinical Practice

Guideline (CPG) ?

How many CPGs are there?

GIN Library contains 6476
(by 29 March 2015) guidelines registered

What should be the

standard number

ONE
that makes a CPG trustworthy?

The eight standards from the IOM

1. Establishing transparency

The processes by which a

CPG is developed and funded
should be detailed explicitly
and publicly accessible.

Vague recommendations e.g.

Patient with such (x) condition should
be offered the intervention (y) if
clinically appropriate.
Clinicians should follow up with
patients given the intervention every 4
weeks, or sooner if necessary.

2. Conflict of interest (COI) -Definition

Any clash between the member's selfinterest (personal gain or to their family)

&
CPG recommendations

3.Guideline development group

composition (GDG)
A multidisciplinary team

Patient /carer/representative
Why it is important?

The Hypertension Clinical Guideline

CG127, Aug 2011

4. Clinical practice guideline

systematic review (SR) intersection
The new definition by the IOM of CPG:
CPGs are statements that include
recommendations intended to optimized
patient care that are informed by a systematic
review of evidence and an assessment of
benefits and harms of alternative options

An interactive slide from IOM

5. Establishing (QoE) and rating

strength of recommendations
Each recommendation should provide:

A clear description of potential benefits or/harms.

Supporting evidence /or lack of it
Strength of recommendation

How is the (QoE) evidence

categorized in
?

++++

High

We are very confident that the true effect is close

to that of the estimate of the effect.

+++

Moderate

We are moderately confident in the effect

estimate: The true effect is likely to be close to the
estimate of the effect, but there is a possibility that
it is substantially different.

++

Low

Our confidence in the effect estimate is limited:

The true effect may be substantially different from
the estimate of the effect.

Very low

We have very little confidence in the effect

estimate: The true effect is likely to be
substantially different from the estimate of effect.

Strength of recommendation in

GRADE

What is meant by the

"strength of recommendation"?
Recommendations to administer, or not administer,
an intervention, should be based on the tradeoffs
between benefits on the one hand, and risks, burden
and, potentially, costs on the other.
If benefits outweigh risks and burden, experts will
recommend that clinicians offer a treatment to typical
patients.

GRADEs strong recommendation if...

Strong methods
Large & precise effect
Few down sides of therapy

A weak recommendation if ...

Benefits and risks and burdens are finely

balanced,
Weak methods
Small effect
Imprecise estimate
Substantial down sides

6. Articulation of recommendations
Under what circumstances
A patient should be given the
intervention
QoE
The strength of recommendation
should be stated in proximity to each
recommendation.

e.g. 2 from the ACP:

Ann Intern Med. 2011;155:625-632

ACP recommends:
Pharmacologic prophylaxis with heparin or a
related drug for venous thromboembolism in
medical (including stroke) patients unless the
assessed risk for bleeding outweighs the likely
benefits.
(Moderate-quality evidence, Strong recommendation).

Factors panels should consider in deciding

on a strong /weak recommendations
1. Magnitude of treatment effect (large effect?)
2. Precision of estimate of treatment Effect (95% CI)
3. Method design & [ Bias , confounding, chance]

4. Burden of Therapy
5. Risks associated with therapy
6. Costs
7. Patients values

Study limitations
Inconsistency of results
Indirectness of evidence
Imprecision
Reporting bias

The State of Art: QoE, Recommendation

& Interpretation
QoE

Recommendation Interpretation

High

Strong

High

Weak

Low

Strong

Low

Weak

Apply to most patients without

reservation. (RCTs/SRs)
Most patients would want, some would
not, depends on individuals
circumstances, (RCT, or overwhelming
observational) e.g.
Apply but may change if new evidence
becomes available (Observational)
e.g. from the past {HRT}
Decision can NOT be made

7. External review
Full spectrum or stakeholders
Addressing Responses
Should be made available to the
general public for comment before
publication

8. Updating
Proposing a date for updating

Monitoring the literature

Modification in response to new evidence

Conclusion

A trustworthy CPG should comply

with 1-8 standards

Can you remember the 8 Standards?

1. Establishing transparency

2. Management of conflict of interest (COI)

3. GDG composition
4. Clinical practice guidelineSR intersection
5. QoE & strength of recommendations
6. Articulation of recommendations
7. External review
8. Updating

Feb 2012

Types of CPGs
DE novo
Adapted
Adopted

A new approach to CPG adaptation in

Saudi Arabia:
Adaptation of practice guidelines to a
country-specific context using the
GRADE/DECIDE evidence to decision
framework

Project Methodology

Adaptation

Development

Adoption

Adolopment

Guideline Ad-o-lopment
Ad-o-lopment = Adaptation + Adoption +
Development
Approach to the development of guidelines that
begins with identifying existing evidence syntheses,
including systematic reviews, HTAs, and evidence
reports, which may have been produced to support
previous guidelines and address specific clinical
questions.
Followed by the updating of the evidence syntheses
and development of guideline recommendations
specific to the healthcare setting.

Evidence Synthesis
Panels prioritized questions to be included in
guidelines (online surveys)
McMaster guideline leaders updated literature
searches
Conducted literature searches specific to the
Saudi healthcare setting: patients values and
preferences, cost-effectiveness & economic data
Produced evidence summaries: GRADE Evidence
Profiles and Summary of Findings Tables

Formulating Recommendations
Online training modules for panels and 1-day
workshop on guideline development
In-person panel meetings, facilitated by
McMaster guideline leaders
Recommendations formulated using the
Evidence-to-Decision (EtD) framework

Evidence-to-Decision Framework
Factors that bear on recommendations and
their strength
Enables formulation of recommendations
tailored to the specific healthcare setting,
through consideration of the factors outlined
in the framework (e.g. patients values and
preferences in local setting, resources
acceptability, feasibility)

Collaboration Model

Saudi Experts (Medical

Societies)

Saudi Center for EBHC

Project Management & Facilitation

Project coordination (e.g. workshops, panel meetings, communication etc.)
Facilitate guideline topics selection by stakeholders and decision makers
Recruit panel members
Facilitate communication with panels
Review final reports
Disseminate guidelines (website, mobile apps, print media, BMJ, newsletters)

Our mission at the Saudi Center for EBHC

To promote the awareness and practice of Evidence-based
medicine across the Kingdom, through training, awareness
campaigns, and the creation of robust and nationally agreed on
clinical practice guidelines (CPGs)

Initiative
The Ministry of Health of Saudi Arabia (KSA) partnered with McMaster
University to develop multiple CPGs for the local healthcare setting
based on the GRADE approach and the GRADE/DECIDE evidence to
decision (EtD) framework

Target
Produced 10 CPGs in a 4-month time period (Sep Dec 2013)

10 Completed CPGs
Diagnosis of Deep Vein Thrombosis

Saudi Scientific Hematology Society

Anticoagulant Therapy for Atrial Fibrillation

Saudi Heart Association

Anticoagulant Therapy for Venous Thromboembolism

Saudi Scientific Hematology Society

Anticoagulant Therapy for Acute Stroke Management

Saudi Stroke Association

Venous Thromboembolism prevention in Stroke

Saudi Stroke Association

Allergic Rhinitis in Asthma

Saudi Allergy, Asthma and Immunology Society

Cervical Cancer Screening and Treatment

Saudi Obstetric and Gynecology Society

Breast Cancer Screening

Saudi Oncology Society

Role of Vitamin D, Calcium, and Exercise in Fracture

Prevention

Saudi Osteoporosis Society

Timing of Initiation of Hemodialysis

Saudi Society of Nephrology and Transplantation

Diagnosis of Deep Vein Thrombosis

Saudi Scientific Hematology Society

How were the CPG topics selected?

Number of topics suggested by individual departments of the Ministry of Health

Suggested topics screened by McMaster Group for feasibility of adaptation

Screened topics presented to Ministry decision makers for final selection of

guideline topics

Recruited multidisciplinary panel of local experts relevant to each CPG topic

Results
Produced 10 CPGs with 80 recommendations achieved in 4
month time period
Produced a Manual for CPG development for Saudi Arabia

Dissemination

Printed CPGs

Newsletters

EBHC website

BMJ Best Practice

Mobile apps

Question/Problem
Benefits and harms
Quality of evidence
Values and
Preferences
Resource use
Impact on health
equity
Acceptability
Feasibility
Recommendation

E
vidence to decision

Where is CPGs in the

Hierarchy of Evidence

Where to search for CPG?

Guidelines International Network (G-I-N)

www.g-i-n.net

National Guidelines Clearing House

www.guidelines.gov

PubMed
Google

Guideline international network (GIN)

National Guideline Clearing House

(NGCH)

National institute for health and care

excellence (NICE)

A tool to assess the CPGs

Agree II domains
Domain 1. Scope and Purpose
Domain 2. Stakeholder Involvement
Domain 3. Rigour of Development
Domain 4. Clarity of Presentation

Domain 5. Applicability
Domain 6. Editorial Independence
Overall Assessment

Conclusions 1
In this unique collaboration, we established and applied a
methodology for adaptation of CPGs in 4-month period
The experience to produce adapted CPGs in a short period is
feasible but challenging
We succeeded because we had:
o Committed stakeholders
o Strong scientific support (McMaster Group)
o Effective project management (EBHC and McMaster Group)

Conclusions 2
Ad-o-lopment approach allows for efficient
production of guidelines
Support and facilitation from trained
methodologists to help with development of
guidelines
Evidence-to-Decision framework allows for
formulation of recommendations specific to
the local healthcare setting

The MOH/EBHC Mobile App

The MOH/EBHC Mobile App

Acknowledgment
Canadian McMaster working group
EBHC Staff