Meta-analysis

Montarat Thavorncharoensap, Ph.D.

Faculty of Pharmacy,
Mahidol University
Meta-analysis
 Meta-analysis: “.. A quantitative approach for sys
tematically combining the results of previous res
earch in order to arrive at conclusion about the b
ody of research.

Meta- analysis is the analytical or statistical part of syste

matic review!

 Petitti D.B., Meta analysis, decision analysis, and cost-effectiveness analysis. New York: Oxford Univ
ersity Press, 1994.
Why meta-analysis?
 Expanding volume of published literature
 Different or controversial results from studies of th

e same topic
Example: systematic review
Example: Meta-analysis
Literature review VS Systematic revi
ew VS Meta-analysis
High
Level of evidence
RCT study

Cohort study

Case control
study

Cross sectional
study

Case report,
Case series

Expert opinion

Low Arch Dis Child 2005.90;840-4

BMJ 2001;323;334-336
Steps in conducting a systematic review/ Me
ta analysis
Formulating review question/ Developing protocol

Searching & Selecting studies

Study quality assessment

Extracting data from studies

Data analysis and interpretation

Information from included study
 Bibliographic details
 Study characteristics
◦ Design/method
◦ Participants
◦ Interventions
◦ Outcome measures
 Study results
◦ Continuous: Mean, SD, and/or SE
◦ Dichotomous: Number of events and sample size in ea
ch groupn
Descriptive synthesis

 Describe studies
 Highlight similarity and differences
 Identify patterns of factors
Type of data to be combined in meta-a
nalysis
 Dichotomous
◦ Relative Risk, Odd Ratio, Hazard Ratio
◦ Risk Difference
 Continuous
◦ Mean Difference (MD)
◦ Standard mean difference
Model used in meta-analysis
 Fixed effect – Use when there is no heterogeneity
◦ Inverse variance method
◦ Mentel-Haenszel method
◦ Peto method
 Random effect
◦ Dersimonian-Laird method

Generally, the results from fixed effects model are similar t

o that of random effects model. However, the 95%CI of ran
dom effects model is wider. (Random effects model is then
more conservative!!)
 Understanding forest plot
Structure:
 Line of No effect
 Box: Location and size
 Pooled estimate
 95% Confidence interval
 Heterogeneity test
 Result of meta-analysis:
forest plot

Pooled analysis

Line of no effect
Ann Intern Med 2003; 138: 795-805.
Example:

Kim DH, Rogers JR, Fulchino LA, Kim CA,

Solomon DH, Kim SC (2015) Bisphosphonates and
Risk of Cardiovascular Events: A Meta-Analysis.
PLoS ONE 10(4): e0122646. doi:10.1371/journal.
pone.0122646
Example

JAMA. 2010;303(11):1077-1083
Factors associated with the quality
of meta-analysis studies
 Publication bias.
 Quality of studies combined in meta-analysis. (
garbage in-garbage out)
 Heterogeneity
 Publication Bias
 Publication bias can distort findings because trials with st
atistically significant results are more likely to get publishe
d, and more likely to be published without delay, than trial
s without significant results.
 Among published trials, those with significant results are
more likely to get published in English, more likely to be si
ded, and are more likely to be published more than once
which means that they are more likely to be identified and
included in the review.

Solution: included both published and un-published, both E

nglish and non-English
Test of publication bias: Funnel plot
Identifying Publication bias

Funnel plot
 a scatterplot of standard error (or inverse of S
E) versus estimated effect size for a group of
studies

 The results from smaller studies will be more

widely spread around the mean effect becaus
e of larger random error.
 The plot will look like a symmetrical funnel if th
ere is no publication bias.
Effect size (OR)
Effect size (OR)
 Quality of study combined in SR/Met
a-analysis

Garbage in Garbage out

If the methodological quality of trials is inadequate the
n the findings of reviews of this material may also be c
ompromised.
Solution: Assess study quality
Heterogeneity
Issue: Are the studies combinable?
 Heterogeneity is variation between the results

of a set of studies of an event)

 Heterogeneity
Heterogeneity is variation between the results of a set of studies
Issue: Are the studies combinable?

Mixing orange with apple!

Possible causes of heterogeneity
Heterogeneity can be caused by
◦ Difference in characteristics of participants (Conditi
on, eligibility criteria, etc)
◦ Difference in treatment characteristic (Type of drug
, dose, duration, mode of administration, etc)
◦ Difference in outcome measuring (Type, follow-up
duration, ways of measuring outcome, definition of
an event)
◦ Unexplainable/ chance
 Test of Heterogeneity
 Chi-square test (Q statistics)
◦ Limitation: Low statistical power (A cut-off significant leve
l of p < 0.1 is recommended)
 I2 = Percentage of total variation across studies th
at is due to heterogeneity rather than chance
◦ I2 lies between 0% (no observed heterogeneity) to 100%
◦ I2 of 25%, 50% and 75% mean low, moderate and high h
eterogeneity (BMJ 2003; 327: 557-60)
 Visual inspections of graphical representation of th
e data (forest plot) can also be useful since test of
heterogeneity has low power!
Methods for dealing with heterogeneit
y
 Change scale of outcome variable to remove heterogene
ity
◦ For example: changing binary outcome into absolute measure
 Exclude studies
 Analyses groups of studies separately
 Use random effect models
 Include covariates in a regression model (meta-regressio
n)
◦ A regression analysis can be performed to examine whether the h
eterogeneity can be explained by one or more factors across all s
tudies
How to report SR/Meta-analysis
 PRISMA Statement (Preferred Reporting Items f
or Systematic Reviews and Meta-analyses): (ne
w version of QUOROM:Quality of Reporting Of
Meta-analysis)
◦ The aim of the PRISMA Statement is to help authors improve the
reporting of systematic reviews and meta-analyses.
◦ The PRISMA Statement consists of a 27-item checklist and a four
-phase flow diagram
 MOOSE Statement (Reporting of Meta-analysis
of Observational Studies in Epidemiology)
Annals of Intern Med 2009;151;264-9
2 x 2 table
Disease No disease
Exposed A B A+B
(treatment)
Non C D C+D
exposed
(control)
A+C B+D A+B+C+D
 Relative risk (RR)

Relative risk = Cumulative incidence in exposed gr.

Cumulative incidence in unexposed gr.

= [a /(a+b)] / [c / (c+d)]
 Therefore,
RR = 1 means there is no association between
exposure and disease.
RR > 1 means exposure = risk factor
RR <1 means exposure = protective factor
Example: RR interpretation
 RR = Cumulative incidence of disease in exposed gr
Cumulative incidence of disease in unexposed gr.

Therefore,
RR = 2 means
 The exposed group is 2 times more likely to have disea

se when compared to unexposed group.

RR = 0.4 means
 The exposed group is 60 % less likely to have disease

when compared to unexposed group.

 Odd Ratio (OR)
 Odds of an event = the number of event
the number of non event
 Odd ratio = odds of exposure in cases = A/C = AD/BC
odds of exposure in control B/D

Disease No disease

Exposed A B A+B
(treatment)
Non C D C+D
exposed
(control)
= (a/c) / (b/d)
A+C B+D A+B+C+D
= ad/bc
 Odd Ratio (OR)
 OR = AD / BC
 Interpretation: Same as RR

Disease No disease

Exposed A B A+B
(treatment)
Non C D C+D
exposed
= (a/c) / (b/d)
(control)
= ad/bc A+C B+D A+B+C+D
Confidence Interval (CI)
 Confidence interval is used to express the degre
e of confidence in an estimate (such as odd rati
o, relative risk)
 Confidence Interval (CI) gives range within which tha
t “true value” probably lies.
 95% CI - if we repeated the experiment with similar
populations an infinite number of times, the results
would fall within the CI 95% of the time. 95% certain
that the “true value” will fall within the 95% CI range
.
Confidence Interval (CI)
 For Odd ratio and Relative Risk, if 95
% CI contains 1 means there is no sig
nificant difference.
 For risk difference and mean differen

ce if 95% CI contains 0 means there is

no significant difference.
Example: RR

N Eng J Med 2006;354:1706-1717

 Odd ratio
Disease No disease
Exposed A B A+B
(treatment)
Non C D C+D
exposed
(control)
A+C B+D A+B+C+D

AD
OR 
BC
1 1 1 1
var(ln OR)    
A B C D
95%CIforOR  exp[ln OR  1.96SE (ln OR)]
Example: OR

(28*151)/(176*51)
 Fixed effects model-the inverse varian
ce-weighted method

wiTi ( wi Ln(ORi ))
PoolT  T  Ln(OR) 
wi wi

1
wi 
vari

1 1
var(T )  var( Ln(OR)) 
wi wi
Number of study = i
 Example: Combining OR using the
inverse variance-weighted method

Hint: Calculated from Ln(OR)!

Example: Combining OR using the inv
erse variance-weighted method