Introduction to Meta Analysis, 2nd Edition

Visit the link below to download the full version of this book:

https://medipdf.com/product/introduction-to-meta-analysis-2nd-edition/

Click Download Now

 Contents vii

12 RANDOM-EFFECTS MODEL 65
Introduction 65
The true effect sizes 65
Impact of sampling error 66
Perforrning a random-effects meta-analysis 68
Summary points 70

13 FIXED-EFFECT VERSUS RANDOM-EFFECTS MODELS 71

Introduction 71
Definition of a summary effect 71
Estimating the summary effect 72
Extreme effect size in a large study or a small study 73
Confidence interval 73
The null hypothesis 76
Which model should we use? 76
Model should not be based on the test for heterogeneity 78
Concluding remarks 79
Summary points 79

14 WORKED EXAMPLES (PART 1) 81

Introduction 81
Worked example for continuous data (Part 1) 81
Worked example for binary data (Part I) 85
Worked example for correlational data (Part 1) 90
Summary points 94

PART 4: HETEROGENEITY
15 OVERVIEW 97
Introduction 97
Nomenclature 98
Worked examples 98

16 IDENTIFYING AND QUANTIFYING HETEROGENEITY 99

Introduction 99
Isolating the variation in true effects 99
Computing Q 101
Estimating -r 2 106
The fl statistic 109
Comparing the measures of heterogeneity 111
Confidence intervals for -r 2 114
Confidence intervals (or uncertainty intervals) for fl 115
Summary points 116
viii Contents

17 PREDICTION INTERVALS 119

Introduction 119
Prediction intervals in primary studies 119
Prediction intervals in meta-analysis 121
Confidence intervals and prediction intervals 123
Comparing the confidence interval with the prediction interval 123
Summary points 125

18 WORKED EXAMPLES (PART 2) 127

Introduction 127
Worked example for continuous data (Part 2) 127
Worked example for binary data (Part 2) 131
Worked example for correlational data (Part 2) 134
Summary points 138

19 AN INTUITIVE LOOK AT HETEROGENEITY 139

Introduction 139
Motivating example 140
The Q-value and the p-value do not tell us how much the effect size varies 141
The confidence interval does not tell us how much the effect size varies 142
The fl statistic does not tell us how much the effect size varies 142
What fl tells us 142
The fl index vs. the prediction interval 145
The prediction interval 145
Prediction interval is clear, concise, and relevant 147
Computing the prediction interval 147
How to use fl 149
How to explain heterogeneity 149
How much does the effect size vary across studies? 150
Caveats 150
Conclusion 150
Further reading 151
Summary points 151
The meaning of fl in Figure 19.2 151

20 CLASSIFYING HETEROGENEITY AS LOW, MODERATE, OR HIGH 155

Introduction 155
Interest should generally focus on an index of absolute heterogeneity 155
The classifications lead themselves to mistakes of interpretation 158
Classifications focus attention in the wrong direction 158
Summary points 158
 Contents ix

PART 5: EXPLAINING HETEROGENEITY

21 SUBGROUP ANALYSES 161
Introduction 161
Fixed-effect model within subgroups 163
Computational models 172
Random effects with separate estimates of -r 2 174
Random effects with pooled estimate of -r 2 181
The proportion of variance explained 189
Mixed-effects model 192
Obtaining an overall effect in the presence of subgroups 193
Summary points 195

22 META-REGRESSION 197
Introduction 197
Fixed-effect model 198
Fixed or random effects for unexplained heterogeneity 203
Random-effects model 206
Summary points 212

23 NOTES ON SUBGROUP ANALYSES AND META-REGRESSION 213

Introduction 213
Computational model 213
Multiple comparisons 216
Software 216
Analyses of subgroups and regression analyses are observational 217
Statistical power for subgroup analyses and meta-regression 218
Summary points 219

PART 6: PUTTING IT ALL IN CONTEXT

24 LOOKING AT THE WHOLE PICTURE 223
Introduction 223
Methylphenidate for adults with ADHD 226
Impact of GLP-1 mimetics on blood pressure 228
Augmenting clozapine with a second antipsychotic 228
Conclusions 231
Caveats 231
Summary points 232

25 LIMITATIONS OF THE RANDOM-EFFECTS MODEL 233

Introduction 233
Assumptions of the random-effects model 234
X Contents

A textbook case 234

When studies are pulled from the literature 235
A useful fiction 237
Transparency 238
A narrowly defined universe 238
Two important caveats 239
In context 239
Extreme cases 240
Summary points 241

26 KNAPP-HARTUNG ADJUSTMENT 243

Introduction 243
Adjustment is rarely employed in simple analyses 243
Adjusting the standard error 244
The Knapp-Hartung adjustment for other effect size indices 246
t distribution vs. Z distribution 247
Limitations of the Knapp-Hartung adjustment 248
Summary points 249

PART 7: COMPLEX DATA STRUCTURES

27 OVERVIEW 253
28 INDEPENDENT SUBGROUPS WITHIN A STUDY 255
Introduction 255
Combining across subgroups 255
Comparing subgroups 260
Summary points 260

29 MULTIPLE OUTCOMES OR TIME-POINTS WITHIN A STUDY 263

Introduction 263
Combining across outcomes or time-points 264
Comparing outcomes or time-points within a study 270
Summary points 275

30 MULTIPLE COMPARISONS WITHIN A STUDY 277

Introduction 277
Combining across multiple comparisons within a study 277
Differences between treatments 278
Summary points 279

31 NOTES ON COMPLEX DATA STRUCTURES 281

Introduction 281
Summary effect 281
Differences in effect 282
 Contents xi

PART 8: OTHER ISSUES

32 OVERVIEW 287

33 VOTE COUNTING - A NEW NAME FOR AN OLD PROBLEM 289

Introduction 289
Why vote counting is wrong 290
Vote counting is a pervasive problem 291
Summary points 293

34 POWER ANALYSIS FOR META-ANALYSIS 295

Introduction 295
A conceptual approach 295
In context 299
When to use power analysis 300
Planning for precision rather than for power 301
Power analysis in primary studies 301
Power analysis for meta-analysis 304
Power analysis for a test of homogeneity 309
Summary points 312

35 PUBLICATION BIAS 313

Introduction 313
The problem of missing studies 314
Methods for addressing bias 316
Illustrative example 317
The model 317
Getting a sense of the data 318
Is there evidence of any bias? 320
How much of an impact might the bias have? 320
Summary of the findings for the illustrative example 324
Conflating bias with the small-study effect 325
Using logic to disentangle bias from small-study effects 326
These methods do not give us the 'correct' effect size 327
Some important caveats 327
Procedures do not apply to studies of prevalence 328
The model for publication bias is simplistic 328
Concluding remarks 329
Putting it all together 330
Summary points 330

PART 9: ISSUES RELATED TO EFFECT SIZE

36 OVERVIEW 335
xii Contents

37 EFFECT SIZES RATHER THAN p-VALUES 337

Introduction 337
Relationship between p-values and effect sizes 337
The distinction is important 339
The p-value is often misinterpreted 340
Narrative reviews vs. meta-analyses 341
Summary points 342

38 SIMPSON'S PARADOX 343

Introduction 343
Circumcision and risk of HIV infection 343
An example of the paradox 345
Summary points 348

39 GENERALITY OF THE BASIC INVERSE-VARIANCE METHOD 349

Introduction 349
Other effect sizes 350
Other methods for estimating effect sizes 353
Individual participant data meta-analyses 354
Bayesian approaches 355
Summary points 357

PART 10: FURTHER METHODS

40 OVERVIEW 361

41 META-ANALYSIS METHODS BASED ON DIRECTION AND p-VALUES 363

Introduction 363
Vote counting 363
The sign test 363
Combining p-values 364
Summary points 368

42 FURTHER METHODS FOR DICHOTOMOUS DATA 369

Introduction 369
Mantel-Haenszel method 369
One-step (Peto) formula for odds ratio 373
Summary points 376

43 PSYCHOMETRIC META-ANALYSIS 377

Introduction 377
The attenuating effects of artifacts 378
Meta-analysis methods 380
Example of psychometric meta-analysis 381
Comparison of artifact correction with meta-regression 384
 Contents xiii

Sources of information about artifact values 384

How heterogeneity is assessed 385
Reporting in psychometric meta-analysis 386
Concluding remarks 386
Summary points 387

PART 11: META-ANALYSIS IN CONTEXT

44 OVERVIEW 391

45 WHEN DOES IT MAKE SENSE TO PERFORM A META-ANALYSIS? 393

Introduction 393
Are the studies similar enough to combine? 394
Can I combine studies with different designs? 395
How many studies are enough to carry out a meta-analysis? 399
Summary points 400

46 REPORTING THE RESULTS OF A META-ANALYSIS 401

Introduction 401
The computational model 402
Forest plots 402
Sensitivity analysis 404
Summary points 405

47 CUMULATIVE META-ANALYSIS 407

Introduction 407
Why perform a cumulative meta-analysis? 409
Summary points 412

48 CRITICISMS OF META-ANALYSIS 413

Introduction 413
One number cannot summarize a research field 414
The file drawer problem invalidates meta-analysis 414
Mixing apples and oranges 415
Garbage in, garbage out 416
Important studies are ignored 417
Meta-analysis can disagree with randomized trials 417
Meta-analyses are performed poorly 420
ls a narrative review better? 420
Concluding remarks 422
Summary points 422
49 COMPREHENSIVE META-ANALYSIS SOFTWARE 425
Introduction 425
Features in CMA 426
xiv Contents

Teaching elements 427

Documentation 427
Availability 427
Acknowledgments 427
Motivating example 428
Data entry 428
Basic analysis 429
What is the average effect size? 430
How much does the effect size vary? 430
Plot showing distribution of effects 431
High-resolution plot 432
Subgroup analysis 433
Meta-regression 435
Publication bias 438
Explaining results 439

50 HOW TO EXPLAIN THE RESULTS OF AN ANALYSIS 443

Introduction 443
The overview 444
The mean effect size 444
Variation in effect size 444
Notations 444
Impact of resistance exercise on pain 445
Correlation between letter knowledge and word recognition 450
Statins for prevention of cardiovascular events 455
Bupropion for smoking cessation 460
Mortality following mitral-valve procedures in elderly patients 465

PART 12: RESOURCES

51 SOFTWARE FOR META-ANALYSIS 471
Comprehensive meta-analysis 471
Metafor 471
Stata 472
Revman 472

52 WEB SITES, SOCIETIES, JOURNALS, AND BOOKS 473

Web sites 473
Professional societies 476
Journals 476
Special issues dedicated to meta-analysis 477
Books on systematic review methods and meta-analysis 477

REFERENCES 479
INDEX 491
 List of Tables

Table 3.1 Roadmap of formulas in subsequent chapters 19

Table 5.1 Nomenclature for 2 x 2 table of outcome by treatment 34
Table 5.2 Fictional data for a 2 x 2 table 34
Table 8.1 Impact of sample size on variance 51
Table 8.2 Impact of study design on variance 52
Table 14.1 Dataset 1 - Part A (basic data) 82
Table 14.2 Dataset 1 - Part B (fixed-effect computations) 83
Table 14.3 Dataset I - Part C (random-effects computations) 85
Table 14.4 Dataset 2 - Part A (basic data) 86
Table 14.5 Dataset 2 - Part B (fixed-effect computations) 87
Table 14.6 Dataset 2 - Part C (random-effects computations) 89
Table 14.7 Dataset 3 - Part A (basic data) 90
Table 14.8 Dataset 3 - Part B (fixed-effect computations) 91
Table 14.9 Dataset 3 - Part C (random-effects computations) 93
Table 16.1 Factors affecting measures of dispersion 111
Table 18.1 Dataset I - Part D (intermediate computations) 128
Table 18.2 Dataset 1 - Part E (variance computations) 128
Table 18.3 Dataset 2 - Part D (intermediate computations) 131
Table 18.4 Dataset 2 - Part E (variance computations) 131
Table 18.5 Dataset 3 - Part D (intermediate computations) 135
Table 18.6 Dataset 3 - Part E (variance computations) 135
Table 19.1 Relationship between observed effects and true effects in
Figure 19.2, Panel A 152
Table 21.1 Fixed effect model - computations 164
Table 21.2 Fixed-effect model - summary statistics 167
Table 21.3 Fixed-effect model -ANOVA table 169
Table 21.4 Fixed-effect model - subgroups as studies 170
Table 21.5 Random-effects model (separate estimates
of -r 2 ) - computations 176
Table 21.6 Random-effects model (separate estimates of -r 2 ) - summary
statistics 177
Table 21.7 Random-effects model (separate estimates of t2 ) - ANOVA
table 180
Table 21.8 Random-effects model (separate estimates of -r 2 ) - subgroups
as studies 181
 xvi List of Tables

Table 21.9 Statistics for computing a pooled estimate of -r 2 183

Table 21.10 Random-effects model (pooled estimate of -r 2 ) - computations 183
Table 21.11 Random-effects model (pooled estimate of -r 2 ) - summary
statistics 185
Table 21.12 Random-effects model (pooled estimate of -r 2 )-ANOVA
table 187
Table 21.13 Random-effects model (pooled estimate of -r 2 ) - subgroups as
studies 188
Table 22.1 The BCG dataset 200
Table 22.2 Fixed-effect model - Regression results for BCG 200
Table 22.3 Fixed-effect model - ANOVA table for BCG regression 200
Table 22.4 Random-effects model - regression results for BCG 207
Table 22.5 Random-effects model - test of the model 207
Table 22.6 Random-effects model - comparison of model (latitude)
versus the null model 211
Table 26.1 Knapp-Hartung computations for ADHD analysis 244
Table 26.2 Original vs. Knapp-Hartung 246
Table 26.3 Impact of using r distribution on the confidence interval width 248
Table 28.1 Independent subgroups - five fictional studies 256
Table 28.2 Independent subgroups - summary effect 257
Table 28.3 Independent subgroups - synthetic effect for study I 257
Table 28.4 Independent subgroups - summary effect across studies 258
Table 29.1 Multiple outcomes - five fictional studies 264
Table 29.2 Creating a synthetic variable as the mean of two outcomes 265
Table 29.3 Multiple outcomes - summary effect 267
Table 29.4 Multiple outcomes - impact of correlation on variance of
summary effect 269
Table 29.5 Creating a synthetic variable as the difference between two
outcomes 271
Table 29.6 Multiple outcomes - difference between outcomes 272
Table 29.7 Multiple outcomes - Impact of correlation on the variance of
difference 274
Table 38.1 HIV as function of circumcision (by subgroup) 344
Table 38.2 HIV as function of circumcision - by study 345
Table 38.3 HIV as a function of circumcision - full population 346
Table 38.4 HIV as a function of circumcision - by risk group 346
Table 38.5 HIV as a function of circumcision/risk group - full population 347
Table 39.1 Simple example of a genetic association study 352
Table 41.l Streptok.inase data - calculations for meta-analyses of
p-values 367
Table42.1 Nomenclature for 2 X 2 table of events by treatment 370
Table 42.2 Mantel-Haenszel - odds ratio 371
 List of Tables xvii

Table 42.3 Mantel-Haenszel - variance of summary effect 372

Table 42.4 One-step - odds ratio and variance 375
Table 43.1 Fictional data for psychometric meta-analysis 382
Table 43.2 Observed (attenuated) correlations 382
Table 43.3 Unattenuated correlations 383
 List of Figures

Figure 1.1 High-dose versus standard-dose of statins (adapted from

Cannon et al., 2006) 4
Figure 2.1 Impact of streptokinase on mortality (adapted from
Lau et al., 1992) 10
Figure 4.1 Response ratios are analyzed in log units 30
Figure 5.1 Risk ratios are analyzed in log units 34
Figure 5.2 Odds ratios are analyzed in log units 36
Figure 6.1 Correlations are analyzed in Fisher's z units 40
Figure 7.1 Converting among effect sizes 44
Figure 8.1 Impact of sample size on variance 51
Figure 8.2 Impact of study design on variance 52
Figure 10.1 Symbols for true and observed effects 60
Figure 11.1 Fixed-effect model - true effects 62
Figure 11.2 Fixed-effect model - true effects and sampling error 62
Figure 11.3 Fixed-effect model - distribution of sampling error 63
Figure 12.1 Random-effects model - distribution of the true effects 66
Figure 12.2 Random-effects model - true effects 66
Figure 12.3 Random-effects model - true and observed effect in one
study 67
Figure 12.4 Random-effects model - between-study and within-study
variance 68
Figure 13.1 Fixed-effect model - forest plot showing relative weights 72
Figure 13.2 Random-effects model - forest plot showing relative weights 72
Figure 13.3 Very large studies under fixed-effect model 74
Figure 13.4 Very large studies under random-effects model 74
Figure 14.1 Forest plot of Dataset 1 - fixed-effect weights 84
Figure 14.2 Forest plot of Dataset 1 - random-effects weights 85
Figure 14.3 Forest plot of Dataset 2 - fixed-effect weights 88
Figure 14.4 Forest plot of Dataset 2 - random-effects weights 90
Figure 14.5 Forest plot of Dataset 3 - fixed-effect weights 92
Figure 14.6 Forest plot of Dataset 3 - random-effects weights 94
Figure 16.1 Dispersion across studies relative to error within studies 100
Figure 16.2 Q in relation to df as measure of dispersion 102
Figure 16.3 Flowchart showing how 7'2 and P. are derived from Q and df 104
Figure 16.4 Impact of Q and number of studies on the p-value 105
Figure 16.5 Impact of excess dispersion and absolute dispersion on T2 107
Figure 16.6 Impact of excess and absolute dispersion on T 108