The Cochrane Collaboration: Medicine's Best-Kept Secret

The Cochrane Collaboration

Medicine’s Best-Kept Secret

By Alan Cassels

Foreword by Sir Iain Chalmers

Illustrations by Jeremy Gordaneer

Copyright © 2015 Alan Cassels

Agio Publishing House

Smashwords Edition

Agio Publishing House, 151 Howe Street, Victoria BC Canada V8V 4K5

© 2015, Alan Cassels. All rights reserved.

Without limiting the rights under copyright reserved above, no part of this publication may be reproduced, stored in or introduced into a retrieval system, or transmitted, in any form or by any means (electronic, mechanical, photocopying, recording or otherwise), without the prior written permission of both the copyright owner and the publisher of this book.

The Cochrane Collaboration: Medicine’s Best-Kept Secret

978-1-927755-30-3 (trade paperback)

978-1-927755-31-0 (kindle/mobi ebook)

978-1-927755-32-7 (epub ebook)

Cataloguing information available from Library and Archives Canada.

Agio Publishing House is a socially-responsible enterprise, measuring success on a triple-bottom-line basis.

To Jini: The first ever awardee of the Chris Silagy Prize

Table of Contents

Foreword by Sir Iain Chalmers

Preface

One: An aha Moment

Two: Iain Chalmers, Archie Cochrane, and the Jibe That Changed Everything

Three: An Unusual Library: For Knowledge-based Health Care

Four: Getting Airborne While Building the Airplane

Five: The High Human Cost of Not Acting on Quality Research

Six: Connecting with Consumers

seven: A Shining Light: Drugs and Money

eight: Cochrane and Disaster Response: Evidence Aid

Nine: Gold-Standard Evidence: Why National Healthcare Systems Need Cochrane

ten: Global Perspectives: The Need for Information

ELEVEN: It’s a Start

Epilogue

Acknowledgements

Endnotes

About the Author

Foreword by Sir Iain Chalmers

In 1972, when Archie Cochrane published his seminal work, Effectiveness and Efficiency: Random Reflections on Health Services,¹ he asked a vital question: how can we have rational health services if we don’t know which of the things being done are useful and which are useless or possibly even harmful?

Archie went on to indicate where one might start to find the answers. He said that not all professional opinions are equal: some are informed by mediocre or no research; others are supported by more rigorous research, such as unbiased treatment comparisons made within randomized trials.

Being introduced to randomized trials in Archie’s book was, for me, like being given a compass to negotiate the jungle of incompatible clinical opinions that I faced as a junior doctor. I started looking out for reports of randomized trials in my chosen field, obstetrics. The bibliographic databases at that time hadn’t indexed these studies satisfactorily, which meant searching journals for them ‘by hand’. A scheme was outlined for evaluating the studies found. Over the next ten years we extended our searches to the point at which we felt able to begin analyzing similar studies together, sometimes combining their results to reduce the likelihood of being misled by the play of chance.

In this book by bestselling author and health policy researcher Alan Cassels, you’ll learn that Archie Cochrane judged my specialty – obstetrics – the most unscientific in medicine! That jibe helped to provide the impetus to tackle this situation, and about a hundred of us from around the world collaborated to show what could be done. Archie Cochrane himself recognized our progress the year before he died. He referred to our collection of systematic reviews of controlled trials of care during pregnancy and childbirth a real milestone in the history of randomized trials and in the evaluation of care, and expressed his hope that it would be widely copied by other specialties.

This presented a new challenge. Why shouldn’t the approach we had used successfully for maternity care be extended across all of health care? The key to establishing the international collaboration needed to take up Archie’s new challenge was to involve people who were generous-spirited, who respected other people’s views, and who were team builders.

In this book you will read about some of these people, and others who have contributed to the collaboration over the past 20 years. Cochrane’s name was built into the initiative when a Cochrane Centre was established in Oxford in 1992, and the Centre convened a meeting the following year at which the international Cochrane Collaboration was founded. The Collaboration now has well over 30,000 contributors in over a hundred countries, who prepare, maintain and disseminate thousands of systematic reviews of research evidence relevant to improving health care. One result of reviewing such a mass of research is that it has laid bare the poor quality and irrelevance to patients and clinicians of much medical research.

I left the Cochrane Collaboration ten years after helping to establish it, and I have deliberately not tried to keep myself informed about its development. However, I was asked to give the 1st Archie Cochrane Lecture at the Cochrane Colloquium in Quebec City in 2013, so I had to do some informal research to help me prepare my talk. I began the talk by reminding my audience of the principal motivation for many if not most of those who contribute to the Collaboration: when decisions in health care are not informed by up-to-date, systematic reviews of research, patients suffer and sometimes die.

Keeping reviews up to date is a particular challenge for the Collaboration because – uniquely – its publicly stated mission is to prepare and maintain systematic reviews of treatments. This is also the Collaboration’s ‘Achilles heel’. Cochrane reviews have earned a deserved reputation for being of generally high quality, but some of them don’t live up to the resulting expectations, and they often take an age to move through the preparation, publication and updating processes. I acknowledged that the Collaboration was very aware of these criticisms and was taking them seriously.

I made clear in my talk that I appreciated that the Cochrane Collaboration had been one of the principal contributors to the global explosion in systematic reviews over the previous 20 years; had promulgated the ethical and scientific principles of systematic reviews; trained tens of thousands of people in systematic review methods; established novel editorial systems for assuring quality in preparing reviews; created and made publicly available the best bibliographic source of reports of controlled trials in the world, as well as the most widely used software for preparing systematic reviews; and had established a network of methods groups and a database of methodological research.

This was all well and good, but given the burgeoning production of systematic reviews by people outside the Cochrane Collaboration, and the development of sources of information other than Cochrane reviews, was there anything special about the Collaboration? Indeed, were others providing the same or a better service to patients and health professionals more efficiently?

I have no sentimentality about the Cochrane Collaboration. But if the organization didn’t exist, something like it would need to be invented to serve the information needs of patients and health professionals. If that could be achieved through means other than the Cochrane Collaboration, I wouldn’t shed any tears at the demise of the organization. However, the Collaboration does remain unique in some important respects. It registers the titles of its planned reviews and publishes the protocols of reviews being prepared. It has developed the editorial and technical infrastructure to assure quality and to update and correct its reviews. And it has established exceptional international coverage.

I suggested in my talk that one of the Collaboration’s most precious features has been its tradition of self-criticism. One of the things I remember with pleasure about the founding Colloquium in Oxford in 1993 was that there weren’t any bullshitters there: bullshitters are uncomfortable in the Cochrane Collaboration because, however distinguished they may be, they get challenged, sometimes by very ‘junior’ people, if they manifest the arrogance that is not uncommon among senior doctors. There is a strong tradition of challenging authority within the Cochrane Collaboration, and there’s even an annual prize for the best (evidence-based) criticism of the Collaboration – whether self-criticism or criticism from outside.

After interviewing me for nearly two hours on the origins of the Cochrane Collaboration, Alan Cassels ended by asking me what I am most proud of. I said that, Pride is not something which I find comes particularly easily; but if you had asked me instead what I am most pleased about, I would say that it is the self-criticism and generosity of spirit among the people contributing to the Cochrane Collaboration. These are the organization’s most important qualities.

In the pages of this book you will learn about some of the generous-spirited people who continue to do their best, despite the frustrations, to ensure that the Cochrane Collaboration serves the people with timely, trustworthy research information relevant to choices in health care.

Preface

If you read this preface as part of a book printed on paper, then consider it a minor miracle. This book almost never got published.

For more than a decade, I had been an outside observer of the Cochrane Collaboration, an occasional lurker at its annual Colloquium. This often required a trek to some exotic locale such as Keystone, Colorado or Melbourne, Australia to meet the people responsible for the best there is in healthcare research. My lurking at those meetings must have been noticed. The Cochrane leadership approached me in 2011 and asked me to consider writing a book about the organization.

The first sentiment that came to my mind was the glib expression of Groucho Marx: Did I really want to belong to any club that would have me as a member? I had meetings and teleconferences with the organizers, and they explained that a book could showcase the social history of the organization and help to mark its 20th-anniversary celebrations taking place in Quebec City, Canada, in September, 2013.

I felt a little reluctant – I’m not the showcasing type. I’m a hopeless failure at PR and typically don’t do promotional writing, because often there is little worth showcasing. It didn’t help that social history sounded a little pompous, and I was pretty sure I didn’t have a clue what it was or why anyone would actually care. In hindsight, the term social history was a misnomer: to academics it means something very specific, and this was not the book I wrote.

I felt pretty confident that maybe the world didn’t need a book about the Cochrane Collaboration. There were already so many stories – legends really – about many of the characters who struggled to create a movement known as evidence-based health care. Many of those characters – including quirky activists and punctilious academics, some who were the early founders of the Cochrane Collaboration – inhabited stories that almost reach the status of folk tales in the world of medical research.

Then there was the irony of creating something in print – a book, on paper, with a cover – which was not lost on me. The Cochrane Collaboration has always been a virtual organization. Its enormous, influential output has always been electronic. Cochrane (which is what the Collaboration is now called after the latest round of rebranding) began at roughly the same time as computers were becoming common in homes and offices, and we began communicating through them.

Paper has always been superfluous while the thousands of systematic reviews of healthcare interventions produced by the Cochrane Collaboration – what some would call the very bedrock of what we can say we really know in health care – are in computers, accessible anywhere to anyone with internet access. Those reviews, plus thousands more pages describing the methods and the careful documentation that support the work of the Collaboration, existed in cyberspace long before anyone ever used the term in the cloud.

After much thought, I agreed to write the book because the leadership gave me some measure of solace that I, as a writer, wouldn’t starve in the process. I did actually find the beginnings of the Collaboration to be a very compelling, critically important story worth telling. But most of all, I did want to showcase this organization, which, apart from healthcare professionals and researchers, almost no one on the planet has ever really heard of.

I’m naturally attracted to those who fight against corruption and misleading medical research and the books I have written tend to highlight these struggles. The Collaboration is a big and diverse – some might even say overly bureaucratic these days – organization but within it there are people who have done and continue to do remarkable things, and display incredible courage and conviction in fighting off the adverse influences of money and power on health research. They have often won huge victories that influence everyone on the planet who seeks health care. Not bad for an organization that is largely unheard of.

Quite simply, I figured that this ‘secret’ organization was making the world a better place, and it would help if more people knew about it. Hey, maybe this was a club I did want to belong to! If these accomplishments needed a megaphone, then maybe I was the one who could give it to them.

Every day around the globe, people grapple with illness, not knowing if a recommended drug, test, or medical procedure will ultimately improve their lives, make no difference, or hasten their demise. Among the sheer variety of medical questions we will ask at one time in our lives, there will always be, for all of us, the hardest question of all: Is there anywhere I can go for reliable, trustworthy information about how to treat my condition?

If you have never given a second thought to medical research and have never tried to second-guess your doctor’s orders, then this book might rattle your worldview. It is not only about one of medicine’s best-kept secrets, but also about the organization that exposes secrets – especially those related to what we know and what we don’t know about health care.

Today, more than 20 years after its creation, many doctors, nurses, researchers, and public health officials who make big decisions affecting the health care we receive consult the Cochrane Collaboration. It is a network of more than 30,000 people around the world, mostly volunteers, who collect and summarize research on healthcare interventions. For most of the world, Cochrane works in the shadows, its members working largely in obscurity, trying to accumulate knowledge to help people decide if the health care offered will truly help – and not harm – them.

You may rightly ask, Don’t medical people always offer what’s going to help and not harm their patients?

No. They don’t.

In fact, what they currently don’t know is alarming.

Think of healthcare interventions as a pie sliced into thirds. The first third contains all those healthcare interventions – pharmaceuticals, surgeries and tests, for example —underpinned by reliable evidence. They have been fairly studied and we can say, with more or less certainty, that they work. Call that piece of pie evidence-based.

The next third of pie comprises all those interventions that are either insufficiently studied or where proof is unknown; that we think are likely to help but for which there is not yet definitive proof that the benefits exceed the harms.

The final third contains all interventions for which the available evidence shows that the drug or procedure is likely harmful. And. We. Should. Stop. Doing. It.

Feelings of confusion may turn to suspicion as you wonder, But how do I know which piece of pie is being served up by my doctor? If you find this thought uncomfortable, that’s a good sign. You might have just had an aha moment similar to the one that struck Scottish physician Archie Cochrane.

He wondered, in a very public way, why his profession wasn’t systematically looking for reliable proof behind what he and his fellow doctors were offering their patients. He saw this as an unconscionable state of affairs, so he threw down a gauntlet at doctors, shaming them into action.

That gauntlet was picked up by Dr. (now Sir) Iain Chalmers, a young British obstetrician whose incredible zeal for the truth concerning medical approaches to pregnancy and childbirth fuelled the beginnings of this organization. He named it after his mentor and challenger.

That aha moment – as I show in the first chapter – had the power to change a person’s life. It changed my life, too, though not in a way I’d expected.

Over the two years I spent researching and writing this book, I traveled to Madrid, Oxford, Paris and Quebec City. I carried out more than 160 interviews with people both inside and outside the Cochrane Collaboration. Those interviews were video recorded and made part of the 20-year Cochrane anniversary celebrations, and many are now viewable on the Cochrane website www.cochrane.org. I talked to those critical of the Collaboration, those who lead it, and many who are part of the 30,000-or-so volunteers who work to make health care more evidence-based, more rational, and more lifesaving.

In January of 2013, shortly after I handed in a 100,000-word manuscript to Wiley, the Cochrane Library’s London-based publisher who was lined up to publish this book, Wiley’s executives got cold feet and withdrew from the project. Apparently they too were thinking like Groucho, not wanting to be part of a club that would have Alan Cassels as a member. I understood this in hindsight: big companies that worry about their brand don’t want anyone tinkering with it.

There are more layers of politics to this organization than one can ever contemplate. Many of the bigwigs in the Collaboration are skilled researchers and clinicians, but also skilled politicians, steeped in the myriad power struggles that make up modern health care. In an organization where every snort or wheeze about medical evidence holds huge consequences for the gargantuan business of health care out in the