Medicine shouldn’t be about authority, and the most important question anyone can ask on any claim is simple: ‘how do you know?’ This book is about the answer to that question.

There has been a huge shift in the way that people who work in medicine relate to patients. In the distant past, ‘communications skills training’, such as it was, consisted of how not to tell your patient they were dying of cancer. Today we teach students – and this is a direct quote from the hand-outs – how to ‘work collaboratively with the patient towards an optimum health outcome’. Today, if they wish, at medicine’s best, patients are involved in discussing and choosing their own treatments.

For this to happen, it’s vital that everyone understands how we know if a treatment works, how we know if it has harms, and how we weigh benefits against harms to determine the risk. Sadly doctors can fall short on this, as much as anybody else. Even more sadly, there is a vast army out there, queuing up to mislead us.

First and foremost in this gallery of rogues, we can mislead ourselves. Most diseases have a natural history, getting better and worse in cycles, or at random: because of this, anything you do, if you act when symptoms are at their worst, might make a treatment seem to be effective, because you were going to get better anyway.

The placebo effect, similarly, can mislead us all: people really can get better, in some cases, simply from taking a dummy pill with no active ingredients, and by believing their treatments to be effective. As Robert M Pirsig said, in Zen and the Art of Motorcycle Maintenance: ‘the real purpose of the scientific method is to make sure nature hasn’t misled you into thinking you know something you actually don’t know’.

But then there are the people who brandish scientific studies. If there is one key message from this book – and it is a phrase I have borrowed and used endlessly myself – it is the concept of a ‘fair test’. Not all trials are born the same, because there are so many ways that a piece of scientific research can be biased, and erroneously give what someone, somewhere thinks should be the ‘right’ answer.

Sometimes evidence can be distorted through absent-mindedness, or the purest of motives (for all that motive should matter). Doctors, patients, professors, nurses, occupational therapists, and managers can all become wedded to the idea that one true treatment, in which they have invested so much personal energy, is golden.

Sometimes evidence can be distorted for other reasons. It would be wrong to fall into shallow conspiracy theories about the pharmaceutical industry: they have brought huge, lifesaving advances. But there is a lot of money at stake in some research, and for reasons you will see in this book, 90% of trials are conducted by industry. This can be a problem, when studies funded by industry are four times more likely to have a positive result for the sponsor’s drug than independently funded trials. It costs up to $800m to bring a new drug to market: most of that is spent before the drug comes to market, and if the drug turns out to be no good, the money is already spent. Where the stakes are so high, sometimes the ideals of a fair test can fail.¹

Equally, the way that evidence is communicated can be distorted, and misleading. Sometimes this can be in the presentation of facts and figures, telling only part of the story, glossing over flaws, and ‘cherry picking’ the scientific evidence which shows one treatment in a particular light.

But in popular culture, there can be more interesting processes at play. We have an understandable desire for miracle cures, even though research is frequently about modest improvements, shavings of risk, and close judgement calls. In the media, all too often this can be thrown aside in a barrage of words like ‘cure’, ‘miracle’, ‘hope’, ‘breakthrough’, and ‘victim’.²

At a time when so many are so keen to take control of their own lives, and be involved in decisions about their own healthcare, it is sad to see so much distorted information, as it can only disempower. Sometimes these distortions are around a specific drug: the presentation in the UK media of Herceptin as a miracle cure for breast cancer is perhaps the most compelling recent example.³

Sometimes, though, in promoting their own treatments, and challenging the evidence against them, zealots and their friends in the media can do even greater damage, by actively undermining the public’s very understanding of how we know if something is good for us, or bad for us.

Homoeopathy sugar pills perform no better than dummy sugar pills when compared by the most fair tests. But when confronted with this evidence, homoeopaths argue that there is something wrong with the whole notion of doing a trial, that there is some complicated reason why their pills, uniquely among pills, cannot be tested. Politicians, when confronted with evidence that their favoured teaching programme for preventing teenage pregnancy has failed, may fall into the same kind of special pleading. In reality, as this book will show, any claim made about an intervention having an effect can be subjected to a transparent fair test.⁴

Sometimes these distortions can go even deeper into undermining the public’s understanding. A recent ‘systematic review’ of all the most fair and unbiased tests showed there was no evidence that taking antioxidant vitamin pills can prolong life (in fact, they may even shorten it). With this kind of summary – as explained beautifully in this book – clear rules are followed, describing where to look for evidence, what evidence can be included, and how its quality should be assessed. But when systematic reviews produce a result that challenges the claims of antioxidant supplement pill companies, newspapers and magazines are filled with false criticisms, arguing that individual studies for the systematic review have been selectively ‘cherry picked’, for reasons of political allegiance or frank corruption, that favourable evidence has been deliberately ignored, and so on.⁵

This is unfortunate. The notion of systematic review – looking at the totality of evidence – is quietly one of the most important innovations in medicine over the past 30 years. In defending their small corner of retail business, by undermining the public’s access to these ideas, journalists and pill companies can do us all a great disservice.

And that is the rub. There are many reasons to read this book. At the simplest level, it will help you make your own decisions about your own health in a much more informed way. If you work in medicine, the chapters that follow will probably stand head and shoulders above any teaching you had in evidence-based medicine. At the population level, if more people understand how to make fair comparisons, and see whether one intervention is better than another, then as the authors argue, instead of sometimes fearing research, the public might actively campaign to be more involved in reducing uncertainties about the treatments that matter to them.

But there is one final reason to read this book, to learn the tricks of our trade, and that reason has nothing to do with practicality: the plain fact is, this stuff is interesting, and beautiful, and clever. And in this book it’s explained better than anywhere else I’ve ever seen, because of the experience, knowledge, and empathy of the people who wrote it.

Testing Treatments brings a human focus to real world questions. Medicine is about human suffering, and death, but also human frailty in decision makers and researchers: and this is captured here, in the personal stories and doubts of researchers, their motivations, concerns, and their shifts of opinion. It’s rare for this side of science to be made accessible to the public, and the authors move freely, from serious academic papers to the more ephemeral corners of medical literature, finding unguarded pearls from the discussion threads beneath academic papers, commentaries, autobiographies, and casual asides.

This book should be in every school, and every medical waiting room. Until then, it’s in your hands. Read on.

Ben Goldacre

August 2011