‘There is no way to know when our observations about complex events in nature are complete. Our knowledge is finite, Karl Popper emphasised, but our ignorance is infinite. In medicine, we can never be certain about the consequences of our interventions, we can only narrow the area of uncertainty. This admission is not as pessimistic as it sounds: claims that resist repeated energetic challenges often turn out to be quite reliable. Such “working truths” are the building blocks for the reasonably solid structures that support our everyday actions at the bedside.’

William A. Silverman. Where’s the evidence?, Oxford: Oxford University Press, 1998, p165

Modern medicine has been hugely successful. It is hard to imagine what life must have been like without antibiotics. The development of other effective drugs has revolutionized the treatment of heart attacks and high blood pressure and has transformed the lives of many people with schizophrenia. Childhood immunization has made polio and diphtheria distant memories in most countries, and artificial joints have helped countless people to be less troubled by pain and disability. Modern imaging techniques such as ultrasound, computed tomography (CT), and magnetic resonance imaging (MRI) have helped to ensure that patients are accurately diagnosed and receive the right treatment. The diagnosis of many types of cancer used to spell a death sentence, whereas today patients regularly live with their cancers instead of dying from them. And HIV/AIDS has largely changed from a swift killer into a chronic (long-lasting) disease.

Of course many improvements in health have come about because of social and public health advances, such as piped clean water, sanitation, and better housing. But even sceptics would have difficulty dismissing the impressive impact of modern medical care. Over the past half century or so, better healthcare has made a major contribution to increased lifespan, and has improved the quality of life, especially for those with chronic conditions.^{1, 2}

But the triumphs of modern medicine can easily lead us to overlook many of its ongoing problems. Even today, too much medical decision-making is based on poor evidence. There are still too many medical treatments that harm patients, some that are of little or no proven benefit, and others that are worthwhile but are not used enough. How can this be, when every year, studies into the effects of treatments generate a mountain of results? Sadly, the evidence is often unreliable and, moreover, much of the research that is done does not address the questions that patients need answered.

Part of the problem is that treatment effects are very seldom overwhelmingly obvious or dramatic. Instead, there will usually be uncertainties about how well new treatments work, or indeed whether they do more good than harm. So carefully designed fair tests – tests that set out to reduce biases and take into account the play of chance (see Chapter 6) – are necessary to identify treatment effects reliably.

The impossibility of predicting exactly what will happen when a person gets a disease or receives a treatment is sometimes called Franklin’s law, after the 18th-century US statesman Benjamin Franklin, who famously noted that ‘in this world nothing can be said to be certain, except death and taxes’.³ Yet Franklin’s law is hardly second nature in society. The inevitability of uncertainty is not emphasized enough in schools, nor are other fundamental concepts such as how to obtain and interpret evidence, or how to understand information about probabilities and risks. As one commentator put it: ‘At school you were taught about chemicals in test tubes, equations to describe motion, and maybe something on photosynthesis. But in all likelihood you were taught nothing about death, risk, statistics, and the science that will kill or cure you.’⁴ And whereas the practice of medicine based on sound scientific evidence has saved countless lives, you would be hard pressed to find a single exhibit explaining the key principles of scientific investigation in any science museum.

And concepts of uncertainty and risk really do matter. Take, for example, the logical impossibility of ‘proving a negative’ – that is, showing that something does not exist, or that a treatment has no effect. This is not just a philosophical argument; it has important practical consequences too, as illustrated by experience with a combination pill called Bendectin (active ingredients doxylamine, and pyridoxine or vitamin B6). Bendectin (also marketed as Debendox and Diclectin) used to be widely prescribed to women to relieve nausea in early pregnancy. Then came claims that Bendectin caused birth defects, which were soon taken up in an avalanche of law suits. Under pressure from all the court cases, the manufacturers of Bendectin withdrew the drug from sale in 1983. Several subsequent reviews of all the evidence provided no support for a link with birth defects – it was not possible to show conclusively that there was no harm, but there was no evidence that it did cause harm. Ironically, as a result of Bendectin being withdrawn, the only drugs available to treat morning sickness in pregnant women are those for which considerably less is known about their potential to cause birth defects.⁵

The most that research can usually do is to chip away at the uncertainties. Treatments can be harmful as well as helpful. Good, well-conducted research can indicate the probability (or likelihood) that a treatment for a health problem will lead to benefit or harm by comparing it with another treatment or no treatment at all. Since there are always uncertainties it helps if we try to avoid the temptation to see things in black and white. And thinking in terms of probabilities is empowering.⁶ People need to know the likelihood of a particular outcome of a condition – say stroke in someone with high blood pressure – the factors that affect the chance of a stroke happening, and the probability of a treatment changing the chances of a stroke happening. With enough reliable information, patients and health professionals can then work together to assess the balance between the benefits and harms of treatments. They can then choose the option that is likely to be most appropriate according to individual circumstances and patient preferences.⁷

Our aim in Testing Treatments is to improve communication and boost confidence, not to undermine patients’ trust in health professionals. But this can only happen when patients can help themselves and their health professionals critically assess treatment options.

In Chapter 1 we briefly describe why fair tests of treatments are necessary and how some new treatments have had harmful effects that were unexpected. In Chapter 2 we describe how the hoped-for effects of other treatments have failed to materialize, and highlight the fact that many commonly used treatments have not been adequately evaluated. Chapter 3 illustrates why more intensive treatment is not necessarily better. Chapter 4 explains why screening healthy people for early indications of disease may be harmful as well as helpful. In Chapter 5 we highlight some of the many uncertainties that pervade almost every aspect of healthcare and explain how to tackle them.

Chapters 6, 7, and 8 give some ‘technical details’ in a non-technical way. In Chapter 6 we outline the basis for fair testing of treatments, emphasizing the importance of ensuring that like is compared with like. Chapter 7 highlights why taking into account the play of chance is essential. Chapter 8 explains why it is so important to assess all the relevant reliable evidence systematically.

Chapter 9 outlines why systems for regulating research into the effects of treatments, through research ethics committees and other bodies, can put obstacles in the way of getting good research done, and explains why regulation may therefore fail to promote the interests of patients. Chapter 10 contrasts the key differences between good, bad, and unnecessary research into the effects of treatments; it points out how research is often distorted by commercial and academic priorities and fails to address issues that are likely to make a real difference to the well-being of patients.

Chapter 11 maps what patients and the public can do to ensure better testing of treatments. In Chapter 12 we look at ways in which robust evidence from research into treatments can really make for better healthcare for individual patients. And in Chapter 13 we present our blueprint for a better future, ending with an action plan.

Each chapter is referenced with a selection of key source material, and a separate Additional Resources section is included at the end of the book (see p184). For those who wish to explore issues in more detail, a good starting point is the James Lind Library at www.jameslindlibrary.org. You will find the free electronic version of the second edition of Testing Treatments at a new website – Testing Treatments Interactive (www.testingtreatments.org) – where translations and other material will be added over the coming years.

We authors are committed to the principle of equitable access to effective healthcare that is responsive to people’s needs. This social responsibility in turn depends on reliable and accessible information about the effects of tests and treatments derived from sound research. Because healthcare resources everywhere are limited, treatments must be based on robust evidence and used efficiently and fairly if the whole population is to stand a chance of benefiting from medical advances. It is irresponsible to waste precious resources on treatments that are of little benefit, or to throw away, without good reason, opportunities for evaluating treatments about which too little is known. Fair testing of treatments is therefore fundamentally important to enable equitable treatment choices for all of us.

We hope that you, the reader, will emerge from Testing Treatments sharing some of our passion for the subject and go on to ask awkward questions about treatments, identify gaps in medical knowledge, and get involved in research to find answers for the benefit of yourself and everybody else.