The Zocor ad introduced in chapter 4 mentions two benefits for people who have high cholesterol and heart disease: the drug lowers cholesterol levels, and it reduces the chance of death from a heart attack. The drug has other benefits as well: it reduces the chance of a nonfatal heart attack, and it lowers the chance of developing a weakened heart—a condition known as heart failure—which makes sense, since fewer heart attacks mean less weakening.
All the things Zocor affects—cholesterol levels, the chance of fatal and nonfatal heart attacks, and heart failure—are called outcomes. As you’ve learned, the benefit of drugs or other interventions is measured in terms of how they affect outcomes. In chapter 4, we listed some questions that you should ask whenever you hear a message about how an action can reduce your risk. The first question to ask is this: “Reduced risk of what?” The answer is the outcome.
Only when you understand the outcome under consideration does it make sense to ask the second question: “How big is the risk reduction?” Without a clear picture of the seriousness or importance of the outcome, you can’t decide if the risk reduction is big enough to make the action worthwhile. In fact, if you don’t care about the outcome, there’s no reason to even bother learning the size of the benefit.
So how do you decide if you care about the outcome? The following illustration organizes outcomes according to their direct impact on you. We call it the pyramid of benefit. At the bottom are things that don’t have a direct impact on you—for example, laboratory measurements, blood tests, and X-rays. You don’t directly experience or feel these outcomes; for instance, you don’t feel any different if your cholesterol level is really low or really high. On the pyramid, these kinds of outcomes are called surrogate outcomes.
As you move up the pyramid, you encounter patient outcomes, where the impact of the outcomes becomes increasingly direct: you clearly feel symptoms such as pain and nausea, for example, and you directly experience events such as being hospitalized, requiring nursing home care, or needing an operation. And since it is hard to think of anything that has a more direct impact on you, death appears at the top of the pyramid. Because the pyramid may be useful in helping you decide which outcomes you care about, the following sections examine it in some detail.
Better Test Results
Many medical interventions change surrogate outcomes, such as laboratory measurements of cholesterol levels or X-ray results that measure bone density. Since you cannot directly feel these surrogate outcomes, why should you care about them? The reason is that these outcomes often “stand in” for, or represent, patient outcomes higher on the pyramid. A surrogate is something that stands in for something else.
Unfortunately, surrogate outcomes often do a poor job of “standing in” for patient outcomes. Keep this in mind! Even if a drug improves a surrogate outcome, it doesn’t mean that the drug will improve a corresponding patient outcome. Judging benefit based solely on a surrogate outcome requires a big leap of faith.
As an example, consider a drug that has been demonstrated to lower cholesterol levels (a surrogate outcome). A host of medical studies have shown that lower cholesterol levels are linked to fewer heart attacks (a patient outcome). Does this mean that the drug will reduce the number of heart attacks? It may—but it may not. Improving a surrogate outcome does not automatically mean improving a patient outcome. (See the Learn More box on page 58.)
Consider a health message, adapted from a real ad for an osteoporosis drug, claiming that “drug X improves bone density by 35 percent.” (Osteoporosis means thinning bones.) This improvement may sound pretty encouraging. But let’s take a closer look. To start, ask yourself, “What is the outcome?” The answer is greater bone mineral density. Unless you know what bone density means, the 35 percent statistic won’t matter to you. Bone density is a way of gauging how strong bones are. But you don’t feel bone density—it’s a surrogate outcome. The only reason we care about bone density is because weak, thin bones are more apt to fracture. And bone fractures—something people directly experience—are important: they hurt, and they can leave people disabled. For the elderly, hip fractures in particular are often the first step toward institutionalization or even death.
But what if drug X improved your bone density (measured by a T-score) but did not lower your chance of a hip fracture, the most disabling kind of bone fracture? This is not so far-fetched. Lots of things other than bone density affect your chance of fracturing a bone, including factors that increase your chance of falling, such as poor vision, poor balance, or even how your house is set up (lots of clutter, throw rugs). And increased bone density is not the only ingredient of bone strength—for example, bone architecture (the underlying structure of the bone) may matter even more. As it turns out, although many drugs improve bone density, few have been shown to reduce hip fractures.
If drug X works only on the surrogate outcome (bone density), but this benefit does not translate into the corresponding patient outcome (fewer hip fractures), you would be wise to be hesitant about exposing yourself to the cost, inconvenience, and potential side effects of the drug. This would be true no matter how big an effect drug X has on the surrogate outcome.
You cannot reliably assume that improving a surrogate outcome will translate into better health. Nevertheless, you will hear about surrogate outcomes all the time (especially in drug ads), because they are easier and faster to measure than patient outcomes: a drug may lower cholesterol levels right away, but it will take years to see whether the drug really results in fewer heart attacks. If you hear about a medical intervention that improves a surrogate outcome, ask this question: “Has the intervention also been shown to have a beneficial effect on what people experience or feel?”
The correct answer is b. Tumor shrinkage is a surrogate outcome. It may translate into longer, better lives—but it may not. Unfortunately, we have many examples of cancer drugs that shrink tumors but do not extend life (some even shorten it). Surrogate outcomes matter only to the extent that they improve the corresponding patient outcome (the outcome they stand in for). So knowing how much the drug shrinks the tumors isn’t really important—unless the drug makes people feel better or live longer.
Now you know why we consider surrogate outcomes to be less important than patient outcomes.
Fewer Symptoms of Disease
The next step on the pyramid addresses the benefit of fewer symptoms of disease. Symptoms are the disease-related sensations people feel. The most familiar symptom is pain: headache, back pain, joint pain, chest pain. Many other common symptoms can be very unpleasant though not painful: upset stomach, shortness of breath, lightheadedness, and runny nose, for instance.
Most people do not need to think long and hard about the importance of interventions that make them feel better when they are sick. That’s why we believe that a reduction of symptoms is a very important outcome. For most people, symptom relief matters so much that they are eager to get to the next questions and ask for the numbers: “How big is the benefit? That is, how likely is it that I will feel better if I take this action? And how much better will I feel?”
In fact, some people wouldn’t even bother asking about the numbers. People with symptoms can often see for themselves whether the intervention works, by trying it. For example, if your headache doesn’t improve after taking a headache medicine, you may decide that the pills don’t work. Alternatively, if the headache goes away completely in twenty minutes, you’ll probably decide that the drug is a winner.
The correct answer is b. Symptom improvement is not a foolproof test of benefit. Some people feel better just because they do something. You may have heard of the placebo effect: people sometimes experience a benefit even when they take an ineffective sugar pill or when they receive a faked surgery. And some symptoms, by their very nature, wax and wane spontaneously. People with back pain know this quite well: on some days, their back feels great; on other days, it feels awful. These two factors—the placebo effect and spontaneous improvement—can lead people to judge an intervention as being beneficial when in fact it is not. For these reasons, the most trustworthy test of an intervention for current symptoms is a randomized trial—a true experiment, in which people are randomly given either the treatment or a placebo and then undergo a standardized symptom assessment. If the treatment works, the people who were randomly chosen to receive it will, on average, do better than those randomly chosen to receive the placebo.
Deciding whether a treatment works by trying it is possible, of course, only when you have symptoms—this strategy doesn’t work with interventions that are designed to reduce your risk of some future event. If you take Zocor, for example, you won’t feel your “reduced risk” of a future fatal heart attack. The only way to gauge the benefit of interventions that reduce a future risk is to know the numbers: your starting and modified risks, as outlined in chapter 4.
Fewer Deaths from Disease
Because reducing the number of deaths from disease is a very important outcome, it appears at the top of the pyramid. There isn’t a lot of ambiguity here—death is really important to most people. So interventions that reduce the number of deaths almost always matter (a lot) to people. But you should still go ahead and find out the numbers. You might be surprised: you might not care so much about an intervention that reduces a very rare cause of death, or even one that reduces a common cause of death by a very, very small amount.
We should also acknowledge that, in some cases, reducing the risk of death is not always the most desirable effect. For someone with terminal cancer or end-stage Alzheimer’s, minimizing suffering—symptoms—may be more important than prolonging life.
In this chapter, we showed you how the pyramid of benefit can help you decide how much to care. Of course, the less you care about the outcome, the less important it is to bother thinking about the size of the benefit.
Pay attention to the outcome that is addressed in any health message—it may not be the one you care about most. When you hear about how a drug affects a blood test result, such as cholesterol or bone density scores, remember to keep your eyes on the prize: cholesterol, other lab tests, X-rays, and so on may matter. But they may not. Do people who take the drug live longer? Do they feel better? Or does the drug just make their test results look better? What really matters to you is how these things translate into how you feel, how they affect your chance of living the way you want to live, and whether they affect your chance of dying.
Publication Details
Copyright
Know Your Chances: Understanding Health Statistics is hereby licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported license, which permits copying, distribution, and transmission of the work, provided the original work is properly cited, not used for commercial purposes, nor is altered or transformed.
Publisher
University of California Press, Berkeley (CA)
NLM Citation
Woloshin S, Schwartz LM, Welch HG. Know Your Chances: Understanding Health Statistics. Berkeley (CA): University of California Press; 2008. Chapter 5, Not All Benefits Are Equal: Understand the Outcome.