A Special Case of Exaggerated Numbers: Survival Statistics

If there were a hall of fame for exaggeration, survival statistics would get a lifetime achievement award. Survival statistics are widely used. Unfortunately, they are also widely misused and easily misunderstood, even when you know the starting and modified risks. Because these statistics come up so often in cancer screening, we’d like to explain how they work.

Cancer screening means testing people who have no symptoms to look for hidden, early evidence of cancer. The assumption is that if you find cancer in very early stages, when a tumor is small, it is less likely to have spread to other parts of the body and is more likely to be curable.

There are many tests that can be used for screening, including mammography for breast cancer, colonoscopy for colon cancer, and PSA testing for prostate cancer. You should be clear that how a test gets used determines whether it is a screening test. When a woman who has no signs or symptoms of breast cancer goes for her annual mammogram, she is getting a screening test. But when a woman feels a lump in her breast and gets a mammogram, she is not getting a screening test—she is getting a diagnostic test in response to a symptom.

You’ll often hear survival statistics quoted to demonstrate the value of screening. The most common statistic used is a 5-year survival rate. There’s nothing special about a span of 5 years. Survival statistics can be calculated for any time period. The lung and prostate cancer examples we’ll discuss in the next pages use both 5- and 10-year survival statistics. For either time frame, the issues that you need to understand are identical.

Here’s a typical example of how survival statistics are misused. This example is an excerpt from a story published in the Los Angeles Times in 2006:¹

Study Calls for Routine CT Scans for Smokers

The use of advanced CT imaging to detect Lung tumors in their still-treatable early stages greatly increases survival rates, and smokers should be routinely screened just as women are for breast cancer, according to a report today in the New England Journal of Medicine.

Imaging yielded an estimated 10-year survival rate of more than 90%, researchers said. Currently, about 5% of the 174,000 lung cancer patients diagnosed each year survive for 10 year. . . .

“This is compelling evidence that you can use CT screening to find lung cancer . . . and when you find it early and take it out early, you can cure a high percentage of patients,” said Dr. Claudia I. Henschke of Cornell University’s Weill Medical College in New York City, who led the study.

These results sound almost miraculous: CT scans for smokers seem to change the 10-year survival rate for lung cancer patients from 5 percent to over 90 percent. Does this prove that CT screening works? The short answer is no. To understand why not, you have to consider exactly how a 10-year survival rate is calculated. It is a fraction that, in this case, tells you what proportion of a group of people diagnosed with lung cancer is still alive 10 years later. Imagine 1,000 people diagnosed with lung cancer 10 years ago. If 50 are alive today, the 10-year survival rate is 50/1,000, or 5 percent. If 900 are alive today, the 10-year survival rate is 900/1,000, or 90 percent. Yet even if CT screening raised the 10-year survival rate to 90 percent, it is entirely possible that none of these patients will get an extra day of life.

The best way to understand this is to work through two thought experiments. First, consider a group of people with lung cancer who will all die at age 70. If they first receive the diagnosis when they are 67, their 10-year survival rate is 0 percent. (Do you see why? If they all die at age 70, none will be alive 10 years later.) But if these same people had received CT scans that found their cancer before they had symptoms, they would have been diagnosed earlier—at, say, age 57—and their 10-year survival rate would have been 100 percent. Yet death would still come at age 70. Earlier diagnosis always increases survival rates, but it doesn’t necessarily mean that death is postponed. This diagram helps you review how this happens.

In other words, making the diagnosis earlier does not in itself mean that you delay death. It may simply mean that you know about the cancer for a longer time.

Comparing survival rates when people are diagnosed with a disease in different ways—for instance, based on a CT scan versus based on symptoms—is always misleading. That’s because the CT scans are great at finding early, hidden cancers, and survival rates always go up when diagnoses are made in early stages.

A second thought experiment helps to explain why CT scans, which find so many minute tumors, can inflate survival rates even if no lives are saved. It turns out that some forms of cancer look like cancer under the microscope but do not behave the way we might expect cancer to—that is, they do not develop into a relentlessly progressive, deadly disease. These nonprogressive cancers grow so slowly that they never cause symptoms, nor do they affect how long a person lives. Consequently, the only way we can diagnose these nonprogressive tumors is by screening.

Prostate cancer provides the most familiar example. We can find microscopic evidence of prostate cancer in around half of 60-year-old men if we look hard enough. Yet only 3 in 1,000 will die from prostate cancer over the next 10 years. How can this be? Because prostate cancer isn’t just one disease: it’s a spectrum of disorders. Some forms of prostate cancer grow very rapidly and kill the men who have them. Some grow so slowly that, even without treatment, men die of something else before the cancer causes symptoms. And other forms look like cancer under the microscope but never grow at all or may regress spontaneously. While this phenomenon is best understood in prostate cancer, it probably occurs in all cancers.

Now imagine a city in which 1,000 people are found to have lung cancer following evaluation for cough and weight loss (symptoms of lung cancer). At 10 years after diagnosis, 50 are alive and 950 have died, which equals a 10-year survival rate of 5 percent. Now let’s start again—but this time imagine that everyone in the city is screened with CT scans. In this scenario, 5,000 are given a cancer diagnosis: the 1,000 who had symptoms, plus 4,000 others who actually have nonprogressive cancers. These 4,000 would not die from lung cancer in 10 years. Consequently, the 10-year survival rate for lung cancer in the city would increase dramatically—to 81 percent—because these healthy people would appear in both parts of the fraction: 4,050/5,000. But what has really changed? Some people were unnecessarily told that they had cancer (and may have undergone harmful therapy), but the same number of people (950) still died. We have sketched out this scenario in the following diagram.

The idea that screening can increase survival rates without actually saving lives is not just a theory. It was confirmed in a randomized trial of regular chest X-ray screening at the Mayo Clinic involving more than 9,000 male smokers.² In that trial, 5-year survival rates were almost two times higher for those who had been screened than for those who had not been screened (35 percent versus 19 percent), but death rates were the same in the two groups. In fact, the death rate was slightly higher in the screened group. Consequently, doctors do not recommend lung cancer screening with chest X-rays.

Survival statistics and death statistics sound like two ways of talking about the same thing. But they are not flip sides of the same coin. To understand how this works, it helps to look at how these kinds of statistics are calculated.

Let’s start with survival statistics. As an example, let’s calculate the 5-year survival rate for a particular type of cancer (we’ll call it cancer X). As explained earlier, the 5-year survival rate for a group of patients diagnosed with cancer X is the number of people in the group who are alive 5 years after their cancer diagnosis, divided by the number originally diagnosed. Here’s the formula:

Now let’s calculate a death statistic, which typically refers to a 1-year time frame (you can think of it as presenting a 1-year risk of death). For example, the death rate (or mortality rate, as researchers call it) for cancer X is the number of people in a group who die from that cancer over 1 year, divided by the number of people in the group. Here’s the formula: The key thing to notice is how the denominator (the bottom number in the fraction) for survival statistics differs from the denominator for death statistics. For survival rates, the denominator is the number of patients diagnosed with the illness; for death rates, it is all the people in a group (which could be, for instance, the population of a country, an age group, or some other defined group).

Survival statistics are always distorted by screening. As you saw in the last two illustrations (pages 91 and 93), survival statistics can go up even when the same number of people die. So survival statistics cannot tell you whether fewer people are dying from cancer.

In contrast, death rates can tell you what is really happening. If finding early cancer delays death, the annual death rate indeed goes down, since fewer people are dying in that year. But if screening is finding only more nonprogressive cancer, the death rate doesn’t change.

But comparing survival rates and death rates is like comparing apples and oranges. In fact, when you look at our best national cancer data, you will find no relationship between improved 5-year survival rates and death rates over time.³ For example, the 5-year survival rate for melanoma, the most deadly form of skin cancer, has improved from 49 percent in 1950 to 92 percent in the most recent data available. But death rates have actually gone up during this same period, from 1 death in 100,000 to almost 3 deaths in 100,000.⁴

So when you hear about changes in survival rates among people who have been screened for a disease, you cannot automatically assume that these changes reflect fewer deaths from the disease. Such changes could simply mean that we are diagnosing more early cases of disease but not delaying death, or that we are finding more nonprogressive disease. In other words, improved survival rates among people who have been screened for a disease provide no evidence that screening works to reduce deaths. The only way to know whether screening works is to show that death rates are lower in the screened group. Screening may well reduce deaths from a disease, but survival statistics are never a good guide to whether this is happening.

When used for the right purpose, however, survival statistics can be a good guide. The Learn More box on the next page shows you how survival statistics can be used appropriately to estimate your prognosis (your chance of surviving for a fixed period of time with a particular disease) and to judge the benefit of a treatment.

This discussion about survival rates and death rates may seem a little counterintuitive. To help make sure you understand the ideas we’ve covered, take a look at the following chart, which appeared in a 1990 brochure sent to patients by the M. D. Anderson Cancer Center, a major academic medical center in Texas.

The correct answer to the first question is c. It turns out that U.S. survival rates for prostate cancer—not just the survival rates for M. D. Anderson’s patients—also went up dramatically during this period, from about 40 percent in I960 to 90 percent in 1990. This increase in 5-year survival reflects the growth of prostate cancer screening with the PSA blood test. In fact, the average U.S. 5-year survival rate of 90 percent was, ironically, higher than the survival rate at M. D. Anderson. To judge whether fewer people are dying of prostate cancer, death rates are the right measure to use. Unfortunately, this line in the chart looks pretty flat—suggesting that deaths from prostate cancer in the United States did not change much over this time. Comparing the 5-year survival rates of M. D. Anderson’s patients with overall U.S. mortality rates is nonsense.

The correct answer to the Extra Credit question is c, 70 percent. Here’s how we did the calculations. First, we wrote out the fraction for the 5-year survival rate of 40 percent:

Then we added the 1,000 people with nonprogressive cancer to both the numerator and the denominator: Although this example may be a little difficult to follow at first, it allows you to see for yourself how dramatically 5-year survival rates can change when nonprogressive cancers are diagnosed—although no one has lived any longer.