Estimating Mortality Due to Disease and Malnutrition

Les Roberts of Johns Hopkins University described mortality surveys conducted in eastern Democratic Republic of Congo (DRC) using spatial sampling techniques.²² Two different methods were used, grid-based (quadrate) sampling and multistage cluster sampling. A very brief survey was used to try to determine mortality rates during a recent period of violence and displacement. The researchers encountered a variety of problems, including a large percentage of vacant residences (people had fled since the war began or whole households had been killed or died) and a number of security issues and geographical barriers.

Using a handheld global positioning system unit, the team selected clusters of households from clinic catchment areas (the areas that individual clinics are supposed to serve). Rural areas tended to be oversampled, so weighting was used to compensate, although comparing the weighted and unweighted results showed no difference in mortality rates. The team concluded that in eastern Congo there was no correlation between population density and mortality, and therefore most mortality was likely caused by violence, rather than communicable diseases, which spread more quickly among denser populations. Most mortality was due to the disintegration of the social fabric, which led to physical insecurity and ultimately deaths from violence. However, the small number of children in the population relative to a “normal” sub-Saharan African population was also a clear indicator of poor health conditions.

Roberts found that spatial sampling techniques tended to bias the sample toward large households or families, in comparison with cluster sampling methods, because large households are more likely to be captured since they physically take up more room than do smaller households. Nevertheless, the technique was sensitive enough to reveal that mortality during the period of study had tripled over the baseline mortality rate for the population under study (as measured at the beginning of the study). Urban displaced populations had higher mortality than rural displaced populations, perhaps because of urban-centered violence.

This study showed the change in mortality over time by comparing estimates at the end of the study to those at the beginning of the study (1999-2001). Commenting on Roberts' presentation, Fritz Scheuren of the Urban Institute noted that it is important to distinguish between people who refuse to participate in a survey and those who cannot be reached and to use repeated measures to check the accuracy of estimates. One problem with mortality surveys is that respondents often have trouble with recall and may tend to displace deaths or situate them in the recall interval when they actually occurred in other periods not covered by the survey. This may be especially problematic if respondents are suffering from trauma. Overall, however, Scheuren felt that the survey was excellent considering the grueling circumstances in which it was conducted.²³

In a small group discussion, participants discussed the advantages and disadvantages of various sampling methods for estimating mortality rates. The group agreed that epidemiologists could benefit substantially from consultations with sample survey statisticians, rather than using standardized methods in every crisis. There should be more data management in the field, and surveillance techniques need to be adapted for use in open populations (e.g., self-settled refugees). If time and resources make it feasible, an event history should be collected from each person in a survey, using a memorable point in the past as a starting point (such as a national or religious holiday).²⁴ Indirect methods are helpful for reducing bias, but they have so far been underused in emergency settings. Although there are two different types of data needs in crises—immediate and historical—a hybrid type of estimation method can address both needs.

Statisticians pointed out that some epidemiologists currently do not keep track of those in the sample who are not reached (noncontacts), but simply keep sampling in order to reach a quota of households. The noncontact rate should actually be estimated in advance and used to calculate the total sample size needed, with no substitutions allowed for noncontacts. This approach obviously creates practical problems because the staff needed to carry out surveys with large sample sizes in the field is lacking. Yet reporting the noncontact, nonresponse, and vacancy rates when a study is published would help to deflect some criticisms of the methodology and hence, of the estimates themselves.

Estimating Mortality Due to Human Rights Abuses

Human rights abuses—violations of international standards of human rights, such as kidnapping, rape, torture, and genocide—are traditionally documented through testimonies and clinical or forensic evidence. Now these methods are also being used in combination with epidemiological and demographic methods. Cluster sampling is not appropriate for measuring mortality in cases of human rights abuses because they are not random events. Some nonprobability sampling techniques that might work in these situations include: network sampling, in which more than one individual can report on another individual in the survey, multiplicity or “snowball” sampling, which uses a process of chain referral by members of the population of interest, and adaptive sampling, which is a technique often used for studying disease patterns among wildlife in which if a diseased animal is discovered, then additional animals are sampled in the same area.

Some types of probability sampling may still be useful for measuring human rights abuses, but it depends on how much knowledge a researcher has about the probability of the events he or she is trying to measure. Finally, those who perpetrate human rights abuses sometimes record demographic data that can later be used by human rights investigators and criminal tribunals or truth commissions. Collecting and working with this type of data may be more akin to working with administrative data for historical demographic research. Obviously, as with any political data, the reliability of such lists is somewhat suspect. Researchers have to take the source of the data into account, especially when the source is the group that perpetrated the abuses or those who oppose that group. They both might have reasons for falsifying data. Depending on the ultimate use of the data, precision and reliability may be more or less important: for example, data precision is generally very important in legal cases, such as human rights tribunals.

Patrick Ball of the American Association for the Advancement of Science presented some of his work on measuring human rights abuses in a variety of settings. One effective method is called capture-recapture, in which one member of the sample is selected, “returned” to the population, and if the same person is selected again, she or he is “tagged.” Originally developed for use in counting populations of animals or fish, for human rights research the method uses administrative data that were kept by the perpetrators. Because more than one witness may report a killing, some killings may go unreported, and there are often several sources of testimonies, multiple lists of killings are often available and must be used to estimate the actual number of murders. According to the theory behind the capture-recapture method, the level of overlap, or the number of killings reported to two independent lists, makes it possible for investigators to estimate the number of killings that occurred in the total population including those killings that were not recorded in either list. The method was used in Guatemala following the civil war in that country (1954-1996). Total killings were calculated by region, ethnicity, period, and perpetrator. The study estimated that approximately 200,000 people were murdered during the conflict.²⁵

Ball and his colleagues also conducted a study during the recent war in Kosovo (1999) using lists from three different organizations. They found that killings and migration during the conflict were highly correlated. The total estimates of deaths corresponded to estimates from other sources, so these findings appeared to be valid.²⁶ There are limitations to this method, however. It requires three large lists for comparison and there is a “heterogeneous catch ability”: in other words, the lists must be randomly ordered or the validity of the method cannot be ensured. The advantages of this method are that in most of these situations lists are easy to find and hence estimating the magnitude of mortality is relatively easy.

Noting the huge value of the work in Guatemala, William Seltzer of Fordham University observed that it was so well done that even the Guatemalan army did not argue with the results. Capture-recapture methodology should be compared with typical demographic analyses (such as cluster sampling techniques, etc.), and postenumeration surveys can also be used to check the validity of estimates.²⁷ Capture-recapture is a standard technique among wildlife researchers, but it has been adapted for special application in this instance, so there are some additional problems with the technique. As Seltzer noted, sometimes it can be difficult to match records from the different lists if names are spelled differently or especially if several people have the same name. Some reports of abuses may be outside of the period of interest because of recall biases—for example, if those who reported the abuse remembered the wrong date. In addition, certain classes of events are unlikely to be “captured;” therefore, it is important to stratify the lists (perhaps by date or location) and use different types of estimation within different strata.

A small group discussed different approaches for collecting data on human rights abuses. In order to prevent bias, they suggested surveying many houses to validate reports of an event (this is called many-to-one reporting) and using indirect methods. Education and training in human rights are vital for humanitarian aid workers so that they can recognize human rights abuses, report them properly, and learn how to collect data on them. Human rights data are not just important for historical analysis, but could be used in real time to prevent genocide or other abuses.

Footnotes

21 For a full explanation of the difference between the CMR and the CDR, see National Research Council 2001:8.

22 For a full report of this study, see Roberts et al. 2001.

23 Due to the ongoing violence and insecurity in the Democratic Republic of Congo, researchers were extremely limited in where they could travel, and the surveys were conducted under very dangerous conditions.

24 An event history is a timeline of personal life events often used in demographic research to study changes over time; migration histories and fertility histories are common examples.

25 For further discussion of the capture-recapture method, see Ball et al. 1999.

26 Ball's work was submitted as evidence by the prosecution against former Yugoslav president Slobodan Milosevic at the International Criminal Tribunal for the Former Yugoslavia (ICTY). See Ball 2000 and Asher et al. 2002 for more information on the study.

27 Postenumeration surveys are conducted after a survey to determine any biases in the sample.