Measurement Difficulties

The primary outcome of interest is the inability of a patient to work. There are, however, difficulties with this outcome measure: "inability" is a concept that cannot be directly measured. Other difficulties stem from the fact that answering such a question entails determining whether an individual is unable to work at a particular job.

Assessing inability to work must take into account the patient's physical, psychological, and social well being, as well as the type of job the patient would perform (Hu, Lahiri, Vaughan et al., 1997). Attempts to construct a simply administered test to measure inability to work would likely be flawed. For example, patients may over- or underestimate their ability to work, which means that a questionnaire, whether self-administered or administered by another, is not guaranteed to be accurate. Compounding this problem is that perfect accuracy is not guaranteed by increasing the complexity of the test. For example, patients who are highly motivated to work might be able to "pass" a test consisting of administered questions and of physical exertion, and still be unable to work for 40 hours per week. Similarly, patients less motivated to work may not "pass" such a test, even though they are, in fact, physically able to work. Therefore, the ability of any current test or any practical test to distinguish between those patients who are truly able to work and those who are truly not is questionable. Finally, although the examples here have incorporated questionnaires to "predict" inability to work, employing biochemical or clinical tests to measure ability to work does not circumvent these problems. One is still faced with the difficulty of attempting to measure inability to work.

These difficulties in measuring inability to work are reflected in the literature. Our literature searches (performed in Phase 1 of this project) demonstrated that there is no single widely accepted measure for inability (or ability) to work either for patients with CRF specifically or across multiple diseases (Kutner, Brogan, and Fielding, 1991; Wolcott, Nissenson, and Landsverk, 1988; Ferrans and Powers, 1985; Gutman, Stead, and Robinson, 1981; De-Nour and Czaczkes, 1975). In other words, there is no "gold standard" test for inability to work. Therefore, it appears that practical and ethical methods to measure a patient's true inability to work remain to be developed.

As such, it is possible only to infer a patient's inability to work. It must be recognized, however, that no inferential measurement of inability to work is perfect, particularly if the inference is based upon a relatively simple test. Nevertheless, it is theoretically possible to obtain information about the relative accuracy of certain inferences and perhaps to determine which (if any) inferential or surrogate measures of a patient's inability to work are better than other measures.

These inferential or surrogate measures include:

• Employment status. This term refers to whether a patient is employed at the time he/she was asked about his/her employment status. Because it is possible that some unemployed patients are able to work but it is not possible that patients who are unable to work do in fact work, employment status equals or underestimates the true ability to work; employment, in turn, can be treated as an estimate of the complement of (or the inverse proportion of) true inability to work.
• Self-reported inability to work. This measure refers to whether an individual states that he/she is unable to work. The outcomes "reported ability" and "reported inability" to work are both found in the literature. Because it is possible that some patients overestimate their ability (or inability) to work (Kruger and Dunning, 1999; McKillop, Berzonsky, and Schlenker, 1992; Lachman and Jelalian, 1984), reported inability to work is not a perfect reflection of their true inability to work. It is also not possible to determine whether self-reported ability to work is more or less accurate than employment status as a surrogate measure of inability to work. Barriers to employment are particularly relevant when considering self-reported inability to work. These barriers are discussed under the definition of "functional status."
• Functional status. Functional status is comprised of social, psychological, and physical aspects. For the purposes of this evidence report, our definition for this term is based on the Kidney Disease Quality-of-Life (KDQOL^TM) questionnaire and SF-36. Included in "functional status" are physical functioning, role limitations caused by physical health problems, role limitations caused by emotional health problems, social functioning, emotional well-being, pain, energy/fatigue, and general health perceptions.
  Functional status can be examined both as an outcome variable (i.e., a dependent variable) and as a predictor variable. Thus, one may not only attempt to ascertain the relationship(s) between other variables (e.g., patient age, creatinine levels) and functional status, but also attempt to ascertain the relationship(s) between functional status and the above-noted inferential measures of employment.
  Self-reported ability to work and functional status are unlikely to correlate strongly with employment status because of current existing nonclinical barriers to employment. Assuming all other factors are equal, the presence of such barriers will cause the percentage of patients who report they are able to work and who have relatively high functional status scores to exceed the percentage of patients who are actually employed. These considerations suggest that self-reported ability to work and functional status could be more accurate (although still imperfect) measures of true inability/ability to work than employment status. On the other hand, patients' estimates of their own abilities may be imperfect; therefore, these measurements may either overestimate or underestimate a patient's ability to work, depending on the patient's motivation.
• Receiving disability. This term refers to whether patients are receiving Social Security disability benefits. This measurement can serve as a benchmark for determining the impact (expressed in terms of number or percentage of patients) of any proposed modification of the disability criteria currently used in the Listings.

Published Studies on Difficulties in Measuring (In)ability To Work

Though SSA uses its own operational definitions of "disability" and "inability to work," the definitions of these terms in the clinical literature vary considerably. In this section, we explore some of these variations.

There are complex relationships among pathologies, physical and mental functions, and social functions (such as the ability to carry out gainful employment) (Verbrugge and Jette, 1994; Hahn, 1993; LaPlante, 1991). A broad categorization of the different levels of function has emerged (Nagi, 1991; Rettig and Levinsky, 1991; World Health Organization, 1980; Nagi, 1979; Nagi, 1965); however, exact definitions from various sources are overlapping and not in complete agreement (see Table 1). This broad hierarchy is briefly described here:

• Disease process. A past or current disease or pathological condition underlies and is the causative factor or one of the causative factors for the condition or health state.
• Physical or mental function. These are biological or psychological consequences of the condition that occur from the molecular and cellular level up through the organ system level.
• Limitations of action. As a consequence of the impairment, there is a loss of ability to carry out certain action(s), such as the movement of an arm or leg, detection by a sensory organ, or the performance of a simple mental task.
• Limitations in basic activities. Because of the action inability, certain basic activities are limited or impossible, such as walking, running, sensory awareness of the social and physical environment, or carrying out sequential mental tasks.
• Disability/disablement. Because of the limitation in basic activities, the individual may be unable to or have limited ability to engage in physical and mental tasks and complex social activities, such as housework, economically gainful work, social interaction with family and community, procreation, and recreation, that are normally part of the social environment or are required by the physical environment. Unlike the above levels of function, the existence and extent of disability/disablement is determined not only by the functional capabilities of the individual, but also by the demands of the social and physical environment (Verbrugge and Jette, 1994; Hahn, 1993). Also, in some cases, a disability can be ameliorated or even completely removed by behavioral, medical, or device-related interventions, even though none of the above levels of function or pathology has changed (Verbrugge and Jette, 1994; Hahn, 1993).
• Quality of life. Suboptimal performance in any of the above areas can diminish an individual's quality of life. Quality of life may not be directly relevant to employment disability, but it is involved in the consideration of disability in general.

Table

Table 1. Impairment and disability terminology.

This evidence report is concerned only with conditions that limit one's ability to engage in substantial gainful activity (SGA). However, determination of an individual's capacity to engage in SGA requires in-depth and subtle knowledge of the other levels of function described above and a mapping of these functional abilities to job requirements. The medical classification of pathologies is based on disease etiology or causation, not on the physical, mental, and social consequences of disease; therefore, there has been a general failure of strictly medical diagnoses to inform disability policy and to produce universal or consistent criteria for disability (Hahn, 1993; Stone, 1984).

The establishment of criteria for determining employment disability involves unique problems beyond defining the concept of disability in general. In a market economy in which economic competition is an inherent part of gainful work, there is the further complication that the mere ability to go through the motions of work does not ensure that the work will in fact be competitive and gainful. There are also a number of factors that are difficult to assess in terms of physical or medical observations. These include the amount of pain involved in otherwise performable activities, the amount of time and energy outside working hours that are expended dealing with the disease or its consequences, and whether a person with a terminal condition should be required to work as close as possible to the time of genuine inability to work or death. This concept is so deeply imbedded in social, cultural, and economic frameworks that are unique in time and place that it is only possible to approach the answer to the question of "who can work?" by instead answering by direct observation the question of "who does work, and under what circumstances?"

Answering the question of which patients with ESRD can work is made even more difficult by the fact that all such patients are currently found disabled at the third step of the SSA disability sequential evaluation process. Thus, the fact that a patient is not working may simply be because he or she would rather receive disability than be gainfully employed or that a patient anticipates (rightly or wrongly) that he or she will be unable to work in the near future. One can, however, gain some insight into the relationship between the impairment caused by CRF and the ability to engage in substantial gainful activity. This can be done by following those patients who are gainfully employed at the time they are diagnosed with ESRD in spite of the fact that they qualify for disability. The measurable medical parameters at the point that these patients stop working would be the criteria that ideally belong in the Listings. This is because these parameters indicate the point in disease at which well-motivated patients cannot (or do not) continue to work. Also, the point at which essentially no patients continue to work is important because this point is when education, skills, and unique job requirements no longer influence one's ability to work. Using measurable medical parameters at these two time points is compatible with the purpose of the SSDI screen-in process, which aims to identify conditions with which no one should be expected to work, regardless of skills and present job requirements. In the SSA disability sequential evaluation process, such individual background and job factors are considered in steps 2, 4, and 5. The Listings are intended to contain conditions that automatically result in a favorable disability determination for the most severely impaired patients, regardless of job factors.

A further difficulty is encountered because no diagnostic cut point for determining who is unable to work will be perfect. Any "test" for inability to work will invariably misclassify some individuals as being unable to work when, in fact, they can work, and/or individuals may be misclassified as not disabled when, in fact, they are unable to work. Therefore, the challenge becomes one of maximizing the ability of any "test" for disability to correctly determine whether a person is able or unable to work, while minimizing the ability of any such "test" to incorrectly determine an individual's inability to work. Issues related to determining the predictive value of a "test" for disability are further discussed in Appendix E.

In summary, the evidence report cannot directly assess inability to work. This is not due to a lack of literature on the subject or lack of data per se. Rather, it is due to the impracticality of directly measuring this concept. This problem is inherent to all studies of disability. Consequently, the evidence report must focus on outcomes that can be or have been measured.

Outcomes Used in the Evidence Report

As discussed above, all of the outcome measures listed in this subsection are inferential measures of inability to work. Employment status, functional status, self-reported ability to work, and SSA disability status are all considered.

For the purposes of this evidence report, the definition of functional status is based on the KDQOL^TM questionnaire, which is a disease-specific expansion of the SF-36 that was used in the USRDS DMMS Wave 2 study (discussed later in this report). Concepts included here are physical functioning, role limitations caused by physical and/or emotional health problems, social functioning, emotional well being, pain, energy/fatigue, and general health perceptions.

Issues Not Addressed in This Evidence Report

It is important to list the issues that are not addressed in the evidence report. These items include:

• Kidney transplantation. SSA has requested that this procedure not be covered in the evidence report.
• Differences in outcomes between hemo- and peritoneal dialysis. There is a potential for considerable selection bias when determining which form of dialysis a patient should receive, which precludes addressing this issue in a satisfactory manner.
• Questions about dialysis duration. Dialysis duration is an indirect measurement of the patient's severity of disease. Thus, questions about physiological functioning as measured by laboratory tests are more direct and will be assessed in this report.
• Questions about the efficacy of EPO. Although this may be of considerable interest in nephrology, questions involving this drug are peripheral to the major questions of this report. EPO is administered primarily to ameliorate anemia. The key questions address anemia directly, among other physiological states. Issues about whether EPO should be administered are questions concerning appropriate clinical practice, not disability.
• Questions about rehabilitation and patient education. While these questions are of considerable interest, they are not directly related to the key questions of this report.

Kidney Disease

• American Association of Kidney Patients (www.aakp.org)
• About Epogen (wwwext.Amgen.com/cgi-bin/genobject/productEpogen/tig__5Znvfv)
• Directory of Kidney and Urologic Diseases Organizations (www.niddk.nih.gov/health/kidney/pubs/kuorg/kuorg.htm)
• Forum of End Stage Renal Disease Networks (www.esrdnetworks.org/)
• Hypertension, Dialysis, & Clinical Nephrology (www.hdcn.com/)
• Kidney and Urologic Diseases Statistics for the United States (www.niddk.nih.gov/health/kidney/pubs/kustats/kustats.htm)
• National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) (www.niddk.nih.gov/)
• National Kidney Foundation (www.kidney.org/)
• The Nephron Information Center (nephron.com/
• Nephrology News and Issues (www.medicalnews.com/nephrology/)
• RENALNET (www.renalnet.org/renalnet/renalnet.cfm)
• United States Renal Data System (www.med.umich.edu/usrds/)

Disability and Rehabilitation

• Disability Resources Monthly (DRM) Guide to Resources on the Internet (www.geocities.com/~drm/)
• Disability Statistics Center (dsc.ucsf.edu/)
• Employment Project's Homepage. Efforts to remove work disincentives (www.teleport.com/~enygma/employ/)
• National Institute on Disability and Rehabilitation Research (NIDRR) (www.ed.gov/offices/OSERS/NIDRR)
• National Organization on Disability (www.nod.org/)
• National Rehabilitation Information Center (NARIC) (www.naric.com)
• Research Institutes, Universities, Rehabilitation Centres (www.gladnet.org/research.htm)
• Vocational Evaluation and Work Adjustment Association (VEWAA) (www.vewaa.org/)