Methods

Many different methods have been employed to assess the quality of medical care. This section presents a matrix for classifying these methods, discusses some principles, and raises problems associated with some of the methods. A detailed discussion of each method is beyond the scope of this report but discussions can be found in some of the articles listed in the annotated bibliography attached to the report as Appendix A.

Quality assurance methods differ mainly in two regards: the type of data to be used and the source from which the data are obtained. Figure 1 presents a summary of the possible interrelationships among these two matters. Some definitions will facilitate consideration of the figure.

Methods for collecting data

Resource and organization data, gathered by a variety of agencies, include the number of hospital beds in a community, vital statistics, and the ratio of physicians to population.

Hospital discharge abstract includes a uniform basic set of data collected for all patients discharged from all hospitals and expressed in standardized terms, definitions, and classifications. This basic data set can be augmented by problem or disease-specific information.

Figure 1

Quality of Care Assessment:  Possible relationships between the type of data by which the assessment of the quality of care will be made and method by which that data will be collected.

Claims form includes basic data used by a physician or health facility in billing insurance intermediaries for reimbursement. By employing uniform terms, definitions, and classifications, it can be linked to or incorporated with the Hospital Discharge Abstract.

Encounter form includes a uniform basic data set as part of the record for each patient visit; a statement of the patient's problems, diagnosis, procedures such as laboratory tests, services provided and disposition of the patient is included. It can be linked to a claims form and in some instances to a hospital discharge abstract.

Source-oriented medical record is the traditional manner in which the medical record is kept. Each provider participating in the care, such as nurses and physicians, initiates data, usually in different parts of the record.

Problem-oriented medical record is a recent innovation in medical record keeping. The record system is basically time-sequential; the patient's salient problems are defined, and the record clearly links each problem to its clinical investigation and management. Such records provide a clear indication of the physician's clinical logic and differ from source-oriented records in that for any given encounter all the data relating to the encounter are found in one place in the chart, regardless of who entered them and are related to the appropriate problem.

Direct observation of practitioners is data acquisition by trained observers who watch and record information about what a health professional actually does in practice.

Simulation techniques employ actors, computers, and other means to simulate clinical situations which enable an assessment of a health professional's response.

Patient interview is the collection of data by interviewing patients (face-to-face, by telephone, or by mailed questionnaire) who sought services from a given health provider.

Tracer disease strategy is an approach to selecting and using indicators to assess the quality of medical care. It relies on sampling sets of discrete units of process for which outcome can be identified. Analysis of the sample data provides a basis for generalizing about the effectiveness of medical processes, provider performance, and organizational management at a collection site.

Population survey is the collection of data by means of household interviews or health examination surveys from properly selected random samples of a defined population or an entire population, both those who have and those who have not actually used health services during a specified time period, such as two weeks or a year.

Criteria for the application of data

After the types of data and methods of data collection have been determined, criteria must be established for applying the data to the assessment of the quality of care. Evidence suggests that the results of quality of care assessment will vary with the purpose for which the criteria were established, who determined the criteria, and the methods used in the establishment of criteria. The broad range of possible approaches illustrates the complexity of attempting to assess quality on a uniform and objective basis:

The purpose for establishing criteria can be administrative alone, or designed to serve an integral role in the care of the patient.

Criteria can be set by groups of providers external to the provider group whose care is being reviewed, or established by the group subject to review.

The manner in which criteria are established can be explicit (predetermined and understood by the group in advance) or implicit (judgments relying on the subjective opinion of the individual judge).

Explicit criteria can be normative judgments based on the opinions of an individual or a group of appropriate experts, or empirical judgments based upon data obtained from objective assessments, evaluations, or analyses.

The way in which criteria are set may help determine the degree of compliance. Local physicians may be less cooperative with criteria established at the national or state levels than those established locally, but local criteria may not reflect the most desirable practice of health care based on more objective analysis.

A later section of this report (see p. 28) discusses the setting of standards, which are defined as the expected level of compliance with criteria. The setting of standards raises many of the same issues involved in establishing criteria. Although the issues are similar, the setting of standards is considered separately because, in the context of this report, standards represent a regulatory decision which may involve many factors other than the technical criteria, whereas the establishment of criteria is a technical task based on professional competence.

Examination of Figure 1 suggests that the choice of a method of quality assurance is a complex undertaking. Yet even this figure oversimplifies the task. Other variables to be considered in developing a method to assess quality of care include the extent of concern about availability, accessibility and cost of care; the costs of the method; whether criteria for adequate care will be set externally or internally; and whether the criteria will be derived administratively by a public agency.

The committee recommends that the process of setting criteria for the use of data in a system of quality assessment be kept flexible so that the system may combine, as appropriate, the values of various techniques in criteria setting.

Structural, process, and outcome data

Any method of assessing the quality of health care should either measure improvements in the health of the population served by the system as well as in the population not served, or should measure a set of variables that have a highly predictable and reproducible correlation with changes in health status. In addition, a quality assurance system should assist in efforts to understand the relative effect of personal health care services on health status as against the effect of other influences such as the environment.

In Figure 1, the “type of data to be used” is arranged so that proceeding down any column yields categories in which increasingly more direct measurements of health can be made. But the categories in the left-hand columns--“process” and #8220;structure”--provide insufficient data by any collection method to serve as the sole basis for assessing medical care quality. In most cases, research has not substantiated the relationship between these categories and health status. A number of commonly accepted structural indicators of good quality, such as the extent of a provider's training, have not been consistently shown to account for all the differences in quality of medical care, although structural factors may make measurable improvements in the processes of care. Such factors as class rank and grades in medical school, a physician's board certification status, the place of physician training, the presence of a full-time radiologist or pathologist on the staff of a hospital, and the percentage of patients autopsied may have some relationship to measurable quality, but they are not sufficient factors by themselves. Such factors as the number of journals read, participation in continuing education as it is currently practiced in most situations, and membership in medical societies have no identifiable relationship to outcomes of health care.

Inherent unreliability notwithstanding, certain structural and process information that can be related to outcome of care is useful in assessing the quality of care. An example is standards for safety in electrical equipment used in care. Many other structural aspects of care need to be assessed in terms of effects on outcomes. When it can be conclusively shown that a process or structural factor is causally related to outcomes, greater reliance can be placed on it.

Data source, cost, and reliability

Possible sources of data are displayed along the vertical axis of Figure 1. In general, unless sampling of data is performed, data obtained from the sources listed toward the bottom of this axis are more expensive than data obtained from sources toward the top. However, except for data derived from hospital case fatality rates or complication rates, which can be obtained from a hospital abstract or medical record, patient outcome data are generally available only from patient interviews and population surveys. Such outcome data are usually expensive, although methods recently tested suggest that under certain circumstances functional capacity or disability and distress levels can be assessed reliably and collected at the time of and during hospitalization.^*

Only through the use of a community survey or survey of a registered population can information be obtained from a properly selected random sample of those in a population who used services as well as those who did not. To assess the effectiveness of health services, users have to be compared with nonusers.

Besides content, there are questions about data with respect to reliability and validity^** and to the amount of professional time needed for its acquisition. Data abstracted from the medical record may have more to do with the quality of data recorded by the physician than with the quality of care provided. If the medical record is not kept in a neat, legible, problem-oriented format, mistakes made in abstracting or judging the record may be so numerous that the information is less useful. On the other hand, data obtained from claim forms, encounter forms, or hospital discharge abstracts may not be sufficiently detailed to judge fairly all aspects of the performance of an individual provider, or the care of a particular patient. Observations of a practitioner's work require a great amount of time and are difficult to standardize. Finally, no matter what source is used, the volume of primary data collected on individual cases is likely to be large.

Detailed and accurate data, such as those contained in the medical record, are often necessary for the delivery of good quality medical care. But if the data are used in the review of all care in a region or large institution, the quantity becomes unmanageably large. An accurate abstract of pertinent data can reduce the quantity and enable data aggregation for statistical profiles. And the development of aggregated statistics can be further simplified, reduced in cost, and frequently made more accurate, by sampling. Simpler data to begin with, such as that contained in an abstract, are more likely to retain their accuracy when handled successively in data processing.

Specific methods in quality assessment

Recent research in quality assurance and the relationships indicated by Figure 1 prompt the committee to make the following observations on the application of specific methods:

• --Because so few of the persons who have developed quality assurance methods have worked collaboratively, or have tested alternative methods, there is little consensus on an optimal method or set of methods. Available comparisons of different methods to assess the quality of care show wide variations among methods in the level of acceptable care found.^* The comparisons suggest that assessment of quality of care is affected by the method used to measure it. They also suggest that the use of explicit process criteria to assess medical care may foster a dramatic increase in the number of services without a corresponding improvement in the health of the population.
• --Other important issues are raised by the trade-offs between the use of explicit or implicit and normative or empirical judgments in the formulation of criteria. Explicit judgments are more reliable and involve less physician time. But implicit judgments permit the review of all aspects of a case because predetermined criteria--essential for explicit judgments--are seldom available for every relevant aspect of a patient's condition and treatment.

Recommendations about methods

The implementation of the PSRO program will afford an opportunity to test the validity of some of the committee's observations. An examination of a number of the predecessor programs, such as some of the EMCROs,^* QUAP,^** TAP^*** and some material developed by the Foundations for Medical Care, has led the committee to believe that it is probable that the method chosen by most of PSROs will be based on explicit judgments of process data derived from a modified type of hospital claim form.

There are a number of problems with this approach. First, as indicated before, this type of review is likely to result in increased costs without corresponding improvement in health status.

Second, innovation may be stifled. As long as correlations are lacking between process and outcome, there will be a tendency to formalize lists of procedures to be employed in given diseases and regard noncompliance as inadequate care. Not all practitioners, however, agree on a “proper” procedure. In the treatment of myocardial infarction, for instance, there is some evidence to indicate that suitable patients treated at home are as likely to recover as those hospitalized.^**** Yet current practice dictates protracted hospitalization, and if that were made a standard practice under the PSRO program efficacious alternatives could be discouraged or even proscribed.

A third problem is the potential for increased hazard to the patient. Prescriptions of lengthy hospital stays, more drugs, more x-rays, and the like increase the risk to the patient--a risk that needs to be balanced by demonstrable benefits. And increased demand for tests adds to the demand for manpower, and ultimately to the cost of care.

The preceding examination of data requirements and methods of review leads to two recommendations for initiating a quality assurance system.

The committee recommends that data for a quality assurance program be based on the universal use of decentralized and regionalized hospital discharge abstract systems that employ a uniform basic data set. ^*Each abstract contains data on the patient's place of residence, demographic characteristics, procedures, diagnosis, length of stay and health status at time of discharge. In all, fifteen basic items have been recommended, including one for the total of charges of the hospital stay. The hospital discharge abstract data set is so constructed that it can be linked to similar uniform basic data sets of claims review and encounter forms. Regional or institutional comparisons of items contained in this data set such as the utilization rate for a specific operation or death rates in myocardial infarction would be useful to quality assurance systems as a first step. If the current PSRO legislation does not permit the use of this approach, comparisons should be made between the PSRO approach and the recommended approach.

Use of the abstract puts the emphasis where it should be at the initiation of a quality assurance program on both process and outcome measurements. As more experience is gained with understanding information from the abstract, the Uniform Hospital Discharge Abstract data set could be augmented by the inclusion of data about those medical care processes which have been shown to be correlated with beneficial patient outcome. The committee acknowledges that this data set allows only a beginning assessment of outcome and alone will not be sufficient to analyze case mix differences among hospitals. We believe, however, that the Uniform Hospital Discharge Abstract data set is the simplest, most acceptable, and reliable uniform data set that can be utilized by the entire hospital community immediately. The committee recommends that processes of medical care for which information is lacking to correlate them with beneficial patient outcomes not be included in a quality assurance program except on a special study basis, even though the processes constitute usual and customary practices.

Once it is under way, a quality assurance program can add data from ambulatory care and disease-specific studies of the outcomes of care for patients who have visited a health provider as well as data on the health status of people who have not. An example of such a study would be a population survey that determined the characteristics as rates for those in the population who have hypertension but have either not been diagnosed or, having been diagnosed, have not been properly treated.

Summary of conclusions about methods

What conclusions can be drawn from the current knowledge about methods that can be useful for both the initiation of PSROs and the longer range developments of quality assurance mechanisms? Informed opinions differ on the validity and reliability of any one method of collecting and evaluating data for a quality assurance program but a few propositions are attracting some consensus.

• --Any method should be oriented, to the extent possible, to establishing and utilizing reliable relationships between structural characteristics and processes of care, and the achievement of desired outcomes of that care. Establishing these relationships is not so much a quality assurance method as it is a clinical research technique that should form a logical underpinning for application in any method. The PSRO program should and can be oriented around this fundamental emphasis.
• --The use of explicitly predetermined criteria should be avoided unless they are grounded in objective studies showing beneficial patient outcome. The use of explicit criteria based on norms of current practice or even consensus of experts, without reference to patient outcome, can stifle innovation, increase costs, increase demand for scarce manpower resources, and increase the risk of harm to the patient from the care itself.
• --The importance of the medical record in organizing decision-making processes is gradually being recognized. However, medical records have been arranged more for the convenience of those entering the data than of those using the record for care. Increasing complexity of records is forcing their reorganization to simplify and expedite the clinical interpretation of the information and reflect the clinical logic of the physician. The problem-oriented record represents an important example of this development.
• --The use of any method poses cost-benefit questions. Some methods, such as observing practitioners at work, are valuable but very expensive. These trade-offs must be openly discussed and costs and benefits analyzed before major commitments are made to specific methods.
• --None of these methods is definitive and none is fully proved. Quality assurance on a major scale is a new enterprise and experimentation should be the rule rather than the exception. In spite of the lack of a definitive method, there is sufficient information available about certain approaches and methods to establish a first-generation quality assurance system as described above.

The committee recommends that carefully designed research studies be conducted to relate process and structural factors causally to outcomes. Until such time as this research provides reliable information, quality assurance should be based on a judicious combination of structural, process, and outcome factors. Experimentation in methods should be the rule and not the exception.

In sum, concerning methods, the committee recommends that a quality assurance program should build initially on the “Uniform Hospital Discharge Abstract Basic Data Set.” As the program develops beyond this initial data set, both process and outcome measurements should be used, attempting always to relate care to outcomes and employing only those process measures for which research has demonstrated a definite correlation with patient outcome. Any quality assurance program, including PSRO, should expand to cover ambulatory care as soon as possible.

Standards^*

Whatever methods are used by a PSRO or future quality assurance program, some standards will have to be employed against which to measure provider performance. There formulation is certain to create controversy. Questions of uniformity and stringency arise, as well as the questions of who will formulate the standards and for what purpose. Nevertheless, a few principles seem incontrovertible.

Uniformity of standards is a desirable principle. The patient's right to expect good treatment wherever he lives would be sacrificed if a provider in one state were held to different standards than one similarly situated in another state. There are, however, practical justifications for deviations from uniform standards. Ability to meet uniform standards may be beyond the control of the individual practitioner. An isolated rural family practitioner with a heavy patient load may not be able to follow all standards that would apply to an urban practitioner with a normal work load. Rigid adherence to uniform standards in this rural situation could result in denying some patients access to any care at all. Other departures from standards may be justified in terms of different patient characteristics, such as a tendency not to follow medical instructions, poor nutritional habits, or drug abuse. Deviations from standards should not, however, be made on a subjective basis, such as the practitioner's relative competence or lack of knowledge, but should reflect differences in available resources and population characteristics. Although medical care standards should be related to the desired changes in health outcome of the population being served, it must be recognized that medical care is often not the most important variable affecting health status.

Another principle is that providers subject to review should not be wholly responsible for formulating the criteria by which they will be reviewed. Epidemiological and statistical skills should also be employed in setting criteria, with providers participating in the establishment of standards, particularly at the local level.

The technology of quality assurance is in its infancy. The formulation and application of standards will have to be an evolutionary process. One way to encourage evolution is to provide some flexibility permitting local initiative in the initial establishment of most standards. The PSRO legislation provides this flexibility. Local initiative must be balanced, however, by the need for a reasonable degree of uniformity to ensure the equitable treatment of providers. Any future legislation enabling quality assurance must not only permit but also require some local setting of standards and the constant reformulation of standards on the basis of both field experience and the results of evaluative research. (The Federal function in collecting and analyzing this experience is discussed in more detail under the recommendation for Federal management. See p. 40.)

The committee recommends that standards be formulated initially at the local review level, and the Federal activity focus on the review, comparison and approval of locally developed standards. After some experience is gained with standards and more knowledge is acquired concerning the relationship of care processes to outcomes, selected standards should be promulgated as national guidelines, but not with the regulatory force that would preclude further local innovation. Both the formulation and application of standards should be subject to periodic modification as the methods evolve to assure quality of health care.

The Costs of Quality Assurance

Some activities now exist to enhance quality in the health care delivery system. But the establishment of a quality assurance program, such as PSRO, that requires the participation of essentially all providers of care will add substantially to current expenditures for the effort. The increases can be attributed to acquiring data, establishing quality assurance systems, maintaining them, and possibly to effects on the provision of care itself, which may become more costly as a result of imposing a quality assurance system. Some additional costs can be justified in terms of the value received from the assurance of the quality of care. Whether these costs are offset by decreases in the overall costs of care as a byproduct of the quality assurance program remains to be seen. In any event, the committee assumes that there will be a decreasing rate of return from the use of more elaborate and expensive quality assurance methods. Therefore, continuous attention should be given to the analysis of costs of alternative methods compared with the effectiveness of the method for enhancing quality of care.

Quality assurance costs. Most of the studies and estimates on health care data suggest that costs of quality assurance will be high. In the New Mexico and San Joaquin medical care foundations, which use complex data for review of hospital and ambulatory care, preliminary estimates of the costs are two to five percent of the overall care cost. These costs, however, also cover the collection of some information that would be needed in any process of third-party claims review.

Because of the methods of cost accounting currently used in the health care field, it is very difficult to develop reliable cost estimates. But the potential for substantial data collection and analysis costs argues for maximum use of sampling techniques which will minimize not only the burden on providers but the purchasers of care as well.

The minimal requirements for quality assurance will be substantial in quantity and cost. The apparently inexorable expansion of demands for data will only increase that cost.

The committee recommends that health information systems be designed so that quality assurance data are compatible with those collected for other purposes in order to restrain the normal tendency of bureaucracies to collect more data than can be justified by a clear expression of need or use; and that sampling techniques be used wherever they are appropriate to the purpose.

Costs of establishing quality assurance systems. Costs of starting a quality assurance system (planning, design, development and installation of hardware and software, and initial staffing) can be covered at the outset by either of two sources but the eventual result is the same: the consumer-taxpayer pays the bill. If providers, who are required to participate in quality assurance programs, also have to bear the cost of those programs, they will pass it on to the consumers. Since consumers of health care services include government, and government's revenue comes from taxation, the taxpayer would support the government's consumer role as well as his own. If government assumes all or part of the start-up costs for quality assurance, as is the case with PSROs, the money could come from general tax revenues or Social Security tax funds. Whichever formula is applied will occasion some redistribution of income among the population. Thorny as the issue is, it is not an integral part of this committee's considerations. We merely emphasize that it is an important issue, and point out that general revenues come from a more progressive tax base than does the Social Security trust fund.

The committee recommends that the Federal government bear the total costs of establishing quality assurance programs by nonprofit organizations with assurances that local sources will be able to bear their share of the costs of maintenance.

The costs of maintenance. The line between start-up and maintenance expenses is not clearcut but is sometimes drawn arbitrarily by legislation as a means of limiting the Federal role. The PSRO legislation is not clear when, if at all, a direct Federal subsidy will cease. The committee believes that users of data from a quality assurance program, such as providers, third-party payers, educational programs, and planning agencies, will have a greater motivation to participate in the implementation of that system if they are asked to participate in its operating costs.

The committee recommends that all users of data from the quality assurance program should participate substantially in the operating costs of the program.

The overall costs of health care. There is an interdependent relationship in health care among the factors of access, cost, and quality. Changing one usually affects the other two. In a real world of constrained resources, a reasonable balance has to be struck among the three factors. The difficulties in achieving that balance are substantial.

It is often assumed that if a quality assurance program has a lean administrative apparatus and modest data requirements it will save resources by controlling overutilization. But this assumption may be valid only if the system is concerned solely with overutilization. If the system were concerned with both under- and overutilization, it might increase costs. If the quality assurance program relies heavily on process criteria, the standards for practice may require more procedures or units of care than current practice, especially if many new developments in the technology of medicine are accepted uncritically; thus compliance with standards could necessitate more costly care than good medical practice would dictate.

A balance among access, cost, and quality also can be thrown out of kilter by a wrongly placed concern about the quality factor. Quality assurance systems can make practitioners so concerned, even preoccupied, with the criteria of care that have been set that they practice defensive medicine or overtreat their patients--a response that diminishes treatment resources for the rest of the population. If a quality assurance system is to minimize unnecessary increases in costs of care, it must be designed for the benefit of the population as a whole, not only the population receiving health care.

Improving Performance: Education and Sanctions

A function of quality assurance should be to change both provider and consumer behavior, using a minimum of coercion and punishment. Education of the provider is the keystone in achieving the quality assurance goals agreed on by this committee. The basis of any set of sanctions against providers must be the transformation of behavior toward improved quality.

Inadequate performance by a provider can be more than a matter of individual error. There also can be failure in the health care system as well, in which case system changes can improve performance. Often it is assumed that inadequate performance on the part of practitioners is due to the actions of just a few. The corollary then is that regulation of performance can be expediently accomplished by “rascal-hunting.” While it is undoubtedly true that some practitioners of health care are error-prone and incompetent, it is also true that the health care system can enhance rather than diminish error by even the most well-intentioned. This concept lies at the root of the committee's recommendations in this section.

To the maximum extent feasible, the quality assurance system should serve as an educational device to improve provider performance, although state licensing bodies and existing continuing education programs will continue to play a role. The purpose of quality assurance is the elevation of performance of all providers, not the detection and punishment of a few offenders or incompetents.

The failure of an organization or institution, such as a hospital, to meet the requirements of quality assurance can be due to poor management, insufficient feedback, and the like. In these circumstances, the isolation of individual failure may not only be difficult but also inappropriate and misleading. Failure of the health system may be as much a factor in providing poor health care as individual failure of health professionals. There also must be sanctions designed to change the behavior of the system, as distinct from the behavior of individuals in it. These sanctions might include the imposing of technical assistance, close monitoring of performance for a probationary period, or reducing reimbursement to an institution under Medicare and Medicaid, although we recognize that the current law requires an “all-or-none” response.

Positive incentives should be used as much as possible. Methods must be developed to reward satisfactory--and more important, exemplary--performance. This effort can be enhanced by publicizing the results of quality assurance services, awarding special commendations, and offering merit awards for innovation and high quality performance.

When sanctions must be applied, they should be appropriately selected from a graduated schedule of possible actions. There is sufficient evidence--from nursing home regulation or physician licensure, for example--that when only a severe sanction is available, its imposition is rare. While not wishing to encourage frequent imposition of sanctions, the committee believes that a set of sanctions graded in severity will provide a more flexible means of securing provider compliance. Examples of possible sanctions include mandatory continuing education, dissemination of information about performance, fiscal penalties (particularly in the form of reduced reimbursement, either per procedure or against total reimbursement payable), restrictions on practice, and suspension of licensure.

Provider profiles developed through quality assurance should be made available to appropriate state licensure bodies and specialty boards. It has been established that state licensure bodies under provider control seldom initiate disciplinary action on the grounds of medical incompetence.^* This is due largely to the fact that licensure statutes, which were enacted at the behest of providers, do not include either occasional or recurrent incompetence as a basis for discipline. Nonetheless, action against a provider's license could serve as a potentially effective sanction. For specialty boards, evidence of continued failure of any provider to meet quality assurance standards should be the basis for withdrawal or modification of specialty qualifications.

A substantial amount of information about provider performance will be amassed by quality assurance systems. This information, in statistically aggregated form and checked for validity and reliability, should be digested and disseminated to the public in a manner reasonably calculated to be constructive and understandable. This should help improve choices by consumers and provide incentives for improved practitioner performance.

But there are a number of precautions to be recognized in implementing public information about provider performance:

• --The relative crudeness of quality assurance methods implies that health care institutions should be the initial subjects of information disclosure. Attribution of error among individual practitioners is often difficult and could be misleading. And for diseases of infrequent incidence, individual practitioners will not process enough patients to enable an assessment of their performance. However, where a valid basis does exist for describing facts relevant to the performance of individual practitioners, the committee believes that dissemination of information identifying practitioners by name is justified.
• --Education and changes in behavior being the prime objectives of quality assurance, information about provider performance should contribute to those objectives. Hence, in addition to public dissemination, and in most cases prior to it, providers should be given information about performance.
• --Any provider named in information planned for public dissemination should be given a reasonable period of time to review the information and offer suggestions for correction and explanation. All information to be disseminated should be placed in context. Public dissemination of information about individual practitioners' performance should be applied only after less stringent measures fail to improve performance.
• --Information to be publicly disseminated should be released in sequence according to its potential for individual onus. The first to be released would be aggregate statistical data describing the frequency of events an their outcomes by institutions and services; next would be data relating to specific health care processes and their objectively established efficacy; and finally, as a last resort, data about the performance of individual practitioners.
• --Information designed for the public should be readily accessible to the public and not buried in obscure reports. Dissemination of it should be handled by the officially designated quality review agency in an authorized report available to anyone.

Management and Control

Management of a quality assurance program is not necessarily the same as control of it, the latter entailing formulation of policy that is implemented by management. The PSRO legislation emphasizes the role of physicians in both functions. The committee believes, however, that providers and consumers both are concerned with quality assurance. The relative participation and responsibilities of each must be explored, as must the Federal role in administration.

The following discussion and recommendations address the degree of professional expertise required for participation at the several levels of the quality assurance system--the initial review level and the supervisory level--and the appropriate participation of consumers in the management and control of the program.

Expertise: review level. At the initial review level, expertise is most important. It is possible for consumers^* to understand the issues in quality assurance but the judgments to be made require both technical knowledge and professional commentary. This is not to say that the initial review process must be the exclusive province of the professional. Consumers can play at least two important roles at this level. The history of self-regulation in all fields suggests that there is an inherent constraint, arising out of identification with one's peers, upon members of a profession who review the performance of a fellow member. The presence of consumers during the review process, far from compromising the expertise of the professional, can enhance it by fostering objectivity.

The consumer's other function is self-education. Patients who have little knowledge of health care and its limitations expect far too much of the provider. They can react with hostility when the healing arts don't heal. But it is in the quality assurance process that the capacities and limits of health care are best revealed. Consequently, participation by consumers in quality assurance, particularly perceptive consumers who play leadership roles in the community, will ultimately help to close the gap between public expectations and provider performance.

The committee recognizes that consumer participation may be more effective for some questions than others, particularly at the beginning of a quality assurance program. There are four general categories of inquiry that will arise at various levels of quality assurance review.

The first is whether the given process of health care is one of established efficacy. The second is whether the care provided by a given practitioner to a given patient meets the standards of quality. The third is the performance of certain providers (individuals and institutions) compared with others. The fourth, based on information obtained in the other three, is to decide on an optimal allocation of resources.

In the committee's view, consumer participation is more likely to be appropriate and effective in the proportions illustrated in Figure 2.

Expertise: supervisory level. The committee's examination has been limited to publicly authorized quality assurance systems, such as PSROs, in which it is understood that nonproviders will assume some supervisory responsibilities in the inevitable bureaucracies. But, it is essential that providers seek and be accepted for supervisory positions so that their expertise will be available to the government in its administration of quality assurance programs.

Control. In its technical definition, control refers to the formal composition of the governing body of a legal entity. For example, comprehensive health planning agencies are putatively controlled by nonproviders because the enabling legislation mandates their majority. But because of the expertise, interest, and resolve of providers, the effective control of many CHP agencies rests with providers.

In the case of quality assurance, control of the program at the Federal level will ultimately rest with nonproviders through the government. Control is exercised more directly at the bureaucratic level by the nonproviders who dominate HEW. The remaining question is the form of control at the intermediate supervisory levels and at the local review level.

At the local review level, the importance of expertise makes it necessary to place majority control with providers. Consumers should constitute less than a majority for reasons that already have been stated.

At the supervisory level, foreseen as state or regional advisory councils, consumers should participate extensively. Whether they should control the entity should be the subject of experimentation. Enabling legislation should not require either provider or consumer control but should permit the development of both forms for comparison as to effectiveness. The same opportunity to experiment with various forms should be permitted any Federal advisory body.

Management. Effective management is not usually related to whether the manager is a provider or consumer. This implies that the most skilled managers should be sought for a quality assurance program irrespective of their status. Nevertheless, managers with experience as providers are possibly the most suitable at the review level where frequent interaction with other providers is essential. At the supervisory level, the situation is less clear and the probable result will be a mixture.

Figure 2

Relative Roles of Providers and Consumers in Review of Care and Supervision of Quality Assurance Systems Level of Participation

The committee recommends that the review process at the local level be controlled and preferably managed by providers with a minority of consumers participating. At the supervisory and advisory levels, control should rest variously with providers and consumers, and management should include nonproviders, such as statisticians, epidemiologists, and systems analysts.

Quality assurance is sufficiently important that the fragmentation characteristic of public administration of health programs should be minimized. This can be achieved in a number of ways; the committee offers only a few general recommendations:

• --A single Federal agency should have the full responsibility of supporting and monitoring local quality assurance programs. The National Center for Health Statistics should have responsibility for determining uniform terms, definitions, classifications and standards for data processing functions. Focus of Federal responsibility will ensure that the agency is visible and responsible to providers, which can be enhanced by having the agency report directly to the Assistant Secretary for Health, HEW.

Many problems can be avoided by focusing Federal responsibilities in a single agency. The current confusion surrounding the administration of the PSRO programs at the Federal level illustrates the importance of this point. The Federal health bureaucracy is fragmented, and it is unclear who bears the responsibility for the administration of quality assurance programs. There is an office charged with responsibility for the PSRO programs but it is in jurisdictional conflict with other agencies that have a role in quality assurance activities. The structure makes unclear to providers the source of rules and regulations and technical assistance.

• --In order to ensure the pertinence and availability of research and development findings, the agency responsible for quality assurance should have the authority to expend funds for applied research and development in quality assurance methods and practices. The evaluation of programs should be shared between the bureau having operational responsibility and another evaluation function located outside the operating level. Other agencies should be mandated to fund basic research related to quality assurance.

One of the keys to the success of quality assurance is a research program to help develop the technology of quality assurance. If one agency administers a program and another administers research and development funds, it is unlikely that the research results will meet all the needs of the administering agency. This problem can be addressed by fusing the quality assurance research and development responsibility with responsibility for its administration.

The evaluation of the quality assurance program should be carried out both by the operating bureau for those operational matters where a direct feedback to management is desirable and by an evaluation function at a different administrative level to examine those issues which relate to the basic nature of the program. By this split of the evaluative function, both ongoing managerial response and objectivity concerning major program changes should be served.

Some additional responsibility to support fundamental research into new methods of quality assurance should be given to an agency that is not responsible for day-to-day operational problems. The Bureau of Health Services Research and Evaluation would seem to be an appropriate location for this fundamental research support.

• --The Federal agency with the responsibility for administration of quality assurance should also possess the authority to determine the needs for data about provider performance and control application of that data to quality assurance.

The committee has recommended that the National Center for Health Statistics be designated to collect and interpret data with respect to the delivery of health care services. Having no operational authority, NCHS should be responsive to the requirements of the quality assurance program. Additionally, NCHS should be governed by the decisions of the quality assurance agency with respect to the use of the data bearing on that aspect of health care.

• --The agency responsible for quality assurance at the Federal level should have the power to assure that policies for payment to providers under all Federal health care financing programs support the actions of quality assurance programs in administering sanctions.

To make effective the sanctions that the quality assurance programs choose to administer, they must have the authority to impose fiscal sanctions. The Social Security Administration, or any successor agency for financing health care, should accept, implement, and, if necessary, delegate authority for any fiscal sanctions imposed by the quality assurance agency.

• --A Federal clearinghouse should be established for (1) information related to standards for quality assurance and (2) analyses and information respecting provider performances as measured by quality assurance systems, other statistical data, and special studies. This function should be assigned to the Federal bureau charged with responsibility for quality assurance programs rather than those agencies concerned primarily with the financing of services.

The committee has recommended that standards for quality assurance be formulated initially at the local level. This implies a passive role for the Federal government in the early stages, one of monitoring performance of local review agencies. Data needs at the various levels of the quality assurance system will vary, and each level will wish to abstract data as appropriate. The performance of quality assurance at the local level will require retrievability of data at that level. Data are also needed at the national level in order to facilitate comparisons and contrasts across the country. This is particularly true of information about standards.

There are at least three types of information that should be centralized. Because the development and application of quality assurance standards are decentralized, each local review unit should forward information about local standards to a central repository for comparison and analysis. Also, aggregated statistical data on provider performance within regions across the country should be centralized, at least on a sample basis, so that comparisons can be made between provider performance in various regions. And the central repository should serve as a clearinghouse for literature, studies and analyses on the quality of care in this country or elsewhere. At present, if a group of providers in one part of the country wishes to establish a quality assurance program, they must rely largely on their own ability to learn about other experiences.

Research, Development, and Evaluation

By means of the PSRO program, this nation is beginning a major innovative social experiment that the committee would like to see committed to a goal of improving the effectiveness of health care. Little is known about the varying quality of provider performance and its relation to this goal or the appropriate methods to assess that performance. The committee has noted earlier that the quality of care has been a matter of public and professional concern for decades but that only in the past few years has sufficient public interest developed to establish a formal program for quality assurance. In part, this interest arises from the expansion of the governmental role in financing health care. But it also manifests an increasing conviction among providers and consumers that a better job must be done in health care. The effectiveness of present methods for quality assurance are not sufficient. A substantial continuing program of research is needed. Results of that research must be rigorously evaluated before they are deployed in the operation of the quality assurance system. In addition, every quality assurance system should have an evaluation design available--and resources to conduct the evaluation--before it becomes operational. Every local quality assurance program should have a minimal level of evaluation activity that feeds back to management of the system. But recognizing that prospective evaluations are costly in time and money, some programs should have more complex evaluation protocols to establish definitively the value of particular techniques. The overall program evaluation responsibility should be assigned to a group that is not directly responsible for operation of the quality assurance program in order to assure objectivity. We also recognize that sufficient evaluation expertise does not exist currently and needs to be developed.

Research requirements go beyond the methods of quality assurance. There is need for investigation of provider and consumer behavior, alteration of which is part of quality advancement. Ways must be found to encourage innovation in quality assurance methods. Special funds could help programs undertake and analyze promising projects, such as a survey of the health status and health needs of that portion of a population that was not utilizing health services. Project grants for these purposes should be made available in addition to sums paid to quality assurance programs for start-up and maintenance costs.

The committee recommends that a substantial research and development program be launched to augment and improve the technology of quality assurance, including data collection and analysis, and that, before quality assurance systems become operational, designs and baseline data for their evaluation will be developed and ready for implementation.

Some plans for evaluation must be developed for the nation as a whole for those aspects which need larger numbers than are available locally. Moreover, provision should be made in the design of any quality assurance system for the systematic feedback of evaluation results and field experience to facilitate the periodic re-examination and retooling of quality assurance programs. In addition, R |andsymbol| D programs should be developed to design methods of altering provider and consumer behavior, and for innovative quality assurance projects.