U.S. flag

An official website of the United States government

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Hughes RG, editor. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville (MD): Agency for Healthcare Research and Quality (US); 2008 Apr.

Cover of Patient Safety and Quality

Patient Safety and Quality: An Evidence-Based Handbook for Nurses.

Show details

Chapter 8Health Services Research: Scope and Significance

; .

Author Information and Affiliations

Background

The provision of high-quality, affordable, health care services is an increasingly difficult challenge. Due to the complexities of health care services and systems, investigating and interpreting the use, costs, quality, accessibility, delivery, organization, financing, and outcomes of health care services is key to informing government officials, insurers, providers, consumers, and others making decisions about health-related issues. Health services researchers examine the access to care, health care costs and processes, and the outcomes of health services for individuals and populations.

The field of health services research (HSR) is relied on by decisionmakers and the public to be the primary source of information on how well health systems in the United States and other countries are meeting this challenge. The “goal of HSR is to provide information that will eventually lead to improvements in the health of the citizenry.”1 Drawing on theories, knowledge, and methods from a range of disciplines,2 HSR is a multidisciplinary field that moves beyond basic and applied research, drawing on all the health professions and on many academic disciplines, including biostatistics, epidemiology, health economics, medicine, nursing, operations research, psychology, and sociology.3

In 1979, the Institute of Medicine defined HSR as “inquiry to produce knowledge about the structure, processes, or effects of personal health services”4 (p. 14). This was expanded upon in 2002 by AcademyHealth, the professional organization of the HSR field, with the following definition, which broadly describes the scope of HSR:

Health services research is the multidisciplinary field of scientific investigation that studies how social factors, financing systems, organizational structures and processes, health technologies, and personal behaviors affect access to health care, the quality and cost of health care, and ultimately our health and well-being. Its research domains are individuals, families, organizations, institutions, communities, and populations.5

More specifically, HSR informs and evaluates innovations in health policy. These include changes in Medicare and Medicaid coverage, disparities in access and utilization of care, innovations in private health insurance (e.g., consumer-directed health plans), and trends among those without health insurance.6–10 The health care industry continues to change, and HSR examines the impact of organizational changes on access to care, quality, and efficiency (e.g., growth in for-profit hospital systems). As new diagnostic and treatment technologies are introduced, HSR examines their impact on patient outcomes of care and health care costs.

The definition of HSR also highlights the importance of examining the contribution of services to the health of individuals and broader populations. HSR applied at the population level is particularly important in understanding health system performance and the impact of health policy on the public’s health. In the United States, the National Healthcare Quality Report,11 National Healthcare Disparities Report,12 and Healthy People Year 201013 exemplify our capacity for monitoring quality and assessing change. These reports tell us that the American quality of care is inconsistent and could be substantially improved. The associated cost of health care services is monitored by the Centers for Medicare & Medicaid Services (CMS). CMS reports tell us that American health care is the most expensive in the world, consuming approximately 16 percent of America’s gross domestic product.14

Beyond health policy, HSR examines the process of care and the interactions of patients and providers. For example, HSR methods have been developed to describe doctor-patient communication patterns and examine their impact on patient adherence, satisfaction, and outcomes of care.15–17

Advances in HSR measurement methodologies have made possible policy innovations. Prospective payment of hospitals, nursing homes, and home health care by Medicare became possible with the development of robust case-mix measurement systems.18 CMS was able to initiate a pay-for-performance demonstration, rewarding hospitals with better quality performance, using valid and robust measures of quality.14 Innovations in health care policy are frequently made possible by advances in measurement of indicators of health system performance.

History of Health Services Research

The history of HSR is generally considered to have begun in the 1950s and 1960s with the first funding of grants for health services research focused on the impact of hospital organizations.19, 20 On the contrary, HSR began with Florence Nightingale when she collected and analyzed data as the basis for improving the quality of patient care and outcomes.21 Also significant in the history of HSR was the concern raised about the distribution, quality, and cost of care in the late 1920s that led to one of the first U.S. efforts to examine the need for medical services and their costs, undertaken in 1927 by the Committee on the Costs of Medical Care.22 The committee published a series of 28 reports and recommendations that have had a significant impact on how medical care is organized and delivered in the United States.23 Other key reports of historical importance to HSR were, for example, the national health survey in 1935–1936 by the Public Health Service, the inventory of the nation’s hospitals by the American Hospital Association’s Commission on Hospital Care in 1944, and studies by the American Hospital Association’s Commission on Chronic Illness on the prevalence and prevention of chronic illness in the community.23

In 1968, the National Center for Health Services Research and Development was established as part of the U.S. Public Health Services to address concerns with access to health services, quality of care, and costs. The Center funded demonstration projects to measure quality and investigator-initiated research grants. In 1989, Congress created the Agency for Health Care Policy and Research and broadened its mission to focus attention on variations in medical practice, patient outcomes of care, and the dissemination of evidence-based guidelines for the treatment of common disorders. Later Congress reauthorized and renamed the agency, Agency for Healthcare Research and Quality (AHRQ). AHRQ provides Federal leadership for the field, investing in methods for quality measurement, development of patient safety methods, and health information technology (e.g., electronic health records and decision support systems).

The Federal role in HSR has expanded over time, and investments in HSR are made by multiple Federal agencies. In addition to AHRQ, the U.S. Department of Veterans Affairs, Centers for Disease Control and Prevention, the National Institutes of Health, CMS, and other Federal agencies fund HSR. The diversification of funding comes, in part, from the recognition that HSR is important in managing health care systems, such as the Veterans Health Administration, and provides essential information on the translation of scientific discoveries into clinical practice in American communities, such as those funded by National Institutes of Health. It is estimated that total Federal funding of HSR was $1.5 billion in 2003, of which AHRQ was responsible for approximately 20 percent.24

Private funding of HSR has also grown over time. Funding by private foundations has a significant role and complements Federal funding. Among the many foundations funding HSR are the Robert Wood Johnson Foundation, Commonwealth Fund, Kaiser Family Foundation, Kellogg Foundation, and Hartford Foundation. Other private funding sources include the health care industry, for example, pharmaceutical companies, health insurers, and health care systems.

Goals for Health Services and Patient Outcomes

The goal of health services is to protect and improve the health of individuals and populations. In a landmark 2001 report, Crossing the Quality Chasm: A New Health System for the 21st Century,25 the Institute of Medicine (IOM) of the National Academy of Sciences proposed that the goals for health services should include six critical elements:

  1. Patient Safety: Patients should not be harmed by health care services that are intended to help them. The IOM report, To Err Is Human,26 found that between 46,000 and 98,000 Americans were dying in hospitals each year due to medical errors. Subsequent research has found medical errors common across all health care settings. The problem is not due to the lack of dedication to quality care by health professionals, but due to the lack of systems that prevent errors from occurring and/or prevent medical errors from reaching the patient.
  2. Effectiveness: Effective care is based on scientific evidence that treatment will increase the likelihood of desired health outcomes. Evidence comes from laboratory experiments, clinical research (usually randomized controlled trials), epidemiological studies, and outcomes research. The availability and strength of evidence varies by disorder and treatment.
  3. Timeliness: Seeking and receiving health care is frequently associated with delays in obtaining an appointment and waiting in emergency rooms and doctors’ offices. Failure to provide timely care can deny people critically needed services or allow health conditions to progress and outcomes to worsen. Health care needs to be organized to meet the needs of patients in a timely manner.
  4. Patient Centered: Patient-centered care recognizes that listening to the patient’s needs, values, and preferences is essential to providing high-quality care. Health care services should be personalized for each patient, care should be coordinated, family and friends on whom the patient relies should be involved, and care should provide physical comfort and emotional support.
  5. Efficiency: The U.S. health care system is the most expensive in the world, yet there is consistent evidence that the United States does not produce the best health outcomes27–30 or the highest levels of satisfaction.31 The goal is to continually identify waste and inefficiency in the provision of health care services and eliminate them.
  6. Equity: The health care system should benefit all people. The evidence is strong and convincing that the current system fails to accomplish this goal. The IOM report, Unequal Treatment,32 documented pervasive differences in the care received by racial and ethnic minorities. The findings were that racial and ethnic minorities are receiving poorer quality of care than the majority population, even after accounting for differences in access to health services.

Crossing the Quality Chasm concludes that for the American health care system to attain these goals, transformational changes are needed.25 The field of HSR provides the measurement tools by which progress toward these goals is assessed, as seen in the National Healthcare Quality Report.11 Equally important, health services researchers are developing and evaluating innovative approaches to improve quality of care, involving innovations in organization, financing, use of technology, and roles of health professionals.

Evaluating the Quality of Health Care

HSR evaluation of quality of care has proven to be an inexact science and complex, even though its definition is relatively simple: “Quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”1 This definition draws attention to the importance of the application of current professional knowledge in the diagnostic and treatment processes of health care. The goal of quality care is to increase the likelihood of achieving desired health outcomes, as expressed by the patient.

The complexity in measuring quality comes from gaps in our knowledge regarding which services, for which patients, will actually improve the likelihood of desired health outcomes. Also, patients need not have the same desired health outcomes and therefore might not receive the same care for an identical health problem, further complicating the measurement of quality of care. Quality measurement has advanced substantially, but it remains early in its development.

The conceptual framework widely applied in evaluating quality comes from years of research and the insightful analysis of Avedis Donabedian.33 He formalized the conceptual model for describing, analyzing, and evaluating the quality of care using three dimensions: (1) structure, (2) process, and (3) outcome. This model is applied in the evaluation of health services and the accreditation of health care providers and organizations.

Seminal research about variation in the quality of care patients received brought to focus the need to monitor and improve the quality of health care. Wennberg and Gittelsohn34, 35 found wide variation in practice patterns among community physicians, surgical procedures, and hospitals. Brook and colleagues36 found that a small number of physicians were responsible for a large number of improperly administered injections. This was the precursor to research on the appropriateness of procedures and services under specific circumstances36, 37 as well as the development of practice guidelines and standards for quality care.38 Yet the challenge of research on variations in care is the implication of the inappropriateness of care. The challenge is determining whether there is a direct relationship between rates of utilization, variations in appropriateness, and quality of care.

One of the challenges in understanding quality, how to measure it, and how to improve it is the influence of physical, socioeconomic, and work environments. Income, race, and gender—as well as individuals within society and organizations—influence health and risks to health.40 Researchers have found that differences in internal factors, such as collaborative relationships with physicians, decentralized clinical decisionmaking, and positive administrative support, impact nurse and patient outcomes41, 42 and the quality and safety of care.43 Differences in external factors, such as insurance and geographic location, can influence access to available health care professionals and resources, what type of care is afforded patients, and the impact of care on patients. The structure, process, and outcome dimensions of quality are influenced by both internal and external factors.

Structure of Health Care

The structure of health care broadly includes the facilities (e.g., hospitals and clinics), personnel (e.g., number of nurses and physicians), and technology that create the capacity to provide health services. Structural characteristics are expected to influence the quality of health care services. One component in the accreditation of health care facilities (e.g., hospitals, nursing homes) is the review of the adequacy of structural characteristics, including staffing, on-call resources, technology, and support services (laboratory, pharmacy, radiology). The structural resources of health care facilities and organizations are the foundation upon which quality health care services are provided.

Process of Care

The interactions between the health care providers and patients over time comprise the process of health care. The process of care may be examined from multiple perspectives: the sequence of services received over time, the relationship of health services to a specific patient complaint or diagnosis, and the numbers and types of services received over time or for a specific health problem. Examining the time sequence of health care services provides insights into the timeliness of care, organizational responsiveness, and efficiency. Linking services to a specific patient complaint or diagnosis provides insights into the natural history of problem presentation and the subsequent processes of care, including diagnosis, treatment, management, and recovery. Examining the natural history of a presenting health complaint across patients will reveal variations in patterns of care. For example, presenting complaints for some patients never resolve into a specific diagnosis. An initial diagnosis may change as more information is obtained. Patients may suffer complications in the treatment process. Also, the process of care may provide insights into outcomes of care (e.g., return visit for complications). Generally it is not possible to examine the process of care and determine how fully the patient has recovered prior health status by the end of the episode of treatment. For this reason, special investigations are needed to assess outcomes of care.

Evaluation of the process of care can be done by applying the six goals for health care quality.25 Was the patient’s safety protected (i.e., were there adverse events due to medical errors or errors of omission)? Was care timely and not delayed or denied? Were the diagnosis and treatments provided consistent with scientific evidence and best professional practice? Was the care patient centered? Were services provided efficiently? Was the care provided equitable? Answers to these questions can help us understand if the process of care needs improvement and where quality improvement efforts should be directed.

Outcomes of Care

The value of health care services lies in their capacity to improve health outcomes for individuals and populations. Health outcomes are broadly conceptualized to include clinical measures of disease progression, patient-reported health status or functional status, satisfaction with health status or quality of life, satisfaction with services, and the costs of health services. Historically, quality assessment has emphasized clinical outcomes, for example, disease-specific measures. However, disease-specific measures may not tell us much about how well the patient is able to function and whether or not desired health outcomes have been achieved. To understand the patients’ outcomes, it is necessary to ask patients about their outcomes, including health status, quality of life, and satisfaction with services. HSR has developed valid and robust standardized questionnaires to obtain patient-reported information on these dimensions of health outcomes. As these are more widely applied, we are learning about the extent to which health care services are improving health.

Public Health Perspective on Health Services

Another perspective on health care services comes from the field of public health in which preventive health services are conceptualized at three levels: primary, secondary, and tertiary prevention.44 Primary prevention includes immunizations, healthy lifestyles, and working and living in risk-free environments. Primary prevention seeks to prevent disease or delay its onset. Examples of primary prevention include immunizations against infectious disease; smoking prevention or cessation; and promotion of regular exercise, weight control, and a balanced diet. Secondary prevention includes the range of interventions that can reduce the impact of disease morbidity once it occurs and slow its progression. With the increasing burden of chronic diseases, much of the health care provided is directed at secondary prevention. Tertiary prevention is directed at rehabilitation for disabilities resulting from disease and injury. The goal of tertiary prevention is to return individuals to the highest state of functioning (physical, mental, and social) possible. The public health framework expands the structure, process, and outcome conceptual model by identifying the role and value of health services at three stages: prior to onset of disease, disease management, and disease recovery and rehabilitation.

Methodologies and Data Sources Used in Health Services Research

The interdisciplinary character of HSR draws on methods and data sources common to the many disciplines that form the intellectual underpinnings of the field. This section discusses the measurement of effectiveness and efficacy of health services and some of the methods and data sources used to understand effectiveness. Effectiveness is one of the six goals of health services. Effectiveness is interrelated with the other five goals, and some of these interrelationships are discussed.

Efficacy and Effectiveness

An important distinction is made between efficacy and effectiveness of health services. Efficacy is generally established using randomized controlled trial (RCT) methods to test whether or not clinical interventions make a difference in clinical outcomes. A good example is the series of studies required for Food and Drug Administration approval of a new drug before it is certified as safe and efficacious and allowed to be used in the United States. Efficacy research is generally done with highly select groups of patients where the impact of the drug can be validly measured and results are not confounded by the presence of comorbid conditions and their treatments. The efficacy question is: What impact does a clinical intervention have under ideal conditions?

In contrast, effectiveness research is undertaken in community settings and generally includes the full range of individuals who would be prescribed the clinical intervention. Many of these individuals will have multiple health problems and be taking multiple medications, unlike those who were recruited to the RCT. Effectiveness research is seeking to answer the question: Who will benefit from the clinical intervention among all those people in the community who have a specific health problem(s)?

Both efficacy and effectiveness questions are important. Logically, effectiveness research would be conducted after finding the clinical intervention to be efficacious. However, there are many treatments for which no efficacy information exists; the treatments are accepted as common practice, and it would not be ethical to withhold treatments from a control group in an RCT. As a result, effectiveness research may not have the benefit of efficacy findings.

The routine use of an RCT to evaluate efficacy began in the 1960s and is the accepted procedure for evaluating new medications. However, this standard is not applied across all health care services and treatments. Most surgical procedures are not evaluated using an RCT. Intensive care units have never been evaluated using an RCT, nor are nurse staffing decisions in hospitals or the evaluation of many medical devices. We currently accept different standards of evidence depending on the treatment technology. As a result, the level of evidence guiding clinical and public health decisionmaking varies.

Methods for Effectiveness Research

A variety of methods are used to examine effectiveness of health services. RCT methods are not usually applied in effectiveness research because the intervention being studied has demonstrated efficacy or is acknowledged as accepted clinical practice. When this is true, it would be unethical to randomly assign individuals who would be expected to benefit from the intervention to a control group not receiving an efficacious treatment. We will discuss when RCT methods can be used to test effectiveness and provide several examples. More commonly, effectiveness research uses statistical methods for comparing treatments across nonequivalent groups.

RCT and Policy Research

RCT study methods can be used to compare the effectiveness and costs of services across randomly assigned representative population groups. In an RCT, study participants are randomly assigned to two or more groups to ensure comparability and avoid any selection bias. At least one group receives an intervention (clinical, organizational, and/or financial), and usually one group serves as a control group, receiving a current standard of care, sometimes referred to as “usual care.” Two examples of effectiveness research using an RCT methodology to answer policy questions are described.

Health insurance experiment

Probably the first application of RCT methods in effectiveness research was undertaken in the 1970s as a health insurance experiment. The experiment was designed to test the impact on cost and health outcomes of different levels of insurance deductibles and copayment rates. A total of 3,958 people, ages 14–61, were randomized to a set of insurance plans and followed over 3 to 5 years.45

The economic impact of receiving free care in one plan versus being in a plan requiring payment out-of-pocket of deductibles and co-insurance had the expected impact on utilization. Those paying a share of their medical bills utilized approximately one-third fewer doctor visits and were hospitalized one-third less frequently.

The impact on 10 health measures of free health insurance versus paying a portion of medical care costs out of pocket was evaluated. The findings were that there was largely no effect on health as measured by physical functioning, role functioning, mental health, social contacts, health perceptions, smoking, weight, serum cholesterol, diastolic blood pressure, vision, and risk of dying.46 The exceptions were that individuals with poor vision improved under free care, as did low-income persons with high blood pressure.

Medicare preventive services experiment

A more recent example of RCT methods applied in HSR is the Baltimore Medicare Preventive Services Demonstration. The study evaluated the impact on cost and outcomes of offering a defined preventive services package to Medicare beneficiaries. This was compared to usual Medicare coverage, which paid for few preventive services. The preventive services coverage being evaluated included an annual preventive visit with screening tests and health counseling. The physician could request a preventive followup visit during the year, which would also be covered. Medicare beneficiaries (n = 4,195) were randomized to preventive services (the intervention group) or usual care (the control group). Sixty-three percent of those in the intervention group had at least one preventive visit. Significant differences were found in health outcomes between intervention and control groups. Among the 45 percent with declining health status, as measured by the Quality of Well-Being scale,47 the decline was significantly less in the group offered preventive services. Mortality was also significantly lower in the intervention group. There was no significant impact of preventive services on utilization and cost.48

Comparative Clinical Effectiveness and RCTs

The passage of the Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (MMA) included provisions for the funding of comparative effectiveness studies. AHRQs’ Effective Health Care Program (authorized under MMA Section 1013) informs comparative clinical effectiveness efforts by conducting and supporting research and evidence syntheses on priority topics to CMS.

Comparative effectiveness studies ask the question: Which of the alternative treatments available is best and for whom? Interest in this question reflects how advances in science have provided multiple treatment options for many conditions. Currently, there is no systematic process by which treatment options are compared and matched to the needs of different types of patients. Frequently, patients are started on one treatment and then may be prescribed alternative treatments if they cannot tolerate the treatment or if it is not as effective as expected. RCT methods can be used to evaluate comparative effectiveness of an intervention in treatment and control populations. This is ethical to do when there is no evidence that the treatments are not equivalent.

An example of a comparative effectiveness study using RCT methods is the CATIE study, testing alternative antipsychotic medications in the treatment of schizophrenia. A study of 1,493 persons with schizophrenia compared five of the newer antipsychotic medications (second generation) and also compared them against one of the first-generation antipsychotic medications.49, 50 The findings were surprising to many. The second-generation antipsychotics were no more effective in controlling psychotic symptoms than the first-generation drug. There was one exception, the drug Clozapine.51 Furthermore, second-generation medications showed significant side effects that can affect health outcomes. These included weight gain, metabolic changes, extrapyramidal symptoms, and sedation effects. Each medication showed a somewhat different side-effect risk profile. From a positive perspective, the findings indicated that the clinician and patient can choose any of these medications as first-line treatment except Clozapine, which is generally used for treatment-resistant cases due to more intensive clinical monitoring requirements. The ultimate choice of treatment will depend on the patient’s ability to tolerate side effects that vary by drug.

The conduct of any RCT is resource intensive, requiring the recruitment of participants, and participants must give informed consent to be randomized. The rationale for making this investment may depend on the importance of the policy or practice issue. As shown, RCT methods can be applied to address policy and clinical care concerns with effectiveness. To the extent that the RCT includes a broad cross-section of people who would be affected by a policy or receive a clinical treatment, this methodology provides robust effectiveness findings.

Comparing Effectiveness and Costs Across Nonequivalent Groups

A range of statistical methods can be used to compare nonequivalent groups (i.e., groups receiving different treatments or exposures when there has been no random assignment to ensure comparability of group membership). It is not practical to review all the specific statistical approaches that can be applied. In general, the statistical methods seek to adjust for nonequivalent characteristics between groups that are expected to influence the outcome of interest (i.e., make the comparisons fair). Statistical adjustment for nonequivalent characteristics is referred to as “risk adjustment.” The foundations for risk adjustment come from multiple disciplines. Epidemiologic methods are routinely used to identify and estimate disease and outcomes risk factors. These methods are applicable in comparative effectiveness evaluations.52

Operations research uses methods for creating homogeneous groups predictive of cost or disease outcomes. These methods are used to make fair comparisons across provider practices and health plans and to control the cost of health care. They also have been used in designing payment systems, including diagnostically related groups used in Medicare’s Prospective Payment System to reimburse hospitals for care rendered to Medicare beneficiaries, and resource-based relative value scales used in Medicare’s physician payment system. Diagnostically related groups are used to standardize and rationalize patient care in hospitals—provided largely by nurses and other health professionals—and resource-based relative value scales are used to standardize and rationalize patient care in outpatient settings—care provided largely by physicians and nurse practitioners. Other disciplines also contribute to our understanding of risk factors for the range of health outcomes, including mortality, health and functional status, quality of life, and rehabilitation and return to work

The basic form of a nonequivalent group comparison includes adjusting the outcomes of each group for the risk factors that are known to affect the occurrence and/or severity of the outcomes being evaluated.53 For many disease outcomes, risk factors include demographic characteristics (age, gender), disease-specific risk factors (e.g., health behaviors, environmental exposures, and clinical indicators of risk), and indicators of health status (e.g., presence of comorbid conditions). After adjustment for risks factors, variations in access to care and quality of care (e.g., choice of treatment and adherence to treatment) would be expected to explain the remaining observed variation in outcomes. Ideally, the nonequivalent group comparison makes it possible to compare the effectiveness of alternative treatments and assess the impact of poor access to care. One limitation of this methodology is the limit of current knowledge regarding all relevant disease risk factors. Even when risk factors are known, limits on data availability and accuracy of risk factor measurement have to be considered.

Risk adjustment methods are also used to make cost comparisons across health care providers to determine which providers are more efficient. Instead of adjusting for disease risk factors, adjustments are made for the costliness of the patient mix (case mix) and differences in costs of labor, space, and services in the local area. Comparisons may be made to assess efficiency of providing specific services (e.g., hospitalization, office visit, or laboratory test). These comparisons would use case-mix measures that adjust for the costliness of different mixes of hospital episodes.18 Comparisons of the total cost of care for insured populations would apply case-mix measures that adjust for disease and health factors that affect total cost of care.54

Data Sources for Effectiveness Research

A range of data sources is used in effectiveness research, including administrative and billing data, chart reviews and electronic health records, and survey questionnaires. The following discussion identifies major attributes of each category of data source.

Medical records

Medical records document the patient’s presenting problem or condition, tests and physical exam findings, treatment, and followup care. The medical record is generally the most complete source of clinical information on the patient’s care. However, medical records are generally not structured to ensure the physician or other provider records all relevant information. The completeness of medical record information can vary considerably. If the patient does not return for followup care, the medical record may provide no information on outcomes of care. If a patient sees multiple providers during the course of treatment, each with its own separate medical record, complete information on treatment requires access to all the records. Lack of standardization of medical records also can make abstracting records for research very resource intensive.

Administrative and billing data

Health care providers generally have administrative and billing data systems that capture a limited and consistent set of data on every patient and service provided. These systems uniquely identify the patient and link information on insurance coverage and billing. Each service received by the patient is linked to the patient using a unique patient identifier. Services are identified using accepted codes (e.g., ICD9-CM, CPT), together with date of service, provider identifier, and other relevant information for billing or management reporting. Administrative data make it possible to identify all individual patients seen by a provider and produce a profile of all services received by each patient over any defined time period. Administrative data are comprehensive and the data are generally complete (i.e., no problems with missing data). The primary limitation is the data set collected by administrative systems is very limited and lacks the detail of the medical record.

Administrative data systems can provide some insights into quality and outcomes of care. AHRQ has developed software that provides quality indicators and patient safety measures using one administrative data set, hospital discharge abstracts.55, 56 Utilization-based indicators of outcome include rehospitalization, return to surgery during a hospitalization, and incidence of complications; some systems include information on death. Administrative data can efficiently provide quality and outcomes indicators for defined populations and for health systems. Other applications of administrative data include assessing efficiency, timeliness, and equity. The limitation is that there are many health conditions and health outcomes that cannot currently be measured using administrative data.

Survey questionnaires

Neither the medical record nor the administrative data capture information on the patient’s experience in health or patient-reported outcomes of care. Survey questionnaires are routinely used to obtain information on patient satisfaction in health plans. A widely used example is the Consumer Assessment of Healthcare Providers and Systems or CAHPS.57

Information on the impact of health conditions on health and functional status has to come from the patient. This may be obtained at the time of a visit or hospitalization. However, to assess patient outcomes of care, systematic followup of patients after the completion of treatment is generally required. This can be done using mail questionnaires, telephone interviews, or in-person interviews. The HSR field has developed health-status and quality-of-life measures that can be used no matter what health conditions the patient has.47, 58–60 Numerous condition-specific measures of outcome are also used.53

Effectiveness research relies on a range of data sources. Some are routinely collected in the process of medical care and patient billing. Others may require special data collection, including medical record abstracts to obtain detailed clinical data and survey questionnaires to gain information on the patient’s perspective on treatment and outcomes. Efficient strategies for examining effectiveness may use administrative data to examine a limited set of data on all patients, and a statistically representative sample of patients for in-depth analysis using data from chart abstracts and survey questionnaires.

Using HSR Methods To Improve Clinical Practice

HSR research tools can be applied in clinical settings to improve clinical practice and patient outcomes. These tools are used as part of quality improvement programs in hospitals, clinics, and health plans. Two examples illustrate applications to improve quality-of-care performance.

Evidence-based treatment

For many chronic medical conditions, clinical research has evaluated the efficacy of diagnostic methods and treatment interventions. As a result, evidence-based reviews of research literature can provide a basis for establishing quality-of-care criteria against which to judge current practice. In a national study of quality of medical care, it was found that only 55 percent of patients received evidence-based treatments for common disorders and preventive care.61 The researchers examined treatment for a range of health conditions, using a national sample of medical records abstracts. For each quality criterion, a classification was applied to determine if the quality-of-care deficiency was one of underuse, overuse, or misuse. Greater problems were found with underuse (46 percent) than with overuse (11 percent). Quality of care varied by condition: senile cataracts scored highest, 78 percent of recommended care received, and alcohol dependence scored lowest, 10 percent of recommended care received. Overall, only about half of recommended care was received, frequently due to underuse of services.

Researchers have sought to identify why rates of conformance with evidence-based treatments are low. Frequently cited barriers to evidence-based practice include physician disagreement with the evidence, perception that patients will not accept treatment, low ratings of self-efficacy as a provider of the treatment, and difficulty of integrating the evidence-based treatment into existing practice.62 More needs to be learned how to assist health care providers to overcome barriers to the adoption of evidence-based practices.

The described data sources and methods can be applied in clinical settings to assess conformance to evidence-based quality criteria and provide feedback to clinicians. If electronic health records are available, the feedback and reminders may be directly incorporated into the medical record and seen by the clinician at the time of a visit. Intermountain Health Care utilizes its electronic health records to monitor adherence to evidence-based quality standards and to provide decision support to clinicians when seeing patients. This strategy has contributed to substantial improvements in their quality performance.63

Outcomes management system

In 1988, Paul Ellwood proposed the adoption of outcomes management system (OMS) as a method to build clinical intelligence on “what treatments work, for whom, and under what circumstances.”64 OMS would require linking information on the patient’s experience with outcomes of care and information on diagnosis and treatment that would usually come from the medical record.

In 1991, the Managed Health Care Association, an employer organization, brought together a group of employers and their health plan partners who were interested in testing the OMS concept in health plans.65 To do so would require a set of methods that could be widely applied across health plans with differing information systems. The methodology chosen was for each of 16 health plans to identify all adult enrollees with at least two diagnoses of asthma over the previous 2 years. A stratified sample was chosen with half of the enrollees having more severe asthma (e.g., hospitalization or emergency room visit in the past 2 years) and the other enrollees having less severe asthma (outpatient visits only). Each adult received a questionnaire asking about their asthma treatment and health status. Followup surveys were done in each of 2 successive years to track changes over time.

The findings were compared to national treatment recommendations for adult asthma.66 Across the health plans, 26 percent of severe asthmatics did not have a corticosteroid inhaler, and 42 percent used it daily, as recommended.67 Only 5 percent of patients reported monitoring their asthma using a home peak flow meter. Approximately half of adults with asthma reported having the information they needed to avoid asthma attacks, to take appropriate actions when an asthma flare-up occurs, and to adjust medications when their asthma gets worse. Health plans used the baseline findings to develop quality-improvement interventions, which varied across health plans. Followup surveys of the patient cohort provided feedback to health plans on their success in improving asthma treatment and outcomes over time.

Conclusion

This chapter has provided a definition and history of the field of health services research and discussed how this field is examining quality-of-care issues and seeking to improve quality of care. Comparisons of current practice to evidence-based standards with feedback to clinicians and the integration of patient-reported outcomes are two examples of how HSR tools can be used to provide quality-improvement information for health care organizations. These examples utilize multiple data sources, including medical records, patient surveys, and administrative data. The opportunities for nurse researchers to provide invaluable contributions to the growing field of health services research are innumerable.

References

1.
Institute of Medicine. Improving information services for health services researchers: a report to the National Library of Medicine. Washington, DC: National Academy Press; 1991.
2.
Ginzberg E, editor. Health services research: key to health policy . Cambridge, MA: Harvard University Press; 1991.
3.
Institute of Medicine. Health services research: training and workforce issues . Washington, DC: National Academy Press; 1995.
4.
Institute of Medicine. Report on health services research . Washington, DC: National Academy Press; 1979.
5.
Lohr KN, Steinwachs DM. Health services research: an evolving definition of the field. Health Serv Res. 2002 Feb;37(1):7–9. [PubMed: 11949927]
6.
Altman SH, Reinhardt UE, editors. Strategic choices for a changing health care system . Chicago: Health Administration Press; 1996.
7.
Brown LD. Knowledge and power: health services research as a political resource. In: Ginzberg E, editor. Health services research: key to health policy. Cambridge, MA: Harvard University Press; 1991. pp. 20–45.
8.
DeFriese GH, Ricketts TC III, Stein JS, editors. Methodological advances in health serivces research. Chicago: Health Administration Press; 1989.
9.
Shi L. Health services research methods. Albany, NY: Delmar Publishers, Inc; 1997.
10.
Shortell SM, Reinhardt UE, editors. Improving health policy and management: nine critical research issues for the 1990s. Chicago: Health Administration Press; 1992.
11.
Agency for Healthcare Research and Quality. National health care quality report. Washington, DC: Department of Health and Human Services; 2006.
12.
Agency for Healthcare Research and Quality. National health care disparities report. Washington, DC: Department of Health and Human Services; 2006.
13.
U.S. Department of Health and Human Services. Healthy people 2010. 2nd ed. 1 & 2. Washington, DC: U.S. Government Printing Office; Nov, 2000.
14.
Centers for Medicare and Medicaid Services. Medicare demonstration shows hospital quality of care improves with payments tied to quality. CMS Office of Public Affairs. [Accessed November 14, 2005]. Available at: http://www​.cms.gov/apps​/media/press/release.asp?Counter=1729.
15.
Schmid Mast M, Hall JA, Roter DL. Disentangling physician sex and physician communication style: their effects on patient satisfaction in a virtual medical visit. Patient Educ Couns . 2007 Sep;68(1):16–22. Epub 2007 May 4. [PubMed: 17482418]
16.
Beach MC, Roter DL, Wang NY, et al. Are physicians’ attitudes of respect accurately perceived by patients and associated with more positive communication behaviors? Patient Educ Couns. 2006 Sep;62(3):347–54. [PMC free article: PMC3119350] [PubMed: 16859867]
17.
Johnson RL, Roter D, Powe NR, et al. Patient race/ethnicity and quality of patient-physician communication during medical visits. Am J Public Health. 2004 Dec;94(12):2084–90. [PMC free article: PMC1448596] [PubMed: 15569958]
18.
Averill RF, Goldfield N. Wallingford, CT: 3M Health Information Systems; 2003. All patient refined diagnosis related groups (APR-DRGs) version 20.0: methodology overview.
19.
Emery FE, Trist EL. The causal texture of organizational environments. Hum Relat. 1965;18:21–31.
20.
Terreberry S. The evolution of organizational environments. Adm Sci Q. 1967;12:590–613.
21.
Gill G. Nightingales: the extraordinary upbringing and curious life of Miss Florence Nightingale . New York: Random House; 2004.
22.
Committee on the Costs of Medical Care. The five-year program of the committee on the costs of medical care. Washington, DC: The Committee; 1928. (Publication 1)
23.
Starr P. The social transformation of American medicine: the rise of a sovereign profession and the making of a vast industry . New York: BasicBooks; 1982.
24.
Coalition for Health Services Research. Federal funding for health services research. Washington, DC: AcademyHealth; 2004.
25.
Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century Institute of Medicine . National Academy Press; Washington, DC: 2001.
26.
Kohn LT, Corrigan JM, Donaldson MS, editors. A report of the Committee on Quality of Health Care in America, Institute of Medicine . Washington, DC: National Academy Press; 2000. To err is human: building a safer health system. [PubMed: 25077248]
27.
Anell A, Willis M. International comparison of health care systems using resource proviles. Bull World Health Organ. 2000;78(6):770–8. [PMC free article: PMC2560778] [PubMed: 10916914]
28.
Davis K. International health policy: common problems, alternative strategies. Health Aff. 1999;18(3):135–43. [PubMed: 10388209]
29.
Sochalski J, Aiken LH. Accounting for variation in hospital outcomes: a cross-national study. Health Aff. 1999;18(3):256–9. [PubMed: 10388223]
30.
Starfield B, Shi L. Policy relevant determinants of health: an international perspective. Health Policy. 2002;60(3):201–18. [PubMed: 11965331]
31.
Schoen C, Osborn R, Doty MM, et al. Toward higher-performance health systems: adults’ health care experience in seven countries. Health Aff. 2007;26(6):w717–34. [PubMed: 17978360]
32.
Smedley BD, Stith AY, Nelson AD, editors. Unequal treatment: confronting racial and ethnic disparities in healthcare, Institute of Medicine . Washington, DC: National Academy Press; 2003. [PubMed: 25032386]
33.
Donabedian A. Evaluating the quality of medical care. Milbank Memorial Fund Quarterly. 1966;44:166–203. [PubMed: 5338568]
34.
Wennberg JE, Gittelsohn A. Small area variations in health care delivery. Science. 1973;192(4117):1102–8. [PubMed: 4750608]
35.
Wennberg JE, Gittelsohn A. Variations in medical care among small areas. Scientific American. 1982;246(4):120–34. [PubMed: 7079718]
36.
Brook RH, Chassin MR, Fink A, et al. A method for the detailed assessment of the appropriateness of medical technology. Int J Technol Assess. 1986;2(1):53–63. [PubMed: 10300718]
37.
Kahn KL, Kosecoff J, Chassin MR, et al. Measuring the clinical appropriateness of the use of a procedure: can we do it? Med Care. 1989;26(4):415–22. [PubMed: 3352334]
38.
Brook RH. Practice guidelines and practicing medicine: are they compatible? JAMA. 1989;262(21):3027–30. [PubMed: 2810647]
39.
Chassin MR, Kosecoff J, Park RE, et al. Does inappropriate use explain geographic variations in the use of health services? A study of three procedures. JAMA. 1987;258(18):2533–7. [PubMed: 3312655]
40.
Evans RG, Barer ML, Marmor TR, editors. The determinants of health of populations. New York: Aldine De Gruyter; 1994. Why are some people healthy and others not?
41.
Mark BA, Salyer J, Wan TTH. Professional nursing practice: impact on organizational and patient outcomes. J Nurs Adm. 2003;33:224–34. [PubMed: 12690254]
42.
Aiken LH, Clarke SP, Sloane DM. Hospital staffing, organization, and quality of care: cross-national findings. Nurs Outlook. 2002;50:187–94. [PubMed: 12386653]
43.
Clarke SP, Sloane DM, Aiken LH. Effects of hospital staffing and organizational climate on needlestick injuries to nurses. Am J Public Health. 2002;92:1115–9. [PMC free article: PMC1447200] [PubMed: 12084694]
44.
The future of the public’s health in the 21st century, Institute of Medicine . Washington, DC: National Academy Press; 2003. Committee on Assuring the Health of the Public in the 21st Century.
45.
Newhouse JP, Manning WG, Morris CN, et al. Some interim results from a controlled trial of cost sharing health insurance. N Engl J Med. 1981;305:1501–7. [PubMed: 6795505]
46.
Brook RH, Ware JE, Rogers WH, et al. Does free care improve adult’s health? Results from a randomized controlled trial. N Engl J Med. 1983;309(23):1426–34. [PubMed: 6355851]
47.
McDowell I, Newell C. Measuring health: a guide to rating scales and questionnaires . New York: Oxford University Press; 1987.
48.
Burton LC, Steinwachs DM, German PS, et al. Preventive services for the elderly: would coverage affect utilization and costs under Medicare? Am J Public Health. 1995;85(3):387–91. [PMC free article: PMC1614868] [PubMed: 7892924]
49.
Lieberman JA, Stroup TS, McEvoy JP, et al. Effectiveness of antipsychotic drugs in patients with antipsychotic schizophrenia. N Engl J Med. 2005;353:1209–23. [PubMed: 16172203]
50.
Stroup TS, Lieberman JA, McEvoy JP, et al. Effectiveness of olanzapine, risperidone, and piprasidone in patients with chronic schizophrenia following discontinuation of a previous atypical antipsychotic. Am J Psychiatry. 2006;163:611–22. [PubMed: 16585435]
51.
McEvoy JP, Lieberman JA, Stroup TS, et al. Effectiveness of clozapine versus olanzapine, quetiapine, and risperidone in patients with chronic schizophrenia who did not respond to prior atypical antipsychotic treatment. Am J Psychiatry. 2006;163:600–10. [PubMed: 16585434]
52.
Armenian HK, Shapiro S, editors. Epidemiology and health services. New York and Oxford: Oxford University Press; 1989.
53.
Iezzoni LI. Risk adjustment for measuring health care outcomes . Ann Arbor, MI: Health Administration Press; 1994.
54.
Weiner JP, Starfield BH, Steinwachs DM, et al. Development and application of a population-oriented measure of ambulatory care case-mix. Med Care. 1991;29(5):452–72. [PubMed: 1902278]
55.
McDonald K, Romano P, Geppert J, et al. Rockville, MD: Agency for Healthcare Research and Quality; Aug, 2002. Measures of patient safety based on hospital administrative data—the patient safety indicators (Prepared by the University of California San Francisco-Stanford Evidence-based Practice Center under Contract No 290-97-0013) Technical review 5. AHRQ Publication No 02-0038. [PubMed: 20734521]
56.
Davies SM, Geppert J, McClellan M, et al. Rockville, MD: Agency for Healthcare Research and Quality; May, 2001. Measures of patient safety based on hospital administrative data—the patient safety indicators (Prepared by the University of California San Francisco-Stanford Evidence-based Practice Center under Contract No 290-97-0013) Technical Review 4. AHRQ Publication No 02-0035.
57.
Hargraves JL, Hays RD, Cleary PD. Psychometric properties of the Consumer Assessment of Health Plans Study (CAHPS™) 2.0 adult core survey. Health Serv Res. 2003 Dec;38(6 Pt 1):1509–27. [PMC free article: PMC1360961] [PubMed: 14727785]
58.
Bergner M, Bobbitt RA, Carter WB, et al. The sickness impact profile: development and final revision of a health status measure. Med Care. 1981;19(8):787–805. [PubMed: 7278416]
59.
Shaw JW, Johnson JA, Coons SJ. US valuation of the EQ-5D health states: development and testing of the D1 valuation model. Med Care. 2005 Mar;43(3):203–20. [PubMed: 15725977]
60.
Ware JE, Sherbourne CD. The MOS 36-item short form health status survey (SF-36). I. Conceptual framework and item selection. Med Care . 1992;30:MS253–65. [PubMed: 1593914]
61.
McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348(26):2635–45. [PubMed: 12826639]
62.
Cabana MD, Rand CS, Pose NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–65. [PubMed: 10535437]
63.
James B. Quality improvement opportunities in health care—making it easy to do it right. J Manag Care Pharm. 2002;8(5):394–9. [PubMed: 14613407]
64.
Ellwood PM. Shattuck Lecture—outcomes management: a technology of patient experience. N Engl J Med. 1988;318:1549–56. [PubMed: 3367968]
65.
Steinwachs DM, Wu AW, Skinner EA. How will outcomes management work? Health Aff. 1994;13:153–62. [PubMed: 7988991]
66.
National Heart, Lung and Blood Institute. Bethesda MD: National Institutes of Health; 1991. National Asthma Education Program Expert Panel Report: Guidelines for Diagnosis and Management of Asthma. Publication No 91-3042.
67.
Steinwachs DM, Wu AW, Skinner EA, et al. Final report submitted to the Managed Health Care Association Outcomes Management Consortium. Baltimore, MD: Johns Hopkins University; 1996. Asthma outcomes in managed health care: outcomes management and quality improvement.

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this page (92K)

In this Page

Other titles in this collection

Related information

  • PMC
    PubMed Central citations
  • PubMed
    Links to PubMed

Similar articles in PubMed

See reviews...See all...

Recent Activity

ClearTurn OffTurn On

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...