Program Rationale

Although our multispecialty group had fared well in commercial payer-based pay-for-performance programs, we found those programs were limited in their ability to promote quality improvement and the adoption of quality and safety systems. The payer-based programs differentially affected some segments of our group, with primary care physicians carrying the greatest burden, while no payer metrics applied to other physician groups. The data lags inherent in payer-based programs were also problematic because they could only serve to retrospectively score performance and were of limited value in managing that performance. As a result, payer-based pay-for-performance, an important contracting mechanism in our market, was not a sufficient catalyst for quality improvement and systems adoption in a climate of increasing focus on quality and transparency.

There were also internal reasons for considering an incentive program. In taking a systems approach to quality and safety, our multi-specialty group had made major investments in electronic medical records, computerized radiology order entry with embedded decision support, and electronic prescribing. However, the adoption of these systems in routine clinical practice was variable due to requirements to change work-flow, concerns over productivity, and a lack of training. To realize the potential of our systems investments, we required more complete adoption than had been achieved with exhortation, pressure from payer-based contracts, and further investments in making the systems user-friendly and efficient. Taken together, these realities—common to physician groups attempting to promote quality improvement and the adoption of safety systems—created a compelling rationale to engage our physician community in the design of an incentive program.

Design Process

To avoid the pitfalls common to previous incentive programs, we began the design of the incentive program with a comprehensive review of both the clinical and economic literature on the subject. Those results, which often raised concerns without necessarily providing solutions, were then shared with representative physician leadership in a series of meetings over 12 months leading up to the program launch. Those discussions explored key design considerations identified by others as crucial to incentive programs, including a consideration of the institutional culture, community context, organizational strategy and structure, organizational stability, infrastructure, quality measurement, nature and size of incentives, and the sustainability of interventions.²⁴

The involvement and awareness of the physicians who ultimately participated in the incentive program’s design have been identified as an essential process component.²⁵ Physician involvement afforded us the opportunity to craft a program with appropriate attention to professionalism and other concerns raised by incentive programs. In addition, it fostered a collaborative approach to quality and safety, which increased the dialogue on that subject among physicians.

Guiding Principles

The explicit goal of our program was to recognize physicians for reaching quality incentive targets and for supporting physician organization goals. From the beginning of the program, we were careful to differentiate this effort from prior payer-based incentives, in that our goal was to avoid creating “winners and losers”; we wanted 100 percent of our physicians to have success because that success was linked to organizational success. We also wanted to avoid the potential negative consequences of payment for performance on disparities26 and therefore decided that all metrics would be “payer-blind.” Based on the literature review and input from our physicians, we developed a simple set of guiding principles for the program:

1. Involve all clinically active physicians. We wanted to ensure that all physician groups would be affected fairly.
2. Focus on activities that are important to our patients and physician organization success. We chose just a few initial organization-wide adoption targets for our program to keep the organization focused on what it needed to achieve to ensure improved quality and safety for our patients and organizational success.
3. Differentiate motivation (incentive) from capability (infrastructure and supporting processes). It is essential that those accountable for performance have the tools to achieve it to avoid a fundamental disconnect.²⁷ The program must recognize that incentives to work harder and faster are doomed to failure, since they will be unsustainable when the incentive is gone. To help address this challenge, we ensured that physicians would receive the maximal support possible, including training and changes to our software systems, to promote adoption. Such focus allows the program to promote working smarter rather than providing a fee for the service of working harder.
4. Complement departmental quality and compensation plans. Some of our physician groups already had systems-adoption and quality improvement goals embedded in their compensation plans. Rather than replace or undermine those existing programs, we focused on areas outside of existing plans.
5. Design with input from our physicians and departmental chiefs of service. To ensure the program would be clinically meaningful, we complemented organization-wide targets (e.g., adoption of electronic medical records) with targets chosen by the departments themselves.
6. Use available metrics. To avoid the creation of a measurement and accountability bureaucracy and to avoid shifting resources to administering the program, we chose to focus our program on areas where there were either existing metrics or a compelling need to develop them because of upcoming pay-for-performance contracts.
7. Remain flexible and evolve. The incentive program was consciously portrayed as a continuing work in progress to explicitly recognize that it would never be error-free. The flexibility in the program allowed for near real-time adjudication of unintended consequences and promoted a sense of fairness. The flexibility and evolution of the program has also allowed us to keep true to the one promise of the program that it will get better.

Program Characteristics

With the guiding principles defined, we sought to take a comprehensive approach to address key aspects of the program. Based on our review of other programs, we focused on four key design issues: (1) eligibility, (2) frequency, (3) size of incentive, and (4) choice of metrics. For each, we conducted a review of the literature for guidance and engaged in an extensive review and revision process with physician representatives.

Eligibility. To be inclusive, we defined eligibility for the program broadly—all physicians who participated in the majority of our managed care contracts and met a minimum level of clinical productivity, measured as at least 50 work Relative Value Units (wRVUs) during the preceding 6 months. Physicians who billed privately were required to submit wRVU data for review to be eligible for the program. Payouts for the program were tiered based on clinical productivity with the tiering structured to ensure that the majority of physicians (65 percent) fell into the highest tier of reward in an effort to fully engage a broad segment of the physician population. Because the incentive program was part of our effort to ensure future success on pay-for-performance arrangements with commercial and public payers, physicians who did not bill for clinical services directly (e.g., trainees) were excluded from the program.

Frequency. As with other aspects of the program, appropriate balances must be considered with regard to the frequency with which the program measures performance and provides feedback/reward. We and others²⁵ have found that commercial payer pay-for-performance contracts, where the withhold return often occurs more than 1 year after the close of the billing year, are too lagged and infrequent to foster physician focus. Annual payments are too infrequent to solidify the relationship between performance and reward. However, more frequent assessments and payments result in greater infrastructure demands and resource needs. To balance these realities, our program runs on a 6-month basis. This provides an opportunity for the program to improve and evolve yet maintain some stability so that physicians do not perceive it as a case of rapidly changing goalposts or “flavor of the month.”

Size of incentive. A review of the literature on the sizing of physician incentives to achieve measurable behavioral change reveals conflicting recommendations. Some suggest that the incentive should be at least 10 percent of annual income,²⁸ while others admit that no real data suggest a more firm specification beyond “enough.”²⁵ In reviewing the program with our physicians, we learned that no sum would be “enough” to “pay” for adopting some of these systems. For example, the adoption of an electronic medical record system not only means lost time in the clinical setting but often translates into evening hours spent converting old records to the new system, time that would otherwise have been spent with family. As a result we found it useful to move the dialogue from a focus on remuneration (payment for time value) to reward (financial recognition of the effort made). This change in both underlying philosophy and rhetoric was not only key to gaining physician acceptance, it also supported the principle that the incentive is only partly about the money. It was also a clear statement of organizational priorities and values.

Nevertheless we still faced the challenge of trying to keep the program affordable yet powerful enough to promote real change. Here we found the economic literature on the Nobel Prizewinning Prospect Theory,²⁹ which explains the evaluation of risk and utility by individuals, to be helpful. According to Prospect Theory, the risks of gains and losses are evaluated differently from what one would otherwise expect by looking at the expected utility. This explains why individuals would rather take a 50/50 chance on losing $200 than a guarantee of a loss of only $100, yet they would accept a guaranteed $100 gain over a 50/50 chance of gaining $200.

We applied Prospect Theory to our program design by phasing the implementation of our program. In the first phase, all participating physicians were given the full financial reward ($500, $1,250, or $2,500 depending upon their wRVU tier) just for meeting the eligibility requirement. However, the communication with the check for that reward made it clear that receiving the remainder of that year’s potential reward would depend upon performance. This phasing allowed us to create the sense that underperformance would lead to our physicians not getting something they otherwise should have received. This creation of an expectation of success proved to be a powerful motivation for our physicians.

Choice of metrics. Based on our pay-for-performance contracts and the goals of our quality and safety initiatives, we chose two metrics for organization-wide focus—the adoption of electronic medical records and the use of electronic radiology order entry—for the initial measurement period. In both cases, the incentive program was preceded by over a year of communications with the physician staff on the importance of these two systems to our future success. Nevertheless, adoption lagged. To be equitable across physicians, those departments for which these organization-wide metrics were inappropriate (e.g., radiology and pathology) had to come up with substitutes where adoption and performance would have an impact on quality and safety (e.g., decreasing turn-around time on final reports).

In addition to the organizationwide metrics, each department was required to develop its own metric related to quality and safety. This ensured clinical relevance and promoted the incentive program as a collaborative effort rather than a central mandate. In the departmental metric selection process, care was taken to avoid selecting metrics that would link incentives to a clinical decision with an individual patient, further reinforcing the systems approach. Some departmental measures were extensions of the organizationwide metrics (e.g., improving the timeliness of electronic visit or operative notes or decreasing the proportion of potentially inappropriate, high cost imaging studies). Others chose metrics related to public reporting (e.g., ACE/ARB use in congestive heart failure or perioperative and pneumonia antibiotic timing); Joint Commission compliance (e.g., hand hygiene and medication reconciliation); or department-based systems adoption (e.g., the deployment of electronic anesthesia flow sheets and electronic prescribing in the emergency department). Table 1 presents the array of metrics used in our first measurement period in the spring of 2007.

Table 1

Initial set of incentive program metrics.

Program Implementation

Our reviews of the literature suggested that the implementation of an incentive program might be even more important than design features.²⁵ In an effort to apply “evidence-based implementation,” we identified six key levers for ensuring program success: (1) embedding the program in a larger quality and safety effort, (2) clear and frequent communications, (3) integrity of performance scoring, (4) transparency, (5) identifiable reward disbursement, and (6) the availability of an appeals mechanism.

Embedding in a larger quality and safety effort. The extant literature on incentives is clear on one point: incentives alone are not sufficient to improve quality and promote systems adoption.²⁷ Like Pasteur, we recognized that “the seed is nothing, the soil is everything.” In our case the “seed” (our incentive program) was unlikely to flourish without appropriate attention to our “soil” (our organizational culture). As a result, we approached the development and implementation of the incentive program from the standpoint that this program was necessary, but not sufficient, to improve quality and systems adoption.

Rather than rely on incentives as an all-powerful, stand-alone magic bullet, our program was embedded as an appropriate next step in a quality and safety strategic planning effort that had begun 3 years earlier and was reaching a critical juncture, where our progress was slowed by the adoption chasm. Other components of that strategic planning included safety culture surveys, quality and safety leadership retreats, integrated delivery system-level efforts focused on information systems adoption,³⁰ and a rewriting of the hospital’s mission statement to reflect an emphasis on quality and safety. In addition, we developed complementary incentive programs—many of which were based on aggregations of the physician incentive program metrics—for the senior leadership of both the physicians’ organization and hospital. This reinforced the organizational commitment to quality and safety and aligned all interests to achieve success.

Clear and frequent communications. Keeping the physicians in an incentive program informed is a key success factor cited in the literature.²⁵ As a result, we developed a broad communications campaign to accompany the incentive program, which included regular newsletter updates; general electronic mail communications to all physicians; targeted electronic mail communications to departments and individual physicians; electronic reminders throughout the performance measurement period; meetings with representative physician groups; updates at “town hall” style meetings; incorporation into regular leadership meetings of department chiefs; and publication of updates, metrics, and targets on the physicians’ organization Web site. The communications program was designed to get the message out “early and often” but to avoid the risk of becoming noisome to busy physicians. As with other aspects of the program, this required a careful balance and two-way communications with our physicians to titrate our strategy.

Integrity of performance scoring. Recent negative experience with a comprehensive program to promote quality in general practices across the United Kingdom illustrates the importance of keeping the scoring of performance simple in order to avoid both confusion and gaming.²² Performance on the two organization-wide performance metrics was evaluated at the individual level and determined 50 percent of the reward payment. These were scored on an “all-or-nothing” basis. The remaining 50 percent was based on the performance on the department’s chosen metric. Here, the unit of analysis varied among the departments, with some choosing to measure at the individual level (e.g., note completion time), while others were at the departmental level (e.g., hand hygiene). In those cases where individual rates were used (e.g., proportion of operative notes completed in 24 hours), there was some room for “partial credit,” although this was limited to maximize the impact of the intervention, and it was neither communicated nor applied until after the end of the measurement period.

We kept the program simple by not using an elaborate scoring algorithm, and thus, we did not need to create a “measurement and accountability bureaucracy.” Also, the 50/50 split on scoring between organizational and more locally defined goals emphasized the collaborative nature of the program.

Transparency. At baseline and after the initial performance measurement period, aggregate data were shared with all of those participating in the program via print publications and on the physicians’ organization Web site. Again, there was a need to strike an appropriate balance, this time between transparency and the appropriate peer pressure it generates with respect to privacy. Based on a previously crafted performance data sharing policy, that dissemination was guided by the application of a “need-to-know” principle. Under that policy—which was adjudicated by the hospital’s and physicians’ organization Medical Policy Committee—all individuals had a need to know their own detailed performance data. Within a department, individual physicians could also see whether colleagues were meeting their performance target (as “Yes/No”) but could not see any individual’s specific performance data. Individual physicians could also see the aggregate performance of other departments. Senior leadership at the organizational and departmental level, whose own incentive program was tied to aggregate performance, had access to individual level performance data.

Identifiable reward disbursement. We took two steps to maximize the impact of the incentive payments. The first was to mail real checks (as opposed to electronic transfers) to physicians’ homes. Although the use of checks required the physicians to get the checks deposited or cashed, the use of actual checks reinforced the distinct nature of the incentive payment. In addition, we had our initial payments coincide with the end-of- year holiday bonus season. The incentive checks were accompanied by a cover letter from the physician organization’s leadership and the department chief of service, an individualized performance report (Figure 1), and a notification of future metrics and targets.

Figure 1

Example of performance communication to individual physician. (EMR = electronic medical record, ROE = radiology order entry).

Availability of an appeals mechanism. Although we made every effort to keep the incentive program simple and get the assignments of accountability and performance measurement correct, the program explicitly acknowledged that it would not be perfect. Physicians who felt that they had not been treated fairly in the program—whether in terms of eligibility or performance assessment—were provided with clear instructions on how to appeal their incentive assignment and were invited to do so. Those appeals were then investigated individually, data were reexamined where appropriate, and a final decision on the validity of the appeal was then made by the Medical Director and ultimately the Chief Executive Officer of the physician organization. Although the proportion of physician appeals was low, 4.7 percent in the spring 2007 measurement period, the availability of an appeals mechanism was a key design feature, which highlighted the commitment to program improvement over time.

Collateral Benefits of the Program

In addition to the preliminary benefits of the program in terms of systems adoption and quality improvement, some important collateral benefits of the program have provided value to the physicians organization. The greatest of these was the heightened interest in quality and safety engendered by the program. Examples of that increased engagement included the participation of all departments in defining quality metrics, increased traffic to our quality and safety Web sites, and an unprecedented level of focused inquiries to the program leadership about quality and safety measurement. That interest integrated nicely with the execution of our quality and safety strategic planning.

Other benefits included the development of more accurate lists of clinically active physicians and their practice group/peers; the identification of low clinical volume physicians for future consideration in contract discussions; enhanced data on the physicians in private practice; early identification of potential billing compliance issues, which were amenable to immediate remediation; improved scheduling information; and increased use of radiology order entry decision support.

The measurement process also improved the integrity of the data we use for payment for performance with commercial payers. The increased scrutiny of our data sources to support this program led to the recognition of failures in extant measurement efforts and improved our pay-for-performance results. In addition, the increased attention paid by our clinicians to our quality and safety systems resulted in important enhancements to our information technology and the identification of opportunities to improve our decision support software, which otherwise might not have come to our attention.

Program Evolution

From the outset we recognized that the program was a trial and that it would only continue as long as it provided value to our physicians and the patients they serve. The evolution of the program to date has been illustrative of the flexibility of the initial approach, with the continued focus on remaining true to collaboratively developed guiding principles. It is expected that the program, which is now in its third incentive period, will continue to evolve with our quality and safety agenda.

Future plans include the retirement of some metrics, where our goals for adoption have been achieved (e.g., adopting electronic medical records); raising the bar in some areas (e.g., note completion time); maintaining focus in areas that required sustained attention (e.g., antibiotic timing in the emergency department); development of new metrics to address our evolving pay-for-performance contracts; and calls for transparency (e.g., electronic prescribing rates). To promote greater transparency and maintain focus—particularly in cases where the adoption or improvement goals have been achieved, leading to metric retirement—we plan to incorporate ongoing performance surveillance into the institution’s quality and safety dashboard, which will be reviewed regularly by the senior leadership and trustees.