Background:

Transitions from hospitals to the ambulatory setting are high-risk periods for patients. The advent of the patient-centered medical home (PCMH) and accountable care organizations (ACOs) has provided an opportunity for true collaboration in which both inpatient and outpatient providers contribute to improving transitions in care. Few studies have rigorously evaluated real-world, patient-centered interventions that take advantage of these new developments in health care.

Objectives:

The goal of this study was to develop, implement, refine, and evaluate a multifaceted, multidisciplinary transition intervention across 2 hospitals and 18 PCMHs within a Pioneer ACO.

Methods:

The study population included adult patients admitted to medical or surgical services at 2 hospitals within the Partners ACO, with primary care physicians at the 18 participating PCMHs, and with a plan to be discharged home. We developed an intervention with the following components: inpatient pharmacist-led medication reconciliation and patient counseling, coordination of care and patient education from an inpatient discharge advocate and the PCMH responsible outpatient clinician, a structured visiting nurse intervention, structured postdischarge phone calls, timely follow-up visits, tools to improve communication among care team members, and home pharmacist visits for selected patients. The study used a stepped-wedge design in which each PCMH practice started in the usual care arm and then, at a randomly selected point in time, changed to the intervention. Outcomes included 30-day hospital readmissions using administrative data and telephone follow-up, and new or worsening symptoms in the 30 days after discharge based on telephone follow-up and medical record review. We analyzed the 2 outcomes by multivariable logistic and Poisson regression, respectively, adjusted for study month, season, patient demographics, risk for postdischarge adverse events, inpatient unit, and primary care practice.

Results:

We enrolled 1657 patients, including 679 assigned to usual care and 978 to the intervention. Receipt of different components of the intervention varied by component and in some cases by hospital, unit, and practice. Thirty-day nonelective readmission rates were 10.9% in the intervention arm and 11.5% in usual care (adjusted odds ratio [OR], 1.08; 95% CI, 0.64-1.85, p = .77). The number of new or worsening symptoms was 0.90 per patient in the intervention arm and 0.92 per patient in usual care (adjusted OR, 0.78; 95% CI, 0.64-0.95; p = .01). The intervention was also associated with a 48% relative reduction in postdischarge adverse events (adjusted IRR = 0.52, 95% CI, 0.30-0.91, p = .02). A priori subgroup analysis found no evidence for effect modification of the intervention on readmission rates, new or worsening symptoms, or adverse events.

Conclusions:

Results showed no difference in adjusted 30-day readmission rates among patients in the 2 study arms, likely owing to lower than expected intervention fidelity and the low proportion of readmissions that are truly preventable in this patient population. However, the intervention was associated with a reduced rate of new or worsening symptoms in the postdischarge period and on postdischarge adverse events—outcomes that are more sensitive to change than readmissions. As with readmissions, efficacy was likely limited by intervention fidelity. Limitations include confounding by indication for some of the intervention components. Further study is needed to explore the causes and effects of low intervention fidelity, to determine the most important components of the intervention, and to explore variation in care by hospital, inpatient unit, and primary care practice.

Patient-Family Advisory Council

Our Patient-Family Advisory Council (PFAC) included patients and caregivers of recently hospitalized patients, thus representing those likely to benefit from the intervention. They were identified and recruited through the Brigham and Women's Hospital (BWH) Office of Patients and Families and the Massachusetts General Hospital (MGH) Institute for Patient Care.

Through their own monthly meetings and our quarterly steering committee meetings, they participated in every aspect of the study, including formulating research questions, finalizing study outcomes, developing and refining the interventions, monitoring study progress, and making plans for dissemination, including reporting results in a manner that would be understood by the public.

We worked with our steering committee on the clinical and policy implications of the study through quarterly meetings throughout the study period (PCORI Methodology Standard PC-4). In addition to PFAC representatives, the steering committee consisted of physician, nursing, pharmacy, information technology, and administrative leadership representing primary care, inpatient care, and transitional care at BWH and MGH, and within the Partners ACO. The committee consisted of approximately 20 members, including 1 or 2 designated PFAC members who participated on a rotating basis for each meeting.

While we based the initial concept for the intervention on the literature and on a model designed by the principal investigator and others, the PFAC members met monthly during the design phase and shaped the design of most of the intervention components, such as the following. (See the Results section for additional details regarding how stakeholders influenced the intervention.)

1. The sicker and more complicated a patient is, the faster the clinical team wants him or her to be seen in follow-up, but the less likely it is that the patient will feel well enough to get to an appointment. Scheduling follow-up appointments must be a negotiation among the inpatient, the patient, and the caregiver within the framework of the outpatient provider's availability.
2. Patients have different kinds of relationships with their primary care physicians (PCPs). Those with close relationships will likely want a quick follow-up appointment with the PCP soon after discharge, regardless of the reason for the hospitalization (eg, even if it was a surgical admission). But this may not be true for everyone. The types and timing of follow-up appointments should be customized for each patient. Patients generally want 1 person to be the point person during a complicated transition such as hospital discharge. Depending on the patient, this could be the PCP, a nurse from the primary care practice, the discharge advocate, or a pharmacist. This person should be identified early on; if possible, the other members of the intervention should defer to the point person and communicate with and through him or her.

The PFAC confirmed the relevance of the research question and study outcomes, while the steering committee ensured the transparency of the research process. The stakeholders played a smaller role in influencing the overall design, rigor, and quality of the study, including participant recruitment, but they provided valuable input into the design of all the patient-facing quantitative survey instruments, and they designed and conducted the qualitative interviews of patient participants under the leadership of the director of the BWH Center for Patients and Families. Perhaps most important, the PFAC and members of the steering committee played (and will continue to play) a major role in translating the research evidence into practice, from helping to convey these results to the public to assisting with manuscript preparation and other dissemination activities to directly applying the results to ongoing improvement activities in care transitions at BWH, MGH, and Partners.

One challenge of this research was balancing various stakeholder perspectives while adhering to the specific aims and research approach as originally proposed and funded. In the end, the principal investigator, with input from the other study investigators, had the final say on how the study was conducted. However (in a process that was built into the design of the study), he gave broad latitude to how the intervention was ultimately designed, implemented, and refined at each study site to best simulate how it would be implemented in the real world, to maximize stakeholder buy-in, and to maximize the generalizability of the findings. This customization of the intervention required constant stakeholder engagement.

Formal Study Protocol

The research question was “What is the impact of a collaborative transition intervention involving hospital and PCMH personnel within an ACO on postdischarge patient outcomes compared with usual care?”

We collected data for the purpose of determining the impact of our intervention on patient safety outcomes. The study population is representative of the population for whom the research question is pertinent. The results of the study will inform several groups of stakeholders, including patients and caregivers (eg, should I belong to an ACO?), hospital and health care executives (eg, how should we set up our health care system to improve transitions of care?), health policy personnel (eg, how should health care be organized and financed to facilitate adoption of transitional care interventions?), and direct care providers (which services should I offer to which patients to optimize patient outcomes?). We used the Standards for Quality Improvement Reporting Excellence guidelines for studies of quality improvement wherever applicable, except that we did not include a discussion of costs.

Study Population

Potential subjects were adult patients admitted to medical and surgical services at BWH and MGH (both academic medical centers within the Partners ACO) who were likely to be discharged back into the community (based on input from each patient's nurse) and whose PCP belonged to 1 of 18 Partners primary care practices that admitted at least 2 patients per month to BWH or MGH, agreed to participate in the study, and met Primed criteria for being a PCMH. (Primed criteria are a standard set of requirements that cover 6 essential building blocks of PCMH practices: (1) electronic health record, (2) patient portal, (3) team-based care, (4) practice redesign, (5) care management, and (6) identification of high-risk patients.) Patients were required to be fluent in either English or Spanish and to be cognitively intact or to have a health care proxy who was willing to provide consent, was planning to live with the patient after discharge, and was willing and able to answer study questions on behalf of the patient. Exclusion criteria included homelessness, police custody, or lack of a telephone.

Study Design

The study employed a stepped-wedge design—an observational study design in which an intervention is sequentially rolled out to different groups (in this case, different primary care practices) at different times (see Figures 1 and 2).¹⁷ For example, if the patient's PCP worked at BWH Primary Care Associates of Brookline and the patient was enrolled in the study in May 2014 (ie, before June 6, 2014, the date of the transition from control to intervention for that practice, see Figure 2), he or she would be assigned to usual care. If a patient with the same PCP was enrolled in July 2014 (ie, after June 6, 2014), he or she would be assigned to the intervention. Thus, each practice had a different amount of time in the usual care and intervention arms, and each one served as its own control. We randomized the order of the rollout to avoid confounding; that is, the primary care practices that were most ready for the intervention (and which may have had other characteristics associated with better implementation or better patient outcomes) did not necessarily get it first (and thus contribute more patients to the intervention arm).

Figure 1

Stepped-Wedge Study Design.

Figure 2

Stepped-Wedge Study Design.

Subject Enrollment

We took several steps to reduce selection bias and maximize the representativeness of the study population. Each day, a trained research assistant received a list of potentially eligible patients (ie, those admitted to medical or surgical services and with a PCP from one of the participating practices) from an automated report. We randomized the order in which these patients were approached for enrollment in the study; patients at the top of the order were likely to be enrolled in the study, while patients at the bottom of the order were unlikely to be enrolled because the daily and weekly quotas for enrollment (about 11 patients a week) would likely have been met before the research assistant got to the bottom of the list. If we had not taken this approach (eg, if we had started at the top floor of the hospital and worked our way down), enrollment in the study would have favored patients on a particular service or floor, leading to a nonrepresentative study population.

Once we selected a patient for possible enrollment, we made multiple attempts to enroll that patient until he or she enrolled, declined, or was discharged from the hospital without enrolling or declining. If we had not taken this approach (eg, if we had tried to enroll a patient once and then moved on to the next patient on the list), we would have biased our enrollment toward patients most likely to be in their rooms at the beginning of each enrollment day (eg, patients who were stable, not in need of diagnostic tests or procedures, unlikely to be walking around the hospital for exercise or leaving their room to smoke). By randomizing the order in which we approached patients and committing to enroll patients once we selected them, we preserved the randomization of patients for enrollment to the extent possible, maximizing the chance that our enrolled patient cohort was indeed a random sampling of medical/surgical patients with Partners PCPs. Note that this individual-level randomization had nothing to do with treatment assignment, only enrollment in the study.

Once we selected a patient, the research assistant reviewed the medical record of the potential subject and asked his or her nurse for additional details to confirm eligibility and for permission to approach the patient regarding informed written consent to participate in the study. Patients did not learn of their allocation to the usual care or intervention arm (based on the practice of the patient's PCP and the date of patient enrollment) until after they agreed to participate.

Study Setting

The study took place on the medical and surgical non-intensive-care units of BWH and MGH (where patients were enrolled in the study, completed baseline data collection, and began the intervention) and in patients' homes, communities, and primary care practices or other health care settings (where additional intervention components were provided and where patients completed data collection by phone call 30 days after discharge).

Intervention

We based our intervention on a conceptual model we developed of the ideal transition in care (Figure 3). The model incorporates work by Naylor et al¹⁸ and by Coleman and Berenson,⁷ best practices in medication reconciliation and information transfer based on our own research,^19-21 the best examples of interventions to improve the discharge process,^18,22,23 and a recent systematic review of discharge interventions.¹² Some of the factors necessary for a high-quality transition in care are complete documentation of clinical information regarding the patient's hospitalization and postdischarge plan of care, clear organization and timely transmission of that information, effective discharge planning, coordination of care among the patient's providers, methods to ensure medication safety, advanced care planning for appropriate patients, and education and coaching of patients and their caregivers to help them learn how to manage their conditions. Successful interventions likely require several of these factors working in concert.

Figure 3

Conceptual Model of the Ideal Transition of Care.

Before the start of the study, we approached each primary care practice associated with Partners, described the study, and invited them to participate. Before a practice moved from usual care to the intervention arm as part of the stepped-wedge methodology, we revisited the practice and engaged in an in-depth discussion of the intervention, including the components that would be conducted by others and those that would be conducted by the practice. We helped them adapt the intervention components they would conduct (eg, nurse-to-nurse communication, postdischarge phone calls, postdischarge visits) and address potential staffing needs and workflows in the following areas:

1. Who would communicate with the inpatient discharge advocates, who would initiate the contact, when would it occur during the hospitalization, and how (eg, phone, email)?
2. Who would make the postdischarge calls, what changes might need to be made to the call script and documentation template, and what actions should be taken in response to certain answers from patients for each question?
3. What actions should be taken during the postdischarge visit and by whom, and what changes might need to be made to the postdischarge note template?

Components of the proposed intervention, as originally designed, included the following (see Figure 3 for how each of these corresponds to a component in the ideal transition in care conceptual model):

1. Inpatient medication safety interventions. An inpatient pharmacist ensured accurate medication reconciliation, including confirming the accuracy of the preadmission medication history, conducting in-depth medication reconciliation at admission and discharge, and communicating with postdischarge providers regarding the discharge medication regimen and reasons for changes. The pharmacist worked with the patients' primary inpatient nurse to educate patients and caregivers about the discharge medication regimen, including indications, special instructions for administration, and reasons for changes from the preadmission regimen. Lastly, prepared patients (and their caregivers) to use medications correctly and safely after discharge, including maximizing adherence and minimizing adverse drug events. This included reviewing barriers to adherence (including cost, side effects, forgetting, lack of access, inconvenience, skepticism about their efficacy) and addressing the barriers or communicating with the primary care practice regarding the need to address them. It also included identifying potential side effects of medications, especially any new medications, and providing education on what to do if these side effects occur. The intensity of the intervention was designed to vary depending on the complexity of the patient's medication regimen, the patient's health literacy and understanding of his or her medications, and the patient's previous medication problems, such as nonadherence or side effects.
2. Inpatient discharge advocate. This component (based on the discharge advocate role in Project Re-engineered Discharge²³) called for a nurse at each of the 2 hospitals to form a therapeutic alliance with the patient, facilitate communication between inpatient and outpatient care teams, ensure the creation of a high-quality discharge plan, facilitate education and preparation of the patient and caregivers for discharge based on their level of health literacy, work with patients to determine what optional interventions they might need, and teach primary inpatient nurses to incorporate these tasks into usual care. The discharge advocate initiated an ongoing dialogue between inpatient and outpatient teams, and facilitated collaborative creation of a discharge plan and scheduling of follow-up appointments and tests within an appropriate time frame. As part of these tasks, the discharge advocate provided and went over with patients and caregivers a predischarge checklist and asked patients and caregivers to identify their most important goals for the postdischarge period, documented them, and took steps to maximize achievement of these goals.
3. Visiting Nurse Association (VNA) appointments. In the week after discharge, Partners Healthcare at Home provided VNA services to qualifying patients (ie, those who were at least temporarily homebound and in need of nursing services). Unlike routine VNA visits, these included a structured template to assess the patient's home situation, current services, and level of support, and to ensure that patients could manage their conditions at home (eg, take their medications, modify health-related behaviors, manage follow-up appointments including transportation to clinic visits, and carry out other aspects of their postdischarge care plan, such as changing dressings). The visiting nurses also verified that patients who could not manage on their own had caregivers who could help them with these tasks. The nurses were encouraged to contact either the inpatient or primary care team with questions or concerns, by phone or email as appropriate. They also wrote structured notes that were placed in the ambulatory electronic medical record (EMR) used by all Partners practices.
4. Responsible outpatient clinician. The responsible outpatient clinician (ROC)—usually a registered nurse (RN) from the patient's PCMH previously assigned to patients of 1 or more PCPs—carried out several tasks: to communicate with the discharge advocate to exchange information and prepare for postdischarge care, to videoconference with the patient while he or she was still in the hospital, to make a postdischarge phone call within 2 days of discharge, to see the patient during the postdischarge clinic visit, and to make 2 additional phone calls during the month after the clinic visit. ROCs were expected to conduct coaching based on the Care Transitions Intervention model,²⁴ to help patients identify and overcome barriers to self-management and help them effectively interact with the health care system at their level of health literacy, to communicate with the PCP, to make plans and arrange follow-up as needed, and to document findings in the EMR.
5. Videoconference. Using mobile technology provided by the study team, the ROC was encouraged to videoconference with the patient prior to discharge; ask the patient about his or her most important goals for the recovery period; discuss the reasons for and the importance of keeping follow-up appointments; remind patients to bring the discharge instructions, personal medical record, follow-up calendar, medication list, and pill bottles to the postdischarge follow-up appointment; discuss the use of the personal medical record; and discuss barriers to keeping follow-up appointments and explore ways to overcome them.
6. Postdischarge phone call. ROCs were given phone call scripts and documentation templates and were encouraged to call all patients within 2 days of discharge. The goals of these calls were to screen for new or worsening symptoms; assure themselves that patients could perform activities of daily living independently or with available help, ensure that patients could carry out the postdischarge care plan, verify the patient's understanding of and adherence to the medication regimen, review danger signs and tell the patient what to do if they occurred, encourage the use of the personal medical record, review follow-up appointments and transportation plans, and identify and manage barriers to keeping the appointments.
7. Multidisciplinary postdischarge PCMH clinic visit. The ROC, PCP, PCMH pharmacist, and other personnel as needed (eg, social worker) were encouraged to work as a team to see all patients within 7 to 14 days of discharge, depending on the acuity of the patient. Following a standardized algorithm, each team member was trained to play a specific role in evaluating the patient's progress along the plan of care, ensuring patient safety, and optimizing postdischarge outcomes, including helping patients meet their postdischarge goals. PCPs and ROCs were encouraged to review the discharge summary, patient instructions, follow-up appointment calendar, patient's personal medical record, and any VNA notes in the EMR; to complete postdischarge medication reconciliation; to review test results finalized after discharge; to communicate with other members of the care team; to continue patient coaching activities; and to arrange additional needed services, appointments, tests, home monitoring, and community resources.
8. High-risk patients received additional interventions as deemed necessary by the care team. For each intervention patient, the inpatient attending and PCP received a menu of additional interventions and were asked to consider them on a case-by-case basis on the basis of perceived risk. Additional interventions included the following:
   • A home visit by Dovetail (a home pharmacist company) with several goals: confirm that inpatient medication reconciliation was done correctly; identify and resolve any discrepancies between the discharge regimen and what medications the patient believed he or she should be taking; screen for barriers to adherence and early side effects, and address them as needed; and work on strategies to maximize medication adherence, identify medication red flags, and provide contingencies when problems arise. Dovetail pharmacists communicated with PCMH practices as needed, provided additional outreach to patients as needed, and documented their findings in the EMR.

     –

     Enrollment in telemedicine programs; for example, daily monitoring and electronic transmission of weights and diuretic dose adjustment by a nurse.

     –

     Specialist (communicating with the PCP or cardiologist as needed) for patients with chronic heart failure who were enrolled in Partners Healthcare at Home.

     –

     Advance care planning: We created a tool to automatically flag patients with a 15% 30-day mortality, using an algorithm created by the MGH Lab of Computer Science based on empirical data obtainable from the EMR. At BWH, the flag triggered a recommendation for an inpatient palliative care consultation with patients, caregivers, and providers regarding goals of care; communication with outpatient providers so this discussion could be continued; and documentation in the EMR. At MGH, the flag triggered a recommendation for inpatient or outpatient palliative care referral as appropriate. We also provided education to MGH residents regarding appropriate inpatient referrals. A home-based palliative care program was available for a limited number of homebound patients.

     –

     Enrollment in the Partners integrated Care Management Program (iCMP), which included intensive and individualized case management.

9. Novel health information technology
   1. A web-based discharge-ordering module to help ensure the quality of discharge documentation by auto-importing certain information from the EMR and requiring completion of structured data fields.
   2. An automated notification system that emailed inpatient attendings and PCPs the results of tests pending at discharge as they became available (BWH only).²⁵
   3. Tools to identify and group text messages and email to all inpatient and ambulatory care team members, facilitated by the Partners Enterprise Patient List (PEPL) application, to improve multidisciplinary communication across care settings.²⁶

Usual Care

Once primary care practices become a PCMH, they usually take several steps on their own to improve transitional care. The question is whether patient outcomes can be improved even further with a standard set of intervention components—informed by the medical literature and a conceptual model of the ideal transition in care—that leverages the PCMH's new capabilities and can be combined with interventions on the hospital side to facilitate coordinated, patient-centered care. This choice of comparator mimics as closely as possible the choices faced by any practice that has become a PCMH and that works within an ACO or other similar integrated delivery system.

Clinician Surveys

To provide information on environmental context at baseline, we surveyed clinicians involved in the transition process (physicians, nurses, pharmacists, and care coordinators) before implementing any interventions. These surveys contained several types of questions: the clinician's assessment of preparedness to deliver transition interventions (eg, role clarity, training, feedback, use of tools and technology), assessment of the quality of the transition process, teamwork climate, and patient safety climate. We assessed the latter 2 factors using Agency for Healthcare Research and Quality (AHRQ) survey instruments,²⁷ and derived the former from surveys used to assess other safety processes, such as medication reconciliation. We surveyed clinicians from each of several roles in the transition process (eg, residents, inpatient attending physicians, nurses, pharmacists, care coordinators, therapists, and physician assistants; outpatient PCPs, nurses, care coordinators, and medical assistants) and from each inpatient unit and primary care practice participating in the study.

Inpatient and Outpatient Inventories of Transitional Care Tasks

We surveyed physician and nurse leadership from all inpatient units and outpatient practices in the study regarding their general staffing levels (eg, number and clinician types), how often they carried out various transitional care tasks, and who usually performed those tasks (Tables 4A, 4B, and 4C, and the Results section).

Table 4A

Inpatient Inventory Results: Summary by Unit (BWH).

Table 4B

Inpatient Inventory Results: Summary by Unit (MGH).

Table 4C

Outpatient Inventory Results: Summary by Practice.

Outcomes

Study outcomes included the following:

1. New or Worsening Signs or Symptoms Within 30 Days of Discharge.
   A trained research assistant contacted patients 30 (±5) days after discharge and administered a questionnaire to identify any new or worsening symptoms since discharge, any health care utilization since discharge, functional status in the previous week, and patient experience, including participation in, understanding of, and ability to carry out the postdischarge care plan. Follow-up questions used branching logic to determine the relationship of any new or worsening symptoms in response to medications or other aspects of medical care, as well as the consequences of these symptoms, including additional health care utilization, functional decline, worry and anxiety, time lost from work, out-of-pocket expenses, and additional caregiver time. This patient-reported outcome (new or worsening symptoms since discharge) has been used in other studies of patient safety, is meaningful to patients, and relates to health decisions patients would need to make. For patients not reachable by phone despite 5 attempts (about 47% of the study cohort), the research assistant reviewed the outpatient medical record for provider reports of any new or worsening symptoms noted during follow-up within the 30-day postdischarge period. Research assistants also reviewed laboratory test results for evidence of renal failure, elevated liver function tests, or new or worsening anemia in the postdischarge period.
2. Hospital Readmissions
   We measured nonelective hospital readmissions within 30 days of discharge using a combination of administrative data for BWH and MGH plus patient report during the 30-day phone call for all other readmissions. Previous studies have shown that this method is effective in capturing almost all admissions for patients discharged from BWH.²⁸
3. Adjudicated Outcomes
   Adverse Events and Preventable Adverse Events
   All cases of new or worsening symptoms, along with all supporting documentation, were presented to teams of 2 trained, blinded physician adjudicators. Each adjudicator independently reviewed the information, along with the medical record, and completed a standardized form to confirm or deny the presence of any adverse events (ie, patient injury owing to medical care rather than to underlying medical conditions). If applicable, they documented the type of event (eg, adverse drug event, hospital-acquired infection, procedural complication, diagnostic or management error), the severity and duration of the event, other consequences of the event (eg, on functional status, health care utilization, out-of-pocket costs), and whether the event was preventable or ameliorable. The 2 adjudicators then met to resolve any differences in their findings and to reach consensus.
   We did not give the adjudicators the stepped-wedge schedule and we purposely delayed adjudication until 2 months after the first practice implemented the intervention, but they could not be fully blinded to intervention status: We could not remove dates from medical records, they might have come across documentation of interventions while adjudicating, and they might have been vaguely aware that later cases were more likely to involve intervention patients.
   Preventable Readmissions
   If patients were readmitted to BWH or MGH, we conducted a thorough evaluation to determine whether and how the readmission could have been prevented. Based on the HOMERUN study of 1000 patients at 12 academic medical centers,²⁹ this process included (1) a standardized patient and caregiver interview to identify possible problems with the transition process and the patient's preparedness to manage the postdischarge care plan, and (2) an email questionnaire to the inpatient teams that cared for the patient during the index admission and the readmission and to the patient's PCP regarding possible deficiencies in the transition process. Using that information and medical record review, teams of 2 physician adjudicators completed a form to determine the preventability of the readmission; which, if any, deficiencies in the transition process contributed to the readmission; and what interventions might have prevented it. As with adverse event adjudications, these physicians worked independently and then met to resolve their differences and reach consensus.
   To improve the reliability of adjudicated outcomes, we trained all adjudicators using standardized cases and held monthly group meetings at which particularly difficult cases could be discussed. We compiled lessons learned from this process in a Frequently Asked Questions document, which we updated and distributed to all adjudicators as appropriate.
4. Other Outcomes
   Patient Satisfaction and Opinions of the Discharge Process
   During the 30-day follow-up phone call, we asked patients about their participation in, understanding of, and ability to carry out the postdischarge plan. These questions included some selected from Care Transitions Measure 3 (CTM-3),³⁰ the Interpersonal Processes of Care survey,³¹ and the HOMERUN study of readmitted patients. On the basis of input from our Patient and Family Advisory Council during monthly meetings, our population of interest cared about the following outcomes and believed that certain adjustments would help inform health care decisions.

Analysis

To evaluate the effects of the intervention (independent variable) on the number of new or worsening postdischarge signs and symptoms per patient, the number of adverse events per patient, and the number of preventable adverse events per patient (dependent variables in the form of number of events per patient), we used multivariable Poisson regression, with the study arm as the main predictor (independent variable). We used an intention-to-treat analysis: If a practice did not start the intervention when it was supposed to according to our randomization, we counted all patients in that practice who were admitted after that point as intervention patients. Covariates in this model—all collected at baseline—included study month (to adjust for temporal trends); season (to adjust for the learning curve of residents and also for seasonal changes in hospital census and patient mix); and patient age, sex, primary language, and race and ethnicity. Research assistants administered several measures at the time of patient enrollment, including of cognitive status (mini-Cog score),³² health literacy (Short Test of Functional Health Literacy in Adults [s-TOFHLA] score),³³ and functional status 1 month before admission (Medical Outcomes Study 12-Item Short Form Health Survey [SF12]) score per patient self-report), caregiver status, Elixhauser comorbidity score,³⁴ median income by Zip code, inpatient unit, primary care practice, number of emergency department (ED) visits in the previous 6 months, and HOSPITAL risk score for potentially avoidable readmissions (hemoglobin level at discharge, oncology service, sodium level at discharge, procedure during the hospitalization, index admission type [elective versus nonelective], admissions in the previous year, and hospital length of stay).^35,36 See Table 1B for details of how we categorized each variable in the final model and the data source for each variable. To summarize, we used administrative data sources when we considered them reliable, and we used patient sources when administrative sources were unavailable (eg, health literacy) or unreliable (eg, race).

Table 1B

Patient Characteristics.

To evaluate the effects of the intervention (independent variable) on nonelective readmissions and preventable readmissions (dependent variables that are dichotomous), we used a similar approach, using multivariable logistic regression. In that model, we were able to cluster by inpatient unit and use primary care practice as a random effect. We used the general linear mixed model (GLIMMIX) procedure in the Statistical Analysis System (SAS) 9.3 statistical package carry out all analyses.

Treatment Analysis

Because we did not expect implementation of the intervention to be perfect, we planned many secondary analyses to evaluate the efficacy of the various intervention components under perfect conditions. First we used documentation in the EMR to determine whether each patient received each of the major components of the intervention (see Table 5 and Results). Then we used the receipt of each intervention as a covariate, 1 at a time, in an otherwise fully adjusted model (except for intervention arm) for each of the major outcomes described above, thus simulating the effects of each intervention component under conditions of perfect adherence. We also created an intervention fidelity score, with the numerator being the number of components received and the denominator being the number of components for which each patient was eligible (eg, if the patient was not referred to the home pharmacist program, that component was removed from the denominator).

Table 5

Intervention Fidelity.

Evolution of the Intervention

During the pilot phase of the study (March through September 2013), during which the intervention was implemented at one practice, and throughout the intervention study period (October 2013 through September 2015), the intervention was iteratively refined in response to input from the Patient Family Advisory Council, the steering committee, and members of the intervention team; cases of adverse events and readmissions from patients in the intervention arm; exit interviews (conducted by PFAC members and the director of the BWH Center for Patients and Families) of patients who had recently completed the intervention; and informal feedback from clinicians, including each primary care practice and several inpatient units at both hospitals. At the same time, the intervention components evolved as a result of factors outside our control, including other efforts to improve transitions at each hospital and at Partners (eg, use of Medicare's Transition Care Management billing codes, which required a phone call within 2 business days of discharge) and limitations in personnel number and type. In general, we tried to standardize the intervention by function across hospitals, units, and practices, using checklists and other tools to maximize reliability and consistency while still allowing for local adaptation. In other words, rather than specify exactly how a task (such as medication counseling) needed to be performed, we gave the sites flexibility regarding how they would implement the task, given their available personnel and institutional culture. One goal of the study was for the PCORI contract to pay for as little of the intervention as possible. The point was for the ACO and PCMH practices to invest in these interventions and increase their sense of ownership of transitional care systems now that incentives are more aligned to improve transitions of care. Also, we did not want the intervention to end once the contract period was over.

Following are brief descriptions of how each intervention component evolved over the course of the study.

1. Inpatient medication safety intervention. The intervention was implemented as designed, but adequately staffing it with inpatient pharmacists was a constant challenge at both hospitals. The hospital pharmacy departments were short-staffed, and intervention pharmacists were often required to cover other responsibilities. At MGH, only certain units agreed to the use of inpatient pharmacists for much of the study period. And because these pharmacists were centrally deployed and not unit-based, they often had difficulty identifying and getting to patients to perform discharge tasks before the patients left the hospital. Over the course of the study, we developed semimanual systems (such as routinely calling unit coordinators where enrolled patients were hospitalized) to proactively identify patients soon to be discharged and in need of patient counseling.
2. Discharge advocate. This component was implemented differently at the 2 hospitals. At BWH, we hired a dedicated nurse practitioner, which led to a high degree of quality control and the ability to perform more tasks, such as using the predischarge preparation checklist to identify and take steps to address barriers to patient self-management at home. The disadvantages included a reduced ability to see every intervention patient because of limited bandwidth. Plans for the nurse practitioner to teach frontline nurses to incorporate these tasks into usual care never came to fruition, in part because of the preparation needed to launch the new electronic medical record at BWH following the end of the intervention period. At MGH, the discharge advocate (DA) role was assumed by attending nurses, an existing special role assumed by 1 frontline nurse each shift, with a focus on patient education and postdischarge planning. This approach enabled us to reach more patients and offered a built-in plan for scale-up, at least in theory, but it led to greater variability in the role and a more limited intervention, focused mainly on communication with the outpatient nurses (ROCs) about each patient.
3. Structured VNA visit. The intervention was implemented as designed, with support from 2 leaders in Partners Healthcare at Home (the network VNA) who served as liaisons between the study and the visiting nurses. However, because the intervention involved only a few patients per nurse, it was difficult for them to develop proficiency. In addition, most patients were not discharged with VNA support, and of those who were, only half had Partners Healthcare at Home as their VNA agency.
4. Responsible outpatient clinician. Each practice identified personnel to assume this role, and each ROC was assigned to a small group of PCPs. The professional certification of the ROCs varied by practice; they included RNs, licensed practical nurses (LPNs), and physician assistants (PAs). MGH used RNs almost exclusively, while BWH used mostly LPNs and some RNs and PAs. The variation in certification influenced how independently the ROCs could manage patients' problems (eg, in response to postdischarge phone calls). Because no practices hired new staff to fill these roles, ROCs were often pulled from other tasks. One practice, the Phyllis Jen Center at BWH, reallocated RNs to conduct postdischarge calls from intervention patients, while calls for nonintervention patients were conducted by LPNs.
5. DA-ROC communication. At BWH, communication was fairly easy because 1 person assumed the DA role. MGH experienced challenges with regard to availability and different communication styles between inpatient and outpatient nurses (ie, outpatient nurses usually sit at a desk and use email, while inpatient nurses move among patient rooms and use phone and pager to communicate, often not checking email until the end of the day). The study project manager at MGH sent an email reminder to both the DA and the ROC to communicate with each other, but this led to diffusion of responsibility. Over time, we refined the procedure, encouraging the ROC to initiate communication on admission by calling the inpatient unit, and encouraging the attending nurse to call or email the ROC before discharge.
6. Postdischarge phone call. Almost all practices at MGH were already making these calls, but they were not always using a standard template to guide the conversation and were sometimes calling only medical patients. We encouraged practices to use a template and to call surgical patients as well, especially if they were part of the study (we alerted practices to study patients by email and an electronic study registry). At BWH, fewer practices were making postdischarge calls, but the hospital's physician organization starting incentivizing practices to do so. As with MGH, we worked to standardize the content of the calls. Transition care management billing codes initiated by the Centers for Medicare and Medicaid Services became the major driver of this initiative, and we synchronized our efforts with those requirements.
7. Postdischarge clinic visits. For almost all practices, PCPs conducted these visits with little help from other personnel, such as ROCs or pharmacists (ie, visits were less team-based than designed). We tried to standardize visit note templates, again using transition care management billing as the driver of standardization. In reality, there was substantial variation in how PCPs conducted and documented these visits.
8. Interventions for high-risk patients.
   1. Home visits by Dovetail pharmacist. This is an area in which Partners provided additional resources, in this case, Partners Population Management. We spent several months working on the logistics of referrals, documenting eligibility, documenting findings, and communicating with providers. Referral rates for the program by inpatient attendings and PCPs was lower than expected, and even when they were referred, many patients declined the program or could not be reached by phone to schedule a home visit. On the other hand, we heard many stories of successful interventions that likely had a profound impact on the patient's postdischarge course.
   2. Congestive heart failure telemedicine program. As with Dovetail, Partners Population Management agreed to subsidize the cost of enrolling any additional patients (who would not otherwise have been eligible) in this program. In reality, very few patients were enrolled—many patients were not eligible for Partners Healthcare at Home (or already had a different agency) or were enrolled in a different congestive heart failure program. The BWH heart failure service was hesitant to enroll patients in our study because of concerns about conflicts with studies they were conducting.
   3. Advance care planning. The automated trigger tool identified very few patients, so we shifted to a referral system initiated by an email to inpatient attendings and PCPs (“Would you be surprised if this patient passed away in the next 6 months?”). Very few patients were identified this way either, even when we subsequently changed the time horizon of the question to 2 years.
   4. integrated Care Management Program. The iCMP, funded by Partners Population Management, was the most robust program already in place to improve transitions of care. We enrolled iCMP patients in our study so we could evaluate the efficacy of that program. To avoid redundancy and patient confusion, we did not provide too many additional services, but we did offer the Dovetail program (iCMP did not provide this service) and the inpatient pharmacist program for MGH patients (not provided at MGH).
9. Information technology.
   1. The web-based discharge-ordering module was implemented as planned.
   2. The automated system to alert providers of the results of tests pending at discharge was also successfully deployed at BWH. The system facilitated closed-loop communication, information transfer, acknowledgment, and transfer of responsibility for the results of tests pending at discharge among responsible inpatient and ambulatory providers during the postdischarge period. The system was never adopted at MGH.
   3. Improvements were made to the Partners Enterprise Patient List application around the time the intervention started. These included functionality to automatically or manually add all members of a patient's care team. We conducted a quality improvement initiative to improve care team identification using PEPL; however, all team members were not entered into the system owing to workflow constraints and cultural issues at both hospitals.²⁶ We set up working groups as part of the quality improvement initiative at both sites to address these issues, with limited success. Use of the group email functionality was modest.
10. Home-based coaching. Despite the fact that this intervention is more evidence-based than most (eg, from the Coleman Care Transitions Intervention), it never received the internal funding it needed to be deployed, despite several efforts (eg, providing it as an additional per diem service of Partners Healthcare at Home, linking it to local community-based organizations, proposing the use of community health workers).

Enrollment and Study Flow

We enrolled 18 PCMH primary care practices to participate in the study, including 8 from BWH (out of 13 approached), 8 from MGH (out of 11), and 2 from non-BWH-MGH Partners practices (out of 9). We also enrolled 2 pilot practices, 1 each from BWH and MGH. Reasons given for not participating included not having dedicated personnel to assume the role of the ROC, recent turnover in practice leadership, and not enough patients admitted.

Figure 4 shows the flow diagram for patient screening and enrollment. Reasons given for not enrolling patients included being unable to complete the screen before discharge, not meeting inclusion criteria or meeting exclusion criteria, assignment to a pilot practice, and patients declining informed written consent.

Figure 4

Study Flow Diagram.

Table 1A shows the enrollment rates (number of patients enrolled per month) and dates of starting enrollment, moving from usual care to the intervention (as randomized and in actuality), and ending enrollment, by practice.

Table 1A

Enrollment Rates and Key Dates of Study Period, by Practice.

Patient Characteristics

Table 1B shows the characteristics of the patients in the 2 arms of the study. Compared with usual care, patients in the intervention arm were less likely to be non-English-speaking (ie, Spanish-speaking, given the inclusion criteria) or to have a caregiver to help them with basic or instrumental activities of daily living. Also, technically, patients in the intervention arm had a statistically significantly higher baseline functional score, but the magnitude of these differences was so small as to be clinically meaningless; statistically significant differences in education level were also seen, again with differences so small as to be clinically meaningless. Overall, the arms were well balanced, especially considering that different primary care practices had different patient populations and spent different amounts of time in the 2 arms of the study.

Baseline Clinician Surveys

Response rate of the baseline clinician survey was 58%. Selected results are shown in Table 3. Many—if not most—clinicians believed that they did not have sufficient time to perform transitional care tasks and that they had received insufficient training or feedback on their performance in this area, including education of patients. They also gave mediocre assessments of the quality of transitions of care from their vantage point. Their scores on the patient safety culture survey were also mediocre.

Table 3

Selected Results of Provider Surveys.

Inpatient and Outpatient Inventories of Transitional Care Tasks

In general, we found inconsistent performance of most transitional care tasks in both settings. We also found tremendous variation by practice and by unit, based on the surveys of physician and nurse leadership (Tables 4A, 4B, and 4C). For example, for most tasks in the inpatient setting, some units never performed the task while others performed it most or all of the time. In most of those cases, there was a lack of role clarity in terms of who was supposed to perform that task. The tasks are listed in Box 1.

Intervention Fidelity

Table 5 shows the frequency with which different interventions were delivered to patients in the intervention arm, based on review of documentation in the electronic medical record. In general, most patients did not receive most intervention components, even those that were supposed to be delivered to all intervention patients. Not surprisingly, the rate of receipt of interventions was even lower for components reserved for selected patients. Most patients were not discharged with VNA services (75% of BWH patients and 72% of MGH patients), and of those who were, many (almost half) received services from a different agency than Partners Healthcare at Home (because of location, need for certain services, insurance issues, past relationship with a different agency, or patient or case manager preference). Only a small percentage of patients (20% of BWH patients and 15% of MGH patients) were referred for the Dovetail home pharmacy program by their inpatient or outpatient providers, even with prompting from our project manager, and of those who were referred, only a little over half accepted (this proportion did increase somewhat over the course of the study with changes to our logistics and to patient messaging, ie, to remove any stigmatization of referral to the program).

Quantitative Outcomes

Table 6 shows the results of the main quantitative outcomes. The intervention had no effect on readmission rates or on preventable readmission rates in either unadjusted or adjusted models. However, for new or worsening signs or symptoms within 30 days of discharge, while we saw no effect in unadjusted analysis, we found a 22% relative reduction in fully adjusted models; this effect was robust to many different models (results not shown). In addition, the intervention was associated with a significant 48% reduction in postdischarge adverse events and a significant 63% reduction in preventable postdischarge adverse events in fully adjusted and clustered models, and similar effects in the unadjusted models.

Table 6

Quantitative Outcomes.

Regarding postdischarge functional status (adjusted for preadmission functional status), we found a small and borderline significant difference in the intervention arm compared with usual care in the unadjusted analysis, but this difference essentially disappeared in the adjusted analysis.

Table 7 shows the rates of postdischarge adverse events in the control and intervention arms by type of event. Significant reductions were noted in 2 types: adverse drug events and procedural complications. Table 8 shows examples of adverse events noted in the control arm that were judged to be preventable by the adjudicators and thought to be potentially addressable by the intervention, thus demonstrating how the intervention might theoretically have achieved the outcomes noted above.

Table 7

Rates of Adverse Event Type, by Arm.

Table 8

Examples of Adverse Events in the Control Arm That Might Have Been Prevented by the Intervention.

Subgroup Analyses

We ran many prespecified subgroup analyses to determine the effects of the intervention on various patient populations. Table 9A shows the subgroup analyses on readmissions; Table 9B shows the results on new or worsening signs and symptoms, Table 9C for postdischarge adverse events, and Table 9D on preventable postdischarge adverse events. To determine the statistical significance of this effect modification, we ran interaction terms in multivariable models (subgroup × arm); these models required fewer covariates in order to converge. No effect modification was seen in the subgroup analyses of readmissions, new or worsening symptoms, or postdischarge adverse events. In the analysis of preventable postdischarge adverse events, we found borderline evidence of effect modification, with patients on the surgical service, at BWH, and with an Elixhauser score less than 5 seeming to benefit more than other patients from the intervention.

Table 9A

Subgroup Analysis for 30-Day Readmissions.

Table 9B

Subgroup Analysis for New or Worsening Symptoms and Signs.

Table 9C

Subgroup Analysis for Postdischarge Adverse Events.

Table 9D

Subgroup Analysis for Preventable Postdischarge Adverse Events.

Patient Survey Results

In unadjusted analyses, the intervention was associated with few markers of patient experience, at least in intention-to-treat analyses. Surprisingly, the intervention was associated with a lower likelihood of agreeing or strongly agreeing with the statement “After I left the hospital, I was able to follow the diet they ordered for me.” This result might have occurred because of a low response rate for that question (49%) or because patients were now more aware of what that diet was without having been given sufficient resources to help them adhere to it. On the other hand, we found positive associations with 1 question and 1 statement that were of borderline significance: “How often did [the medical team] ask if you might have problems actually doing the recommended treatment [after discharge]?” and “After I left the hospital, I knew how to contact my doctor if I needed to.” In adjusted analyses, these associations were no longer statistically significant, although the diet question remained of borderline significance (Table 10).

Table 10

Effect of Intervention on Patient Experience.

Qualitative Analysis

Eight focus groups were conducted, involving 21 clinicians (Table 2). The 2 coders reviewed the themes they had documented and resolved discrepancies. During our focus groups of clinicians, we identified several themes related to barriers to and facilitators of implementation of the intervention. Box 2 lists the themes, while Box 3 provides illustrative quotes for selected themes. Not surprisingly, the barriers and facilitators were often opposite characteristics of the same attributes (eg, lack versus presence of institutional commitment to improving care transitions).

Table 2

Focus Groups.

On-Treatment Analysis

Using the intervention fidelity data from Table 5, we were able to correlate the receipt of each intervention component with outcomes, producing an on-treatment analysis. By definition, control patients were considered not to have received these interventions and so were grouped with intervention patients who did not receive each component, compared with those intervention patients who did. The components were put into the model 1 at a time, along with covariates, to avoid collinearity. The results are summarized in Table 11.

Table 11

On-Treatment Analyses.

Interestingly, a PCP visit within 14 days was associated with higher rates of readmission, new or worsening symptoms, and preventable adverse events. We saw a similar but nonsignificant trend for postdischarge phone calls and nonsignificant trends toward improved outcomes among those who received the inpatient pharmacist intervention. Finally, we saw a trend for the discharge advocate to be associated with better outcomes at BWH, while the DA was associated with worse outcomes at MGH.

Mixed Methods

We performed several analyses to correlate structure, process, and outcome, especially as they pertained to the ability to carry out interventions and the success of those interventions. Table 12 shows an example—a correlation between the inpatient and outpatient inventories (ie, capacity to carry out transitional care interventions in each inpatient unit and outpatient practice) and patient outcomes. We saw no statistically significant findings but a few of borderline significance; for example, a correlation between the number of transitional care interactions typically performed at postdischarge PCP visits with fewer new or worsening symptoms.

Table 12

Effects of Inpatient and Outpatient Inventories on Readmission and Adverse Event Rates.

Context for Study Results

In this study as implemented, we found that a multifaceted, multidisciplinary intervention had no effect on adjusted 30-day readmission rates. However, the intervention was associated with an approximately 22% lower rate of new or worsening signs or symptoms in the 30 days after discharge (an absolute difference of about 20 events per 100 patients), a 48% reduction in postdischarge adverse events (our primary outcome, an absolute difference of about 10 events per 100 patients), and a 63% reduction in preventable postdischarge adverse events (an absolute difference of approximately 6 events per 100 patients). The increasing relative reduction for these 3 outcomes in this order would be expected, as each one is successively more sensitive to interventions.

The lack of effect of the intervention on hospital readmissions was likely due to several factors, including lower than expected intervention fidelity and a low proportion of readmissions that were truly preventable. Regarding the latter, on the basis of a recently published multicenter study of 1000 patients (of which Dr Schnipper was senior author) and using a 360° approach and a rather utopian view of how transitions could be delivered, we estimated that only 27% of readmissions are likely to be preventable.³⁷ It might be easier to reduce postdischarge signs and symptoms, postdischarge adverse events, and preventable postdischarge adverse events, which are very important outcomes to patients and caregivers. However, the efficacy of the intervention on these outcomes was also likely affected by low intervention fidelity. We suspect that the effect of the intervention on new or worsening symptoms and on adverse events would have been greater had intervention fidelity been higher. Efficacy was also likely affected by the lack of postdischarge coaching, which is one of the more evidence-based interventions in the literature²² but one that was never adopted by Partners Healthcare. Another factor that may have affected the success of the intervention is the fact that some practices started the intervention later than their assigned start date, and we conducted an intention-to-treat analysis based on the assigned date. But these delays were relatively small and affected only a few practices, so the overall effect was likely small.

To the extent that the intervention was successful, we believe (on the basis of exit interviews of patients and caregivers conducted by the PFAC, as well as the focus groups of clinicians) that its effects were attributable to several features, including improved patient/caregiver engagement in the hospital, improved communication between inpatient and outpatient clinicians, pharmacist interventions to improve medication safety, and perhaps better postdischarge follow-up.

Putting the results of this study in context, we should note that the literature on interventions to improve the transition of care is confusing.¹² Although several studies report successful interventions, many—often using similar components—report unsuccessful ones. Many recent studies have focused on single-component interventions, looking for a “silver bullet” that is effective, easy to implement, and relatively inexpensive. Almost all of them have been unsuccessful. Our bridge diagram of the ideal transition in care is a good analogy for many reasons. Like a bridge, a good transition requires many supports—the more supports, the stronger the structure. A silver bullet solution is unlikely for this problem.³⁸ Another conclusion to draw from our study as it relates to the literature is that success is often the result of adequate resources and attention to a thousand details of implementation.³⁹ These interventions are not just pills to be administered. Research of the type conducted here allows us to learn not just whether something works, but how, when, where, and why it works … or does not work.¹⁷ It is also important to keep in mind that readmission is not the only important end point, although the financial implications of readmission often drive these efforts. It might be much easier to reduce postdischarge pain and suffering, the importance of which should not be underestimated.

Generalizability of the Findings

This study demonstrates the potential for multifaceted interventions to achieve their aims within an accountable care organization, the many barriers to successful implementation, and possible ways to overcome these barriers. Threats to the generalizability of the study include its implementation at 2 academic medical centers, although that setting had the required structure given the design of our study (ie, hospitals and patient-centered medical homes integrated within an ACO). Some of the barriers we identified might have been unique to our particular health care setting, but it is likely that many are fairly universal. In other words, our findings are likely generalizable to other academic medical centers within a large ACO.

Implementation of Study Results

Why was intervention fidelity low? On the basis of our focus groups and our inventories of transitional care tasks, we believe a primary factor was the lack of internal resources to pay for several intervention components, which meant that existing personnel were too stretched to conduct them reliably or thoroughly. We saw substantial variation in resource allocation toward these tasks at baseline by inpatient unit and outpatient primary care practice. The lack of resources was likely driven by the existing fee-for-service structure of health care, in which incentives are not completely aligned with preventing readmissions, even with the federal Pioneer ACO program of which Partners is a member. Our mixed methods analysis was unable to prove these hypotheses, but these results might have been subject to the “ecological fallacy” (asking about conditions in general and applying the answers to specific patient interactions). For example, just because a particular primary care practice is well (or poorly) staffed overall does not mean that a particular patient received abundant (or inadequate) care from his or her providers.

This lack of aligned incentives to improve transitions of care in the Partners ACO is notable. In theory, the federal Pioneer ACO program is supposed to reward value of care over volume of services by allowing integrated health care systems to share in savings accrued measured against an expenditure benchmark. However, because the rest of the health care system (at least at Partners, and likely for many others) is still largely fee-for-service, the Pioneer program might not be enough by itself to encourage large health care systems to completely change the way they do business. For example, with few exceptions (such as bundled payments for a limited set of procedures) hospitals are still paid when patients are readmitted. It is likely that a much larger proportion of revenues would need to be at risk (as high as 60% by some estimates) before large health care systems begin to redesign the way care is delivered, focusing on areas such as preventive health, primary care, mental and behavioral health, and transitions of care, where up-front investments can lead to better outcomes and lower health care costs in the future. Of all these investments, those in transitions of care might accrue benefits most quickly.

Patient-centered medical homes are starting to change their structures to promote patient self-management, care coordination, and other goals consistent with this intervention. For some advanced PCMHs, it could be rather easy to adopt the goals of the study's interventions that are under their purview. However, if PCPs are still paid for office visits (and little else, as is largely the case), it will be difficult for them to spend additional time communicating with or visiting patients while they are in the hospital, communicating at length with hospital-based providers, calling patients or visiting them at home, or doing other things that improve transitions of care. Furthermore, PCMHs often have little direct control over hospital personnel and, therefore, their ability to carry out the inpatient side of the intervention or to coordinate with outpatient providers. If hospitals and PCMHs are part of the same ACO, greater potential exists to improve this situation, but only if the ACO is properly incentivized.

To be fair, some barriers to implementation had little or nothing to do with resources and bandwidth. For example, because any given clinician had only a few intervention patients, it was difficult to build the intervention into the workflow or even identify which patients needed it. We had hoped that the intervention would become the new standard of care, but this did not happen during the study period, owing in part to staffing limitations but also because of logistical challenges. In addition, we may have had “too many cooks” implementing the program, creating additional coordination needs and causing problems with quality control. Having fewer clinicians delivering the intervention might have been more effective. In fact, our PFAC told us that patients prefer having 1 point person to rely on, but we did not incorporate this advice into our intervention as much as we could have. We spread the intervention over several clinician types to minimize the additional burden on any of them, to minimize additional costs, and to play to each clinician's expertise, but this may not have been the right approach. While hiring a single additional person to perform the discharge coach role for a group of patients might seem more expensive than asking existing personnel to do a little more, in the end all additional tasks come with a cost, whether hidden or not, and the former approach might ultimately be more effective.

We encountered competing programs, such as the integrated Care Management Program, and competing priorities, such as early day discharges and short lengths of stay, both of which save money and increase revenue under the current payment system. Logistical challenges included enrolling patients in the home pharmacist program, finding a good way for inpatient and outpatient nurses to communicate with each other (outpatient nurses use email and are stationed at a computer most of the day, while inpatient nurses communicate by phone or pager and move from patient room to patient room most of the day), and identifying patients about to be discharged in time for inpatient pharmacists to provide counseling. Preparation for a new electronic medical record at BWH diverted leadership (not to mention time, resources, and energy) away from other quality improvement and patient safety initiatives. And the implementation lacked information technology programs that could have kept everyone on the same page regarding the status of each patient (eg, recently discharged, in need of a phone call) or the intervention itself (eg, received inpatient pharmacist counseling).

Finally, while we did try to incorporate PFAC input into the design of the intervention, we could have involved the council members earlier in the process and incorporated their input more thoroughly, which could have increased the efficacy of the intervention. For Partners, much work is needed to improve transitions of care, but this study has provided many lessons and some needed momentum to continue the process. Possible actions include the following (including an evaluation of their efficacy, taking into account all the implementation lessons learned from this study):

1. Inpatient nurses incorporate the discharge advocate role into usual care, including communicating with outpatient nurses, using the discharge preparation checklist, identifying needed services tailored to each patient, coaching patients and caregivers and conducting motivational interviews.
2. More inpatient pharmacists (ideally unit-based and already familiar with the patients and with the other members of the care team) provide a variety of medication safety interventions at the exact time they are needed.
3. More outpatient nurses (ideally RNs) complete transitional care tasks, including postdischarge phone calls as well as interprofessional office visits and follow-up coaching interventions.
4. More support for home-based coaching; for example, community health workers for high-risk patients (eg, those who have trouble with the discharge readiness checklist).
5. Build enhancements into Partners' new EMR (Epic) to better support transitions of care, including improved templates for postdischarge calls and visits, a dashboard to track patients throughout the continuum of care and to track interventions that have been performed, reports to track tasks still requiring completion, and risk stratification tools to identify patients in need of additional interventions.
6. Consolidate the discharge coach role across as few people as possible. For example, roles 1, 3, and 4 could all be done by the same person for a given patient.

Many of these actions will require concerted efforts regarding staffing, training, policies, procedures, and workflow.

Moving beyond Partners, we need to consider what it will take to improve transitions of care in the United States. A concerted research effort is required to continuously evaluate what we already know, the implications of that knowledge, and what studies still need to be done. Any implications for how health care is organized and financed should be made explicit and effectively communicated to the public and to policymakers. Lessons learned regarding implementation and sustainability should also be effectively communicated to the appropriate stakeholders, including leaders of hospitals, primary care practices, and integrated health care systems. Requirements in staffing, training, workflow, policies, and technology will have to be effectively disseminated. And where our metrics or risk adjustment methods fall short, new metrics and methods will be required. We are proud to be part of PCORI's Transitional Care Evidence to Action Network, which is beginning this journey.

Subpopulation Considerations

The results of the subgroup analyses were less helpful than we hoped. However, some of the findings of borderline significance—such as a possibly greater effect of the intervention on surgical patients, younger patients, and patients with fewer comorbidities—were unexpected and deserve further exploration in future studies.

Study Limitations

Our study had several limitations. For example, our on-treatment analyses were limited by variable amounts of confounding by indication, not to mention co-interventions and natural variation by inpatient unit and outpatient practice. This limited our ability to accurately determine the efficacy of each component.

Second, our measure of readmission is imperfect, as we do not have statewide or national data (which take over a year to become available). However, our combination of administrative data for Partners readmissions plus self-report for non-Partners readmissions is as accurate as we can make it with the data we have, and has been shown to be fairly reliable in previous studies conducted at Partners.⁴⁰

Third, our outcome adjudicators were not blinded. We did not tell them the exact nature of the stepped wedge (ie, which practices went live with the intervention at which times), but we cannot exclude the possibility that they noted some of the intervention components while conducting chart review; we could not blind them to admission dates, and it is possible that they vaguely understood that later hospitalizations were more likely to involve patients in the intervention arm. Also, we cannot exclude the possibility that the adjudication process slowly changed over time, and this could lead to bias because later cases were more likely to involve intervention patients.

Fourth, while the relatively short study period (no more than 22 months for any given site) and adjustment for study month should mitigate most concerns about our results being confounded by general improvements in transitions over time, we cannot exclude the possibility of confounding, as most intervention patients were admitted later in the study period (by design), and our model might not have accounted for some nonlinear improvements in health care delivery over time.

Fifth, while we adjusted for several social determinants of health at least indirectly (eg, health literacy, cognitive status, functional status, need for a caregiver, and median income by Zip code), we did not capture information on other social determinants, such as access to transportation, money for prescriptions, healthy diet, and getting enough sleep. The impact of these factors on our study outcomes is important but was not the focus of this study. As for effect modification by these factors, we purposely limited the number of subgroup analyses, as is commonly recommended, and did not choose social determinants (other than health literacy, an indirect measure) because they are often harder to collect and we wanted to specify patients who were most likely to benefit from the interventions and who could be easily identified by most health systems. Nevertheless, this is a limitation. Social complexity (as opposed to medical complexity) might identify patients most likely to benefit from certain transitional care interventions; therefore, this should be explored in future studies.

Sixth, because the intervention was often implemented over time rather than all at once in each practice, the intention-to-treat analyses biased toward the null. Also, we did not collect baseline data on the primary outcomes for each unit or clinic to determine margins for improvement or for adjustment for confounding.

One potentially concerning finding is that the effect of the intervention on new or worsening signs or symptoms was seen only in the risk-adjusted model. This raises the possibility that the findings are simply an artifact of our risk-adjustment strategy. However, we knew from the outset that the 2 arms of the study were likely to be different, because each primary care practice, with its own patient population, spent different amounts of time in the 2 arms of the study because of the stepped-wedge design. In addition, the effects of the intervention were robust to the number of predictors in the model. We ran a model with only 6 covariates and then the full model with 14 covariates, and the results were virtually the same, suggesting that the results are not just an artifact of our particular risk-adjustment strategy. Plus, the intervention had an even greater effect on postdischarge adverse events—new or worsening symptoms judged to be the result of medical care. Thus, we believe these effects are real.

The on-treatment analyses were likely affected by variable effects of confounding by indication (ie, when clinicians are concerned that a patient is at high risk for poor postdischarge outcomes, they are more likely to implement transitional care interventions, such as an early follow-up appointment). For example, we knew in advance that the inpatient pharmacist intervention was the least likely to be affected by confounding by indication: The pharmacists saw intervention patients whenever they could, limited by their availability. The same was true for the discharge advocate at BWH (a single nurse practitioner co-investigator). Most of the other interventions—including the discharge advocates at MGH, the postdischarge phone calls, and the postdischarge visits—were delivered by personnel outside our direct control and were thus more susceptible to confounding by indication. Indeed, the first group of interventions (those least likely to be affected by confounding by indication) were associated with better outcomes, while the latter group were associated with worse outcomes, especially the postdischarge visits. In a separate study conducted at Florida State University (submitted for publication) we also found that early postdischarge visits were associated with an increased rate of postdischarge adverse events, suggesting confounding by indication.

Nevertheless, it was reassuring to see the first group of interventions associated with improved outcomes, even if they were of borderline significance. This lack of statistical significance was likely due to limited statistical power, given that an individual component can be only so effective and each component was delivered to a limited number of patients. It is highly plausible that these several components combined, even if not delivered to every patient, could produce a statistically significant improvement in outcomes when analyzed in a manner less subject to confounding (ie, the stepped-wedge methodology, with an intention-to-treat analysis based on when each practice was selected to move from usual care to the intervention). This theory is substantiated by Table 8, in which we describe several adverse events that occurred in the control arm; these events were judged to be preventable by the adjudicators and deemed by us to be potentially addressable by our intervention, thus providing biological plausibility for our findings.

Future Research

Several future directions are indicated for this research, for Partners, and for the field of transitions in general. For this research, further exploration is required to better elucidate the connections among environmental context, intervention fidelity, and intervention efficacy. We also want to explore in more depth the natural variation by inpatient unit and primary care practice to understand baseline variation in postdischarge outcomes and the likelihood of benefiting from the intervention. For example, baseline performance might be a result of contextual factors (eg, patient safety culture), structural factors (eg, type and quantity of staffing), or processes of transitional care (eg, how often conducted and by whom).

Likelihood of benefiting from the intervention could be the result of room for improvement (ie, baseline performance, baseline processes), enough staffing to take on new roles, and contextual factors, such as management culture and patient safety culture. Each of these could be correlated with intervention fidelity and then with patient outcomes.