Study Component 1: Literature Review Identifying TC Core Components and Measures

A work group of ACHIEVE investigators that included patients, family caregivers, and stakeholders reviewed TC literature to define a core set of TC components. The literature search updated the literature review conducted before Project ACHIEVE's proposal submission to include newly published articles (January 2013-July 2015) using the terms “Patient Readmission” (OR similar terms) OR “Continuity of Patient Care” AND “Care Transition” (OR similar terms). The work group screened 909 abstracts and reviewed the full text of 303 articles.

Starting with preliminary definitions of core TC components as identified in the original ACHIEVE proposal and through months of deliberation, the work group established an organizing framework based on common categories of problems faced by the target population (Medicare beneficiaries at risk of poor outcomes) and used findings from the updated literature review to update and finalize the core TC components. The team mapped a specific TC case study to the resultant framework as a form of validation. This process yielded the following 8 TC core components published in the research team's article, “Components of comprehensive and effective transitional care” in the Journal of the American Geriatric Society in June 2017.³

1. Patient engagement: Identify what matters to patients, assess their needs, engage them in shared decisions, and foster mutual respect and accountability.
2. Family caregiver engagement: Identify what matters to caregivers, assess their needs, engage them in shared decisions, and foster mutual respect and accountability.
3. Patient education: Implement continuous interactive teaching and learning among health care professionals, patients, and their family caregiver(s).
4. Family caregiver education: Involve family caregivers in decision-making and teach them necessary skills.
5. Patient and family caregiver well-being: Acknowledge patient and caregiver emotions; foster coping skills and support strategies.
6. Complexity management: Anticipate and prevent poor outcomes; ensure optimal medication management.
7. Care continuity: Exchange timely information with providers; ensure appropriate follow-up and resources.
8. Accountability: Define and fulfill team members' roles; support performance improvement.

Limitations: This study component's limitations include that the TC component framework was validated only through 1 case study, the literature review specifically lacked a formal systematic approach, and special attention was not given to subpopulations known to experience higher risk of poor posthospital outcomes (eg, individuals with cognitive impairment, low health literacy). In addition, because much of the TC literature focuses on readmissions, findings from the review itself may not generalize to other patient and family outcomes, though we attempted to minimize this issue through our multimethod approach, which included devising a framework based on common patient/family caregiver problems and concerns.

Study Component 2: Patient and Family Caregiver Focus Groups and Key Informant Interviews

Ensuring representation of ethnically diverse individuals (36% African American, 19% Latino) and rural-dwelling participants (21%), we spoke with 138 patients and 110 family caregivers from 6 health networks across the nation via focus groups and individual interviews. Audio recordings were transcribed and analyzed using principles of grounded theory to identify themes and the relationships among them. The results from this analysis were published by the research team in the Annals of Family Medicine in May 2018 as the manuscript, “Care transitions from patient and caregiver perspectives.”⁴

Patients and family caregivers identified 3 desired outcomes of care transitions:

1. To feel cared for and cared about by medical providers
2. To have a clear understanding of who is responsible for their care plan (ie, unambiguous accountability from the health care system)
3. To feel prepared for and capable of implementing care plans on hospital discharge

Interviewees linked 5 TC services or provider behaviors to these desired outcomes:

1. Using empathic language and gestures
2. Anticipating the patient's needs to support self-care at home
3. Using collaborative discharge planning
4. Providing actionable information
5. Providing uninterrupted care with minimal handoffs

The ACHIEVE research team, scientific advisory council, and stakeholder advisory group reviewed the results from components 1 and 2 and formulated a set of 22 TC strategies through an iterative, modified Delphi consensus-building approach⁵ to be evaluated in aim 2's prospective study.

Limitations: A primary limitation of study component 2 is that we were unable to directly compare patients' perception of the TC components received with those actually delivered by health care providers in order to confirm perceived linkages to outcomes.

Aim 2: Determine which evidence-based TC strategies or groups of these strategies most effectively yield desired outcomes for patients and family caregivers overall and among diverse patient and caregiver populations in different types of care settings and communities.

Study Component 3: Retrospective Analysis

The retrospective analysis, a longitudinal retrospective cohort study, sought to compare trends in readmission rates of hospitals that implemented varying TC strategies during a time when hospitals undertook widespread experimentation attempting to reduce readmissions. To collect data regarding TC strategy implementation, the American Hospital Association (AHA), America's Essential Hospitals (AEH), and Joint Commission Resources (JCR)—all Project ACHIEVE collaborators—distributed a web-based, cross-sectional Hospital Transitional Care Effort Adoption Survey to their US hospital membership (June 2015-March 2016). Apart from federal and psychiatric hospitals, which were not surveyed, membership among these groups provides a comprehensive view of the approximately 5200 short-term acute care hospitals nationwide.⁶ The survey was developed based on national care transition initiatives (eg, the Centers for Medicare & Medicaid Services [CMS] Community-Based Care Transitions Program, the Quality Improvement Organization [QIO] Integrating Care for Populations and Communities, and the Care Transitions Intervention/Coleman) and included 13 TC strategies from these initiatives,^** including questions about the timing of implementing them. We obtained retrospective Medicare claims data (2009-2014) from Research Data Analytics Center (ResDAC) for patients who received care at hospitals completing the survey. TC strategies were grouped through factor analysis, latent class analysis, and literature and expert review. We found high levels of missing data regarding specific implementation dates of TC strategies. We knew that the activation of the Hospital Readmission Reduction Program (HRRP) in 2012 prompted many hospitals to initiate TC efforts in an effort to reduce readmissions. To address the problem of missing implementation dates, we assumed that TC implementation began at some point during the study period (2010-2014) and compared readmission rates at the study onset (quarter 1 of 2010) with its conclusion (quarter 3 of 2014). We used health care utilization in the 12 months before the index hospitalization as a model covariate based on other researchers' readmission models. Associations between the implemented TC strategy combinations and the primary outcome of unplanned 30-day hospital readmission rates were assessed through mixed-effects logit models controlling for patient, hospital, and community covariates (eg, patient comorbidities, hospital ownership of palliative care or skilled nursing facilities [SNFs], and Area Deprivation Index; see Appendix D1.3 for a full list).

There was no sampling frame for the survey, as our partners distributed the survey links to all their members who were nonfederal, short-term general, or specialty hospitals to maximize participant response. Compared with the 4967 short-term acute care hospitals in the 2015 AHA hospital survey, a total of 370 eligible hospitals completed the survey.^*** An analysis of their characteristics revealed concordance with the characteristics of all US hospitals except for a higher representation of academic medical centers (12.2% in our sample vs 5.1% in all AHA hospitals), rural hospitals (40% vs 29.1%, respectively), and >300-bed hospitals (27.3% vs 15.3%, respectively). The sample hospitals varied in the combinations of the 13 TC strategies they implemented, with 303 distinct combinations of the 13 TC strategies reported by hospitals. The prevalence of implemented individual strategies ranged from 78% (transition summary for patients and family caregivers) to 14% (patient/family caregiver TC needs assessment).

We mapped TC strategies into 5 groups based on factor and latent class analyses, including (1) care plan (implemented by 59% of hospitals), (2) shared decisions (31% of hospitals), (3) identification of high risk (62% of hospitals), (4) medication reconciliation (72% of hospitals), and (5) cross-setting information exchange (16% of hospitals). Among these TC strategy groups tested, hospitals implementing cross-setting information exchange (Table ES2) showed the largest decline in risk-adjusted readmission rates during the 4.75-year study period, with a 1.53% absolute reduction (from 15.24% to 13.71%) and 10% relative risk reduction (P < .001). Compared with hospitals implementing at least 1 of the 5 groups of TC strategies, hospitals not implementing any of these groups had a lower initial 30-day readmission rate at 14.28%, and an associated nonsignificant (P = .17) 2% risk-adjusted decline to 14.03%. Of note, hospitals not implementing any of the TC strategy groups differed in some important ways, too, as they had lower proportions of dual-eligible patients or patients with disabilities. Compared with hospitals reporting none of the groups of TC strategies, those using cross-setting information exchange had a greater risk-adjusted decrease in readmission rates over the study period (mean difference in risk-adjusted slopes, 1.28%; 95% CI; 0.36%-2.20%).

Table ES2

Cross-Setting Information Exchange Group and Its Component Strategies.

Limitations: The methodology for this study component was retrospective and observational. Despite controlling for potential confounders (eg, hospital ownership of SNF or palliative care, proportion of primary care providers in the community), due to the observational nature of the study, we are unable to draw strong causal inferences about the relationships found between TC strategy implementation and hospital readmissions. TC strategy implementation data were also self-reported, and 46.8% of hospitals had data missing for TC implementation dates. Given the degree of missing data for specific dates and the HRRP effective date (October 1, 2012), we assumed that implementation occurred during the study period (2010-2014). We also ran sensitivity analyses of hospitals with known implementation dates (n = 197); the results from this analysis showed significantly greater readmission rate reductions for hospitals implementing each of the TC strategy groups than for those not implementing each of the groups. Another limitation relates to the possibility that study hospitals are not representative of all hospitals nationwide; the low participation of hospitals yields the potential for selection bias.

Study Component 4: Prospective Analysis

Using prospectively collected implementation and outcome data, the prospective study sought to evaluate associations of patient exposure to groups of TC strategies with patient-reported and health care use outcomes. We hypothesized that the effectiveness of different groups of TC strategies on patient and caregiver outcomes would vary by patient, caregiver, hospital, and community characteristics.

Methods: After completion of the retrospective survey, a prospective cohort design was employed to collect data about the implementation of an expanded set of 22 TC strategies by 42 collaborating hospitals, and among patients discharged from these hospitals along with their family caregivers when possible. TC strategy exposure was assessed at the patient level where appropriate (6 strategies), and exposure to the remaining 16 TC strategies was measured at the hospital level and then assigned to participating discharged patients from those hospitals (see Table D2.6). We sent hospitals a web-based survey in which they reported their implementation of specific TC strategies. The team validated their responses by site visits. Other TC strategies were measured at the patient level, and patients were coded as being exposed or unexposed to specific TC strategies or groups of TC strategies based on their survey responses. Medicare patients (N = 7939) completed mailed surveys 51 days or more (a time delay mandated by CM)S after hospital discharge, and their family caregivers (N = 2112) completed telephone surveys about their TC experience and outcomes. (Patient exclusions included (1) in-hospital death; (2) transferred to another acute-care hospital; (3) discharged against medical advice; (4) admission for primary diagnosis of psychiatric condition, rehabilitation, or medical treatment of cancer; (5) current prisoner; or (6) currently under suicide watch.) Some family caregivers were recruited at the same time as were patients in the hospital and surveyed approximately 2 weeks after discharge (time 1 [T1]); others were nominated by patients in their response to the mailed survey and surveyed approximately 2 months after patient discharge (time 2 [T2]). Patient survey data were linked to Medicare claims data obtained through ResDAC and through Kaiser claims data via their electronic health record (EHR) system. Five groups of TC strategies were identified using an approach similar to that of the retrospective analysis. General linear mixed-model and logistic regression analyses were used to assess associations between TC strategy groups and health care use (eg, 30-day readmissions, 7- and 30-day ED visits), patient and caregiver experiences, and patient-reported outcomes (PROs).

Table D2.6

TC Strategy Prevalence by Survey.

Ultimately, Project ACHIEVE's measurements of patients' exposure to 5 TC strategy groups revealed varied exposure to strategies and groups of strategies. Among the 7939 patients in the sample, 27.2% were exposed to patient communication and care management; 24.9% were exposed to home-based trust, plain language, and coordination; 26.3% were exposed to hospital-based trust, plain language, and coordination; 39% were exposed to patient/caregiver assessment and provider information exchange; and 6.4% were exposed to assessment and teach back. In addition, 25.7% of patients were not exposed to any of these groups (no-TC group). Importantly, patients in the no-TC group may have been exposed to individual or multiple TC strategies, just not to any of the specific groups listed above.

The hospital-based trust, plain language, and coordination TC strategy group (Table ES3)—which included TC strategies of (1) plain-language communication at the hospital, (2) promote trust in the hospital, (3) medication reconciliation, (4) postdischarge care consultation, (5) identify high-risk patients and intervene, and (6) transition summary for patients and caregivers—was significantly associated with positive results for nearly every patient outcome assessed (Table ES4). To adjust for multiple comparisons, statistical significance was conservatively set at P ≤ .01.

Table ES3

Hospital-Based Trust, Plain Language, and Coordination Group: Required TC Strategies and Definitions.

Table ES4

Risk-Adjusted Associations of Hospital-Based Trust, Plain Language, and Coordination With All Outcomes.

Among family caregivers surveyed 2 weeks postdischarge (T1 caregivers), none of the TC strategy groups were individually associated with improvements in caregiver experience. However, those reporting that they experienced none of the TC strategy groups (no-TC group) had a 47% (95% CI, 0.31-0.53; P < .01) lower likelihood of feeling prepared. In other words, although no specific TC strategy group was associated with more positive outcomes at 2 weeks, caregivers of patients not experiencing any of the 5 TC strategy groups reported feeling less prepared to care for the patient.

These findings were somewhat different among caregivers surveyed >51 days postdischarge (T2 caregivers). Among T2 caregivers, those exposed to the patient communication and care management TC group reported nearly 20% higher likelihood of feeling like health care providers had been there for them (risk ratio [RR], 1.19; 95% CI, 1.10-1.27; P < .001) and a 23% higher likelihood of assigning a rating of “excellent” to the care of their loved one since being home (RR, 1.23; 95% CI, 1.06-1.39; P < .01). In other words, family caregivers who continued providing care more than 2 months after discharge reported better experiences if they had received TC strategies related to plain communication and care management. Among T2 caregivers, unlike T1 caregivers, the no-TC group (not exposed to any of the strategies) was not significantly associated with caregiver experience outcomes.

Limitations: As with the retrospective study, the observational design of the prospective study precludes our proving causality from the associations identified in our analyses. Additionally, exposure to TC strategies was assessed by 2 approaches. For patient-centered TC strategies, patients were deemed the best source to assess exposure (eg, whether the hospital expressed care and concern for patients as individuals). For other TC strategies, implementation and exposure were assessed through surveys of participating hospitals and validated by the site visits. The hospital survey results were then assigned to all patients discharged from that particular hospital. Although the potential for self-report bias exists (ie, hospital staff may overreport strategies implemented by the hospital), we conducted in-person site visits to help validate hospital reports of implementation. The study may be susceptible to potential misclassification bias due to the unknown extent of individual patients receiving a hospital-reported strategy. Again, our site visits aimed to validate the adoption and implementation of TC strategies to minimize the risk of this bias.

Due to unforeseen barriers with potential interference with the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey, we were unable to survey patients until ≥51 days after hospital discharge. Fortunately, as family caregivers do not take the HCAHPS survey, we were still able to collect information about the discharge experience from their perspective starting at 2 weeks postdischarge (T1). However, this delay for patients and T2 caregivers likely increased the threat of recall bias and of selection bias, in that sicker patients may have died before receiving their survey. Finally, selection bias may be an issue, as study hospitals may not be representative of all hospitals nationwide due to our attempt to ensure the inclusion of hospitals participating in national care transition programs and the high representation of Kaiser Permanente hospitals (31% of study hospitals and 46% of the patient sample). However, because our research question was to determine which groups of TC strategies are effective for which populations, it was important that our sample include hospitals that were implementing TC strategies.

Aim 3: Identify barriers and facilitators to the implementation of specific TC strategies or clusters of TC strategies for different types of care settings and communities.

Study Component 5: Provider Focus Groups

Providers who participated in community-based TC programs (eg, CMS Community-Based Care Transitions Program, QIO Integrating Care for Populations and Communities) were recruited to participate in interviews. We conducted 12 focus groups and 15 individual interviews with 63 providers from 13 states with broad geographic diversity who engaged in multiagency, cross-setting collaborative TC efforts to understand which contextual factors influence TC strategy adoption and adaptation to improve the transition from hospital to home or other site of care. Participants represented providers throughout the care continuum (eg, Area Agencies on Aging, hospitals, SNFs).

The findings revealed that the implementation of TC efforts vary based on available resources, community demographics, and interagency collaboration. Specifically, those communities with funding from CMS Community-Based Care Transitions Programs were more likely to implement evidence-based TC models (eg, Project RED [Re-Engineered Discharge], Project BOOST [Better Outcomes for Older adults through Safe Transitions], Transitional Care Model) in their entirety, whereas others relying on institutional support were typically implementing strategies very specific to identified problems (eg, population characteristics). Adaptations were made to integrate TC strategies into workflows or to gradually incorporate strategies in a multicomponent approach while increasing stakeholder buy-in. Providers also noted the importance of identifying key contextual factors that helped or hindered TC implementation. Furthermore, they described the development and implementation of strategies to mitigate initial barriers and sustain programs, as well as how they modified interventions following implementation.

Limitations: Due to the sampling strategy's focus on gathering information from participants engaging in community-level TC efforts, participants may not be representative of all providers. In addition, despite our efforts to ensure that each focus group participant felt comfortable voicing their opinions, it is possible that some participants may have felt hindered by the inclusion of participants from various levels within their organization in each focus group.

Study Component 6: The Provider Survey

The provider survey was designed to understand interagency information exchange, collaboration, and provider experience and assessments of TC. Informed by the provider focus groups (study component 5), the ACHIEVE team developed and administered a web-based survey to 948 providers from across the care transition continuum, including hospital-based providers from the 42 hospitals engaged in the prospective study (ACHIEVE study component 4) and downstream (eg, postacute) and ambulatory providers who also provided health care services to patients discharged from the participating hospitals.

Key findings reveal opportunities for improved communication between hospitals and ambulatory/downstream providers. About a quarter (28%) of downstream and half (50%) of ambulatory providers reported that hospitals informed them of patients' admission to the hospital; less than half of each (46% and 48%, respectively) were informed of their patients' discharge. Two-thirds (66%) of downstream and half (51%) of community clinical providers reported receiving a hospital discharge summary for “all or almost all” of the associated hospitals' patients, indicating that as many as a third of “receiving” facilities and half of ambulatory providers are not receiving basic hospital stay information that is critical for appropriate patient care.

Limitations: Due to study methodology (web-based survey via public link), we do not have response rate data available. We also did not request participant demographics. Because providers included in the study were employees of or TC collaborators with prospective study hospitals, they are also prone to the same potential for selection bias as the prospective study hospitals and may not be representative of all providers nationally. Specifically, we anticipate that participants may be more likely to participate in cross-setting TC programs or to collaborate with hospitals during care transitions than providers nationally may be. Due to this participation, they may be biased to rate hospitals more favorably or, conversely, feel comfortable identifying issues given their involvement in TC initiatives.

Study Component 7: Hospital Site Visits (Phases 1 and 2)

The research team conducted site visits in 2 phases (phase 1, n = 22; phase 2, n = 29). Phase 1 (Scott et al⁷) served as a pilot from which we refined the site visit approach for phase 2 by increasing the duration of the visits from 1 day to 1.5 to 2 days and adjusting the survey tool. Phase 2 site visits were conducted with most (29 of the 42 hospitals in the prospective study; the Kaiser hospitals (n = 13) had participated in phase 1 site visits, with no reported changes in their TC implementation in the intervening time) participants in the prospective study (study component 4) that were recruited through a purposive sampling strategy to ensure a diverse sample of hospitals nationwide (see Table E3.1). In addition to providing information regarding facilitators, barriers, and other contextual influences on TC efforts, and expanding the role of patient and family advisory council (PFAC) members in site visit interviews, we used phase 2 site visits to validate participating hospitals' reporting of their TC strategy implementation in the prospective study (study component 4).^**** Approaches to data collection included (1) interviews with diverse internal and external TC stakeholders, (2) observation of interdisciplinary rounds, and (3) document review. Interviews were recorded and transcribed verbatim. Structured content analyses were conducted on both the phase 1 and phase 2 data sets to generate themes.

Table E3.1

Sites and Locations, Phase 1 and Phase 2 Hospital Site Visits.

Facilitating factors identified in the phase 1 analyses included (1) collaborating within and beyond the organization, (2) tailoring care to patients and family caregivers, and (3) generating buy-in among staff. Barriers included (1) poor integration of TC services into hospitals' standard processes, (2) unmet patient or family caregiver needs, (3) underused TC services, and (4) lack of physician buy-in. Based on experience from the phase 1 site visits, phase 2 site visits were lengthened with modification of the interview guides and the addition of meetings with PFACs. The findings largely validated the results of the phase 1 site visits: Top facilitators included collaboration, communication, and staff and leadership buy-in, while top barriers included poor integration, limited availability of services, and population social needs (eg, transportation, housing, food). Compared with phase 1, health care professionals more clearly voiced the importance of addressing social determinants of health and the challenges in clinical–community linkage, perhaps highlighting the heightened attention to such challenges in the health care environment over the study period.

Limitations: Despite our efforts to hold separate focus groups for senior leadership and for lower staff echelons, it is possible that hospital site visits may have been hindered by the inclusion of participants from various levels of an organization in each focus group. In addition, hospitals included in this sample may not be representative of all hospitals nationally.

Aim 4: Develop recommendations for dissemination and implementation of the findings on the best evidence regarding how to achieve optimal TC services and outcomes for patients, caregivers, and providers.

Research on the dissemination and implementation of effective TC strategies is underdeveloped. As a complex research study tasked with learning what works best in TC, active dissemination and implementation of our findings is not in the scope of Project ACHIEVE. However, to make the next step—translating research findings into practice—more effective, we have outlined dissemination and implementation recommendations to help Project ACHIEVE's main findings reach the stakeholders who may benefit most from them.

The development and publication of interim findings emerging from aim 1 of the study offered an opportunity to pilot test principles and steps in the dissemination plan drafted by the ACHIEVE team. We believe this was an important step to hone these strategies and learn important lessons that could be applied to the dissemination of the study's ultimate findings. Therefore, the team pilot-tested a 6-step strategy based on the Agency for Healthcare Research and Quality's Dissemination Planning Tool to disseminate findings from study component 2 (patient and family caregiver interviews and focus groups). This dissemination strategy included leveraging Project ACHIEVE's extensive network of research partners and stakeholders and customizing messages to each target end user.

The dissemination pilot was successful, as measured by its wide reach. Our efforts resulted in the peer-reviewed publication obtaining an Altmetric score (a weighted count of all of the mentions [eg, social media, mainstream media] tracked for an individual research output [eg, article] and is designed as an indicator of the amount and reach of the attention an item has received) of 226—among the top 5% of research articles rated by the platform—and attention in Kaiser Health News, The Washington Post, CNN Español, and >40 other outlets nationwide. A final dissemination plan was drafted based on the pilot, which will guide dissemination of the final results.

Overall conclusions: Through Project ACHIEVE, we sought to answer 2 primary questions: what matters most to patients and family caregivers during care transitions, and how hospitals can improve care transitions for patients and families. Regarding the first goal, we found that what matters most remains grounded in fundamental needs, despite the growing complexity of the health care environment. In focus groups and interviews, patients and family caregivers reported wanting to feel cared for and cared about as well as to be prepared to implement the postdischarge care plan, and they reported wanting a clear understanding of who is responsible for their care plan. They described numerous activities providers could employ to achieve their desired outcomes, leading us to develop a conceptual framework in which key processes of care (anticipating needs and providing actionable information, engaging in collaborative discharge planning and providing uninterrupted care, and communicating with empathy) can lead to improved TC outcomes.

Consistently, hospitals that did not employ the groups of TC strategies we evaluated in the retrospective and prospective analyses experienced smaller reductions in readmissions or poorer patient outcomes. The Project ACHIEVE findings indicate that providing support for an organized, bundled approach to implementing TC strategies yields improved patient-centered outcomes and experiences. Through this analysis, we found that the group of TC strategies characterized by an emphasis on trust, plain-language communication, and tailored care planning that included both pre- and postdischarge activities (ie, bridging activities) was most consistently and strongly associated with various improved patient outcomes, including reduced health care use and improved experience and PROs.

These findings contribute to the field in several ways. First, to our knowledge, Project ACHIEVE is the largest known prospective TC study, collecting data from 42 diverse hospitals across the nation. In addition, we collected data directly from a representative sample of nearly 8000 Medicare beneficiaries regarding their TC experience, self-reported outcomes, and health care use. While our results largely validate extant research showing positive outcomes associated with care coordination, fostering trust, and clearly communicating with patients and providers across care settings,^8-11 ACHIEVE also shows that, together, these activities address the spectrum of patients' needs during care transitions, resulting in positive self-reported outcomes and reduced health care use. The consistency in our findings across self-reported measures of patient outcomes and objective health care use measures is unique and strengthens their validity. In addition, ACHIEVE's goals, methods, and instruments were developed in partnership with clinicians, patients, family caregivers, policy makers, and other stakeholders. We anticipate these findings to be of particular interest to acute and postacute facilities, health systems, payers, and policy makers as the US health system transitions from the fee-for-service system to value-based care. In particular, health systems may find value in implementing these strategies to ensure that the needs of patients and their family caregivers are met.

Quality Improvement Methods

Project ACHIEVE employed continual quality improvement methodology in its SAG engagement, iteratively adjusting strategies based on active and nonactive member feedback and evaluations. Multiple improvement cycles focused on strategies that responded to 2 SAG member concerns: (1) to be included in and knowledgeable of study activities, and (2) to feel connected to the study's purpose and to each other.

Patients, caregivers, and stakeholders who were on the research team participated in weekly project calls and discussions. For SAG members, we initially engaged them through 3 conference calls and 1 in-person meeting per year, which we maintained over the study period. However, the team modified this strategy to strengthen stakeholders' input into and guidance of Project ACHIEVE. The most effective engagement techniques included the following:

• Generating a stakeholder engagement optimization plan to integrate the SAG into the project work plan, including a roadmap of opportunities for SAG guidance
• Closing the feedback loop with SAG members about how their suggestions informed research efforts
• Creating an “elevator speech” about ACHIEVE that SAG members can use to succinctly describe the project and their role in it to their networks
• Co-chairs conducting regular check-ins with each SAG member regarding their responses to engagement efforts and adjusting engagement strategies accordingly
• Providing a monthly email newsletter including project updates, recent research in care transitions, and personal bios of SAG and research team members to help build community
• Inviting SAG members to participate in research team calls once a month so they could remain engaged in and informed of the project's activities and contribute their ideas and perspectives to the topics discussed

Strong Interpersonal Engagement From SAG Co-Chairs

Carol Levine and Terry Davis, SAG co-chairs and members of the core research team, worked closely with University of Kentucky (UK) staff, setting agendas for calls and meetings. Importantly, they also actively engaged SAG members individually and as a group during phone calls and annual face-to-face research team meetings. They learned to identify members' voices and called them by name on the calls, building important relationships. They solicited thoughts and concerns and asked whether other members had similar or different reactions. They used “teach back” to confirm clear and complete understanding of an issue or decision. Carol and Terry also made individual calls to SAG members to solicit additional advice in their areas of expertise and experience.

SAG co-chairs brought members' ideas to research team meetings and fully credited the member who made the suggestion. For example, 1 member suggested an “elevator speech” to help SAG members describe Project ACHIEVE to colleagues. SAG members and UK staff implemented this idea and updated it after annual meetings. The elevator speech was presented at a national PCORI conference and received a great deal of interest.

SAG Retention

Despite some fluctuations owing to individuals' illnesses or increases in stakeholders' personal or professional demands, we retained approximately 22 to 24 active members throughout the study period. Of the 16 unique organizations represented on the SAG, 2 resigned their involvement due to the need to avoid conflict between professional and advisory roles. Of the 8 stakeholder organizations represented on the research team, all were retained (Table B1a). Of note, a former co-chair of the SAG and member of the research team had to step down due to other professional obligations.

Study Development

From project conception, the ACHIEVE team engaged patients and caregivers as co-investigators on the core research team. The overall approaches are outlined below. More details are provided in Appendix B1.

1. Patients and family caregivers as co-investigators. Two patients shared their lived experience of poorly coordinated care transitions and provided critical input and guidance throughout the proposal development process, while 2 caregivers and advocates actively contributed to research planning conference calls. Their input and engagement directly influenced the formulation of the research questions and study design.
2. National patient/family caregiver survey. With extensive input from patients and caregivers through multiple iterations, the research team developed a survey to solicit feedback on how we were “planning a research project to figure out how to improve the hospital discharge process or when patients go from the hospital to home or another place such as a nursing home.” This survey was completed by >100 patients and caregivers in April 2014. Respondents were asked if they had any “advice for these researchers who want to help patients take care of themselves after they leave the hospital.” We also gained valuable feedback on what patients and caregivers thought about our research plans.

Recruitment

The SAG assisted with recruitment of hospitals, patients, and family caregivers throughout the course of the study, as described below:

1. Recruiting hospitals and health systems to participate. The SAG's involvement enabled ACHIEVE to reach its goals of recruiting diverse hospitals across the country and helped build potential avenues for dissemination and implementation of study findings, as participating hospitals are already invested in the project.
2. Advocating for the inclusion of patient and family advisory council (PFAC) members in phase 2 site visit hospital interview sessions and adding questions about PFACs to the hospital survey.
3. Providing ideas for materials for hospitals to use in their recruitment of patients and family caregivers (postcards, magnets, ACHIEVE posters with contact information of local coordinators). Ultimately, we secured >9000 patient surveys from 42 hospitals across the country and achieved a patient survey response rate of 57%.
4. Providing ideas and input into materials and strategies to improve the recruitment of time 1 (T1) caregivers (fact sheets, times to call, language for recruitment script) when their recruitment was challenging.

Data Collection

The SAG provided guidance and feedback regarding data collection in the following ways:

1. Urging ACHIEVE site visit moderators to assure participating hospitals that site visits were not to evaluate the hospital but to gather information about TC implementation experiences. We adjusted the interview protocol accordingly.
2. Providing input and feedback into patient and family caregiver survey development and editing to improve readability and brevity. The SAG insisted on the survey taking no longer than 15 minutes to complete. They also provided specific feedback on questions to omit from the survey or to rephrase to be clearer. As a result of this explicit guidance, we cut 28 questions from the prospective survey.
3. Providing input into pursuing 2 different time periods for the family caregiver survey (T1, at 2 weeks after hospital discharge; time 2 [T2], after the patient interview approximately 2 months after hospital discharge) as an option, as well as guidance and input into survey content for each time period.

Dissemination

1. Providing input and feedback on the development, design, and messaging of dissemination materials about Project ACHIEVE to be provided to patients, family caregivers, and providers (eg, 1-page summaries targeting each end user).
2. Sharing study findings with diverse stakeholder groups to ensure that the findings reached their intended audiences.

Methods

Ethics

No human subjects were involved in this component of the study.

Results

This review of the research literature guided the ACHIEVE team to refine the definition of TC core components, identify key strategies that providers could use for the implementation and delivery of TC core components, and develop and finalize the metrics that could be used to assess relevant processes and outcomes. The multimethod approach described earlier yielded the identification of 8 TC core components that are summarized below. Table C1.1 provides comprehensive definitions for each component and was a part of a formal report submitted to PCORI. A modified version describing the process and findings was published in the Journal of the American Geriatric Society.³

Table C1.1

Definitions of TC Core Components.

1. Patient engagement: Identify what matters to patients, assess their needs, engage them in shared decisions, and foster mutual respect and accountability.
2. Family caregiver engagement: Identify what matters to caregivers, assess their needs, engage them in shared decisions, and foster mutual respect and accountability.
3. Patient education: Ensure continuous interactive teaching and learning among the health care professionals, patients, and their family caregiver(s).
4. Family caregiver education: Involve family caregivers in decision-making and teach them necessary skills.
5. Patient and family caregiver well-being: Acknowledge patient and caregiver emotions; foster coping skills and support strategies.
6. Complexity management: Anticipate and prevent poor outcomes; ensure optimal medication management.
7. Care continuity: Exchange timely information with providers; ensure appropriate follow-up and resources.
8. Clinical/team/organizational accountability: Define and fulfill team members' roles; support performance improvement.

Methods

Data collection

Focus group and individual key informant interviews were conducted in English and Spanish with patients and their family caregivers separately between March 1, 2015, and March 1, 2016. To increase participation among frail older patients and those with disabilities, some interviews occurred at the bedside or by phone. The interviews averaged 45 minutes in duration, and focus groups averaged 75 minutes. Focus groups had both a facilitator and a note taker. The research team designed the interview guide to capture the hospitalization and care transition experience of patients and family caregivers and to learn from participants what mattered most to them during their journey from hospital to home. This included addressing the patient's time in the hospital, the discharge process, transition from hospital to home, and access to services during and after the hospitalization (see Table C2.1 for sample questions and Appendix C2.1 for full interview guides).

Table C2.1

Patient and Caregiver Interview Guide Sample Questions.

Ethics

Informed consent was obtained from all participants before conducting interviews or focus groups. The medical IRB at UK approved the study protocol, as did IRBs at each partner site conducting interviews, for example, Boston Medical Center, Louisiana State University, University of Pennsylvania, and Kaiser Permanente. UK's IRB served as the IRB of record for Telligen.

Results

Participant characteristics

Overall, 248 participants (138 patients and 110 caregivers) were interviewed from 9 subpopulations. Patients were 61 years old on average; 57% were female; 30% were married; 18% were Hispanic, with 49% non-Hispanic White and 34% non-Hispanic Black; 44% had low health literacy; and 29% screened positive for depression. All were Medicare beneficiaries, and 66% were insured by both Medicare and Medicaid. Caregivers were predominantly female (84%) and were an average age of 56 years (range, 19-81 years); 24% had low health literacy, and 15% screened positive for depression (see Table C2.2).

Table C2.2

Patient and Caregiver Interview Participant Demographics.

Major Themes

Participants identified 3 outcomes as integral to optimal care transitions: (1) clear understanding of who is responsible for their care plan, (2) feeling prepared and capable, and (3) feeling cared for and cared about.

1. Clear understanding of who is responsible for their care plan. Patients and family caregivers want a clear understanding of who is responsible for different aspects of their care throughout their care transitions until they recover or reach a stable state of health. Participants often described this in terms of knowing who on their health care team they could contact if they have problems. One family caregiver described the abandonment she felt when problems arose at home and she did not know whom to turn to for counsel:
   It's like being thrown out in the middle of a lake and… [you're] frantically searching for somebody on the other side of the lake to help you.
2. Feeling prepared and confident of executing the care plan on discharge. Patients and family caregivers want providers to tell them what to expect when they leave the hospital and to be given the tools to care for themselves. They want to be prepared for potential issues and know what to do if they occur. When feeling prepared, participants reported feeling more confident and able to adhere to care plans and felt more trust toward providers, as indicated by one patient:
   They had set up the doctor's appointments, and they filled the prescriptions in the pharmacy downstairs for me… they explained it line by line… that all helped.
   Conversely, it was common for caregivers to report anxiety about being properly trained to provide home care, as was described by one caregiver:
   Somebody should've advised me on what the aftercare was gonna be. If you're gonna hold me accountable to be the nurse, then you need to train me to be the nurse.
3. Feeling cared for and cared about by health professionals. Patients and family caregivers want clear expression by their health care providers demonstrating quality care delivery and compassion that indicates that the providers care about them. It gives them confidence that their health care needs matter. Participants reported poor experiences when empathy and compassion were lacking and unanimously expressed the desire for the health care team to convey sincere concern and commitment to patients' well-being. Specifically, doctors and nurses who knew participants' names and sat down when speaking to them exuded a sense of patience and compassion. However, participants more frequently noted that empathy and support were lacking, sowing doubt and mistrust and engendering a sense that care delivery was transactional instead of compassionate:
   If it weren't for family at my hospital, you would not be cared for… when you call for [the nursing staff], they may or may not come. —Patient

In addition, 5 themes related to key processes of care—defined as care transition services and/or provider behaviors—were identified: (1) providing actionable information, (2) engaging in collaborative discharge planning, (3) communicating with compassion and empathy, (4) anticipating patient and family caregiver needs, and (5) providing uninterrupted care.

1. Providing actionable information. Patients felt capable when information was tailored, understandable, and accompanied by clinical skills training with confirmed comprehension (ie, teach back). Conversely, if staff did not provide training and information that was actionable, participants reported feeling overwhelmed, stressed, incapable of adhering to care plans, and ultimately abandoned:
   We struggled for information every single day…I would've liked more guidance. —Caregiver
2. Engaging in collaborative discharge planning. Involving patients and families in discharge planning not only made patients feel supported, but many participants characterized it as being crucial to ensuring that an appropriate, effective plan was in place. When their input was not sought, however, not only could critical information be missed, but it made participants feel disregarded and disrespected:
   I was very glad that they included me… my involvement… was crucial. —Caregiver
   I felt that the hospital… made decisions for me that I didn't want made and they never consulted me… .it just really upset me that they made all these arrangements without asking me. —Patient
3. Communicating with compassion and empathy. A desire for empathic and compassionate communication emerged repeatedly throughout focus groups and interviews. Such communication was sometimes characterized specifically as patients and family caregivers wanting providers to know their names and sit when talking to them. It was also described as wanting their health care team to convey concern for their well-being, ask questions, and actively listen, as one patient describes:
   [Good communication] is like, “How are you doing? How are you feeling?” And pay attention to what we are saying. And [providers] assume, “Oh no, you have this, you have this, you have this.”
4. Anticipating needs of patients and family caregivers to support self-care at home. Patients and caregivers shared that it was difficult to anticipate all the supplies, medications, treatment regimens, and care they would require after their hospitalization. They looked to their medical team in the hospital as being critical in helping them plan for a safe transition and to help manage the needs they might develop postdischarge:
   It's not until you get into the wilderness at home that you realize… the vastness of what you don't know. —Caregiver
5. Providing uninterrupted care through the point of the patient's recovery. Experiencing a seamless health care experience across the continuum (eg, hospital, home care, skilled care) was desired by patients and family caregivers, who felt that care continuity cultivated a sense of trust and confidence in their medical care. Some participants specifically indicated wanting providers to know the patient's history and acknowledge them as a person. One participant expressed his concerns about the lack of continuity among providers in the hospital compared with the providers he regularly sees:
   Hospitalists are not familiar with you whatsoever… you can't deal with your regular doctor, you have to deal with the hospitalists, and I hate that because… they're not familiar with you, they don't know your health history, anything.

When ACHIEVE focus group/interview participants' desired outcomes were realized, they characterized care as excellent and trustworthy, family caregivers experienced less distress, and reported adherence to discharge plans was increased.⁴ Patients and family caregivers overall shared similar perspectives in terms of desired care transition outcomes. There were, however, key differences between them in the processes of care identified as essential within the themes of collaborative discharge planning and anticipating needs. Family caregivers overwhelmingly emphasized the importance of engaging them in discharge planning. They also stressed the importance of eliciting and anticipating family caregiver needs so they would be prepared and confident to deliver home care and have the resources to successfully implement the care plan.

Figure C2.1 provides a conceptual model illustrating how care transition services and provider behaviors (processes of care) support the desired outcomes identified by participants. For example, when health professionals provide (1) actionable and timely information and (2) uninterrupted care with minimal handoffs, patients and family caregivers are more likely to feel a clear understanding of who is responsible for their care plan. In addition, when health professionals (1) anticipate both patient and family caregiver needs and (2) involve them in discharge planning, both patients and family caregivers are more likely to feel prepared and confident in their ability to execute the discharge plan.

Figure C2.1

Conceptual Model, Relationship Between Provider Behaviors and Services Across the Care Continuum, and Care Transition Outcomes Desired by Patients and Caregivers.

When providers perform these behaviors and communicate with compassion and empathy, patients and family caregivers feel cared for and cared about. Importantly, these processes of care integrate well into the Project ACHIEVE conceptual model (Figure C2.2), in which communication, coordination (eg, uninterrupted care, providing actionable information), and engagement (eg, collaborative discharge planning, anticipating needs) are central to successful TC.

Figure C2.2

Project ACHIEVE Conceptual Model, Adapted From CFIR.

The findings from patient and family caregiver focus groups were used to (1) finalize the list of TC strategies that Project ACHIEVE assesses, (2) refine the phase 2 hospital site visit interview guide, and (3) guide the development of patient and family caregiver surveys to collect data on TC strategy implementation and outcomes.

Lessons Learned

Two major activities were accomplished in aim 1 to identify what matters most to patients and family caregivers during care transitions: (1) a multimethod process by which the extant literature informed a catalog of evidence-based TC core components and (2) in-depth interviews/focus groups with nearly 250 patients and family caregivers (study component 2). Importantly, in their own words, patients and family caregivers highlighted wanting to feel cared about and prepared for care transitions, as well as wanting a clear understanding of who is responsible for their care plan. These desires were reflected somewhat in the literature review as well, through the TC components emphasizing the importance of interactive, respectful engagement and education of patients and family caregivers; the importance of delivering continuity in care; and the demonstration of accountability on behalf of the health care organization for patients' goals of care.

The extant TC literature documents the importance of a variety of specific TC strategies (eg, organizing follow-up care and home services, addressing financial and psychosocial barriers during care, providing teach-back communication, ensuring medication reconciliation, and coordinating between physicians).^{3, 45-48} Our multimethod process of identifying TC core components also suggested the importance of employing diverse strategies specifically targeting common concerns and problems faced by patients and family caregivers and engaging them in the process.³ Nevertheless, patients and family caregivers interviewed in study component 2 repeatedly noted these TC components to be severely lacking or inconsistently delivered, suggesting that our health care system struggles with accountability to patients and delivering satisfactory TC.⁴

Based on these collective findings, the ACHIEVE research team, scientific advisory council, and SAG finalized and evaluated 22 TC strategies in aim 2's prospective study.

Methods

Time frame

Researchers administered the hospital survey from June 2015 through March 2016 and obtained retrospective Medicare claims data (2009-2014) for patients receiving care at hospitals completing the TC implementation survey (see Table D1.4).

Table D1.4

Characteristics of Participating Hospitals Compared With US Hospitals.

Measures

The hospital survey was developed based on national TC initiatives aimed at reducing readmissions, such as CMS CCTP, QIO ICPC, HEN, Project RED, Project BOOST, CTI/Coleman, and TCM/Naylor. Survey respondents indicated whether their organization previously or currently implemented specific TC practices or strategies included in established TC models and/or recommended by professional and scientific organizations. Although the survey asked for specific implementation dates for TC strategies, data were frequently missing for this question (46.8%). Consequently, we assumed implementation to have occurred between 2010 and 2014 (see the “Statistical Model” section below) and conducted a sensitivity analysis using hospitals with known implementation dates (see the “Sensitivity Analyses” section).

We finalized a list of 13 TC strategies for evaluation in the retrospective analysis. TC strategy selection was guided by its presence in the aforementioned TC interventions, as well as findings from ACHIEVE patient and family caregiver interviews and focus groups,⁴ literature review,³ and iterative discussions among members of the ACHIEVE research team, scientific advisory committee (SAC), and SAG.

Data Collection

To efficiently gather TC strategy data from diverse hospitals and health systems nationwide, we administered a web-based survey, using REDCap (https://project-redcap.org), a secure, HIPAA-compliant survey administration platform. Staff from ACHIEVE partner organizations (AHA, AEH, and JCR) directly shared the survey link with their membership. Because our agency partners distributed this public link, we were unable to obtain specific data on response rates. See Appendix D1.1 for the survey instrument.

Data Sources

Other data sources used in this analysis include the following:

1. Medicare FFS claims data (Medicare Provider Analysis and Review [MEDPAR], inpatient, outpatient, carrier, home health, and SNF research identifiable files), obtained through a Research Data Analytics Center (ResDAC) data request
2. Medicare Master Beneficiary Summary File, obtained through a ResDAC request
3. AHA 2015 Hospital Survey, purchased from AHA
4. CMS Hospital Impact File, a publicly available file
5. Dartmouth Atlas of Healthcare Health Service Area and Hospital Referral Region data files, which are publicly available
6. Area Health Resources Files (ARHF), which are publicly available
7. Area Deprivation Index (ADI), a publicly available file using 2010 US Census data files
8. Hospital survey regarding implementation of TC strategies, described above

As our data sources do not identify which individual patients receive specific combinations of TC strategies, we asked hospitals what proportion of their patients were provided with each TC strategy. Only TC strategies that were provided to all or most patients were coded as being implemented. Importantly, 4 hospitals participated in the retrospective analysis as well as phase 1 hospital site visits (see study component 7). We used hospital site visit data to perform a quality check on hospitals' self-reported adoption of TC strategies in the retrospective analysis survey and were satisfied with the accuracy of the responses.

The Medicare claims data contain encounter-level information about health care services received by individual patients, while other data sources contain information about the characteristics and contextual factors of hospitals and communities in which patients received care. For example, AHA data provided hospital characteristics (eg, number of staffed beds, ownership, teaching status, system membership), and the Dartmouth Atlas of Healthcare Health Service Area and Hospital Referral Region provided data on hospital market structure in hospital service areas ([HSAs] eg, number of short-term acute-care hospitals and hospital beds in an HSA). Hospital characteristics and capacity as well as community/environmental system factors are related to both TC strategy implementation and readmissions; therefore, we linked Medicare claims data to hospital- and community-level data using dates of service, location of service, and geographic identifiers to control for potential confounders.

The primary unit of analysis is an episode of care transition for each inpatient hospital stay beginning with an index hospital admission and ending 30 days after discharge; as such, we grouped encounter-level data to the person-episode level. For example, a patient visiting a PCP after hospital discharge represents individual encounter-level data, but in our analysis, it is part of an episode of care transition initiated by the index hospitalization. For each episode, we constructed a dichotomous measure of an unplanned 30-day readmission using the CMS definition. We constructed measures of postdischarge health care use, including the number of days of skilled nursing, inpatient rehabilitation, home health, outpatient physician visits, and ED visits during the episode. Finally, we measured patient characteristics for each episode, including age, sex, race, Medicaid eligibility, the Elixhauser comorbidities present before and on admission, and each patient's community characteristics (based on zip code), including urban/rural designation, poverty rate, and ADI.

Defining TC Strategy Groups

We identified groups of individual TC strategies for analysis using a stepwise process. This included factor analysis (FA) and latent class analysis (LCA) of TC implementation survey data collected from hospitals, informed by the literature and a subject expert review. This yielded grouping of the 13 TC strategies into 5 hospital-level dichotomous variables to indicate whether a hospital implemented a TC strategy group (see Appendix D1.2 for full details).

In brief, we used exploratory FA to identify a condensed set of TC practice combinations implemented by larger groups of hospitals. We estimated unweighted least-squares FA with the 13 dichotomous TC practice strategy variables, using a polychoric correlation matrix and varimax rotation. We referenced findings from Project ACHIEVE patient and caregiver focus groups and obtained expert clinical and research input to determine the final combinations of TC practices through review during regularly scheduled research conference calls and meetings using a modified Delphi approach.⁵ We retained 5 factors with eigenvalues exceeding unity, which we used as indicators of distinct TC strategy groups. We assigned TC strategies with factor loadings approaching or exceeding 0.5 as required elements of a group and TC strategies with factor loadings between 0.2 and 0.5 as optional (see Table D1.1 for the final TC groups). In the final analysis, we also included a group of hospitals (no-TC group) that did not implement any of the 5 TC groups emerging from this process. This group of hospitals may have implemented individual TC strategies but not in the combinations identified in Table D1.1.

Table D1.1

TC Strategy FA Results Evaluated in ACHIEVE Retrospective Analysis.

As an alternative strategy for identifying distinct combinations of TC practices used by hospitals, we conducted an LCA, a nonparametric statistical method resulting in mutually exclusive groups, to classify the TC strategy groups. The 7-class model was selected as a preferred result from LCA based on having the highest Akaike information criterion (AIC) statistic and based on item-response probabilities indicating the best differentiation between classes of hospitals and highest homogeneity within classes. The 7 classes of hospitals identified through LCA corresponded closely with selected groups of TC strategies identified through FA. We tested both approaches in the complete model to gauge the relative effectiveness of LCA vs FA for classifying groups, finding that the FA method performed better in detecting differential trends in 30-day readmissions using the AIC, bayesian information criterion, and J statistics; FA was thus retained for the retrospective analysis (Appendix D1.2).

Risk Adjustment, Confounding, and Covariate Control

We used multivariable statistical models to control for a rich set of patient, hospital, and community characteristics that otherwise may confound the relationship between TC strategies and patient outcomes, such as unplanned readmission within 30 days. Our covariate selection included all of the factors used in the CMS risk standardized hospital-wide all-cause readmission methodology. Additionally, we selected covariates based on existing studies of hospital readmissions and TC outcomes as well as insight gained from ACHIEVE phase 1 site visits⁷ and patient/family caregiver interviews.⁴ See Appendix D1.3 for a complete list of covariates.

• Patient-level covariates: age, race, sex, Medicare eligibility, dual-eligible status, 31 Elixhauser comorbidities, and ADI
• Hospital-level covariates: bed size, academic medical center (AMC), for-profit status, participation in alternative payment models, structure (eg, having rehabilitation service)
• Community-level covariates: proportions of the population who are uninsured, White, in poverty, and have at least a high school education; median household income; and number of primary care providers (PCPs) per 100 000 residents and hospital beds per 100 000 residents

Statistical Model

The primary patient outcome of interest in the retrospective evaluation is the dichotomous measure of 30-day, all-cause unplanned readmission (READMIT). Our base model is a mixed-effects logit model to estimate Pr(READMIT_ijhct = 1) for each patient “i” in episode “j” occurring within hospital “h” and community “c” during year “t”. A random-effects specification accounted for patients clustered within the same hospital, while we estimated each TC combination as a fixed effect. To test for differences in readmission trends over time, we estimated the model using data from 2010 Q1 to 2014 Q3 and added a linear time trend covariate along with interaction terms for each TC combination interacting with the time trend. We summarized model results using average marginal effect estimates from linear probability models rather than odds ratios to compare readmission trends across alternative TC combinations and alternative models.

With the prior 12 months' Medicare FFS coverage criterion and 30-day episode window, our analytic file included the index hospitalizations occurring between January 2010 and September 2014. We define year 1 as January-December 2010 and year 5 as January-September 2014 throughout the report. We chose this comparison window based on ResDAC data availability and the potential policy effects associated with the ACA. The ACA was passed in March 2010, and with it, the HRRP. Although the HRRP effective date was October 1, 2012, the initial performance period was July 2008-June 2011. Therefore, it is likely that many hospitals initiated TC efforts in late 2010 or early 2011. The year 2014 was the most recent data available through ResDAC for the retrospective analysis; therefore, we used it as the end of the study time frame.

Sensitivity Analyses

We conducted a series of sensitivity analyses to assess the robustness of model results, including the following:

1. Stratified analyses for hospitals that were and were not subject to payment penalties under the CMS HRRP
2. A difference-in-difference specification with subgroups of hospitals that reported information on time periods before and after implementation of their TC strategies (n = 197)

TC Strategies and Groups

As noted earlier, we do not know the response rate for the survey because the rate of exposure to it is unknown, a result of our ACHIEVE partner organizations (AHA, AEH, and JCR) directly sharing the survey link with their membership. Hospitals varied widely in their implementation of the combinations of 13 TC strategies (range, 0-13 strategies; mean, 6.36; see Table D1.2a). The per-site implementation of individual strategies ranged from a high of 78% (transition summary for patients and family caregivers) to a low of 14% (patient/family caregiver transitional care needs assessment; Table D1.2b). In all, 303 different combinations of 13 TC strategies were reported by hospitals.

Table D1.2a

Frequency of TC Strategy Adoption.

Table D1.2b

TC Strategies and Activities in US Hospitals Reported in the ACHIEVE Hospital Survey.

Patient Characteristics

Table D1.3 summarizes the demographics of Medicare beneficiary admissions by TC strategy group for years 1 and 5. The sex and race of beneficiaries changed little from 2010 to 2014. Beneficiaries with dual-eligible status plateaued at about 27%; those with end-stage renal disease (ESRD) decreased from years 1 to 5 (from 1.24% to 0.91%). Fewer dual-eligible beneficiaries were exposed to cross-setting information exchange in years 1 and 5 compared with other TC groups, though the proportion was comparable to that of the no-TC group ranging from approximately 22% to 24%.

Table D1.3

Demographics of Medicare Beneficiary Admissions by TC Strategy Group.

Hospital Characteristics

Our final analytic sample of 370 hospitals reflects broad diversity in terms of geographic region, urbanicity, system membership, academic affiliation, and bed capacity and is comparable to hospitals nationwide (Table D1.4). Although the 5 groups are relatively similar across several characteristics, differences do exist (Table D1.5). For example, group 0 (not adopting any TC groups) were more often in the West (27.3% vs 13.2% overall) and less often nonprofit (51.5% vs 62.7% overall). States/territories were categorized into the following regions: Midwest: IL, IN, MI, OH, WI, IA, KS, MN, MO, NE, ND, and SD; Northeast: CT, ME, MA, NH, RI, VT, NJ, NY, and PA; South: DE, FL, GA, MD, NC, SC, VA, DC, WV, AL, KY, MS, TN, AR, LA, OK, and TX; and West: AZ, CO, ID, MT, NV, NM, UT, WY, CA, OR, and WA. In addition, group 5 (cross-setting information exchange) had a lower proportion of hospitals with <100 beds (17.2% vs 33% overall).

Table D1.5

Hospital Characteristics Overall and by TC Group, N = 370.

Primary and Secondary Outcomes

Table D1.6 summarizes the primary and secondary outcomes by study year. In our analytic cohort, the unplanned 30-day all-cause hospital-wide readmission rate decreased from 2010 to 2014 (from 15.27% to 14.44%). The proportion of patients having an outpatient visit within 7 days of discharge slightly increased (from 51.30% to 54.29%), as did that for 30-day mortality (from 4.49% to 5.01%). The average number of days between index and subsequent readmission and average institutionalized days after index did not change, though the average number of days between discharge and first outpatient follow-up visit declined (from 4.21 to 3.71).

Table D1.6

Unadjusted Frequency Distributions for Primary and Secondary Outcome Measures by Year.

Thirty-Day All-Cause Readmission and TC Strategies and Groups

The proportion of patients experiencing a readmission within 30 days of discharge trended downward steadily. During 2010 Q1, 15.36% of patients were readmitted to the hospital for any cause within 30 days of discharge. This rate declined to 14.38% by 2014 Q3, representing a 6.38% relative reduction (0.98% absolute reduction) over 4 years (Figure D1.1).

Figure D1.1

Unadjusted Overall 30-Day All-Cause Readmission Rate Across 370 Hospitals = ~1% Drop.

Intertemporal trends in readmission rates over the study period varied significantly across groups of hospitals based on the combination of TC strategies implemented. Hospitals implementing none of the 5 groups had the lowest readmission rates during Q1 and experienced the lowest reduction of readmission rates, while hospitals implementing any of the 5 groups had greater reductions in readmission rates (Figure D1.2). Hospitals implementing group 5 (cross-setting information exchange) experienced the largest reduction in readmission rates, falling from 15.13% in 2010 Q1 to 13.64% in 2014 Q3 (9.85% relative decline, 1.49% absolute reduction).

Figure D1.2

Unadjusted 30-Day All-Cause Readmission Rate Trend by TC Group Across 370 Hospitals.

Risk-Adjusted Readmission Trends by TC Strategy Group

The multivariate model identified factors associated with increased readmissions. At the patient level, being non-White, male, and dual eligible; having a disability or comorbidity; and having prior use were associated with higher odds of readmissions. At the hospital level, palliative care program ownership, enabling services (eg, health education, case management), and lower competition index scores were associated with lower odds of readmissions. At the community level, higher high school completion rates and more PCPs per 100 000 residents were associated with reduced odds of readmissions, while higher proportions of individuals aged <65 years with health insurance and non-White individuals were associated with increased odds.

Estimates from multivariable logistic regression models confirmed that hospitals implementing group 5 (cross-setting information exchange) experience significantly larger reductions in readmission rates than do other hospitals after adjusting for potential confounders (see Figure D1.3). Specifically, it was associated with a risk-adjusted absolute reduction of 1.53% (from 15.24% to 13.71%) and a 10% relative risk reduction (P < .001) during the 4.75-year period. For hospitals implementing group 1 (care plan), the trending coefficient was 0.00072 (P < .0001), indicating a 6.89% risk-adjusted relative reduction over the study period.

Figure D1.3

Risk-Adjusted 30-Day Readmission Rate by TC Group.

Hospitals not implementing any of the 5 groups of TC strategies had a lower initial 30-day readmission rate, at 14.28%, and an associated nonsignificant (P = .17) 2% risk-adjusted relative decline to 14.03%. Compared with group 0 (no-TC group), the 5 TC groups have a significant decreasing trend, with a type III error of P < .0001. More specifically, those using cross-setting information exchange had a greater risk-adjusted decrease in readmission rates than did the no-TC group. The mean difference in risk-adjusted slopes of the 2 groups (groups 0 and 5) was 1.28% (95% CI, 0.36%-2.20%).

Sensitivity Analysis Results

The stratified analysis for differential effects of TC groups among hospitals at risk vs not at risk of HRRP penalties based on pre-2012 readmission rates showed that each TC strategy group had an independent effect on readmissions, with their declines exceeding those of hospitals implementing none of the groups. In addition, TC strategy groups had an interactive effect that was larger for penalized hospitals. In other words, among penalized hospitals, those implementing TC strategy groups experienced higher readmission rate declines than did those implementing no TC groups.

In the difference-in-difference analysis with the 197 hospitals that reported dates of implementation for TC strategies, the interaction terms of pre-post implementation with TC strategies group revealed a significantly greater decrease in readmissions across all TC groups than that of those not implementing each TC strategy group. See Appendix D1.4 for the results.

We also assessed changes in observation stays over the study period. Although observation stays did increase over the study period—overall, in group 0, and across all 5 TC groups—there were no statistically significant associations in a comparison of the exposure to each TC group vs lack of exposure to each TC group. Therefore, we are confident that the reduction in readmissions in our analysis could not be explained only by the increased observation stays.

Methods

Hospital Participants

We purposively recruited hospitals to ensure that our sample included at least some hospitals with the following characteristics: (1) urbanicity; (2) safety net; (3) critical access; (4) integrated delivery system; (5) alternative payment models (eg, accountable care organizations [ACOs]); and (6) participation in 1 of the following programs: CMS CCTP, QIO ICPC, or HEN, which became the Hospital Improvement Innovation Network in 2016. This purposive sampling originally targeted 40 hospitals; the final sample consisted of 42 hospitals.

Patient Participants

Patient participants were eligible if they were hospitalized on the medical and surgical units at the participating hospitals and had traditional Medicare FFS. Hospital staff at each site approached and recruited eligible patients, who were later contacted by study personnel via mail or phone to participate. Patient exclusion criteria included (1) in-hospital death; (2) transferred to another acute care hospital; (3) discharged against medical advice; (4) admission for primary diagnosis of psychiatric condition, rehabilitation, or medical treatment of cancer; (5) currently incarcerated; or (6) currently under suicide watch. The original target sample size was 12 000 completed patient surveys; this changed to 7500 to 8500 after conducting additional power calculations (see the “Analytic Methods” section).

Family Caregivers

Family caregivers (ie, unpaid friends or family members providing assistance to eligible patients during their hospital stay and/or recovery) were recruited at 2 time points: (1) in the hospital along with patients (T1 caregiver) by hospital staff or by the patient, and (2) when patients completed the survey ≥51 days after discharge (T2 caregiver). Family caregivers were excluded if they were <18 years old or were providing care only in a paid capacity.

Interventions and comparators or controls

Exposure to each group of TC strategies and its relationship with outcomes were analyzed separately. Thus, we compared patients and family caregivers exposed to each TC strategy or groups of TC strategies with their unexposed counterparts; of note, patients may have been exposed to >1 TC strategy group. In addition, patients and caregivers not receiving any of the a priori–specified groups of TC strategies compose the usual care or no-TC group. Patients in the no-TC group may have received ≥1 TC strategies but not the specific combinations defined in the study's TC strategy groups. TC strategies and TC strategy groups are defined in Table D2.1 and Table D2.2, respectively.

Table D2.1

Final TC Strategies and Definitions Measured in Prospective Analysis.

Table D2.2

TC Strategy Groups and Requisite Strategies.

Additional Data Sources

Claims data:

1. 2015-2018 Q2 Medicare FFS claims data (MEDPAR, inpatient, outpatient, carrier, home health, and SNF research identifiable files), obtained through ResDAC
2. 2015-2018 Q2 Medicare Master Beneficiary Summary File, obtained through ResDAC
3. 2015-2018 Q2 claims data from 13 participating Kaiser Permanente hospitals

Hospital demographic data:

1. AHA 2017 Hospital Survey, purchased from AHA
2. FY2018 CMS Hospital Impact File, a publicly available file

Community data:

1. Dartmouth Atlas of Healthcare Health Service Area and Hospital Referral Region data files, which are publicly available
2. ARHF, which are publicly available
3. ADI, a publicly available file using 2010 US Census data files

TC Strategy Measures

Thirteen TC strategies were previously assessed in the retrospective study that were identified from the extant literature and consultation with our stakeholder advisors. TC strategies in the earlier retrospective study were measured solely on hospital-reported implementation. For the prospective study, we were able to build on the list of TC strategies through our findings from ACHIEVE patient and family caregiver focus groups,⁴ the completed TC component literature review,³ and iterative discussions among the core research team, SAG, and SAC. We finalized a list of 22 TC strategies for evaluation in the prospective analysis, 6 of which were more reliably measured from the patient perspective (Table D2.1).

Instrument Development for Primary Data Collection

Data Collection for Patient and Family Caregivers

In brief, during a patient's hospitalization, hospital staff identified eligible patients, recruited them to the study, and obtained HIPAA authorization and contact information for them (and T1 caregiver contact information, if available). Each week, hospital staff sent the research team information about recruited patients and T1 caregivers through a secure online data submission system. T1 caregivers and patients were then contacted via phone and/or mail (see details below) to obtain informed consent and administer the survey. Patients nominated a T2 caregiver on completing their survey. Overall, 42 hospitals recruited 17 638 patients, and 41 hospitals recruited 5031 T1 caregivers for possible participation.

Based on the results from the pilot survey process in which a greater response rate was attained for mailed surveys, patient survey administration followed a 2-wave protocol of direct mailing with telephone follow-up. Researchers sent patients a mail survey packet at least 51 days after hospital discharge, including a cover letter, survey, and $5 cash incentive (the incentive was given on completion of the survey, as specified by their IRB protocol).

Seven days after the initial mailing, researchers sent all patients a reminder postcard; 24 days following the initial mailing, we sent a second survey to nonresponders. Beginning 10 days later, researchers called nonresponders up to 5 times before retiring them. The patient survey average completion date was 75 days (range, 52-259 days) after hospital discharge with a 57%, response rate. Surveys were conducted in English or Spanish depending on the respondents' preference.

T1 and T2 caregiver surveys were administered by phone in English or Spanish with a $5 promised incentive on completion of the survey. Researchers contacted T1 caregivers 14 to 28 days after patient discharge (average completion date of 18 days, with a 28.3% response rate). We contacted T2 caregivers 59 to 259 days after patient discharge (average completion date of 85 days, with a 35% response rate).

Data Collection for Hospital TC Strategy Implementation

We used REDCap (https://www.project-redcap.org), a secure, HIPAA-compliant web-based survey administration platform to efficiently gather TC strategy implementation data from diverse hospitals and health systems nationwide. Surveys were completed by all hospitals recruiting patients and family caregivers for the prospective study. Surveys were accompanied by a 1- to 1.5-day hospital site visit in which at least 2 ACHIEVE investigators conducted focus group interviews with various TC stakeholders. ACHIEVE researchers used transcripts from these site visits to validate hospital survey responses. In some cases, research staff followed up directly with hospitals via phone calls if survey data were missing or contradicted site visit transcripts.

Defining TC Strategy Groups

Multiple conceptual and analytic strategies, which are delineated in Appendix D2.4, informed the development of the TC strategy groups to be evaluated in the prospective study. As planned in the original proposal, we began with the TC strategy groups identified in the retrospective study (see aim 2, Table D1.1) incorporating the additional 9 TC strategies included in the prospective study. At each stage, members of the research team (including experts in statistics and comparative research methodology) reviewed the results in the context of their conceptual relevance to define a priori groups of TC strategies. Specifically, we considered 4 aspects: (1) mapping to the results of the patient and family caregiver focus group findings (study component 2); (2) service settings (eg, hospital based, bridging, home based); (3) considerations for real-world practice (eg, the N in each group must enable comparisons); and (4) sufficient differentiation among groups (eg, minimize overlapping TC strategies among groups). See Appendix D2.4 for a complete summary of this process.

First, FA and LCA were undertaken separately for each data source (eg, patient and hospital). TC strategies tended to group based on the source used to measure them (ie, a hospital survey or patient survey). The results from these analyses (Appendix D2.4) supported the previously identified retrospective study combinations of TC strategies composing group 5 (cross-setting information exchange, renamed patient/family caregiver assessment and information exchange among providers in the prospective analysis), group 1 (care plan, renamed patient communication and care management in the prospective analysis), and group 3 (high risk, renamed home-based trust, plain language, and coordination in the prospective analysis). These analyses supported addition of the TC strategies of identify high-risk patients and intervene to group 4 (medication reconciliation, renamed hospital-based trust, plain language, and coordination).

Next, we ran a finite mixture model (FMM) separately for hospital and patient survey data to model the probability of individuals belonging to each unobserved group and to classify individuals into the groups. FMM was conducted to determine how health care use was predicted by the selected TC strategies and the inferences about how each group behaves. The FMM results using patient data supported adding follow-up appointment and home visits to group 3. Random forest (RF) analyses did not suggest alternate groups; see Appendix D2.4 for the RF analysis results and a full description of the process.

Importantly, because some TC strategies were essentially universally applied (identify caregiver, interdisciplinary approach, and standard protocols), we determined that they did not provide enough discrimination to include in final groups, and we removed them from the prospective TC groups. The TC strategy urgent care plan was also removed from the prospective analysis because it was a component of the TC strategy helpful health care contact. In addition, the PI team concluded that the question defining exposure to the TC strategy of shared decisions did not adequately assess this TC strategy, so we removed it and the TC group based on that strategy (group 2, shared decisions) from the prospective analysis.

Ultimately, 5 groups of TC strategies were identified for final analyses. Table D2.2 displays each group and its required TC strategies. The no-TC group comprised individuals who were not exposed to any of the 5 groups, though they may have been exposed to any number of individual strategies.^* In fact, the TC strategies to which patients in the no-TC group were exposed ranged from a minimum of 5 to a maximum of 15, and patients in all groups may have been exposed to unmeasured strategies. We entered each group as a dichotomous variable in the final model.

Risk Adjustment, Confounding, and Covariate Control

We used multivariable statistical models to control for a rich set of patient, hospital, and community characteristics that might confound the relationship between TC strategies and patient outcomes. We selected covariates based on existing studies of hospital readmissions and TC outcomes, the CMS risk standardized hospital-wide all-cause readmission methodology, insight gained from ACHIEVE phase 2 site visits⁷ and patient/family caregiver⁴ interviews, input from TC experts on our research team, and guidance from the SAC and SAG. See Appendix D2.5 for a full list of covariates.

PRO Covariate List

Patient-level covariates included age (categorical), race (3 categories), sex, ethnicity (Hispanic/non-Hispanic), dual-eligibility status (Medicare/Medicaid), disability, 7 different Elixhauser comorbidities (eg, congestive heart failure, chronic obstructive pulmonary disease), total number of Elixhauser comorbidities, ADI, HCC (eg, cardiovascular, medicine, surgery), and health literacy level.

Patient Experience Outcomes Covariate List

• Age (categorical), race (3 categories), sex, ethnicity (Hispanic/non-Hispanic), dual-eligibility status (Medicare/Medicaid), disability, total number of Elixhauser comorbidities, ADI, HCC (eg, cardiovascular, medicine, surgery), health literacy level, and self-rated physical and mental health status.^50,51

Health Care Use Outcomes Covariate List

• Patient-level covariates: age (and age squared), race, sex, reason for Medicare eligibility (age, ESRD, etc), dual-eligible status, 15 Elixhauser comorbidities (eg, chronic disease, congestive heart failure, peripheral vascular disease), total number of Elixhauser comorbidities, having a comorbidity, ADI, eligible readmission in prior 6 months, hospital service line (eg, cardiovascular, medicine, surgery), number of physician encounters at index admission, number of prior physician encounters within past 6 months, number of inpatient stays in past 6 months, total inpatient days in past 6 months, number of SNF stays in past 6 months, total SNF days in past 6 months, patient distance from admitting hospital, and patient distance from nearest hospital
• Hospital-level covariates: bed size, AMC, for-profit status, teaching-center status, urban/rural by hospital zip code, participation in alternative payment models, hospital system membership, structure (eg, owning a rehabilitation service), Kaiser vs non-Kaiser, Herfindahl-Hirschman hospital competition index, hospitalist staffing ratio, and hospital occupancy rate
• Community-level covariates: in the patient's county of residence, the number of SNF beds, PCPs, and hospital beds per 100 000 residents; and number of community health centers per 1000 residents below federal poverty level

Subgroup Analyses

We performed analyses among the following subgroups, defined as follows:

1. MCCs: Patient with ≥3 Elixhauser comorbidities (excluding obesity, weight loss, fluid and electrolyte disorders, blood loss anemia, deficiency anemia) according to Medicare claims data (Parts A and B)
2. Mental health issues: Patient Medicare claims data (Parts A and B) include diagnostic codes of depression or psychoses.
3. Rural area domicile: Patient zip code registers as having <20 000 population, not adjacent to urban area in the Evaluation Reporting System 2013 rural-urban continuum.
4. Low health literacy (includes limited English proficiency): Patient survey responses of
   1. Was “a little/not at all confident” in filling out medical forms by themselves, OR
   2. Usually asks someone to help them read materials from the hospital, OR
   3. Survey language is non-English.
5. Medicare and Medicaid dual eligible: Patient designated “dual eligible” in Medicare Master Beneficiary Summary file
6. Has disability and is aged <65 years: Patient designated “disabled and younger than 65” in Medicare Master Beneficiary Summary file

Sample Size Calculations

We based original sample size calculations for the patient survey on a 1-way analysis of variance (ANOVA) to compare TC groups and t tests to contrast particular TC strategy groups on continuous outcomes (2 sided, P < .01). A 2-sided t test of any 2 TC groups in a 1-way ANOVA has at least 80% power to detect a standardized effect size of 0.35 when the sample size is 100 patients per TC group (Table D2.3).

Table D2.3

Approximate Sample Sizes Needed per TC Group.

Our target sample size was selected to allow for comparisons within smaller subpopulations of interest. Additionally, the recruitment of patients nested within hospitals might impact the independence of observations, and sample size estimates should also be augmented by the design effect, 1 + (m − 1) ρ, where m is the number of patients per hospital and ρ is the intracluster correlation coefficient (ICC). Data on ICCs in TC intervention studies are limited, but the ICC was expected to be small and the design effect to range between 0.0 and 2.5. We assumed needing approximately 6 TC groups, resulting in an overall sample size of 12 000 patients.

Due to several obstacles outlined in the “Changes to Study Protocol” section, our target sample size was difficult to achieve. Based on the performance of statistical models in the retrospective analysis, we were able to replicate the explanatory power of these models in the prospective analysis to offset the loss in statistical power due to a smaller sample size (see Appendix D2.6 for details). Consequently, we determined that a sample size of 7500 to 8500 patients would be adequately powered to detect differences in outcomes across TC groups (eg, minimal detectable difference of 2.1-2.9 percentage points). We ultimately obtained a sample of 9450 patients. After review, this dropped to 7939 patients after eliminating surveys with <50% of applicable to all (ATA) questions answered.⁵⁷

Given the paucity of research on family caregivers, we did not perform power calculations for the family caregiver sample as part of our original study protocol.

Analytic Methods

For health care use outcomes, we applied a generalized linear mixed model with random effects for hospitals to each of the dichotomous outcomes (eg, 30-day readmission). SAS version 9.4 (SAS Institute, Inc) PROC GLIMMIX was used for analysis following this equation: logit(P(Readmission_ij = 1)) = β₀+β₁ * Patient Demographic + β₂ * Disease + β₃ * Community + β₄ * Hospital+β₅ * TC Group + b_i0 + ε_ij --- eq (1), where i is hospital, j is patients, b_i0 is the random effects across hospitals, ε_ij is the error term, and β0, β1, β2, β3, β4, and β5 are the vectors of coefficients of related covariates.

For the continuous outcome variable of number of institutional days, we used a generalized linear model with zero-inflated negative binomial analysis among those patients who had experienced at least 1 institutional day in the 30 days following discharge. PRO data were collapsed into 3 categories (eg, poor/fair, good, and very good/excellent). Patient experience data were collapsed into 2 categories (eg, excellent vs very good/good/fair/poor) based on HCAHPS protocol for distinguishing the most positive or “top box” score⁵⁷ and entered into logistic regression analyses. The lowest category (eg, poor/fair or very good/good/fair/poor) was the reference for regression analyses. On the PRO outcomes, we performed multinomial regression; on the patient experience outcomes, we performed logistic regression analyses.

For all outcomes, we used SAS version 9.4 to first test independent associations of all model covariates with each outcome (bivariate analyses) before analyzing the full model adjusting for all covariates (risk-adjusted analyses). All tests are 2 sided, and when possible, P values and confidence intervals are reported. Our study tested associations across 5 comparison groups. To adjust for multiple comparisons, we used the conservative Bonferroni correction: We set our significance level at α = .05 divided by 5 to equal .01 (P ≤ .01). By using this method, which is more conservative than other methods, such as calculating a false-discovery rate, we feel confident in our assessment of statistical significance.

Hospital Characteristics

Table D2.4 summarizes the characteristics of participating hospitals and compares them with those of acute care hospitals in the AHA and CMS Impact File. Compared with these national samples, Project ACHIEVE hospitals were more often in the West (40% vs 20%), had at least 300 beds (57% vs 19%), were major or minor teaching hospitals (86% vs 51%), and were nonprofit (79% vs 59%). More than half of study hospitals and those in the CMS Impact File were large urban (60% vs 55%). Notably, 13 of the 42 hospitals were members of the Kaiser Southern California health system.

Table D2.4

Characteristics of Study Hospitals Compared With National Samples.

Patient Characteristics

Figure D2.1 shows a flowchart of patients from recruitment to survey administration. Table D2.5 summarizes the demographics of patients in the final analytic file (N = 7939). Just over half of the sample was female (53.4%), with an average age of 72.3 years. About three-quarters of the sample was White, 9.4% were Black, and 14.7% were Hispanic. Over half (59%) had more than a high school education, and most were eligible for Medicare due to age (74%). Patient demographics across groups of TC strategies were relatively similar, although patient communication and care management had a higher proportion of White patients (82%), and the no-TC group had a slightly higher proportion of patients with disabilities (16%) and dual-eligible patients (19%).

Figure D2.1

Patient Participant Flowchart.

Table D2.5

Patient Characteristics Overall and Across TC Strategy Groups.

TC Strategies and Groups

Hospitals varied in the number of TC strategies implemented, and consequently, patients varied in the strategies to which they were exposed. In addition, some of our TC strategies were sourced by patient survey data exclusively (Table D2.6); thus, patient exposure varied even more. The most commonly adopted strategy was identification of caregiver (100% of hospitals and patients), while the least common was teach back for information and skills (35.7% of hospitals reported implementing it, and 25.7% of patients were exposed to it; see Table D2.6).

Ultimately, 5 TC strategy groups were measured, and patients varied in the strategies and groups of strategies to which they were exposed. Among the 7939 patients in the sample, 27.2% were exposed to patient communication and care management; 24.9% were exposed to home-based trust, plain language, and coordination; 26.3% were exposed to hospital-based trust, plain language, and coordination; 39% were exposed to patient/caregiver assessment and provider information exchange; and 6.4% were exposed to assessment and teach back. In addition, 25.7% of patients were not exposed to any of these groups.

Patient Outcomes

Table D2.7a and Table D2.7b describe the frequencies of PROs and health care use outcomes.

Table D2.7a

Frequency of Patient-Reported Health and Experience Outcomes.

Table D2.7b

Frequency of Health Care Use Outcomes.

Approximately half of patients rated the hospital's preparation as excellent (47.9%) and their care at home as excellent (50.4%). Nearly three-fourths said that health care professionals had “definitely” been there for them (72.3%).

Over half of patients rated their mental health as excellent/very good (60.8%). Across all other PROs, fewer than half of patients rated themselves in the top category (Table D2.7a).

Approximately 10% of patients had an unplanned 30-day readmission; 8.6% had a 7-day ED visit, though nearly one-fifth (19.6%) had 1 within 30 days; and 38% had a PCP visit within 7 days (Table D2.7b).

Bivariate Analysis

Bivariate associations of TC groups with outcomes are presented in Table D2.8a, Table D2.8b, and Table D2.8c.

Table D2.8a

Bivariate Associations Between TC Groups and Selected PROs.

Table D2.8b

Bivariate Associations Between TC Groups and Patient Experience Outcomes.

Table D2.8c

Bivariate Associations Between TC Groups and Health Care Use Outcomes.

Patient-Reported Outcomes

Across all PROs, a greater proportion of patients exposed to the TC strategy group hospital-based trust, plain language, and coordination rated their health positively than did those who were unexposed (Table D2.8a) in the following areas: physical health (51.8% vs 40.7%, P < .001), mental health (68.4% vs 58.1%, P < .001), and participation in daily activities (53.7% vs 46.6%, P < .001). Approximately 45.3% of patients exposed to this TC group experienced pain “not at all/once” in the past week compared with 38.2% of those who were unexposed (P < .001).

Patient communication and care management was associated with positive outcomes for all PROs except pain, which was not significant. Patients exposed to this TC group rated their health as excellent in the following outcomes compared with those unexposed: physical health (49.3% vs 41.4%, P < .001), mental health (66.6% vs 58.6%, P < .001), and participation in daily activities (54.9% vs 46.0%, P < .001).

Home-based trust, plain language, and coordination was associated with positive outcomes for all PROs except pain, which was not significant. Those exposed to this group rated their health as excellent in the following outcomes: physical health (49.5% vs 41.6%, P < .001), mental health (66.4% vs 58.9%, P < .001), and participation in daily activities (52.5% vs 47.1%, P < .001).

Compared with those not exposed to patient/caregiver assessment and provider information exchange, those who were exposed rated their mental health as excellent (63.1% vs 59.3%, P < .001) and participation in daily activities as mostly/completely (49.4% vs 47.9%, P < .001).

Not being exposed to any of the TC groups was associated with more negative PRO scores across all outcomes: physical health (34.1% vs 46.8%, P < .001), mental health (51.0% vs 64.1%, P < .001), participation in daily activities (41.9% vs 50.7%, P < .001), and pain (36.8% vs 41.2%, P < .001).

Risk-Adjusted Associations Among Patients

See Table D2.9a, Table D2.9b, and Table D2.9c for risk-adjusted associations of TC groups with outcomes; significant findings are reported in the text below. Risk ratios (RRs) are reported in the body of the report; odds ratios are provided in Appendix D2.7.

Table D2.9a

Risk-Adjusted Associations Between TC Groups and Selected PROs.

Table D2.9b

Risk-Adjusted Associations Between TC Groups and Patient Experience Outcomes.

Table D2.9c

Risk-Adjusted Associations Between TC Groups and Health Care Use Outcomes.

Family Caregiver Results

T1 and T2 caregiver sample flowcharts are displayed in Figure D2.2 and Figure D2.3, respectively; the sample characteristics are displayed in Table D2.10a. In brief, caregivers tended to be female (72% and 69%, respectively), live with the patient (79% and 84%, respectively), be retired or not working (69% and 76%, respectively), and have at least some college education (68% and 65%, respectively).

Figure D2.2

T1 Caregiver Flowchart.

Figure D2.3

T2 Caregiver Sample Flowchart.

Table D2.10a

Family Caregiver Demographics.

Family Caregiver Experience Outcomes

Table D2.10b displays the frequency of each caregiver experience outcome across both T1 and T2 caregivers. Responses from both time points were relatively consistent, though a higher percentage of T2 caregivers reported that caregiving was “a lot easier” now than at discharge (41.9% vs 33.4%, respectively). More than half of both groups characterized the care received since being home as “excellent” (58.8% of T1 and 52.2% of T2) and reported that health care professionals had been there for them (71.1% and 72%, respectively). Although few in either group indicated that caregiving took “no effort” (12.5% and 13.5%, respectively), more than one-third indicated that caregiving was “not at all stressful” (37.3% and 35.8%, respectively).

Table D2.10b

Family Caregiver Experience Outcomes.

Family Caregiver Bivariate Analysis

In the bivariate analysis (see Table D2.11a and Table D2.11b), more T1 caregivers exposed to patient communication and care management rated the hospital's preparation of the caregiver as excellent (57% vs 44%, P < .01) and said that health care professionals had definitely been there for them (80% vs 67%, P < .001), compared with T1 caregivers unexposed to that TC group. More T1 caregivers exposed to home-based trust, plain language, and coordination rated that health care professionals had been there for them (78% vs 68%, P < .01). In addition, more T1 caregivers exposed to hospital-based trust, plain language, and coordination rated the hospital's preparation of the caregiver as excellent (60% vs 44%, P < .001) and said that their caregiving was not stressful (46% vs 34%, P < .01). Finally, not being exposed to any of the groups had negative associations with rating care received since being home as excellent (46% vs 61%, P < .001), rating the hospital's preparation as excellent (30% vs 53%, P < .01), and reporting that health care professionals had been there for them (62% vs 73%, P < .01).

Table D2.11a

Bivariate Associations Between TC Groups and T1 Caregiver Experience Outcomes.

Table D2.11b

Bivariate Associations Between TC Groups and T2 Caregiver Experience Outcomes.

Among T2 caregivers, more exposed to patient communication and care management rated the hospital's preparation as excellent (55% vs 45%, P < .01), rated care received since home as excellent (60% vs 48%, P < .001), and said that health care professionals had definitely been there (81% vs 68%, P < .001), compared with T2 caregivers who were not exposed to that TC group. In addition, more T2 caregivers exposed to hospital-based trust, plain language, and coordination rated care received since home as excellent (59% vs 49%, P < .01), said that health care professionals had definitely been there (77% vs 70%, P < .01), and said that their caregiving was not stressful (41% vs 34%, P < .01). Not being exposed to any of the groups had negative associations with rating care received since being home as excellent (44% vs 55%, P < .01), rating the hospital's preparation as excellent (40% vs 51%, P < .01), and reporting that health care professionals had been there for them (66% vs 74%, P < .01).

Family Caregiver Risk-Adjusted Associations

Most associations lost their significance (P ≤ .01) after adjusting for other covariates (Table D2.12a and Table D2.12b). Among T1 caregivers, not being exposed to any group was associated with 47% reduced likelihood of rating hospitals' preparation as excellent (RR, 0.53; 95% CI, 0.31-0.83; P = .003; see Table D2.12a). RRs are reported in the body of the report. Odds ratios are provided in Appendix D2.8.

Table D2.12a

Risk-Adjusted Associations Between TC Groups and T1 Caregiver Experience Outcomes.

Table D2.12b

Risk-Adjusted Associations Between TC Groups and T2 Caregiver Experience Outcomes.

The results were slightly different for T2 caregivers, among whom patient communication and care management was associated with a 23% higher likelihood of rating the patient's care since being home as excellent (95% CI, 1.06-1.39; P = .008) and a 19% higher likelihood of saying that health care professionals had definitely been there for them (RR, 1.19; 95% CI, 1.10-1.27; P < .001). Assessment and teach back remained significantly associated, with a 22% lower likelihood of feeling that health care professionals had been there for them (RR, 0.78; 95% CI, 0.59-0.96; P = .01).

Patient Results, Subgroup Analyses

Table D2.13 shows patient characteristics across subgroups. The rural group was primarily non-Hispanic (98%) and White (93%). Over half of the dual-eligible and low-health-literacy subgroups had a high school education or less (64% and 63%, respectively); about a quarter (26%) of both groups were Hispanic, compared with a range of 2% to 15% among other subgroups.

Table D2.13

Patient Characteristics Overall and Across Subgroups.

Bivariate Analysis, Patient Subgroups

Significant findings from the bivariate subgroup analysis are reported in the text but not shown in tables.

Risk-Adjusted Analysis, Patient Subgroups

Generally speaking, the risk-adjusted associations among TC strategy groups and outcomes were preserved across many of the subgroups, except among rural residents whose PROs and health care use were not associated with any of the TC groups. The risk-adjusted associations between TC groups and each outcome are presented in detail in Table D2.14, Table D2.15, and Table D2.16 with a summary of the key findings below. RRs are provided in the body of the report; odds ratios are reported in Appendix D2.9. Appendix D2.10 provides a visual summary of significant findings across all subgroups.

Patient-reported health outcomes

Table D2.14a, Table D2.14b, Table D2.14c, Table D2.14d, Table D2.14e, and Table D2.14f report the PRO findings among each patient subgroup. The TC strategy group hospital-based trust, plain language, and coordination was associated with improved PROs among all subgroups apart from rural and dual-eligible patients:

Table D2.14a

Risk-Adjusted Associations Between TC Groups and Selected PROs for Patients With Dual Eligibility.

Table D2.14b

Risk-Adjusted Associations Between TC Groups and Selected PROs for Patients With MCCs.

Table D2.14c

Risk-Adjusted Associations Between TC Groups and Selected PROs for Patients With Mental Illness.

Table D2.14d

Risk-Adjusted Associations Between TC Groups and Selected PROs in the Rural Subgroup.

Table D2.14e

Risk-Adjusted Associations Between TC Groups and Selected PROs for the Subgroup of Patients With Disabilities.

Table D2.14f

Risk-Adjusted Associations Between TC Groups and Selected PROs for Patients With Low Health Literacy.

• Physical health among patients with MCCs (very good/excellent vs fair/poor: RR, 1.20; 95% CI, 1.11-1.29; P < .001), mental illness (very good/excellent vs fair/poor: RR, 1.38; 95% CI, 1.16-1.58; P < .001), low health literacy (very good/excellent vs fair/poor: RR, 1.24; 95% CI, 1.08-1.39; P = .004), and dual eligibility (very good/excellent vs fair/poor: RR, 1.36; 95% CI, 1.10-1.60; P = .007)
• Mental health among patients with MCCs (good vs fair/poor: RR, 1.23; 95% CI, 1.12-1.33; P < .001; very good/excellent vs fair/poor: RR, 1.16; 95% CI, 1.11-1.21; P < .001), mental illness (good vs fair/poor: RR, 1.36; 95% CI, 1.11-1.58; P = .01; very good/excellent vs fair/poor: RR, 1.33; 95% CI, 1.18-1.45; P < .001), and low health literacy (very good/excellent vs fair/poor: RR, 1.23; 95% CI, 1.12-1.32; P < .001), as well as patients with disabilities (very good/excellent vs fair/poor: RR, 1.36; 95% CI, 1.24-1.53; P = .004)
• Participation in daily activities among patients with MCCs (good vs fair/poor: RR, 1.24; 95% CI, 1.11-1.36; P < .001; very good/excellent vs fair/poor: RR, 1.20; 95% CI, 1.11-1.28; P < .001), mental illness (good vs fair/poor: RR, 1.32; 95% CI, 1.08-1.54; P = .01; very good/excellent vs fair/poor: RR, 1.37; 95% CI, 1.16-1.56; P < .001), and low health literacy (very good/excellent vs fair/poor: RR, 1.27; 95% CI, 1.11-1.43; P < .001), as well as patients with disabilities (very good/excellent vs fair/poor: RR, 1.37; 95% CI, 1.08-1.68; P < .001)

The TC strategy group home-based trust, plain language, and coordination was associated with improved mental health and participation in daily activities among all subgroups apart from rural and dual-eligible patients:

• Mental health among patients with MCCs (good vs fair/poor: RR, 1.17; 95% CI, 1.06-1.27; P = .004; very good/excellent vs fair/poor: RR, 1.10; 95% CI, 1.04-1.15; P < .001) and low health literacy (good vs fair/poor: RR, 1.20; 95% CI, 1.05-1.34; P < .001)
• Participation in daily activities among patients with low health literacy (good vs fair/poor: RR, 1.28; 95% CI, 1.11-1.45; P < .001; very good/excellent vs fair/poor: RR, 1.23; 95% CI, 1.10-1.38; P = .01)

The TC strategy group patient communication and care management was significantly associated with the following improved outcomes:

• Mental health among those with low health literacy (good vs fair/poor: RR, 1.21; 95% CI, 1.06-1.35; P = .01; very good/excellent vs fair/poor: RR, 1.19; 95% CI, 1.07-1.28; P = .002)
• Participation in daily activities among those with low health literacy (very good/excellent vs fair/poor: RR, 1.23; 95% CI, 1.07-1.39; P = .004)
• Less pain in the past week among those with mental illness (good vs fair/poor: RR, 1.44; 95% CI, 1.14-1.75; P < .001)

The TC strategy group patient/family caregiver assessment and information exchange among providers was associated with increased risk of poor outcomes among the following patient subgroups:

• Physical health among those with MCCs (good vs fair/poor: RR, 0.87; 95% CI, 0.78-0.96; P < .01; very good/excellent vs fair/poor: RR, 0.82; 95% CI, 0.73-0.91; P < .001) and low health literacy (very good/excellent vs fair/poor: RR, 0.72; 95% CI, 0.60-0.86; P < .001), as well as patients with disabilities (very good/excellent vs fair/poor: RR, 0.62; 95% CI, 0.39-0.91; P = .01)
• Less pain in the past week among patients with disabilities (good vs fair/poor: RR, 0.53; 95% CI, 0.31-0.87; P = .01)

The TC strategy group assessment and teach back was associated with lower participation in daily activities among those with MCCs (good vs fair/poor: RR, 0.72; 95% CI, 0.55-0.91; P < .01) and mental illness (good vs fair/poor: RR, 0.55; 95% CI, 0.30-0.90; P = .01).

Not experiencing any of the groups was associated with declines in the following:

• Physical health among those with MCCs (very good/excellent vs fair/poor: RR, 0.79; 95% CI, 0.69-0.90; P < .001) and low health literacy (very good/excellent vs fair/poor: RR, 0.68; 95% CI, 0.55-0.84; P < .001)
• Mental health among those with low health literacy (very good/excellent vs fair/poor: RR, 0.86; 95% CI, 0.73-0.97; P = .01) and dual-eligible patients (very good/excellent vs fair/poor: RR, 0.74; 95% CI, 0.54-0.94; P = .007)

Patient experience outcomes

Table D2.15a, Table D2.15b, Table D2.15c, Table D2.15d, Table D2.15e, and Table D2.15f display the patient experience outcomes among each patient subgroup. Consistent with the overall analysis, patient communication and management was consistently associated with excellent patient experience outcomes across all subgroups. This trend was mostly consistent with home-based trust, plain language, and coordination, except among rural patients for 2 of the experience outcomes (hospital preparation and health care professionals being there were not significant). Similarly, while hospital-based trust, plain language, and coordination was positively associated with most patient experience outcomes in the overall analysis, among rural and mental illness subgroups, this association only persisted for hospital preparation; among dual-eligible patients, it was associated with positive outcomes apart from health care professionals being there.

Table D2.15a

Risk-Adjusted Associations Between TC Groups and Patient Experience Outcomes: Dual-Eligibility Subgroup.

Table D2.15b

Risk-Adjusted Associations Between TC Groups and Patient Experience Outcomes: MCCs.

Table D2.15c

Risk-Adjusted Associations Between TC Groups and Patient Experience Outcomes: Mental Illness Subgroup.

Table D2.15d

Risk-Adjusted Associations Between TC Groups and Patient Experience Outcomes: Rural Subgroup.

Table D2.15e

Risk-Adjusted Associations Between TC Groups and Patient Experience Outcomes: Subgroup of Patients With Disabilities.

Table D2.15f

Risk-Adjusted Associations Between TC Groups and Patient Experience Outcomes: Subgroup of Patients With Low Health Literacy.

Patient/family caregiver assessment and information exchange among providers was not significantly associated with any patient experience outcome across any subgroup. Significant findings are reported in the text below. A visual summary of significant findings across all subgroups is available in Appendix D2.10.

Patient communication and management was positively associated with every patient experience outcome across all subgroups, including the following:

• Rating hospital's preparation as excellent among patients with MCCs (RR, 1.19; 95% CI, 1.09-1.30; P < .001), mental illness (RR, 1.48; 95% CI, 1.28-1.68; P < .001), and low health literacy (RR, 1.37; 95% CI, 1.21-1.53; P < .001), as well as among rural patients (RR, 1.50; 95% CI, 1.22-1.75; P < .001), patients with disabilities (RR, 1.65; 95% CI, 1.35-1.92; P < .001), and dual-eligible patients (RR, 1.37; 95% CI, 1.12-1.60; P < .001)
• Feeling that health care professionals had been there for patients with MCCs (RR, 1.31; 95% CI, 1.26-1.36; P < .001), mental illness (RR, 1.38; 95% CI, 1.27-1.47; P < .001), and low health literacy (RR, 1.41; 95% CI, 1.33-1.47; P < .001), as well as among rural patients (RR, 1.44; 95% CI, 1.33-1.51; P < .001), patients with disabilities (RR, 1.50; 95% CI, 1.30-1.66; P < .001), and dual-eligible patients (RR, 1.41; 95% CI, 1.28-1.51; P < .001)
• Rating care since home as excellent among patients with MCCs (RR, 1.26; 95% CI, 1.16-1.37; P < .001), mental illness (RR, 1.49; 95% CI, 1.27-1.70; P < .001), and low health literacy (RR, 1.30; 95% CI, 1.14-1.46; P < .001), as well as among rural patients (RR, 1.40; 95% CI, 1.10-1.70; P = .01), patients with disabilities (RR, 1.44; 95% CI, 1.12-1.75; P < .01), and dual-eligible patients (RR, 1.37; 95% CI, 1.13-1.61; P < .001)
• Overall patient composite rated as excellent among patients with MCCs (RR, 1.28; 95% CI, 1.18-1.37; P < .001), mental illness (RR, 1.60; 95% CI, 1.42-1.77; P < .001), and low health literacy (RR, 1.37; 95% CI, 1.23-1.50; P < .001), as well as among rural patients (RR, 1.39; 95% CI, 1.13-1.63; P < .001), patients with disabilities (RR, 1.63; 95% CI, 1.34-1.88; P < .001), and dual-eligible patients (RR, 1.45; 95% CI, 1.23-1.64; P < .001)

Home-based trust, plain language, and coordination was associated with the following:

• Rating hospital's preparation as excellent among patients with MCCs (RR, 1.22; 95% CI, 1.12-1.33; P < .001), mental illness (RR, 1.26; 95% CI, 1.06-1.46; P = .01), and low health literacy (RR, 1.32; 95% CI, 1.16-1.48; P < .001), as well as among dual-eligible patients (RR, 1.44; 95% CI, 1.20-1.67; P < .001)
• Feeling that health care professionals had been there for you among patients with MCCs (RR, 1.23; 95% CI, 1.16-1.28; P < .001), mental illness (RR, 1.29; 95% CI, 1.16-1.39; P < .001), and low health literacy (RR, 1.24; 95% CI, 1.14-1.32; P < .001), as well as among patients with disabilities (RR, 1.44; 95% CI, 1.24-1.57; P < .001) and dual-eligible patients (RR, 1.39; 95% CI, 1.25-1.48; P < .001)
• Rating care since home as excellent among patients all subgroups: those with MCCs (RR, 1.34; 95% CI, 1.23-1.44; P < .001), mental illness (RR, 1.49; 95% CI, 1.27-1.69; P < .001), and low health literacy (RR, 1.37; 95% CI, 1.21-1.53; P < .001), as well as among rural patients (RR, 1.29; 95% CI, 1.06-1.51; P < .01), patients with disabilities (RR, 1.37; 95% CI, 1.08-1.64; P = .01), and dual-eligible patients (RR, 1.55; 95% CI, 1.30-1.77; P < .001)
• Overall composite as excellent among patients with MCCs (RR, 1.30; 95% CI, 1.21-1.39; P < .001), mental illness (RR, 1.28; 95% CI, 1.09-1.45; P < .001), and low health literacy (RR, 1.32; 95% CI, 1.18-1.46; P < .001), as well as among patients with disabilities (RR, 1.44; 95% CI, 1.18-1.66; P < .001) and dual-eligible patients (RR, 1.58; 95% CI, 1.38-1.74; P < .001)

Hospital-based trust, plain language, and coordination was associated with the following:

• Rating hospital's preparation as excellent among patients with MCCs (RR, 1.39; 95% CI, 1.29-1.50; P < .001), mental illness (RR, 1.27; 95% CI, 1.07-1.47; P = .01), and low health literacy (RR, 1.45; 95% CI, 1.28-1.61; P < .001), as well as among patients with disabilities (RR, 1.54; 95% CI, 1.27-1.78; P < .001) and dual-eligible patients (RR, 1.53; 95% CI, 1.29-1.75; P < .001)
• Feeling that health care professionals had been there for you among patients with MCCs (RR, 1.10; 95% CI, 1.03-1.16; P = .007) and low health literacy (RR, 1.14; 95% CI, 1.04-1.23; P = .01)
• Rating care since home as excellent among patients with MCCs (RR, 1.21; 95% CI, 1.10-1.31; P < .001) and low health literacy (RR, 1.28; 95% CI, 1.12-1.44; P < .001), as well as among dual-eligible patients (RR, 1.33; 95% CI, 1.10-1.57; P = .006)
• Overall patient composite rated as excellent among patients with MCCs (RR, 1.23; 95% CI, 1.13-1.32; P < .001) and low health literacy (RR, 1.32; 95% CI, 1.18-1.46; P < .001), as well as among patients with disabilities (RR, 1.41; 95% CI, 1.15-1.64; P < .01) and dual-eligible patients (RR, 1.34; 95% CI, 1.13-1.53; P < .001)

Assessment and teach back was associated with a reduced likelihood of the following:

• Rating the hospital's preparation as excellent among patients with MCCs (RR, 0.64; 95% CI, 0.52-0.78; P < .001) and mental illness (RR, 0.62; 95% CI, 0.40-0.90; P = .01) and among those with low health literacy (RR, 0.66; 95% CI, 0.50-0.85); P < .001)
• Feeling that health care professionals had been there for patients with disabilities (RR, 0.59; 95% CI, 0.34-0.90; P = .01)
• Overall patient experience composite as excellent among those with MCCs (RR, 0.78; 95% CI, 0.65-0.91; P < .001) and mental illness (RR, 0.62; 95% CI, 0.40-0.90; P = .01). Otherwise, it was not significantly associated with patient experience outcomes among specific subgroups.

Finally, and consistent with the overall analysis, a lack of exposure to any TC group was associated with reduced likelihood of excellent ratings across almost all patient experience outcomes for the MCC subgroup, including rating the hospital's preparation as excellent (RR, 0.72; 95% CI, 0.62-0.82; P < .001), feeling that health care professionals had been there for them (RR, 0.86; 95% CI, 0.78-0.93; P < .001), and overall patient experience composite as excellent (RR, 0.78; 95% CI, 0.68-0.87; P < .001). Otherwise, it was not significantly associated with patient experience outcomes.

Health care use outcomes

Table D2.16a, Table D2.16b, Table D2.16c, Table D2.16d, Table D2.16e, and Table D2.16f show risk-adjusted subgroup analyses for health care use outcomes. In summary, the positive associations of hospital-based trust, plain language, and coordination were largely washed out among subgroups apart from it being significantly associated with reduced risk of a 7-day ED visit (RR, 0.62; 95% CI, 0.42-0.89; P = .008) among patients with low health literacy.

Table D2.16a

Risk-Adjusted Associations Between TC Groups and Health Care Use Outcomes: Dual-Eligible Subgroup (n = 1232).

Table D2.16b

Risk-Adjusted Associations Between TC Groups and Health Care Use Outcomes: Subgroup of Patients With MCCs (n = 5155).

Table D2.16c

Risk-Adjusted Associations Between TC Groups and Health Care Use Outcomes: Subgroup of Patients With Mental Illness (n = 1524).

Table D2.16d

Risk-Adjusted Associations Between TC Groups and Health Care Use Outcomes: Rural Subgroup (n = 1136).

Table D2.16e

Risk-Adjusted Associations Between TC Groups and Health Care Use Outcomes: Subgroup of Patients With Disability (n = 941).

Table D2.16f

Risk-Adjusted Associations Between TC Groups and Health Care Use Outcomes: Subgroup of Patients With Low Health Literacy (n = 1892).

Consistent with the overall analysis, home-based trust, plain language, and coordination was not significantly associated with health care use.

Consistent with overall analysis, patient/family caregiver assessment and information exchange among providers was significantly associated with a reduced likelihood of having a PCP visit within 7 days of discharge (RR, 0.74; 95% CI, 0.57-0.93; P = .01) among the subgroup of patients with mental illness.

Assessment and teach back was significantly associated with a reduced likelihood of having a PCP visit within 7 days of discharge among rural patients (RR, 0.09; 95% CI, 0.01-0.64; P = .01).

Not being exposed to any TC group was associated with a significantly reduced risk of a 7-day ED visit among those with low health literacy (RR, 0.58; 95% CI, 0.38-0.89; P = .01), patients with disabilities (RR, 0.38; 95% CI, 0.17-0.82; P = .01), and dual-eligible patients (RR, 0.46; 95% CI, 0.24-0.86; P = .01). It was also significantly associated with a lower risk of a 30-day ED visit among dual-eligible patients (RR, 0.58; 95% CI, 0.38-0.87; P = .006).

Lessons Learned

Prospective analysis

Three of the 5 defined groups of TC strategies evaluated through this study showed consistent trends in their association with a broad spectrum of patient outcomes, including health care use, PROs, and patient experience. These 3 groups, (1) patient communication and care management; (2) home-based trust, plain language, and coordination; and (3) hospital-based trust, plain language, and coordination, were also characterized by an emphasis on patient and caregiver-centered communication and coordination activities (eg, plain-language communication, promote trust, postdischarge care consultation). However, while the associations with patient communication and care management were strongest for patient experience outcomes, and home-based trust, plain language, and coordination was strong for physical and mental health and patient experience outcomes, it was hospital-based trust, plain language, and coordination that showed the most positive associations across all categories of outcomes. See Table D2.17 for the required TC strategies for this group and their definitions.

Table D2.17

Hospital-Based Trust, Plain Language, and Coordination Group: Required TC Strategies and Definitions.

The patient-desired experience outcomes were derived from the patient and caregiver focus groups/interviews in phase 1 of the study, which revealed the importance patients and family caregivers placed on feeling (1) cared for and cared about, (2) prepared to implement the care plan, and (3) having a clear understanding of who is responsible for their care plan. These desired outcomes appeared to be more likely achieved with 3 of the TC strategy groups that were characterized by quality of communication between providers and patients, communication with patients across the transition, and coordination of services after discharge. These findings validate the conceptual framework emerging from our qualitative research with patient and family caregivers in phase 1 of the study. In essence, provider behaviors, such as anticipating patient needs and providing actionable information, providing uninterrupted care and engaging in collaborative discharge planning, and communicating with empathy, may lead to improved TC outcomes (Figure D2.4). Our framework described in the original proposal (Figure D2.5) emphasizing communication, coordination, and engagement anticipates this outcome.

Figure D2.4

Adapted Conceptual Framework for How Patient-Centered Processes of Care May Lead to Improved Transitional Care Outcomes.

Figure D2.5

Project ACHIEVE Conceptual Framework, Adapted From CFIR.

However, of the 3 TC strategy groups that deliver both better patient experience and perception of health after hospitalization, only hospital-based trust, plain language, and coordination was associated with the third type of outcomes, better use of medical services, among the whole group of patients studied. We believe this may be due in part to the potential for patients to be more involved in their care when providers communicate in plain language and their potential to be more adherent to their care plan and medication instructions when they trust hospital providers. It may also be due in part to the influence of medication reconciliation, which was unique to this TC strategy group, on these outcomes.

Conversely, 2 of the TC strategy groups (patient/family caregiver assessment and information exchange among providers and assessment and teach back) were often associated with poorer PROs, patient experience, and health care use. These groups were characterized by hospitals performing patient assessments (risk assessment, language assessment, goal assessment, TC needs assessment) and the exchange of information among care settings. Hospitals' use of teach back was included in 1 of these groups, which also had the fewest number of hospitals reporting this organization-wide approach. Of note, the TC strategies within these groups were all hospital reported; therefore, patients' perception of their delivery remains unknown.

The findings from the prospective and retrospective studies do not completely align. We think this may be due in part to the following reasons. First, the retrospective study collected data about many TC efforts in a national sample of hospitals in 2015. It is possible that enhanced health information exchange, EHR interoperability, and increased integration of health systems across acute and postacute settings has improved since these data were collected. Thus, the cross-setting information exchange was more universal in 2017, which meant weaker contrasts between hospitals when the prospective hospital data were collected. It is also possible that the hospitals that were able to participate in the more labor-intensive prospective study are inherently different than hospitals nationally, with a higher degree of sophistication in their TC efforts. Again, for these hospitals, it is possible that the specific strategies pertaining to exchanging information among acute and postacute providers are not as much of a distinguishing feature as they were in the retrospective analysis. Finally, the prospective study evaluated a larger number of TC strategies than did the retrospective study, and strategies that were included in the most successful groups were not measured in the prior study (eg, plain-language communication and promote trust).

In summary, we found that patients who were discharged from hospitals that emphasized building trust and communication with patients, in combination with clinical and care management processes, experienced improved outcomes.

Retrospective

We did not perform stratified subgroup analyses for the retrospective study. However, even after controlling for hospital- and community-level factors, including the implementation of TC groups, we found that the patient characteristics of being non-White, male, and dual eligible and having a disability, comorbidity, and prior use increased the risk of readmissions, demonstrating that these populations are still at increased risk of poor outcomes despite the implementation of TC strategy groups.

Prospective

Our subgroup analyses revealed findings similar to those from the pooled analysis, with a few distinctions. Significant effects found in some subgroups or across the entire sample disappeared in the rural subgroup, though patient communication and care management remained significantly associated with all patient experience outcomes in this and all subgroups. The no-TC groups were associated with better health care use outcomes or PROs among rural patients, indicating that this group may require a different set of TC strategies than those measured in our study.

The group with low health literacy/limited English proficiency experienced positive, significant associations with the 3 aforementioned communication-based TC groups across numerous PROs and patient experience measures, with hospital-based trust, plain language, and coordination also being associated with some improved health care use outcomes. It was notable that among all the subpopulations studied, patients with low health literacy uniquely experienced better perception of mental health and participation in daily activities associated with the patient communication and care management group. The element of this TC group that is not part of other groups is the provision of a helpful health care contact. This hypothesis should be tested further and, if proven to be a strategy of specific importance for people with low health literacy, would be worthy of broad dissemination as a best practice. Moreover, health literacy may have a direct and clinically important effect on the ability of patients to understand and complete PRO questionnaires.

The MCC group also saw positive and significant associations with PROs and patient experience for the 3 aforementioned groups focusing on communication and patient care coordination. Although strong associations were identified with improved patient experience in the dual-eligible group, our analysis did not demonstrate similar improvements across other outcomes for the 3 communication-based TC groups.

Of note, not being exposed to any TC group was significantly associated with lower risk of either 7-day or 30-day ED visits among patients with low literacy, disabilities, and dual eligibility. It is possible that the most effective grouping of TC strategies to reduce unnecessary ED use was not assessed in the study.

T2 caregivers experienced more significant associations with TC groups than did those at T1, with patient communication and care management showing the most positive results. This makes sense, as this group contains several “bridge” TC services, or those including both pre- and postdischarge activities (eg, postdischarge care consultation, access to urgent care, and transition summary for patients and family caregivers) to help ensure that there is contact and information provided postdischarge, as well as plain-language communication at home and the hospital. Somewhat surprisingly, however, caregivers did not have better experiences when they received strategies that included a formal assessment of their needs. This may indicate that these needs assessments were not sufficiently paired with appropriate follow-up, building expectations for services not ultimately received. However, our data do not allow us to know exactly when assessments led to follow-up activities.

In summary, the results from the pooled analyses remained consistent across many subgroups, with some exceptions for specific outcomes and subpopulations. A different set of strategies may be required for rural patients, specifically those whose self-reported health outcomes and health care use outcomes were not positively associated with the groups of TC strategies evaluated in this study.

Retrospective

Due to the study's observational design, the estimated associations between TC strategies and readmission trends should not be interpreted as causal and may be influenced by unmeasured patient, hospital, and community characteristics or unrelated temporal trends in spite of our efforts to include many measured characteristics as model covariates.

TC strategy implementation data were self-reported by hospitals through a cross-sectional survey conducted in 2015-2016 and were associated with readmission data from 2009 to 2014. Hospitals' assessment of past TC strategy adoption may have been limited by recall bias. Many hospitals did not report detailed information about when they initiated implementation, thereby limiting the study's ability to evaluate temporal relationships between TC strategies and readmission trends with precision.

Attempting to mitigate the potential for self-report bias regarding TC strategy implementation, we used data collected through the study's phase 1 hospital site visits (study component 7) to confirm survey results among 4 hospitals that participated in both study components. Comparing the qualitative site visit notes to hospitals' survey responses revealed no inconsistencies. Therefore, we feel confident in the survey's validity as a measure of the use of different TC strategies.

Another limitation of this retrospective study was the failure of the survey to inquire about some important TC practices (eg, plain-language communication) that respondent hospitals may have used. In addition, the intention-to-treat method may underestimate a TC strategy group's effects because it assumes implementation for patients who may not have received the identified strategies. Conversely, because we surveyed hospitals (and not other potential sources of TC interventions), the survey responses may not provide a complete view of all the community TC resources available to and/or received by patients and caregivers in a given setting.

Prospective

The prospective study, just as the retrospective study, is observational, and we are not able to attribute causation to the relationships between TC strategy groups and the study outcome measures. It is also possible that despite our conservative use of the Bonferroni method to set our significance value, some of the associations found were due to chance given the number of comparisons explored. In addition, it is important to recognize that each of our sources of data about TC strategies has limitations. Patient and caregiver measures are self-reported and subject to the usual sources of measurement error associated with such data, as well as the potential bias associated with missing data. These include differences in how TC strategies are understood and interpreted by respondents, differences in the ability to observe and recognize receipt of TC strategies, and differences in the ability to accurately recall details about TC strategies several weeks or months after patients and caregivers have experienced the transition. This limitation was heightened with the required ≥51-day delay in when we were able to survey patients about their TC experience. This delay also increased the potential for selection bias toward a healthier sample, in that sicker patients may have died in the intervening weeks after discharge. Although our response rate of 57% for the patient survey compares favorably with that of similarly designed surveys (eg, HCAHPS surveys of patient experience typically average approximately 30%), there remains the possibility that participants were systematically different from nonparticipants in ways we were unable to measure. Because patients were initially recruited by hospitals and not formally consented until contacted for the survey, we did not collect demographic information about those who did not participate. Thus, we are unable to compare the characteristics of participants with nonparticipants.

Including patient-reported exposure data is an asset because it ensures that for some TC strategy exposures that are more accurately assessed by the patient (eg, promote trust), the patient voice is included in measuring exposure. However, it includes the limitation that we collected data from patients about outcomes (eg, self-reported health) at the same time we collected data about their exposures. Patients with self-perceived “good” outcomes may have thus been more likely to state that hospitals provided certain TC strategies. However, we believe this potential is limited for 2 reasons: (1) Only 6 of 22 strategies included a patient-sourced component, and (2) the TC strategy group with the most successful results was consistently associated with positive outcomes that were both subjective (patient reported) and objective (reduced readmissions, ED visits).

TC strategy implementation data were cross-sectional and self-reported. However, during phase 2 of the study, we validated hospitals' TC implementation survey data with qualitative data collected through hospital site visits. Although we have a high degree of confidence in the validity of these data, the possibility of measurement error remains.

Although our study hospitals were similar to hospitals nationwide according to some characteristics, they also differed in several as well. We oversampled rural hospitals for 2 reasons: (1) Rural hospitals lack the capital of large urban hospitals, and we therefore wanted to ensure retention of some rural hospitals if the external environment changed or organizational priorities shifted; and (2) the analysis was at the patient level. Usually, rural hospitals are smaller, and to obtain similar numbers of patients served by urban hospitals, we needed more rural hospitals in the sample. With our oversampling strategy, our analytic data set included 21% rural hospitals, compared with 14% all CMS inpatient prospective payment system (IPPS) hospitals, which we see as sufficiently comparable to generate generalizable results.

Our sample also included more large hospitals, which may have more resources for TC, and more hospitals in the West, where managed care is more widely adopted. Notably, the research team purposely recruited Kaiser hospitals (13 of the 42 [31%]) to participate, allowing an evaluation of the impact of an integrated delivery system; however, Kaiser patients ultimately accounted for 45.7% of the sample. To control for the potential confounding effect of Kaiser affiliation as an integrated health system, we included it as a dummy variable in the health care use models. Although Kaiser affiliation was associated with decreased 30-day readmissions in the unadjusted analysis, no significant association was found in the adjusted analyses. In addition, the study cohort of hospitals had overall lower readmission rates (~10%) than those of hospitals nationwide (15.3%). The inclusion of higher-performing hospitals likely derives from our sampling strategy, which attempted to ensure inclusion of hospitals participating in national care transition programs. As such, nearly every hospital applied at least 3 of the TC strategies identified by our study, including interdisciplinary approach, identification of caregiver, and standardized protocols, which may ensure better performance in readmissions. Of note, using an interdisciplinary approach to care delivery is a core component of several evidence-based TC models, and it is possible that this finding reveals a higher structural and resource investment in TC among study hospitals than among hospitals nationwide.

In addition, hospitals may have implemented locally developed, more innovative TC strategies not measured by our study. Given that Project ACHIEVE aimed to evaluate existing TC efforts with commonly implemented strategies and wanted to balance the need for data collection and manageable length of each survey for each participant group, some TC practices used by hospitals may have been omitted from our survey and thus not measured.

The goal of this analysis was to evaluate combinations of TC strategies employed by hospitals from their own perspective by analyzing their associations with numerous PROs and health care use outcomes to ultimately provide recommendations to health systems. Thus, our approach to identifying TC strategy implementation is largely hospital centered. However, as a result, it may not provide a complete view of all the TC resources available to and/or received by patients and caregivers in a given community. Nonetheless, 6 of our TC strategies were sourced by patient survey data because they were viewed as being more accurately measured from that perspective (eg, plain-language communication).

Methods

Interview Guide

Provider focus group interview guides were developed to elicit discussion about influential community characteristics, the nature of TC problems to be addressed through quality improvement efforts, implementation barriers and facilitators, perceptions of program sustainability, and methods for monitoring and evaluating progress. We started with the interview questions from the QIO ICPC evaluation effort, and we revised, added questions, and finalized the interview guide based on input from ACHIEVE research team members and stakeholders through an iterative process. Interviews assessed contextual factors relevant to selecting, deploying, and adapting TC strategies considered most promising for patient populations within a community and/or identified problems in the organization and community; when necessary, the probing question was adapted depending on the roles of the participants or on known characteristics of the community. The interview protocol included the following topics (see Appendix E1 for the interview guide):

1. Community characteristics
2. Factors facilitating the development and sustenance of transitional efforts
3. The process for identifying and addressing problems in TC
4. A description of TC interventions, including adaptations and target populations
5. Barriers associated with TC interventions

Most questions were asked consistently in all interviews, with probes added as necessary; questions that were not applicable to certain participants were not asked.

Procedures

The ACHIEVE team conducted phone-based focus groups (all but 1 group interview was conducted by phone) and individual interviews from March 2015 to September 2015 to identify factors that influenced the selection of interventions and implementation approaches of TC programs within different care settings or that relied on multistakeholder participation. Although most communities participated in group interviews (12 groups with 48 providers), some providers preferred to participate in individual interviews (15). Because of a scheduling issue, all PCPs were interviewed individually (n = 12). Group interviews lasted 1.5 to 2 hours, and individual interviews lasted 30 to 60 minutes. Importantly, the facilitators were experienced in conducting interviews and focus groups remotely and had >1 facilitator to help ensure equitable participation. Each group interview began with introductions, which helped facilitators ensure that all participants were active. If certain participants were less engaged in conversation, facilitators would solicit information from them directly.

Interviews were audio recorded, and, in most cases, a note taker was present. Recordings were transcribed and integrated with the staff notes to create the final transcripts. In cases where a notetaker could not be present, the interviews were recorded and transcribed; transcripts were then revised by a team member who listened to the recording. The same methodology was applied to the focus groups and individual interviews.

Contextual Factors That Influence TC Interventions

Facilitators to Implementation

Methods

Sample and recruitment

Eligible participants included health care providers (hospital, ambulatory, or downstream) affiliated with the hospitals involved in recruiting patients/family caregivers for the prospective study^* (study component 4). To recruit participants, we gave eligibility criteria—which included each provider being involved in TC with the hospital and matching the example target roles and care settings according to each survey type displayed in Table E2.1—to each hospital's ACHIEVE coordinators, who then provided email addresses for the nominated providers. Our target sample size for this survey was 929 providers across the 3 provider types.

Table E2.1

Provider Survey Target Care Setting and Provider Types.

Table E2.3 displays characteristics of the 43 participating hospitals compared with AHA hospitals and the CMS Impact File.^** Table E2.4 and Table E2.5 display the characteristics of study respondents by provider role. Compared with national hospitals, study hospitals were more often in the West (40% vs 19%, respectively), had >300 beds (53% vs 14%), and were nonprofit (81% vs 50%), large urban (56% vs 41%), and major teaching (37% vs 11%).

Table E2.3

Provider Survey Participant Characteristics Compared With National Samples of Hospitals.

Table E2.4

Roles of Study Respondents.

Table E2.5

Downstream and Ambulatory Provider Survey Respondent Characteristics.

Measures

The study team developed 3 separate provider surveys, 1 for each category. Surveys were designed to capture information about TC services and collaborations with hospital providers, perceived barriers and facilitators in delivering those TC services, and the organizational and community contexts in which those TC services are delivered. Informed by literature reviews and provider focus group findings, the research team developed draft surveys. We then conducted 28 cognitive interviews in 2 rounds to test how different iterations of survey questions were understood by participants. Hospitalists, social workers, care coordinators, nurses, administrators, and PCPs from a range of organizations (eg, hospitals, outpatient clinics, home health agencies, and SNFs) participated in these interviews. The results from these interviews as well as feedback from both the SAG and SAC were used to refine and finalize the surveys.

A cross-sectional design enabled us to efficiently capture information from various perspectives based on the qualitative data collected from earlier phases. Given the high demands on participants' time, a web-based mode was determined to be the best way to gather information from a diversity of geographically dispersed providers.

The survey development team (UK and Westat teams) conducted an 11-week pilot test with 5 hospitals in 2017 to test the functionality of the web surveys and the effectiveness of the recruitment methodology, as well as to identify items needing additional refinement. Item variability was assessed, and the research team openly debated whether items were redundant or lacked sufficient variability to remain. Based on the results from 110 respondents, researchers removed 13 questions and revised 10 items across the 3 surveys. The final surveys included the following content areas (see Appendix E2.1 for the downstream provider survey):

1. Information about patients: downstream and ambulatory providers' awareness of patients' hospital admission or discharge; providers' assessment of patient information provided
2. Communication with family caregivers: efforts to engage family caregivers
3. Health information technology: modalities for downstream and ambulatory providers' patient information exchange with the hospital
4. Support for TC: provider perspectives on the organizational and senior leadership support for providing TC services
5. Access to medical resources: provider perspectives on patient access to services and health-related community resources
6. Working with other providers: providers' perspectives on their relationship with other providers in the community when providing TC
7. Overall coordination with hospital: downstream and ambulatory providers' overall assessment of the hospital's coordination with the provider
8. Overall assessment of TC: overall assessment on how well each organization provides TC to patients

Data Collection

The coordinating site (UK) sent nominated providers an individual survey link through REDCap (https://www.project-redcap.org/), a HIPAA-compliant web-based platform, from November 2017 to April 2018. Email reminders with continuously active survey links were sent weekly via REDCap to nonresponders over 4 weeks before individuals were retired. If the hospital did not want UK to contact its providers directly, UK generated a public link to the survey(s) with a corresponding email template allowing for the hospital's distribution.

The provider surveys were not a component of the original proposal's power analyses. Though the original target sample size was 3000, the team updated the target sample size to 929 through contract modification along with other changes described below. The downward revision of the target sample size was based in part on the smaller-than-expected numbers of providers actively involved in TC effort/project implementation at each organization, which was necessary for meaningful data, and in part because the relatively simple analysis plan for the provider survey (basic descriptive analysis) did not require a large sample size.

Due to the mode of survey administration (web-based links, some of which were public and not associated with specific respondents), we were unable to calculate response rates or obtain data regarding reasons for noncompletion.

Analytical and statistical approaches

Basic descriptive statistics (eg, frequencies) were conducted to present findings across each survey type. Valid percentages are reported with missing responses removed from the denominator and reported separately (Table E2.4, Table E2.5, and Table E2.6). A summary of key descriptive survey findings follows. In addition, we conducted several psychometric analyses (eg, inter-item correlations and reliability, confirmatory FA, site-level and overall reliability) to identify conceptually meaningful and reliable composite measures for the provider surveys and determine site-level and overall reliability of these measures (Table E2.7 and Table E2.8). Analyses and results are available on request.

Table E2.6

Provider Survey Responses.

Table E2.7

Final Reliability of Composite Measures in Provider Survey.

Table E2.8

Percentage Positive of Provider Survey Composite Measure Components.

Methods

Study Design

Participants

Phase 1

In partnership with local ACHIEVE hospital coordinators, the team used purposive sampling supplemented by convenience sampling techniques to ensure that participants from each site had sufficient representation from the following TC stakeholders: leadership, TC team, internal stakeholders, community partners, and recently discharged patients/family caregivers (see Table E3.2). The composition of individuals interviewed was adjusted at each site depending on the structure and leadership of the site's TC efforts.

Table E3.2

Sample Participants in Site Visit Sessions.

Data Collection

Analytical Approach

Changes to the Original Study Protocol

No major changes occurred to the original study protocol for this component for either phase.

Ethics

The phase 1 study protocol received IRB approval from UK and from each of the co-PI's organizations. The phase 2 UK IRB was the IRB of record for all organizations and was responsible for reviewing and approving the protocol.

Phase 1

The site visits provided insight into the relative effectiveness of different approaches to leading TC implementation in various contexts, including common facilitators of and barriers to effective care transitions. The details of this analysis were published in The Joint Commission Journal on Quality and Safety.⁷

Three broad categories of facilitators were described (Figure E3.1). The first was collaborating within and beyond the hospital. Coordination with community partners was most frequently reported as a way to do so, which might include downstream providers (eg, SNF, home health), nonprofit organizations (eg, elder care networks, faith-based ministries), or public institutions (eg, health, police, or fire departments), and often consisted of formal partnerships and/or regular, face-to-face meetings.

Figure E3.1

Heat Map of TC Facilitators.

Disease isn't what brings patients back to [the] hospital. What brings them back is the inability to see their doctor, inability to get food, inability to get their meds. So targeting these things, bringing [community] services in that get at these needs—once this is realized, doctors can do their part, and community supports can do their part, and we don't have all this time wasted. —Leadership

Tailoring to needs of patients and family caregivers was another key facilitator. The primary way that sites tailored programming to the needs of patients and families was by adapting TC protocols based on ongoing evaluation of their effectiveness. A participant from a use management team explained that at their site, “complex case conferences focusing on high utilizers have been very instrumental in reducing readmissions.”

Finally, generating buy-in was also commonly cited as a facilitator, specifically through the strategic prioritization of TC services, which was accomplished when leadership committed to prioritizing TC and created an organizational culture supporting TC efforts.⁷ A chief medical officer at one site noted that TC services were a top strategic priority because, “admission to the hospital and discharge are the scariest parts of any process, the parts where there's a likelihood of errors to occur.”

Similarly, 4 categories of barriers were revealed through analysis of site visit transcripts (Figure E3.2). Among them was poor integration of TC services, within which the lack of coordinated implementation of TC programs was most frequently cited. In other words, when TC programs were not uniformly implemented across an organization, service delivery was fragmented or siloed. As one participant noted in their organization, “high-risk patients are over-resourced. They get 9 calls a day, and there's no one who has a central knowledge of the patient.”

Figure E3.2

Heat Map of TC Barriers.

Another common barrier leading to reported increases in readmissions was when patients' needs (eg, stable housing, transportation) were not adequately addressed. As one participant noted, “what brings patients back to the hospital is not necessarily disease,” but it can also be their underlying social or pscyhological issues. In addition, patients who are discharged without sufficient education on how to care for themselves may also be at risk of readmission or adverse outcomes, but education could be compromised due to a lack of time or resources. As one member of a specialty care management team noted, “There are unrealistic expectations that patients can manage their self-care at home. The focus is, ‘Are you ready to get out of the hospital?’ vs ‘Are you ready to live your life at home?’”

In addition, underused TC services were also common barriers, including inadequate medication review or home health quality and breadth and underused palliative care. Much of this underuse seemed to be rooted in a lack of understanding of team members about the nature and scope of TC services. For example, a home health representative noted,

The physician's understanding of home health is getting better, but there are still provisions that they don't understand. We are not based out of social services, so if patients need help but have no skilled need, we have to say that we cannot provide services.

Phase 2

Although the focus of the meetings was to learn about stakeholders' perspectives on TC strategies offered by the hospital as well as how transitions are managed from the hospital to the next site of care, we also learned that sites were in various stages of implementation. Some organizations experienced unsuccessful implementation, while others had redesigned delivery of TC services to improve care. Sites often reported that TC implementation was in progress.

The phase 2 findings largely validated those from phase 1 regarding facilitators and barriers. Three main facilitators to implementing TC strategies emerged from the initial coding: (1) collaboration, (2) communication, and (3) staff and leadership buy-in of TC.

Collaboration occurred both within and beyond organizations' systems and was systematically integrated into many sites through regular internal staff meetings to discuss care transitions, regular meetings with community or downstream partners, or formal agreements and contracts with community and downstream partners to collectively improve TC quality. Communication covered a variety of subtopics, including efficient use of information management tools (eg, information technology or EHRs), information exchange with community and downstream providers, shared access to patient information, and other communication regarding handoffs between providers. Staff and leadership buy-in were commonly described as imperative to effective TC implementation. Strong leadership support of TC efforts helped cultivate staff engagement and prioritization of these efforts:

We've been able to establish really good relationships with our home care agencies to work along with them. —TC team member

The communication piece is key, and I think that, certainly, patient-centered care has driven that. —Postacute facility partner

It helps to have collaborative champions, not just in nursing but also case management, they'll be a champion. —Internal stakeholder

The top 3 barriers to implementing TC strategies were poor integration, limited availability of services, and population needs. Poor integration was described as TC efforts being inconsistently applied or siloed across an organization. Inconsistent use and frequent siloes were highlighted as key barriers to improving patient care outcomes. Limited availability of services included organizations' inability to provide access to postdischarge physician appointments or specific downstream care services that were needed for safe transitions, reflecting limited downstream and community-based resources in the hospital area. Most hospitals described their population's social determinants of health (SDOH) and population needs as challenges to TC, including unaddressed psychosocial needs, poverty, homelessness, large number of older adults, lack of support at home, low literacy, and lack of transportation:

When people are really needing [services] to go home, they're on waiting lists forever for a lot of in-home services. —Leadership/management

That is one aspect we have very little control over. Transportation to their primary care doctors, diet, food, electricity, heating. Housing is a big thing. —Internal stakeholder

I think it's a huge challenge for us because everybody wants to do the right thing, and sometimes we go off and isolate it as siloed areas. —Leadership/management

The phase 2 site visits added the engagement of PFACs at each site, if available. Notably, the PFAC members often commented that they focused on responding to hospital ideas for building design or process improvement and/or generating new ideas for hospital process improvement efforts. Other key patient and caregiver engagement efforts involved including patients in their own discharge process—including during bedside rounding—and identifying family caregivers of patients, assessing family caregivers on postdischarge needs, and engaging them in the discharge process.

Phase 2 site visit data were also used to validate hospitals' self-reported employment of TC strategies for the prospective study (study component 4). Coders noted the current presence of TC strategies as defined by the Project ACHIEVE research team (see Table D2.1 in the Aim 2 section, study component 4). Figure E3.3 provides a heat map illustrating the relative presence of specific strategies. Six strategies were mentioned by >85% of the 29 sites.

Figure E3.3

TC Strategies Described and Assessed on Site Visits.

Lessons Learned

Provider focus groups/interviews (study component 5) and site visits (study component 7) provided rich information about perspectives from different TC stakeholders regarding what facilitates and what hinders implementation of TC efforts. Across these components, we found similar themes: Cross-site collaboration, communication, and partnership with different stakeholders (eg, downstream providers, nonprofit organizations, or public institutions) are essential to effective TC; TC efforts should be well integrated within the organization as well as within the community; and leadership and staff support and buy-in to these efforts are critical to their success. Such integration includes selecting the most appropriate intervention to address root causes of readmissions, implementing the intervention across settings, and ensuring that evaluation and monitoring inform a process of continuous improvement to increase the likelihood of success.

Specifically, collaboration was viewed as important on a provider-to-provider basis in provider surveys (study component 6). These findings complement site visit findings in both phases, in which internal and external collaboration and partnership were viewed as integral facilitators of successful TC. However, opportunities exist for improved collaboration of clinical providers with community-based or social service organizations. Despite our results from provider focus groups and site visits revealing that patients' unmet social needs were a barrier to effective TC, partnership between providers and community organizations was lacking. In fact, less than a third of hospital and ambulatory providers rated their relationships with CBOs positively. If providers view patients' psychosocial needs as a contributing factor to their unnecessary hospitalizations, improved coordination with organizations charged with meeting those needs is merited.

Although there was a great deal of consensus regarding most facilitators and barriers to TC among different provider groups, some divergence in perspectives was also present based on the provider/organization type and their past experience with formal TC efforts. For example, in provider interviews, PCPs, who are typically not involved in community-level TC programs, indicated that readmissions and poor outcomes were generally the result of different root causes from those reported by providers who were more directly involved in TC partnerships. Of note, the provider survey results also indicated a general lack of systematic communication with ambulatory (including PCP) providers, where only about half were alerted to their patients' admission and discharge from the hospital. Such findings highlight a gap in cross-setting information exchange, as a lack of information sharing may be associated with providers' poor perceptions of care coordination and overall TC quality.

Methods

Developing a framework

The DIW drafted a dissemination plan in June 2017 (see Appendix F1) based on recommendations from PCORI⁸¹ and others.⁸² Members of the research team used the Toolkit for Dissemination Planning posted from the AHRQ's Patient Safety and Quality Tools and Resources⁸³ (Figure F1). This toolkit breaks down the development of a dissemination plan into 6 discrete, actionable steps, emphasizing the need for successful plans to be targeted to a specific audience (eg, nuanced push-messaging strategies) for each product worthy of dissemination.

Figure F1

Dissemination Plan Components From AHRQ's Dissemination Planning Tool.

Although active dissemination was not a funded goal of ACHIEVE, which is an effectiveness research study, the team decided to leverage its impressive team of stakeholders to pilot test the framework using the preliminary findings from the project. Based on this targeted, multistep approach, in May 2018, the Project ACHIEVE DIW pilot-tested a dissemination plan to broadly share findings from study component 2 (patient and family caregiver interviews and focus groups), which were published in the Annals of Family Medicine.⁴ Specifically, this dissemination strategy included the following:

1. Identifying the research findings to share: what matters most to patients and family caregivers in care transitions
2. Identifying target end users: patients and family caregivers, health care providers
3. Leveraging Project ACHIEVE's extensive network of research partners and stakeholders: On DIW conference calls, participants brainstormed target outlets and audiences and volunteered to share the findings with their networks through various means (eg, newsletters, conferences, listservs, webinars). An example of the workflow created and activated by ACHIEVE partners for 1 specific target end user is shown in Figure F2. This process would be repeated for each target end user.
4. Customizing messages for each target end user and facilitating the dissemination of findings through the preparation and distribution of press packets. The ACHIEVE coordinating site (UK) drafted the following documents, which were vetted and edited by research team and stakeholder advisors before being shared with all research team members, SAG members, and SAC members.
   1. Press release, customizable to each partner organization
   2. Drafted tweets, customizable to each partner organization
   3. One-page summaries of main findings, customized for each end user^*
5. Evaluate the results. In August 2018, the ACHIEVE team evaluated the pilot by tracking all partner dissemination activities and media coverage between May and August 2018. We also referenced the article's Altmetric score (Altmetric is a platform that tracks online attention for research by pulling data from a variety of sources, including social media outlets, mainstream media, blogs, and online reference managers).
6. Dissemination work plan. In September 2018, ACHIEVE team members, including research team, SAC, and SAG members, met to discuss preliminary results and future dissemination of the research findings. Additionally, in June 2019, the SAG held a conference call to discuss dissemination efforts. Based on the input gathered, the team drafted a final dissemination plan for the study's prospective findings (Appendix F2).

Figure F2

Dissemination Workflow With Partner Activation.

Lessons Learned

Through the process of constructing and deploying a methodical plan for disseminating specific messages emerging from our results, we learned several lessons to apply to future dissemination efforts. Apart from the broader lesson—the AHRQ 6-step process described in Figure F1 provides an excellent roadmap for dissemination planning—we also learned the following:

1. Stakeholders are valuable active partners for dissemination.
   We did not merely engage our stakeholder partners in pushing out messages from our research, we engaged them in crafting the messages, just as we sought their guidance in the research process. Stakeholders were present in early discussions regarding how to frame and present the findings, and they reviewed/edited drafts of summaries and tweets. After numerous review cycles, we finalized products that our partners co-owned. We believe this heightened their engagement in sharing them with their networks. We also worked with our partners to identify promising upcoming opportunities to share the findings and incorporated those into partner-specific dissemination plans.
2. View dissemination as an interactive, iterative process, not a 1-time activity.
   We found that sharing research findings with a public audience was effective when it was interactive and when we collaborated with partners as they promoted the findings. Although this applies to numerous venues, our primary experience during this pilot was with social media. We supplied draft tweets to our partners and invited them to customize the messages and post them on their organizations' social media accounts, which we found to be successful. We tagged key partners and engaged them in conversation, used popular relevant hashtags, and posed questions to users.
   We also learned the importance of monitoring how messages are being interpreted and responding if necessary to allay confusion.
3. Ensure that the message is compelling, direct, and succinct.
   Media-savvy stakeholder partners reiterated the importance of providing targeted information in lay terms that includes recommendations or actions for the end user. The message from our pilot dissemination plan was clear and concise, which made it easier to remain consistent in our diverse dissemination efforts. Across all media (eg, press releases, 1-page summaries, tweets), the message included 3 main points related to what patients and caregivers wanted from transitions, as well as 5 actions that providers can take to help achieve those outcomes.
   We believe another reason why our results were widely shared is that they resonated with patients and family caregivers, affirming their personal experiences. We believe that the clarity of the patient and family caregiver voice—in the lay media articles and in the original publication through direct quotations—played a significant role in the accessibility of our message. We also learned that reporters work on a deadline and need quick access to patients or caregivers who are willing to share their stories. Based on this experience, we have assembled a list of stakeholders whom we can contact to share their stories if needed.
4. Remove barriers to accessing the message.
   We believe that having our results shared through platforms providing public access facilitated their dissemination. For example, we published the article targeted for the pilot dissemination plan in an open-access journal,⁴ meaning anyone could read it without a subscription. We also shared 1-page summaries on project webpages and through social media. In addition, Kaiser Health News (KHN) published a story about the findings, which quickly spread to nearly 40 media outlets nationwide (note: although the study's findings were ultimately shared in nearly 60 outlets, 39 were reprints of the KHN article). KHN uses a Creative Commons License, meaning that any other outlet can republish its content, allowing for broader sharing.

Implications

Another lesson learned from this effort is the resources required to undertake such an effort. Although rigorous conduct and evaluation of a dissemination plan is outside the scope of Project ACHIEVE, even the activities documented above required substantial resources that could be leveraged while the study teams were already meeting but that may be harder to corral outside a funded project. Based on the efforts we expended, we estimate it would consume 6 months of 1 full-time employee's effort to manage the drafting of tailored press packets, coordinate input and feedback from stakeholder partners, and evaluate the process. A more rigorous evaluation and more targeted dissemination effort would require more investment, both from the study coordinator and from study partners.

Limitations

The dissemination pilot described above achieved success in the attention it garnered for the targeted findings. However, these findings pertain specifically to results from 1 manuscript resulting from only 1 study component, ACHIEVE study component 2 (see aim 1), which aimed to identify the TC outcomes and components that matter most to patients and their family caregivers. Overall, Project ACHIEVE includes 7 study components with 4 manuscripts already having been published (see the Related Publications section for a full list), but we applied this concerted dissemination effort to 1 published manuscript.⁴ In addition, our evaluation of the success of this effort does not directly measure the dissemination of messages to each specific target audience, nor does it extend into its impact on hospital practice, adoption of specific TC strategies, or patient and family caregiver behaviors or experiences—the desired end goals of such efforts. Although we understand the importance of evaluating dissemination efforts regarding the desired end goal, such an undertaking is outside the scope of Project ACHIEVE.

Despite these limitations, this pilot test provides an excellent roadmap for how to engage stakeholders in the dissemination and implementation process, including how to prepare them to be partners. The findings disseminated above had material benefit for patients and providers. While we targeted these “end users” specifically through our organizational contacts' partnering stakeholders, the message of these findings had such a broad appeal that we also relied on the lay media (eg, The Washington Post) to reach these groups. After all, most of us have been patients before. Our main findings or recommendations from the prospective study will most likely target hospitals and health care systems specifically—who we want to adopt and implement our recommended set of TC strategies—and therefore, the process of dissemination will necessarily change based on that target end user. Given the pilot experience with this modifiable 6-step strategy and our lessons learned (eg, engage stakeholders, create compelling messages, disseminate iteratively), we anticipate being able to successfully translate the process described above to different messages targeting diverse stakeholders and end users.

Table 1 (PDF, 335K)

Table 2 (PDF, 214K)

Table 1. Characteristics of Participating Hospitals (PDF, 447K)

Table 2. Implementation of Transitional Care Strategies and Activities in 370 Hospitals (PDF, 467K)

Table 3. Groups of Transitional Care Strategies Based on Factor Analysis -- Adjusted with Patient and Caregiver Focus Group Findings and Clinical Input (PDF, 452K)

Table 4. Probability of Implementing Transitional Care Strategies Among Seven Latent Classes of Hospitals (PDF, 453K)

Table 1. Stratified analysis of hospitals who were and were not subject to payment under CMS HRRP (PDF, 120K)

Figure 1. 30-day risk adjusted readmission trends --Not Penalized (PDF, 460K)

Figure 2. 30-day risk adjusted readmission trends--Penalized (PDF, 370K)

Table 2. Stratified Analysis of Penalized vs. Non Penalized Hospital Readmission Rate Reduction During Study Period (PDF, 140K)

Table 3. Comparison of Slopes, Penalized vs. Non-penalized by TC Groups (PDF, 160K)

Figure 3. Trend in Slopes, Penalized vs. Non-penalized by TC Groups (PDF, 158K)

Table 4. Difference in Difference Specification with hospitals with TC implementation date (pre and post implementation) (PDF, 142K)

Table 1. TC Groupings in Retrospective Analysis (PDF, 484K)

Table 2. Factor Analysis Results, Patient Survey, Home Visits Based on Receipt (PDF, 461K)

Table 3. Latent Class Analysis Results, Patient Survey, Home Visits Based on Receipt (PDF, 459K)

Table 4. Factor Analysis Results, OTCIA Survey (PDF, 442K)

Table 5. Latent Class Analysis Results, OTCIA Survey (PDF, 451K)

Table 6. Finite Mixture Model (FMM), Patient Survey (PDF, 436K)

Table 7. Finite Mixture Model (FMM), OTCIA Survey (PDF, 531K)

Table 8. Random Forest Analysis Results (PDF, 463K)

Table 9. How TC Strategies in Prospective Study Map to Prospective and Retrospective TC Groups^a (PDF, 549K)

Table 4. Approximate sample sizes needed per TCC cluster (PDF, 429K)

Table 1. Risk-Adjusted Associations between TC Groups and Selected Patient Reported Outcomes (PDF, 94K)

Table 2. Risk Adjusted Associations between TC Groups and Patient Experience Outcomes (PDF, 136K)

Table 3. Risk-Adjusted Associations between TC Groups and Healthcare Utilization Outcomes (PDF, 270K)

Table 1. Risk-Adjusted Associations between TC Groups and Time 1 Caregiver Experience Outcomes (PDF, 89K)

Table 2. Risk-Adjusted Associations between TC Groups and Time 2 Caregiver Experience Outcomes (PDF, 139K)

Table 1. Risk-Adjusted Associations between TC Groups and Selected Patient Reported Outcomes Dual-Eligible (PDF, 181K)

Table 2. Risk-Adjusted Associations between TC Groups and Selected Patient Reported Outcomes Multiple Chronic Diseases (PDF, 203K)

Table 3. Risk-Adjusted Associations between TC Groups and Selected Patient Reported Outcomes Mental Illness (PDF, 202K)

Table 4. Risk-Adjusted Associations between TC Groups and Selected Patient Reported Outcomes Rural Subgroup (PDF, 147K)

Table 5. Risk-Adjusted Associations between TC Groups and Selected Patient Reported Outcomes Disabled Subgroup (PDF, 203K)

Table 6. Risk-Adjusted Associations between TC Groups and Selected Patient Reported Outcomes Low Health Literacy Subgroup (PDF, 203K)

Table 7. Risk-Adjusted Associations between TC Groups and Patient Experience Outcomes, Dual-Eligibility Subgroup (PDF, 192K)

Table 8. Risk-Adjusted Associations between TC Groups and Patient Experience Outcomes, Multiple Chronic Conditions Subgroup (PDF, 324K)

Table 9. Risk-Adjusted Associations between TC Groups and Patient Experience Outcomes, Mental Illness Subgroup (PDF, 191K)

Table 10. Risk-Adjusted Associations between TC Groups and Patient Experience Outcomes, Rural Subgroup (PDF, 137K)

Table D2.15e. Risk-Adjusted Associations between TC Groups and Patient Experience Outcomes, Disabled Subgroup (PDF, 182K)

Table D2.15f. Risk-Adjusted Associations between TC Groups and Patient Experience Outcomes, Low Health Literacy Subgroup (PDF, 182K)

Table D2.16a. Risk-Adjusted Associations between TC Groups and Healthcare Utilization Outcomes, Dual-Eligible Subgroup (N=1232) (PDF, 177K)

Table D2.16b. Risk-Adjusted Associations between TC Groups and Healthcare Utilization Outcomes, Multiple Chronic Diseases Subgroup (N=5155) (PDF, 177K)

Table D2.16c. Risk-Adjusted Associations between TC Groups and Healthcare Utilization Outcomes, Mental Illness Subgroup (N=1524) (PDF, 216K)

Table D2.16d. Risk-Adjusted Associations between TC Groups and Healthcare Utilization Outcomes, Rural Subgroup (N=1136) (PDF, 260K)

Table D2.16e. Risk-Adjusted Associations between TC Groups and Healthcare Utilization Outcomes, Disability Subgroup (N=941) (PDF, 221K)

Table D2.16f. Risk-Adjusted Associations between TC Groups and Healthcare Utilization Outcomes, Low Health Literacy Subgroup (N=1892) (PDF, 270K)

Table 1. Risk-Adjusted Associations between Patient Communication and Care Management TC Group and Patient Outcomes Across Overall Sample and Among Subgroups (PDF, 184K)

Table 2. Risk-Adjusted Associations between Home-based trust, plain language, and Coordination TC Group and Patient Outcomes Across Overall Sample and Among Subgroups (PDF, 153K)

Table 3. Risk-Adjusted Associations between Hospital-based trust, plain language, and Coordination TC Group and Patient Outcomes Across Overall Sample and Among Subgroups (PDF, 154K)

Table 4. Risk-Adjusted Associations between Patient/Family Caregiver Assessment and Information Exchange among Providers TC Group and Patient Outcomes Across Overall Sample and Among Subgroups (PDF, 153K)

Table 5. Risk-Adjusted Associations between Assessment and Teach back TC Group and Patient Outcomes Across Overall Sample and Among Subgroups (PDF, 154K)

Table 6. Risk-Adjusted Associations between No TC Group (none of the groups) and Patient Outcomes Across Overall Sample and Among Subgroups (PDF, 152K)

Table 1. SAG D&I Workshop Dissemination Recommendations – How this might be used? (PDF, 108K)