An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Douglas F. Zatzick, MD, Joan Russo, PhD, Peter Thomas, JD, Doyanne Darnell, PhD, Harry Teter, JD, Lauren Whiteside, MD, Jin Wang, MS, PhD, and Gregory Jurkovich, MD.
Author Information and AffiliationsCurrently, high-quality patient-centered care is not the standard of care throughout US trauma care systems. Injured trauma survivors treated in trauma care systems frequently receive fragmented care that is not coordinated across hospital, emergency department (ED), outpatient, and community settings. Postinjury care is frequently not individualized to integrate the patient's most pressing posttraumatic concerns and preferences into medical decision-making.
The aim of this randomized comparative effectiveness trial was to compare 2 approaches to the delivery of patient-centered care for hospitalized injured trauma survivors: a novel patient-centered care transition service delivery model vs enhanced usual care that included notifications to nurses of trauma survivors' postinjury concerns.
The care transition intervention included a Master's-level social worker who first elicited and then addressed each injured trauma survivor's posttraumatic concerns; whenever possible, the social worker attempted to integrate the trauma survivor's concerns and preferences into medical decision-making, and problem solve with the patient. The intervention team provided 24-hour cell phone contact, 7 days a week, that included text messaging services aimed to enhance care coordination and reduce ED visits. The primary outcome examined was the severity and domain of patient-identified postinjury concerns. Other primary outcomes examined included posttraumatic stress disorder and depressive symptom severity. Secondary outcomes examined included physical function, automated statewide health service use, satisfaction with the physical and emotional aspects of care, and injury risk behaviors. Regression analyses assessed intervention and control group outcome differences over time.
Of the 171 patients, motor vehicle, pedestrian, or bicycle-related crashes represented approximately 50% of the injuries. Other events (eg, falls, work injuries) represented approximately 36% of injuries. More than 80% patient follow-up was attained at each time point. Concerns elicited from patients spanned physical health, work/finance, social, psychological, medical, and legal domains. Intervention patients demonstrated clinically and statistically significant reductions in the percentage of any severe postinjury concerns when compared with controls longitudinally (Wald χ2 = 11.29; P = .01) and at the 6-month study time point (Control [C] = 74%, Intervention [I] = 53%; Fisher exact test, P = .02). Comparisons of ED use data yielded clinically significant differences at the 6-month time point (1 or more 3- to 6-month ED visits; C = 30.2%, I = 16.5%; relative risk [95% CI], C vs I = 2.00 [1.09-3.70], P = .03) that did not achieve significant differences over time when compared with baseline (F3,507 = 2.24, P = .08). The intervention did not significantly affect other symptomatic, risk behavior, or functional outcomes.
The investigation found that patients who received the intervention had a clinically and statistically significant reduction in the percentage of any severe postinjury concerns expressed over the course of the 6 months after injury hospitalization. Clinically important ED health service use reductions were observed, but the intervention did not affect other symptomatic or functional outcomes. These initial findings are provocative and could lead to further investigation to develop a body of knowledge that would inform sustainable American College of Surgeons' clinical guideline requirements for US trauma care systems.
Because this was a multifaceted intervention, the investigation did not yield information about which components of the treatment were effective in targeting specific outcomes. No analyses were conducted to examine lasting intervention effects. The investigation did not find substantial age, ethnoracial, or traumatic brain injury group differences in treatment outcomes.
Each year in the United States, more than 30 million individuals present to acute care medical trauma centers and emergency departments (EDs) for the treatment of traumatic physical injury.1-6 Annually, 1.5 to 2.5 million Americans are so severely injured that they require inpatient hospitalization.1-5 Injured trauma survivors present to acute care medical settings after both intentional (eg, gunshots, stabbings, physical assaults) and unintentional (eg, natural disasters, motor vehicle crashes) injury events.7 Estimates suggest that approximately 1.5 million American youths and adults experience traumatic brain injury (TBI) annually, although many of these patients may not require injury admissions.8,9 Physical injury with and without TBI constitutes a major public health problem for both civilian and veteran trauma-exposed patient populations.10,11 Traumatic injury is a leading cause of death for individuals younger than age 45 years and accounts for 12% of medical expenditures in the United States.1-3,5 In a nationwide US study, more than 40% of injured trauma survivors reported they were unable to return to work 12 months after their hospital admission.12 Globally, traumatic injury is estimated to account for approximately 9% of the world's burden of disease.13-15
In 2001, an Institute of Medicine report, Crossing the Quality Chasm, identified patient-centered care as 1 of the 6 specific components of high-quality care.16 Increasingly, the adoption of a patient-centered model of health care delivery is being advanced as a critical goal of clinical medicine.17,18 At the core of a patient-centered approach is the belief that patients' perspectives can be understood and ultimately integrated into medical decision-making and comparative effectiveness research.19-22
A key element of trauma-focused patient-centered care is the empathic identification of each patient's individual needs, concerns, and values.20,21 Another key element particularly relevant to the acute care medical context is the coordination of care transitions across inpatient, outpatient, and community service delivery settings.16 Postinjury, the provision of a continuous helping relationship may help reduce care fragmentation and optimize acute care transitions.20,21 Other elements of high-quality postinjury patient-centered care can include the incorporation of patient caregivers into care where appropriate and the integration of peer support interventions.16,17,22
Currently, the standard of care throughout US trauma care systems does not incorporate high-quality patient-centeredness in practice.21,23 Upon discharge from an acute trauma care setting, injured trauma survivors frequently experience disjointed and ineffective care as they transition into outpatient and community settings. Postinjury care often prioritizes a medical practitioner's knowledge in medical decision-making to the detriment of the patient's most pressing posttraumatic concerns and preferences.
Prior investigation has demonstrated the reliability and validity of patient-centered posttraumatic concern assessments.20,21 These investigations have shown that the longitudinal trajectories of posttraumatic concerns mirror the trajectories of postinjury posttraumatic stress disorder symptoms and functional limitations.20,21 Prior investigations have also established that greater concern severity and burden are risk factors for the longitudinal development of high levels of posttraumatic stress disorder symptoms over the course of the year after injury.20,21 Commentary has encouraged increased incorporation of patients' perspectives, including addressing needs and concerns, into comparative effectiveness trials.17,22 A key methodological approach in comparative effectiveness research is the development and implementation of pragmatic randomized clinical trials that are conducted in real-world practice settings, include representative patient populations, and target a broad spectrum of outcomes relevant to patients, providers, and policy makers.24
Several papers have suggested that research with trauma victims requires an increased attention to ethical considerations beyond the standard guidelines for conducting research with human participants.25-29 Acutely injured patients often surrender control over emergency care decisions and frequently undergo treatment procedures about which they may have minimal awareness or understanding. Traumatically injured patients may, therefore, come to passively accept loss of autonomy as an unavoidable circumstance of their hospitalization. Also, the traumatic injury and subsequent medical procedures induce psychological distress, which could be exacerbated by research participation. An accumulating body of literature suggests that adequate protection against these risks can be enhanced by appropriate investigative methods in trauma-exposed populations.25-29
Care management treatment models are in place across US health service delivery sectors and could incorporate enhancements to patient-centered care derived from comparative effectiveness trials.30 Care management treatment models can target multiple aspects of patient-centered care delivery, including tailoring treatments to each patient's concerns and preferences and enhancing care coordination. In particular, by coordinating care across inpatient acute care and emergency, outpatient surgical subspecialty, primary care, and community service delivery sectors, care management treatment models could reduce postinjury care fragmentation.30,31 Few comparative effectiveness trials have assessed the delivery of high-quality patient-centered care for injured trauma survivors transitioning from inpatient trauma center settings to outpatient care and community rehabilitation.21,32
Additionally, relatively few investigations have made the needs and concerns of patients a central element of comparative effectiveness trial design by comprehensively incorporating patient-centered methods into screenings, treatment targets, and outcome assessments. In one of the few such randomized clinical trials, investigators randomized patients in a general medical practice to a standardized patient-centered need and concern elicitation intervention vs a care-as-usual comparator condition.32
At the time it was envisioned, the proposed comparative effectiveness model was novel in that, rather than targeting for improvement a specific physical or mental health disease entity, the investigation focused on eliciting and improving patients' posttraumatic concerns as well as the integration of each patient's unique constellation of needs and concerns into medical decision-making. The project focused on the development and implementation of patient-centered care transitions, care management, and outcome evaluation among injured trauma survivors treated in acute care medical settings. The approach developed in the application could facilitate a more central role for patient perspectives in the design and implementation of comparative effectiveness research; ultimately, the findings have excellent potential to be leveraged across disease states and service delivery contexts.
Injured trauma survivors and their family members, frontline trauma center providers, and US trauma care system policymakers need to better understand how a patient-centered care transition intervention can affect these key outcomes. A multidisciplinary team that includes patients, clinical investigators, frontline trauma center providers, and national trauma surgery policymakers implemented and evaluated the intervention.
This investigation was a comparative effectiveness trial designed to evaluate posttraumatic concerns, psychological symptoms, physical function, health service use, satisfaction with care, and injury risk behaviors in patients randomized to receive a patient-centered care transition intervention vs enhanced usual postinjury care. The study hypothesized that patients who received the novel care transition intervention would demonstrate significant improvements in a broad spectrum of patient-centered outcomes, including reductions in the number and severity of individual patients' posttraumatic concerns, reductions in posttraumatic stress and depressive symptoms, improved physical function, more efficient patterns of health service use, enhanced satisfaction with care, and reductions in injury risk behaviors.
The study multidisciplinary stakeholder team included clinical investigators (n = 5), patients (n = 6), patient advocates (n = 2), frontline trauma center providers (n = 5), and trauma center policymakers (n = 4). More than a decade before the initiation of the study, members of the study team began to ask representative samples of injured trauma survivors to voice their most pressing postinjury concerns and needs.20,21 Four years before the initiation of the study, the clinical investigator, patient, patient advocate, frontline trauma center provider, and trauma center policymaker study team members collaborated to develop national trauma center guidelines for patient-centered and psychosocial care. In May 2011, the multi-stakeholder study team organized an American College of Surgeons' policy summit that focused on mental health and patient-centered care for US trauma care systems. As part of this policy summit, patient members of the team presented in their own words their experiences of traumatic injury and recovery. While giving injured trauma survivors a “voice” at the summit, these narratives did not move surgical policymakers to develop requirements or clinical guidelines for patient-centered care. In contrast, at the May 2011 summit, presentations that included information from randomized comparative effectiveness trials and standardized outcome assessments convinced surgical policymakers to develop US trauma care system policy mandates and best practice guidelines for managing related posttraumatic stress disorder (PTSD) and alcohol use problems. The stakeholder team realized that, to optimally integrate patient-centered care into US trauma care systems, they would need to use the best scientific methods that capture the highest-quality data; the now completed PCORI proposal is the product of the multidisciplinary stakeholder team's effort.34
The composition of the study team (eg, the patient and peer advocacy stakeholder group) changed over the course of the 3-year protocol. One senior patient co-investigator and 1 senior peer advocate co-investigator had substantial input into all phases of the PCORI project. At the beginning of the project, these 2 stakeholders gave input on the study design and reviewed study instruments and protocols. Over the course of the study, regular telephone conferences (eg, monthly) between these stakeholders and the principal investigator provided opportunities for ongoing input in the study implementation. These 2 stakeholder coinvestigators encouraged the investigation to incorporate a friend and family intervention component, as well as to develop the peer pilot intervention.
For the peer pilot intervention, an additional injured peer was recruited to become the lead interventionist. This project member was a physical injury survivor who was trained by MD/PhD members of the study team to take the lead in the peer interventionist role. Over the course of the pilot study, the peer interventionist intervened with 4 peer pilot intervention cases and presented her experience of the pilot at the end-of-study policy summit. Three additional patient and peer advocate stakeholders participated in the end-of-study summit development and implementation. Thus, a diverse group of stakeholders contributed to the planning, implementation, and policy summit phases of the study protocol.
The study team, which included the clinical investigators, frontline trauma center providers, patient and peer advocate stakeholders, and national trauma center policymakers, as well as PCORI program staff, planned and successfully conducted a Patient-Centered Care Policy Summit with the American College of Surgeons Committee on Trauma; the summit was held on September 23, 2016, at the American College of Surgeons Policy offices.35 The study team orchestrated the review of 2 other comparative effectiveness trials at the policy summit (1 PCORI-sponsored trial and 1 Department of Defense–sponsored trial).36,37 The American College of Surgeons is currently considering clinical care recommendations for the incorporation of patient-centered care within US trauma care systems. The patient stakeholders and peer advocates voiced the importance of patient engagement in care within US trauma systems. The patients and peer advocates also voiced the importance of future investigations that assessed the impact of including injured peer interventionists as part of multidisciplinary trauma center teams.
The investigation—a randomized comparative effectiveness trial—evaluated 2 readily implementable approaches to the delivery of patient-centered care for injured trauma survivors treated acutely in hospitals and EDs. The 2 compared approaches—a novel patient-centered care transition service delivery model vs nurse notification of patient concerns and emotional distress that constituted a modest enhancement to usual care—were selected in part because they can be feasibly implemented in the acute care medical context. The care transition intervention included a Master's-level social worker who first elicited and then addressed each injured trauma survivor's unique constellation of posttraumatic concerns; whenever possible, the social worker attempted to integrate patient concerns and preferences into medical decision-making. The study incorporated automated electronic health record (EHR) screening for high levels of emotional distress as well as computerized decision support and caseload supervision. The intervention team also provided patients with the opportunity for 24-hour, 7-days-per-week cell phone contact that included text messaging. This intervention element aimed to enhance care coordination across acute care inpatient and surgical subspecialty and primary care outpatient service delivery sectors, as well as reduce unnecessary ED use. Patients' posttraumatic stress and depressive symptoms,38,39 functional impairment,40 and satisfaction with care30,41,42 were assessed at baseline in the surgical ward and again 1, 3, and 6 months after the injury. Patients' health service use was assessed over the course of the 12 months after injury.43
Patients included in the study were female and male survivors of intentional and unintentional injuries, aged ≥14 years, who were admitted to the University of Washington's Harborview level I trauma center (Harborview) inpatient surgical ward or ED for ≥24 hours. A previously developed electronic health record screen was used to assess the population of admitted injured trauma survivors at risk of developing high levels of emotional distress.44 The screen used 10 data elements that are both associated with increased risk for PTSD and readily available in any robust EHR.
Consenting patients who scored ≥3 on the EHR screen were subsequently evaluated for study participation. Patients included in the study were required to have ≥3 posttraumatic concerns at the time of their baseline interview as well as substantial postinjury emotional distress as manifested by a score of ≥35 on the PTSD Checklist, a score of ≥10 on the 9-item Patient Health Questionnaire (PHQ-9), or a score of ≥1 on the PHQ-9 item 9 suicide assessment.38
Patients were excluded only if they required immediate psychiatric intervention (ie, self-inflicted injury, active psychosis), were not Washington State residents, or were currently incarcerated; non-English-speaking patients were also excluded from the protocol. The University of Washington Institutional Review Board approved all study procedures before protocol initiation and each participant provided written informed consent. For adolescent participants, parental consent was obtained before adolescent assent. Study recruitment occurred over an 18-month period, from March 2014 through September 2015.
When patients declined to participate, the recruiter asked the patient why they did not want to participate. The answers were recorded verbatim and then categorized into 1 of 5 refusal reasons/categories: inconvenience, privacy concerns, not interested, undo distress, and refusal without giving a reason. Randomization occurred in a 1:1 ratio according to random assignments of blocks of 4 or 6 patients that were computer-generated by the investigation's biostatistician. Team members conducted randomization. The research assistants who conducted all baseline assessments and follow-up interviews were blinded to block sizes and intervention or control group status.
Harborview is the major level I trauma center for the Pacific Northwest; Harborview admits patients from throughout the WWAMI (Washington, Wyoming, Alaska, Montana, and Idaho region). Harborview has a large annual admission volume (approximately 6000-7000 injured trauma survivors annually); otherwise, its organizational characteristics are similar to other US academic level I trauma centers.24 These features made Harborview an ideal setting for developing and implementing the comparative effectiveness trial.
Table 1 outlines the recruitment, intervention, and follow-up elements of both conditions. The research assistants performed the baseline recruitment and concern elicitation for both intervention and control conditions. Similarly, research assistants blinded to the conditions performed the 1-, 3-, and 6-month concern assessments.
Protocol Elements Received by Patients in the Patient-Centered Care Transition Intervention vs Enhanced Usual Care Control Study Arms.
The research assistants received comprehensive training that included shadowing the principal investigator during routine Harborview consultation liaison clinical encounters. They also shadowed the principal investigator and coinvestigators for training on approaching injured patients on the ward to obtain informed consent and conduct interviews. Research assistants also were trained to deliver the 1-, 3-, and 6-month follow-up interviews; these trainings included practice in the elicitation of the posttraumatic concern items. Finally, drift assessments and recalibration retraining were periodically performed for the research assistants who conducted baseline and follow-up interviews.
The investigation compared 2 approaches for delivering patient-centered care to hospitalized injured trauma survivors with high levels of posttraumatic emotional distress: a novel patient-centered care transition service delivery model vs enhanced usual care that consisted of nurse notification of patients' postinjury concerns. The 2 approaches were selected in part because they can be feasibly implemented in the acute care medical context. Below, the 2 approaches are described in detail.
Care management treatment strategies have been flexibly used to target medical/surgical and psychiatric disorders in medical settings.42,45,46 The study team had previously developed care management approaches in comparative effectiveness trials that targeted postinjury mental health conditions and associated improvement in functional outcomes.30,31
The care management intervention element of this trial specifically targeted each patient's unique constellation of postinjury concerns. Care management procedures were derived from previously implemented acute care medical care management interventions.30,31 For patients randomized to the care management condition, the Master's of Social Work (MSW) intervention team member visited them by the bedside in the hospital ward or by the gurney in the ED. The research assistant elicited the patients' concerns; as follow-up to the research assistant concern elicitation, the MSW care manager independently elicited and targeted for improvement each patient's unique constellation of posttraumatic concerns and needs. The care manager asked about treatment preferences and scheduled ongoing times to meet or call the patient during the initial days and weeks postinjury. Treatment preferences the care manager might have discussed with the patients included the option of receiving continued care from specialty surgical providers vs receiving care from a new or preexisting primary care provider. For mental health specialty care, treatment preferences might have included the option of receiving ongoing counseling from the care manager vs an early, stepped-up referral to a community mental health provider who could address more intensive emotional distress or counseling needs. The care manager might also assist in locating a community provider who could address patients' psychotropic medication needs. The care manager also gave the patients the study team's 24-hour/7 days per week telephone contact number and encouraged the patients to text or call with questions, needs, and concerns. Care management patients were given a choice of treatment options, and the care manager shared information and deliberated medical treatment decisions with each patient to develop a patient-informed, individually tailored treatment plan.19 For care management patients with high levels of postinjury mental health symptoms, psychiatric consultation and community mental health referrals were made available.
As in prior study team trials, stepped-up care was available for patients with specific postinjury symptomatic presentations. For patients with alcohol use problems or other risk behaviors, the care manager delivered a motivational interviewing intervention.30,31 For patients with high levels of PTSD and/or depressive symptoms, the care manager delivered cognitive behavioral therapy elements.47 Psychopharmacologic consultation and community referral was also recommended as stepped-up care procedures for symptomatic patients. Patients received the intervention over the course of the 6 months after the injury event.
Patients in the control condition underwent informed consent, both electronic health record and in-person PTSD screenings, baseline surgical ward evaluation, and blinded follow-up interviews. At the termination of the surgical ward/ED interviews, the study research assistant contacted the nurses of all patients in the usual care arm of the investigation. With each patient's nurse, the research assistant reviewed the nature and severity of the individual patient's posttraumatic concerns and, if pertinent, level of emotional distress. Nurse notification of patient concerns constitutes an enhancement to usual trauma center/ED posttraumatic care. Nurse notification was selected because it represented an optimal, feasibly implemented comparator condition for comparative effectiveness trials targeting American College of Surgeons' policy.48 Of note, while existing guidelines recommended that nurses follow up on patients concerns, competing time demands may have preempted some of these efforts for enhanced usual care patients.
Prior investigation suggests that usual posttraumatic care after hospital discharge includes routine surgical, primary care, and ED visits, as well as the occasional use of specialty mental health services.30,31 In this investigation, all usual care ED use was captured using the Emergency Department Information Exchange System (EDIE).43
ED health service use was assessed using the EDIE to collect population-based usage data over the course of the 12 months after the injury; this follow-up outcome assessment extended 6 months beyond patients' exposure to the intervention. Patients' posttraumatic concerns, symptomatic, risk behavior, and functional outcomes were assessed by patient self-report at baseline and again 1, 3, and 6 months after the injury event. Research assistants collected these assessments through telephone interviews, which have been found to be reliable and valid in the assessment of injured trauma survivors.42,49,50 To minimize bias, the research assistants who conducted the telephone follow-up interviews were blinded to the patients' study group assignments.51
Table 2 describes the timing and administration of all study outcome assessments. Further detail of specific assessments and measures is provided below.
Study Assessments and Timing of Administration.
As in prior study team investigations,20,21 the baseline and follow-up interviews began with the assessment of each patient's unique constellation of postinjury concerns. The baseline, 1-, 3-, and 6-month concern assessment asked each patient, “Of everything that has happened to you since you were injured, what concerns you the most?” Patients could express an unlimited number of concerns. At each follow-up time point, research assistants reassessed the presence or absence of previously endorsed concerns. Following each concern elicitation, patients were asked to rate the severity of the concern on a scale from 1 to 5, with 1 being not at all concerning and 5 being extremely concerning. Based on a prior study team investigation, a severe concern was defined as a concern that the patient rated as a 5.20,21 The study team assessed whether each patient expressed 1 or more severe concerns at each time point; the study team used this dichotomized assessment in the outcome analyses. The study team also calculated the average severity rating for all concerns for each individual patient at each time point, and used this continuous variable in the outcome analyses.
Procedures for coding the posttraumatic concern domains were derived from previously described content analytic methods.20,21 A previously developed code book that described concern domains and coding procedures was used.20,21 The initial concern question and its explanation constituted the unit of analysis. Raters independently coded each concern into 1 of the previously established concern domains.20,21 The frequency of patients reporting themes from 1 or more domains along with the concern severity was tabulated.20,21 Approximately 10% (80/735) of concerns were coded by 2 raters. A κ statistic was used to assess interrater reliability, with values ranging from 0.77 to 0.78.
The PTSD Checklist, a 17-item self-report questionnaire, was used to assess PTSD symptoms.38 The instrument yields both a continuous PTSD symptom score and a dichotomized diagnostic cut point for symptoms consistent with a DSM-IV diagnosis of PTSD.52 A series of investigations have demonstrated the reliability and convergent and construct validity of the PTSD Checklist across trauma-exposed populations.38,53,54 Increasing symptom scores on the PTSD Checklist have been linked to such important postinjury outcomes as reductions in physical function and impairments in the return to work.12 At baseline, patients were asked such questions as, “Since the event in which you were injured, how often have you been bothered by repeated, disturbing memories, thoughts, or images of the event in which you were injured?” Patients were then asked to respond using a 5-point Likert scale (1 = “not at all” to 5 = “extremely”). PTSD symptoms for the month prior were assessed again during the 1-, 3-, and 6-month follow-up interviews. In a prior investigation with injured trauma survivors conducted by the study team, Cronbach's α for the 17-item scale was .92.30,42 In a study of injured motor vehicle crash survivors, a correlation of 0.93 was documented between the PTSD Checklist total score and the gold standard Clinician-Administered PTSD Scale diagnostic scale.53
The investigation used the PHQ-9 to assess depressive symptoms. The instrument yields both a continuous depressive symptom score and a dichotomized diagnostic cut point for symptoms consistent with a DSM-IV diagnosis of depression. The PHQ-9 item asks specifically about suicidal ideation and/or intent. The PHQ-9 has established reliability and convergent and construct validity when used to assess patients in general medical settings.39 At baseline, patients were asked such questions as, “In the past month, how often have you been bothered by little interest or pleasure in doing things?” Patients were then asked to respond using a 4-point Likert scale (0 = “not at all” to 3 = “nearly all the time”). During the 1-, 3-, and 6-month follow-up interviews, the PHQ-9 was administered for the month before the interview. In a prior investigation with injured trauma survivors conducted by the study team, Cronbach's α for the 9-item scale was .97.30,42
The investigation used the SF-36 to assess functioning and quality of life outcomes. The 8 domains assessed included physical function, pain, general health, role physical function, role emotional function, vitality, social function, and mental health. The SF-36 has established reliability and validity and the measure has been used extensively with traumatically injured populations.40,42,55,56 The study team used the Physical Component Summary (PCS) subscale to assess postinjury physical function. In prior investigations conducted by the study team, Cronbach's α for the MOS SF-36 PCS was .90.30,42
ED health service use was assessed using the EDIE system, developed by Collective Medical Technologies.43,57 EDIE is a novel clinical informatics tool that aggregates ED visits in real time for the population of patients who present to any ED in Washington and Oregon. EDIE is currently integrated into the medical record at the University of Washington and Harborview Medical Center. For the purposes of this trial, EDIE allowed a population-based, 12-month follow-up of all ED visits across Washington and Oregon for the intent-to-treat sample of intervention and control patients.
Items that assessed satisfaction with posttraumatic care were adapted from previous studies of care management interventions for patients in primary and acute care medical settings.30,41,42
The study team used the Alcohol Use Disorders Identification Test (AUDIT), a 10-item screening instrument for the early identification of problem drinkers,58 to assess alcohol use problems. At baseline, the AUDIT evaluated drinking behaviors in the year before the injury. The reliability and validity of the AUDIT are well established in traumatic injury populations and the scale has been widely used as a screening instrument in general medical settings.58 The Drug Abuse Screening Test was used to assess drug use problems.59 A single item assessed whether patients had a recent history of weapon carrying.60
Over the past 2 decades, the study team has developed research methods that allow for the empirical assessment of reactions to research participation, including participant burden.26 These methods involve asking all research participants key questions that weigh the costs and benefits of research study participation. The participants were asked to respond to the four following statements: “Had I known in advance what participating would have been like, I still would have participated”; “I gained something positive from participating”; “Participating upset me more than I expected”; and “I felt free to skip questions and/or parts of the study.” Patient participants could respond with “true,” “mostly true,” “uncertain,” “mostly false,” or “false.” All 171 patient participants were asked these items at baseline in the surgical ward and again during the 1-, 3-, and 6-month follow-up interviews.
The investigation determined injury severity at baseline during the index admission from the medical record ICD-9-CM using the Abbreviated Injury Scale and Injury Severity Score.61 Traumatic brain injury and chronic medical conditions were also prospectively identified in the medical record.49 Race, ethnicity, and gender were assessed through patient self-report. Laboratory toxicology results, insurance status, length of hospital and intensive care unit stays, and other clinical characteristics were abstracted from the electronic health record and trauma registry. The study included many other assessments of clinical, demographic, and injury characteristics, which are listed in Table 2.62-66
A previously developed EHR screen was used to assess the population of admitted injured trauma survivors at risk for developing emotional distress.44 The screen used 10 data elements that are both associated with increased risk for posttraumatic stress disorder and readily available in any robust electronic health record system. In a prior investigation, when the 10 data elements were used to predict scores on the PTSD Checklist of ≥35, the electronic health record screen demonstrated adequate sensitivity (0.71), specificity (0.66), and area under the receiver operating characteristic curve (0.72).44 Consenting patients who scored ≥3 on the electronic health record screen were subsequently evaluated for their number of posttraumatic concerns. The 10 data elements include an ICD-9-CM PTSD diagnosis, other ICD-9-CM psychiatric diagnosis, other ICD-9-CM substance use diagnosis or positive blood alcohol on admission, tobacco use, female gender, non-White ethnicity, uninsured, public or veteran insurance status, intentional injury (eg, physical assault, stabbing, gunshot or other violent trauma), and intensive care unit admission and EHR documentation of any prior trauma center visits.
The research assistant administered the concern assessment to each consenting patient; the assessment asked, “Of everything that has happened to you since you were injured, what concerns you the most?” Subsequent open-ended items further explored the nature and severity of each concern. Patients could express an unlimited number of concerns. Following the administration of these open-ended questions, patients were asked to rate the severity of each concern on a scale from 1 to 5, with 1 being not at all concerning and 5 being extremely concerning. Next, patients were administered the PTSD Checklist (PCL-C) and the PHQ-9. Patients were screened into the randomized portion of the study if they had ≥3 concerns and either a score ≥35 on the PCL-C, a score ≥10 on the PHQ-9, or a response ≥1 on item 9 of PHQ-9. These screen-in criteria were used because, in previous investigations, injured trauma survivors with ≥3 concerns and high scores for PTSD and/or depression demonstrated the greatest symptomatic and functional impairment over the course of the year after injury and thus may derive the greatest benefit from study intervention procedure.
Upon recruitment, patients were asked to provide at least 2 contact information items for themselves (eg, phone number and mailing address). Recruiters probed patients for as many contact information items as possible, including alternate contact sources such as family or friends. If a patient could no longer be reached through his or her personal contact information, follow-up interviewers contacted the patient's alternate contact sources to either collect new contact information for the patient or pass a message on to them. If interviewers were unable to reach patients after repeatedly trying to contact them through the information provided at recruitment, the follow-up interviewers searched for new information in the following ways: Check Harborview medical records for new information; search Yellow Pages; use search engines (eg, Google, Bing); or search social media (eg, MySpace, Facebook, Google+). Patients who could not be contacted or were deceased were considered lost to follow-up.
The study hypothesized that the novel care transition intervention, when compared with nurse notification, would be associated with significant improvements in patient-centered outcomes, including standardized assessments of the severity of individual patients' posttraumatic concerns. The study also hypothesized that intervention patients would demonstrate reductions in the percentage of concerns across medical/physical and mental health domains. The study also hypothesized that patients who received the intervention would demonstrate reductions in posttraumatic stress disorder and depressive symptoms when compared with patients receiving enhanced usual care. Secondary outcomes compared across intervention and control conditions included physical function, injury risk behaviors, physical and emotional satisfaction with care, and automated statewide health service use. Exploratory analyses assessed the impact of the intervention on outcomes for patients with and without a traumatic brain injury. To further assess for treatment effect heterogeneity, gender, ethnicity, and age were tested as interactions with treatment effects.
A detailed analytic plan was created before the conduct of data analyses. The prespecified primary outcome analysis began with assessments of concern severity and domains. Two concern severity variables were used as the dependent variable in the outcome analyses: the dichotomous assessment of whether each patient expressed 1 or more severe concerns at each time point and the continuous assessment of the average severity rating for all concerns of each individual patient at each time point. For the concern assessments, the study team used both the full sample of N = 171 patients and the subsample of patients who endorsed enduring concerns at each time point (n = 102). Changes in concern domains over time were also assessed.
Next, the primary outcome analysis examined changes over time across groups for PTSD and depressive symptoms. The analysis plan specified the secondary outcomes to be assessed, including physical function, health service use, satisfaction with care, and injury risk behaviors.
The investigation used mixed effects regression models to test hypotheses.67-69 The effect of major interest was the time by treatment group interaction term. Regression models included time categories, intervention, and intervention by time interactions. Other analyses adjusted for relevant clinical and demographic characteristics, including age, gender, race, and traumatic brain injury. Longitudinal data collected prospectively from injured trauma survivors was characterized by correlated intra-individual observations, missing data, and dropouts. Mixed-effects random-coefficient regression methods were selected because of their superior ability to model longitudinal data with these characteristics. The study team used SAS version 9.4 and SPSS version 24.0 for all analyses.
Some attrition was expected in the study sample. Prior studies by the investigative group have consistently achieved follow-up completion rates ≥80% between 6 and 12 months postinjury.30,31 Assumptions about the nature of missing data are important to the type of statistical analysis chosen. Full information maximum likelihood estimates from mixed random-effects models at least accommodate missing data that are missing at random (MAR); MAR data depend on previously observed variables.67,70-72 The study team used statistical logistic models to determine which, if any, demographic or clinical characteristics, including treatment group membership, were predictive of subject attrition. Any factors observed to explain trends in missing data were used as covariates in subsequent analyses. Past studies found no sources of consistent variation to explain missing data.23,24 Based on the relatively low attrition rates and the inability to find consistent variation in past investigations, the study team believes that MAR is a reasonable assumption.
Criteria and standards for the approach to treatment effect heterogeneity were derived from the PCORI Methodology Report published for public comment on July 23, 2012.73 Because of the limited information in the prior literature and the lack of large studies with subgroup analyses, the approach to heterogeneity of treatment effect (HTE) was considered somewhere between a descriptive and exploratory HTE analysis. Prior analyses suggest that patients of different ages, genders, and ethnicities who have traumatic brain injuries may experience varying impacts of traumatic injury regarding PTSD symptoms, depressive symptoms, and physical function outcomes.42,74-79 However, none of these investigations have previously established variations in posttraumatic concerns or severity in response to these subgroup factors. Therefore, the study adopted a descriptive/exploratory approach to the heterogeneity of treatment effect analyses. To explore for treatment effect heterogeneity, the study team examined treatment effect by time interactions for gender, age, traumatic brain injury, and ethnoracial group. It was understood a priori that the study would not be adequately powered to examine differences across these multiple subgroups.
Please reference previous sections on the methods and outcome variables for explication of the study protocol. First, the study added an injured trauma survivor family and friend protocol element at the behest of the patient and patient advocate stakeholders on the investigative team. Second, in the third year of the study, a small (n = 4) injured peer interventionist intervention was conducted, also at the recommendation of the patient and patient advocate stakeholders.
Compared with all other patients admitted to the trauma center during the investigation period, the 171 randomized study patients were significantly more likely to be female, non-White, more severely injured, hospitalized in the intensive care unit (ICU), and have a greater overall length of hospital stay (Table 3).
Trauma Registry Population vs Study Patient Comparisons.
For the 171 randomized patients, the mean age was 42.4 years (SD, 16.0) and 56.7% (n = 97) of patients were female (Table 4). Race and ethnicity were self-reported: 56.1% (n = 96) of patients identified as White, 15. 8% (n = 27) identified as African American, 12. 9% (n = 22) identified as American Indian, 7.0% (n = 12) identified as Hispanic, 5.3% (n = 9) identified as Asian, and 2.9% (n = 5) identified as Pacific Islander. Injury events in the population included falls (31.6%, n = 54), motor vehicle crashes (23.4%, n = 40), motorcycle crashes (12.9%, n = 22), pedestrian accidents (10.5%, n = 18), physical assaults (7.6%, n = 13), gunshots (4.1%, n = 7), bicycle accidents (3.5%, n = 6), stabbings (2.3%, n = 4), work injuries (1.2%, n = 2), sports injuries (0.6%, n = 1), and other (2.3%, n = 4). Intervention patients were less likely than control patients to have 1 or more chronic preinjury medical conditions (χ2 [3] = 9.23; P < .05); no other significant differences existed between intervention and control patients on any demographic, clinical, or injury characteristics.
Baseline Patient Characteristics.
Fifty-six patients declined to enroll in the project. Of those who declined, 21 patients identified the time commitment and/or inconvenience of the protocol as their reason for declining, 11 patients were concerned about their privacy, 11 patients stated they were not interested, 7 patients felt that taking part in the study would cause them distress, and 6 patients refused to give a reason for not wanting to participate.
The investigation attained ≥80% follow-up at each time point for patients randomized to the intervention and control conditions (Figure 1). Of the 171 patients, 26% were missing 1 or more assessment at the 1-, 3-, or 6-month time point. Observation from the CONSORT diagram (Figure 1) and additional analyses that broke down lost to follow-up categories revealed some differences across the 2 groups that the study team believes did not achieve clinical significance because of the relatively small number of patients in each group. Three patients in the intervention group, but no patients in the control group, withdrew from the study. Two patients in the intervention group, but no patients in the control group, died during the 6-month protocol. Fourteen patients in the control group and 8 patients in the intervention group were not able to be contacted at the 6-month time point. Of the patients who were missing follow-up data, 7 control patients and 5 intervention patients completed only the baseline interview, zero control patients and 2 intervention patients completed only the baseline and 1-month interviews, and 8 control patients and 6 intervention patients completed only the baseline, 1-, and 3-month interviews.
Patient Flow Through Protocol (CONSORT).
Regression analyses that systematically assessed patterns of missing data revealed that only gender was associated with missing assessments (61% of men vs 38% of women; P < .01). Men and women were equally balanced, however, across intervention and control group conditions (Table 4).
On average, the total time spent with each intervention subject was 6.49 hours (SD = 4.57 hours). More than half of patient clinical encounters occurred over the telephone. Approximately 95% of patients (81/85) had at least 1 care management telephone contact over the course of the 6 months after the injury. Approximately 65% (55/85) of patients communicated with the study by text messaging; 63.5% (54/85) of patients sent texts to the study team's 24/7 cell phone and 64.7% (55/85) of patients received texts from the study team. Of patients, 94% (80/85) had any voicemail communication with the study team; 87.1% (74/85) left a voice message for the study team and 94.1% (80/85) received at least 1 voice message from the care manager interventionist or other study team members.
Approximately a third of intervention patients (n = 27) received 1 or more motivational interviews targeting alcohol use. Most intervention patients (N= 76) were engaged in the delivery of cognitive behavioral therapy elements. Also, the need for psychopharmacologic intervention was assessed in most intervention patients (n = 74).
Patients in both the intervention and control groups reported a variety of concerns that spanned the 6 previously articulated content domains (Table 5). Regression analyses in both the intent-to-treat sample and the subsample of patients with enduring concerns demonstrated significant reductions in the percentage of any severe concerns in intervention patients when compared with controls. At the 3-month postinjury time point, 71.4% of control patients vs 76.2% of intervention patients endorsed 1 or more severe concerns. At the 6-month postinjury time point, 73.7% of control patients vs 52.4% of intervention patients endorsed 1 or more severe concerns (Fisher exact test, P = .02; Table 6). For the continuous analyses of mean concern severity, significant reductions among intervention patients relative to controls were observed in the subsample with enduring concerns but not in the intent-to-treat sample (Table 6). No significant differences in concern domains were observed for intervention and control group patients over time (Table 6).
Posttraumatic Concern Domains: Definitions and Examples.
Posttraumatic Concerns Severity and Domains over Time.
No clinically or statistically significant differences between intervention and control group patients were observed for the PCL-C and PHQ-9 longitudinally over the course of the 6 months after the injury (Table 7). Similarly, intervention and control group patients did not demonstrate significant 1-month, 3-month, or 6-month differences in either PTSD or depressive symptom levels.
Symptomatic and Functional Outcomes Over time.
There were no significant group differences in physical functioning as assessed by the MOS SF-36 over the course of the 6 months after the injury (Table 7). Correspondingly, intervention and control group patients did not demonstrate baseline, 1-month, 3-month, or 6-month differences in physical function. Patients in the 2 groups demonstrated no significant differences in alcohol-, drug-, or violence-related injury risk behaviors over time. Comparisons between intervention and control patients on measures of physical and emotional satisfaction with care did not attain statistical significance.
Patients in the intervention group demonstrated clinically important reductions in ED visits over the course of the year after injury hospitalization (Figure 2). The intervention was associated with clinically significant cross-sectional differences (1 or more 3-to 6-month ED visits; C = 30.2%, I = 16.5%; relative risk [95% CI], C vs I = 2.00 [1.09-3.70], P = .03) that did not achieve longitudinal statistical significance (F3,507 = 2.24, P = .08).
One or More ED Visits Over Time.
Because patients in the intervention group were less likely to have 1 or more chronic conditions, and chronic conditions may influence ED use, regression models were adjusted for chronic conditions. In these adjusted models, the intervention remained associated with nearly identical clinically significant cross-sectional differences (P = .03) that again did not achieve longitudinal statistical significance (P = .08).
The investigation did not find any significant treatment group by time interaction effects for race, age, or traumatic brain injury across outcome domains. For gender, a significant group by time by ED use interaction was identified such that at the 3- to 6-month postinjury time point, women in the intervention group were significantly more likely to demonstrate reductions in ED use (F3,501 = 3.08, P < .05).
For the first item, “Had I known in advance what participating would have been like, I still would have participated,” 97% of participants responded favorably with “true” or “mostly true” at baseline in the surgical ward. In the follow-up interviews, between 93% and 98% of participants responded favorably to this item. For the item “I gained something positive from participating,” between 84% and 87% responded “true” or “mostly true” over the 4 time points. For the item “Participating upset me more than I expected,” 77% to 89% responded “false” or “mostly false” over the 4 study time points. For the item “I felt free to skip questions and/or parts of the study,” between 93% and 99% of patients responded “true” or “mostly true” over the 4 study time points.
This investigation contributes to the evolving literature on patient-centered early interventions for injured trauma survivors who are initially treated in surgical inpatient settings and then require continuous care transitions. The investigation demonstrates the feasibility and suggests the effectiveness of a patient-centered care transition intervention in reducing posttraumatic concern severity and reducing postinjury ED health service use. This initial investigation could have important implications for US trauma care systems nationally. Trauma centers could consider customizing trauma care programs to incorporate elements of patient-centered care. Key program elements for consideration include the provision of a continuous helping relationship that links the trauma center to community services and includes eliciting and addressing patients' most pressing needs and concerns after injury.
The comparative effectiveness trial was novel in that posttraumatic concerns constituted a primary trial screening and intervention focus. The results of this investigation suggest that a patient-centered care transition intervention significantly reduces the severity of concerns expressed by injured trauma survivors. Patients in the intervention group demonstrated significant reductions in the percentage of any severe concerns expressed over the course of the 6 months after the injury. These reductions were most prominent at the 6-month endpoint, when approximately half of intervention patients expressed at least 1 severe concern as compared with three-quarters of control patients.
The literature review revealed 1 other investigation that used a patient-generated concern construct as a primary intervention target. In 1 of the few such randomized clinical trials, McLean and Armstrong randomized patients in a general medical practice to a standardized patient-centered need and concern elicitation intervention vs a care as usual comparator condition.32 Patients randomized to the concern elicitation condition reported increased satisfaction with health care visits.
The trial was also innovative because it used a population-based ED registry, EDIE, to track health care use outcomes associated with the care transition intervention. The investigation found substantial reductions in ED use among patients receiving the care transition intervention, even when adjusting for the presence of preexisting chronic medical conditions. Over the course of the year after injury, intervention group patients demonstrated clinically significant cross-sectional reductions in ED use that did not achieve longitudinal statistical significance.
Prior investigations by the study team have used stepped collaborative care interventions with care management components to effectively target postinjury PTSD symptoms and associated functional impairments.30,31 This investigation was innovative because postinjury concerns were directly targeted and followed as a distinct patient-centered outcome. Although the intervention reduced posttraumatic concern severity, concern domain, PTSD and depressive symptoms, physical function, and injury risk behaviors were not significantly affected. For treatment process measures, the care transition intervention was not associated with increased psychotropic medication usage or enhanced care satisfaction. The social worker interventionists' clinical directives differed from those in recent acute care collaborative care trials that have successfully targeted care processes and symptomatic and functional outcomes; in this investigation, the social worker often focused on eliciting and targeting for improvement posttraumatic concerns rather than symptomatic targets.30,31 The pattern of postinjury severe concern reduction from 3 to 6 months deserves further comment. In observing the 3- and 6-month postinjury concerns results, at the 3-month time point, the intervention and control group are nearly equivalent in the percentage of any severe postinjury concerns expressed. At the 6-month time point, however, a separation occurs between the 2 groups, with the intervention manifesting reductions in the percentage of any severe concerns relative to the control group. Thus, there is a difference in the direction and magnitude for severe concerns across intervention and control groups between the 3- and 6-month time points.
Interestingly, the 3- to 6-month pattern of reduction in the severity of any severe concerns mirrors the pattern of 3- to 6-month intervention and control group differentiation in prior stepped collaborative care trials that have targeted PTSD.30,31 Future investigations could further explore the observation that, regardless of the primary outcome target, multifaceted interventions in acutely injured trauma survivors may require 3 to 6 months of intervention activity to achieve clinically and statistically significant differentiation between intervention and control group patients.
Based on the study team's experience of the investigation's rollout, the study team believes that the intervention can be feasibly delivered by frontline trauma center providers. On average, the intervention required just over 6 hours of interventionist time per patient; this time commitment could present a barrier to implementation among frontline acute care providers. Other barriers include the need for 24/7 cell phone coverage.
The investigation is hypothesized to be highly generalizable to other US level I trauma centers. The characteristics of the Harborview level I trauma center, beyond admission volume, are typical of other US level I trauma center sites. Prior behavioral interventions initiated at Harborview have demonstrated excellent national implementation potential.24,30,31,80 Also, patients with PTSD are more likely to be female and from diverse backgrounds, which may explain some of the differences between the study sample and the external population for sampling at the Harborview level I trauma center.
The investigation did not detect substantial differences in intervention effects by traumatic brain injury status, race/ethnicity, gender, or age. The investigation may have been underpowered to detect treatment effect heterogeneity.
Because this intervention was multifaceted, the investigation did not yield information regarding which components of the treatment were effective in targeting specific outcomes. Thus, the study team can only postulate that the patient-centered care management procedures may have targeted reductions in concern severity, while telephone availability and care coordination may have been associated with observed reductions in ED health service use. Future investigations could test specific intervention components in relation to hypothesized outcome effects, or they could attempt to disentangle the effects of these individual components of the multifaceted intervention.47 Also, the intervention extended to the 6-month injury time point and the investigation did not conduct a formal posttreatment outcome assessment. The study was conducted at a single level I trauma center and may not generalize to diverse US trauma care systems nationally. The results also may not generalize to acute care ED settings nationwide. The investigative sample was relatively small and included a large subpopulation of ethnically diverse women. The study team acknowledges that an enhancement to usual postinjury care, which included nurse notification of patient postinjury distress, was used as the comparator condition.
Finally, these findings are limited by challenges in the naturalistic follow-up of posttraumatic concerns over time. Although attempts were made to capture concerns at each time point, patients had difficulty with comprehensively listing all their endorsed concerns. Further methodologic refinement of concern coding and quantitative analyses could be a productive area for future investigations. The study team also notes that although a systematic process was in place to train the research assistants to conduct the baseline and follow-up interviews, no reliability estimates or kappa statistics were obtained that specifically addressed concordance for the concern elicitation procedure. However, the study team has no reason to believe that systematic differences would occur in the way concerns were elicited for patients in the intervention and control conditions by blinded research assistants. Therefore, the study team believes there would be minimal overall impact of variations in concern elicitation for the results of the comparative effectiveness trial.
Patient stakeholder study team members have voiced the importance of peer-integrated injury interventions. On September 23, 2016, the study team convened a PCORI-sponsored policy summit with the American College of Surgeons' Committee on Trauma, the regulatory governing body for US trauma care systems nationally. A major summit recommendation was that additional research on peer-integrated interventions was required to inform American College of Surgeons' regulatory policy. A possible future research direction would be a comparative effectiveness trial of a multidisciplinary team care transition intervention that integrates injured trauma survivor peer interventionists.
The investigation randomized 171 patients to intervention (n = 85) and nurse notification group conditions (n = 86). Intervention patients demonstrated clinically and statistically significant reductions in the percentage of any severe postinjury concerns when compared with nurse notification patients longitudinally (Wald χ2 = 11.29; P = .01) and at the 6-month study time point (C = 74%, I = 53%; Fisher exact test, P = .02). Comparisons of ED use data yielded clinically significant cross-sectional differences (1 or more 3- to 6-month ED visits; C = 30.2%, I = 16.5%; relative risk [95% CI], C vs I = 2.00 [1.09, 3.70], P = that did not achieve longitudinal statistical significance (F3,507 = 2.24, P = .08). The intervention did not significantly affect symptomatic, risk behavior, or functional outcomes.
To the study team's knowledge, this is the first care management intervention to document reductions in patient postinjury concerns at the 6-month postinjury time point. Also, this is the first investigation to report an association between reductions in statewide ED use and a patient-centered care transition intervention.
The investigation has many limitations that contextualize the study findings. The intervention extended to the 6-month injury time point and the investigation did not conduct a formal posttreatment outcome assessment. Because the intervention was multifaceted, the investigation did not yield information regarding which components of the treatment were effective in targeting specific outcomes. The investigation did not find substantial gender, age, ethno-racial, or traumatic brain injury group differences in treatment outcomes; however, the investigation may have been underpowered to detect these subgroup differences.
In May 2011, the study team convened an American College of Surgeons' policy summit and patient stakeholders voiced the need for patient-centered care transition interventions. This comparative effectiveness trial provides initial evidence that a patient-centered care transition intervention can reduce patient concern severity and diminish ED use. These initial findings are provocative and could lead to further investigation to develop a body of knowledge that would inform sustainable American College of Surgeons' clinical guideline requirements for US trauma care systems.
Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (IH-1304-6319). Further information available at: https://www.pcori.org/research-results/2013/does-care-management-help-patients-recover-serious-injury
Zatzick DF, Russo J, Thomas P, et al. (2018). Does Care Management Help Patients Recover from a Serious Injury? Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/8.2018.IH.13046319
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
