An official website of the United States government
NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
James P. Guevara, MD, MPH, Alexander Fiks, MD, MSCE, Thomas Power, PhD, Katherine Bevans, PhD, Lisa Snitzer, MSS, Siobhan Leavy, EdD, Denise Stewart, BS, Caroline Broomfield, BS, Stephanie Liu, MS, Salima Shah, BS, Robert Grundmeier, MD, Jeremy Michel, MD, Steven Berkowitz, MD, Nathan Blum, MD, Matthew Bryan, PhD, and Heather Griffis, PhD.
Author Information and AffiliationsChildren with attention-deficit/hyperactivity disorder (ADHD) experience fragmented health care among families, teachers, and clinicians. Fragmentation can result in poor communication, coordination of services, and outcomes. Requirements for “meaningful use” of electronic health records (EHRs) have elevated the role of patient portals as a standard communication tool for patients with chronic health conditions. Similarly, care managers are increasingly used for communication and care coordination for patients with chronic health conditions.
We aimed to (1) compare the effectiveness of an electronic patient portal designed to communicate family-centered treatment preferences and goals to providers and educators (tier 1) with the electronic patient portal combined with a care manager to facilitate communication and care coordination (tier 2) for children with ADHD on ADHD symptoms (primary), goal attainment (secondary), and patient-reported outcomes (secondary); (2) assess treatment initiation and adherence and family engagement as mediators of intervention treatment effects; and (3) explore individual, family, and community factors that moderate intervention treatment effects.
The study design was a randomized clinical trial. We recruited 11 urban and suburban primary care pediatric practices. Children were eligible if they had ADHD, were 5 to 12 years old, and had received ADHD treatment from participating practices in the past year. Children were excluded for past year histories of autism and/or bipolar, psychotic, conduct, or suicidal disorders. Eligible children were consented and randomly assigned 1:1 within site to tier 1 (online patient portal only) or tier 2 interventions (online patient portal plus care manager). Both groups used an online patient portal that included secure online access to an ADHD dashboard containing family treatment preferences and goals, parent and teacher ADHD symptom scales, ADHD resources, and parent-teacher information-sharing. Care managers provided ADHD education to families; communicated quarterly with parents, teachers, and clinicians; and coordinated care with the patients' ADHD care team. Measures included changes in ADHD symptoms (Vanderbilt Parent Rating Scales [VPRS]) as the primary outcome; changes in goal attainment (Goal Attainment Scale [GAS]) and patient-reported outcomes (PROs) (parent-proxy and child-reported PROs) as secondary outcomes; and mental health services use (Services Assessment for Children and Adolescents) and family engagement with treatment (ADHD Engagement Scale [AES]) as mediators. Parents completed measures at study visits 1, 2, 3, and 4, while children 8 to 12 years old completed child PROs at study visits 2, 3, and 4. Patient addresses were geocoded to census tracts and overlaid with census and American Community Survey data. Differences in outcome measures between groups were assessed using intention-to-treat analysis and generalized linear and marginal mixed-effects models. Mediation analyses involving treatment initiation and adherence and family engagement were proposed for statistically significant outcomes. Geocoded data on participants supplemented individual and family data to explore individual, family, and community factors moderating treatment effects. Stakeholder parent, teacher, and clinician partners participated in biannual meetings and provided recommendations regarding study planning, conduct, analysis, and dissemination of findings. Nineteen parents in the tier 2 intervention completed semistructured interviews concerning their involvement with the care manager.
A total of 303 eligible children and their parents were enrolled. Most children were 8 to 12 years old, male, in public schools, and taking ADHD medications at study visit 1. There were no between-group differences in demographic or neighborhood characteristics. A total of 206 (68%) parents and 92 (30%) teachers used the patient portal during the study period with no differences between groups. Care managers completed a mean of 2.2 (range, 0-6) care management sessions with parents and 0.5 (range, 0-3) sessions with teachers. In multivariate models adjusted for seasonality, race/ethnicity, urbanicity, parent education, income, and medication status, there were no significant intervention × time effects on VPRS, parent PRO (school performance, student engagement, peer relationships), child PRO (school performance, student engagement, peer relationships, family relationships, teacher connectedness), or GAS scores. At study visit 4, there were no between-group differences in mental health services use (64% vs 72%, P = .31) or in AES scores. Given no significant findings, mediation analyses were not performed. In interviews, parents described sporadic contact and lack of face-to-face engagement with care managers as limiting their engagement. No significant moderators were identified.
Care management did not produce outcomes that differed from patient portal use alone among children with ADHD. However, there was poor engagement with care managers, which likely influenced results. Future studies should consider methods to better engage families with care managers and patient portals to improve outcomes.
This study took place in a single geographic area and integrated health system. Results may not be generalizable to other areas and systems. Care managers primarily communicated via email, text, and phone, which improved feasibility but limited acceptability among some families who preferred in-person interactions.
Attention-deficit/hyperactivity disorder (ADHD), which is characterized by inattention, impulsivity, and hyperactivity,1-4 is the most common chronic behavioral disorder in children.5,6 According to estimates from the National Survey of Children's Health, 5.4 million children aged 4 to 17 years in the United States have been diagnosed with ADHD.7 Prevalence estimates range from 4% to 12% of school-age children in community samples,6 with boys diagnosed more than twice as often as girls (13.2% vs 5.6%, respectively).7,8 Indicative of the long-term impact of the condition, 29% to 65% of children with ADHD continue to exhibit symptoms into adulthood.9,10 The total costs of ADHD are estimated to be greater than $50 billion, with affected children having health care costs 3 times higher than those without ADHD.11
ADHD may result in significant impairment in areas of profound importance to affected children and their families, including self-esteem, interpersonal relationships, and academic achievement.12 In addition, co-occurring chronic conditions are common and increase the burden of ADHD on individuals, families, and society.6,8,13 The long-term prognosis for children with ADHD is concerning: They are less likely to graduate from high school than unaffected children,12,14 more likely to suffer from poor social relations and delinquency,15,16 and rate significantly lower in health-related quality of life in all psychosocial areas.13 Untreated adolescents and young adults consistently demonstrate higher rates of substance abuse, driving infractions, delinquency, and poorer employment outcomes than those who are treated.14,17-20
Evidence-based treatments improve outcomes for children with ADHD. The effectiveness of treatment for ADHD is supported by a large number of clinical trials and consists of psychotropic medications (eg, methylphenidate), behavior therapy, and school-based modifications either alone or in combination.21-27 Unfortunately, engagement in and adherence to ADHD treatment is poor, limiting the effectiveness of evidence-based strategies. Despite the availability of proven treatments, many children with ADHD do not receive evidence-based care over the long term.28 Research has found poor adherence and inconsistent use of effective treatments in community studies, with only a third of children receiving treatment over a year.29-31 Of those who are treated, a majority of children (80%) receive medication, and less than half receive behavioral interventions.31 Even among those receiving appropriate medication, about 25% of children receive only a single stimulant prescription in a year.32,33 Less than half receive monthly or more frequent mental health visits.28 Although poor treatment adherence is a problem for many children with ADHD, it particularly affects youth of ethnic minorities.34 Male and White children are more likely to receive consistent medication prescriptions than female and Black or Hispanic children.35 These differences in treatment adherence might result from concerns about medication safety, despite evidence showing a well-established safety profile for most ADHD medications.36 Additionally, the use of treatments that do not match families' treatment preferences or social and cultural norms may reduce treatment initiation and long-term adherence.15 Results from our prior research indicate that families of children with ADHD from poor, urban, and minority communities are often incompletely informed of treatment options, do not routinely have a chance to voice preferences, and are not consistently involved in decision-making, which may be associated with increased health care costs and greater impairment.37-40
National guidelines prioritize shared decision-making (SDM) as a strategy to engage families in treatment and improve adherence and outcomes. SDM is a process in which families and clinicians jointly participate in decision-making, exchange information and treatment preferences, and work together to decide on a treatment plan.41 SDM is particularly helpful for conditions like ADHD that have multiple evidence-based options and in which variation exists in how families weigh their risks and benefits.42 In fact, the American Academy of Pediatrics recommends that clinicians engage families, be attentive to family preferences and goals, and weigh benefits and harms before deciding on a treatment course.27 In addition, the Institute of Medicine (now the National Academy of Medicine) has prioritized research on the comparative effectiveness of SDM,43 and the 2010 Patient Protection and Affordable Care Act supports its implementation in clinical settings.44 Our prior research found that families who were more easily able to contact their child's doctor outside of office visits were more likely to report high levels of SDM,38 suggesting that interventions that facilitate communication and information-sharing outside the context of office visits may improve SDM and engagement in care.
Fragmentation in health care and poor communication across systems adversely impact SDM and engagement and adherence to treatment by children with ADHD and their families. Children with ADHD often use services across the physical and mental health and education systems. Fragmentation of services between these systems impairs communication and collaboration between families and primary care providers (PCPs), mental health providers, and educators, and leads to suboptimal outcomes for children.45-49 This fragmentation impairs communication and coordination between providers and educators in delivering treatment responsive to a family's preferences and goals.50 Poor communication and coordination may result in duplication of services, inadequate monitoring of treatment, and, more importantly, treatment that is inconsistent with a family's preferences and goals. Reasons for fragmentation in the systems of care for children may be related to a lack of accountability for coordinating care, frequent changes in providers that lead to disruptions in care, insufficient knowledge of treatments, few community resources to assist families, distrust among providers, lack of administrative support to undertake coordination activities, and managed care policies that carve out mental health care.51-54 The quality of care and the uptake of evidence-based therapies have been reported to be low in poorly integrated systems.55-57 However, little attention has been devoted to developing and testing communication strategies as a means to improve ADHD care. Prior findings suggest that more than knowledge is needed.48
One such strategy is the use of electronic patient portals, increasingly common online health care applications that allow patients to interact and communicate with their health care providers and manage their health.58,59 Portals have the potential to overcome fragmentation, promote SDM, and improve patient outcomes via electronic communication tools. Portals have grown in popularity as clinicians and families have become interested in having families manage their own health information within electronic systems.60-62 One pediatric study on ADHD found that community pediatricians using a portal were significantly more likely to collect information from parents and teachers than pediatricians who were not using a portal.63 In family medicine, use of a portal focused on preventive care resulted in increased patient activation and greater patient-centered care, and users were more likely to receive needed preventive care.64 At our institution, we found that implementation of an ADHD patient portal that used email surveys to gather parent and teacher rating scales resulted in a significant increase in rating scale use.65,66 This resulted in the use of an ADHD portal as standard of care. Unfortunately, disparities have been found in studies of portal use, with White and privately insured families more likely to use these tools.67-71 This highlights the importance of evaluating the benefit of portal use among population subgroups.
An alternative strategy investigated in this study to promote communication and SDM is the use of care managers, who function similarly to patient navigators to promote patient engagement and coordinate care. In studies of adolescents and adults with depression, care managers have been used as part of multifaceted collaborative care interventions in primary care settings and have demonstrated favorable findings with regards to depressive symptoms and functioning.72-77 Systematic reviews of these trials have consistently demonstrated that care provided in this manner is associated with modest but sustained improvement in outcomes (standardized mean difference = 0.25; 95% CI, 0.18-0.32) relative to usual care.78,79 Interventions that included regular planned supervision and use of care managers with mental health training had better outcomes relative to less rigorous interventions.80,81 Care managers have become broadly implemented, particularly for cancer and mental health care.82,83 However, study of care managers in ADHD has been limited. We previously piloted an ADHD care manager intervention and found it to be feasible and acceptable.84
Information is lacking on how patient portals compare with care managers in promoting communication and care coordination for ADHD. However, this comparison may not be relevant or feasible given the ubiquitous nature of patient portals. Therefore, the objectives of this study (Figure 1) were as follows:
ADHD Health Care System.
Aim 1 sought to determine the incremental benefit of supplementing a patient portal with a care manager. Aim 2 sought to test whether the effects of improved communication were mediated through treatment adherence, family engagement in treatment, or both. Aim 3 sought to explore racial and socioeconomic factors that might moderate treatment effects. This study broadens the definition of SDM by focusing on a chronic childhood condition (ADHD) with longitudinal decision-making between patients/parents and multiple providers.
By recognizing patients and parents as experts, prioritizing their treatment preferences and goals, and necessitating information-sharing over time across the health and school systems that impact children, SDM in ADHD is an ideal prototype to define strategies to foster patient-centered care, engagement, adherence, and improved outcomes in varied pediatric chronic conditions (eg, asthma, depression). As such, the lessons learned regarding interventions to foster SDM in ADHD in this protocol are likely to yield generalizable knowledge for fostering patient-centered pediatric care. This project addressed the PCORI methodology standard for identifying gaps in evidence concerning effective communication strategies to promote SDM for children with ADHD.
To ensure the patient-centeredness of this application, we sought to include patients, families, and stakeholders in each of 4 key domains of the ADHD care system (Figure 1). We recruited key patient and stakeholder representatives using known contacts from previous research, the Children's Hospital of Philadelphia (CHOP) Parent Advisory Council, and recommendations from colleagues (Table 1).
Characteristics of Key Patient and Stakeholder Representatives.
We successfully recruited 2 patient and family representatives (Lisa Snitzer from the Mental Health Association of Southeastern Pennsylvania and Denise Stewart from CHOP's Parent Advisory Group), a school representative (Siobhan Leavy from Chichester School District), a mental health representative (Dr Steven Berkowitz from Hall-Mercer Community Behavioral Health Center), and a physical health representative (Dr Nathan Blum from CHOP). Each of these patient and stakeholder representatives participated in the preapplication planning and continued their participation during the 3-year study period as coinvestigators. Their time and effort during the study period was compensated through subcontracts or paid consultancy agreements. These key patient and stakeholder representatives contributed 5% to 10% effort through attending weekly study meetings; serving as leads for stakeholder advisory groups; reviewing drafts of study procedures, protocols, and presentations; providing guidance on ongoing study recruitment, data collection, and intervention implementation; interpreting study results; promoting the study among their professional networks; participating in presentations; and assisting with dissemination of research findings. For example, stakeholders helped to select secondary outcome measures for the study. Stakeholders also helped review the language to use when recruiting families to the study, based on what kind of language would be relatable to parents of children with ADHD. When discussing the study with clinicians at participating practices, stakeholders also advised the study team on language based on what clinicians would find most helpful for patients and their families. The stakeholders were instrumental in the decision to move from the original strategy of 2 waves of recruitment to 1 continuous wave of recruitment.
We formed separate caregiver, teacher, and clinician stakeholder partner groups involving parents, teachers, and primary care clinicians of children with ADHD (Table 2). We recruited primary clinicians from participating pediatric primary care practices, caregivers through CHOP's Parent Advisory Council and from nominations from participating PCPs, and teachers from Dr Leavy's known education contacts. These groups met biannually and were led by key patient and stakeholder representatives (Table 1). Partner group meetings were sometimes combined so that groups could hear each other's perspectives, although group meetings were typically held separately. The groups generally met in the evenings, shared a meal, and had childcare to promote attendance. Some of the group meetings were held by conference call, depending on the availability of group members to attend in person. All in-person meetings allowed members to call in to the meeting as well. These stakeholders provided advice and consultation regarding development of the engagement measure (ADHD Engagement Scale [AES]), participant recruitment, study procedures, outcome measures, care manager intervention implementation, and interpretation of study results. As stakeholder partners provided input on the study only twice a year, their role was distinct from that of the CHOP key patient and stakeholder representatives, who were part of the weekly research team meetings and provided more frequent feedback to the study. Stakeholder partners were compensated at $100/hour of meeting time.
Caregiver, Teacher, and Clinician Stakeholder Groups.
We conducted a prospective, randomized comparative effectiveness trial to (1) compare the effectiveness of an electronic patient portal designed to communicate family-centered treatment preferences and goals to providers and educators (tier 1) with the electronic patient portal combined with a care manager to facilitate communication and coordination of care (tier 2) for children with ADHD; (2) assess treatment initiation and adherence and family engagement as mediators of intervention treatment effects; and (3) explore individual, family, and community factors that moderate intervention treatment effects. Eligible children and their parents were randomly assigned 1:1 to the ADHD portal alone (tier 1) or the ADHD portal plus care manager (tier 2). Randomization was stratified by practice, sex, and age groups (5-7 years old and 8-12 years old) to ensure balance between groups. Allocation concealment (blinding of the treatment assignment) was implemented using sealed, opaque envelopes, along with stratification and randomly permuted blocks of unequal sizes to prevent providers and patients from manipulating the randomization to favor any treatment. Treatment assignment was made at the time of enrollment following informed consent, and patients were followed with measures related to ADHD symptoms, goal attainment, and patient-reported outcomes collected at 4 study visit time points. Study visit 1 was completed at baseline and was typically completed in-person with study staff at enrollment. Study visit 2 was completed between 2 and 5 months postenrollment electronically or by phone with study staff. Study visit 3 was completed between 5 and 8 months postenrollment electronically or by phone. Study visit 4 was completed between 8 and 12 months postenrollment electronically or by phone. The wide time window allowed for any lag in reaching families so that they could complete study measures and interventions. The study was approved by the IRB at CHOP and was registered with ClinicalTrials.gov (identifier NCT02716324) before patient enrollment.
The study was conducted at 11 of the 31 primary care pediatric practices affiliated with CHOP, an integrated pediatric health care system located in the Philadelphia metropolitan area. Practices included urban (Karabots, Cobbs Creek, South Philadelphia, CHOP campus, Chestnut Hill) and suburban (Haverford, Drexel Hill, Media, Broomall, HighPoint, Indian Valley) locations. A common electronic health record (EHR) was available at all of the practices that included the ADHD-specific patient portal known as the ADHD Assistant. The 11 practices were recruited to participate using letters of invitation and in-person presentations. Incentives for practices to participate were provider education on ADHD and opportunities to improve communication among teachers, parents, and clinicians. We initially sought out 15 pediatric practices to participate in the study by using the Pediatric Research Consortium (PeRC) at CHOP to help identify practices to recruit. PeRC is designed to serve as a liaison between clinical investigators and the practices of the CHOP primary care network. This role was achieved by taking advantage of CHOP's expanded organizational and technological infrastructure in the ambulatory component of the CHOP Care Network, including 31 pediatric and adolescent care practices and the CHOP campus. PeRC has assembled an experienced staff of epidemiologists, clinical informaticians, and administrators that use the ambulatory EHR system to gather data across the pediatric network, conduct research, and develop an array of integrated, evidence-based decision support tools. The EHR is currently in use in all 31 primary care practices and the CHOP campus, and it has captured data for >400 000 patients since its inception in 2002. PeRC contacted 15 potentially eligible practices to ask for their consent to be contacted by our team about the study. Study team members then made in-person recruitment visits to each practice to meet with medical directors, clinicians, and staff regarding study participation incentives (such as the availability of care management for all tier 2 study participants) and requirements for participation. Of the original 15 practices contacted, 4 declined to participate.
Children were eligible for the study if they received ADHD care at a participating practice, had an ADHD diagnosis code (ICD-9 codes 314.XX) recorded at an ambulatory visit in the past year, and were age 5 to 12 years. Lists of potentially eligible children at each practice were identified from EHRs. The lists were circulated to primary care clinicians at the practices to confirm they received ADHD care at the practice. Children were excluded if they had a diagnosis of autism spectrum disorder (ICD-9 codes 299.XX), conduct disorder (ICD-9 codes 312.XX), psychosis (ICD-9 codes 298.X), bipolar disorder (ICD-9 codes 296.XX), or suicide behavior (ICD-9 codes E950.0-958.9) in the past 12 months, as treatment for these diagnoses can overwhelm ADHD care managers. To obtain an unbiased sample, we drew a stratified random sample of approximately 300 eligible children from participating practices, taking into consideration the race/ethnicity/sex of the patients served in order to achieve a representative sample of children from the greater Philadelphia area. Parents of eligible children were mailed recruitment letters signed by their PCP. Included in the mailing was a stamped self-addressed postcard to permit families to opt out. Families who did not opt out within 2 weeks were called to screen for eligibility, provide study information, and schedule a consent visit. We selected additional children from the same practice, sex, and age groups (5-7 years old and 8-12 years old) strata for any families that opted out and if recruitment targets were not met.
Families who consented to participate were contacted to complete study measures at study visits 1, 2, 3, and 4 using all available means, including letters, telephone calls, text messages, and emails to ensure data were collected thoroughly and systematically from all participants. Contact information for each participant was updated every 3 months using interpersonal exchanges and clinic EHRs. Participation incentives included compensation to complete study measures, quarterly newsletters, and child birthday card mailings, all of which have been used in prior work with urban populations to maintain >80% participant follow-up.85
We tested the comparative effectiveness of a tiered communication intervention to facilitate SDM over the course of 9 months. The tiered communication intervention was designed to facilitate SDM between parents and clinicians through the consistent use of the ADHD Preferences and Goals Instrument (PGI).88 The ADHD PGI consists of a parent-reported 46-item tool that queries parents on their treatment preferences (medications and/or behavioral therapy) and goals for treatment. The ADHD PGI was completed by parents through the portal and electronically scored to facilitate SDM at the time of a clinic visit.
Tier 1 consisted of an electronic patient portal designed specifically for ADHD, which has become standard practice for the care of children with ADHD at CHOP. The portal was designed to (1) capture and share patient and family treatment preferences and goals using the ADHD PGI; (2) monitor ADHD symptoms from parent and teacher reports, treatment receipt, and medication adverse effects; (3) provide a repository of ADHD educational materials; and (4) permit information-sharing between parents and teachers with parental consent. The ADHD portal sent electronic ADHD symptom rating scales to a child's parent (Vanderbilt Parent Rating Scales [VPRS]) and to the child's teacher (Vanderbilt Teacher Rating Scales [VTRS]) via email. Teachers could print out their individual responses, so as to include the VTRS in a child's school report. Parents were permitted to view VTRS data, but teachers could only view VPRS information if parents gave consent. The frequency of emails from the portal varied from weekly to every 3 months at the discretion of the clinician. The portal did not permit parents or teachers to free text comments to clinicians.
Tier 2 consisted of the electronic patient portal combined with an ADHD care manager. The care manager was responsible for communicating information and facilitating coordination of care. Using data captured in the portal, the care manager communicated with families at the beginning of the study to confirm their treatment preferences and goals, provided additional education on ADHD treatment, monitored attainment of goals, and provided education on common concerns among patients with ADHD and their families. The care manager contacted families at least every 3 months by phone, text, or email, as needed, to assess treatment use, identify new concerns, and assist families with problem-solving. Frequency was determined to coincide with frequency of office visits for ADHD. Using virtual communication tools, including email, text messaging, or phone, the care manager communicated with pediatric clinicians, mental health providers, and teachers to clarify family treatment preferences and goals and to address emerging treatment issues. Care managers reached out to pediatric clinicians, mental health providers, and teachers at least every 3 months, depending on the needs of the family and the contact information provided by the family (ie, if the child was seeing a mental health professional) and expected to have a dialogue with the professional that could be relayed to the caregiver. Care managers were permitted to continue the intervention for an additional 1 to 3 months to ensure resolution of outstanding issues with families. The care managers completed a fidelity checklist after each encounter to assess self-reported task completion (0 = not completed, 1 = partially completed, 2 = fully completed). In addition, the care manager summarized clinically relevant encounters in the EHR, facilitating communication with clinicians. We assessed the number of parent-completed portal uses and care management sessions completed by participants.
Measures proposed for this study (Table 3) corresponded to our conceptual framework (Figure 1) and assessed ADHD symptoms as the primary outcome, goal attainment and patient-reported outcomes as secondary outcome measures, and ADHD engagement and treatment as mediator measures. These measures were endorsed by our caregiver and stakeholder partners as important to them. Measures were collected at 4 study visits, 3 months apart, using Research Electronic Data Capture (REDCap). The REDCap surveys were sent to participants via email with a link to the survey items. We followed up with nonresponders by phone. We permitted participants up to 12 months from enrollment to complete the final study visit.
Study Measures and Timing.
To measure ADHD symptoms, we used the ADHD symptoms scales of the VPRS, a standardized validated measure of ADHD symptom severity that is commonly employed in clinical care and ADHD research studies. We collected VPRS scores at study visits 1, 2, 3, and 4.86,87 The VPRS is a public domain tool (https://www.nichq.org/sites/default/files/resource-file/NICHQ_Vanderbilt_Assessment_Scales.pdf) that includes 18 items corresponding to the DSM-5 ADHD symptom criteria.111 The VPRS is rated on a 4-point Likert scale (0 = “never” to 3 = “very often”), but the scales were restricted to the 18 ADHD inattention and hyperactive/impulsive symptom items from the follow-up version of the scale with total scores ranging from 0 to 54. In validation studies, the internal consistency of the overall VPRS was excellent (α = .90-.94), and concurrent validity was high (r = 0.79) in relation to diagnostic interviews. The minimal clinically important difference (MCID) in VPRS scores was prespecified by clinician stakeholders to be 4 points, although the study was powered to find a 2.5-point difference between groups. This determination was primarily suggested by coinvestigators Drs Power and Blum, who use ADHD symptom scores in their clinical research. The study team then discussed the use of 4 points as the MCID and came to consensus.
To determine goal attainment, we identified family treatment goals through use of the ADHD PGI.88 Families selected a goal they believed was attainable during the study period. Goals were categorized as academic, behavioral, or relational by consensus of the study team. The Goal Attainment Scale (GAS) was used to allow parents to rate the degree to which their goal was met.89-91 The GAS response categories are ordered from 0 (“no change”) to 6 (“goal completely met”). Parents completed the GAS at study visits 1, 2, 3, and 4.
We incorporated the following patient-reported outcomes (PROs), which patient and stakeholder representatives identified as most important: school performance, student engagement, peer relationships, family belonging, and teacher connectedness. These PROs consisted of Patient Reported Outcomes Measurement Information System (PROMIS short forms and Healthy Pathway scales, each of which are brief, reliable, and precise measures of patient-reported health status.92-96 The PROs were completed by children aged 8 to 12 years (child reported) and by parents of all children (parent reported) at study visits 1, 2, 3, and 4.
Parents completed the AES at study visit 4. It was developed and validated as part of this study and consisted of 4 domains: patient- and family-centered care, access, communication, and understanding. The AES consists of 28 items rated on a 4-point Likert scale (1 = “not at all/a little bit” to 4 = “very much”). See “Aim 2” under the “Analytic and Statistical Approaches” section for information on AES development.
ADHD medication use for each participant during the study period was determined by abstraction of the EHR. ADHD medication (stimulants, α-agonists, and atomoxetine) prescription fills and dates were identified in the medication tab, which corresponded to those prescribed by primary care clinicians and mental health clinicians. Mental health service use for each participant during the study period was determined by parent completion of the Services Assessment for Children and Adolescents (SACA) at study visit 4. The SACA is a well-validated, client-reported tool and provides information on any mental health services use, ambulatory services use, and inpatient service use.
We used the Consolidated Criteria for Reporting Qualitative Research (COREQ) to report the conduct of the interviews.99 We purposively selected tier 2 participants who had ≥2 care management sessions for semistructured interviews. We sought to conduct up to 20 interviews, or fewer if thematic saturation was reached before 20 interviews. The interviews were conducted by phone or in-person, depending on participant preference. An interview guide was developed at the conclusion of the study based on feedback from the care managers and sought to query participants on barriers and facilitators to the tier 2 intervention (see Appendix A). Interviews were conducted by a trained research assistant with a bachelor's degree, who was provided with training on how to conduct qualitative interviews. All interviews lasted approximately 30 minutes and were audio-recorded and transcribed. Field notes made by the interviewer supplemented the audio recordings. Transcripts were analyzed using Grounded Theory, in which we sought to uncover emerging themes. All transcripts were reviewed and coded separately by 2 investigators, with disagreements settled by consensus. We used NVivo qualitative software (QSR International) to conduct the analysis of the interviews.
Our target sample size was 300. For our primary outcome, we assumed moderate levels of correlation of the 2 VPRS subscale scores on hyperactivity and inattention (correlation = 0.4-0.8) from one study visit to the next, which would provide a standard deviation of the sum of the 2 scores between 4.9 and 5.5. With an initial sample of 303, we had a power of 0.87 to detect a difference as small as 2.5 points on the VPRS between groups with α = .05, 240 participants (80%) with follow-up data, and a marginal model for longitudinal data (generalized estimating equation [GEE] with a first-order autoregressive working correlation structure and robust variance estimates). Actual follow-up proportions at each study visit ranged from 240 to 300. The 2.5-point difference is below our MCID of 4 points.
The time frame for the study intervention and participation was 9 to 12 months. This period was chosen in order to correspond to a school year. We had initially proposed to recruit participants in 2 waves at the beginning of each academic year, but we modified our recruitment plan to continuously recruit and enroll over the course of the year in order to meet our recruitment goal of 300 participants within the 3-year study period. The 9- to 12-month period was felt to be sufficient to observe clinically significant changes in VPRS scores.
Study data were collected and managed using the secure web-based REDCap database at CHOP, an active participant of the REDCap Consortium.97,98 Participants entered outcome data using a web-based interface that supports encrypted data transfer from its origin to the database housed in a secure data center in Norristown, Pennsylvania. REDCap supports a simple interface for validated data entry, audit trails for tracking data manipulation, export procedures for seamless data downloads to statistical packages, and procedures for importing data from external sources. All participants were assigned a unique study identifier (ID), which permitted linking of study measures in the REDCap database.
Baseline data was completed at the time of an in-person consent study visit (study visit 1). Subsequent study visits were completed by participants using automated email reminders sent by REDCap, with follow-up telephone calls conducted by research staff blinded to study arm assignment for those participants who did not respond. Supplemental data for VPRS scores and medication status with corresponding time stamps were pulled from the CHOP EHR for a subset of patients when their data was missing within a REDCap survey, as these data are collected as part of clinical care. If a particular survey data point was missing, a research assistant queried the EHR for relevant data with 3 months of the assigned survey time point. Data were merged to REDCap data via study ID and time point in days from enrollment.
The numbers and dates of care manager encounters were determined by a tracking spreadsheet maintained by the care managers. Parent and teacher usage of the patient portal was extracted from the EHR for each participant's time of study participation. Care manager and patient portal usage was abstracted into the REDCap database. All data management and analyses were conducted on secure data and software servers that have daily back-up. This approach facilitated data acquisition, prevented data theft and loss, and achieved maximum protection from unauthorized access.
To account for missing data, we employed imputation methods for 2 of the main outcomes (ADHD symptoms and PRO measures) as follows.
ADHD symptoms were measured using the VPRS, which is composed of 18 symptom items rated on a 4-point Likert scale that were averaged for each participant for each time point (range, 0 to 3 for each item). To account for missingness on the VPRS, we imputed values within the 2 domains for the ADHD symptoms scale (2 subdomains, 9 questions each). If a participant for a particular follow-up period had 1 or 2 items missing within a domain, the average of the nonmissing items for the respective domain was assigned. No between-study arm differences between pre- and postimputation ADHD mean scores were observed by case status: tier 1 or tier 2 (Appendix B, Table 1). VPRS subdomains with >2 missing items were not imputed.
Parent-reported PRO and child-reported PRO measures were averaged for each domain for each time point. Parent domains included school performance (5 items), student engagement (4 items) and peer relationships (7 items); child domains included school performance (5 items), student engagement (4 items), peer relationships (7 items), family belonging (6 items), and teacher connectedness (9 items). To account for missingness on the PRO measures, we imputed values within each parent or child domain. If a participant for a particular follow-up period had 1 or 2 items missing within a domain, the average of the nonmissing items for the respective domain was assigned. A separate GEE model was used for each domain. No differences between pre- and postimputation PRO measures were observed by case status (Appendix B, Tables 2 and 3). PRO measures with >2 missing items were not imputed.
Goal attainment was measured using the GAS, which was coded via a 7-point Likert scale at the 9- to 12-month follow-up: (1) never, (2) occasionally, (3) a little less than half, (4) sometimes, (5) a little more than half, (6) very frequently, and (7) always. Goal type was coded by 2 independent reviewers and the study team came to consensus on goals that were not clearly categorized. Goals were collapsed into 3 main types: school performance, behavior, and relationships. We did not impute missing values for the GAS. An overall GEE model for goal attainment is reported, as well as stratified models by each of the 3 goal types.
To determine differences in the comparative effectiveness of the tier 1 vs the tier 2 intervention for children with ADHD, we followed the standard of an intention-to-treat repeated-measures longitudinal analysis that modeled study visit 1, 2, 3, and 4 measurements as outcomes; these were clustered by practice site, and permitted continuous, count, and binary outcomes. To check for randomization of patients, patient characteristics were compared between the groups in the tier 1 and tier 2 interventions using χ2 tests for categorical variables, and t tests/Wilcoxon rank sum tests for continuous characteristics. To assess bivariate associations between patients in the care manager and portal group vs portal alone and outcomes, we used t tests for ADHD symptoms, parent and child PRO measures, and goal attainment. These analyses were stratified by time point. For the primary outcome of ADHD symptoms, and secondary goal attainment and PRO measures, linear mixed-effects and marginal (GEE) models were implemented. For binary outcomes of the proportion of children with normalized symptoms and goals attained, marginal models (GEE) were implemented and weighted as needed for any dropout.100,101 As these models have different assumptions, using both allowed us to confirm that assumptions were met. We followed usual guidelines for reporting results (CONSORT),102 and reported all outcomes in terms of interactions of intervention status and time in days. Thus, the β coefficients for the interaction terms represent the adjusted difference in outcome measures between the 2 groups over time. The season in which the survey was completed (summer: June 16 to September 15, fall: September 16 to December 15, winter: December 16 to March 15, spring: March 16 to June 15) was also included to account for the potential influence of seasonality on outcomes,103,104 as participants were enrolled at different times of year. Models included adjustments for child age (5-7 years and 8-12 years), sex, race/ethnicity (White, Black, Hispanic, other), free/reduced school lunch, metropolitan status (urban, suburban), Supplemental Security Income (SSI) status, ADHD medication status, and school type (public, private, charter); and for parent education level (less than high school, some college, college degree). In addition, to account for clustering, clinic site was included in each model as a random effect.
We conducted sensitivity analyses to examine the impact of intervention dosage intensity on the main outcome of ADHD symptoms. We stratified tier 2 participants by intervention dosage (frequency of interaction of parents or teachers with the portal/care manager): 0 to 1 sessions, 2 sessions, or ≥3 sessions. We compared differences in ADHD symptom score changes over time by intervention dosage using a linear mixed-effects model that adjusted for seasonality.
We originally proposed to assess whether family engagement, treatment initiation, and adherence were mediators of intervention treatment effects; this analysis proposed to focus on the role of engagement and treatment initiation at study visit 2 or adherence at study visit 3 and 4 as mediators of the effect of the intervention on the study outcomes. Because none of the primary or secondary outcome measures differed between groups, we were unable to proceed with formal tests of mediation. Instead, we sought to focus on differences in family engagement and treatment initiation and adherence between treatment groups. For each of the previously described sets of potential mediators, we report distributions by case/control as bivariate differences via t tests or contingency tables/χ2 analyses where appropriate. As a subgroup analysis, we assessed the association between family engagement and our main outcome, VPRS scores. The latter analysis would enable us to assess whether there were any effects of family engagement on ADHD symptoms regardless of treatment group.
To assess family engagement, we created a series of items (ie, the AES) from review of the literature and prior instruments in other fields. We used confirmatory factor analyses to find distinct domains for which each item loading was >0.7. Our first round of analyses resulted in 3 factors: patient- and family-centered care, communication, and understanding. Upon presentation to the study team, several items were dropped from the initial domains, but additional items resonated with the team as important concepts on which the intervention was focused. Thus, the study team came to consensus on items that should be included. These items represented 4 conceptual domains. The 4-factor structure was supported in confirmatory factor analyses. Although the patient- and family-centered care and understanding domains remained the same, the original communication domain was split into 2 factors, resulting in the following domains: care team accessibility; patient- and family-centered care; communication among members of the care team; and understanding of ADHD. To compare patients in the tier 1 and tier 2 groups by engagement status, we used t tests for each of the 4 domains separately. Further, linear mixed-effects models were implemented for each engagement domain as the outcome, with intervention status and season as covariates clustering for clinic site. We further assessed for item responsiveness using item response methods.
We explored treatment initiation and adherence via the SACA and medication status.98 On the SACA, patients reported service use ever, as well as service use within the past 9 months. Categorizations include any service use, ambulatory service use (any community mental health or outpatient clinic, private professional, or in-home provider), and overnight stay (psychiatric or medical unit, residential treatment center, group home, or foster home). Differences between intervention groups and treatment initiation were assessed via χ2 tests. In addition, treatment initiation and adherence were assessed via medication status throughout the study period. On follow-up surveys at study visits 1, 2, 3, and 4, participants were asked whether their child was taking or not taking medication. If data was missing for a particular time point, we used medication status within the EHR if available.
When we explored individual, family, and community factors that moderate treatment effects, the same longitudinal data models supported estimates of effect modification by means of 3-way interaction terms of time × treatment × effect modifier as well as subgroup analyses. Thus, the β coefficients for the 3-way interaction terms represent the adjusted difference in outcome measures between the 2 groups over time by the level of a third factor. Factors of importance explored in this aim included parent education level (high school or more vs less than high school); and child race/ethnicity, medication status, and neighborhood variables (eg, median income, metropolitan status). To obtain the community- level variables, we geocoded each participant was via street address, city, and zip code. American Community Survey (ACS) Census Bureau tract–level information, including median income, percentage of high school graduation, population density, and percentage of poverty were linked to participant tracts.
Urbanicity was defined as home residence in Philadelphia County (urban) vs a suburban county. Bivariate differences between intervention status and the above factors were assessed using χ2 tests for categorical characteristics, and t tests/Wilcoxon rank sum tests for continuous characteristics. Moderation models were conducted separately for each outcome, following the same modeling strategy in aim 1 with marginal (GEE) models accounting for clinic site and seasonality. Stratified analyses were proposed for any significant 3-way interactions.
As described previously, we had originally proposed to recruit 300 eligible participants in 2 waves corresponding to school years in years 1 and 2. However, we fell behind in our recruitment. Thus, in order to meet timely recruitment goals, we modified our recruitment plan to continuously recruit eligible participants throughout the year until we achieved our goal of 300 participants. To adjust for different times of enrollment, we adjusted for seasonality in our analyses above.
To better understand why the tier 2 intervention (online patient portal plus care manager) was not more effective than the tier 1 intervention (online patient portal alone), we sought to conduct semistructured interviews with tier 2 participants' parents at the conclusion of their participation in the study (see the “Interviews” section above for the methods we employed). These modifications were approved by the IRB at CHOP and by PCORI.
We identified 3118 potentially eligible participants with ADHD from the 11 participating practices (Figure 2). Of these, we excluded 2815 for the following reasons: ADHD not managed by PCP (n = 1914), had an exclusionary condition (n = 516), did not have active ADHD (n = 89), had a parent who did not speak English (n = 27), was not in age range (n = 100), other (n = 60), or declined to participate (n = 105). A total of 303 eligible children were enrolled and randomly assigned: 149 to the tier 1 intervention (online patient portal alone) and 154 to the tier 2 intervention (online patient portal plus care manager). Of these, 273 (90.1%) completed the study, as defined by completing the final VPRS.
Flow Diagram of Study Participants.
Following randomization, children in both study arms had similar demographic characteristics (Table 4). The average age was 8.5 years old, with most being >8 years of age. Two-thirds of children were male. Children were racially and socioeconomically diverse, with 50.5% residing in urban Philadelphia and the remainder in a suburban location. Over half qualified for free or reduced school lunch (54.1%) or attended public schools (59.7%). As expected, a majority (53.4%) of children were receiving an ADHD medication at study visit 1 (ie, baseline).
Demographic Characteristics of Study Participants.
We successfully geocoded all participant residences to unique census tracts and overlaid those census tracts with census tract data from the 2016 ACS (Table 5). We used data on population density as a surrogate marker of neighborhood urbanicity; median household income, percentage unemployment, percentage poverty, and percentage high school graduation as surrogate markers for neighborhood socioeconomic status; and percentage African American as a surrogate marker for racial segregation. As shown in Table 5, there were no differences in census tract variables between intervention groups, although twice as many participants in the tier 1 intervention neighborhoods were living below the poverty threshold as those in the tier 2 intervention neighborhoods. There was a slightly greater proportion of tier 1 participants from neighborhoods with a higher percentage of African Americans.
Neighborhood Characteristics of Study Participants.
Over the course of the study period, 206 (68.0%) parents of study participants accessed the ADHD patient portal to complete a parent-reported VPRS (Table 6). The average number of VPRS surveys completed as study visits was 2.3 out of a possible 4, but there was no difference between groups in the mean number of VPRS surveys completed. Few participants (30.4%) had a teacher access the portal to complete a VTRS. The average number of VTRS surveys completed was low, and there was no difference between groups.
Portal-Completed VPRS and VTRS Surveys.
Among those randomly assigned to the tier 2 intervention, 96% of parent participants completed at least 1 parent care management session during the study period (Table 7). The average number of completed parent sessions was 2.2, with a range of 0 to 5 sessions. Only a third of participants had a teacher complete a care management session. The average number of completed teacher sessions was 0.5 with a range of 0 to 3 sessions. When care management fidelity checklists were examined, 66% of parent and 63% of teacher sessions were rated as fully completed on all relevant items (Appendix B, Table 4). Initial parent sessions had higher fidelity (70.1% vs 63.1%) than subsequent parent sessions. Initial teacher sessions had lower fidelity (53.1% vs 85.7%) than subsequent teacher sessions, but there were few (n = 21) subsequent teacher sessions.
Care Management Sessions by Caregivers and Teachers.
To compare the effectiveness of the tier 1 vs tier 2 interventions, we examined their effects on primary and secondary outcomes. Table 8 shows the primary outcome, mean VPRS scores, by study visit. Completed VPRS scores include supplemented VPRS scores pulled from the EHR. Study visit 1 refers to study visit 1 (ie, baseline) measures, study visit 2 refers to measures completed between month 2 and month 5, study visit 3 refers to measures completed between month 5 and month 8, and study visit 4 refers to measures completed between month 8 and month 12. All 303 participants completed study visit 1 measures, 258 (85%) completed the study visit 2 measures, 236 (78%) completed the study visit 3 measures, and 273 (90%) completed the study visit 4 measures. To supplement VPRS scores that were not completed for the study, we abstracted 54 VPRS surveys from the EHRs of 45 participants. These included 5 study visit 1 VPRS surveys, 18 visit 2 VPRS surveys, 17 study visit 3 VPRS surveys, and 14 visit 4 VPRS surveys. Mean VPRS scores decreased over time, indicating clinical improvement in ADHD symptoms, but there were no statistically significant differences in mean scores between groups at any study visit (Table 8).
Imputed VPRS Scores by Study Visit.
Tables 9 and 10 show imputed parent-proxy and child-reported PRO measures by study visit.
Imputed Parent PRO Scores by Study Visit and Intervention Arm.
Imputed Child PRO Scores by Study Visit and Intervention Arm.
Modest increases over time in unadjusted parent-proxy PRO measures were observed across all parent domains, but differences between groups were not statistically significant. The results over time in unadjusted child-reported PRO measures were mixed with some measures showing modest decreases over time, whereas others showed no changes. Again, there were no statistically significant differences between groups at any of the study visits.
Table 11 shows GAS scores for participants in both groups. Of the original sample of 303 participants who completed a goal at the first study visit, 262 participants (86.5%) completed at least 1 subsequent GAS score at a follow-up study visit. The majority of goals were academic, followed by behavioral, and then relational. There was modest change in GAS scores over time, with average scores of 3 indicating “change sometimes.” There were no differences in GAS scores between groups at the various time points.
Parent Goal Attainment by Study Visit and Intervention Arm.
To assess changes between groups in our primary outcome, VPRS scores, we developed mixed-effects regression models that examined “intervention × time” (I × T) interactions on VPRS scores (Table 12). All models included adjustment for seasonality given the different seasons of study entry by participants. The I × T interaction in the full model was not significant (Table 12), indicating no difference between groups in changes in VPRS scores over time. Time (in days) was significant, suggesting that VPRS scores decreased an average of 0.015 points/day or roughly 5 points over the course of a year for both groups, a clinically meaningful improvement. Urban location and medication status were also both significant; children living in urban residences had VPRS scores that were 3.4 points greater than children living in suburban residences, and children taking ADHD medications at the time of a VPRS survey had VPRS scores that were 4.6 points less than children who were not taking ADHD medications at that time.
Adjusted VPRS Scores.
To assess changes between groups in our secondary outcome, GAS scores, we similarly developed mixed-effects regression models that examined I × T interactions on GAS scores (Table 13). The I × T interaction for the full model was not significant (Table 13), indicating no difference between groups in changes in GAS scores over time. The time variable was also not significant, suggesting that there was no significant change in GAS scores over time. Only the median household income variable was significant, indicating that higher median incomes were associated with lower GAS scores. The GAS was scored such that lower scores indicated that one's goal was less attained or not attained. In addition, we fit separate models on individual goal attainment categories (academic, behavioral, and relational; (Appendix B, Tables 5-7) and again found no significant I × T interactions.
Adjusted GAS Scores.
To assess changes between groups in our other secondary outcome measure, PRO scores, we similarly developed mixed-effects regression models that examined I × T interactions on parent-reported PRO and child-reported PRO scores. Separate and full models that examined the potential confounding effects of individual covariates on I × T interactions are shown in Tables 14 to 16 for parent-reported PRO measures and Tables 17 to 21 for child-reported PRO measures. The I × T interactions for the full 3 parent PRO models were not significant, indicating no difference between groups in changes in parent PRO scores over time. Similarly, the I × T interactions for the full 5 child PRO models were not significant, showing no between-group differences in changes in child PRO scores over time. All time variables with 1 exception were nonsignificant, suggesting no change in parent or child PRO scores over time. Only the Child School Performance model showed a significant time variable (−0.001; 95% CI, −0.002 to 0.000), suggesting that children reported lower school performance over time.
Adjusted Parent PRO School Performance Scores.
Adjusted Parent PRO Student Engagement Scores.
Adjusted Parent PRO Peer Relationships Scores.
Adjusted Child PRO School Performance Scores.
Adjusted Child PRO Student Engagement Scores.
Adjusted Child PRO Peer Relationships Scores.
Adjusted Child PRO Family Relationships Scores.
Adjusted Child PRO Teacher Connectedness Scores.
We conducted a preplanned sensitivity analysis to determine if there was a dose-response in which greater engagement among participants in the tier 2 intervention (online patient portal plus care manger) resulted in greater declines in VPRS scores (Table 22). In this case, we assessed engagement by the number of care management sessions attended (0 to 1, 2, or ≥3). We found that those tier 2 intervention participants who received 2 or ≥3 care management sessions experienced statistically significantly greater decreases in VPRS scores than those who only participated in 0 or 1 sessions. This suggests that greater engagement of participants with the care management intervention may have resulted in greater symptom improvement. However, this result was nonrandomized and may reflect selection bias. Results were in the same direction when we examined VPRS score changes by number of teacher care management sessions completed (Table 23).
Adjusted VPRS Scores by Number of Parent Sessions.
Adjusted VPRS Scores by Number of Teacher Sessions.
Because none of our primary or secondary outcomes were significant, we were unable to assess for mediation of effects by family engagement in treatment and mental health services use. We did, however, assess associations between family engagement in treatment and intervention group status and between family engagement in treatment and the primary outcome, change in VPRS scores. We also assessed associations between mental health services use and intervention group status.
As a validated measure of family engagement in ADHD treatment did not exist, we developed and validated a measure of engagement, the AES. We conducted a review of the literature and focus groups with parents of children with ADHD to identify item concepts. We then conducted cognitive interviews with parent stakeholders to test the appropriateness and understandability of items from parents' perspectives. Following focus groups and cognitive interviews, we included 28 items in a preliminary AES (Appendix B, Table 8). All items had 4-point response scales (1 = “not at all/a little bit” to 4 = “very much”). Study participants completed the AES at study visit 4. Scales were labeled Care Team Access (5 items), Patient Family-Centered Care (PFCC; 6 items), Communication (3 items), and Understanding of ADHD (5 items). Item response theory modeling showed that all items adequately discriminated between parents with different levels of engagement. The Care Team Access and PFCC domains showed ceiling effects, with the majority of respondents selecting the highest item-level response category. Missing data was minimal (<6%). Domain mean value ranged from 2.52 ± 1.06 for Communication to 3.49 ± 0.64 for Care Team Access. Cronbach α values for domains ranged from .86 for Communication to .93 for PFCC, indicating that items within domains hang well together. We compared mean scores for each domain between groups (Table 24). The Care Team Access domain had the highest overall mean of 3.48, while the Communication domain had the lowest overall mean of 2.54. There was no difference between groups in average engagement across any of the domains. Though not statistically significant, the care manager group (tier 2) did report higher scores across all domains except PFCC.
Mean (SD) AES Scores by Intervention Arm.
We assessed the association between each of the 4 AES domains and intervention group status. We regressed each of the 4 domain scores on intervention group status, controlling for race, urbanicity, education, medication status, median household income, and season, and we clustered by practice site (Tables 25-28 show individual models). There were no significant associations between intervention group and any of the 4 domains of family engagement, suggesting that engagement in treatment was not different between groups. Of the individual covariates, current medication status was associated with the PFCC and Understanding of ADHD domains of Family Engagement.
Adjusted Care Team Access AES Scores.
Adjusted PFCC AES Scores.
Adjusted Communication AES Scores.
Adjusted Understanding of ADHD AES Scores.
We assessed the association between each of the 4 AES domains and final VPRS scores. We regressed final VPRS scores separately on each of the 4 domain scores controlling for intervention group and season and clustered by practice site (Tables 29-32 show individual models). The Access to Care, PFCC, and Understanding of ADHD domains showed strong associations with lower VPRS scores, while the Communication domain was marginally associated with VPRS scores, suggesting that greater engagement was associated with lower ADHD symptom scores. Intervention × AES domain interactions were all nonsignificant, suggesting no effect of the intervention with any domain of engagement on ADHD symptoms (see Appendix B, Tables 9-12).
Adjusted VPRS Scores and Intervention × Care Team Access AES Score.
Adjusted VPRS Scores and Intervention × PFCC AES Score.
Adjusted VPRS Scores and Intervention × Communication AES Score.
Adjusted VPRS Scores and Intervention × Understanding of ADHD AES Score.
We examined differences in mental health services use between groups (Table 33). Of those who completed the SACA at study visit 4, 65% reported any past mental health services use. Most of this past services use was for ambulatory mental health services. Few (5%) experienced a psychiatric hospitalization. Of those who reported any past service use and who answered the question about current use, 67% reported any mental health services use during the 9-month study period, with most of this being ambulatory services. There were no differences between groups in any or past 9-month mental health services use. In addition, there were no differences between groups in ADHD medication use over the study period (data not shown). Appendix B, Table 13 shows differences in mental health services use by more refined categories.
Descriptive Characteristics of Study Participants' Use of Services.
To determine whether individual, family, and community factors modified the effects of the interventions on ADHD symptoms (heterogeneity of treatment effects [HTE]), we included models with 3-way interactions involving intervention group, time, and either race/ethnicity, medication status, urbanicity, parent education level, or neighborhood median household income. None of the 3-way interactions were significant, suggesting that there was no HTE between the tier 1 and tier 2 interventions. Tables 34 to 38 show the separate models of each of the 3-way interactions.
Adjusted VPRS Scores, I × T × Race/Ethnicity Interaction.
Adjusted VPRS Scores, I × T × Urbanicity Interaction.
Adjusted VPRS Scores, I × T × Education Interaction.
Adjusted VPRS Scores, I × T × Medication Status Interaction.
Adjusted VPRS Scores, I × T × Median Household Income Interaction.
To better understand how the care managers interacted with study participants, we conducted semistructured interviews with 19 purposively selected caregivers who participated in the tier 2 intervention (Table 39). Interview participants were all female, 13 were from urban primary care practices, and 6 were from suburban practices. Participants ranged in age from 33 to 65 years old with a mean of 42.8 years. Of the children discussed in the interviews, 10 were Black or African American, 5 were White, 1 was American Indian or Alaskan Native, 1 was Asian, 1 was multiracial, and 1 identified as other. We identified 4 primary themes. First, caregivers perceived that care managers provided necessary education and information concerning ADHD and strategies to address related conditions (eg, how to develop a homework report card to assist children with homework completion). This information was compiled in educational packets that were distributed to participants at enrollment. Second, caregivers perceived that care managers improved their understanding of ADHD and its treatment; however, this knowledge could create conflicts if other caregivers lacked this understanding, because other caregivers might not agree with this understanding. Third, caregivers perceived that care managers improved communication with their child's school and clinic. However, they felt that communication with care managers was sporadic and infrequent. Fourth, caregivers felt that care managers assisted them in tracking goals and maintaining treatment plans, but the lack of face-to-face contact impeded building more trusting relationships to help with goal attainment.
Themes from Qualitative Interviews With Parents in the Tier 2 Intervention Arm.
In this comparative effectiveness study of 2 communication strategies, an electronic patient portal combined with a care manager vs a portal alone, we found no difference in primary or secondary outcomes between the 2 groups. We observed an overall improvement in ADHD symptoms over time but little to no change in goal attainment or PROs in both groups. This suggests that electronic communication using patient portals may be sufficient to provide necessary communication to improve ADHD symptoms; however, engagement with both the portal and care manager was low.
The finding that patient portals can improve ADHD symptoms is consistent with previous studies employing patient portals for ADHD. Epstein and colleagues found that community pediatricians using a portal were significantly more likely to collect information from parents and teachers than pediatricians practicing usual care.105 Nagykaldi and colleagues found that using a portal focused on preventive care resulted in increased patient activation and greater patient-centered care, and users were more likely to receive needed preventive care.106 However, disparities have been found in portal use, with White and privately insured families more likely to use these tools.107 Our study did not support the findings that racial/ethnic disparities in portal use exist, but there was a relatively low rate of portal use overall among all participants.
There was a lack of consistent engagement by parents not only with the portal but with the care manager. It is not entirely clear why parents did not engage to a greater extent with the care manager than with the portal. Our study results showed no difference in AES scores between groups, and our exit interviews with 19 parents suggest that communication with care managers was sporadic and infrequent and lacked face-to-face contact to build rapport and trust. The care manager intervention was structured to have check-ins every 3 months with parents and teachers, so as to coincide with 3- to 6-month visit schedules between families and clinicians. However, it was difficult for the care manager to schedule calls with parents and teachers. In some instances, we were unable to identify names and contact information for teachers. Parents in our exit interviews indicated a preference for more frequent face-to-face communication.
Face-to-face contact may be an essential ingredient in engaging patients in ADHD care that was absent from our study. For example, a study of community health workers who were representative of the population served and who provided weekly face-to-face communication showed beneficial effects on the quality of care and a reduction in hospital days.108 In addition, systematic reviews of collaborative care trials among adults with depression have consistently demonstrated that care management involving face-to-face contact is associated with modest but sustained improvement in depression outcomes (standardized mean difference = 0.25; 95% CI, 0.18 to −0.32) relative to usual care at 6 and 12 months.109,110 Subgroup analyses showed that care management with regular planned supervision of care managers and use of care managers with mental health training had better outcomes relative to less rigorous interventions. Although our study employed regular supervision of care managers through discussion of cases at weekly meetings and training in ADHD, our care managers were unable to engage regularly in face-to-face sessions given logistical constraints.
Given the lack of differential effectiveness between treatment arms in the current study and prior studies showing benefits from patient portals, we would recommend that clinicians, teachers, and parents use portals as a primary means of communicating and coordinating care for children with ADHD. Most prior studies of patient portals have not used tools to permit communication directly between parents and school staff. Our patient portal, however, permitted sharing of information not only between parents or teachers and clinicians but also between parents and teachers. The latter communication strategy may have helped to improve communication and coordination. However, portal use by both parents and teachers was still relatively modest and may require additional strategies to bolster usage.
We did not find HTE by individual, family, or community characteristics. Goel and colleagues found that White patients from higher socioeconomic strata were more likely to use patient portals than ethnically and racially diverse patients from lower socioeconomic strata.107 Our study found that differences by race/ethnicity, parental education level, urban residence, and neighborhood median household income were not associated with any of our outcomes. This may have been the result of a lack of engagement by most study participants without regards to differences in race/ethnicity, residence, education level, or neighborhood.
Our study was conducted in a single geographic area within an integrated pediatric health care system. Results may not be generalizable to other geographic areas or other types of health care systems. In addition, our care managers primarily employed electronic means of communication (text, email, phone), because this was feasible across multiple practices and did not require travel. However, parents who participated in our exit interviews perceived a lack of rapport and trust with care managers without face-to-face contact. Additionally, we did not enroll teachers in our study due to complexities with multiple school district ethics boards. This limitation did not permit us to interview teachers to discern their perceptions of the intervention. Finally, we lacked a no-intervention control group, as use of the electronic portal was considered standard of care at our institution during the study period. This limited our ability to assess any benefits of the portal over no portal care.
Given these limitations, we would still recommend future studies involving patient portals and care managers/coordinators for children with ADHD to test strategies to improve patient engagement. Such studies could employ more intensive and frequent strategies, such as face-to-face contact, than were employed in this study, as postintervention parent surveys suggested this approach. In addition, attention to engagement strategies that may help retain parents in interventions and meet individual needs could potentially be very useful to examine.
We conclude that care management combined with a patient portal did not produce outcomes that were different from electronic patient portal use alone among children with ADHD. Overall, there was poor engagement by parents with portals and with care managers, which may have biased results toward the null. There was no HTE as a function of race/ethnicity or socioeconomic factors. Future studies should consider methods to better engage families and teachers with care managers and patient portals to improve outcomes. Given the qualitative feedback from participants in this study, such methods should consider additional face-to-face sessions or video conference calls with participants and an increased frequency of contacts.
We would like to thank the clinicians and patients at CHOP Care Network Karabots, Cobb's Creek, South Philadelphia, CHOP campus, Chestnut Hill, Haverford, Drexel Hill, Media, Broomall, HighPoint, and Indian Valley for participating in this study. We would also like to thank all our patient and stakeholder partners who participated in biannual stakeholder meetings to provide important guidance and insight.
Research reported in this report was funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#CDR-1408-20669). Further information available at: https://www.pcori.org/research-results/2015/improving-communication-about-care-goals-children-adhd-adhd-link
Qualitative Interview Guide (PDF, 122K)
Ancillary Tables 1-18 (PDF, 176K)
Non-Imputed Vanderbilt Parent Rating Scale Scores by Study Visita,b (PDF, 83K)
Non-Imputed Parent Patient Reported Outcomes Scores by Study Visit and Intervention Arma,b,c (PDF, 45K)
Non-Imputed Child Patient Reported Outcomes Scores by Study Visit and Intervention Arma,b,c (PDF, 70K)
Fully Implemented Caregiver and Teacher Care Management Fidelity Checklists for the Intervention Arm (PDF, 35K)
AES item-level dimensionality and item response theory (IRT)a,b,c (PDF, 100K)
Adjusted Vanderbilt Parent Rating Scale Scores, Intervention by Access Engagement Scorea,b,c,d (PDF, 72K)
Adjusted Vanderbilt Parent Rating Scale Scores, Intervention by PFCC Engagement Scorea,b,c,d,e (PDF, 87K)
Descriptive Characteristics of Study Participants' Use of Servicesa,b (PDF, 70K)
Guevara JP, Fiks A, Power T, et al. (2020). Improving Communication about Care Goals for Children with ADHD. Patient-Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/11.2020.CDR.140820669
The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.
This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License which permits noncommercial use and distribution provided the original author(s) and source are credited. (See https://creativecommons.org/licenses/by-nc-nd/4.0/
Your browsing activity is empty.
Activity recording is turned off.
See more...
