Does an Advanced Electronic Tracker Help Families Manage Children's Asthma Symptoms Better Than a Standard Electronic Tracker?

Parent/Stakeholder Engagement in the Current Proposal

We engaged 10 parents of children with asthma in the design and validation of the paper-AT. While developing the current proposal, we worked with 1 parent representative who used the paper-AT for her child. Upon funding, we recruited 5 more parents for a total of 6 parents. We also engaged 2 PCPs to provide insight in facilitating e-AT implementation at clinics. We also involved other stakeholders, including representatives from insurance companies (SelectHealth and Medicaid), the Utah Department of Health (school nurse and asthma program), and the Intermountain Healthcare (IH) system (pediatric and primary care clinical program directors), as well as an external advisory committee of recognized health care leaders. Overall, we engaged 16 parents and 13 stakeholders to solicit their input in designing and testing the paper-AT and e-AT, and in developing the current proposal, including objectives, study design, and outcome measures. Input was received through focus group sessions⁴³ and facilitated discussions.

Methods to Identify/Recruit Parents/Stakeholders

We recruited parent partners from a cohort of parents of children who used the paper-AT. To ensure a good representation in characteristics that could affect participation, we divided parents into 2 groups based on their adherence with weekly use of the paper-AT, and selected 2 frequent and 2 infrequent users. We also solicited referral from PCPs, who helped us enroll 2 more parents. We recruited community stakeholders based on established relationships from previous collaborations.

Engagement Frequency

After funding, we held a 2-day kickoff retreat with all stakeholders. We watched 2 PCORI videos on stakeholder engagement in research and discussed expectations regarding their participation and required trainings (eg, Human Subjects Protection, HIPAA). We discussed the provisional study protocol for additional stakeholder input, reviewing and recording direction regarding study objectives, design, and outcome measures. During the study, we held monthly meetings with clinic care coordinators (CCCs), parents, and the research team, and quarterly meetings with the entire team to discuss progress and issues raised in the study. Other parent engagement activities included (1) developing clinic/participant recruitment materials, (2) participating in monthly research progress conference calls, (3) reviewing performance data and assisting in solving issues, (4) monitoring project goals and providing ongoing support, and (5) conducting site visits to address barriers to study recruitment. The advisory community members were included in the various work phases, providing their input through phone calls or emails, or via face-to-face meetings (2 times per year).

Perceived or Measured Impact of Parent/Stakeholder Engagement

Study Rigor and Quality

Although stakeholder engagement led to the selection of outcome measures that were meaningful to them, changes to the initial design (due to stakeholders' request) to allow the e-AT to be provided to all participants and clinics, reduced the rigor and quality of the study and led to a more complex study design (see the Methods section).

Transparency of the Research Process

We engaged parents and other stakeholders through the entire research process. All major decisions regarding the study process were made with parent partners and stakeholders, often using language understandable to them, leading to increased transparency. We can share upon request the study documentation (eg, study protocol, programming code and data definitions) and deidentified data, so others may reproduce results.

Adoption of Research Evidence Into Practice

Parent stakeholders were eager to facilitate clinic recruitment based on their own experience using the e-AT. They also worked to improve participant's adherence by developing patient recruitment materials and a Facebook page to strengthen parent support and to increase their ability to sustain ongoing use of the e-AT and study participation. Their presence during clinic recruitment and outreach visits increased the study's credibility and motivated clinic participation.

Setting/Initial Design/Objectives

The study involved 11 primary care clinics, with 9 clinics belonging to the IH system. These clinics are located in Salt Lake City and rural regions in northern and southern Utah. IH is a regional, not-for-profit integrated health care delivery system, with 22 hospitals and 185 clinics and urgent care facilities in Utah and southeastern Idaho,⁴⁹ providing care to ∼1 680 000 patients, which includes ∼60% of Utah's residents and 85% of Utah's children.⁵⁰ We enrolled children 2-17 years old with persistent asthma and their parents. The study was initially a prospective, cluster-randomized trial with 480 children (2-12 years old) and parents from 10 clinics to be randomized to the e-AT vs usual care.

Study Design Change/Updated Objectives

We made changes (see details under the “Perceived or Measured Impact of Parent/Stakeholder Engagement” section) to the study design, age criteria, and outcome measures, and 1 clinic was added (making 11 clinics), leading to a new design with staggered implementation of clinics, split in 2 groups, early (started on January 1, 2014) vs late (started on April 1, 2014) starting clinics, with a 3-month delay period between the 2 starting dates. We then cluster-randomized clinics to standard vs intensive e-AT. Figure 2 illustrates the updated study design.

Figure 2

Study Design and Randomization Procedures.

Final objectives included the following:

• Objective 1: To assess the effectiveness of the new ambulatory care model, by comparing outcomes for the (1a) child (QOL, asthma control, and interrupted/missed school days due to asthma); (1b) parent (satisfaction and interrupted/missed workdays due to a child's asthma); (1c) clinic (ED/hospital visits due to asthma), between clinics randomly assigned to the standard vs intensive e-AT intervention; and (1d) use of several nonrandomized comparisons to determine the e-AT effectiveness relative to usual care periods or control groups (see the “Five Nonrandomized Comparisons Between e-AT Interventions and Usual Care” section for details) in which the e-AT was not used.
• Objective 2: In the intervention groups, to assess whether the effect on child outcomes varies across parent characteristics (high vs low health literacy, Medicaid vs private insurance, frequent vs infrequent users).
• Objective 3: In the intervention groups, to determine (demographic, socioeconomic, behavioral, and technology) factors associated with sustained parent participation in asthma self-management (defined as sustained use of the e-AT).

Forming the Study Cohort

We conducted an outreach visit and education at clinics that met the eligibility criteria and showed interest to participate in the study.

Interventions

Standard e-AT Intervention

The e-AT, web (Figure 3) and mobile web (Figure 4) versions have 4 components: (1) patient portal, (2) survey questionnaire, (3) report generator creating color-coded dynamic run charts (Figure 5), and (4) decision support pop-up messages/recommendations) tied to asthma control scores, displaying green, yellow, and red zones, with personalized feedback (email/text messages tailored to each patient).^43,46 The standard e-AT features include (1) automated reminders to sustain use; (2) real-time result graphing; and (3) alerts that are sent to patients/parents (via email/text messages) and the PCP's office (via clinic email and e-AT clinic web-based dashboard) on the patient's deteriorating asthma control; and (4) real-time decision support (pop-up messages recommending appropriate actions) based on color zones: green (continue current care), yellow (follow asthma action plan, and contact PCP if score remains in the yellow zone the following week), and red (contact PCP immediately for evaluation). The dashboard (Figure 6) facilitates patient management by a CCC or PCP, providing real-time access to a patient's asthma control status, poor asthma control alerts, graphs/decision support, adherence to weekly e-AT use, and patient contact methods. Following the alert, the clinic proactively contacts patients/parents to identify and address asthma care issues and adjust treatments if needed using step therapy.³⁵

Figure 3

e-AT Web Version.

Figure 4

e-AT Mobile Web Version.

Figure 5

User Interface (Dynamic Run Chart): Standard e-AT Intervention.

Figure 6

Care Coordinator Dashboard.

Intensive e-AT intervention

Intensive e-AT was developed to increase participant adherence with weekly use through additional motivational features (Figure 7), including a progress bar, which adds 25 points/participant completion of an assessment. At 100 points (∼4 assessments per month), a pop-up message is generated that includes fireworks, a congratulatory message, and a notice that an incentive ($10 gift certificate) is being sent by mail. The incentive system resets to 0 after it reaches 100 points. The second motivational feature is the leader board, which allows participants to see adherence of the 5 best e-AT users, motivating them to increase their own adherence. Standard e-AT users were also sent the gift after 4 assessments per month but did not see the progress bar, leader board, fireworks, or congratulatory message. Yet, the graph about the child's asthma status, recommendations about management, and clinic dashboard were similar in both standard and intensive interventions.

Figure 7

User Interface (Dynamic Run Chart): Intensive e-AT Intervention.

Follow-up

Study Outcomes

Data Collection and Sources

At enrollment, parents completed surveys addressing demographic, socioeconomic, and behavioral factors; parent satisfaction; and education level (replaced health literacy). A technology factor survey also included 2 open-ended questions assessing barriers/facilitators of parent participation. Baseline surveys were collected at the time of enrollment on paper, but follow-up surveys were collected online using the Research Electronic Data Capture (REDCap) database. Outcomes data, including ED/hospital admissions—using ICD-9 and ICD-10 codes and oral corticosteroid use, 1 year pre- and 1 year post-e-AT initiation—were collected using the IH EDW,⁵³ an integrated database linking administrative and clinical data from IH hospitals and clinics. The EDW was also used to collect patient demographics (age, gender, race, and insurance) and to create a matched control group. Table 1 lists data elements, validated survey instruments and references, data sources, and frequency of data collection.

Table 1

Timeline of Data Collection and Survey Instrument and Data Collection Source.

Statistical Analysis

We used SAS 9.4 statistical software (https://www.sas.com/en_us/software/sas9.html) to conduct all analyses. The new design allowed randomized comparisons of 2 levels of e-AT intensity, but not e-AT effectiveness vs usual care. Consequently, we included 2 general categories of analyses:

1. Randomized comparisons of standard vs intensive e-AT, comparing QOL and secondary outcomes.
2. Four nonrandomized analyses to assess the e-AT effectiveness compared with usual care.
   1. The first analysis assessed longitudinal change within-patient (both standard and intensive e-AT users) in outcomes (QOL, satisfaction, asthma control, interrupted/missed school days, and interrupted/missed workdays) compared between baseline (before e-AT initiation) and follow-up assessments while using the e-AT.
   2. The second analysis (early vs late starting clinics) or clinic-level analysis compared oral steroid use and hospital/ED admissions between enrolled participants in clinics that started the e-AT early vs participants in clinics that started the e-AT late (see also the “Comparison of ED/hospital admissions and oral steroid use between patients enrolled…” subsection).
   3. The third analysis (early vs late starting patients) or patient-level analysis involved both standard and intensive e-AT and compared oral steroid use and ED/hospital admissions between enrolled participants that started the e-AT in the first year vs those that started the e-AT.
   4. in the second year, during the 1-year period before their e-AT start dates (serving as usual care period) when late-starting participants were not using the e-AT (see also the “Comparison of ED/hospital admissions and oral steroid use 1 year post-e-AT initiation …” subsection).
   5. The last analysis (limited to ED/hospital admissions and oral steroid use) compared participants in the e-AT (standard and intensive) groups vs nonrandomized matched controls or participants with persistent asthma from clinics that did not participate in the e-AT study.

Objective 1: Assess the Effectiveness of the New Ambulatory Care Model

Objective 2: Assess Whether the Effect on Child Outcomes Varies Across Parent Characteristics (High vs Low Education, Medicaid vs Private Insurance, Frequent vs Infrequent Users)

We assessed the e-AT (standard and intensive interventions) effectiveness in different parent subgroups (eg, high vs low education, private vs Medicaid insurance). Health literacy was replaced by education because of lack of time by CCC and difficulty of data collection during clinic visits.

For ED/hospitalization admissions, we used Cox regression to relate the risk of first hospital/ED admission to the cumulative proportion of e-AT use (numbers of weeks) before that admission in a 3-month window, along with indicator variables for low education and Medicaid insurance at baseline, and an interaction term between indicator variables. The baseline hazard function was stratified by clinic site. We also assessed the dose-response relationship between risk of hospital/ED admissions and the cumulative proportion of e-AT use (categorized), comparing patients with vs without low education and Medicaid vs private insurance subgroups.

For asthma control, we applied GEE for binary ACT scores (19-25 vs 5-18) with logistic link function and robust SEs under an independence working covariance model to relate the odds of good control during a given week to the cumulative proportion of e-AT use (in a 3-month window) before that week, using the same sets of covariates (plus clinic site as a covariate rather than a stratification factor) and interaction terms to compare dose-response relationships of the odds of good asthma control with prior cumulative proportion of e-AT use between low vs high education and Medicaid vs private insurance.

For QOL, we used GEE by applying a normal outcome model with an identity link function, and QOL measurements at 3, 6, and 12 months each related to the cumulative proportion of e-AT use before the week of QOL measurement, using the above covariates and interaction terms. We also compared dose-response relationships of mean QOL with prior cumulative proportion of e-AT use across the 2 subgroups.

Objective 3: Determine Factors Associated With Sustained Parent Participation in Asthma Self-management

We defined a dichotomous outcome for each month as “frequent” (if ≥3 weekly assessments recorded/month) or “nonfrequent” e-AT users otherwise. We used a logistic regression model to relate the odds of frequent e-AT use to sets of predictor variables after adjusting for clinic (categorical factor), with the following sequence:

1. We applied subject-matter considerations to define baseline predictors, including demographic (parent age, sex, and race/ethnicity), socioeconomic (marital status, highest education completed, household income, number of children in household, and primary language spoken), behavioral, and technology factors, and subdivided predictors into modifiable and nonmodifiable.
2. We examined correlation matrices, variance inflation factors, and contingency tables to identify any collinear factors and reduce the list of predictors, if needed.
3. We examined the association of e-AT use frequency with each predictor by fitting separate GEE for each predictor, with clinic used as a categorical covariate.
4. We introduced the assignment to the intensive e-AT as a time-dependent covariate in each GEE model. We examined interaction terms between baseline predictors and treatment assignment to determine if the association of sustained participation and predictor variables differs in time period when the intensive e-AT was applied vs time period when it was not applied.
5. We used multivariable GEE regressions to relate frequent use to each individual modifiable risk factor after adjustment for nonmodifiable factors, the time-dependent intensive e-AT indicator variable, and clinic as covariates.

Strategies for Handling Missing Data

Analyses described above were all applied to all available data, allowing all nonmissing variables to be included regardless of whether measurements were missing at a time point, as missing data seemed to occur at random. In general, analyses using the mixed effects models utilize likelihood-based statistical inference and remain unbiased so long as data are missing at random. Also, analyses using GEE remain nearly unbiased as long as data are missing completely at random, making use of all nonmissing measurements of variables in the model, including baseline covariates.

Conduct of the Study

All changes to the original protocol, including reasons for changes, are described in previous sections of this report. Any changes that occurred after the study start date were documented in the study protocol. We obtained PCORI and IRB approvals before implementation of the study and subsequent changes.

Objective 1: Assess the Effectiveness of the New Ambulatory Care Model

Of the 11 participating clinics, 2 had only 1 patient enrolled. These 2 clinics (and the 2 patients) were excluded from all analyses. Of the 325 children/parents enrolled, 318 (97.2%) completed their baseline surveys and were included in descriptive analysis of baseline characteristics. Figure 8 (CONSORT diagram) shows the study population at different times. The dropout rates at 3, 6, and 12 months were similar between the 2 groups, including 10.0%, 12.7%, and 20.9%, respectively, for the standard e-AT intervention, and 13.3%, 13.3%, and 20.0%, respectively, for the intensive intervention. (The 4 ClinicalTrials.gov tables [patient flow, baseline characteristics, outcomes and statistical analysis, and adverse events] are available at ClinicalTrials.gov.)

Figure 8

CONSORT Diagram.

Randomized Comparisons Between Standard and Intensive e-AT Interventions

Of the 318 children, 261 (82%) received the standard e-AT intervention and 57 (18%) the intensive e-AT intervention. Table 2 provides baseline characteristic distribution of the study population.

Table 2

Baseline Characteristics of Children Who Receive Intensive vs Standard e-AT Interventions.

Primary outcome (QOL)

Overall baseline average QOL scores were 77.5 (95% CI, 71.1-83.8) points for the standard e-AT and 79.2 (95% CI, 72.5-85.9) for the intensive e-AT (P = .713). There was a significant QOL improvement over baseline at 3, 6, and 12 months (P < .001) following implementation of the standard and intensive e-AT interventions individually and overall (Tables 3 and 4). However, when compared together there was no significant difference (P = .961) between the 2 interventions. Figure 9 shows change from baseline (x-axis) at different follow-up times, with highest QOL increase occurring at 3 and 12 months, for both interventions. Overall, both interventions were effective in improving QOL over time. Although participants on the standard e-AT intervention tended to have highest QOL scores at all follow-ups, no statistical difference were found between them. Therefore, additional motivational mechanisms built within the intensive e-AT intervention to improve user adherence did not make any differences in outcomes.

Table 3

Summary of QOL by Clinic and by Timeline.

Table 4

QOL/Intensive vs Standard e-AT Interventions at Different Follow-ups (Fixed-effects Model).

Figure 9

Least-Squares Means of Change (x-Axis) in QOL: Intensive vs Standard.

Asthma control

Overall, asthma control scores increased significantly (P < .001) in both intensive and standard e-AT interventions individually and overall (Figure 10 and Tables 5 and 6), across all follow-up quarters. However, increases in asthma control among participants on the intensive e-AT tended to be lower than increases seen in those on standard e-AT. However, when compared, there was no significant difference (P = .926) in asthma control between the 2 groups.

Figure 10

Least-Squares Means of Change (x-Axis) in Asthma Control Scores: Intensive vs Standard.

Table 5

Summary of Asthma Control Score per Quarter and per Clinic.

Table 6

Asthma Control/Intensive vs Standard Interventions at Different Follow-ups (Fixed Effect).

Interrupted/missed school days

Similar to QOL and asthma control, number of interrupted/missed school days were reduced significantly (P < .001) across all follow-ups (month) in both intensive and standard e-AT interventions individually and overall (Figure 11). Also, overall reduction in number of interrupted/missed school days among participants on the intensive e-AT tended to be lower (especially at 3 and 6 months) than reductions seen in patients on standard e-AT (Table 7). Yet, when compared, there was no significant difference (P = .493) in asthma scores between the 2 intervention groups.

Figure 11

Least-Squares Means of Change (x-Axis) in Interrupted/Missed School Days: Intensive vs Standard.

Table 7

Interrupted/Missed School Days/Intensive vs Standard Interventions Across Follow-ups (Fixed Effect).

Interrupted/missed workdays

Fixed-effects analysis showed no change in the number of interrupted/missed workdays across all follow-up periods for the intensive e-AT, but there was a significant (P < .001) reduction in standard e-AT overall and at 3, 6, and 12 months. Although those patients receiving the intensive intervention tended to have fewer interrupted/missed workdays, there was no significant difference (P = .119, except at 6 months) between intensive and standard e-AT interventions (Figure 12 and Table 8).

Figure 12

Least-Squares Means of Change (x-Axis) in Interrupted/Missed Workdays: Intensive vs Standard.

Table 8

Interrupted/Missed Workdays/Intensive vs Standard Interventions Across Follow-ups (Fixed Effect).

Parent satisfaction

Satisfaction scores ranged from 1 to 5 (1 = very dissatisfied, 2 = dissatisfied, 3 = neutral, 4 = satisfied, and 5 = very satisfied). Patients in the intensive group experienced a slight but not significant (P = .09) reduction of 0.14 points in satisfaction score, while patients in the standard e-AT experienced a significant (P < .001) but minimal reduction of 0.24 points in satisfaction score. Overall, there was no significant difference (P = .244) between the 2 groups.

ED/hospital admissions

The average ED/hospital admission was 0.15 (0.50 SD) for the overall study population, and 0.22 (0.60 SD) during preintervention and 0.09 (0.35 SD) during postintervention. Compared with baseline, both intensive (relative risk [RR], 0.12; 95% CI, 0.05-0.25; P < .001) and standard (RR, 0.08; 95% CI, 0.04-0.16; P < .001) interventions achieved a significant reduction in ED/hospital admissions. However, there was no significant difference (P = .505) in ED/hospital admissions between the 2 intervention groups.

Longitudinal Within-Participant Analyses

These analyses assessed within-patient longitudinal change in outcomes (QOL, satisfaction, asthma control, interrupted/missed school days, and interrupted/missed workdays) compared between baseline (before e-AT initiation) and follow-up assessments while using the e-AT.

Quality of life

Overall, average QOL scores were 79.1 at baseline and 90.9, 90.0, and 90.6 at 3, 6, and 12 months, respectively. The increase from baseline was the highest (12 points) at 3 months and then plateaued around 90 through the end of the study (Table 9). Patterns of increase (P < .001) in QOL scores occurred not only overall but also at almost all individual clinics (Figure 13 and Figure 14).

Table 9

Longitudinal Within-Participant Analysis of QOL: Summary by Clinic and Timeline.

Figure 13

Least-Squares Means of Change (x-Axis) in QOL Scores Over Time.

Figure 14

Change in QOL Between Different Follow-up Time and Baseline Score, and by Clinic.

Asthma control

Average asthma control scores at baseline and quarters 1, 2, 3, and 4 were 18.8, 22.3, 22.8, 22.8, and 22.9, respectively, with the highest increase occurring in the first quarter and then plateauing at about 22 to 23 for the remaining follow-up quarters (Figure 15). Overall, asthma scores increased significantly (P < .001) in most clinics and across all follow-up quarters (Figure 16).

Figure 15

Least-Squares Means of Change (x-Axis) in Asthma Control Scores Over Time.

Figure 16

Change in Asthma Control Between Different Follow-up Times and Baseline Score, and by Clinic.

Interrupted/missed school days

The overall average number of interrupted/missed school days at baseline was 1.91, and 0.79, 0.52, and 0.79 at 3, 6, and 12 months, respectively, with the largest reduction occurring by 3-month follow-up. A reduction (P < .001) in the number of interrupted/missed school days was seen across most clinics (Figure 17) and follow-up schedules (Figure 18).

Figure 17

Least-Squares Means of Change (x-Axis) in Number of Interrupted/Missed School Days Over Time.

Figure 18

Change in Interrupted/Missed School Days Between Different Follow-ups and Baseline, and by Clinic.

Interrupted/missed workdays

The overall average number of interrupted/missed workdays was 0.72 at baseline and 0.27, 0.25, and 0.19 at 3, 6, and 12 months, respectively, with the largest reduction occurring at 3-month follow-up. A reduction in the average number of interrupted/missed school days was seen (P < .001) at most clinics (Figure 19) and follow-up schedules (Figure 20).

Figure 19

Least-Squares Means of Change (x-Axis) in Number of Interrupted/Missed Workdays Over Time.

Figure 20

Change in Interrupted/Missed Workdays Between Different Follow-ups and Baseline, and by Clinic.

Parent satisfaction

The overall average satisfaction score was 4.7 at baseline and was reduced (−0.21) to 4.5 points (P < .001) at 12 months (Figure 21). Fixed modeling analysis showed a significant reduction of points in satisfaction score. However, this reduction in parent satisfaction was slight and not clinically meaningful, as the 12-month score (4.5) was still within the highest satisfaction range (4-5). (Note: satisfaction score addressed multiple facets of overall asthma care, beyond the influence of the e-AT). Also, when scores were dichotomized to >3 vs ≤3, analyses showed no significant differences between baseline and 12-month scores.

Figure 21

Change in Parent Satisfaction Between Different Follow-ups and Baseline, and by Clinic.

Comparisons of the Frequency of Hospital/ED Admissions and Oral Steroid Use Between the Year Following Each Patient's Enrollment Visit and the Year Before the Enrollment Visit

The analysis compared overall hospital/ED admissions and oral steroid use in the study population (both standard and intensive e-AT), 1 year before initiation vs 1 year postinitiation of the e-AT.

ED/hospital admissions

The average ED/hospital admission was 0.15 (0.50 SD) overall, and 0.22 (0.60 SD) during preintervention and 0.09 (0.35) postintervention, and with an adjusted RR from preintervention to postintervention of 0.68 (95% CI, 0.49-0.95; P = .024), demonstrating significant reduction in the postintervention period. At individual clinics, ED/hospital admissions were significantly reduced in 2 clinics and tended to have a lower RR (though not significant) at most of the remaining clinics (Figure 22), probably because of limited power.

Figure 22

Adjusted Rate Ratio of ED/Hospital Admissions From Preintervention vs Postintervention, and by Clinic.

Steroid use

The overall average oral steroid use was 0.63 (1.1 SD and 0.74 [1.2 SD]) days during preintervention and 0.51 (0.95 SD) days postintervention, and with an adjusted RR from preintervention to postintervention of 0.74 (95% CI, 0.61-0.91; P = .003), demonstrating significant reduction in the postintervention period. At individual clinics, oral steroid use was significantly reduced in 2 clinics and tended to have a lower RR (although not significant) at most of the remaining clinics (Figure 23), probably because of limited power.

Figure 23

Adjusted Rate Ratio of Oral Steroid Use From Preintervention vs Postintervention, and by Clinic.

Comparison of ED/Hospital Admissions and Oral Steroid Use 1 Year Postinitiation of e-AT for Patients Who Started the Intervention Early With Those Who Started the e-AT Later, During a 1-Year Period Before Starting the e-AT Intervention

This patient-level analysis compared rates of ED/hospital admissions and oral steroid use during 1-year post-e-AT initiation for patients who were enrolled in the first year (between January 2014 to December 2014, also known as early starting patients) and were using the e-AT (or were in the intervention period), with rates during a 1-year period before enrollment of patients who were enrolled in the second year (between January 2015 to December 2015, or late starting patients), when they were not using the e-AT (usual care period).

Rationale

Participants in the control group (late starting patients were receiving usual care) later became e-AT users (or met study eligibility criteria) and their preenrollment usual care time periods (including change in seasons) coincided with the postenrollment time periods of those who enrolled early, limiting the confounding effects of seasonal change and secular trends in asthma care.

Due to sparseness of data (Table 10), we were not able to fit a GLME model with either fixed or random site effects to compare the rates of ED/hospital admissions and oral steroid use between the 2 groups. We had to combine all 1-year-before-enrollment data for late starting participants and compare it to all 1-year-after-enrollment data for early starting participants. Overall, the average number of ED/hospital admissions was high, at 0.18 (0.57 SD) per year, in the late starting patients (during the time they were not exposed to the e-AT) and low, at 0.10 (0.38 SD) per year, in the early starting patients (during the time they were exposed to the e-AT). However, when comparing late vs early patients, this difference was not statistically significant (P = .183), with an RR of 1.74 (95% CI, 0.77-3.93). In addition, the average numbers of oral steroid use were similar, with 0.53 (0.92 SD) per year in the late starting patients and 0.53 (0.99 SD) per year in the early starting patients; we found no significant differences (P = .937) between the 2 groups.

Table 10

Number of Early vs Late Starting Patients by Clinic.

Nonrandomized Comparison of Outcomes Between Patients From Randomized Clinics While Using the e-AT vs a Matched Control Cohort of Children With Persistent Asthma From Nonparticipating Clinics

The analyses in the preceding 2 sections allowed us to compare rates of poor asthma control in different periods, some in which e-AT was used and some in which it was not used. These comparisons of care outcomes with vs without e-AT at different times could be confounded by secular trends in asthma care. The nonrandomized comparison in this section addresses this potential bias. We compare asthma outcomes during the same periods in patients (cases) from randomized clinics that were using the e-AT to a matched control group of patients from nonparticipating clinics.

Comparison of ED/hospital admissions

There were 928 participants (325 cases and 603 matched controls) for the analyses (Table 11). The overall average ED/hospital readmissions (per 1000 days) was 0.42 (1.36 SD) for cases and 0.23 (0.99 SD) for controls. Among cases, ED/hospital readmissions were 0.59 (1.64 SD) during the pre-e-AT period and 0.24 (0.97 SD) during the post-e-AT period. However, among controls, ED/hospital readmissions were 0.23 (1.01 SD) during pre-e-AT initiation and 0.24 (0.97 SD) during post-e-AT initiation. Random-effects analysis showed the ED/hospital readmissions for cases were significantly reduced among cases (RR, 0.43; 95% CI, 0.27-0.67; P = .0003) but not among controls (RR, 1.04; 95% CI, 0.69-1.58; P = .817). The intervention effect was RR of 0.41 (95% CI, 0.22-0.75; P = .0041).

Table 11

Baseline Characteristics Between Cases and Matched Controls.

Comparison of oral steroid use

Overall average oral steroid use (per 1000 patient days) was 1.71 (2.90 SD) for cases and 1.93 (6.0 SD) for controls. Among cases, oral steroid use was 2.02 (3.14 SD) during the pre-e-AT initiation and 1.41 (2.60 SD) during post-e-AT initiation. Among controls, oral steroid use was 1.87 (3.12 SD) during preinitiation and 1.99 (7.89 SD) during postinitiation of the e-AT. Fixed-effects models showed the oral steroid use for cases was significantly reduced preinitiation to postinitiation of e-AT among cases (RR, 0.69; 95% CI, 0.52-0.93; P = .0139) but not controls (RR, 1.06; 95% CI, 0.87-1.31; P = .5565). The intervention effect was RR, 0.65 (95% CI, 0.46-0.93; P = .0191).

Objective 2: Assess Whether the Effect on Child Outcomes Varies Across Parent Characteristics

Demographics

Analysis involved the overall study population (325 children), for which 53 (16.3%) had Medicaid coverage, 203 (62.5%) had private insurance, 5 (1.5%) were self-paying, and 64 (19.7%) had unknown insurance. The median number of children in these families was 3 (interquartile range [IQR], 2-4); 226 (83%) were from married families, and 243 (89%) had 1 member of the household with some college education (Table 12). Overall, average adherence (as measured by frequency of weekly use of the e-AT) to e-AT dropped in the first month and stabilized afterward. The overall average adherence was about 65% at the end of the 1-year study period, slightly lower in patients using the intensive e-AT vs those on standard e-AT intervention (Figure 24).

Table 12

Family Demographics of Children With vs Without ED/Hospital Admissions.

Figure 24

Average Cumulative Proportion of e-AT Adherence (or Use) by Week.

Objective 3: Determine Factors Associated With Sustained Parent Participation in Asthma Self-management

Analysis of individual univariate factors revealed that higher income (β = .2343; SE = .0784; P = .0028) was associated with sustained e-AT use, while being married (β = −1.3327; SE = .4801; P = .0055) and having high education (β = −1.2596; SE = .6088; P = .0387) were associated with less sustained e-AT use; however, none of these factors was significant in a multivariable model, when controlling for time-dependent intensive intervention. Further, analyses of open-ended questions revealed that improved management of asthma symptoms, provider and staff support, and use of gift card/incentives were the most common factors that facilitated use of e-AT in clinics, while lack of physician buy-in or staff support, and lack of time and e-AT integration with the electronic medical records were commonly reported barriers.

Decisional Context

When comparing clinics randomized to intensive vs standard e-AT, we found no statistical differences in all outcomes. We also found no significant differences in oral steroid use and ED/hospital admissions when comparing outcomes between early starting clinics and late starting clinics during the 3-month delay period. Similarly, we did not find any statistically significant differences (although late starting patients had higher ED/hospital readmissions) between early starting patients and late starting patients. However, when we compared outcomes between receiving the e-AT intervention relative and not receiving the e-AT (or usual care), we found clinically and statistically significant differences. Specifically, we found a significant improvement over time in outcomes from baseline (with no e-AT use) in either standard or intensive or both e-AT interventions overall (see Figures 9-12), as well as in individual clinics. We saw this change across all outcomes (except parent satisfaction), including improvement of child's QOL and asthma control, reduction of child's interrupted/missed school days and parent's interrupted/missed workdays, and reduction of child's oral steroid use and ED/hospital admissions. Also, when comparing the year following patient's e-AT enrollment to the year before enrollment, we found a significant reduction in oral steroid use and ED/hospital admissions overall, and in individual intervention arms, as well as at most clinics. Further, when comparing with matched controls from clinics that did not implement e-AT, we found participants from clinics that received e-AT interventions had significantly lower oral steroid use and ED/hospital admissions.

The current ambulatory asthma care model is reactive and inefficient, and focuses on episodic acute care, addressing treatment of recurrent asthma attacks. Our study is among the few to demonstrate multiple outcomes for the child, parent, and clinic can be improved through implementation of a potentially low-cost health information technology (the e-AT), promoting early identification of signs of asthma control deterioration to prompt timely interventions. Our intervention represents a shift in ambulatory asthma care, from the current reactive acute care to a new model that is continuous and proactive, focusing on prevention of asthma attacks. Our results are helpful not only to families but also to health care providers, payers, and other community stakeholders willing to improve asthma care.

Study Results in Context

Previous studies have reported that adults who engage in effective self-management of a chronic disease have less activity limitation, improved QOL, improved perceived disease control, and improved medication adherence.^13,17-20 However, self-management interventions in children with asthma remain rare. The few previous studies of self-management interventions have either not evaluated the effect on asthma outcomes^22,23 or have reported mixed results with no effect^21,24-27 or little effect on asthma outcomes,^28,29 limiting their use in clinical practice.⁵⁶ Our intervention is among the few to demonstrate a significant improvement in child, parent, and clinic outcomes. With the e-AT, improvements in asthma control and QOL exceed those previously reported. For instance, Voorend-van Bergen reported a 2-point change in ACT scores as the minimal important difference.⁵⁷ In our e-AT intervention, improvement of ACT scores for patients who received the e-AT was 4 points already by the first quarter and was sustained through the end of the 12-month study period. Contrary to Rikkers-Mutsaerts's study (consisted of weekly asthma control monitoring with the asthma control questionnaire and treatment advice by a web-based algorithm), which reported improvement in QOL at 3 but not 12 months, our study of e-AT, a web-based intervention, showed sustained QOL improvement throughout the study duration.⁵⁸

Studies have reported that parents often lack critical information and tools to guide them in recognizing and acting on warning signs.³³ The e-AT has the potential to improve child asthma self-management, and thereby outcomes, by (1) increasing patient/parent roles in managing symptoms, including provision of self-care decision support; and (2) increasing awareness of chronic asthma symptoms through regular assessment and prompting patients to seek appropriate and timely visits with their PCPs when chronic control is deteriorating instead of waiting until they experience a severe asthma attack.^5,59-61 Ultimately, parents are empowered to recognize and act on early warning signs of declining asthma control or impending asthma attacks, including subtle symptoms that are usually missed. By increasing awareness of chronic asthma symptoms, the e-AT led to improved treatment of asthma, which ultimately resulted in better QOL and reduced use of acute health care services, potentially from optimal use of asthma medication.⁶²

Asthma guidelines recommend health care providers complete regular assessments of chronic asthma control, as it prompts timely and appropriate adjustments of therapy.⁶ However, studies continue to show that providers often lack or do not assess information about chronic asthma control. Lack of assessment results in failure to prescribe or appropriately adjust asthma therapy, leading to under treatment.^63,64 Provider failure to assess asthma control is attributed to a busy schedule, lack of resources, limited ability and time to support ongoing monitoring outside clinical encounters, and lack of an effective follow-up process between regular clinic visits.^63,64 The e-AT provides care providers with an effective way to monitor patients outside clinical encounters, providing real-time information about child asthma control status over time to capture early signs of deteriorations. Longitudinal information about a patient's clinical status is available to the provider for use in medical decision making rather than the provider relying on 1 assessment at the time of an acute visit and patient/family recall.

Despite potential benefits, implementation of self-management interventions in clinical practice remains low, and their uptake by patients and patients' adherence is often poor and declines significantly over time with high participant attrition.^65,66 van der Meer et al, for instance, reported a low adherence with weekly use of the ACQ for self-management of about 53% at 12 months.¹⁶ In our study, although adherence decreased over time, parent engagement remained high and stayed above 60% at the end of the 12-month study period. High adherence was likely due to our approach of providing training and support to PCPs' offices to deliver the intervention themselves. Participants were motivated to comply with weekly use of the e-AT knowing that the PCP would examine their data (based on verbal communication with some participants and clinic staff).

Implementation of Study Results

Implementing e-AT at clinics requires a designated CCC and support from PCPs. In a typical clinic, asthma patients are identified (and approached) at the time of an office visit or by mail/phone invitation. Clinic personnel enroll the patient (providing training, connecting families to the e-AT), which takes about 30 minutes. Although not formally measured, ongoing monitoring is very efficient. For instance, in one clinic, the CCC spent 30 to 40 minutes per week on average managing about 100 patients (this included receiving alerts, logging into the e-AT dashboard, reviewing the list of patients with low scores, making follow-up calls to identify and address issues, and scheduling an appointment with the PCP if needed). The e-AT dashboard has a tab that provides the list of patients (usually about 5/week) with low scores (ie, ones that trigger an alert) for the CCC to follow up, allowing the CCC to quickly focus attention only on those in need of follow-up (rather than reviewing the data of each patient) and thereby minimizing workload.

Morrison et al reported that digital interventions are poorly described, with insufficient information about barriers and facilitators to their uptake and utilization.⁶⁷ In our survey, the most important barriers to clinical staff adoption of e-AT in clinics were lack of physician buy-in and staff support. Clinic-level implementation needs to focus on securing PCP buy-in through education and training. The anticipated risks of e-AT use are minimal and primarily related to disclosure of protected health information. Fortunately, the e-AT has rigorous security protections as well as adherence to HIPAA privacy and data security requirements. Overall, we observed no adverse events.

Generalizability

The e-AT supports the use of evidence for best asthma care practices in the ambulatory care setting and helps meet self-management needs of not only children but also parents and PCPs. Also, our study showed outcome improvements with use of the e-AT among disadvantaged populations (see the “Subpopulation Considerations” section), and the e-AT may be generalized to inner-city children who experience asthma care and outcome disparities. Therefore, dissemination of our e-AT care model can lead to major improvements in asthma care and outcomes, with potential reductions in asthma-related health care costs in the United States, and can serve as a model to improve care for other pediatric chronic diseases.

Subpopulation Considerations

Underserved populations, especially low-income Hispanic and black children, experience asthma care and outcome disparities compared with well-served populations. In our study, we found, among those who used the e-AT, that having Medicaid insurance (compared with private insurance) and low education level (compared with high education level) tended to be associated with shorter time to hospital readmission. Yet, we noted nonsignificant improvements for frequent e-AT users with Medicaid or low education level. Also, our subgroup analysis demonstrates that QOL and asthma control also significantly improved among Hispanic patients (the largest minority population in Utah), other minority patients, and those with Medicaid, who also achieved improvements that were higher across all follow-up quarters than the overall study population. Thus, disadvantaged populations seem to benefit more from this new care model, suggesting the e-AT might also be a tool to address asthma disparities among disadvantaged and minority populations.

Study Limitations

Our study has several limitations, resulting from the following:

1. We changed the design to compare intensive to standard e-AT interventions, rather than a randomized comparison of the e-AT intervention with usual care (control). We relied on several nonrandomized controls and pre-post comparisons to evaluate the e-AT effectiveness compared with usual care. Our study results should be interpreted with caution as unrecognized biases may have been introduced by using nonrandomized controls and pre-post comparisons. Thus, factors other than e-AT may be responsible of our results.
2. Enrollment was unbalanced across the clinics in the randomized comparison (standard vs intensive e-AT), with 3 of the 4 high-patient-enrolling clinics (>20 patients per clinic) randomly assigned to the standard intervention and only 1 clinic assigned to the intensity intervention, leading to 60 enrolled patients in the standard intervention vs 60 enrolled patients in the intensive e-AT intervention. As a result, the power to estimate between-site variation in QOL and other outcomes was limited due to variable participant enrollment numbers across the clinics, particularly for statistical inferences under mixed models that account for intersite variation, and for nonnormally distributed outcomes. As a result, we also used fixed-effects analyses that applied statistical inferences specifically to participating clinics. However, when the fixed-effects approach is used, generalization of results from the current study sample to all other potential samples not in the study (beyond the specific enrolled clinics) may be limited. Applicability of results should be based on subject-matter considerations, since it depends on other unmeasured clinic factors.
3. Assignment of some (21%) patients to the standard e-AT before randomization led to different periods of postrandomization follow-up, adding complexity to the statistical analysis. Fortunately, 79% of the planned 1-year evaluation periods and all of the planned 12-month QOL assessments occurred after randomization, so the cost in statistical power incurred by variable evaluation periods after randomization was minimal.
4. We initially intended to use, but did not actually perform, multiple imputation analyses to address missing data. We instead used all available data in our analysis because missing data seemed to occur at random (however, we did not conduct formal analyses of missingness). Therefore, our study results may be biased due to dropout rates that exceeded 10% in some outcomes.

Future Research

We plan to disseminate the e-AT across multiple clinics in Utah. We shared our results with IH leaders, and they have expressed interested in disseminating the e-AT care model in their system. We are discussing with them the potential for funding and resources needed to achieve this goal. We hope to use this opportunity to conduct a rigorous pragmatic study that will allow better comparison of e-AT effectiveness relative to usual care. Schools are an additional setting for dissemination, as the e-AT can support school nurses' work to efficiently monitor many children with asthma, extending their reach to those from low socioeconomic households and racial/ethnic minorities experiencing asthma care disparities.