Background:

Although children's health insurance coverage has expanded, some eligible children are still uninsured and others experience frequent coverage gaps. Community health centers (CHCs) care for low-income patients, many of whom are eligible for public coverage; thus, CHCs are an ideal setting for testing interventions to increase children's health insurance coverage rates.

Objectives:

We designed an intervention that used electronic health record (EHR)–based tools to help CHCs provide children's insurance enrollment assistance. We hypothesized that the group of children for whom the tool was used would have increased odds of gaining and maintaining coverage, increased likelihood of having return visit(s) after an uninsured visit, lower odds of being uninsured at subsequent visits, and increased rates of receiving recommended pediatric care.

Methods:

We implemented EHR-based tools in 4 intervention CHCs and selected 4 matched control CHCs that did not implement the tools. Using mixed methods, we assessed tool adoption and impact of tool use on insurance coverage, care utilization, and receipt of recommended care over 18 months after tool implementation, comparing 3 pediatric groups: (1) patients in the intervention clinics for whom the tools were used (n = 2240), (2) patients in the intervention clinics for whom the tools were not used (n = 12 784), and (3) patients from matched control clinics (n = 12 227). We quantitatively measured whether tool use affected insurance coverage, care utilization, and pediatric care quality.

Results:

Overall use of the insurance assistance tool was low. The most commonly used tool, the Tracking and Documentation Form, was applied to 15% of targeted patients; it was also used, unexpectedly, among children with no clinic visit and adult patients (often when their child was being assisted). A key factor underlying these results was the effect of concurrent policy initiatives. Patients for whom the tool was used had higher rates of continuous insurance coverage during follow-up (62% vs 59%), and were 76% more likely to gain insurance and 29% less likely to lose insurance coverage compared with those for whom the tools were not used. Among patients with an uninsured visit, those for whom the tools were used were 83% more likely to have a return visit and 54% less likely to be uninsured at all return visits. Intervention patients had significantly higher rates of several recommended care quality indicators, including well-child visits, human papillomavirus vaccination, and up-to-date childhood and adolescent immunization status.

Limitations:

The intervention was not randomly allocated. Despite statistical regression adjustment, unmeasured differences among participating clinics may have affected results. The follow-up period may have been too short to adequately assess tool uptake and impact. Concurrent policy initiatives also may have affected tool use.

Conclusions:

Tool use was low but had significant impact. This pragmatic trial, the first to evaluate EHR-based health insurance enrollment support tools, suggests such tools can increase insurance enrollment, prevent coverage loss among CHC patients, and promote delivery of recommended pediatric care services. Future efforts should engage diverse stakeholders in collaboratively designing tools that can be adapted to changing initiatives and clinic environments and can support enrolling family units.

Patient/Stakeholder Engagement

Based on the established framework for identifying key groups of stakeholders for clinical effectiveness research,³⁶ we identified a diverse group of relevant stakeholders that included policymakers, payers, providers, clinic leadership, administrators, and staff, as well as patient advocates and patient families. We engaged the stakeholders in all phases of the project: developing the proposal, adapting study methods, understanding the context of obtaining and maintaining public health insurance for children, and designing and refining the tool. The stakeholder engagement process for this project is described in detail elsewhere³² and summarized here.

The idea of developing EHR tools to help clinics identify patients whose insurance had expired or was nearing expiration originated from a CHC staff member. We focused this stakeholder idea on children's health insurance coverage and developed the study concept, design, and approach in collaboration with CHC leaders, clinicians, patient representatives, and policymakers from our Patient Engagement Panel and Practice-Based Research Network. Involvement in study development led to formally appointing a patient coinvestigator who was actively involved throughout the study and participated in dissemination activities, including conference presentations and manuscript publications.

To develop our intervention, we employed a user-centered design approach. “User-centered design” broadly describes processes employed to ensure that end-users influence computer software design.³⁷ Sequential strategies involved in user-centered design include initial interviews to collect background data about needs and expectations; subsequent interviews to collect data about sequence of work; focus groups to discuss issues and requirements; on-site observation to collect information about the environment; walk-throughs and simulations to evaluate prototypes; usability testing; and follow-up interviews about user satisfaction.³⁸ Using these strategies, we actively engaged stakeholders and solicited their feedback and insights about the development and use of study tools. We applied user-centered design to ensure that study tools were appropriate, relevant, usable, and patient and family centered.

During the preimplementation and tool development phases of the study,^32,39 we conducted qualitative interviews with clinic staff and families of pediatric patients (n = 44) to understand their perspectives on obtaining, keeping, and managing health insurance for their children. Our interviews revealed that families think health insurance is very important, but they find the process of applying for insurance to be onerous and confusing. They trust their CHCs and clinic staff to help them manage the application process and to track information about their applications. Clinic staff were enthusiastic about supporting families with the insurance enrollment and retention process. We also held a focused advisory group meeting with policymakers from the State of Oregon, patient representatives, community leaders, and members of the research team (n = 13) to exchange ideas and information about the project, state health insurance exchange, and Affordable Care Act. Based on learnings from these activities, we made a stakeholder-driven decision to develop our tools for clinic staff as the primary users.

Our tool development team then created a set of tool prototypes that were shared with stakeholders from the 3 participating clinic systems (CHC leaders, staff, and families; n = 14) at a half-day retreat. The research team summarized findings from the staff and family interviews, presented possible tool options, and solicited feedback on the prototypes. Next, during 2 on-site clinic visits, the research team engaged key clinic staff (n = 6) in Think-Aloud exercises to further refine the prototype tools.⁴⁰ In this user-centered design process, key users “thought out loud” while performing specified tasks and reviewing the tool prototypes. We completed an iterative series of 5 cycles of review and refinement with CHC staff and patient stakeholder involvement. We also engaged state policy stakeholders at key points during this process. A health literacy consultant reviewed final prototypes, and we incorporated suggestions for improving readability and usability into the final tool design.

When the prototypes were finalized, we built and activated the tools. We conducted beta-testing across the 4 intervention sites with insurance eligibility specialists, schedulers, and front desk staff (n = 16). After final modifications, we made the tools available to all staff at intervention sites. Six weeks after the tools were implemented, we conducted usability trials with end-user clinic staff (n = 24). Based on these trials, we redesigned some of the tools for ease of use and modified some functions to enhance clarity. A year after tool implementation, we conducted follow-up visits at 3 of the intervention clinics (n = 12) to observe insurance assistance processes and use of the tools and to assess perceptions of the tools.

As the project neared completion, we held a final stakeholder event with patients, clinicians, staff, and administrators from the study clinics as well as policymakers, patient representatives, and staff from other interested clinics (n = 28). We provided a project status update, discussed preliminary findings, and sought stakeholder input on ways to increase use of the insurance support tools as well as future directions for research.

Overview

The study period was the first date of tool implementation (June 1, 2014) through 18 months postimplementation (November 30, 2015). We addressed Q1—rate of and factors associated with tool adoption—using a convergent, mixed methods design, wherein we collected qualitative and quantitative data before and after tool implementation, and converged them to explain tool use patterns.⁴¹ We addressed Q2 and Q3—associations between tool use and changes in insurance coverage, use, and preventive care receipt—using a retrospective cohort design that involved quantitative data only.

Study Setting

We conducted this study at 8 (4 intervention and 4 control) CHCs serving low-income, ethnically diverse populations, including many publicly insured and uninsured pediatric patients, who would potentially benefit from the tools. CHCs were members of OCHIN, Inc, a 501(c)(3) that provides health information technology to >400-member CHCs in 19 states.^42,43 OCHIN member clinics share a single instance of the Epic EHR, which enabled us to build and implement the tools described here on a single EHR platform across all intervention sites and to obtain EHR data from all study sites.

Sample

Four OCHIN member clinics volunteered to be intervention sites. We selected 4 comparison sites from a pool of 38 qualifying nonintervention clinics, using a propensity score technique⁴⁴ that selected comparison sites that were the closest possible matches to the intervention sites based on patient population demographics and date of EHR implementation. Because of the small clinic sample size, we were ultimately able to match on only 3 factors: ratio of children to adults, percentage Hispanic ethnicity, and length of EHR experience.

Intervention

We developed a suite of EHR tools via a user-centered design process, described in more detail above as well as in our publications.^31,32,45 Briefly, through direct workflow observation at CHCs and interviews with CHC staff, we identified the following deficiencies in the process of providing health insurance assistance to patients: (1) lack of information on patients' insurance end dates, (2) lack of automated systems for identifying patients in need of assistance, and (3) lack of a standardized system for tracking assistance provided to patients. We then employed low-fidelity wireframe prototyping, beta-testing, and usability testing to design health insurance tools to address these gaps.

We created the tools for use specifically by those CHC staff who assisted patients with health insurance enrollment (eg, outreach and enrollment assistants/eligibility specialists; front desk [registration/scheduling] staff).⁴⁵ Staff in these roles at both intervention and control sites have instituted insurance review and support workflows to ensure that as many of their patients as possible are insured. They regularly check insurance status and identify patients who may need help with health insurance applications. In addition to insurance verification at time of scheduling, patients may come to the attention of enrollment assistants/eligibility specialists in a number of other ways: being referred from the front desk at check-in, being contacted through active outreach by clinic staff, by word of mouth from the community, or by seeking assistance on their own. For those patients who require assistance, enrollment staff may complete, submit, and track both new and renewal applications on their behalf.⁴⁵ Enrollment staff maintained expertise in insurance eligibility and enrollment rules and procedures, the range of qualified health plan options and insurance affordability programs, and the needs of underserved and vulnerable populations. The majority of enrollment staff at all sites were bilingual in English and Spanish.

The suite of tools was embedded into the OCHIN Epic EHR Registration module and consisted of the following:

1. Tracking and Documentation Form for Insurance Applications (Tracking Form; Figure 2) provided electronic functionality for documenting assistance provided to patients who applied for public health insurance. Specifically, we designed the Tracking Form to record relevant insurance application data, including fields to capture the status of the application (submitted, pending, approved, and denied), the date application was submitted, the insurance type, the enrollment assistant who opened the form, and a free text field for entering notes about missing information or action steps for the next appointment. The form also included a field for entering a follow-up date to support the workflows for checking the status of an application.
2. Reporting/Roster Tools (Roster Tool; Figures 3 through 6) enabled clinic staff to create a daily report listing patients who have a Tracking Form and (1) for whom insurance assistance has already been initiated and are in need of follow-up (IMPACCT Follow-up Report), or (2) who have an upcoming clinical appointment and might need insurance enrollment support (IMPACCT Upcoming Appointments Report). Access to the Tracking Form for each patient included on the report was readily available from these reports, facilitating updates to the form.
   The IMPACCT Roster Tools can be accessed by searching the library of reports available within the electronic health record.
   An IMPACCT Follow-up Report can be selected from the library of available reports and is ready to run.
   Running the IMPACCT Follow-up Report results in a roster of patients for whom insurance assistance has already been initiated and follow-up is needed; used primarily by insurance assistants/eligibility specialists.
   An IMPACCT Upcoming Appointments Report can be selected from the library of available reports and is ready to run.
3. Insurance Coverage Information (pop-up warnings, coverage verification option, appointment information insurance data, Department Appointments Report): To meet clinics' need to identify patients in need of health insurance application assistance, we made health insurance information visible in a few different locations in the EHR Registration module:
   1. Pop-up Alerts (Figure 7): Notifications that popped up when clinic staff closed the registration page of, scheduled an appointment for, or checked in a patient whose health insurance was soon to expire or who appeared eligible but not enrolled in public health insurance
   2. Coverage Verification Option (Figure 8): A new dropdown menu option under Coverage Verification that allows staff to indicate that public health insurance coverage may soon be expiring for a patient
   3. Appointment Information Insurance Data (Figure 9): New fields in the Appointment Information section that display health insurance data (Medicaid Redetermination Date, Medicaid Application Status, Insurance Assistant Appointment)
   4. IMPACCT DAR (Figures 10 and 11): A special Department Appointments Report (DAR) created to highlight insurance expiration data (Medicaid Redetermination Date, Medicaid Application Status, Insurance Specialist Appointment Date). Users could add any of these data elements to an existing, customized DAR.

Figure 2

Tracking and Documentation Form for Insurance Applications.

Figure 7

Pop-up Alert.

Figure 8

Coverage Verification Option.

Figure 9

Appointment Information Insurance Data.

Figure 10

IMPACCT DAR.

Figure 11

Accessing IMPACCT DAR.

Figure 3

Accessing Reporting/Roster Tools.

Figure 4

Reporting/Roster Tools—IMPACCT Follow-up Report.

Figure 5

IMPACCT Follow-up Report.

Figure 6

Reporting/Roster Tools—IMPACCT Upcoming Appointments Report.

The IMPACCT DAR, designed for Front Office staff use, creates a roster of patients for scheduling/check-in, along with relevant health insurance information.

Front office staff can add new IMPACCT data elements (eg, insurance expiration date, next appointment with insurance assistant) to appointment information by selecting and running the IMPACCT DAR from the Available Settings menu under Scheduling/Department Appointments.

We designed all study tools to be accessed and used by CHC staff; individual patients for whom the tools were used were unaware of their use and thus were passive participants in the study. The tools were available for use with any patient (ie, not limited to children or Medicaid enrollees); staff initiated tool use based on clinic-specific workflows.

Implementation

We built study tools on OCHIN's single EHR platform and distributed them centrally from OCHIN into the study sites' EHR. Figure 12 illustrates the timeline of the iterative process undertaken for tool development and implementation. We initially rolled the tools out on June 1, 2014. Over the course of the next year, CHC staff were periodically engaged in activities related to tool implementation and evaluation, including on-site training, usability testing, and postimplementation process evaluation. We made updates to the tools based on feedback from these activities. Initially, in response to usability testing, we redesigned some of the tools for ease of use and modified language on several functions to enhance clarity, such as expanding drop-down list options and adding data fields. Next, when we discovered that the accuracy of some of the insurance coverage data provided by the State of Oregon was unclear, we obtained additional data elements from the state to more precisely inform the algorithm that calculated the insurance end dates embedded in the IMPACCT tools. We worked with analysts from the State of Oregon throughout the study to iteratively improve our process of obtaining regular data feeds. Following implementation of the improved insurance end date algorithm, we learned that another barrier to wider adoption of the tools was the lack of Tracking Form reporting capability. We met with CHC stakeholders to elaborate reporting specifications and developed new reporting functionality accordingly. We implemented the final iteration of the tools on April 2, 2015.

Figure 12

Timeline of Iterative EHR Tool Development and Implementation Process.

Analyses—Q1 Tool Uptake

Quantitative

Measurement was limited to the Tracking Form, the only tool for which usage data could be extracted from the EHR and readily quantified. To evaluate how frequently it was used at the study CHCs, we used a retrospective cohort design to measure its use in the 18 months after the tools were implemented at intervention clinics. We evaluated use of the other tools (reports, pop-up alerts, and coverage verification options) using qualitative data.

We extracted EHR data to assess Tracking Form use (when the tool was opened and edited within a patient chart) and to collect patient demographics and visit data. We assessed patients' insurance coverage using OCHIN EHR data (coverage status at a visit) and Oregon Medicaid enrollment data, which specified coverage start and end dates. Patients covered by Oregon Medicaid retain the same Medicaid identification number over time; our 2 quantitative data sources were linked using this identification number via established methods.^46-48 Medicaid data also include a case number that identifies households, which enabled us to measure how long each patient's household was “established” at a clinic. Because Oregon requires all enrollees to recertify their Medicaid enrollment annually, we designed our study was with an 18-month intervention period as a means of ensuring that all Medicaid patients would have the opportunity to use enrollment assistance at least once during the study.

We retrospectively identified 3 parallel pediatric (aged 0-19 at time of visit) cohorts for analyses. Study cohorts included all pediatric patients with at least 1 clinical encounter in the study period, regardless of insurance status. “Intervention patients” (n = 2240) were those with ≥1 clinical visit at an intervention clinic, and for whom the Tracking Form was used. Because tool use was determined by CHC staff, patients did not self-select into the intervention group; all patients at intervention clinics had the opportunity for the tools to be used on them. “Within-clinic comparison patients” (n = 12 784) were those for whom the tool was not used, despite having ≥1 clinical visit at an intervention clinic during the study period. We identified the within-clinic comparison group to control for differences in health outcomes owing to changes in other clinic procedures, staffing, and workflows that may have occurred during the study period. “Control clinic patients” (n = 12 227) were patients with ≥1 clinical visit at matched control sites that did not have access to the tools. Because of the insurance review and support workflows in place at both intervention and control sites, control site patients would have had equal opportunity for the tools to be used on them had the tools been available in their clinics. Clinic staff at the intervention sites also used the tools for 2 unanticipated groups outside the study cohorts: children with no clinical visit (n = 969) and adult patients (n = 3207; see Figure 13).

Figure 13

Overview of Study Groups.

We calculated the number of patients for whom the Tracking Form was used (numerator; intervention patients) over the number eligible for tool use (denominator; all eligible patients in the intervention CHCs aged 0-19 with ≥1 clinical visit). We selected the total patient population as our denominator, given that any patient had the opportunity to be a “tool use patient.” Using chi-square tests, we compared demographic, care use, and coverage characteristics of intervention patients vs the 2 comparison groups.

Analyses—Q2 Tool Impact on Coverage and Health Care Visits

We structured analyses to address our second study question to test the following 3 hypotheses: (1) Tool use would be associated with higher odds of gaining/maintaining Medicaid insurance coverage; (2) among children with an uninsured visit, tool use would be associated with higher odds of returning for a clinical visit, because coverage would remove an access barrier; and (3) among children with an uninsured visit who returned for a second visit, tool use would be associated with lower odds of being uninsured at subsequent visits. These analyses are premised on the assumption that uninsured children would benefit most from the tools. To test these hypotheses, we conducted the following analyses within relevant subsets of our 3 cohorts (intervention patients, within-clinic comparison patients, and control clinic patients):

1. Among patients who could be linked to Medicaid data (those with a Medicaid identification number in the EHR), we assessed (1) the odds of gaining coverage and (2) the odds of losing coverage during the study period. The assessment period was June 1, 2014, to September 30, 2015. Medicaid enrollment data used to evaluate this outcome were available only through September 30, 2015, so these analyses ended at that date.
2. Among those who had an uninsured visit within 6 months of the tools' implementation date, we calculated the odds of returning for a second visit. The assessment period was December 1, 2013, to November 30, 2015. We selected this outcome based on the recommendation that most children should have at least 1 wellness visit per year.⁵²
3. Among patients in (2) who did return for a visit, we calculated (1) the odds of being uninsured and (2) the odds of being insured by Medicaid at return visit(s) within the assessment period of December 1, 2013, to November 30, 2015.

In these analyses, our assessment period extended to 6 months before tool implementation, to include patients who would have been identified as uninsured or about to lose coverage, if the tool were used for panel-based outreach. We calculated odds ratios using adjusted generalized estimating equation (GEE) models to account for within-clinic clustering and adjusted for demographic and utilization covariates.

Analyses—Q3 Tool Impact on Receipt of Recommended Care

We structured analyses to address our third study question to test the hypothesis that tool use would be associated with higher rates of recommended care receipt in the 18 months following tool implementation, compared with within-clinic comparison patients and control clinic patients.

We assessed receipt of recommended quality using the Child Core Set of health care quality measures set out by the Children's Health Insurance Program Reauthorization Act (CHIPRA). We extracted data from the EHR and used CHIPRA technical specifications published in March 2015,⁵³ the most current version available during our study's assessment period. The following indicators were deemed most appropriate to our study population and yielded sufficient sample size for analysis: well-child visits in the first 15 months of life; well-child visits in the third, fourth, fifth, and sixth years of life; adolescent well-care visit; body mass index (BMI) assessment for children and adolescents; human papillomavirus (HPV) vaccine for female adolescents; chlamydia screening in women; childhood immunization status; and immunizations for adolescents. We assessed all measures among eligible subpopulations over the 18-month period after study tools were implemented (June 1, 2014, to November 30, 2015). Using chi-square statistics, we tested between-group differences in the percentage of eligible children meeting each measure. We also estimated adjusted odds ratios for meeting each measure, comparing the intervention group with within-clinic comparison patients for whom the tools were not used, and with control clinic patients.

For all quantitative analyses, we estimated odds ratios using adjusted GEE models to account for within-clinic clustering; we set statistical significance was at α = .05. We assumed a compound symmetry correlation structure and applied a robust sandwich variance estimator to account for possible model misspecification.⁵⁴ Because our propensity score matching technique was insufficient to balance on many differences in the cohort populations, we adjusted all models for a range of patient covariates: gender, age, race/ethnicity, language, Federal Poverty Level (FPL), number of encounters, new/established patient status, household members on case, and length of time household established with clinic. We also analyzed the 3 cohorts as described above using an intent-to-treat (ITT) approach, in which all children with ≥1 clinical visit at an intervention clinic, regardless of tool usage, were compared with the group of control clinic patients (Appendix). We were cognizant of the fact that ITT analyses might not be valid in the presence of low uptake rates, which provided little information about the efficacy of the tools and made it more susceptible to type II error.⁵⁵ We conducted all analyses using SAS software, version 9.4 (SAS Institute, Inc). The Oregon Health & Science University IRB approved the study.

Q1: Tool Uptake

Tool Use Rates

Overall, use of the suite of tools we developed to support insurance activities was low. Over the course of the study period, the Tracking Form was used for approximately 15% of all pediatric patients with a clinical visit at an intervention site (Figure 13 and Table 1). Although we could not track the other tools quantitatively, qualitative assessments suggest that the Pop-up Alerts and Roster Tool were also used on a small subset of patients. Additionally, we observed that Tracking Form use changed over time (Figure 14): An immediate spike in use rates occurred when the tool went live in June 2014, then utilization dipped for several months. After we conducted additional site visits and trainings that fall, starting in November 2014, tool use increased gradually but steadily throughout the remainder of the follow-up period but remained low overall.

Figure 14

Tool Use by Month with Key Study Dates Noted.

Qualitative data offer several explanations for this tool use pattern. A key factor was the effect of concurrent initiatives. Most notably, clinic staff were initially very enthusiastic about the tools' potential to alert them when a patient's insurance was due to expire. However, because of Affordable Care Act insurance expansion implementation in Oregon, the state received an overwhelming influx of Medicaid applications just before the study period began, which extended many coverage expiration dates to prevent coverage loss owing to a longer processing times for renewal paperwork.

Because of everything with OHA [the Oregon Health Authority] being so crazy, renewal dates are all over the place. A lot of people … are not renewing and they're just being extended. [The state is] extending expiration dates so that … people don't lose coverage, even though they were supposed to lose coverage, like, back December 31st. But they're extending them because there's so many people applying, and they're so short staffed and, you know, so they just keep extending, extending, extending so that they have time to … renew medical benefits. [Clinic 5, Eligibility Specialist]

Despite a greater amount of insurance application activity, when staff realized that the coverage end dates shown in the tools did not coincide with the state's extended coverage end dates, they stopped using the tools:

We were excited because [the tools were] going to tell us when it's time to renew. We're going to be able to research all these patients and call them…. But then it didn't really work. And then the front [desk staff] would schedule appointments because [the tool] would say it's time to renew. But then when they would come [in] I would call and [the state] would say, no, it's not time. So that was kind of a bummer. We thought it would work, and … help not only the patient but, you know, us too. [Clinic 5, Eligibility Specialist]

We think that the insurance is supposed to be expired by January. Then we realize that it's been extended 3 more months. When we check [in] 3 more months, the next day there's another 3 more months. So my staff are like, “How can I do any follow-up?” … There is not a real and exactly accurate redetermination date for us to support our patients. [Clinic 4, Eligibility Specialist Supervisor]

In addition, during the study period, new health insurance outreach initiatives required CHCs to use specific processes to track insurance application assistance. In 2013, HRSA announced a new funding opportunity for CHCs to extend insurance assistance to all community members, whether patients or not.²² The CHCs participating in the study received this HRSA community outreach grant funding, which changed their workflows and reporting requirements. Although some overlap occurred between the data fields in the study tools and the HRSA data tracking requirements, HRSA required the collection of additional information that was not included in the Tracking Form, including but not limited to number of family members being assisted or included on each health insurance application. The intervention CHCs also participated in an alternative payment program that required staff to document every type of patient interaction, including insurance assistance, using specified reporting methods.

The study tools had been designed based on prior workflows and were not easily adapted to the workflows necessary to meet the requirements for these new initiatives, so clinics developed alternative tracking mechanisms to document the newly required data. As a result, CHC staff were tracking HRSA data and insurance-related interactions in clinic-developed spreadsheets and not using the study tools, which would have required time-consuming and duplicative data entry.

In addition, staff did not receive clear direction from management about using the various recording systems available to them. One staff member said she did not use the tools because she had not been instructed by a manager to do so; another said she had been instructed to use only the clinic's spreadsheet, not the tools under study:

I'm just doing what we were told to do … to keep track on the [spreadsheet] insurance tracking list. And that's how I go and put all the information. [Clinic 4, Eligibility Specialist]

These concurrent initiatives not only created confusion about which tools to use for which purposes, but also increased the eligibility specialists' workloads. As a result, the eligibility specialists had little time to conduct proactive outreach activities using the Roster Tool to identify patients whose insurance was nearing expiration.

Respondent: It's been really hard for me to get into that list. I know I should be working on it. But there is … no way unless we blocked certain time from our schedule to get into the list and see which ones need to re-apply.

Interviewer: What are the barriers?

Respondent: The workload on working with appointments every hour. And sometimes we have families of 7 or 8 people. And, even though I do the application online, it's fairly time consuming … because sometimes people don't have all the information with them. So unless we block the schedule … there is not enough time to do the follow-up list. [Clinic 1, Eligibility Specialist]

Tool Use by Patient Characteristics

Although tools were intended for use with all pediatric patients, actual tool use appeared to be driven, in part, by patient characteristics. Children for whom the Tracking Form was used differed from those for whom it was not used, most notably by ethnicity: 95% were Hispanic, compared with 65% of within-clinic comparison patients; and 88% used Spanish as their primary language, compared with 51% (P < .001 for all differences; Table 1). Qualitative results concur that patients' ethnicity and language drove the use of the insurance assistance tools. Eligibility specialists reported that Spanish-speaking patients often requested application assistance and often referred other families that needed insurance assistance.

Some [Spanish-speaking patients] don't want to apply just because they don't think they qualify. [Clinic 4, Front Desk Worker]

Since we've been here for a long time, the word of mouth with the Hispanic families has been the word of mouth, basically. They know where we are and they just come. [Clinic 3, Eligibility Specialist]

I can be like a liaison between the client [and the state]. And especially if they're Spanish speaking, they tend to be more intimidated to call their case worker to go into the office. [Clinic 2, Eligibility Specialist]

In addition, intervention patients were older, had lower household incomes, were more likely to be established clinic patients at their first study visit, and had more clinical encounters in the study period compared with within-clinic comparison patients (P < .001 for all comparisons; Table 1). Among patients with a Medicaid ID, intervention patients were also more likely to have another household member on their Medicaid case (P < .001; Table 2).

Table 2

Medicaid Measures and Coverage Status, Among the Subset of Patients With a Medicaid ID.

Tool Use Beyond the Intended Study Population

Although the tools were designed for use with all pediatric patients seen at the study clinics, we observed tool use in 2 groups not included in our main study population: (1) children who were not established patients and (2) adults. Children who were not established clinic patients (had no clinical visits during the study period; n = 969) were older, less commonly Hispanic or Spanish speaking, and less likely to have a Medicaid ID than intervention patients (P < .001 for all; Table 3). Among adults for whom the tool was used, most were female (65%), under age 50 (77%), with household incomes ≤138% of FPL (88%), Hispanic (86%), and Spanish speaking (80%). Of these adults, 23% were assisted with insurance but had no medical visit during the study period. Almost a third of adults with a Medicaid ID did not share a case number with other household members, indicating that the Tracking Form was also used for “single” adults, not just family members of children already being assisted.

Table 3

Demographic and Encounter Characteristics, Individuals With Insurance Tracking Form Use Outside the Study Population.

Our qualitative work helps explain these findings. The eligibility specialists often aided entire families with their health insurance; when helping a child apply, the specialist screened the rest of the household and assisted all eligible family members at the same appointment.

The eligibility specialist pulls a paper [application] and begins to fill it out with the family…. She writes in the names and info about all the family members. [Clinic 1, Fieldnotes]

Interviewer: How long does it take you to complete an application with the family?

Respondent: It depends. If it's just 1 person, it takes about 45 minutes. But if it's a family, a big family, 5 or 6, it takes me a little bit more than an hour. [Clinic 4, Eligibility Specialist]

Because EHRs are designed to be used at the patient level (ie, users work in only 1 patient chart at a time, and the Tracking Form was linked to an individual patient's chart), the study tools did not allow eligibility workers to document assistance provided to a family unit without conducting multiple data entries.

Q2: Tool Impact on Coverage and Health Care Visits

Tool Use Was Associated With Higher Odds of an Uninsured Child Returning for a Subsequent Clinic Visit

Among the subset of patients with ≥1 uninsured visit within 6 months of tool implementation, the adjusted odds of returning for ≥1 follow-up visit was significantly higher in the intervention group than in the within-clinic comparison group (aOR, 1.83; 95% CI, 1.39-2.40) and also higher than in the control clinic group, although the difference was not statistically significant (aOR, 1.64; 95% CI, 0.92-2.91; Table 4).

Table 4

Follow-up Visit and Coverage Status Among the Subset of Patients With ≥1 Uninsured Visit Within 6 Months of Tool Implementation.

Q3: Tool Impact on Receipt of Recommended Care

Tool Use Was Associated With Higher Rates of Receipt of Quality Recommended Care

In the 18 months after tool implementation, intervention patients for whom the Tracking Form was used had higher rates of most assessed CHIPRA indicators compared with both within-clinic comparison patients and control clinic patients (Table 5). For example, 63% of eligible intervention patients had the recommended number of well-child visits in the first 15 months of life, compared with 39% of within-clinic comparison patients for whom the tools were not used (P < .001) and 53% of control clinic patients (P = .02). Female adolescents for whom the tool was used were more likely to complete HPV vaccination by their 13th birthday (54%) vs 36% for within-clinic comparison patients and 38% for control clinic patients (P < .001 for both). Intervention patients were more likely to be up-to-date on most immunization indicators, compared with both comparison groups. The only measures for which intervention patients did not perform significantly better than either comparison group was BMI assessment and chlamydia screening among sexually active women. Many of these observed differences remained significant when adjusted for covariates and clinic clustering (see Table 5). Results of ITT analyses were mixed (Appendix, Table A.4).

Table 5

CHIPRA Measures of Recommended Care Receipt in the 18 Months After Tool Implementation.

Implications

This study contributed new knowledge about the effectiveness of innovative health insurance enrollment systems and EHR-based data tools in supporting CHCs as they assist families with the process of obtaining and retaining children's health insurance coverage and deliver high-quality, evidence-based pediatric care. Based on the many promising findings from this first study, we believe it is important to continue pursuing this line of inquiry, which has significant future implications for improving patients' health.

We have already embarked on 2 exciting next steps. First, we are working with the leadership of the study intervention clinics to use our tools and findings in crafting effective strategies for meeting their 2016-2017 organizational goal of “No Gaps In Coverage” for all of their Medicaid-eligible patients. Second, findings from this study directly informed the design and implementation of a hybrid effectiveness-implementation trial, the Community-based HIT Tools for Cancer Screening and Health Insurance Promotion (CATCH-UP) study, funded by the National Cancer Institute (NCI R01 CA181452, DeVoe PI). We have shared the final algorithms and functionalities of the IMPACCT health insurance enrollment assistance tools, training materials, pre- and postimplementation qualitative findings, and best practices with the CATCH-UP study. The 5-year CATCH-UP study is building on lessons learned, stakeholder engagement, and findings from IMPACCT by developing a next generation of health insurance enrollment tools and spreading the updated suite of tools to 23 additional OCHIN clinic sites. The CATCH-UP health insurance tools include updated and expanded tracking and reporting functionality, including new data fields to support HRSA outreach and enrollment reporting. In addition, the new tool structure captures the number of individuals in a family unit, without identifying who those family members are. CATCH-UP will formally study implementation support strategies to identify key facilitators for increasing the tools' use as well as their effectiveness with adult patients. Upon completion of the CATCH-UP study, and based on the learnings from both IMPACCT and CATCH-UP (if the updated version of the health insurance enrollment tools prove effective), our ultimate goal is widespread dissemination of the tools, not only throughout the OCHIN clinic network (>450 clinic sites) but nationwide.

The potential long-term impact of our work is to maximize health insurance enrollment, promote insurance coverage stability, improve health outcomes, enhance the efficiency of health care system performance, reduce health care disparities for vulnerable populations, and ultimately improve the health of at-risk populations.

Limitations

To accommodate CHC stakeholder requests early in the study, the intervention was not randomly allocated to the 4 CHCs initially recruited into the study—all 4 received the intervention. Instead, we used propensity score techniques to identify the 4 CHCs that were closest to the intervention clinics based on patient panel characteristics and date of EHR implementation. However, significant differences remained between intervention and control sites, probably because of the small number of nonintervention CHCs available for matching. We attempted to address this imbalance through additional statistical regression adjustment. Although this approach is not as closely controlled as a more traditional randomized trial, it was chosen to suit this pragmatic, real-world evaluation. This approach was also more ethically appropriate for research in this setting because it enabled us to avoid recruiting CHCs with the possibility of being randomized into a control arm.

Unmeasured differences between the intervention and control CHCs may have affected results. As with any study, unobserved differences may have existed between study groups that affected insurance enrollment, which in turn may have affected the study results. Through our qualitative work, we learned that eligibility specialists, the primary users of the Tracking Form, had similar responsibilities at both the intervention and control sites, but were unable to compare them quantitatively.

We focused on the impact of the tools on the children for whom the tools were used. The addition of a within-clinic comparison cohort—patients seen in CHCs with access to the tools but for whom the tools were not used—provided another comparison group for quantitative analyses and accounted for potential clinic-level effects. Additionally, we adjusted analyses for patient- and clinic-level confounders, but some model-based adjustment may have inadequately controlled for the differences between study groups. We also assessed clinic-level effects in a secondary ITT analysis, which expectedly showed few significant results. This, too, reflects the nature of pragmatic studies in which tool use is an optional rather than a required part of the intervention, as with a more controlled study. Furthermore, the tools were used on a small number of patients, so we did not assess clinic-level changes in uninsured rates.

Because EHRs are not structurally designed to link records for families, we could not directly assess whether tool use was affected by the number of individuals being assisted in a given family. We could use Medicaid enrollment data to assess family size, but only among household members who ever received Medicaid. The length of our follow-up period was 18 months, which may not have been sufficiently long to assess uptake and impact. In addition, it was technically feasible to quantify only Tracking Form use; while we evaluated use of the other tools in qualitative data collection, we were not able to assess whether that affected the outcomes of interest here. Based on the relatively low amount of tool use, as well as the other limitations noted above, inferences drawn from this study should be considered tentative.

Appendix: Intent-to-Treat (ITT) Analyses