Effectiveness of Decision Support for Families, Clinicians, or Both on HPV Vaccine Receipt

Setting

This study was conducted within The Children’s Hospital of Philadelphia (CHOP) Pediatric Research Consortium (PeRC), a 2-state (New Jersey and Pennsylvania), hospital-owned, primary care practice-based research network including more than 202 000 children.³⁰ Of the 25 PeRC practices, 18 primarily suburban practices not involved in resident teaching and all 4 urban, resident teaching practices participated in the study (Fig 1). All practices use the ambulatory EHR, EpicCare (Verona, WI). Before the start of the study, we confirmed that insurance plans accepted by the CHOP Care Network covered the cost of the HPV vaccine. HPV vaccines could be received at preventive visits, acute visits, and nurse-only visits. At the study start, no practice had implemented routine vaccine reminder calls for adolescent vaccines, and only 5 practices (3 intervention) used automated telephone calls to remind families of upcoming, preventive care visits that were already scheduled.

• Download figure
• Open in new tab
• Download powerpoint

FIGURE 1

Randomization of study subjects. Girls were randomly assigned as they became eligible during the study period. Adolescents vaccinated at family planning visits were excluded to protect confidentiality.

Study Design and Patient Population

The 22 participating primary care practices were first randomized at the practice-level to EHR-based clinician-focused vaccine alerts, education, and audit and feedback or to no practice-level intervention. Nested within this design was a patient-level randomized intervention of automated educational reminder calls. The 1-year intervention began on May 10, 2010.

The study population included all girls 11 through 17 years of age due for at least 1 dose of the HPV vaccine during the study period (Fig 1). To focus on adolescents actively cared for at study practices, each subject was required to have had a preventive visit within 15 months of randomization. Although EHR-based alerts appeared for girls who had not had such a visit and were due for the HPV vaccine, they were not included in the study population.

Clinician-Focused Intervention

The clinician-focused intervention consisted of 3 components: (1) EHR-based alerts for all routine adolescent vaccinations programmed to appear prominently whenever any patient encounter at an intervention practice was opened within the EHR (Supplemental Fig 3),^24,25 (2) a 1-hour presentation delivered in person or online to introduce the intervention, provide site-specific data on HPV vaccination rates derived from the EHR, and present evidenced-based information on adolescent vaccine safety, efficacy, and strategies for overcoming barriers to vaccine receipt,³¹ and (3) 3 quarterly performance feedback reports generated from EHR data and delivered by a research assistant including individual, practice, and network rates of captured immunization opportunities for adolescent office visits (Supplemental Fig 4). The EHR-based alerts offered suggestions but required no action or documentation on the part of the clinician. Control practices received no EHR-based alerts for adolescent vaccines, no education, and no feedback on adolescent vaccination rates.

Family-Focused Intervention

The family-focused intervention consisted of 3 distinct types of automated telephone calls based on an EHR-generated roster and delivered by an outside vendor (Televox, Mobile, AL; Supplemental Table 6). (1) Intervention subjects with scheduled well-visit appointments and study vaccines due received reminder calls 2 business days before the appointment; (2) those who had not had a well visit within the past 10 months but were due for study vaccines and did not have a well visit scheduled in the future received up to 2 reminder calls to schedule an appointment; and (3) those due for dose 2 or 3 of HPV vaccine received a reminder call to schedule an appointment with a second reminder call 1 month later if needed. Each call listed the vaccines due, emphasized that vaccine receipt was recommended by the adolescent’s clinician, and referred families to an Internet site that linked to educational materials on adolescent vaccination from the CHOP Vaccine Education Center (http://www.chop.edu/service/vaccine-education-center/home.html; Supplemental Fig 5). The study Internet site was set up outside the Vaccine Education Center so that use tied to the study could be independently tracked.

Randomization

For the clinician-focused intervention, we first stratified the 22 sites into urban resident teaching versus other practices. We then sorted practices within these groups by their baseline rate of HPV #1 vaccination and the randomization process alternatively assigned practices to intervention and control groups to ensure that baseline rates were comparable between the 2 groups. We used systematic random sampling to ensure good balance between treatment and control groups. For the family-focused intervention, we randomly assigned subjects within each of 2 age categories (11–13 or 14–17 years) within the 22 practices by using randomly permuted blocks with unequal block sizes to ensure both blinded allocation and balanced assignment within each practice. Patient-level randomization was stratified by age, with categories chosen based on local vaccination patterns, since intention to vaccinate and actual vaccination rates are higher among older adolescents.^6,8,9,15 A statistician (Dr Localio) generated the allocation sequence and implemented the randomization.

Outcomes

The outcomes were HPV vaccination rates (the cumulative incidence of vaccination) and time to vaccine receipt. Vaccination rates were measured separately as the proportion of the population eligible for HPV vaccine dose 1 (HPV #1), HPV vaccine dose 2 (HPV #2), or HPV vaccine dose 3 (HPV #3) who received the vaccine during the study period. The first eligible day was the 11th birthday, or, for HPV #2 and HPV #3, the date of eligibility based on previous dose receipt. Follow-up time began at randomization among those >11 years of age who had not been vaccinated before the study start, or on the date of eligibility for those who became eligible during the study period. Follow-up ended with receipt of vaccine, attendance at a family planning visit, or the end of the study. A family planning visit censored subsequent observations because these visits are confidential, and receiving a telephone call might disclose the confidential visit. For HPV #2 and HPV #3, we measured time until 50% of the study population had received the vaccine dose. Because no more than 25% of children received HPV #1, we measured time to 15% complete as the outcome.

Covariates

We collected data from the EHR on demographic and clinical characteristics of study participants associated with HPV vaccine receipt (Table 1). Vaccine refusal was measured based on documentation by the clinician in a patient’s problem list, a standard approach at study practices to document families refusing multiple vaccines.

Statistical Analysis

Separately for each HPV dose, we compared the time of eligibility and vaccination and constructed Kaplan-Meier plots revealing overall vaccination rates among eligible subjects over time. To adjust for possible differences across sites in patient characteristics not balanced by randomization, we implemented Cox proportional hazard regression models accounting for the clustered design and including covariates. Standardized Cox regression by using weights equal to the inverse of the probability of treatment assignment for each patient given her individual characteristics³² was used to generate standardized estimates of the cumulative probability of receiving a vaccination and time to vaccine receipt. We confirmed that assumptions of these models were met. We report bias-corrected bootstrap confidence intervals (CIs; from 999 samples) for these estimates and their differences, again accounting for the clustered design.³³

Using standard methodology,³⁴ we next calculated the incremental cost of vaccinating each additional girl based on study arm, accounting for the fixed costs of programming the clinician-focused alerts, generating the rosters for the family calls and delivering clinician education and feedback reports, and the variable costs of each additional telephone call. Fixed costs were spread across 3 years, providing a conservative estimate of true costs because the costs of health information technology interventions are generally recovered over a longer time period.

Data were complete on all variables used in the analysis. The CHOP Institutional Review Board approved the study, and the requirement for consent from individual girls/families and clinicians was waived.