Effectiveness of Decision Support for Families, Clinicians, or Both on HPV Vaccine Receipt
OBJECTIVE: To improve human papillomavirus (HPV) vaccination rates, we studied the effectiveness of targeting automated decision support to families, clinicians, or both.
METHODS: Twenty-two primary care practices were cluster-randomized to receive a 3-part clinician-focused intervention (education, electronic health record-based alerts, and audit and feedback) or none. Overall, 22 486 girls aged 11 to 17 years due for HPV vaccine dose 1, 2, or 3 were randomly assigned within each practice to receive family-focused decision support with educational telephone calls. Randomization established 4 groups: family-focused, clinician-focused, combined, and no intervention. We measured decision support effectiveness by final vaccination rates and time to vaccine receipt, standardized for covariates and limited to those having received the previous dose for HPV #2 and 3. The 1-year study began in May 2010.
RESULTS: Final vaccination rates for HPV #1, 2, and 3 were 16%, 65%, and 63% among controls. The combined intervention increased vaccination rates by 9, 8, and 13 percentage points, respectively. The control group achieved 15% vaccination for HPV #1 and 50% vaccination for HPV #2 and 3 after 318, 178, and 215 days. The combined intervention significantly accelerated vaccination by 151, 68, and 93 days. The clinician-focused intervention was more effective than the family-focused intervention for HPV #1, but less effective for HPV #2 and 3.
CONCLUSIONS: A clinician-focused intervention was most effective for initiating the HPV vaccination series, whereas a family-focused intervention promoted completion. Decision support directed at both clinicians and families most effectively promotes HPV vaccine series receipt.
- CDS —
- clinician-focused decision support
- CHOP —
- The Children's Hospital of Philadelphia
- CI —
- confidence interval
- EHR —
- electronic health record
- HPV —
- human papillomavirus
- PeRC —
- Pediatric Research Consortium
- Tdap —
- tetanus toxoid, reduced diphtheria toxoid, and acellular pertussis, adsorbed
What’s Known on This Subject:
Despite proven health benefits, human papillomavirus (HPV) vaccination rates are among the lowest of all routine immunizations. No previous large-scale trial has compared the benefit of automated decision support directed at clinicians, families, or both in any context.
What This Study Adds:
We found that a clinician-focused intervention was most effective for initiating the HPV vaccine series, whereas a family-focused intervention supported completion. Decision support directed at both clinicians and families most effectively promotes HPV vaccine series receipt.
Vaccinating children is among the highest priorities of the nation’s health care system.1 With the licensure of the human papillomavirus (HPV),2 tetanus toxoid, reduced diphtheria toxoid, and acellular pertussis, adsorbed (Tdap),3 and meningococcal conjugate vaccines4 between 2005 and 2007, efforts to promote vaccination have increasingly focused on adolescents.5 However, adolescent vaccination rates are lower than rates for early childhood immunizations, ranging from 78% for Tdap to only 35% HPV series completion for girls.6
Parental concerns, clinician beliefs and practice styles, and adolescents’ patterns of health care utilization limit receipt of these immunizations, especially for the HPV vaccine. Reluctant to immunize prepubertal girls against sexually transmitted infections and concerned about safety and efficacy, parents often delay HPV vaccination beyond the recommended starting age of 11 to 12 years.7–10 Clinicians similarly postpone recommending HPV vaccine in response to perceived parental concerns,11,12 doubts about long-term safety and efficacy,13 and inaccurate beliefs about who is at risk, leading to missed opportunities for vaccination.12,14–17 Additionally, adolescent attendance at preventive visits declines with age, limiting opportunities for vaccination.18,19
Delays in initiating HPV vaccination adversely impact girls’ health. Although infection usually clears, one-quarter of girls ages 14 to 19 years are infected with at least 1 strain of HPV, and serotypes associated with a high-risk of developing cervical, anal, and other genital cancers are common.20 In addition, the vaccine is effective only if received before infection, and 3 doses over at minimum 6 months are recommended for full protection.21,22
Recognition of these obstacles triggered calls to develop innovative systems to foster adolescent vaccine delivery.5,23 Research in this area is warranted since interventions using electronic health record (EHR)-based, clinician-focused decision support (CDS) to support early childhood and influenza vaccination have had mixed results,24,25 and no published studies of EHR-based alerts have addressed adolescent vaccination. In addition, although basic reminder calls to families have proven effective in fostering vaccination,26 only 2 studies of reminder calls for adolescent vaccination have been published, revealing mixed results.27,28 Although CDS has been defined as including alerts to clinicians or families,29 researchers have not conducted large-scale trials of automated, EHR-based decision support directed at both clinicians and families in any context. Given multiple family and clinician barriers to HPV vaccination, this strategy may better address obstacles to vaccine receipt than either family-focused or CDS alone.
To address these knowledge gaps, we conducted a cluster-randomized clinical trial to test the benefit of clinician and family directed decision support, delivered by using the EHR and telephone, on receipt of HPV vaccine for adolescent girls. To minimize contamination, the practice was the unit of randomization for the clinician-focused intervention, since each child might receive vaccines from multiple clinicians within each practice. The family-focused intervention was randomized at the individual level. We hypothesized that providing decision support either to families or clinicians would improve vaccination rates compared with no decision support, and that decision support for both clinicians and families would be more effective than either approach alone.
This study was conducted within The Children’s Hospital of Philadelphia (CHOP) Pediatric Research Consortium (PeRC), a 2-state (New Jersey and Pennsylvania), hospital-owned, primary care practice-based research network including more than 202 000 children.30 Of the 25 PeRC practices, 18 primarily suburban practices not involved in resident teaching and all 4 urban, resident teaching practices participated in the study (Fig 1). All practices use the ambulatory EHR, EpicCare (Verona, WI). Before the start of the study, we confirmed that insurance plans accepted by the CHOP Care Network covered the cost of the HPV vaccine. HPV vaccines could be received at preventive visits, acute visits, and nurse-only visits. At the study start, no practice had implemented routine vaccine reminder calls for adolescent vaccines, and only 5 practices (3 intervention) used automated telephone calls to remind families of upcoming, preventive care visits that were already scheduled.
Study Design and Patient Population
The 22 participating primary care practices were first randomized at the practice-level to EHR-based clinician-focused vaccine alerts, education, and audit and feedback or to no practice-level intervention. Nested within this design was a patient-level randomized intervention of automated educational reminder calls. The 1-year intervention began on May 10, 2010.
The study population included all girls 11 through 17 years of age due for at least 1 dose of the HPV vaccine during the study period (Fig 1). To focus on adolescents actively cared for at study practices, each subject was required to have had a preventive visit within 15 months of randomization. Although EHR-based alerts appeared for girls who had not had such a visit and were due for the HPV vaccine, they were not included in the study population.
The clinician-focused intervention consisted of 3 components: (1) EHR-based alerts for all routine adolescent vaccinations programmed to appear prominently whenever any patient encounter at an intervention practice was opened within the EHR (Supplemental Fig 3),24,25 (2) a 1-hour presentation delivered in person or online to introduce the intervention, provide site-specific data on HPV vaccination rates derived from the EHR, and present evidenced-based information on adolescent vaccine safety, efficacy, and strategies for overcoming barriers to vaccine receipt,31 and (3) 3 quarterly performance feedback reports generated from EHR data and delivered by a research assistant including individual, practice, and network rates of captured immunization opportunities for adolescent office visits (Supplemental Fig 4). The EHR-based alerts offered suggestions but required no action or documentation on the part of the clinician. Control practices received no EHR-based alerts for adolescent vaccines, no education, and no feedback on adolescent vaccination rates.
The family-focused intervention consisted of 3 distinct types of automated telephone calls based on an EHR-generated roster and delivered by an outside vendor (Televox, Mobile, AL; Supplemental Table 6). (1) Intervention subjects with scheduled well-visit appointments and study vaccines due received reminder calls 2 business days before the appointment; (2) those who had not had a well visit within the past 10 months but were due for study vaccines and did not have a well visit scheduled in the future received up to 2 reminder calls to schedule an appointment; and (3) those due for dose 2 or 3 of HPV vaccine received a reminder call to schedule an appointment with a second reminder call 1 month later if needed. Each call listed the vaccines due, emphasized that vaccine receipt was recommended by the adolescent’s clinician, and referred families to an Internet site that linked to educational materials on adolescent vaccination from the CHOP Vaccine Education Center (http://www.chop.edu/service/vaccine-education-center/home.html; Supplemental Fig 5). The study Internet site was set up outside the Vaccine Education Center so that use tied to the study could be independently tracked.
For the clinician-focused intervention, we first stratified the 22 sites into urban resident teaching versus other practices. We then sorted practices within these groups by their baseline rate of HPV #1 vaccination and the randomization process alternatively assigned practices to intervention and control groups to ensure that baseline rates were comparable between the 2 groups. We used systematic random sampling to ensure good balance between treatment and control groups. For the family-focused intervention, we randomly assigned subjects within each of 2 age categories (11–13 or 14–17 years) within the 22 practices by using randomly permuted blocks with unequal block sizes to ensure both blinded allocation and balanced assignment within each practice. Patient-level randomization was stratified by age, with categories chosen based on local vaccination patterns, since intention to vaccinate and actual vaccination rates are higher among older adolescents.6,8,9,15 A statistician (Dr Localio) generated the allocation sequence and implemented the randomization.
The outcomes were HPV vaccination rates (the cumulative incidence of vaccination) and time to vaccine receipt. Vaccination rates were measured separately as the proportion of the population eligible for HPV vaccine dose 1 (HPV #1), HPV vaccine dose 2 (HPV #2), or HPV vaccine dose 3 (HPV #3) who received the vaccine during the study period. The first eligible day was the 11th birthday, or, for HPV #2 and HPV #3, the date of eligibility based on previous dose receipt. Follow-up time began at randomization among those >11 years of age who had not been vaccinated before the study start, or on the date of eligibility for those who became eligible during the study period. Follow-up ended with receipt of vaccine, attendance at a family planning visit, or the end of the study. A family planning visit censored subsequent observations because these visits are confidential, and receiving a telephone call might disclose the confidential visit. For HPV #2 and HPV #3, we measured time until 50% of the study population had received the vaccine dose. Because no more than 25% of children received HPV #1, we measured time to 15% complete as the outcome.
We collected data from the EHR on demographic and clinical characteristics of study participants associated with HPV vaccine receipt (Table 1). Vaccine refusal was measured based on documentation by the clinician in a patient’s problem list, a standard approach at study practices to document families refusing multiple vaccines.
Separately for each HPV dose, we compared the time of eligibility and vaccination and constructed Kaplan-Meier plots revealing overall vaccination rates among eligible subjects over time. To adjust for possible differences across sites in patient characteristics not balanced by randomization, we implemented Cox proportional hazard regression models accounting for the clustered design and including covariates. Standardized Cox regression by using weights equal to the inverse of the probability of treatment assignment for each patient given her individual characteristics32 was used to generate standardized estimates of the cumulative probability of receiving a vaccination and time to vaccine receipt. We confirmed that assumptions of these models were met. We report bias-corrected bootstrap confidence intervals (CIs; from 999 samples) for these estimates and their differences, again accounting for the clustered design.33
Using standard methodology,34 we next calculated the incremental cost of vaccinating each additional girl based on study arm, accounting for the fixed costs of programming the clinician-focused alerts, generating the rosters for the family calls and delivering clinician education and feedback reports, and the variable costs of each additional telephone call. Fixed costs were spread across 3 years, providing a conservative estimate of true costs because the costs of health information technology interventions are generally recovered over a longer time period.
Data were complete on all variables used in the analysis. The CHOP Institutional Review Board approved the study, and the requirement for consent from individual girls/families and clinicians was waived.
Of 25 practices approached, 22 practices volunteered to participate yielding a total study sample of 22 486 adolescent girls (Fig 1). The characteristics and number of study participants were similar across the 4 study arms (Table 1). Seventy-nine percent of subjects had not received any doses of HPV at the study start.
We collected multiple measures to assess the success of the implementation of both interventions. During the 12-month study period, 14 534, 4608, and 4622 calls were made to girls due for HPV #1, HPV #2, and HPV #3, respectively. A total of 47% of calls were listened to for >10 seconds and 3% <10 seconds; 46% resulted in a message left on an answering machine, and 4% were not answered. In families receiving care at urban practices, calls were slightly more likely to result in no answer (9% vs 4%) or a hang up in <10 seconds (7% vs 3%). Although all calls mentioned the informational Internet site, only 154 visits to the site occurred. For the clinician educational program, 60% of clinicians attended a live session, 14% viewed the recorded session online, and 26% did not participate. Clinician participation in the training by practice ranged from 45% to 100%. EHR-based vaccine alerts for HPV occurred at a total of 36 280 visits during the intervention.
The combined clinician and family-focused intervention resulted in significantly higher rates of HPV vaccine receipt relative to no intervention (Tables 2, 3, and 4). Rates were similar in the unadjusted (Fig 2 A, B, and C) and standardized results (Tables 3 and 4). The control group had standardized final vaccination rates of 16%, 65%, and 63% among those eligible for HPV doses 1, 2, and 3, respectively. The combined intervention increased the standardized vaccination rate from 16% to 25%, from 65% to 73%, and from 63% to 76%, respectively, compared with no intervention.
Although rates of HPV #1 vaccination were significantly higher in the clinician- than in the family-focused group (24% vs 18% vaccinated), rates of HPV #2 and HPV #3 among those eligible were significantly higher in the family-focused compared with the clinician-focused group (71% vs 64% and 73% vs 67%, respectively.) Additionally, rates of HPV #1 were significantly higher in the combined intervention group compared with the family-focused group (25% vs 18%), whereas rates of HPV #2 and HPV #3 were significantly higher in the combined group compared with the clinician-focused group (73% vs 64%, and 76% vs 67%, respectively; Tables 3 and 4). The intervention performed similarly among older and younger adolescents.
Time to Vaccination
The control group reached 15% vaccination for HPV #1 and 50% vaccination for HPV #2 and 3 after 318, 178, and 215 days, respectively. Girls receiving the combined intervention, compared with neither, reached 15% vaccination for HPV #1 a mean of 151 days faster, and achieved 50% vaccination for HPV #2 and HPV #3 68 and 93 days faster, respectively (Fig 2 A, B, and C and Tables 3 and 4). The time to vaccine receipt was shorter with the clinician-focused than the family-focused intervention for HPV #1, but the reverse was true for HPV #2 and HPV #3.
Table 5 details the incremental costs of vaccinating girls in each study arm for each dose of HPV vaccine. The assumptions underlying these calculations are also listed. The incremental cost of the more effective intervention versus no intervention for each additional dose was low, $6 for CDS for HPV #1, and $10 and $6 for the family-focused intervention for doses 2 and 3, respectively. The combined intervention added $24 compared with CDS for HPV #1, and $42, and $189 compared with the family-focused decision support for HPV #2 and 3.
This randomized trial was novel in comparing the benefits of automated decision support directed at families, clinicians, or both on HPV vaccine receipt. We found that the combined clinician and family-focused decision support intervention was most effective in improving vaccination rates and shortening the time to vaccine receipt for HPV doses 1, 2, and 3. The clinician-focused intervention was more effective than the family-focused intervention for HPV dose 1, but less effective for doses 2 and 3.
By separately examining receipt of the initial and subsequent doses of HPV, this trial was designed to compare the benefit of the clinician- and family-focused intervention on vaccine initiation versus continuation. Distinguishing these effects was especially important because of the complexity of having clinicians recommend and families accept the initial vaccine dose coupled with the need for girls to subsequently complete the 3-dose series. Combining multiple evidence-based strategies, the clinician focused intervention increased vaccination rates by 8 percentage points for HPV #1, an impact larger than the median benefit of 3.8% points for vaccination reported in systematic reviews of on-screen, point of care decision support,35 or the 6% median benefit of academic detailing36 and 5% benefit of audit and feedback shown in systematic reviews including a mix of adult and pediatric-focused studies in varied clinical settings.37 In contrast, the family intervention had little impact on HPV #1. Previous research, primarily from surveys, has described the importance of clinician recommendation to vaccine receipt.11,38,39 Our trial results confirm the central role of the clinician in promoting HPV vaccine receipt and validate using CDS to do so.
In contrast to the results for the first vaccine dose, once families accept the initial vaccine dose, family-focused decision support was more effective in promoting series completion. For HPV #2 and #3, nearly all of the benefit of the intervention resulted from the family-focused decision support designed to bring girls to the office for the vaccine as soon as it was due. The impact of the clinician-focused intervention for HPV #2 and #3 was likely reduced because, although girls could receive these doses at routine preventive or acute visits with clinicians who had the benefit of point-of-care, on-screen alerts, these visits are normally infrequent for adolescent girls.18 The effectiveness of the family-focused intervention in our diverse practice network contrasts with the failure of a telephone reminder system to improve adolescent vaccination rates in an urban, underserved population with unreliable telephone numbers,27 but is consistent with a trial in 4 primarily suburban practices in which 94% of the intervention population successfully received calls as well as reminder letters.28 The results of these studies underscore the importance of reliable contact information as a prerequisite for effective family-focused intervention.
In this trial, the incremental costs per each additional girl vaccinated for the single most effective intervention (clinician-focused for HPV #1, family-focused for HPV #2 and #3) for each HPV dose were low, ranging from $6 to $10. All costs, including for the combined intervention, were substantially lower than for an immunization navigator program designed to bolster adolescent vaccination as well as preventive care, which cost $465 per additional adolescent fully vaccinated.40 The navigator study exclusively targeted urban adolescent girls and assessed the outcome of complete vaccination, which limits direct comparison. The costs in our study were somewhat higher than a school-based recall intervention for adolescent vaccines, which cost between $1 and $6 per adolescent immunized.41 However, the recall mechanism in that study involved retrieving students already in class, a captive population. Additional work, beyond the scope of this trial, is needed to determine the cost-effectiveness of the family and clinician-focused interventions.
This study had several limitations. Although the study population of adolescent girls was large and diverse, our study was confined to 1 health system. However, we were able to conduct the intervention for all eligible adolescent girls at each of the 22 sites, enhancing generalizability. Additionally, by including only girls who had a well-child visit within 15 months, we likely had a more easily contacted population than for the practices overall, potentially improving results. The 12-month duration of the trial limited our ability to assess patterns of vaccine receipt throughout adolescence and likely explains why vaccination rates for HPV #1 in all study arms were ≤25%, below the national average of 35%. In addition, the finite number of practices meant that the cluster-based, clinician-focused intervention had far less statistical power than did the nested randomization of girls within sites for the family-focused intervention. Due to the limited number urban sites available from the network, the study lacked adequate power to compare intervention success by urban versus suburban practice setting. Additionally, the family-focused intervention included 2 educational reminder calls each for HPV #2 and #3; results may not generalize to more intensive family-focused interventions. Future studies that examine the mechanisms of the intervention, including the utility of repeat calls and the impact on missed opportunities and office visits, will be helpful in optimizing our approach for HPV and other adolescent vaccines.
Focused on a highly-effective vaccine that reduces cancer risk but is, as yet, poorly adopted, this trial demonstrated that clinician- and family-focused decision support complement each other in improving vaccine delivery to adolescent girls. Given the success of this intervention, future research should be directed at understanding how automated decision support based on EHR data and delivered to clinicians via EHRs and to families via telephone, text message, e-mail, or patient portals can support the provision of evidence-based care in varied clinical contexts. Our results suggest that a focus on either one alone is likely to be inadequate to fully realize the benefits of EHR implementation for vaccine delivery.
We thank the network of primary care physicians, their patients, and families for their contribution to clinical research through the PeRC at CHOP. In addition, we recognize David Milley for technical assistance in creating the study Internet site, and Linda Tague, Deborah Nasca, and Cylestine Mercer for their help in implementing the interventions.
- Accepted February 7, 2013.
- Address correspondence to Alexander G. Fiks, MD, The Pediatric Research Consortium, The Children’s Hospital of Philadelphia, 3535 Market Suite, Room 1546, Philadelphia, PA 19104. E-mail:
Dr Fiks contributed to the conception and design of the study, acquisition of data, analysis and interpretation of data, drafted the article, and approved the final article as submitted. Dr Grundmeier contributed to the conception and design of the study, acquisition of data, analysis and interpretation of data, critically reviewed the article, and approved the final article as submitted. Ms Mayne contributed to the acquisition of data, analysis and interpretation of data, critically reviewed the article, and approved the final article as submitted. Mr Song participated in the analysis and interpretation of data, critically reviewed the article, and approved the final article as submitted. Dr Feemster contributed to the conception and design of the study, analysis and interpretation of data, critically reviewed the article, and approved the final article as submitted. Mr Karavite contributed to the analysis and interpretation of data, critically reviewed the article, and approved the final article as submitted. Mr Hughes and Mr Massey contributed to the conception and design of the study, critically reviewed the article, and approved the final article as submitted. Dr Keren contributed to the conception and design of the study, analysis and interpretation of data, critically reviewed the article, and approved the final article as submitted. Dr Bell contributed to the analysis and interpretation of data, critically reviewed the article, and approved the final article as submitted. Dr Wasserman contributed to the analysis and interpretation of data, critically reviewed the article, and approved the final article as submitted. Dr Localio had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr Localio contributed to the conception and design of the study, acquisition of data, analysis and interpretation of data, critically reviewed the article, and approved the final article as submitted.
The authors of this article are responsible for its content. No statement may be construed as the official position of the Agency for Healthcare Research and Quality of the US Department of Health and Human Services. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Eunice Kennedy Shriver National Institute of Child Health & Human Development or the National Institutes of Health.
FINANCIAL DISCLOSURE: Dr Fiks is a coinventor of the “Care Assistant” that was used to provide the clinician-focused, point of care decision support in this study. He holds no patent on the software and has earned no money from this invention. No licensing agreement exists. Dr Grundmeier is a coinventor of the Care Assistant that was used to provide the clinician-focused, point of care decision support in this study. He holds no patent on the software and has earned no money from this invention. No licensing agreement exists. Additionally, a member of Dr Grundmeier’s family received speaker’s fees from Merck. Dr Feemster received an honorarium from Pfizer, Inc for participation on an advisory board regarding changing attitudes toward vaccines among clinicians in April 2012. Additionally, she received an honorarium from Abbot Laboratories for serving as a speaker at Influenza Forum Europe in November 2012. To manage conflicts of interest, data management and analyses were overseen and results were independently reviewed and verified by one of the authors (Dr Localio) who had no conflict of interest. The other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This research was conducted by the Children’s Hospital of Philadelphia under contract to the Agency for Healthcare Research and Quality, contract number HHSA 290-07-10013, Task Order 4, Rockville, Maryland. This research was also supported by award number K23HD059919 from the Eunice Kennedy Shriver National Institute of Child Health & Human Development. Funded by the National Institutes of Health (NIH).
- ↵US Department of Health and Human Services. Office of Disease Prevention and Health Promotion. Healthy People 2020. Washington, DC. Available at: www.healthypeople.gov/2020/default.aspx. Accessed June 5, 2012.
- Markowitz LE,
- Dunne EF,
- Saraiya M,
- Lawson HW,
- Chesson H,
- Unger ER,
- Centers for Disease Control and Prevention (CDC),
- Advisory Committee on Immunization Practices (ACIP)
- Broder KR,
- Cortese MM,
- Iskander JK,
- et al.,
- Advisory Committee on Immunization Practices (ACIP)
- Bilukha OO,
- Rosenstein N,
- National Center for Infectious Diseases, Centers for Disease Control and Prevention (CDC)
- Szilagyi PG,
- Rand CM,
- McLaurin J,
- et al.,
- Working Group on Adolescent Vaccination in the Medical Home
- Waller J,
- Marlow LA,
- Wardle J
- Daley MF,
- Crane LA,
- Markowitz LE,
- et al
- Fiks AG,
- Grundmeier RW,
- Biggs LM,
- Localio AR,
- Alessandrini EA
- Fiks AG,
- Hunter KF,
- Localio AR,
- et al
- ↵Osheroff JA, Pifer EA, Teich JM, Sittig DF, Jenders JA. Improving Outcomes With Clinical Decision Support: An Implementer's Guide. Chicago, IL: Healthcare Information and Management Systems Society; 2005.
- Freed GL,
- Clark SJ,
- Butchart AT,
- Singer DC,
- Davis MM
- Davison AC,
- Hinkley DV
- Drummond MF,
- O'Brien B,
- Stoddart GL,
- Torrance GW
- Copyright © 2013 by the American Academy of Pediatrics