Abstract
BACKGROUND AND OBJECTIVES: Human papillomavirus (HPV) vaccination rates lag behind vaccination rates for other adolescent vaccines; a bundled intervention may improve HPV vaccination rates. Our objective is to evaluate the impact of quality improvement (QI) training plus a bundled practice-based intervention (provider prompts plus communication skills training plus performance feedback) on improving HPV vaccinations in pediatric resident continuity clinics.
METHODS: Staff and providers in 8 resident clinics participated in a 12-month QI study. The intervention included training to strengthen provider communication about the HPV vaccine. Clinics also implemented provider prompts, received monthly performance feedback, and participated in learning collaborative calls. The primary outcome measure was eligible visits with vaccination divided by vaccine-eligible visits (captured HPV vaccination opportunities). Practices performed chart audits that were fed into monthly performance feedback on captured HPV vaccination opportunities. We used conditional logistic regression (conditioning on practice) to assess captured vaccination opportunities, with the time period of the study (before and after the QI intervention) as the independent variable.
RESULTS: Overall, captured opportunities for HPV vaccination increased by 16.4 percentage points, from 46.9% to 63.3%. Special cause was demonstrated by centerline shift, with 8 consecutive points above the preintervention mean. On adjusted analyses, patients were more likely to receive a vaccine during, versus before, the intervention (odds ratio: 1.87; 95% confidence interval: 1.54–2.28). Captured HPV vaccination rates improved at both well-child and other visits (by 11.7 and 13.0 percentage points, respectively).
CONCLUSIONS: A bundled intervention of provider prompts and training in communication skills plus performance feedback increased captured opportunities for HPV vaccination.
- CORNET —
- Continuity Research Network
- EHR —
- electronic health record
- HPV —
- human papillomavirus
- LC —
- learning collaborative
- MO —
- missed opportunity
- P-chart —
- process control chart
- PDSA —
- Plan-Do-Study-Act
- QI —
- quality improvement
- VIS —
- vaccine information statement
Most cases of cervical cancer1 and >60% of oropharyngeal cancers are related to human papillomavirus (HPV) infection.2 Despite the vaccine having been recommended for adolescent girls since 20073 and for boys since 2011,4 HPV vaccination rates in the United States lag behind those for other adolescent vaccines. The Centers for Disease Control and Prevention, the President’s Cancer Panel from 2012 to 2013, and conductors of systematic reviews5–7 have recommended reducing missed opportunities (MOs) to administer HPV vaccines to increase HPV vaccination coverage.8,9
Several strategies might reduce MOs. Provider prompts is 1 strategy, yet few researchers have evaluated prompts for the HPV vaccine. One randomized clinical trial revealed that prompts combined with practice-based performance feedback reduced MOs and improved HPV vaccination rates.10,11 However, a second randomized clinical trial12 revealed that prompts alone reduced MOs slightly in 1 of 2 practice networks only and failed to raise HPV vaccination rates substantially. It is implied in these results that prompts alone may not be sufficient to raise vaccination rates. A second strategy relates to the finding that parents often report that the HPV vaccine was not recommended by their adolescent’s physician.9 Thus, an additional potential strategy is to strengthen provider recommendations.8 A third possible intervention is performance feedback for providers. Performance feedback can change provider behavior for many health services,13,14 including vaccination,11,15 because physicians do not accurately evaluate their own performance.16 Its effectiveness depends on how the feedback is provided; feedback is more effective when provided more than once, in multiple formats, and when it includes a goal.14 Therefore, in this quality improvement (QI) study, we combine provider prompts, training in giving a strong recommendation for the HPV vaccine, and performance feedback, with a goal of reducing MOs for HPV vaccination.
We conducted a QI study over a 12-month period in 8 practices from a national network of pediatric resident continuity clinics that are based at academic medical centers. Whereas much of the literature concerns MOs, our outcome was captured opportunities (eligible visits with HPV vaccination divided by HPV vaccine–eligible visits), which are the inverse of MOs (to display improvement as a positive change). Our specific aim in the project was to increase captured opportunities for HPV vaccination by 10%. A secondary goal was to improve documentation (by 10%) of reasons why the HPV vaccine was not given when due, to better identify issues leading to under-vaccination. We also performed a qualitative analysis of monthly practice diaries in which Plan-Do-Study-Act (PDSA) cycles were documented to assess office changes that were associated with greater success in improving captured opportunities.
Methods
Context
This 12-month QI study was conducted in a national, practice-based research network that is composed of pediatric resident continuity practices (Continuity Research Network [CORNET]).17 An e-mail newsletter used to encourage participation was distributed to all 107 CORNET practices, with a goal of enrolling 12 to 14 practices overall. Eight CORNET practices volunteered to participate in the prompt intervention (5 other practices participated in a standing order intervention, described elsewhere). Maintenance of certification credit was provided for faculty as an incentive to participate, in addition to $1000 per practice to compensate for time needed for chart reviews. At the time of the baseline period, none of the practices had ways to prompt providers when an HPV vaccination was due for a patient, and vaccination was not generally recommended outside of well visits.
Interventions
Basic QI Training
Initial QI training involved four 15-minute videos18 used to introduce the following topics: the model for improvement, leading change, initiating a QI project, and QI measurement. All videos focused on practice improvement in primary care.
Strong Provider Recommendation
Participants watched a 30-minute webinar on “Making a Strong Recommendation About HPV Vaccine to Parents in Primary Care,” based on slides created by the Centers for Disease Control and Prevention.19 The webinar covered communication strategies including: (1) recommending the HPV vaccine in the same way and at the same visit as other recommended adolescent vaccines, (2) providing accurate and succinct answers to frequently asked questions about the vaccine, and (3) training the entire office team to feel comfortable talking about the HPV vaccine to provide consistent messages supporting HPV vaccination.
Provider Prompts
Office staff members implemented prompts from nurses to remind the providers to give the HPV vaccine to adolescents who were due at any type of visit. Because it was already common practice to review vaccinations at well visits, the prompt intervention targeted acute and chronic care visits. Nurses or medical assistants were expected to review the vaccination record before or at every visit for any adolescent and to prompt the provider to order the vaccine. The prompt mechanisms varied by practice (eg, Vaccine Information Statement [VIS] forms placed on the clinician’s desk in the examination room; a laminated, erasable sign that was used to indicate which vaccines were due; a highlighted patient schedule for the day; printed immunization registry data). Some practices also used electronic health record (EHR) decision support in addition to the nurse prompt, but this was not the sole cue at any site. In addition, practices used visual cues (ie, door signs and posters) to raise awareness of the project for staff, clinicians, and parents.
Performance Feedback
Practices performed monthly chart reviews (10 charts per month)20 for adolescents aged 11 to 17 years who were due for an HPV vaccine at the visit. Practices received monthly feedback reports with run charts describing the overall (across 8 sites) and site-specific progress for HPV vaccine captured opportunities; they were asked to share these reports with providers and staff. Each practice also filled out a diary, describing specific changes the practice worked on during each month (using PDSA cycles).
Monthly Learning Collaborative Telephone Calls
We conducted monthly calls via webinar with intervention practices to teach improvement science, drive PDSA cycles, share practice run charts, and discuss barriers and successes with strong provider recommendations as well as prompt implementation. Learning collaborative (LC) faculty provided coaching to address obstacles and encourage those meeting their objectives.
Improved Documentation
Practices also worked on improving documentation if a vaccine was refused; they edited their EHR note templates to encourage documentation. When possible, they added the following choices to document when a patient did not get an HPV vaccine when due: refused, referred to later date, or unable to obtain consent.
Practice Teams
In addition to nurses, medical assistants, and pediatric residents, each practice team included at least 1 QI lead who was a continuity clinic attending physician. At some sites, receptionists were included for specific parts of the intervention (ie, scheduling follow-up appointments). The practice teams met monthly to review data and plan PDSA cycles. They communicated with other providers in the practice via e-mail and educational sessions for staff and residents.
Academic Pediatric Association Core Team
The project was led by individuals with expertise in immunization delivery (C.A., C.M.R., S.G.H., S.J.S., P.G.S.), QI (K.J.M., W.S.), and CORNET directors and staff (P.M.D., J.R.S., N.D.). Two immunization delivery experts (C.M.R., S.J.S.) led the monthly LC calls.
Measures
The target population was all adolescents aged 11 to 17 years who had a visit to a participating practice during the 12-month intervention period (baseline data were for those with visits in the 12 months before intervention). Visits were considered vaccine-eligible if the adolescent was eligible for the HPV vaccine at the visit on the basis of the 2013 Advisory Committee on Immunization Practices guidelines21; that is, HPV dose 1 if none previously, HPV dose 2 if >30 days from HPV dose 1, and HPV dose 3 if >24 weeks from HPV dose 1 and >12 weeks from HPV dose 2.
The primary outcome measure was captured HPV vaccination opportunities (the number of eligible visits with vaccination divided by the number of vaccine-eligible visits). Practices gathered retrospective data for the 12 months before the intervention (10 charts per month) and monthly during the intervention period by using randomly selected charts to report on patient characteristics, visit types, captured opportunities, and the reasons patients were not vaccinated if due (process measure). Reasons were coded as refused, deferred to later date, not mentioned and/or unclear (missing data), or other.
Analysis
We assessed (1) captured opportunities for HPV vaccination by practice for all visits and by visit type and (2) reasons patients were not immunized if due. We compared the baseline period to intervention periods by using descriptive statistics and Pearson’s χ2 test. We also assessed the odds of HPV vaccination at the index visit using conditional logistic regression and conditioning on practice, with the time period of the study as the independent variable.
The primary outcome measure was plotted on a statistical process control chart (P-chart) by using Microsoft Excel QI Macros. Upper and lower control limits were set at 3 σ. Special cause (evidence of a change in the system) was noted when 8 consecutive data points were above the preintervention mean.
One researcher (C.M.R.) reviewed the monthly practice diaries to examine common themes and better understand the relationship between practice changes and improvement in captured opportunities.
Ethical Considerations
The Research Subjects Review Board of the University of Rochester approved the project; the 8 CORNET sites obtained institution-specific institutional review board approval.
Results
In Table 1, we show patient demographics in the 8 continuity clinics. Most (7 out of 8) practices were hospital-based, 5 were in urban settings, and all participated in the Vaccines for Children program; most patients were publicly insured.
Practice Demographics
As shown in the P-chart of monthly chart reviews (Fig 1), captured opportunities for HPV vaccination increased by 16.4 percentage points (from 46.9% to 63.3%) overall. Special cause was demonstrated by centerline shift after the intervention.
P-chart of captured opportunities for HPV vaccine, monthly chart reviews. CL, control limit; LCL, lower control limit; UCL, upper control limit.
Comparing the intervention period to the baseline period, we found that adolescents were more likely to receive a vaccine at an eligible visit during the intervention than before the intervention (odds ratio: 1.87; 95% confidence interval: 1.54–2.28). HPV captured vaccination rates improved at both well child (from 65% to 77%) and other visits (from 27% to 40%), with P < .001. Improvement in captured opportunities varied by practice, from 2.5 to 30 percentage points (Fig 2). Practices with the lowest initial rates made the least improvement; practices with the middle range at baseline made the most improvement. Documentation of refusals and deferring vaccination increased from 9.3% to 17.1% and from 5% to 12.8%, respectively, whereas no documentation for an MO declined from 80.2% to 65.5%, with P < .001 overall.
Captured opportunities for HPV vaccination by practice.
Monthly diaries were used to describe the changes in the prompt intervention, barriers encountered, parent responses, and lessons learned (Table 2). Barriers included nursing shortages, staff changes, and decreased response to the prompts at busy periods. A minority of parents continued to be resistant to vaccination, and safety was a concern that was raised by several parents. Lessons learned from successful practices included sending frequent communications about the project and meeting frequently about practice change.
Qualitative Responses From Monthly Diaries
Regarding a strong provider recommendation, providers noted that staff recommendations were not as strong as expected at baseline but appeared to improve because of the intervention. Practice comments toward the end of the study were as follows:
The staff continues to get resistance from some parents. Overall seems less over the past 6 months. The 2 areas for this are: (1) my child is not sexually active yet, “I’ll wait.” and (2) The thought that the HPV is only for girls to prevent cervical cancer only; there is a hesitancy to give it to their sons. Some staff have stepped up their education and overall approach to the vaccine. I believe this change in staff has led to overall less resistance from the parents.
[We have] much less hesitancy or resistance than in the earlier part of the year. It now feels like the HPV is automatic, not just an add-on.
Practices with little improvement (<5%) lacked consensus for the QI activity from staff and providers. In addition, more complicated interventions (such as having the nurse place an LED light on the computer) were difficult to sustain. The simplest interventions, such as putting the VIS sheet on fluorescent paper, led to greater improvement (>25%). In addition, successful practices made it a priority to update new staff and to review monthly data with the QI team.
Discussion
In this QI study in pediatric resident continuity clinics, we found that a bundled intervention consisting of a combination of provider prompts from nurses and EHRs, provider and staff training in giving a strong recommendation for the vaccine, and performance feedback to providers increased rates of captured opportunities for HPV vaccination by 16 percentage points. However, there was great variation in improvement by practice, ranging from 2.5 to 30 percentage points. In addition, absence of documentation for MOs declined from 80% to 66%.
The range of improvement for practices was broad and provides lessons for practice improvement. Practices with the greatest increases in captured opportunities used the following approaches: frequent communications for staff and residents, changing EHR note templates to improve documentation, using multimodal prompts (paper, VIS, EHR), and having a nurse champion who reviewed medical records ahead of time. By improving documentation, practices have a better understanding of causes of under-vaccination. A previous study that was focused solely on EHR prompts without other interventions was not effective.12 This may be because of “prompt fatigue,”22,23 indicating the necessity to use multimodal cues. However, prompts need to be simple to be sustainable. For example, a fluorescent VIS sheet worked well as a prompt, whereas stamping the immunization registry printout had an intermediate effect. We also believe that the addition of performance feedback to prompts contributed to the large effects. In addition, practice champions can improve adherence to guidelines, including hand hygiene,24 vaccine delivery,25 and infection prevention.26 Finally, the success of our intervention supports other studies in which researchers have found that multicomponent interventions are more effective27,28 than single-component ones.
Two practices with the lowest baseline rates of captured opportunities experienced the least improvement. In 1 case, the barriers were potentially identifiable a priori, whereas in the other practice they were not. In 1 practice, the project champions had trouble convincing all providers and nurses in the practice to support vaccinating at illness visits. Practice engagement is a critical component of successful QI initiatives.29 It is important to involve the entire office staff in the intervention30 and to engage leadership in practice change for it to be successful.31 A second practice experienced major staff turnover; hence, the intervention became a low priority. Such real-life factors can dramatically diminish practices’ abilities to improve the delivery of preventive services.32 Because primary care practices face a multitude of barriers to practice improvement,33 a potential target for future research is to identify factors that predict the likelihood of improvement from modifiable practice features.
At the other extreme, the highest performing practice at the baseline period also experienced a low rate of improvement in captured opportunities. At the high range of performance, there is minimal room for improvement, and alternative strategies may be needed to reach the remaining few MOs. This is consistent with other studies that have revealed that high-performing practices make fewer advances than midrange performers in QI interventions.34,35
The highest rate of captured opportunities in any intervention practice was 84%, with an average of 63%. We believe rates of captured opportunities are unlikely to reach 100% for HPV vaccination in most practices because of several factors. First, parent delay and refusal are likely to contribute to ∼20% of HPV under-vaccination.36 Although a strong recommendation can impact vaccine refusals, as shown by a 12% increase in captured opportunities at well visits, other strategies are needed to reach the most vaccine-hesitant parents. Second, a few patients will be too sick to be vaccinated. Lastly, some days are likely to be too busy to allow time for nurses to assess vaccination status or to draw up and administer vaccines during acute visits. At busy times in the practice, such as during influenza season, the staff had particular difficulty reviewing vaccination records from multiple sources, and, consequently, some practices noted a drop in nurse prompts. During times of stress, practices may need to adjust the intervention.29
One strength of this study was the multidisciplinary nature of the practice teams, which included attending physicians, residents, nurses, and staff. Collaborating with staff from different areas in a practice has been shown to enhance teamwork and communication and counters the “silo effect.”37 In particular, training residents in both QI and in providing a strong recommendation is important because residents tend to use the skills learned during residency during their later practice.38,39 A second strength was having a multisite network of practice-based teams that allowed providers to learn from one another.40 Such LC models are now increasingly used for a variety of health topics. A third strength is potential sustainability, which was enhanced by providers earning Maintenance of Certification Part IV credit for participation; this has been shown to motivate physicians to participate in QI for HPV vaccination.41 Finally, the 8 practices involved in the intervention care for a large number of adolescents, so this intervention has the potential to impact a sizeable population.
There are several characteristics of the setting and patient population that may limit generalizability. The intervention was performed in pediatric resident continuity clinics serving mostly low-income populations; findings may not be generalizable to other settings. Academic practices may be particularly motivated to implement newer guidelines and are expected to teach current recommendations. On the other hand, these large practices with rotating residents are complex environments in which to implement practice change. There may be selection bias in practices volunteering to participate. The practices received a modest incentive for performing the randomly selected chart reviews. Also, the research and LC faculty leads were supported by grant funding. To scale up this intervention, we would need to integrate such core funding into local, state, or national QI efforts.
Conclusions
In this QI study, we demonstrate that a bundled intervention consisting of multimodal provider prompts combined with training on the delivery of a strong provider recommendation for HPV vaccination and performance feedback to providers can substantially increase captured opportunities for HPV vaccination in pediatric resident continuity clinics. For best results, practices should have an engaged champion, communicate frequently with the practice team, ensure that all providers understand and support the initiative, and incorporate ongoing performance feedback. If scaled up, such a QI intervention might help raise HPV vaccination rates across the United States.
Acknowledgments
We appreciate the collaboration of the physicians and staff at the participating primary care practices. We thank Beth King for her assistance with this study.
Footnotes
- Accepted November 13, 2017.
- Address correspondence to Cynthia M. Rand, MD, MPH, Department of Pediatrics, University of Rochester School of Medicine and Dentistry, PO Box 777, 601 Elmwood Ave, Rochester, NY 14642. E-mail: cynthia_rand{at}urmc.rochester.edu
FINANCIAL DISCLOSURE: During this study, Dr Humiston was a consultant to the Immunization Action Coalition; the other authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by grant 5U66IP000673 from the Centers for Disease Control and Prevention.
POTENTIAL CONFLICT OF INTEREST: Dr Humiston has consulted for the Immunization Action Coalition; the other authors have indicated they have no potential conflicts of interest to disclose.
References
- Copyright © 2018 by the American Academy of Pediatrics
In this issue
