Effectiveness and Cost of Immunization Recall at School-Based Health Centers
BACKGROUND AND OBJECTIVE: Effectiveness of recall for immunizations has not been examined in the setting of school-based health centers (SBHCs). We assessed (1) immunization rates achieved with recall among sixth-grade girls (demonstration study); (2) effectiveness of recall among sixth-grade boys (randomized controlled trial [RCT]); and (3) cost of conducting recall in SBHCs.
METHODS: During October 2008 through March 2009, in 4 Denver public SBHCs, we conducted (1) a demonstration study among 265 girls needing ≥1 recommended adolescent vaccine and (2) an RCT among 264 boys needing vaccines, with half randomized to recall and half receiving usual care. Immunization rates for recommended adolescent vaccines were assessed 6 months after recall. First dose costs were assessed by direct observation and examining invoices.
RESULTS: At the end of the demonstration study, 77% of girls had received ≥1 vaccine and 45% had received all needed adolescent vaccines. Rates of receipt among those needing each of the vaccines were 68% (160/236) for tetanus toxoid, reduced diphtheria toxoid and acellular pertussis vaccine, 57% (142/248) for quadrivalent meningococcal conjugate vaccine, and 59% (149/253) for the first human papillomavirus vaccine. At the end of the RCT, 66% of recalled boys had received ≥1 vaccine and 59% had received all study vaccines, compared with 45% and 36%, respectively, of the control group (P < .001). Cost of conducting recall ranged from $1.12 to $6.87 per recalled child immunized.
CONCLUSIONS: SBHC-based recall was effective in improving immunization rates for all adolescent vaccines, with effects sizes exceeding those achieved with younger children in practice settings.
- CI —
- confidence interval
- CIIS —
- Colorado Immunization Information System
- HPV —
- human papillomavirus vaccine
- MCV4 —
- quadrivalent meningococcal conjugate vaccine
- RCT —
- randomized controlled trial
- SBHC —
- school-based health center
- TdaP —
- tetanus toxoid, reduced diphtheria toxoid and acellular pertussis vaccine
What's Known on This Subject:
The National Vaccine Advisory Committee highlighted the importance of settings complementary to the medical home for immunization delivery among adolescents, including school-based health centers (SBHCs). The effectiveness and cost of recall for immunizations in SBHC settings has not been studied.
What This Study Adds:
SBHC-based recall was effective in improving immunization rates among adolescents, with effects sizes exceeding those achieved in practice settings. Average costs per child who was immunized ranged from $1.12 to $2.34 in 3 schools, but was $6.87 in 1 school.
The licensure of 3 new adolescent vaccines in the past 5 years1–3 has focused increased attention on the challenges of vaccination in this age group.4 Adolescents are less frequently seen for primary care than younger children,5,6 are often seen without parents,4 and are more frequently uninsured than younger children.7,8 In addition, there is a general lack of awareness of recommended vaccines for this group,4,8 and difficulties with immunization record scatter across multiple sites of care.9,10 The National Vaccine Advisory Committee recently published recommendations highlighting the importance of settings complementary to the medical home for immunization delivery among adolescents, including schools.11
The school-based health center (SBHC) model, in which comprehensive primary and preventive health care services are provided on school campuses, is an approach that has been shown to be highly effective in increasing preventive health care, including immunizations, to adolescent populations from socioeconomically disadvantaged backgrounds and those with limited access to care.12–14 The number of SBHCs in the United States has increased from 200 in 1990 to 1909 in 2009.15 As of 2008, only 6 states reported having no SBHCs.
The effectiveness and cost of conducting immunization recall in a SBHC setting has not previously been studied. We assessed, within 4 SBHCs, (1) immunization rates achievable with recall among sixth-grade girls needing tetanus toxoid, reduced diphtheria toxoid and acellular pertussis vaccine (TdaP), quadrivalent meningococcal conjugate vaccine (MCV4), or first human papillomavirus vaccine (HPV) (demonstration project); (2) effectiveness of recall among sixth-grade boys in increasing rates of receipt of TdaP and MCV4 (randomized controlled trial [RCT]); (3) rates of missed opportunities for vaccines; and (4) cost of conducting recall in a SBHC setting.
The study protocol was approved by the Colorado Multiple Institutional Review Board.
Our study included the largest 4 of the 6 Denver public middle schools with established SBHCs. All 6 schools serve predominately low-income, minority populations.12 All sites entered immunization data into the Colorado Immunization Information System (CIIS). Vaccines from the Vaccines for Children program or from Section 317 Immunization Grant Program16 supplies are available for eligible students.
We included all sixth graders who had enrolled in 1 of the SBHCs as of November 2008 (∼80% of sixth graders), and needed 1 or more of the following immunizations: TdaP, MCV4, or the first dose HPV (girls only). Seventh and eighth graders were not included because the SBHCs preferred to target the group most likely to need immunizations. Adolescents whose parents chose not to have their child participate in CIIS were excluded (<1%). The study was divided into a demonstration project among girls and an RCT among boys. All girls were included in a demonstration project rather than an RCT because of the SBHCs’ concern that an RCT might compromise their opportunity to complete the HPV series within the school year. Boys were randomly assigned within each school at the level of the individual student to either intervention or control by use of a SAS random number generation program (SAS 9.1, SAS Institute, Cary, NC).
According to Denver public school policy, at least 1 parent or guardian of all middle school children must come in person to register their child for school and, if desired, to enroll their child into the SBHC. Parents were asked to bring immunization records to registration, and, if a child was enrolled in the SBHC, these records, as well as the electronic records from the SBHC and from the CIIS, were reviewed. SBHC personnel identified needed vaccines and asked parents to consent for each vaccine, including each individual HPV dose in the series. If a parent did not consent at the time of registration, they could mail in their consents later. In addition, the intervention groups in the RCT were afforded 2 additional opportunities to consent for vaccines before the beginning of recall. Study personnel were only able to reliably track the rates of consent for each vaccine at the time of registration. At the time of registration, 82% of girls and 70% of boys enrolled in the SBHC had parental consent for at least 1 immunization.
All female and male students in the intervention group were recalled if they had parental consent to receive ≥1 needed vaccine. Students in the demonstration project or in the intervention arm of the trial were recalled up to 2 times by 1 of 3 methods: a pass sent to the student in their classroom, a phone call to the classroom, or a staff member of the health center walking into their classroom to escort them to the clinic. Outcomes were assessed 6 months after the onset of the intervention. For students randomly assigned to usual care, the SBHC immunized adolescents with all vaccines for which there was a signed consent when they presented for any type of visit.
The 2 major outcomes among both girls and boys were (1) the proportion receiving ≥1 needed study vaccine, including Tdap, MCV4, or (among girls) a first dose of HPV and (2) the proportion receiving all study vaccines needed. Because of time limitations, the major outcome regarding the HPV vaccine was initiation, rather than completion of the 3-dose HPV series. However, the proportions of girls who received a second or third dose were tracked as secondary outcomes. Among boys and girls, we also calculated percentages who were consented for individual vaccines at the time of registration and who received these vaccines. Finally, within each gender group, we calculated the percentage of students who received at least 1 vaccine but did not receive all needed vaccines. The data source for immunization outcomes was CIIS. Immunizations were counted even if given outside the SBHC, because recall could have resulted in a visit to a different site for immunization.
Data Analysis of Immunization Outcomes
The study was originally powered to detect a 15% difference between control and intervention groups based on a total sample size of 400 students across all schools. Because the RCT was only conducted in boys, the total randomly assigned was 264, which provided 84% power to detect an 18% overall difference between the intervention and control groups. All analyses were based on intention to treat. Students who needed vaccines but were not consented were included in the denominator. For boys, results for both outcomes were compared between intervention and control groups by using χ2 statistics. Statistical analyses were performed by using SAS software (SAS 9.2, SAS Institute, Cary, NC).
The cost analysis focused on the costs associated with performing recall and did not include start-up costs. Personnel time included time spent by using CIIS to generate a list of students who needed immunizations, looking up students’ class schedules, setting appointment times, and notifying students to come to the clinic for shots. To estimate the time associated with recall, an observer visited each school twice to directly observe the tasks performed by SBHC employees and document the time spent. The documented times were averaged for each site. Other costs consisted of supplies necessary to performing recall (recall notices, paper supplies for tracking, copy costs) as determined from purchase invoices.
Characteristics of the schools where the 4 SBHCs are based and their student populations as well as baseline immunization rates are shown in Table 1. Among the 715 sixth-grade students enrolled in the SBHCs, 74% needed 1 or more vaccines and were included in either the demonstration project (girls, n = 265) or the RCT (boys, n = 264). Among the 265 girls, only 12 had received a first HPV, and all of these needed another vaccine; therefore, no girls were excluded from the study on the basis of having received a single HPV vaccine. Boys were randomly assigned to either usual care (n = 131) or to the recall intervention (n = 133). There was no significant difference between the 2 groups of boys when comparing the available demographic variable, mean age (intervention, 11.6 years; control, 11.6 years; P = .93).
Six months after study onset, 77% of girls had come in for at least 1 vaccine and 45% had received all study vaccines, with some variation among the 4 schools (Fig 1). As shown in Table 2, parents were more likely to initially consent for TdaP than for HPV, with intermediate levels for MCV4. Final rates of receipt of each vaccine among those who needed it were slightly higher for TdaP (68%, 95% confidence interval [CI] 61%–74%) and similar for MCV4 (57%, 95% CI 51%–64%) and HPV (59%, 95% CI 53%–65%), but differences did not reach statistical significance. Of the 253 who needed a first HPV at baseline, 25% (64/253) received a second HPV dose, and none completed the series. Among girls who received the first HPV, 43% (64/149) received a second dose.
At the end of the trial, 66% of the recalled boys overall had received at least 1 vaccine in comparison with 45% of those not recalled (P < .001, Fig 2). As shown in Table 2, rates of consent for TdaP and MCV4 at registration did not vary substantially by vaccine. Among those who needed each vaccine in the intervention group, 63% (78/124) received TdaP and 60% (80/133) received MCV4 in comparison with 37% for TdaP and 38% for MCV4 in the control groups (P < .01 for both comparisons). Fifty-nine percent of the intervention and 36% of the control groups received both TdaP and MCV4 (P < .001, Fig 3). The effect size ranged from 12 to 44 absolute percentage points among the 4 schools.
Missed Vaccines in Adolescents Who Received at Least 1 Vaccine
Among girls seen for at least 1 vaccination (n = 203), 34% (n = 68) did not receive 1 needed vaccination and an additional 8% (n = 16) missed 2 or more needed vaccinations. The percentage of girls seen who needed but did not receive each vaccine was 22% (42/191) for HPV, 24% (44/186) for MCV4, and 8% (14/174) for TdaP. Rates of consent among those who missed individual vaccines were 17% (7/42) for HPV, 66% (29/44) for MCV4, and 57% (8/14) for TdaP. Among boys in the intervention group who received a vaccine (n = 133), only 8% (n = 10) did not receive needed vaccines; 9% (8/88) missed MCV4; and 3% (2/80) missed TdaP. Rates of consent among those who missed these vaccines were 38% (3/8) for MCV4 and 50% (1/2) for TdaP.
Costs of Conducting Recall in a SBHC
As shown in Table 3, the average cost of recall per child who was immunized ranged from $1.12 to $2.34 in 3 of the schools, but was substantially higher, $6.87, in 1 school. The cost per child receiving all study vaccinations ranged from $1.53 to $2.94 in 3 schools and, in the fourth, was $10.21. The major difference between the 3 schools with similar costs and the fourth school with higher costs was related to the much greater time spent by SBHC staff in recall efforts at the fourth school.
Although reminder/recall within a practice setting has been shown to be effective in increasing immunization rates for young children,17,18 reminder/recall effectiveness for adolescent populations has not been extensively studied and has not been evaluated in the setting of a SBHC. It has been suggested that SBHCs would be ideal settings for reminder or recall efforts because they serve a “captive audience” and their patient population is finite, with minimal migration within the school year.12 Our data demonstrate that SBHC-based recall was, indeed, effective at increasing rates of adolescent immunization overall, with effect sizes that were substantially higher than most previous studies of reminder/recall in practice settings in 2 of the 4 schools. The cost of conducting recall ranged from ∼$1 to $2.50 per child receiving a vaccination in 3 of the 4 schools studied.
Inadequate access to preventive health care for adolescents is a persisting problem in the United States, especially for those from a racial or ethnic minority, those who come from families with low incomes, and those who are underinsured or uninsured.5,19–25 SBHCs were developed to improve access to primary health care, especially for children at risk for limited access, such as low-income adolescents. Previous research has shown that low-income adolescents using SBHCs have higher rates of preventive visits, lower rates of emergency department care, and higher immunization rates than low-income adolescents in the same schools who receive care at community health centers.12,14,26 This study is the first to assess the effectiveness and cost of conducting recall within a SBHC setting among a middle-school population.
The success of our recall in the SBHC setting contrasts sharply with data from Szilagyi et al27 who demonstrated minimal effectiveness of autodialer reminder-recall among low-income adolescents within a practice setting. This reflects the fact that SBHCs do not face some of the barriers practices encounter when attempting recall, especially incorrect contact information for patients, the difficulty of determining who is still an active patient in the practice, and the hurdle imposed by the need for the patient to come into the practice for vaccinations. The overall observed effect size in our RCT was higher than that seen in most trials of recall conducted within private practice settings, where median absolute changes in rates have been estimated to be 8.2%.17 The effect size was not uniform among all schools in our trial, however, because little effect was seen at SBHC 2. This appeared to be due to contamination of the control group, initially suggested by an immunization rate in the control group that was twice as high as those seen in other schools. Qualitative interviews with staff post hoc subsequently confirmed that; because of a strong immunization champion at this site, students were often being recalled without respect to their group of randomization.
Despite the high rate of receiving at least 1 vaccine among those recalled, however, the number who did not receive all study vaccines needed when seen was almost 1 in 10 among boys and >3 times this rate among girls. The gender differences likely reflect the fact that girls required more vaccines than boys. The most commonly missed vaccine was MCV4 among boys and both MCV4 and HPV among girls, in roughly equal proportions. Not obtaining consent at registration appeared to be a major factor associated with not being fully vaccinated, especially for HPV. Other than these data, we have no insight into why needed vaccines were missed, but possibilities include student refusal, uncertainty regarding receipt elsewhere, student illness, vaccine supply issues, or competing time demands. TdaP was rarely missed, presumably because Colorado has mandated since 2007 that children entering sixth grade receive Tdap.28 Of note, although rates for TdaP more than doubled within the intervention groups between registration and the end of the 6-month intervention, >30% of our study population remained unimmunized for TdaP at the end of the intervention, illustrating the difficulty of achieving coverage even with a mandate in place. The study’s focus on initiation of HPV, rather than completion of the series, the requirement for separate consents for each HPV vaccine, and limitations imposed by the duration of the school year may have contributed to failure to complete the series in any female student.
The cost of conducting recall in a SBHC setting was modest in 3 schools, but higher in the fourth. All schools were trained in the same methods and reported using all 3 recall methods, but for unknown reasons the school with the highest costs spent almost 3 times the amount of time conducting recall. Because of the variability in published methods used to conduct recall, to calculate costs and to specify ages and vaccines being targeted, it is difficult to compare our findings with previous literature. The most expensive of the previous cost-effectiveness studies of reminder/recall for routine childhood immunizations are those including a case management component or a staged intervention including tracking and outreach29,30 with reported costs as high as $12 022 per additional child immunized. In a study comparing the cost-effectiveness of more commonly employed methods of short-term reminder/recall in a health maintenance organization setting, costs per child immunized were $9.80 with use of the autodialer, $10.50 with use of a letter reminder, and $7 with use of a combination of these methods.31 The costs per child immunized that we report are, in general, substantially lower than other published cost-effectiveness studies for reminder/recall for childhood vaccines,29–32 likely because of the relatively inexpensive means of retrieving students and almost universal success in immunizing once they are retrieved.
This study has several limitations. Our data may not be generalizable to settings other than urban schools with a SBHC with the availability of Vaccines for Children and Section 317 vaccine for eligible students. Although we assessed costs, we did not fully address the sustainability of intervention because the resources to conduct recall were provided to the schools by the study. Other than tracking the percentage of children consented at registration for vaccines, we were also unable to determine why students who responded to recall did not receive all needed study immunizations. It was not possible for us to determine where vaccinations were received; therefore, some study vaccines may have been given outside of the SBHC, either because of the recall or unrelated to it. Because vaccination outside of school that was not triggered by a school recall would be expected to occur with equal frequency in both groups, this seems unlikely to introduce significant bias in the outcomes of the trial. The accuracy of the cost analyses depended on the quality of record-keeping at each school, and we did not include the costs of implementing a registry.
This study demonstrates that recall conducted in the setting of a SBHC can be effective in increasing immunization rates among adolescents, at a relatively low cost. The success of the intervention is particularly notable because it was conducted in a setting serving primarily low-income adolescents, a group that has, historically, been difficult to reach.5,23–25 Although SBHCs are not very prevalent currently, their numbers may increase in the setting of recent health legislation establishing SBHCs as sites for comprehensive primary care, which should enhance opportunities for billing and enhanced federal support. Our data add to the growing body of evidence demonstrating that SBHCs can serve an important role in increasing immunization coverage for hard-to-reach adolescents.
- Accepted February 1, 2012.
- Address correspondence to Allison Kempe, MD, MPH, Children's Outcomes Research Program, 13199 E Montview Blvd, Suite 300, Aurora, CO 80045. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This study was funded by the Centers for Disease Control and Protection, grant 3U01P000129-02W1: Evaluation of vaccination reminder/recall systems for adolescent patients.
COMPANION PAPER: A companion to this article can be found on page e1437, online at www.pediatrics.org/cgi/doi/10.1542/peds.2011-1714.
- Bilukha OO,
- Rosenstein N,
- National Center for Infectious Diseases, Centers for Disease Control and Prevention (CDC)
- Broder KR,
- Cortese MM,
- Iskander JK,
- et al.,
- Advisory Committee on Immunization Practices (ACIP)
- Markowitz LE,
- Dunne EF,
- Saraiya M,
- Lawson HW,
- Chesson H,
- Unger ER,
- Centers for Disease Control and Prevention (CDC) Advisory Committee on Immunization Practices (ACIP)
- Rand CM,
- Szilagyi PG,
- Albertin C,
- Auinger P
- Oster NV,
- McPhillips-Tangum CA,
- Averhoff F,
- Howell K
- Allison MA,
- Crane LA,
- Beaty BL,
- Davidson AJ,
- Melinkovich P,
- Kempe A
- ↵National Assembly on School-Based Health Care. School-Based Health Centers, National Census School Year 2004-05. 2007. Available from: http://www.nasbhc.org/atf/cf/%7BCD9949F2-2761-42FB-BC7A-CEE165C701D9%7D/Census2005.pdf. Accessed August 12, 2011
- ↵US Department of Health and Human Services. Immunization Grant Program (Section 317). 2011 [cited December 20, 2011]. Available from: http://www.hhs.gov/recovery/programs/cdc/immunizationgrant.html. Accessed December 20, 2011
- Ziv A,
- Boulet JR,
- Slap GB
- ↵Colorado Department of Public Health and Environment. State board of health rules pertaining to the infant immunization program, the vaccines for children program, and the immunization of students attending school. January 21, 2009 [cited March 18, 2011] (6 CCR 1009-2). Available from: http://www.cdphe.state.co.us/dc/immunization/BOHRulesMarch2009.pdf. Accessed March 18, 2011
- Rodewald LE,
- Szilagyi PG,
- Humiston SG,
- Barth R,
- Kraus R,
- Raubertas RF
- Lieu TA,
- Capra AM,
- Makol J,
- Black SB,
- Shinefield HR
- Copyright © 2012 by the American Academy of Pediatrics