OBJECTIVE: To evaluate the effectiveness and cost of bidirectional short messaging service in increasing rates of vaccination and well child care (WCC) among adolescents.
METHODS: We included all adolescents needing a recommended adolescent vaccine (n = 4587) whose parents had a cell-phone number in 5 private and 2 safety-net pediatric practices. Adolescents were randomized to intervention (n = 2228) or control (n = 2359). Parents in the intervention group received up to 3 personalized short messaging services with response options 1 (clinic will call to schedule), 2 (parent will call clinic), or STOP (no further short messaging service). Primary outcomes included completion of all needed services, WCC only, all needed vaccinations, any vaccination, and missed opportunity for vaccination.
RESULTS: Intervention patients were more likely to complete all needed services (risk ratio [RR] 1.31, 95% confidence interval [CI] 1.12–1.53), all needed vaccinations (RR 1.29, 95% CI 1.12–1.50), and any vaccination (RR 1.36, 95% CI 1.20–1.54). Seventy-five percent of control patients had a missed opportunity versus 69% of intervention (P = .002). There was not a significant difference for WCC visits. Responding that the clinic should call to schedule (“1”) was associated with the highest effect size for completion of all needed services (RR 1.89, 95% CI 1.41–2.54). Net cost ranged from $855 to $3394 per practice.
CONCLUSIONS: Bidirectional short messaging service to parents was effective at improving rates for all adolescent vaccinations and for all needed services, especially among parents who responded they desired a call from the practice.
- CI —
- confidence interval
- CIIS —
- Colorado Immunization Information System
- EHR —
- electronic health record
- HPV —
- human papillomavirus
- ITT —
- MCV —
- meningococcal conjugate vaccine
- NNT —
- number needed to treat
- RR —
- risk ratio
- Tdap —
- tetanus-diphtheria-acellular pertussis vaccine
- WCC —
- well child care
What’s Known on This Subject:
Adolescent vaccination rates lag behind other childhood vaccines. Text messaging to improve uptake of adolescent vaccines has been shown to be effective in academic centers but has not been studied in other settings.
What This Study Adds:
This study, done in 5 private and 2 safety-net practices, used a bidirectional text message as a behavioral prompt and showed text messaging was effective at increasing uptake of all adolescent vaccines. Costs were similar to other reminder/recall methods.
Childhood vaccination is commonly cited as one of the greatest public health achievements in history. However, for several vaccinations, and in certain populations, vaccination rates remain suboptimal. Among the interventions for increasing vaccination rates that have been studied, “reminder/recall” is considered one of the more effective.1 Reminders, meaning prompting a parent that a child is due for an upcoming vaccine, and recalls, meaning alerting a parent that a child is overdue, have traditionally been performed using mailed messages or telephone calls.2
Reminder/recall may be particularly useful for adolescents. Adolescents tend to visit medical providers less frequently than younger children, and are less likely to receive preventive visits.3 Also, several of the recommendations for adolescent vaccinations are less than a decade old, so parents may not be aware of these newer recommendations. One of the adolescent vaccines requires a 3-dose series, complicating delivery. Current recommendations for adolescent vaccination include a tetanus-diphtheria-acellular pertussis (Tdap) vaccine at age 11 to 12, a meningococcal conjugate vaccine (MCV) at age 11 to 12 with a booster at age 16, the 3-dose human papillomavirus (HPV) vaccine series, to be started at age 11 to 12, and a yearly influenza vaccine.4 Adolescent vaccination rates lag well behind other childhood rates, particularly for HPV vaccine.5 Although traditional reminder/recall, using mail and telephone calls, has been shown to be effective in adolescent patients,6,7 it remains underused.8 A national survey showed that only 16% of pediatricians routinely use reminder/recall, primarily because traditional methods of reminder/recall are time-consuming or expensive.9
Short messaging service, better known as text messaging, has potential advantages over traditional reminder/recall methodologies. However, the use of text messaging for reminder/recall remains understudied, particularly in private practice settings.10 Also, previous studies of reminder/recall, including text messaging, have primarily relied on unidirectional prompts, meaning that the recipient cannot respond. There is some evidence that encouraging a recipient to make a plan may serve as a behavioral prompt that may be more effective at accomplishing a desired health outcome than a simple reminder.11 Therefore, the objectives of this study were (1) to evaluate the effectiveness and cost of bidirectional text messaging in increasing rates for all adolescent vaccines and well child care (WCC), and (2) to compare results by type of response to the text message.
The study protocol was approved by the Colorado Multiple Institutional Review Board as an expedited protocol, not requiring consent. We conducted the study in 5 urban/suburban private pediatric practices and 2 safety-net practices in Colorado from September 2012 to August 2013. Practices in the greater Denver area were contacted by e-mail and/or telephone for interest in participation followed by meetings with interested practices to confirm participation. Practices were chosen purposefully based on their patient populations to represent a diverse cross section. Race/ethnicity was not routinely collected at the 5 private practices. In general, they are typical of the racial/ethnic breakdown of Colorado and accept between 10% and 35% Medicaid. The populations of the 2 safety-net practices, which were part of a larger system, were approximately 50% Hispanic, 30% non-Hispanic white, 10% African American, and 10% other, with 82% public insurance, 15% private insurance, and 3% uninsured.
All study practices participate in the Colorado Immunization Information System (CIIS), which includes all immunizations administered at participating sites, which in Colorado includes most primary care practices, school-based health centers, local public health agencies, and some pharmacies. Data in CIIS undergo routine quality checks and standard data validation checks for all new interfaces.
The intervention was developed collaboratively with the 7 practices in the study. Adolescent, parent, and provider input was solicited through a series of focus groups and key informant interviews. Based on the feedback from this process, a series of collaborative meetings were held with the intervention practices, during which the logistics of the intervention were developed.12
Study Design and Population
This was a randomized controlled trial with randomization at the patient level within each practice using random number generation (SAS 9.3; SAS Institute, Cary, NC). Providers were blinded to group allocation. The study population was a sample of adolescents aged 11 to 17 years seen at their practice at least once in the preceding 2 years. Adolescents were eligible for the study if they needed ≥1 of the targeted adolescent vaccines (Tdap, MCV4, and HPV) or WCC, defined as no WCC in the previous year. In the case in which an adolescent had ≥1 siblings who also met inclusion criteria, only 1 adolescent from the household was randomly chosen to be in the study. Parents and adolescents were blinded to which child was chosen for study inclusion; therefore, eligible nonstudy siblings received the same intervention as the study adolescent without analysis of their data.
Administrative data from the practices’ electronic billing systems, including historical data, were merged with CIIS data, and these combined data were used to determine which adolescents were eligible. Parent cell-phone numbers were determined from administrative data by scanning electronic telephone fields for cell-phone numbers (SearchBug, Encinitas, CA).
Short Messaging Service Intervention
Parents of adolescents in the intervention group were sent a text message with the following script: “We show [first name] is due for a [vaccine OR checkup or vaccine and checkup]. REPLY 1 for us to call you to schedule, 2 if you will call us, or STOP to end messages [practice name and phone number].” Replies of STOP were removed from further messages. All other replies received the following automated response: “This is an automated response. For emergencies, call 911. If you replied 1, we’ll call you soon. For questions or to schedule, please call the office.” Parents who responded in any way were removed from further text messages. The study team tracked all responses. Lists of parents who responded 1 and 2 were provided to the practices weekly. Practices agreed to call all parents who responded 1 to schedule appointments. Appointment schedules were pulled from electronic practice schedules and tracked weekly. Any patient not already removed from future texts who had an upcoming appointment was removed from further text messages. Among those who did not respond, up to 3 more text messages were sent every 2 weeks, so that the last text message was sent approximately 6 weeks after the first. Text messages were sent using MessageMedia (San Francisco, CA).
Parents of patients in the usual care group received no reminders as part of the study, and none of the practices used any reminders during the study.
Primary outcomes were completion of (1) all needed immunizations and/or WCC, (2) WCC, (3) all immunizations, (4) any immunization, and (5) missed opportunities for immunization, defined as any visit to the practice during the study period in which a vaccination was needed but not given. Secondary outcomes included results by individual vaccine and by parental response to the text message. All outcomes were assessed 6 months after the last text message.
The primary analysis was intention-to-treat (ITT) in which all individuals were analyzed within the group to which they were randomized. Per-protocol analyses were limited to those intervention parents who received at least 1 of the text messages (delivery status provided by MessageMedia). Like the intervention group, the usual care group included only parents who had access to a cell-phone. Neither randomization nor analysis was stratified by gender. Comparisons between the ITT and usual care groups and baseline demographics were conducted to determine balance between the groups. Generalized linear random effects models were used to determine differences between the ITT and per protocol (PP) groups for primary and secondary outcomes. All models were adjusted as necessary for age, insurance, and potential random effects of clinic. Relative risks were generated by using a log link with a binomial distribution in the regression model.13
The cost analysis was conducted for the 5 private practices that participated in the study, from the practice’s perspective. The safety-net clinics were not included in the cost analysis because their scheduling processes were markedly different from the private practices and therefore not comparable. The 4 cost activity areas related to the text messaging intervention were development of the text message, staff training, data management, and implementation. Practices’ ability to link data between clinical, scheduling, billing, and text messaging systems varied. Therefore, intervention costs were estimated by using 3 different hypothetical scenarios for each of the 5 clinics based on practice costs related to various data linkage capabilities: scenario 1: no electronic system linkage between clinical, scheduling, billing or text messaging and no cell-phone field in electronic data; scenario 2: system linkages between clinical, scheduling and billing but not text messaging and with a cell-phone field; scenario 3: each of the 4 systems linked and cell-phone field. Cost data were derived by identifying staff involved with text message reminder/recall and estimating their time related to these activities during the start-up and implementation period (2 months). Therefore, the length of time for the cost analysis was a 3-month period including development, training, and implementation. The study team performed timed observations at each clinic of staff performing duties, such as answering questions regarding the text messages and scheduling appointments. Job titles of the relevant staff were linked to the job titles and relevant median wages from the Bureau of Labor Statistics for the State of Colorado 2012.14 Benefits were added and calculated as 30% of wages. Cost of vaccines was not included in the analysis. Summary costs per outcome were estimated for each of the 3 scenarios.
Figure 1 shows the consolidated standards of reporting trials diagram showing the selection of the study population, which included 2228 parents randomized to intervention and 2359 to control. Details of the study population are shown in Table 1. At baseline, there were no significant differences between intervention and control groups in terms of age or up-to-date status for any of the primary outcomes.
Eighty-four percent (1877/2228) of parents who were sent a text message received the message according to the carrier. Among those parents who were sent a text message, 30% (n = 662) responded by text message. The most frequent response was that the parent would like to receive a phone call from the clinic to schedule an appointment (1, 41%), followed by that the parent would call the clinic at a later time (2, 28%), followed by STOP, meaning end future messages (22%). Nine percent of parents responded in some other way (such as with a question). Most responses (65%) occurred with the first text message.
Primary outcomes for the ITT analysis are shown in Table 2. Intervention patients were more likely to complete all needed services, meaning all needed vaccinations and WCC (risk ratio [RR] 1.31, 95% confidence interval [CI] 1.12–1.53, number needed to treat [NNT] 35), complete all needed vaccinations (RR 1.29, P = .0006, NNT 32), and complete any vaccination (RR 1.35, P < .0001, NNT 19). Seventy-five percent of patients in the control group experienced a missed opportunity for vaccination compared with 69% in the intervention group (P = .002). Analyzed alone, there was not a significant difference between intervention and control groups for WCC.
Also shown in Table 2 are results by parental response to the text messages. A response requesting a call from the clinic (1) resulted in the highest effect size. Adolescents whose parents replied STOP were less likely than the control group to receive the needed services, and this finding was significant for all study outcomes. Those who did not respond, which was most of the intervention population, still generally had favorable outcomes compared with controls.
With the exception of the MCV booster dose, there was a significant difference between intervention and control for all of the vaccine doses examined (Table 3). There was a significant difference for each HPV dose, with 16% of the intervention group who needed any dose receiving it compared with 12% among controls (P < .0001). However, most of the study population was missing their first dose of HPV vaccine, and although there was a significant difference, the overall effect on the first dose was small (11% vs 9%, P = .04). For patients for whom HPV was the only needed service (about half the study population), 19% of the intervention group compared with 15% of the control group received at least 1 dose of the vaccine (P = .002). In subgroup analyses for HPV series completion, no differences were found by practice type (private, public), age (11–12, 13–15, 16–17) or insurance type (private, public, no insurance).
Per-protocol analyses were performed for all study outcomes and were consistent with ITT analyses.
The practice costs for each of the 3 scenarios are summarized in Table 4. The greater the data linkage capability the fewer resources required for data management and quality and for conducting reminder/recall. Table 5 describes the cost per outcome for each of the 3 scenarios. The average cost per clinic over the intervention period ranged from $855 to $3394 per practice and an average cost per child of $2.64 to $10.48. As a smaller percentage of children completed WCC than vaccinations, the cost per WCC visit was higher than any of the vaccination outcomes, ranging from $61.96 to $245.94.
This study among 5 private and 2 safety-net practices shows that bidirectional text messaging to parents was effective at improving rates for all adolescent vaccinations and for all needed services, even if the parent did not respond to the text message. In addition, those who responded that they desired a call from the practice achieved higher completion rates. The impact of the intervention was consistent with other methods of reminder/recall but overall was modest.2 Missed opportunities for vaccination were high in both intervention and control groups, emphasizing the need for providers to use every opportunity to vaccinate, particularly for adolescent patients.
Text messaging has been shown to be effective at increasing vaccination coverage in previous studies, including children and adolescents, pregnant women, and in an adult travel clinic.10,15–17 One study showed a lack of effectiveness for text messaging for increasing influenza vaccination coverage in a mostly African American population.18 What all of those studies have in common is that they were performed in clinics affiliated with academic medical centers with predominantly low-income, urban populations. In the current study, we purposely sought out a broader range of practice types to better evaluate the effectiveness of text messaging in typical private practice and community health center settings.
It is not clear if the bidirectional nature of our intervention offered much advantage over a unidirectional text message. Parents who responded with an intention, though, were more likely to have their child vaccinated compared with those who did not respond. Bidirectional text messaging can be considered a form of “implementation intention,” meaning asking a person to develop a plan. Psychology research has shown that simply prompting people to develop a plan for a desired action can increase the likelihood of success.19–23 Bidirectional text messaging offers a prompt to form an implementation intention which may “nudge” a parent in the direction of the desired behavior,24 which in this case was to take their child in for a needed vaccination or well-child visit. One previous study showed a small increase in uptake of influenza vaccine by using a similar psychological prompt.11 Such low-cost psychological interventions to change behavior deserve further examination in the field of immunizations.
This study must be considered in the context of low uptake of HPV vaccine.25 For approximately half the study population, a missing dose of HPV vaccine was the only needed service, and approximately three-fourths of the study population was missing the first dose. For some of these adolescents, parents may have refused HPV vaccine at a previous visit and therefore a text message might not be expected to have much effect. Nonetheless, with low national rates for initiation and completion of the HPV vaccine series, our finding of improvement in uptake of HPV vaccine with a simple text message has important implications. Because there are so many adolescents nationally missing HPV vaccine, text messaging could offer a needed boost to low rates at a relatively low cost. However, strategies to address parental and provider attitudes are needed to substantially affect HPV vaccination coverage levels.
Although this study demonstrates that text messaging is effective at increasing vaccination rates in adolescents, it is unknown if practices will use this technology. Few providers use reminder/recall for vaccination presently.8,9 There are several reasons why one might expect practices to adopt text messaging over traditional reminder/recall modalities. Text messaging for reminder/recall has the advantage of scalability. It has the potential to be less expensive than mail or personal phone calls, and certainly can be automated much more easily. It also offers the potential advantage to parents of being more efficient than a telephone call, whether automated or personal, as parents can quickly read the message rather than listen to a telephone reminder that lasts 30 seconds or more. Further, text reminders often remain on the phone, so there is no need for a parent to write down the reminder as there would be with a telephone call. As technology advances and interfaces improve between electronic health records (EHRs) and state immunization information systems, identifying which patients are due or overdue for vaccinations or WCC should become easier for practices to do, and therefore text messaging for such reminders could be automated. Text messaging also could be used by state or local public health departments through local or regional immunization information systems, as centralized reminder/recall has been shown to be more effective than reminder/recall performed by individual practices.8
Similar to a recent cost-effectiveness study of postal and auto-dialer reminder/recall for adolescent vaccines,6 we chose to reflect the cost of text message reminder/recall from the perspective of pediatric practices. However, in contrast to that study, we specifically chose not to address revenue in our cost analysis, as cost and payment for vaccines varies greatly across states and practices.26,27 If we compare only costs between the 2 studies, they were similar, at least for our scenarios 2 and 3 ($1087–$1349 in that study versus $855–$1658). Scenario 1 reflects a “worst-case scenario,” for practices in early stages of EHR adoption or with rudimentary systems. In such cases, use of text messaging would likely be cost prohibitive. Scenario 2 reflects the situation for most practices in this study. The technology exists to use text message reminders in a more automated fashion, though, as reflected in scenario 3, and this analysis may provide a more accurate prediction of what practices in later stages of EHR adoption would spend to use text messages for reminder/recall. Some of our cost was related to the bidirectional nature of our intervention, an optional feature, and practices should consider this in interpreting these data.
This study has strengths and limitations. It is the first study performed in a mixture of public and private settings not affiliated with an academic medical center, and is therefore potentially more generalizable than previous studies. We also had a large sample size. However, it was all in 1 state in urban or suburban areas. We also were not able to fully explore the impact of bidirectionality, as we did not directly compare unidirectional with bidirectional text messaging. Also, the text messages were delivered only in English, which could have blunted the impact of the intervention in non–English speaking families. Finally, our study team did much of the work of data collection and organizing and sending the text messages, which limits assessments of sustainability.
Providers in diverse settings should consider text messaging as a viable method of reminder/recall in their adolescent patient populations, and the use of bidirectionality as a prompt for an intended action deserves further study. Although the cost of text message reminder/recall may be similar to more traditional reminder/recall modalities in the current environment, text messaging, because of its potential for automation and scalability, may represent the future of reminder/recall.
We thank all of the providers and staff at the practices involved; this study would not have happened without them.
- Accepted August 6, 2015.
- Address correspondence to Sean T. O’Leary, MD, MPH, University of Colorado, Department of Pediatrics, Mail Stop F443, 13199 E Montview Blvd, Ste 300, Aurora, CO 80045. E-mail:
Dr O’Leary conceptualized and designed the study, and drafted the initial manuscript and subsequent revisions; Ms Lee coordinated and supervised data collection, and critically reviewed the manuscript; Mr Lockhart assisted with data collection, performed much of the data collection for the cost analysis, and critically reviewed the manuscript; Dr Eisert conceptualized and designed the study, designed and supervised the cost analysis, wrote the initial draft of the cost portion of the manuscript, and critically reviewed the manuscript; Ms Barnard assisted with study design and data collection, and critically reviewed the manuscript; Ms Furniss carried out the final analyses, and reviewed and revised the manuscript; Mr Shmueli carried out the initial analyses including randomization, and reviewed and revised the manuscript; Ms Stokley conceptualized and designed the study, and critically reviewed the manuscript; Dr Dickinson conceptualized and designed the study, supervised the data analysts, and reviewed and revised the manuscript; Dr Kempe conceptualized and designed the study, oversaw all aspects of the study, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.
Portions of this work were presented at the Pediatric Academic Societies’ Annual Meeting, Vancouver, British Columbia, May 2014.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This investigation was funded by the National Center for Immunization and Respiratory Diseases and the Centers for Disease Control and Prevention (5U01IP000310-02). The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- 2.↵Jacobson Vann JC, Szilagyi P. Patient reminder and patient recall systems to improve immunization rates: Edited 2009. Cochrane Database Syst Rev 2009;(1):CD003941
- Akinsanya-Beysolow I,
- Jenkins R,
- Meissner HC,
- ACIP Childhood/Adolescent Immunization Work Group,
- Centers for Disease Control and Prevention (CDC)
- Elam-Evans LD,
- Yankey D,
- Jeyarajah J,
- et al.,
- Immunization Services Division, National Center for Immunization and Respiratory Diseases,
- Centers for Disease Control and Prevention (CDC)
- Tierney CD,
- Yusuf H,
- McMahon SR,
- et al
- Milkman KL,
- Beshears J,
- Choi JJ,
- Laibson D,
- Madrian BC
- McNutt LA,
- Wu C,
- Xue X,
- Hafner JP
- 14.US Department of Labor. Occupational Employment Statistics: May 2012 State Occupational Employment and Wage Estimates Colorado. Available at: http://www.bls.gov/oes/2012/may/oes_co.htm. Accessed February 24, 2015
- Nickerson DW,
- Rogers T
- Gollwitzer PM,
- Bayer UM
- Thaler RH,
- Sunstein CR
- Coleman MS,
- Lindley MC,
- Ekong J,
- Rodewald L
- Freed GL,
- Cowan AE,
- Gregory S,
- Clark SJ
- Copyright © 2015 by the American Academy of Pediatrics