PEDIATRICS Vol. 106 No. 4 Supplement October 2000, pp. 919-923

PREVENTION IN PEDIATRIC CARE:
Impact of Appointment Reminders on Vaccination Coverage at an Urban Clinic

Matilde M. Irigoyen, MD^*, Sally Findley, PhD, Beryl Earle, RN, PhD^§, Kevin Stambaugh^§, and Roger Vaughan, DrPH

From the ^* Division of General Pediatrics, College of Physicians and Surgeons and the  Mailman School of Public Health, Columbia University, New York, New York; and the ^§ New York Presbyterian Hospital, New York, New York.

	ABSTRACT

Top Abstract Methods Results Discussion References

Objective.  To test if appointment reminders blinded to immunization status improve kept-appointment and vaccination coverage rates.

Design.  Controlled trial.

Setting.  Pediatric clinic serving a low-income community in New York City.

Intervention.  Children ages 4 through 18 months (n = 1273) scheduled sequentially for clinic appointments were systematically assigned to 1 of 4 study groups: control (n = 346); postcard (n = 314); telephone call (n = 307); and postcard and telephone call (n = 306).

Outcome Measures.  Kept-appointment and vaccination coverage rates.

Results.  Children assigned to the postcard and telephone group were 1.75 times more likely to keep their appointments than controls (95% confidence interval [CI] = 1.2, 2.5). Children who actually received the postcard and telephone reminders were 2.3 times more likely to keep an appointment than controls (95% CI = 1.4, 3.7). Children who kept appointments were 2.3 times more likely to be up-to-date with their immunizations (95% CI = 1.7, 3.2). The reminders selectively increased vaccination coverage for the subgroup of children who were not up-to-date before the appointment (² = 11.2). The cost of the reminders was $.67 for the postcard and $1.58 for the postcard and telephone. Assuming 5000 visits per year and $100 reimbursement per visit, the return on each dollar invested was $10 for the postcard and $7.28 for the postcard and telephone reminder.

Conclusions.  Appointment reminders blind to immunization status are a practical and cost-effective strategy to increase kept-appointment rates for all children, and, through this mechanism, reach and vaccinate children who are not up-to-date.appointment reminder, vaccination coverage.

Reminding parents that their children's vaccinations are due or delayed is an effective strategy for increasing vaccination coverage.^1-9 However, the logistic and financial barriers associated with creating office immunization tracking and reminder systems have deterred many providers from implementing such systems. By 1995, only 35% of pediatricians and 23% of family physicians were using a manual or a computer-based vaccination reminder/recall system.²

Providers more commonly have systems in place to remind parents of their child's upcoming appointment. These appointment reminders are blind to the child's immunization status and do not require a tracking system. Although appointment reminders are an efficient and cost-effective means of improving kept-appointment rates, few data show that they improve vaccination coverage of children at primary care practices.^10-14 If children are vaccinated at well-child visits, then increasing kept-appointment rates will increase immunization rates.

The 2 studies examining the efficacy of appointment reminders for improving vaccination coverage have shown conflicting results. At a clinic in Cleveland with volunteer families, a computerized-appointment reminder and recall intervention increased infants' kept-appointment rates and on-time immunization.¹⁵ In contrast, a randomized clinical trial conducted at a clinic in Rochester, New York, found mailed appointment reminders increased show rates for well-child visits but had no significant effect on immunization status.¹⁶

We conducted a controlled trial of appointment reminders blind to immunization status at a pediatric clinic in a low-income community in New York City. The study hypothesis was that appointment reminders improved kept-appointment and, through this mechanism, vaccination coverage rates. The study also evaluated the cost-benefit of the appointment reminders.

	METHODS

Top Abstract Methods Results Discussion References

Study Setting

From April through August 1997, we conducted a study at a hospital-affiliated pediatric clinic serving a low-income population in New York City.¹⁷ Most (85%) of the patient visits were fee-for-service Medicaid; <5% were managed care. In 1997, the kept-appointment rate at the clinic was 65% for 22 100 scheduled appointments.

Study Group Assignment

This study focuses on children ages 4 through 18 months, the ages when most immunizations are due. Each patient, sequentially listed in the appointment book, was systematically assigned to 1 of 4 study groups: control (no reminder), postcard (P), telephone call (T), or postcard and telephone call (PT). On day 1, the first patient listed was assigned to the control group and successive patients were assigned to the P group, then the T group, and then the PT group. Assignment was advanced by 1 group for each successive day, restarting with the assignment of the first patient on day 5 to the control group. Assignments were made regardless of whether telephone numbers were available. Through this systematic rotation, each study group was exposed to every weekday. On subsequent visits, children were reassigned without regard to the previous assignment. Children in the same family who had appointments for the same provider on the same morning or afternoon were assigned to the same study group (n = 26). The providers were blind to the group assignment.

The assignment of patients to a study group was made daily, 1 week before the scheduled appointment date. Postcards were mailed 1 week before the appointment date. A bilingual clerk telephoned families in the PT and T groups the weekday evening before the appointment date. Up to 3 calls were attempted to reach each family.

A total of 1273 children were assigned to 4 groups: 346 children to the control group, 314 to the P group, 307 to the T group, and 306 to the PT group. To assess whether characteristics of patients differed by study group, we collected information on gender and insurance status for a random sample of one-third of the children (n = 393).

Outcome Variables

The 2 main outcome variables were kept-appointment rate and vaccination coverage. The outcome of each appointment (kept-not kept) was recorded in a daily log at the clinic and validated against the hospital's billing system. There were 9 discrepancies between the clinic log and the billing system, and these were counted as kept appointments. In addition, children who kept the appointment but were not seen for administrative reasons (eg, they were enrolled in health maintenance organizations with other primary care providers) were considered kept-appointments (n = 5). Cancellations made on the same day of the appointment or during the telephone call the prior evening were considered not-kept appointments (n = 35).

To calculate vaccination coverage, we counted the vaccinations administered up to and including the date of the appointment. Vaccination data were abstracted from all available charts (97%) for children who did not keep appointments and a systematic sample of every third child who kept-appointment (33.9%). Vaccination coverage was estimated based on the harmonized schedule of the Advisory Committee of Immunization Practices, the American Academy of Pediatrics, and the American Academy of Family Physicians, plus a 1-month grace period. Up-to-date coverage rates are reported for the 4-3-1 series: diphtheria-pertussis-tetanus, polio, and measles-mumps-rubella. Because we were interested in how effective the reminders would be for the children who would have been the targets of a vaccination reminder system, we also contrasted the vaccination coverage for the children who were up-to-date before the appointment versus those who were not.

Statistical Analysis

The outcomes were analyzed by study group. Because this study is about reminders that are only effective if they reach the families, we analyzed appointment-keeping and vaccination coverage using intent-to-treat analysis as well as by receipt analysis (Table 1). We tested differences in kept-appointment rates by study groups with ² and odds ratios (OR) with 95% confidence intervals (CI).

In the analysis of vaccination coverage we used sampling weights to adjust for the unequal probability of being selected for vaccination coverage assessment across study groups and kept-appointment rates. The weights were the reciprocals of the proportion assessed for appointment-keeping groups. We computed the OR of keeping appointments and being up-to-date using logistic regression.

Cost Analysis

The costs of the reminders were calculated based on the staff hours expended and other charges, such as postage and telephone service. Total costs were then annualized. Revenue projections were estimated based on 5000 visits per year, the number of visits in a 1-to-2 provider practice, assuming 2 typical reimbursement scenarios, $100 and $50 per visit.

	RESULTS

Top Abstract Methods Results Discussion References

Characteristics of the Study Population

The age of the children (mean = 10.3 months; standard deviation = 4.3) did not differ significantly among study groups (F = .32; P = .81). Likewise, gender did not vary significantly by group (56.3% male, ² = 3.44; P = .33). Medicaid coverage was greater among the T and PT groups compared with the other study groups, but there was no significant difference in Medicaid coverage across all reminder groups compared with the controls (92.5% in the reminder groups vs 89.8% in the control group, ² = 2.75; P = .25). The proportion of children who were up-to-date before the appointment did not differ significantly by study group (² = 4.0; P = .27).

Not all children assigned to reminders actually received them. In the P group, 6.2% of the postcards were returned; we assumed that the rest of the postcards were received. We were not able to reach by telephone 46.6% of the households assigned to the T group. In the PT group, 53.3% of the households did not receive both the postcard and the telephone reminders.

Appointment-Keeping

Children assigned to any reminder group were significantly more likely to keep their appointments than controls (² = 13.7, P = .003; Fig 1). The highest kept-appointment rate was among the children in the PT group, who were 1.75 times more likely to keep their appointments than controls (PT 76.5%, controls 65.0%, 95% CI = 1.2, 2.5).

View larger version (27K): [in this window] [in a new window]   Fig. 1.   Kept-appointment rate of children 4 to 18 months old by assignment and receipt of reminder.

The effectiveness of the reminders on appointment-keeping was even higher among those who received them (² = 12.9, P = .005; Fig 1). Children in the PT group who received both the telephone and the postcard reminders were 2.3 times more likely to keep their appointments than controls (95% CI = 1.4, 3.7).

Children who were not up-to-date before their appointment were significantly less likely to keep their appointments than children who were up-to-date at baseline (² = 9.1; P = .003). Although the reminders improved appointment-keeping for all children regardless of the up-to-date baseline status, the appointment-keeping rate was highest among those who were up-to-date at baseline (² = 8.5; P = .004).

Vaccination Coverage

Although there were no significant differences in vaccination coverage by study group, there were significant differences in vaccination coverage associated with appointment-keeping and baseline up-to-date status (Table 1).

Overall, vaccination coverage rates averaged 84.1% and did not differ significantly among the control and reminder groups (² = 2.66; P = .45). Vaccination coverage by reminder group also did not differ significantly for children who actually received the reminders (² = 2.12; P = .55).

Vaccination coverage differed significantly by appointment-keeping response. Children who kept appointments were 2.3 times more likely to be up-to-date than children who missed appointments (95% CI = 1.7, 3.2).

The reminders were significantly effective at increasing vaccination coverage for the subgroup of children who were not up-to-date at baseline (² = 11.2; P = .011). In the control group, only 9.1% of the children who were not up-to-date at baseline became up-to-date at their visit. In the PT group there was a minimal intervention effect, with only 6.8% of the children who were not up-to-date at baseline becoming up-to-date. But the reminders significantly increased vaccination coverage in the P and the T groups. For the P group, coverage rose to 26.3% (OR = 3.6; CI = 1.3, 10.1), and for the T group, coverage rose to 22.4%. For the subset of children who were not up-to-date at baseline, the postcard and the telephone reminders increased their vaccination coverage threefold compared with controls (OR = 2.9; CI = 1.1, 8.0).

Cost-Benefit Analysis

Table 2 shows the unit costs and revenues associated with each reminder. The cost of the reminders ranged from $.67 for a postcard to $1.58 for the postcard and telephone reminder. For a 5000-visit practice, the volume of a 1 to 2 provider practice, and assuming a reimbursement rate of $100 per visit, annual net revenue generated by the additional kept-appointments would range from $500 to $57 500. Net revenue per dollar invested would range from $.11 with the telephone reminder to $10 with the postcard reminder. Although the postcard reminder yielded the best return on the investment, the greatest net revenue was derived from the telephone-postcard reminder, which would yield $57 500.

	DISCUSSION

Top Abstract Methods Results Discussion References

This study shows that appointment reminders blinded to immunization status are a practical and cost-effective strategy to increase kept-appointment rates for all children, and, through this mechanism, reach and vaccinate children who are not up-to-date. Children assigned to appointment reminders were more likely to keep their appointments than controls, and children who kept their appointments were more than twice as likely to be up-to-date with their immunizations.

The most effective reminder strategy for improving kept-appointment rates was the combined telephone and postcard reminder which increased appointment-keeping twofold. Although the intent-to-treat analysis reflects the real-life situation of outdated phone numbers and addresses, the receipt analysis shows what can be achieved with an effective delivery system. Understanding how families who did not receive the reminders may differ from those who did can help improve the design of future reminder interventions.

Appointment-keeping behavior was associated with a child's baseline immunization status. Children who were not up-to-date at baseline were less likely to keep their appointments than children who were up-to-date. This finding is not surprising, because regular appointment-keeping is required to maintain up-to-date immunization status. The parents of children who were up-to-date at baseline also could be more likely to comply with recommendations from health care providers.

Blinded appointment reminders were effective in reaching the underimmunized without the need for patient-specific tracking. These results are consistent with those of Alemi et al,¹⁵ which looked at the effect of appointment reminders on vaccination coverage in a clinic serving a low-income patient population in Cleveland. Only by focusing on the children who were not up-to-date at baseline did the effect of appointment reminders on vaccination coverage became evident. Our study provides further clarification about the way that the appointment reminders work, namely to selectively increase vaccination coverage among the subgroup of children who are not up-to-date.

One advantage of the blinded reminders is that they do not require an assessment of immunization status before the reminder, making it easier and less costly to implement. Additionally, because most parents leave the clinic with a return appointment there is no need to contact the clinic before the visit, which they would have to do with vaccination reminders. Thus, appointment reminders also save clinic staff resources.

Furthermore, the appointment reminders are cost-effective. The postcard had the highest cost-benefit ratio, probably attributable to the higher receipt rate than that of telephone calls. Had we been able to reach more families by telephone, the success rate and cost-benefit ratio for that reminder could have been higher. This cost analysis does not consider additional savings accrued from improved patient flow and reduced costs associated with a lower outlay for patient recall. The calculations, likewise, do not measure the added health benefits associated with regular preventive visits and the possible benefit of improved patient satisfaction from being made to feel welcome by the practice. These additional benefits should be taken into account when translating these findings to Medicaid managed care, with its capitated payment system.

There are several limitations to this study. We assumed that postcards that were not returned were actually received; however, it is possible that some undeliverable postcards were not returned by the post office. The vaccination coverage of the children who did not keep appointments may have been underestimated if they went elsewhere for vaccinations.¹⁸ Because they had an appointment, we assumed parents intended to bring their children to the practice. However, parents may have decided to change providers since scheduling their next appointment. Finally, the study did not measure the potential impact of repeated reminders on the same child or family.

Appointment reminders may not be feasible in all practice settings. Some inner-city providers do not have appointment systems in place.¹⁹ Appointment reminders also are not applicable to families who use services exclusively on a walk-in basis. In addition, telephone reminders are of limited usefulness in communities without universal telephone service.

Because the appointment reminders are feasible and affordable in most practice settings, they can be utilized as a key strategy to improve immunization coverage. Although we did not examine other primary care markers, it is likely that appointment reminders, by increasing kept-appointment rates, can achieve higher levels for screening and other specific primary care events associated with regular visits to the provider.

Regional immunization registries may be helpful for identifying the needs of children without scheduled appointments.²⁰ If a registry is linked to a practice's appointment system, it can be used to trigger reminders that specify which immunizations are due at the next scheduled appointment. In the meantime, appointment reminders should be adopted by providers to enhance vaccination coverage for children through improved clinic attendance.

	ACKNOWLEDGMENTS

This study was supported by the Northern Manhattan Immunization Partnership, funded by the Centers for Disease Control and Prevention, Grant No. U66/CCU212961-01, Atlanta, Georgia.

	FOOTNOTES

Received for publication Oct 5, 1999; accepted Jun 13, 2000.

Reprint requests to (M.M.I.) Department of Pediatrics, Columbia University, 622 W 168th St, VC 402, New York, NY, 10032. E-mail: mi5{at}columbia.edu

	ABBREVIATIONS

P, postcard; T, telephone call; PT, postcard or telephone call; OR, odds ratio; CI, confidence interval.

	REFERENCES

Top Abstract Methods Results Discussion References

1. Ad hoc Working Group for the Development of Standards for Pediatric Immunization Practices Standards for pediatric immunization practices. JAMA. 1993; 269:1817-1822 [Abstract/Free Full Text]
2. Advisory Committee on Immunization Practices Recommendations of the Advisory Committee on Immunization Practices, the American Academy of Pediatrics, and the American Academy of Family Physicians: use of reminder and recall by vaccination providers to increase vaccination rates. MMWR Morb Mortal Wkly Rep. 1998; 47:715-717 [Medline]
3. Young SA, Halpin TJ, Johnson DA, Irvin JJ, Marks JS Effectiveness of a mailed reminder on the immunization levels of infants at high risk of failure to complete immunizations. Am J Public Health. 1980; 70:422-424 [Abstract/Free Full Text]
4. Yokley JM, Glenwick DS Increasing the immunization of preschool children; an evaluation of applied community interventions. J Appl Behav Anal. 1984; 17:313-325 [CrossRef][Medline]
5. Tollestrup K, Hubbard BB Evaluation of a follow-up system in a county health department's immunization clinic. Am J Prev Med. 1991; 7:24-28 [Medline]
6. Szilagyi PG, Rodewald LE, Savageau J, Yoos L, Doane C Improving influenza vaccination rates in children with asthma: a test of a computerized reminder system and an analysis of factors predicting vaccination compliance. Pediatrics. 1992; 90:871-875 [Abstract/Free Full Text]
7. Szilagyi PG, Rodewald LE, Humiston SG, et al. Immunization practices of pediatricians and family physicians in the United States. Pediatrics. 1994;94:517-523. See comments
8. Linkins RW, Dini EF, Watson G, Partiarca PA A randomized trial of the effectiveness of computer-generated telephone messages in increasing immunization visits among preschool children. Arch Pediatr Adolesc Med. 1994; 148:908-914 [Abstract/Free Full Text]
9. Lieu TA, Capra AM, Makol J, Black SB, Shinefield HR. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics. 1998;101(4). URL: http://www.pediatrics.org/cgi/content/full/101/4/e3
10. Nazarian LF, Mechaber J, Charney E, Coulter MP Effect of a mailed appointment reminder on appointment keeping. Pediatrics. 1974; 53:349-352 [Abstract/Free Full Text]
11. Morrison JP, Percer TL, Stone FM Improved appointment keeping with mailed reminders in a well baby clinic. Mil Med. 1976; 141:703-705 [Medline]
12. Shepard DS, Moseley ABI Mailed versus telephoned appointment reminders to reduce broken appointments in a hospital outpatient department. Med Care. 1976; 14:268-273 [CrossRef][Medline]
13. Quattlebaum TG, Darden PM, Sperry JB Effectiveness of computer-generated appointment reminders. Pediatrics. 1991; 88:801-805 [Abstract/Free Full Text]
14. Dini EF, Linkins RW, Chaney M Effectiveness of computer-generated telephone messages in increasing clinic visits. Arch Pediatr Adolesc. 1995; 149:902-905 [Abstract/Free Full Text]
15. Alemi F, Alemagno SA, Goldhagen J, et al. Computer reminders improve on-time immunization rates. Med Care. 1996;34:OS45-OS51
16. Campbell JR, Szilagyi PG, Rodewald LE, Doane C, Roghmann KJ Patient-specific reminder letters and pediatric well-child-care show rates. Clin Pediatr. 1994; 33:268-272
17. Garfield R, Greene D, Abramson D, Burkhardt S, eds. Washington Heights/Inwood: The Health of a Community II. New York, NY: The Health of the Public Program at Columbia University; 1996
18. Darden P, Taylor J, Slora E, Methodological issues in determining rates of childhood immunization in office practice. A study from Pediatric Research in Office Settings (PROS). Arch Pediatr Adolesc. 1996; 150:1027-1031 [Abstract/Free Full Text]
19. Irigoyen M, Brown RE, Cunningham N Inner city private physicians: profile and relationship with academic medical centers. Pediatr Res. 1998; 43:319
20. Linkins R, Feikema SM Immunization registries: the cornerstone of childhood immunization in the 21st century. Pediatr Ann. 1998; 27:349-354 [Medline]