Objective. To determine the effect of a single telephone call reminder on appointment compliance among adolescents in an inner city, hospital-based clinic.
Methods. A randomized clinical trial was conducted at the Adolescent Clinic, a part of the Ambulatory Pediatric Practice Clinic in Cleveland, Ohio, from December 1995 to November 1996. A total of 703 routine adolescent ambulatory appointments were randomized to receive either a single telephone call reminder 1 day before the appointment or to receive no reminder. A single telephone call attempt was directed primarily to the parent or guardian of the patient in the assignment group. If not available, the reminder message was left with the patient or other family member, or on the answering machine. The study variables selected included age, gender, appointment time, distance from clinic, and payment source, and for the intervention group, the recipient of the reminder telephone call. The outcome measure was the attendance rate.
Results. The intervention group (n= 347) and control group (n = 356) were well balanced for all study variables. The overall attendance rate was 49.8%. Only 204 (58.8%) of the 347 attempted intervention appointments were contacted successfully by telephone. In the attempted intervention analysis, the attendance rate of 55.6% in the intervention group (n = 347), regardless of whether subjects were successfully contacted by telephone, was 26.1% greater than the 44.1% attendance rate in the control group. In the completed intervention analysis, the attendance rate of 65.2% in the successfully contacted individuals within the intervention group (n = 204) was increased by 47.8% over that in the control group. In the univariate analysis, attendance for self-paying patients (25.4%) was worse than that for any group. In the logistic regression analysis, both the reminder telephone call intervention and the payment source were independent predictors of attendance. In the group that was called successfully (n = 204), there was no association between attendance and the recipient of the telephone call.
Conclusions. Telephone reminders are a very effective method of increasing attendance in a hospital-based adolescent clinic. The reminder is a consistently effective intervention whether the message is delivered to the patient, to the parent or other family member, or to a telephone answering machine. Adolescent patients whose visits are not covered by Medicaid or commercial insurance are least likely to attend their clinic appointments, and a telephone call reminder has no effect on this pattern.
A major limitation in health care delivery to adolescents is their poor attendance at ambulatory clinics. The failed appointment rate in this age group can be as high as 60%.1-6 A recent large national survey of 160 pediatric residency continuity clinics with children of all ages including adolescents showed an average appointment failure rate of 31%, with a range between 3% and 80%.4 Failure to keep clinic appointments is a major impediment to effective and efficient health care delivery.7,8 Several studies have demonstrated the effectiveness of telephone and mailed reminders in improving clinic attendance and immunization rates among children and adults.8-13 In addition to the goal of improving the delivery of health care to adolescents, the increasing focus on cost-cutting measures in medical care makes it mandatory to identify and implement systems that maximize the efficiency and effectiveness of the delivery of care to this group as well.
Forgetfulness is cited as one of the major reasons for appointment noncompliance among both adolescents and adults.3,6,14 This claim, at least in adults, is supported by the success of mailed and telephoned reminders in improving appointment compliance.6,15,16 The impact of reminder interventions on attendance, however, has not been examined in an exclusively adolescent medical group. The only study that tested the effect of reminders on appointment compliance among adolescents involved dental rather than medical appointments, and the results of that study concluded that compliance was not improved significantly by mailed appointment reminders.17 The purpose of this study was to examine the effect of a single telephone call reminder on appointment compliance among adolescents in an inner city, hospital-based clinic.
SUBJECTS AND METHODS
In this randomized clinical trial, we tested the effect of a single telephone call reminder on attendance at an adolescent clinic, comparing an intervention group with a control group.
Study Population and Setting
The study patients were selected from the adolescents who receive routine medical and preventive care at the Adolescent Clinic, a part of the Ambulatory Pediatric Practice Clinic in Cleveland, Ohio. This teaching hospital-based clinic is directed by a full-time attending faculty physician, and is staffed by several pediatric resident physicians who rotate through Adolescent Medicine each month. The adolescent appointments involve generalized illness and injury care, physical examinations, immunizations, and contraceptive support. The clinic is conducted 1 full day and several half days each week. All patient appointments are scheduled either by patients or by their guardian, by telephone or in person, and are entered into a computerized scheduling system. There must be written consent by the parent or guardian before the adolescent's initial visit. Appointments are scheduled for either 15 or 30 minutes, and there is no overbooking of patients to compensate for no-show patients.
The study period was 1 year, extending from December 1995 through November 1996. Patients who scheduled their appointments more than 1 working day before their appointment were eligible for the study. A total of 720 scheduled patient visits were selected for the study. This represents almost 70% of all the teen-age clinic visits in the 12-month study period. Each scheduled patient appointment was randomly assigned to either the intervention group that was assigned to receive the telephone call reminder or to the control group that was assigned to receive no reminder of any kind. Because the randomization scheme was based on appointments rather than on patients, some of the patients who had more than one single scheduled visit were included in both the intervention and control groups at some time over the 12 month study period.
Intervention and Data Collection
The appointments selected for the study were identified 1 working day before the scheduled visit. A medical assistant made a single telephone call to the telephone number listed with the patient's name. The call was intended for the parent or other guardian of the patient. If that person was not available, the reminder message was delivered to the patient or other family member. If an answering machine responded to the call, a reminder message was left on the telephone machine. If there was no answer to the single telephone call or if the number was incorrect, disconnected, or otherwise out of service, no additional call attempts were made. The outcome of the telephone intervention and the identification of the recipient of the message were recorded on the schedule. At the end of the each clinic day, the appointment use status (ie, attend, no-show, or cancel) was recorded on the schedule. After the completion of the clinic day, the schedules were audited, and the study information was abstracted. Study variables were based on demographic information that is routinely updated and recorded in the patients' computerized records. The selected study variables included the patient's gender, appointment time (morning or afternoon), distance from clinic (<3 miles, between 3 and 5 miles, or >5 miles), and payment source (self-pay, commercial insurance, or Medicaid), and for the intervention group, the recipient of the reminder telephone call (parent or other family member, patient, or telephone answering machine).
The single outcome that was measured was the attendance rate, which was the percentage of scheduled appointments that were kept.
To compare the characteristics of children randomized to receive or not to receive a reminder telephone message, the mean or frequency values of the study variables of age, gender, time, distance from the clinic, and payment type were compared between the control group and the intervention groups using the Mantel Haenszel χ2for the categorical variables and the two-tailed t test for the continuous variable, age. To determine whether there was anything particularly unique about the patients who were randomized to receive telephone calls but who were not reached successfully, the mean and frequency values of the study variables in this unsuccessfully contacted group were compared with those of the successfully contacted group. Also, the attendance rate of this unsuccessfully contacted group was compared with the attendance rate in the controls by χ2 analysis. The significance level was set at 0.05 for all statistical tests.
Two major study populations were identified for the analyses. In the attempted intervention analysis, the attendance of the adolescents in the control group was compared with that of those randomized to the intervention group, regardless of the success of the intended telephone call. In the completed intervention analysis, the attendance of the patients in the control group was compared with that of those randomized to the intervention group whose households were successfully contacted. In both analyses, rate ratios with confidence intervals (CIs) were calculated by Mantel Haenszel χ2 analysis, comparing the attendance rates in the intervention and control groups. The univariate analysis tested the statistical associations between attendance and the other study variables using χ2analysis for the categorical variables and the t test for the continuous variable, age.
Using logistic regression analysis to simultaneously control for the potentially confounding study variables, adjusted odds ratios (ORs) with CIs were calculated to evaluate the effect of the telephone intervention on the odds of attending clinic appointments.
To assess whether the effectiveness of the telephone reminder message on attendance varied by recipient of the telephone call, χ2 analysis compared attendance rates by category of telephone message recipient within the completed intervention group.
We also tested the hypothesis that the patients with the most frequent appointments would have the poorest attendance, because it has been suggested that this group may simply reschedule multiple appointments because of their attendance failures.6,8 The patients were grouped by their total number of appointments over the study period, and χ2 analysis compared attendance rates by frequency of appointment.
During the 1-year study period, 720 adolescent outpatient visits were originally selected for the study. This represented almost 70% of the total teen-age clinic appointments scheduled in that period. Seventeen of the scheduled appointments were cancelled within 1 working day of the scheduled appointment. Nine of these cancellations were made by patients in the intervention group at the time of the telephone call reminder, and eight of these cancellations were made by patients in the control group who called within 24 hours of the appointment. These 17 cancellations were not included in the study; thus, 703 scheduled visits were actually included in the analysis. Over the 12-month study period, 309 patients had a single appointment, 95 patients had two appointments, and 49 patients had three or more appointments. All of the 703 appointments were accounted for by 453 patients. A total of 347 patient appointments were randomized to the intervention group, and 353 patient appointments were randomized to the control group.
Characteristics of the patients in the intervention and control groups are presented in Table 1. There were no statistically significant differences in the distribution of gender, payment source, distance from the clinic, or appointment time. The mean age of 14.8 years was comparable in the two groups. Greater than 97% of the clinic population was African-American; thus race was not included as a study variable. Greater than 70% of the adolescent clinic population was female.
Of the 347 adolescent appointments assigned randomly to the intervention group, only 204 (58.8%) were contacted successfully by telephone (Fig 1). All patients in the study provided listed telephone numbers. Of the 143 patients who were not contacted successfully by telephone, 19 (13.3%) had telephone numbers that were disconnected or out of service. This indicates that ∼5.7% of the adolescent clinic population could not be contacted by telephone. The successfully contacted group (n = 204) and the unsuccessfully contacted group (n = 143) were compared on the basis of the frequencies and mean values of the study variables. There were no statistically significant differences between the two groups in the values for gender (P = .68), source of payment (P = .64), distance from clinic (P = .35), appointment time (P = .44), or age (P = .26) (data not shown).
The overall attendance rate for the entire 703 scheduled appointments was 49.8% (Fig 2). The rates of appointment attendance and the rate ratios comparing attendance for the intervention and control groups, in the attempted intervention analysis and also in the completed intervention analysis, are presented in Tables 2 and 3, respectively. In the attempted intervention analysis (Table 2), which included all the study patients regardless of whether the telephone call reminders were received successfully, the attendance rate was significantly increased from 44.1% in the control group to 55.6% in the attempted intervention group (rate ratio [RR] = 1.26; P = .002). This represents a 26.1% increase in the attendance rate. In the completed intervention analysis (Table 3), which included the controls and only the patients in the intervention group who were contacted successfully by telephone, the increase in the attendance rate was emphasized further. Attendance was significantly increased from 44.1% in the controls to 65.2% in the group that was contacted successfully by telephone (RR = 1.36; P < .001). This represents a 47.8% increase in attendance. The attendance rate of the unsuccessfully contacted subgroup within the attempted intervention group (42.0%) was not significantly different from the attendance rate in the control group (44.1%) (P = .66) (data not shown).
A univariate analysis involving the total group of study patients (N = 703) tested for statistical associations between attendance rate and the other study variables (Table4). The self-paying patients had an attendance rate of 25.4%, which was significantly lower than that of the patients whose payment source was Medicaid (52.1%) or commercial insurance (52.3%) (P < .001). The positive telephone call reminder effect persisted when tested separately within each of the strata of payment source (Table5). Attendance was increased by >150% within the self-paying group, from 16.7% in the control group to 41.7% in the completed intervention group. Because of the relatively small number of patients in this strata, however, the statistical significance of this increase is marginal (P = .06). Attendance was increased by 82.4% in the commercial insurance group, from 43.1% in the control group to 78.6% in the completed intervention group (P < .01). In addition, among the Medicaid patients, who represent the majority of the study patients, attendance was increased by 39.6%, from 45.4% in the control group to 63.4% in the completed intervention group (P < .001).
Attendance also was weakly associated with gender; females were slightly more likely to attend clinic than were males (P = .07). Attendance rates were statistically correlated with neither the distance from the clinic (P = .54) nor the time of the visit (P = .29). The mean age was comparable in the attending and nonattending groups (P = .30).
A logistic regression model tested for the effect of telephone reminders on attendance while simultaneously controlling for the potential confounding study variables that were identified in the univariate analyses (Table 6). The telephone reminder was a statistically significant independent predictor of clinic attendance (OR = 2.30; P < .001). Source of payment also was a significant independent predictor of attendance (OR = 0.26; P < .001). The borderline statistically significant association between gender and attendance present in the univariate analysis did not persist in the adjusted model (P = .12).
Of the adolescents in the intervention group who were contacted successfully by telephone (n = 204), the recipients of the telephone call were the patient (41.1%); the parent, guardian, or other family member (37.3%); and the answering machine (21.6%). There was no statistically significant association between recipient of the telephone call and attendance (P = .68) (data not shown).
Figure 3 shows the mean attendance rates for patients when categorized by their number of appointments. The attendance rates were very comparable for the patients who made either a single appointment (47.2%) or two appointments (46.3%). The attendance rate for those patients with three or more appointments (56.9%), however, was significantly greater than that for the patients who made either one or two appointments (P = .02).
The results of this study suggest that telephone reminder messages are very effective in increasing attendance rates in a hospital-based adolescent clinic. Less than one half of the adolescents who did not receive a reminder message kept their appointments. Nearly two thirds of the patients who received a telephone call reminder kept their appointments. This reminder effect appears to have been independent of the potentially confounding influences of gender, appointment time, clinic distance, and source of payment. The only other study factor clearly associated with clinic attendance in both the univariate and multivariate analyses was source of payment; the patients whose families would have been held financially responsible for their clinic visits were clearly less likely to attend. The stratified univariate analysis and the logistic regression analysis demonstrated that the source of payment, however, had no moderating effect on the positive association between the telephone call reminders and clinic attendance. That the statistical significance of the reminder effect was less impressive among the self-paying group was most likely attributable to the small sample size in that particular strata. Finally, the telephone call reminder appears to have been effective uniformly, regardless of whether the recipient of the telephone call reminder was patient, parent, or other family member, or even the telephone answering machine.
The issue of appointment compliance is not well understood. Research in this area suggests that clinic attendance is dependent on the condition being treated, the age of the patient, and various other social and psychological factors.1,3,17 Adolescents as well as adults and children of lower socioeconomic status (SES), independent of race and ethnic group, are more likely to miss medical appointments.3,18,19 Adolescents may delay or fail to seek medical care until their symptoms are severe or incapacitating.1 They may fail to attend follow-up appointments, use emergency room services when ill, and terminate treatment prematurely.1 Reasons cited by adolescents as to why they might fail to keep scheduled appointments include forgetting the appointment, interference with school, being too busy, not liking going to the doctor, interference with a job, fear of parental knowledge of the appointment, transportation problems, financial concerns, and no longer feeling sick or in need of treatment.3 In one large survey of adolescents, >50% responded that they would fail to attend their clinic appointments if they believed their problem had resolved.3 Irwin and coworkers found that the most important predictor of compliance with first-scheduled appointments was whether the parents, rather than the patient, made the appointment.2 In a subsequent study that tested compliance with their follow-up appointments, however, the suggested positive effect of parental involvement was no longer apparent.3 Adolescents' compliance with medical and dental regimens may be related to factors such as self-esteem and health locus of control.17,20 According to the Health Belief Model, adolescent compliance with health care advice is more likely among those who feel susceptible to a health problem that they perceive as serious, and also among those who feel that their adherence to medical or dental recommendations is likely to have beneficial effects.17,20,21
Several interventional studies in large pediatric practices, generally among lower SES populations, have demonstrated the effectiveness of both telephone and mailed reminders in reducing the number of broken appointments.9-12 In a large study of children with patient demographics and practice techniques similar to those in many private practices, Quattlebaum and colleagues13 achieved an ∼48% improvement in both well-child and sick visits by means of computer-generated postcard reminders. A meta-analysis of 23 randomized trials involving adult patients over a 25-year period demonstrated that mailed reminders and telephone prompts were consistently useful in reducing broken appointments.8 Computer-generated telephone reminders have been a particularly simple and cost-effective method of increasing childhood immunization visits in inner city children.9-12 In contrast to the generally positive impact of reminders on clinic attendance in the studies that have been published, a large survey of 160 hospital-based pediatric residency continuity clinics found a statistically significant increase in appointment compliance in the few programs that used computerized telephone message reminders only in the univariate analysis. In the multivariate analysis, which controlled for multiple study factors, payer mix was the only independent predictor of attendance.4 The only study that tested the effect of reminders on appointment compliance exclusively among adolescents involved dental rather than medical appointments, and the results of that study found that compliance was not improved significantly by mailed appointment reminders.17
There are several issues to consider when interpreting the results of this study. Perhaps the type of visit (urgent, routine physical examination, follow-up), not measured in this study, may have been a potentially confounding variable if it had been associated with both attendance and the study intervention. The severity of the child's illness, or whatever the patient's perception of the urgency of the visit, may well have been positively correlated with attendance,3,21 whereas the telephone reminder effect might have been attenuated in the more severely ill group that would have attended clinic whether or not they had received a reminder. If forgetfulness is a significant factor affecting appointment compliance, as suggested by several studies,3,6,14 then the likelihood of attendance may well have been inversely correlated with the length of the elapsed interval between making the appointment and the visit day. One might then predict that the telephone reminder effect would have been positively associated with appointment interval. Although parental consent is necessary for their initial appointments, perhaps there would have been a decreased probability of clinic attendance in the subset of adolescents who had sought subsequent medical care in confidence, such as for contraceptive support or pregnancy testing, if their guardian had received the reminder telephone call. Other potentially important, but little explored, areas that may well be related to clinic attendance include patient satisfaction with the clinic, patient perception of the staff attitudes toward them, continuity of the providers, waiting time in the clinic, and appearance and hygiene of the clinic.4,15 Although the above factors, and others not mentioned, may affect attendance, and might have possibly influenced the impact of the study intervention on attendance, the randomization scheme should have distributed equally the potential effects between the two comparison groups. Although such possible nondifferential misclassification would not have systematically influenced the study's conclusion that telephone call reminders improve attendance rates in our adolescent clinic, it could have potentially diluted the strength of the statistical association that we determined.
The literature demonstrates a conflicting pattern of the effectiveness of reminder messages on clinic attendance. The studies vary by population characteristics, clinic settings, and clinic objectives. Because the developmental stage, perceptions of illness, and medical needs of adolescents are unique, one cannot extrapolate confidently the results of our intervention study to other age groups or even to our own larger pediatric clinic, of which the adolescent clinic is a part. Also, because the study population was a fairly homogenous, inner city, lower SES adolescent group, one cannot predict reliably whether the effectiveness of the reminder calls would have varied by strata of SES. Nevertheless, the limitations in generalizability of these results in no way dampen their value for application within our own adolescent clinic, and possibly for other adolescent clinics with similar settings, populations, and health care objectives.
The issue of confidentiality may potentially limit the use of telephone call reminders in some clinic settings. Reminder calls to the homes of adolescents who seek medical care in confidence for circumstances such as sexually transmitted disease treatment, pregnancy testing, and contraceptive support may be perceived as threatening and might actually deter the adolescent patient from even seeking medical care.
As demonstrated, 31.7% of the patients made more than one clinic appointment during the 12-month study period. Although it has been suggested that multiple appointments may represent repeat scheduling by patients who fail to keep their appointments,6,8 the significantly higher attendance rate in the patients in this study with three or more appointments does not support that hypothesis. There are probably multiple reasons to explain the positive correlation of attendance with number of appointments. This trend may be unique to adolescent clinics, where a significant proportion of female patients may attend regularly for contraceptive support. Perhaps those adolescents who make more frequent appointments represent a sicker group of patients who are more motivated to keep their appointments. The positive correlation of appointment frequency with attendance may reflect the adolescent patients' positive experience of care and support that reinforces appointment compliance. Possible factors contributing to the positive correlation between attendance and appointment frequency would be interesting to explore further. Also, if there had been a larger number of patients with repeat visits, it would have been of interest to test the study effect within patients.
With the implementation of the telephone reminder system, one might project an increase in scheduled clinic visits. Campbell et al, however, demonstrated that a higher attendance rate resulted in an increased proportion of used appointments and an overall decrease in the number of scheduled appointments.6 Interestingly, however, the decrease in scheduled appointments was actually beneficial to their clinic, because the improved efficiency of clinic time allowed additional patients to be scheduled because of increased availability of clinic appointment slots that otherwise would have been no-shows. Additional study in our clinic well may show similar trends and allow for the care of increased numbers of patients should there be that demand for adolescent outpatient care. Alternatively, the increased appointment efficiency resulting from a reminder system may actually lead to a consolidation of the entire schedule of the adolescent clinic.
Before this study, there was little enthusiasm in our clinics for the necessary personnel time and commitment required to support a telephone reminder system without any documented evidence to support its effectiveness. Clearly, the attendance in our adolescent clinic is significantly improved by reminder telephone calls. At some point, we may wish to investigate the cost benefits of an automated telephone reminder system as described in several studies.9,10,12Although the telephone reminders are clearly an effective method of improving appointment compliance, at least within our clinic, we need to continue to identify and address those issues that continue as barriers to clinic attendance and health care delivery for adolescents.
We thank Ms Lisa Thiedemann and Ms Mary Anne O'Riordan for their assistance.
- Received June 23, 1997.
- Accepted February 3, 1998.
Reprint requests to (G.O.) Department of Pediatrics, Rainbow Babies and Children's Hospital, 11100 Euclid Ave, Cleveland, OH 44106.
- CI =
- confidence interval •
- OR =
- odds ratio •
- RR =
- rate ratio •
- SES =
- socioeconomic status
- ↵Jay MS, DuRant RH. Compliance. In: McAnarney ER, Kreipe RE, Orr DP, Comerci GC. eds. Textbook of Adolescent Medicine.Philadelphia. PA: WB Saunders; 1992:206–209
- Irwin CE Jr.,
- Millstein SG,
- Ellen JM
- Neinstein LS
- Campbell JR,
- Szilagyi PG,
- Rodewald LE,
- Doane C,
- Roghmann KJ
- Quattlebaum TG,
- Darden PM,
- Sperry JB
- Nazarian LF,
- Mechaber J,
- Charney E,
- Coulter MP
- Kleinman JC,
- Gold M,
- Makuc D
- Alpert JJ
- ↵DuRant RH, Seymore C, Jay S. Adolescents' compliance with therapeutic regimens. In: Hendee WR, ed. The Health Care of Adolescents. San Francisco, CA: Josey-Bass, 1991:468–494
- Copyright © 1998 American Academy of Pediatrics