Delivery of Pediatric After-Hours Care by Call Centers: A Multicenter Study of Parental Perceptions and Compliance
Background. Despite the rapid growth of centralized call centers to provide after-hours triage to patients of multiple providers, little is known about the perceptions of parents regarding this type of care and their compliance with triage disposition recommendations.
Design/Methods. From August through September 1999, randomized samples of after-hours calls were selected each day from computerized records at 4 pediatric call centers at 1) Children’s Hospital, Denver, Colorado; 2) Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; 3) Rainbow Babies and Children’s Hospital, Cleveland, Ohio; and 4) All Children’s Hospital, St Petersburg, Florida. All participating call centers use the same triage software. Calls were randomly selected to yield at least 250 callers with nonurgent dispositions and 100 with urgent dispositions from each site. Telephone surveys to callers were conducted by an external survey unit 3 to 7 days after the call to the call center.
Results. Surveys were completed for 70.5% of those sampled (N = 1561). Parents indicated they were very satisfied or satisfied with aspects of care received from 92.6% (waiting time) to 99.4% (nurse courteousness) of the time. Satisfaction did not differ by site or by recommended disposition of the index call. Most parents (65.2%) reported no preference about speaking with a physician or nonphysician for after-hours care, whereas 27.7% preferred to speak with a physician. Usually speaking with a physician during office hours (odds ratio [OR]: 1.48), feeling it was important that provider knows child’s medical history (OR: 3.47), and respondent having an educational level of college graduate or higher (OR: 1.30) were significant predictors of preferring to speak with a physician. Of the 37.0% (N = 723) of parents who reported any change in their relationship with their primary provider as a result of the after-hours call center, 95.7% (N = 691) assessed the change to be positive. Reported compliance with the call center disposition recommendation was 83.3% for urgent referral, 41.0% for next day, 4.5% for visit at a later time, and 78.2% for home care. The major reason given by parents for noncompliance was reporting that they heard a different disposition (76.9% for urgent to 100% for visit at a later time).
Conclusions. Parental satisfaction with pediatric call centers was uniformly high in 4 different geographic locations, and almost all parents who reported any effect on their relationship with their primary provider assessed it as positive. Compliance with recommendations for urgent evaluation or home care was relatively high but for intermediary dispositions was low. In most cases in which noncompliance occurred, parents reported hearing a different disposition. Additional study is needed to clarify whether noncompliance, especially in cases in which an urgent recommendations was made, is attributable to poor nurse communication of the recommended disposition, parental misinterpretation, or parental difference of opinion.
In recent years, there has been dramatic growth in centralized call centers to manage after-hours calls from patients of multiple health care providers. Such centers are generally based within a local hospital or a service bureau at a geographically distant site. Conservative estimates suggest that at least 25% of all general pediatricians currently sign out their after-hours calls to call centers, and there are over 50 pediatric call centers.1 The majority of call centers employ nurses who use decision-support protocols or guidelines to aid in triage. A number of published protocol books are available,2,3,4 and at least 1 product for pediatric patients is available in a software format.5
Despite the rapid growth of call centers, there has been little evaluation of their clinical validity or of parental perceptions regarding this type of after-hours care. The objectives of the present study were to examine the following factors in parents using pediatric hospital-based call centers in 4 areas of the United States: 1) parental satisfaction with after-hours telephone care, 2) parental preferences regarding the type of provider for after-hours care, 3) parental opinions about the effect of call centers on their relationship with their child’s primary care provider, and 4) parental compliance with disposition recommendations. The investigators hypothesized at the study onset that parents would be satisfied with their care >90% of the time, but that roughly half would prefer to speak with a physician rather than a nurse if they had a choice. In addition, we hypothesized that call centers might disrupt continuity of care with the primary provider and that, therefore, 20% or more of families would report a detrimental effect on their relationship with their provider. We also hypothesized that compliance with an urgent disposition recommendation would be >80%, but that compliance would be lower for dispositions of lower acuity.
During an 8-week period from August 1, 1999, to September 30, 1999, we surveyed callers to 4 after-hours pediatric call centers. Data were collected from pediatric call centers based at children’s hospitals at: 1) Children’s Hospital, Denver, Colorado; 2) Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; 3) Rainbow Babies and Children’s Hospital, Cleveland, Ohio; and 4) All Children’s Hospital, St Petersburg, Florida. The study protocol was reviewed and approved by institutional review boards at each site. All call centers use the same triage and advice software program, Pediatric Triage and Advice System (PTAS; National Health Enhancement Systems, Phoenix, AZ).5 Triage calls at each site are handled by pediatric nurses who have had specialized training in telephone triage. In addition, each call center has a local pediatrician as medical director who is responsible for the quality of care delivered at the call center. Characteristics of the study sites are shown in Table 1.
Triage and Advice System
PTAS is a Windows-based system of computerized protocols that assist with triage and provide advice regarding acute pediatric illness and injury. Pediatric nurses identify the presenting complaints of the caller, access the appropriate computerized PTAS protocol, and ask a series of questions dictated by the protocol. Based on the responses to these questions, the protocols give 1 of the following recommended dispositions: activate the emergency medical system, see immediately at an after-hours facility, see within 4 hours at an after-hours facility, see within 24 hours in physician’s office, see within 72 hours, see within 2 weeks, or home care and advice only. Because small variations in these dispositions can be introduced at individual sites and to focus on the most clinically meaningful categories, for the purposes of this study possible dispositions were grouped into the following categories: 1) patient should be referred for an urgent after-hours evaluation (immediately or within the next 4 hours), termed “urgent;” 2) patient should be referred for evaluation the next day at their primary care physician’s office, “next day;” 3) patient could be seen at a later date, “later time;” or 4) patient doesn’t need to be seen and requires home care and advice only, “homecare.” At each site, providers who subscribe to the after-hours call service have the option of requesting that any patient call for which urgent after-hours care is deemed necessary by nurses be referred to the practice on-call physician for second-level physician triage. This allows a physician from the patient’s practice to make the final decision regarding the necessity of an after-hours visit. The percentage of practices using this option for urgent calls varies in our sites from a low of 20% in St Petersburg to a high of 75% in Cleveland. At all sites, under certain circumstances, the nurse may override the protocol and advise parents to seek after-hours care. Likewise, a parent who feels uncomfortable complying with a recommendation of next-day appointment or home care may override a protocol disposition and request an after-hours evaluation. At each site, the computerized system automatically records the disposition recommended by the protocol, based on responses entered by the nurse, and whether an override by the nurse or parent occurred. If an override occurs that changes the initial recommendation, a final disposition is also recorded.
Random samples of after-hours calls were selected each day from computerized records at each of the 4 study sites. Within each site, all calls were stratified by type of call (nonurgent, including “next day,” “later time,” and “home care,” urgent with second-level triage, and urgent without second-level triage). The number of calls selected each day was based on the average number of calls on weekdays and on weekends for each type of call in the preceding year at each site. Within each call type, 250 calls with nonurgent dispositions, 50 calls with urgent second-level triage dispositions, and 50 calls with urgent without second-level triage dispositions were randomly selected. The disposition used for sampling was the final disposition, reflecting the decision reached using the protocols and any decision to override. Excluded from sampling were nontriaged calls (medication advice calls or callers who refused triage), callers who went to the emergency department before triage, direct referrals to public agencies, poisoning calls, and repeat calls about the same child during the study.
The survey instrument incorporated questions modified from national surveys including the Consumer Assessment of Health Plans Survey and the Behavioral Risk Factor Survey, as well as new questions that were piloted in 2 previous telephone care surveys (44 total questions).6,7 The definition of a chronic condition was defined by a question from the Consumer Assessment of Health Plans Survey and included conditions that had lasted for at least 3 months. Race and ethnicity were determined by self-report using a standardized question from the Behavioral Risk Factor Survey. Telephone surveys were conducted by trained interviewers at a professional survey unit that had no direct relationship with the call centers (Sterling Research Group, Inc, St Petersburg, FL). There were 4 interviewers and 2 rotating supervisors, with additional supervision by a project manager/analyst. The survey was administered using a Computer Aided Telephone Interviewing program to use effective skip patterns and acceptable ranges of responses. All calls were monitored and recorded for quality assurance, with 10% actually audited. Before the survey, a 2-day training session was held with the first day focusing on survey detail to ensure each question was thoroughly understood and the second day on quality assurance review, including examination of Computer Aided Telephone Interviewing and interviewer proficiency.
Interviewees were assured that their compliance was voluntary and that their responses would remain confidential. Names and phone numbers of the randomly selected calls from all 4 sites were electronically transmitted to the survey unit and secured with a password to protect the users’ confidentiality. The calls were conducted across all sites during the same period of time and adjustments for differences in time zones were made. Calls had to be completed between 72 hours and 7 days after the index call to be included in the study. Calls were attempted at least once during the morning, afternoon, and evening hours each day up to a maximum of 14 total attempts/family. Secondary exclusionary criteria included the following: the person who originally called the after-hours call center could not be reached because they had no phone or had left the area (eg, on vacation), the person who originally called the center was not one of the major people responsible for the care of the child (eg, babysitter, neighbor, or friend of the family), the child was in a special living situation (hospital, group home, foster parents, prison, therapeutic school), or the family was not English-speaking. The study populations, exclusions, and response rates are summarized in Fig 1.
Major study outcomes were the parent’s reported degree of satisfaction with various aspects of the call center and with the overall care received (on a 4-point Likert scale), their preferred provider of after-hours care, and whether the call center had impacted their relationship with their primary provider. To decrease reporting bias in assessing compliance, the study interviewers did not tell the parents what the system recorded as the recommended disposition. Instead, parents were asked what they decided to do for their child after speaking with the nurse. Compliance was defined as agreement between what the parent reported doing and what the system recorded as the final recommended disposition. Parents were also asked what they remembered the nurse recommending they do for their child. Their perception of the disposition and the actual system disposition were then compared analytically. Parents were asked why they chose to do something different from what was recommended only if their reported actions differed from what they reported to be the nurse’s disposition.
In assessing satisfaction and parental opinions regarding after-hours care, the entire sample of calls was analyzed. In assessing compliance, we excluded calls from callers who called back and received a different disposition between the index call and the time of the interview because of concern that callers would be unable to remember which disposition related to the index call and might overreport or underreport compliance. In assessing compliance, we also excluded calls that were referred to the physician for additional triage, because we did not know the final disposition of these calls (Fig 1).
Comparisons by site of categorical variables were performed using χ2 tests, and continuous variables were compared using a 1-way analysis of variance. We used unconditional logistic regression to predict physician preference. Factors we thought might be predictive of preferring to speak with a physician, such as the urgency of disposition of the index call, a chronic medical condition in the child, usually speaking with their physician during office hours, and feeling it was important that the provider know the child’s medical history as well as sociodemographic factors that may affect health care delivery choices such as insurance coverage, education, and race/ethnicity, were forced into the model as independent variables. Because the stratified random sampling scheme could affect the results of the logistic regression, if the effect of independent variables on dependent variables differed between disposition types we tested for this possibility. We used a logistic regression to regress the dependent variables on the interaction of the independent variables and disposition type. We found no significant interactions, which validates the results of the logistic regressions reported in the results. All analyses were conducted using SPSS for Windows, version 9.0.1 statistical software (SPSS Inc, Chicago, IL).
Comparisons of characteristics of the populations at the 4 sites are shown in Table 2. Because the sites were chosen to provide a diverse population in different geographic areas, there were notable differences seen, especially with respect to race and ethnicity, marital status, and insurance status of the survey respondents. The percentage of children with chronic illnesses using the call center also differed, and the distribution of these illnesses is summarized in the footnote.
Parental Satisfaction and Perceptions of Care
Parental satisfaction was uniformly high as shown in Fig 2, and intersite differences were only seen with respect to length of time to call back. Overall satisfaction also did not differ significantly by call disposition (Pearson χ2 = 5.59; P = .23). The majority of parents (65.2%) did not have a preference about the type of professional with whom they preferred to speak about health concerns that occurred after hours; 6.1% preferred to speak with a nurse, and 28.6% preferred to speak with a doctor or midlevel provider. Table 3 reports results of a logistic regression predicting parental preference to speak with a physician or mid-level provider for after-hours care. The factors that were predictive were “feeling it was important that the provider know the child’s medical history,” “usually talking with a physician during office hours,” and an educational level of college degree or higher in the respondent. In addition, reporting racial designation to be “other,” (N = 74), a category that included Asian, 27.7%; Native American, 6.8%; native Hawaiian, 1.4%; and unknown, 62.2%; was also predictive of wanting to speak with a physician or mid-level provider. The majority of parents reported no change in their relationship with their provider as a result of after-hours call centers (60.8%, N = 939), or reported a positive effect (37.4%, N = 578). Only 1.7% (N = 26) reported a negative effect and 1.1% (N = 17) did not give an opinion.
Compliance With Disposition Recommendations
As shown in Fig 3, approximately 80% of the most urgent groups and the least urgent groups reported complying with the recommendation made by the system. Compliance with intermediary disposition categories was much lower, with almost no reported compliance with a recommendation to be seen at a time later than 24 hours. Between 76.9% and 100% of noncompliant groups in the 4 disposition categories reported hearing a different disposition than the system disposition. Overall, parents reported doing what they had heard to be the disposition recommendation 92.6% of the time. Reported compliance with specific disposition heard by the parents was 63.6% for later time, 86.6% for next day, 95.2% for urgent, and 96.1% for homecare. Figure 4 demonstrates the percentage of noncompliant respondents who reported doing something more or something less urgent than what was recorded as the recommended system disposition.
Despite the widespread adoption of call centers for the provision of after-hours care for children, relatively little is known about the perceptions of parents regarding this type of care and their level of compliance with recommendations made by nurses at call centers. Our data suggest that parents using after-hours call centers are highly satisfied with the care they are receiving. Most report it has either had no effect or has had a positive effect on their relationship with their primary provider. Parents report complying with disposition recommendations approximately 80% of the time when the disposition is urgent or home care, but report low rates of compliance when the recommendation is intermediary. Parents who do not comply usually report hearing something different from what was documented.
Our results regarding satisfaction corroborate data from 2 earlier studies at single institutions that reported high levels of satisfaction with nurses or physician assistants delivering care at after-hours call centers. Poole et al8 found a mean satisfaction rating of 8.8 on a 10-point satisfaction rating scale, and Pert et al9 reported a rate of satisfied or very satisfied of 88%, although no specifics about the methodology were reported. Comparable satisfaction data for pediatric after-hours care delivered by physicians is difficult to find, but a 1983 study of urban primary care centers and private practices demonstrated that 89% of callers were satisfied, as measured by a dichotomous variable.10 Satisfaction among patients receiving care from family practice residents, of which 20% were calling about a child, was estimated to be 76.7%.11 The consistency of our data across 4 study sites and very high satisfaction rates is strong evidence that delivery of after-hours care by nurses at centralized call centers results in satisfaction levels that are at least comparable, if not higher, than those reported for after-hours care by physicians treating children.
In view of the perceived importance of continuity of care in pediatrics, it is surprising that the majority of parents did not have a preference for the type of professional who provided after-hours care and did not perceive a negative impact on their relationship with their primary provider of having a call center respond to their after-hours concerns. We were unable to find previous literature that focused specifically on such parental preferences. Wanting a physician to provide care was more important for the subgroup of parents who expressed a desire for continuity, who were accustomed to speaking with a physician during office hours, or who were highly educated. The fact that chronic illness did not predict physician preference must be interpreted with caution. At all sites, a significant percentage of children with complex or multisystem disease are likely to contact subspecialty services directly and to bypass the call center used by their primary care physician. The majority of chronic conditions identified by children using our call centers were problems commonly handled by primary physicians, such as asthma, allergies, and chronic otitis media. Therefore, our results pertain to opinions of parents of children with common chronic problems usually handled by primary care physicians but may not pertain to parents of children with complex medical disease followed by subspecialists.
Reported compliance rates for calls given urgent dispositions in our study were similar to one previous report from a call center (83%), in which all emergency department contacts were corroborated by chart review6 and only slightly lower than the rate reported by Crane and Benjamin12 for compliance with resident physicians’ recommendations for urgent referral (94%). In the latter study, methodological differences may have accounted for the higher rates, because the residents’ decisions were being reviewed by an attending physician each morning, and residents were not blinded to the outcome of the study. It is also possible that compliance was higher because the disposition was given by a physician rather than a nurse. Our results differ substantially from those observed by Baker et al13 who reported a compliance rate of only 42% to 46% for urgent referrals by a call center using the same software as described in the present study. They assessed compliance by comparing the system disposition with visits documented in the clinic and hospital chart at their institution, which may have significantly underestimated visits to outside facilities. Differences in the characteristics of the populations studied may also have contributed to differences in compliance, although the uniformity of our data at 4 sites with differing populations do not support this degree of variability.
Compliance within the intermediary disposition categories was substantially lower than the most and least acute categories, with few reporting compliance with a recommendation to be seen at a time later than 24 hours. This may reflect changes in the acute condition about which the index call was made or differences in the way private physicians choose to see patients in the time after the index call. It may also reflect an increasing lack of clarity in the advice delivered by the nurses when the advice is neither clearly for an urgent visit or for no visit. For these intermediary dispositions, assessments of compliance may be less meaningful, because clinical problems that are not urgently evaluated or cannot be quickly resolved with telephone advice are subject to alterations in the course of the disease over time and to additional input from the family’s primary care providers.
Almost all patients who did not comply reported hearing a different disposition than what the system recorded in the computerized record. This may reflect the parents’ reluctance to report doing anything other than what was recommended or confusion with the passage of time and, possibly, input from other providers about what was recommended. It is easy to see how a protocol recommendation for a patient to be seen sometime after 24 hours might become unclear. In this situation, the nurse might advise the patient to call the office in the morning to discuss being seen at a later time. The practice may then decide to see the patient that day, with the parents interpreting this as the disposition recommendation. Our results may also indicate that parents remembered hearing what they wanted to hear or that nurses were unclear in the advice they gave. The cases in which there was noncompliance with a recommendation to be seen urgently are of the greatest clinical concern, especially because the majority reported they thought they were not supposed to be seen at all and 70% reported never being evaluated after their index call. Also of great interest from a health care delivery perspective were the roughly one third of cases for which the system disposition was home care who reported they were referred for urgent evaluation. Additional study is needed to clarify the reasons for misinterpretation or noncompliance in cases such as these.
Our data about compliance is limited by being based on self-report rather than independent assessment. Our findings are similar, however, to other reports in which compliance was independently confirmed. In addition, all data regarding satisfaction and preferences are subject to reporting biases associated with survey administration. We attempted to minimize these by using an independent survey unit and stressing the anonymity of the data to respondents. Our data also do not address the validity of the call center nurses’ recommendations or whether there were any adverse outcomes associated with parental noncompliance. Finally, our data may not be generalizable to call centers that do not resemble those described here. For example, our data may have little relevance to large service bureaus that are located at sites distant from the population being served, that do not use protocols developed for pediatric populations or pediatric nurses for triage, and who do not have a pediatric medical director that is directly responsible for overseeing the quality of care delivered by the center.
The delivery of after-hours care by call centers staffed by nurses has rapidly expanded in the past 10 years in this country and is recently being adopted in other countries.14,15,16,17 Previous studies have demonstrated the value of call centers to physicians in decreasing burnout in primary care and increasing job satisfaction.8,9 Appropriateness of disposition decisions made by call centers has only begun to be assessed, but initial assessments suggest a high rate of appropriateness for urgently referred cases.6,18 The present study demonstrates high rates of satisfaction and acceptance of this type of health care delivery by parents with sick children. In addition, our results do not support a perceived deleterious effect on parents’ relationship with their child’s primary provider with this type of after-hours care. Our results do suggest that parents do not always follow the disposition advised by the center, even in cases for which urgent evaluation was recommended. Better understanding of the reasons for noncompliance will provide insight into the complex information exchange and negotiation that constitutes triage. Although call centers face fiscal challenges as currently structured,19 physicians in many parts of the country have come to rely on them for the provision of after-hours care in recent years, and our study suggests that patient acceptance is high.
- ↵American Academy of Pediatrics, Provisional Section on Pediatric Telephone Care and Committee on Practice and Ambulatory Medicine. Pediatric Call Centers and the Practice of Telephone Triage and Advice: Critical Success Factors. Strategies for Practice Management. 1998:1–6
- ↵Schmitt B. Pediatric Telephone Advice. Boston, MA: Little Brown & Co; 1980
- ↵Katz H. Telephone Medicine and Training. A Handbook for Primary Health Care Professionals. Philadelphia, PA: F. A. Davis Company; 1990
- ↵Brown J. Pediatric Telephone Medicine. 2nd ed. Philadelphia, PA: J. B. Lippincott Co; 1994
- ↵Schmitt B. Pediatric Telephone Triage and Advice System-2000. Phoenix, AZ: National Health Enhancement Systems, Inc; 1996
- ↵Poole SR, Schmitt BO, Carruth T, Peterson-Smith A, Slusarki M. After-hours telephone coverage: the application of an area-wide telephone triage and advice system for pediatric practices. Pediatrics.1993 ;92:670–679
- ↵Gallagher M, Huddart T, Henderson B. Telephone triage of acute illness by a practice nurse in general practice: outcomes of care [see comments]. Br J Gen Pract.1998 ;48:1141–1145
- ↵Marklund B, Bengtsson C, Blomkurst S, et al. Evaluation of the telephone advisory activity at Swedish primary health care centres. Fam Pract.1990 ;7:184–189
- ↵Barber JW, King WD, Monroe KW, Nichols MH. Evaluation of emergency department referrals by telephone triage. Pediatrics.2000 ;105:819–821
- American Academy of Pediatrics