 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ARTICLE |
a Departments of Pediatrics
f Family Medicine
g General Internal Medicine
b Colorado Health Outcomes Program
c Primary Care Research Unit, University of Colorado and the Health Sciences Center, Denver, Colorado
d Children's Outcomes Research Program, Children's Hospital, Denver, Colorado
e Kaiser Permanente Research Unit, Denver, Colorado
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONREFERENCES |
|---|
METHODS. The study population included all pediatric patients enrolled in Kaiser Permanente Colorado whose families called the Children's Hospital after-hours call center in Denver, Colorado, during the period between October 1, 1999, and March 31, 2003. Postcall disposition recommendations were categorized as urgent (visit within 4 hours), next day (visit in >4 hours but within 24 hours), later visit (visit in >24 hours), or home care (care at home without a visit). Compliance with the nurses’ triage disposition recommendations was calculated as the proportion of cases for which utilization data matched the disposition recommendations.
RESULTS. Of the 32968 eligible calls during the study period, 21% received urgent, 27% next day, 4% later visit, and 48% home care disposition recommendations. Rates of compliance with both urgent and home care disposition recommendations were 74%, and the rate of compliance with next day recommendations was 44%. No deaths occurred within <1 week after the after-hours calls. The rate of potential underreferral with subsequent hospitalization was 0.2%, or 1 case per 599 triaged calls. In multivariate modeling, age of <6 weeks or >12 years and being triaged after 11 PM were associated with higher rates of potential under-referral.
CONCLUSIONS. Approximately three fourths of families complied with recommendations for their child to be evaluated urgently or to be treated at home, with much lower rates of compliance with intermediate dispositions. The rate of potential underreferral with hospitalization was low, and age and time of call triage were associated with this outcome.
Key Words: telephone medicine • telephone triage • call center • health care delivery
Abbreviations: KPCO—Kaiser Permanente Colorado • CI—confidence interval
In the past decade, the delivery of health care after office hours has undergone profound changes. Many primary care providers have stopped taking after-hours calls themselves, instead signing out to centralized call centers that manage patient calls regarding acute problems for multiple health care providers. Call centers and service bureaus usually employ nurses, who use decision-support protocols or guidelines to aid in triage. Data from the 1999 to 2000 American Academy of Pediatrics Periodic Survey demonstrated that 21% of members used a centralized call center with staff members using written protocols and 19% used a centralized service with staff members using computerized protocols, although the amount of overlap between these categories was unclear.1 In addition to call centers based within local hospitals or consortia of practices that cover patients of subscribing physicians, health insurance plans have established large, centralized call centers called service bureaus that respond to calls from members throughout the country from a single site. Recent estimates suggest that >100 million people in the United States have access to call centers through a hospital or health maintenance organization.2
Despite the increasing role that call centers are playing in the delivery of after-hours care, the safety of after-hours triage by call centers has been evaluated only rarely. Previous studies demonstrated high rates of appropriateness of urgent referrals by pediatric call centers3,4 and fairly high rates of compliance with triage disposition recommendations among patients referred urgently and those advised to be treated at home.5,6 Only 1 previous study examined underreferral among children triaged by call centers, and that study relied on parental reports of outcomes, rather than actual utilization data.7 The previous study was not sufficiently large to yield precise estimates of underreferral associated with triage by a call center. Estimates of how frequently underreferral is associated with significant consequences would be particularly valuable to physicians who are trying to weigh the risks and benefits of signing out to call centers.
In this study, we examined outcomes for children enrolled in a vertically integrated, staff-model, health maintenance organization whose families accessed a pediatric hospital-based call center regarding urgent problems among their children. Our objectives were to examine (1) actual compliance with nurse disposition recommendations; (2) the frequency of adverse events, including death, and potential underreferral associated with hospitalization in the subsequent 24 hours; and (3) factors associated with potential underreferral.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Study Site
The after-hours call center at the Children's Hospital, which has been in existence for 18 years, employs 16.4 full-time employee-equivalent pediatric nurses and provides after-hours coverage for 98% of Denver and 84% of Colorado private pediatricians, as well as all pediatric patients treated by KPCO in the Denver metropolitan area. Triage nurses must be registered nurses with 
4 years of pediatric clinical experience and with specialized training in telephone triage. The nurses use computerized protocols that guide them through a sequence of questions, the answers to which dictate a recommended triage disposition. If the nurse feels uncomfortable with the parents’ ability to assess the patient accurately or to care for the patient or thinks that the child needs to be treated more urgently than the protocol dictates, for any reason, then he or she may override the protocol disposition and upgrade the urgency of disposition. Nurses are allowed to recommend a disposition decision that is less urgent than dictated by protocol only under special circumstances and with the agreement of a second triage nurse. The medical director of the after-hours call center is a local pediatrician. The program has ongoing quality improvement programs, including monthly review of audiotaped calls and call documentation for all nurses.
Data Sources
Patient name, KPCO identification number, date and time of call, protocol used for triage, triage recommendation, and additional notes by the triaging nurse were recorded automatically in the computerized database of the after-hours call center. We excluded from analysis all calls for which parents could not be contacted by triage nurses, duplicate records of calls, and calls with an unknown triage disposition. In addition, calls that were referred to a physician, resulting in an unknown disposition, were excluded. The after-hours call center data were linked to KPCO administrative databases through the unique KPCO identification numbers. The KPCO administrative databases provided sociodemographic data for the patients and health care utilization data for the 72 hours after the index call. We used International Classification of Diseases, 9th Revision, codes to determine the presence or absence of a chronic condition. We identified chronic health conditions with inpatient and outpatient codes defined by the National Association of Children's Hospitals and Related Institutions Classification of Congenital and Chronic Health Conditions.8
Study Outcomes
Compliance With Disposition Recommendations
Triage recommendations by the nurses were grouped into the following categories: (1) urgent, requiring after-hours evaluation within the next 4 hours; (2) next-day visit in the office (>4 hours but 
24 hours); (3) later visit in the office (>24 hours to 72 hours); or (4) home care, requiring parental supportive care and advice only. The 24-hour period after a call was defined as a single episode of care. If a patient received >1 disposition in the course of multiple calls within 24 hours, then the most urgent disposition recommendation was used as the index call. If there was >1 utilization outcome, for example, an urgent emergency department visit and an office visit, then the most urgent outcome was used for assessment of agreement with the nurse's disposition. If a patient contact at KPCO on the same date as the call was documented but the time of the call was unknown, then the call was excluded, because the KPCO contact could have occurred either before or after the call.
Potential Underreferral Associated With Hospitalization
Underreferral was defined conceptually as triage to a disposition category that was less urgent than the subsequent course of the patient's illness demonstrated to be medically appropriate. To estimate the rate of under-referral associated with hospitalization (as a marker of serious underreferral), we identified a sentinel event that could be defined easily with our existing administrative data. We defined a case as a potential under-referral if a child who was given a nonurgent disposition recommendation was hospitalized within 24 hours after the index call. Previous literature suggested that defining potential underreferral in this way would likely overestimate the true rate of underreferral associated with hospitalization,7 because some patients who were triaged appropriately on the basis of the information available when the family first contacted the call center experienced clinical deterioration after the call.
Analytic Plan
The unit of analysis was call episodes to the after-hours call center, rather than unique patients. The rates of compliance within each disposition category were calculated as the proportion of cases for which utilization data matched the nurses’ disposition recommendations. The rate of potential underreferral was calculated as the proportion of all triaged cases in which a patient was hospitalized within 24 hours after a nonurgent disposition recommendation. This rate was reported both as a percentage of the total number of calls and as the number of triaged calls required to observe a single event. Age-stratified point estimates and 95% confidence intervals (CIs) were also calculated; 
2 tests were used to evaluate the association between underreferral and independent variables, including age (<6 weeks, 6 weeks to 23 months, 2–11 years, or 12 years of age), gender, presence of a chronic condition, and season, day, and time of call. Independent variables that were significant at P .15 in bivariate analyses were entered into a multivariate logistic regression model to evaluate the independent variables associated with underreferral. Variables that did not add significantly to the model at P .05 were removed. The potential for confounding and interactions was also assessed. All analyses were conducted with SAS software (versions 8.02 and 9.1; SAS Institute, Cary, NC).
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
5 calls. Similar to results of previous reports regarding the Children's Hospital after-hours call center,9 the 10 protocols used most frequently by the nurses to triage calls during the study period were vomiting, colds, cough, fever, sore throat, diarrhea, eye with pus, earache, trauma, and rashes.
|
|
|
Rates of hospitalization within 24 hours after a call to the call center varied according to the type of triage disposition and compliance with the disposition. The hospitalization rate was 1.1% (n = 378; 95% CI: 1.0%–1.3%) for all eligible calls. Among the 7093 calls that received an urgent disposition, 4.5% of patients (n = 322; 95% CI: 4.1%–5.0%) were hospitalized within 1 day, compared with only 0.2% (n = 56; 95% CI: 0.2%–0.3%) of the 25929 calls given a nonurgent disposition recommendation. The hospitalization rate among the 1827 calls with an urgent disposition recommendation but noncompliance, 5.7% (n = 104; 95% CI: 4.7%–6.9%), was as high as the rate for all urgently referred cases. In contrast, the rate among the 233 calls with home care dispositions for which the patient instead was examined urgently was 3.0% (n = 7; 95% CI: 1.2%–6.1%), a rate that was significantly higher than the rate for patients given a home care disposition who were not examined urgently (0.2%; P < .001).
The 56 patients who were given a nonurgent disposition recommendation by call center nurses but were hospitalized within the subsequent 24 hours were classified as potential underreferral cases. The potential underreferral rate for the sample, therefore, was 1 case per 599 triaged calls (95% CI: 1 case per 472–901 triaged calls).
Factors Associated With Potential Underreferral
As demonstrated in Fig 3, the rates of potential under-referral varied significantly according to age, with children <6 weeks of age and those >12 years of age having the highest rates (P = .001 for comparison of all groups). Table 2 includes bivariate and multivariate models assessing factors associated with potential underreferral, demonstrating that both age of <6 weeks or >12 years and a triage time after 11 PM were associated with higher rates of potential underreferral. Of calls that occurred after 11 PM, 71% occurred between 11 PM and 3 AM.
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
660 calls, with infants <6 weeks of age and adolescents, as well as calls triaged after 11 PM, being at highest risk of potential underreferral.
Our findings regarding parental compliance were similar to those of 2 previous studies assessing compliance at our institution and at other institutions.5,24 Compliance with either urgent or home care disposition recommendations was reported in 1 study of 4 sites with children's hospital-based call centers to be 80%.5 In a comparative study of triage by nurses versus on-call physicians, parental compliance ranged from 73% to 75% for urgent disposition recommendations and from 74% to 77% for home care disposition recommendations, with no significant differences noted between those triaged by nurses and physicians.24 In our study, as well as in the 2 other studies noted, rates of compliance with intermediate recommendations to be examined by a physician the next day or at a later time were much lower.5,24 This finding suggests that families want their acute problems to be addressed within 24 hours, even if they are judged less urgent by a nurse, unless the problems resolve within that time period.
In our study, approximately one fourth of families did not comply with a recommendation for urgent referral. The hospitalization rate for this subgroup was similar to the rate for urgently referred cases overall, which suggests that these cases were not over-referrals in which the parents down-triaged their child appropriately. Of interest, patients who were judged to require only home care by the nurses but were examined within 4 hours despite those recommendations were hospitalized at a much higher rate than that observed for nonurgently triaged cases overall. The rate was almost as high as the rate for urgently referred cases, which suggests that the disposition might have been up-triaged appropriately by parents in this group. In aggregate, these data suggest that sometimes parents may have difficulty conveying the severity of their children's illness and their judgments that their children need to be examined should not be discounted by call center nurses, whereas increased efforts should be made to facilitate visits and to convince parents of the need for a visit when children are judged by a call center nurse to require urgent evaluation. The role of parents in the triage process is difficult to measure, but these data point out their probable importance for the final outcomes of triaged children. Our findings also demonstrated much higher rates of hospitalization for the group of patients referred for urgent evaluation, compared with the group with nonurgent disposition recommendations; this supports the contention that call center nurses using guidelines seem to be identifying accurately the patients at highest medical risk. These data also corroborated an earlier study, conducted at 4 pediatric call centers, that showed similar differences in hospitalization rates on the basis of parental self-reports.7
From a practicing physician's point of view, under-referral of a patient who is not recognized to be seriously ill and subsequently is hospitalized or experiences a poor outcome is the most serious possible consequence of signing out patients to a call center. Only 1 previous study assessed serious adverse outcomes associated with care by pediatric call centers. Although that study relied on parental reports of outcomes, the estimated rate of potential underreferral in the present study falls within the 95% CI of the previous estimate.7 It is important to note that the potential underreferral rate reported here may overestimate the true underreferral rate. In the previous study,7 audiotapes of all potential underreferrals, defined in the same way as in the present study, were reviewed in a blinded manner, interspersed with randomly selected calls, by expert panelists who were unaware of the final disposition. On the basis of that review, only 60% of potential underreferrals were judged to be true underreferrals, whereas the rest were judged to have been triaged appropriately, with subsequent clinical deterioration that was not predictable at the time of the initial call. Therefore, the potential underreferral rate reported here is likely to be an overestimate of the true underreferral rate.
The magnitude of the underreferral rate is difficult to interpret in the absence of comparable data regarding the alternative mode of after-hours care, namely, physicians taking call primarily. The only available data indicate that call center nurses are more cautious in their triage decisions than physicians. A previous study demonstrated that, when physicians performed a secondary triage of calls designated as urgent by call center nurses, they decreased the urgent referral rate by 50%.3 Although physicians are likely to be less conservative about referral overall, their rate of underreferral is unknown. Although previous reports documented inadequacies in the telephone triage skills of physicians25–29 or compared triage decisions made by physicians, mid-level providers, and nurses,30–33 there have been no studies assessing directly patient outcomes after telephone triage by physicians. Such studies are needed to understand the risks and benefits of alternative forms of after-hours care.
Our study has some important strengths and limitations. Because all KPCO pediatric calls are triaged by the call center and KPCO is a closed-model health care organization, follow-up data should be virtually complete. In addition, the present study included the greatest volume of calls with reported compliance and utilization outcomes, which allowed us to provide more precise estimates of rates of underreferral and to assess risk factors for this outcome. However, our study did not address underreferrals without subsequent hospitalization and did not capture outcomes that occurred after the 72-hour follow-up period. Our data also could not quantify the contribution of parental decisions to averting serious underreferral in cases in which parents made a more-urgent decision than the call center dictated and the child was not hospitalized. The number of potential underreferrals identified was insufficient for identification of specific diagnoses with greater likelihood of leading to underreferral or to rapid clinical deterioration, resulting in hospitalization. Our data did not include calls that were never triaged by the call center because the caller could not be reached or calls that were referred to a physician for which disposition data were unavailable. In addition, our data were collected at a single call center and might not be generalizable to other call centers, particularly those that do not use pediatric nurses for triage, are not located in the same city as the patients, do not have a pediatric medical director who is directly responsible for overseeing the quality of care delivered, or do not use protocols designed specifically for pediatric populations.
Because call centers play an increasingly prominent role in the triage of acute patient problems in this country and internationally, data regarding the effectiveness and safety of care delivered by call centers and the compliance of triaged patients are crucial. This study is the first to assess actual health care utilization after triage and potential underreferral, with subsequent hospitalization, for all children who receive their care within a closed-system health care organization. Our findings support the safety of triage by call centers with respect to short-term adverse outcomes such as hospitalization. Additional data regarding underreferral that does not result in hospitalization but may result in delay of care or suboptimal care are needed. In addition, comparable data regarding outcomes associated with physician triage are needed to inform decisions about the most valid and cost-effective model for telephone triage.
|
FOOTNOTES |
|---|
Address correspondence to Allison Kempe, MD, MPH, Children's Hospital, 1056 E 19th Ave, B032, Denver, CO 80218. E-mail: kempe.allison{at}tchden.org
The authors have indicated they have no financial relationships relevant to this article to disclose.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
This article has been cited by other articles:
 |
|
 |
|
  K. L. Black Standardization of Telephone Triage in Pediatric Oncology Journal of Pediatric Oncology Nursing, August 1, 2007; 24(4): 190 - 199. [Abstract] [PDF]
|
|
|
|
 |
|
 D. A. Hirsh, H. K. Simon, R. Massey, L. Thornton, and J. E. Simon The Host Hospital 24-Hour Underreferral Rate: An Automated Measure of Call-Center Safety Pediatrics, June 1, 2007; 119(6): 1139 - 1144. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Bunik, J. E. Glazner, V. Chandramouli, C. B. Emsermann, T. Hegarty, and A. Kempe Pediatric Telephone Call Centers: How Do They Affect Health Care Use and Costs? Pediatrics, February 1, 2007; 119(2): e305 - e313. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2006 American Academy of Pediatrics. All rights reserved. |
||
|
||
