 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ARTICLE |
a Departments of Pediatrics
b Emergency Medicine, Emory University, Atlanta, Georgia
c Departments of Pediatrics
d Information Systems and Technology
e Emergency Services, Children's Healthcare of Atlanta, Atlanta, Georgia
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONREFERENCES |
|---|
METHODS. From October 2005 through March 2006, hospitalizations at a tertiary care pediatric hospital after calls to its call center were matched with their respective call-center dispositions. The host hospital 24-hour underreferral rate was defined as the percentage of total admissions to the study institution within 24 hours after a call to the call center for treatment of the same illness or injury that had been assigned a nonurgent disposition by the call center. A convenience sample of call-center calls was surveyed for admissions to other facilities. This sample was then combined with admissions to the pediatric hospital to estimate a true 24-hour underreferral rate. Underreferrals were subjected to clinical and statistical analyses.
RESULTS. The host hospital 24-hour underreferral rate was 5.2%. The estimated true 24-hour underreferral rate was 5.95% ± 2.75%. Diagnoses frequently associated with underreferral were gastroenteritis, croup, asthma, and bronchiolitis. Underreferred patients admitted to the study institution were hospitalized for an average of 1.6 ± 1.1 days, compared with 2.8 ± 3.1 days for patients referred by the call center to a higher level of care. The monthly SD of the host hospital 24-hour underreferral rate was 1.56%.
CONCLUSIONS. For the call center studied, the host hospital 24-hour underreferral rate could be determined easily and objectively and approximated reasonably the true 24-hour underreferral rate. The month-to-month variability of the host hospital 24-hour underreferral rate was sufficiently small to allow for meaningful internal trending analyses.
Key Words: call center • telephone medicine • telephone triage • pediatrics • patient safety
Abbreviations: CC—call center • ED—emergency department • AHC—after-hours care • CI—confidence interval
Pediatric telephone triage centers carry a heavy burden of responsibility. Failing to refer a sick child to a higher level of care (underreferral) could severely compromise patient safety. A benchmark measure of the tendency of a call center (CC) to underrefer would be highly desirable. This benchmark should be determined easily, allowing a CC to use serial measurements of this benchmark to identify its underreferral trend as changes are made to protocols, policies, and training procedures. For trending of this benchmark to be meaningful, its variability from month to month should be low and the calls responsible for any trend should be identified easily.
Three studies have examined a CC's tendency to underrefer pediatric patients.1–3 Lee et al1 prospectively compared calls managed by a CC and calls managed by pediatricians. Kempe et al2 used postcall telephone surveys to determine underreferral rates. Those 2 studies are worthy contributions to the telephone triage literature. However, their methods do not lend themselves easily to serial measurements of CCs’ underreferral rates.
A second study by Kempe et al3 linked CC data with the claims information for a closed, managed-care population. This data-mining approach allowed Kempe et al3 to study 32968 calls, with 378 patients admitted within 24 hours after calling the CC. Fifty-six of the 378 admissions had been provided with a nonurgent disposition by the CC. Kempe et al3 chose to define the potential underreferral rate for the CC studied by using total calls as the denominator. Specifically, the potential underreferral rate was defined by Kempe et al3 as being 56 divided by 32968, or 0.17% (1 of every 589 calls).
Use of total calls as the denominator in defining a CC's underreferral rate necessitates that total calls be the basis of study. Without electronic access to insurance claims data for at least a subpopulation of callers, this forces a CC to rely on a follow-up call method, with its inherent costs and volume limitations, when studying its tendency to underrefer. The current study was undertaken to demonstrate the use of an automated measure of a CC's tendency to underrefer, based on patients admitted within 24 hours to a single hospital, and to demonstrate that it approximates reasonably a more-global measure of that CC's tendency to underrefer.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Daily Query of the CC and Inpatient Databases
Although the CC and inpatient databases at Children's Healthcare of Atlanta are not interfaced, the Children's Healthcare of Atlanta Department of Information Systems and Technology was able to create a daily query of these databases that identified, for each call to the CC, matches (according to birth date and last name) that appeared within 24 hours after the call as Children's Healthcare of Atlanta inpatients. During the study period, this report was prepared each day for the calls placed 4 days earlier. For example, on the tenth of the month, a report of all admissions within 24 hours after calls placed on the sixth of the month was available for analysis. Additional information listed in the reports for each call and admission included call and admission dates and times, CC nurse, CC disposition, referral justification (if referred), CC protocol used, admitting diagnosis, CC identification number, and inpatient medical charts number.
Urgent Referral Rate
A protocol branch might end in one of many different dispositions. These dispositions were divided into 3 groups: "911/ED/AHC," "call another provider," and "nonurgent." The 911/ED/AHC group represented all 911 referrals and all referrals to an emergency department (ED) or another after-hours care (AHC) setting for examination. The call another provider group included all dispositions that directed the caller to speak promptly to another provider, where promptly was defined as <4 hours. "Another provider" included pediatricians, subspecialty physicians, and the Georgia Regional Poison Control Center. The nonurgent group included all remaining dispositions not involving prompt examination or prompt telephone contact with another provider. For example, it included the commonly used disposition "call pediatrician in morning." The urgent referral rate for a given time period was defined as the number of 911/ED/AHC dispositions expressed as a percentage of the total number of advice calls managed during that time period.
Many pediatricians who subscribe to CC services request to perform secondary triage for any patient judged to warrant an ED referral by the CC nurse. The disposition of record for a given call, for purposes of calculating the CC urgent referral rate, was the disposition determined to be most appropriate by the CC nurse. For example, if a call was sent to a physician because of his or her standing request to perform secondary triage for ED referrals, then this call was regarded as an ED referral.
Underreferral Rate Calculations
The host hospital 24-hour underreferral rate was defined as the number of children provided with a nonurgent disposition who were subsequently admitted to Children's Healthcare of Atlanta within 24 hours for treatment of the same illness or injury, expressed as a percentage of total calls resulting in admission to Children's Healthcare of Atlanta within 24 hours for treatment of the same illness or injury. This rate, based on hospitalizations at Children's Healthcare of Atlanta, serves as a proxy for the more-global rate including hospitalizations at any institution. In particular, the "true" 24-hour underreferral rate would be the number of children provided with a nonurgent disposition who were subsequently admitted to any hospital within 24 hours for treatment of the same illness or injury, expressed as a percentage of total calls resulting in admission to any hospital within 24 hours for treatment of the same illness or injury. To determine the degree to which the host hospital 24-hour underreferral rate approximates the true 24-hour underreferral rate, a convenience sample of calls to the CC underwent follow-up monitoring via telephone during the study period. This convenience sample was designed to avoid disproportionate sampling of a particular CC shift or day of the week. The rate of admission to a hospital other than Children's Healthcare of Atlanta after a call to the CC, for treatment of the same illness or injury, was then estimated by using standard sample size calculations, with a confidence interval (CI) of 95%. In a similar manner, the sampling results were then used to estimate the number of children admitted to another hospital within 24 hours after a nonurgent CC disposition. These results were then combined with the host hospital 24-hour underreferral rate data.
Calls with the disposition "information only" were eliminated from consideration in analyses of underreferral rates. This disposition is used when a CC nurse does not feel comfortable providing definitive triage advice. For example, this disposition is used routinely when the caller is not with the child.
Analysis of Underreferrals
The frequency with which a given protocol resulted in underreferral was determined and compared with the overall use rate for that protocol during the study. Final diagnoses, lengths of stay, age distributions, and other characteristics of underreferred patients were determined.
Statistical Methods and Institutional Review Board Approval
Categorical variables were analyzed by using 
2 analysis. Sampling errors were estimated by using standard sample size calculations. This study was approved by the institutional review boards of both Children's Healthcare of Atlanta and Emory University.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Twenty-Four–Hour Underreferral Rate
During the study period, 831 children were admitted to Children's Healthcare of Atlanta within 24 hours after contact with the CC, of whom 807 were judged to have been admitted for treatment of the same illness or injury that prompted the call to the CC. Forty-six of the 831 children had received a nonurgent disposition from the CC. Forty-two of those children were judged to have been admitted for treatment of the same illness or injury that prompted the call to the CC, for a host hospital 24-hour underreferral rate of 5.2% (42 of 807 children). With elimination of the subjective determination of the relationship of illness prompting the call to illness leading to admission, the 24-hour Children's Healthcare of Atlanta underreferral rate was 5.5% (46 of 831 children).
During the same study period, a convenience sample of 4986 callers to the CC was studied. Thirteen admissions to a non–Children's Healthcare of Atlanta inpatient facility within 24 hours after contact with the CC were identified. (One call was eliminated from consideration because admission time could not be determined reliably; that caller had been provided with a nonurgent disposition.) Eight of the 13 callers had been provided with an urgent referral disposition by the CC; 5 had been directed to call another provider. A simple extrapolation of these 13 admissions from the sample of 4986 to 126972 total calls during the study period resulted in an estimate of 331 admissions to a non–Children's Healthcare of Atlanta inpatient facility [126972 x (13/4986)]. By treating these 4986 calls as a sample of the 126972 calls and using a 95% CI, the possible range of the number of admissions to other hospitals after a call to the CC during the study period was determined to be 142 to 520 admissions (Table 1).
|
With the use of total calls as the denominator in the formula defining the CC underreferral rate (the method used in the studies by Kempe et al2,3), this rate would range from 1 of 1094 calls to 1 of 3023 calls (95% CI). Stated as a percentage, this rate range is 0.03% to 0.09% (95% CI). The monthly Children's Healthcare of Atlanta underreferral rate during the 6 months studied varied from 3.5% to 7.9% (mean: 5.39%; SD: 1.56%).
Clinical Characteristics of Underreferrals
Table 2 lists the final diagnoses for the 42 underreferrals to Children's Healthcare of Atlanta. Gastroenteritis, croup, asthma, and bronchiolitis accounted for 66% of the cases. The average length of stay was 1.6 days (SD: 1.1 days), compared with 2.8 days (SD: 3.1 days) for patients referred to a higher level of care (P = .016). Only 1 underreferred patient was hospitalized for >3 days (6-day admission for management of an infected branchial cleft cyst). Two underreferred patients were admitted to the ICU for a total of 2 ICU days, one with asthma and the other with new-onset diabetes mellitus. The age distribution for underreferrals was not significantly different from the overall age distribution for cases managed by the CC, as determined with an F ratio (analysis of variance) of 1.6 in an age group analysis with 4 age groups.
|
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
The host hospital 24-hour underreferral rate, as defined in this study, has many advantages as a measure of a CC's tendency to underrefer children in need of prompt examination. It is an objective measurement. The data necessary to determine this rate are likely to be available from the host hospital's computer systems. As illustrated in this study, it is likely that, for many pediatric CCs, their host hospital 24-hour underreferral rates would reasonably approximate their more-global 24-hour underreferral rates. These institutions could determine their host hospital 24-hour underreferral rates at regular intervals with minimal effort after initial configuration. The effects of protocol modifications or training initiatives on a CC's tendency to underrefer could then be monitored and analyzed. Finally, at least for the CC studied, the month-to-month variability of the host hospital 24-hour underreferral rate was minimal, which supports its value as an ongoing monitor of a CC's tendency to underrefer.
A pediatric CC is unlikely to ever achieve a 24-hour underreferral rate of 0%. Some children's conditions are stable in all respects at the time of the call but deteriorate within hours after the call. With respect to that point, it is not surprising that two thirds of the underreferrals in this study involved patients with gastroenteritis, croup, asthma, or bronchiolitis, illnesses that are well known for their variable and unpredictable courses. Nevertheless, this study has demonstrated that a 24-hour underreferral rate of 5.95 ± 2.75% can be achieved by a high-volume pediatric CC. Determining whether a lower 24-hour underreferral rate can be achieved will require additional study.
Neither the host hospital 24-hour underreferral rate nor the more-global 24-hour underreferral rate should be viewed as an absolute measure of CC safety. For example, the rates would fail to identify children admitted 25 hours after inappropriate CC advice or children provided with homecare advice who visited an ED soon after receiving homecare advice, received urgent treatment, and were discharged. Nevertheless, these measures of CC safety provide an automated method of serially monitoring a CC's safety.
The call another provider rate of 20.7% reported in this study is high, compared with similar rates reported in the literature.3 The protocols used in this study were designed specifically to triage acutely ill or injured, previously well children. Accordingly, all protocols contain questions to identify children with chronic medical conditions and those whose acute medical problems have been the subject of aggressive medical evaluation or treatment before the call. Such calls are screened for highly urgent conditions and then referred to the appropriate primary care provider or specialist. Historically, more than two thirds of all calls referred to speak with another provider by the Children's Healthcare of Atlanta CC were referred because of those 2 questions. This philosophy almost certainly reduces underreferrals by the CC.
Review of individual calls that led to an underreferral was quite productive. For example, it became evident that several patients with croup who were admitted within 24 hours had not been prescribed steroids by their pediatricians the morning after the call. This information led to a change in the follow-up instructions to parents of children with croup. In another example, the child admitted with staphylococcal scalded skin syndrome presented with scalp tenderness, minimal erythema, and no fever. The nonfever arm of the rash protocol was modified to identify such a case in the future. Review of calls for patients who were admitted within 24 hours and were referred to a higher level of care by the CC allowed the reviewers to provide substantial positive feedback and medical follow-up information to the CC nurses.
This study has some limitations. The study was approved for and conducted during the winter months of October through March. Seasonal influences could have affected the underreferral rates determined here.
Sampling for admissions to a non–Children's Healthcare of Atlanta facility used the method of convenience sampling. It is possible, for example, that admissions that received a nonurgent disposition occurred with greater frequency among noncontacts than among contacts completed during sampling. Other biases are always possible with a convenience sample. An additional limitation of telephone sampling, in this study and other studies, is determining accurately the timing of an admission to a facility without access to the medical charts. A limitation of our method of identifying admissions to the host hospital was the reliance on matching birth dates and last names. This undoubtedly resulted in missed admissions within 24 hours. However, we do not think that underreferrals would be disproportionately missed with this method.
Comparison with other CCs might be hampered by differences in the categorization of calls among CCs. For example, cases with significant chronic medical conditions are managed by Children's Healthcare of Atlanta CC, but they are managed very conservatively. Most but not all of these calls are returned to a pediatrician or specialist for triage after emergency and highly urgent situations are ruled out. These calls were included in this study. Other CCs might exclude such calls from statistical analysis. As another example, the calls that were categorized as information only by Children's Healthcare of Atlanta CC and were eliminated from additional consideration might be accounted for in a different manner by other CCs.
Finally, whether an admission was or was not related to the illness or injury that prompted the call was sometimes subjective. However, as noted above, few calls were eliminated for this reason. Simply assuming that all admissions are related to the illness or injury that prompted the call makes the host hospital 24-hour underreferral rate easier to determine without significantly compromising data integrity (at least for the CC studied). Despite its limitations, we think that this study demonstrates a method that could be used by other CCs to monitor underreferral trends and to identify both opportunities for improvement and opportunities to provide positive feedback to CC nurses.
|
FOOTNOTES |
|---|
Address correspondence to Joseph E. Simon, MS, MD, Department of Pediatrics, Children's Healthcare of Atlanta, 1639 Tullie Circle, Atlanta, GA 30329. E-mail: joseph.simon{at}choa.org
The authors have indicated they have no financial relationships relevant to this article to disclose.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2007 American Academy of Pediatrics. All rights reserved. |
||
|
||
