PEDIATRICS Vol. 105 No. 4 April 2000, pp. 819-821

Evaluation of Emergency Department Referrals by Telephone Triage

Judson W. Barber, MD, William D. King, MPH, DrPH, Kathy W. Monroe, MD, and Michele H. Nichols, MD

From the Department of Pediatrics, University of Alabama at Birmingham, Birmingham, Alabama.

	ABSTRACT

Top Abstract Methods Results Discussion Conclusion References

Objective.  Telephone triage programs are becoming very common at children's hospitals across the nation. One of the proposed benefits of these programs is the more efficient use of health care resources by triaging patients to the appropriate level of health care. The purpose of this study is to examine the appropriateness of referrals to a pediatric emergency department (ED) by the Pediatric Health Information Line (PHIL), a hospital-based telephone triage program, versus all other sources of referrals.

Methods.  A blinded Delphi rating system was used to review the physician's sheets of 133 consecutive ED referrals by PHIL for medical appropriateness. A total of 260 randomly selected control patients seen in the ED during the same period were similarly reviewed. If 2 of 3 pediatric emergency medicine physicians agreed that an ED visit was appropriate, then it was considered appropriate. A comparison of the 2 groups' ED appropriateness was made using a contingency table ² test. An odds ratio with confidence limits was also calculated. Demographic data were collected for both groups including age, race, gender, and insurance status.

Results.  The PHIL group had an appropriateness rate of 80.2%, compared with 60.5% for the control group (² = 14.6369; odds ratio = 2.65; 95% confidence interval [1.5759,4.5008]).

Conclusions.  This demonstrated that for the period studied, PHIL referrals to the ED had a 33% higher rate of appropriateness than controls. This evidence supports telephone triage as an efficient gatekeeper for health care resources.  Key words:  gatekeeper, telephone, triage, pediatrics, information.

In the past decade there has been a large increase in the number of telephone triage programs in the United States. This is especially true in the field of pediatrics, where many of these programs are based at urban children's hospitals. The Pediatric Health Information Line (PHIL) is 1 such program that has been operated by the Southeast Child Safety Institute at the Children's Hospital in Birmingham, Alabama, since 1995. PHIL is staffed by registered nurses who receive special training in triage protocols and Windows-based computer operation followed by a training period with an experienced telephone triage nurse. PHIL uses the Centramax M (National Health Enhancement Systems, Inc, Phoenix, AZ)¹ telephone triage patient and data management software. Patients and their families are put in contact with PHIL by the answering services of subscribing pediatric practices (including the local health department).

The proposed benefits of telephone triage are many including patient (or parent) education, improved practice satisfaction among participating physicians, and safe and efficient patient use of health care resources. Previous reports in the pediatric literature are few in number and focus primarily on the implementation of the programs. Appropriateness of emergency department (ED) referrals was quoted as 78% in 1 study in which only 1 nonblinded person reviewed a random sample of charts. There was no control group.²

The purpose of this study is to determine how many of the ED referrals made by PHIL are appropriate compared with all other sources of referral. In doing so, more stringent criteria are used than have been used in previous studies. A pilot study demonstrated that PHIL had a nearly 18% higher rate of appropriate referrals to the ED compared with the control group. However, this pilot study lacked the sample size to draw conclusions about the significance of these findings.

	METHODS

Top Abstract Methods Results Discussion Conclusion References

Cases were defined as a series of consecutive referrals to a pediatric ED made by PHIL between October 15, 1997 and October 31, 1997. Controls were patients seen in the same ED during that time period and were randomly selected using a computer-generated random numbers table. A 1:2 case to control ratio was used. The study population contained 133 cases and 260 controls for a total sample size of 393 ED visits as determined using a power of .80 and a 2-tailed  of .05 to detect at least a 15% difference in the rates of ED referral appropriateness. A Delphi rating system was used whereby the physician's sheets from the ED charts of the patients in the study were reviewed by 3 pediatric emergency physicians (J.W.B., K.W.M., and M.H.N). The physicians were blinded to whether the charts were cases or controls as well as their insurance status. Based on the information found on the ED charts, the 3 panel members were asked to decide whether the patients were appropriate ED referrals. If 2 of 3 agreed that a referral was appropriate, then it was considered appropriate. A partial list of medically appropriate ED visits was compiled and agreed on in advance by the reviewing physicians (Table 1). If a patient in either group met 1 of these criteria, then the visit was automatically considered appropriate.

Study patients' demographic information, including age, gender, race, and insurance status was entered into a database. Mean age determinations with standard deviations for cases and controls, as well as cross tabulations and other simple descriptive statistics were performed with Microsoft Excel (Microsoft, Redmond, WA). The ² test of independence with resultant odds ratio calculation was performed with True Epistat (Epistat Services, Richardson, TX). Cornfield's 95% confidence limits for the odds ratio were obtained as well. A Mantel-Haenszel standardized odds ratio for deconfounding was calculated to correct for insurance status. Student's t test was used to analyze the mean age differences between the case and control groups.

	RESULTS

Top Abstract Methods Results Discussion Conclusion References

The demographics of the PHIL (case) group (n = 133) and the control group (n = 260) were similar in many respects, but also had some notable differences (Table 2). The mean age for the PHIL group was 5.33 years (standard deviation ± 4.99) compared with 6.06 years (standard deviation ± 5.34) for the control group. The differences in mean age were not shown to approach statistical significance by the Student's t test (t₃₉₁= .77; P > .2). The PHIL group was 60.2% male. The control group was 53.8% male. Again, this difference was not statistically significant (P > .2). The racial composition of the PHIL group was 68.4% white, 29.3% black, and 2.3% other or unknown. The racial composition of the control group was 33.5% white, 65.4% black, and 1.1% other or unknown (P < .000001). A large majority (80%) of the PHIL patients had a form of private medical insurance. The remaining 20% either had Medicaid or no insurance. This is in contrast to the control group in which 62.3% of the patients had Medicaid, county (indigent), or no insurance. Of the control patients, 37.7% had private health insurance (P < .000001).

By use of the Delphi method, it was shown that 107 of the 133 (80.5%) of the PHIL referrals to the ED were appropriate. Of those referrals judged appropriate, 76 (71.0%) were considered appropriate by all 3 reviewers. By the same method, 158 of 260 (60.8%) control patients were found to be appropriate for the ED. In this group, all 3 reviewers agreed that a patient was appropriate in 114 cases (72.2%). This 20% difference in referral appropriateness was found to be statistically significant ( ² = 14.64; P < .001; Table 3). The calculated odds ratio was 2.66 (95% confidence limits: 1.58 and 4.50).

In an examination of potential confounding factors, it was found that the patients' insurance status had an effect on ED visit appropriateness. Specifically, among patients that had private forms of insurance, the PHIL group (n = 106) was rated appropriate 84.9% of the time compared with 68.4% for the control group (n = 98). This difference was found to be statistically significant (² = 6.95; P < .01) with a calculated odds ratio of 2.60 (95% confidence limits: 1.25 and 5.45; Table 4). In contrast, patients with Medicaid or no insurance, the PHIL group (n = 27) was 63.0% appropriate and the control group (n = 162) was appropriate in 56.2% of the patients. Although a 6.8% difference was found, the higher rate of appropriate ED referrals in the PHIL group did not achieve statistical significance (² = .20; P = .65; Table 5). The adjusted odds ratio, taking insurance status into account, was 1.96 (95% confidence interval: 1.28,2.99).

	DISCUSSION

Top Abstract Methods Results Discussion Conclusion References

Telephone triage systems are becoming increasingly popular in pediatrics. As this area of pediatrics continues to expand, it is important to critically evaluate these systems. To our knowledge, only 1 study has examined the appropriateness of referrals to a pediatric ED by a telephone triage system.² The study described here attempts to apply more rigorous criteria to the ED referrals made by the telephone triage system at our institution.

By using the Delphi method, we were able to show that PHIL had an 80.5% appropriate referral rate during the study. This compared favorably to the 60.8% appropriateness for the control group, which represented all other sources of referral (self, physicians, etc). In its current form, the ED record does not distinguish source of referral. The usefulness of a system that so efficiently acts as a gatekeeper for health care resources is obvious. Because telephone triage systems by design may over-refer patients, it is not clear what the ideal referral rate would be. In over 21/2 years of operation (127 523 total patients), there have been no reported adverse outcomes in patients that PHIL managed without ED referral, so it is unlikely that an ideal rate would be much less than that observed. How much higher than 80% appropriateness can be achieved while maintaining a margin of safety is unknown.

The finding that patients with Medicaid or no insurance had considerably lower rates of appropriateness in both groups was somewhat surprising. In the study population, PHIL did have a 6.8% higher appropriateness rate, but this was not statistically significant. Stratification by insurance status did result in a small sample size (of Medicaid or no insurance) in the PHIL group (n = 27). The observed findings may represent an access to care issue. Although the medical literature does not specifically address access to care in regards to telephone triage, there are many studies that attempt to examine the use of the ED as a routine (ie, nonemergent) source of health care by people with no insurance or public aid (Medicare/Medicaid). Pane et al³ found that public aid/self-pay insurance status was significantly associated with use of the ED for routine care. It should be noted that this study included patients of all ages and only 25% were <20 years old. A later study that involved only children found that there was no association between insurance status and ED use for routine sick care, once demographic and socioeconomic factors were taken into account.⁴ Yet another study suggests that patients with private forms of insurance are more likely to use the ED for nonemergency problems than are patients with Medicaid or no insurance.⁵ Medical appropriateness was higher among all private insured groups compared with the Medicaid group in our study. Even if insurance status was accounted for, the case group had twice the odds of being appropriate as the control group. Further study of Medicaid and no insurance patients served by PHIL is needed to provide more precise estimates of the impact of PHIL on ED appropriateness in this subgroup.

Several limitations to our study should be noted: 1. Although attempts were made to establish rigid criteria for appropriateness, in several cases the judgment of appropriate versus nonappropriate was subjective. 2. The demographic differences between the case and control groups, specifically race and insurance status, certainly influenced the results. However, stratified analyses based on insurance status enabled us to calculate an adjusted odds ratio that better quantified the high rate of appropriateness among PHIL-referred patients. 3. Not all patients referred to the ED follow this recommendation. During the study period, 15 patients (10%) referred in by PHIL did not come to the ED to which they were referred. Because it is not known what happened to these patients, they were excluded from the study. Similarly, there is no easily accessible corresponding group among the controls. 4. Second-level triage by the on-call physician was solicited by PHIL for a small percentage of these referrals (~6%). Initial ED referrals that were changed to a next day office appointment or home care by the on-call physician were not included in this study. Although there are now published guidelines for implementation and administration of telephone triage systems,⁶ the findings in this study may not extend to other systems. Perhaps if the criteria used in this study are applied to other systems, then this matter may become clearer.

	CONCLUSION

Top Abstract Methods Results Discussion Conclusion References

Based on the findings of this study, the PHIL had an estimated 33% higher appropriate ED referral rate than the control group. This estimated effect held true for patients with private forms of insurance. The effect was not as pronounced in patients with Medicaid or no insurance and additional study is required for a more precise estimate of this effect. This evidence supports telephone triage as an efficient gatekeeper for pediatric ED referrals.

	FOOTNOTES

Abstract presented at the Southern Society for Pediatric Research Annual Meeting; February 8, 1998; New Orleans, LA, and at the American Academy of Pediatrics Annual Meeting, Section on Emergency Medicine; October 16, 1998; San Francisco, CA.

Received for publication Jun 23, 1999; accepted Oct 18, 1999.

Reprint requests to (J.W.B) Children's Hospital, Division of Emergency Medicine, 1600 7th Ave S, Birmingham, AL 35233. E-mail: jbarber{at}peds.uab.edu

	ABBREVIATIONS

PHIL, Pediatric Health Information Line; ED, emergency department.

	REFERENCES

Top Abstract Methods Results Discussion Conclusion References

1. National Health Enhancement Systems, Inc. Centramax M. Phoenix, AZ: National Health Enhancement Systems, Inc; 1993
2. Poole SR, Schmitt BD, Carruth T, Peterson-Smith T, Slusarski M After-hours telephone coverage: the application of an area-wide telephone triage and advice system for pediatric practices. Pediatrics 1987; 80:28-31 [Abstract/Free Full Text]
3. Pane GA, Farner MC, Salness KA Health care access problems of medically indigent emergency department walk-in patients. Ann Emerg Med 1991; 20:730-733 [CrossRef][Medline]
4. Halfon N, Newacheck PW, Wood DL, St Peter RF Routine emergency department use for sick care by children in the United States. Pediatrics 1996; 98:28-34 [Abstract/Free Full Text]
5. Baker DW, Stevens CD, Brook RH Determinants of emergency department use: are race and ethnicity important? Ann Emerg Med 1996; 28:677-682 [CrossRef][Medline]
6. Hertz A, Gitterman B, Sherman H. Pediatric Call Centers and the Practice of Telephone Triage and Advice: Critical Success Factors. Elk Grove Village, IL: American Academy of Pediatrics, Provisional Section on Pediatric Telephone Care and the Committee on Practice and Ambulatory Medicine; 1998