PEDIATRICS Vol. 107 No. 5 May 2001, pp. 1133-1137

Detecting Suicide Risk in a Pediatric Emergency Department: Development of a Brief Screening Tool

Lisa M. Horowitz, PhD, MPH^*, , Philip S. Wang, MD, DrPH^§, , Gerald P. Koocher, PhD^*, Barbara H. Burr, MD, MPH^*, Mary Fallon Smith, RN, MSN^¶, Susan Klavon, MSSW, and Paul D. Cleary, PhD

From the ^* Department of Psychiatry, Children's Hospital/Harvard Medical School, Boston, Massachusetts; the  Department of Quality Improvement, Children's Hospital/Harvard Medical School, Boston, Massachusetts; the ^§ Division of Pharmacoepidemiology and Pharmacoeconomics, Brigham and Women's Hospital/Harvard Medical School, Boston, Massachusetts; the  Department of Health Care Policy, Harvard Medical School, Boston, Massachusetts; and the ^¶ Emergency Department, Children's Hospital/Harvard Medical School, Boston, Massachusetts.

	ABSTRACT

Top Abstract Methods Results Discussion References

Objective.  To develop a brief screening tool that will allow emergency department (ED) staff to rapidly and accurately detect suicide risk in child and adolescent patients.

Design.  Cross-sectional survey.

Participants.  One hundred forty-four children and adolescents, mean age of 13.6 years, presenting to an urban university teaching hospital pediatric ED for primarily psychiatric reasons.

Data Collection.  As part of a quality improvement initiative, we developed a 14-item screening survey (the Risk of Suicide Questionnaire [RSQ]) that was administered by a triage nurse to all pediatric mental health patients on admission to the ED. All patients were subsequently administered the 30-item Suicide Ideation Questionnaire (SIQ) by a mental health clinician, which served as the criterion standard assessment of suicidality. Other information collected included demographic and clinical characteristics.

Main Outcome Measures.  Sensitivity, specificity, positive predictive value, negative predictive value (NPV), and area under the receiver operating characteristic curve for responses to individual and combinations of RSQ items, relative to determinations of suicidality by the criterion standard SIQ.

Results.  Four of the items from the RSQ had a predictive c statistic of 0.87, a sensitivity of 0.98, and a NPV of 0.97. Little improvement in predictive ability was obtained by including other RSQ items (c statistic for the most predictive 4-item model = 0.87; c statistic for the model containing all 14 items = 0.90). Among all possible combinations of 4 RSQ items, the combination of items inquiring about current suicidal behavior, past suicidal ideation, past self-destructive behavior, and current stressors yielded the highest sensitivity (0.98), NPV (0.97), and c statistic (0.87), as assessed by the criterion standard SIQ.

Conclusions.  A brief 4-item screening tool can be used by nonmental health clinicians to accurately detect suicidality in children and adolescents who visit an ED. Early and accurate identification of suicidality is a critical first step that could lead to better treatment and improved health outcomes for children and adolescents with mental health concerns.  Key words:  suicidal behavior, suicidal ideation, screening tool, emergency medicine, suicide risk, children, adolescents.

The rate of suicide among adolescents in the United States has tripled since the 1950s^1,2 and now ranks as the third leading cause of death for this age group.³ In addition to completed suicides, suicidal ideation and suicidal attempts are an even greater problem. As many as 12% of children aged 6 to 12 and 53% of adolescents age 13 to 19 have suicidal thoughts⁴; 250 000 adolescents attempt suicide each year⁵; and 8% to 10% of all children in the United States attempt suicide at some point during their childhood.⁶ The Centers for Disease Control Youth Risk Behavior Surveillance System reported that in 1997, 20.5% of high school students had seriously considered attempting suicide, 15.7% had made a plan to attempt suicide, and 7.7% had made an actual suicide attempt.⁷

Despite its high prevalence,^8,9 the risk of suicidal behavior in many children and adolescents is often undetected. Clark¹⁰ investigated adolescents in primary care settings and found that 83% of adolescent patients who had attempted suicide were not recognized as suicidal by their primary care physician. Frankenfield et al¹¹ found that although 47% of primary care physicians reported having an adolescent patient who attempted suicide in the past year, less than half of the physicians surveyed reported that they routinely screened their patients.

Unrecognized suicidality in emergency department (ED) settings is an especially important problem for several reasons. First, increasing numbers of children and adolescents now present to hospital EDs with mental health concerns, primarily self-destructive behavior.^1,12-16 Second, ED staff are increasingly being given the responsibility of triaging children and adolescents with mental health problems to crisis intervention and appropriate follow-up treatments.^17,18 Finally, unrecognized suicidality in the ED is associated with substantial morbidity, potential mortality, and increased health care utilization and costs.¹⁹

Despite the importance of recognizing suicidality in the ED setting, providers with specialized training in recognizing and managing psychiatric crises are often unavailable.²⁰ For example, only 24% of the pediatric hospitals in the United States have mental health specialty services available in the ED.²¹ In the absence of specialists, nonmental health clinicians are responsible for both detecting and managing mental health problems.^17,22 This makes it imperative to have screening tools that can guide ED clinicians in the rapid and accurate detection of suicide risk.

Unfortunately, brief instruments to assess suicidality in children and adolescents in an ED setting are lacking. Some investigators have adapted full-length, adult scales for use with younger populations, but the psychometric characteristics of these measures have not been fully described.^23,24 In addition, existing instruments may not be practical screening tools because of the time required for administration, which can range from 20 to 40 minutes.²⁵ Fewer than 15 minutes of total evaluation time are available before clinical decisions must be made in the majority of ED cases.²⁶ Other suicide assessment instruments are not options for most EDs because they are designed to be administered by trained mental health specialists (ie, Suicidal Behavior Interview²³) or require cumbersome computations by clinicians (ie, Suicide Intent Scale²⁷).

The first aim of the present project was to develop a screening tool to assess suicide risk in children and adolescents. Given its intended use in the ED setting, we sought items that could identify children who were imminently at risk for self-destructive behavior,^23,24,28 rather than items that predict future behavior.^23,29 An additional aim was to identify the smallest number of items that could accurately identify suicidal youths to ensure that the screening tool is practical to administer and not burdensome to ED clinicians. Finally, among possible psychometric properties, we sought to develop a screening instrument with high sensitivity, given the relative importance of detecting children and adolescents at high risk and the potentially devastating consequences of not doing so. This project was part of a quality improvement initiative, and was developed as the first step of implementing a clinical practice guideline for patients at risk for suicide in the ED.

	METHODS

Top Abstract Methods Results Discussion References

Population

Participants consisted of 155 consecutive children and adolescents arriving in the ED of a major tertiary care teaching hospital in Boston between 1997 and 1998 with a chief complaint judged to be psychiatric in nature by the triage nurse. Parents were asked to give permission for their child to complete the gold standard survey. Five children judged to have severe cognitive impairments that interfered with completing the verbal assessment tools were excluded from the project. Four children were excluded because of missing data on either the screening tool or the gold standard assessments. One adolescent refused to participate and was excluded from the results.

Instruments

Risk of Suicide Questionnaire (RSQ) We identified 14 potential screening questions from several sources, including published literature on adolescent suicide risk, interviews with senior mental health clinicians, and items from the Centers for Disease Control and Prevention Youth Risk Behavior Survey.⁷ Questions were reviewed and revised by a panel of mental health clinicians, health services researchers, and survey methodologists for use among children and adolescents presenting to an ED. The adapted items were then pilot-tested among several pediatric ED clinicians and mental health specialists, as well as a sample of child and adolescent psychiatric inpatients and nonpatients, for appropriateness, comprehensibility, and ease of administration. All items were phrased in the form of a question with possible responses of "yes," "no," or "no response." If the child refused, then the accompanying parent or guardian was given the opportunity to answer.

Suicidal Ideation Questionnaire (SIQ)

The SIQ³⁰ was used as a criterion standard to validate the RSQ. The SIQ is a 30-item questionnaire that assesses the frequency of suicidal thoughts in adolescents. Adolescents are asked to rate the frequency with which a cognition occurs on a 7-point scale ranging from "almost every day" to "never." A cutoff score is used to judge the severity level of suicide ideation that requires additional psychiatric evaluation. For children below the 10th grade level, the SIQ-JR, similar to the SIQ but shorter in length (15 items), was administered. A total score of 41 or above was considered significant (31 or above for the SIQ-JR). The SIQ has demonstrated a high reliability (r = 0.97 [SIQ], r = 0.94 [SIQ-JR]) and evidence of validity.^30,31

Other Covariates We also collected information on sociodemographic characteristics (age, race, gender), clinical features (ie, diagnoses, previous suicide attempts, psychiatric history), and other health care utilization measures (ie, medications used, previous psychiatric hospital admissions).

Procedures

Each child/adolescent was administered all 14 potential screening items of the RSQ by a triage nurse in the ED. Once the patient was escorted to an examination room, the triage nurse paged a member of the psychology team, who administered all SIQ items while blind to the patient's RSQ results. Children were told that their answers would be shared with the team of staff members treating them (ie, pediatrician, nurse, mental health clinician). After assessing the patient, the psychology team reported the results to the ED staff members involved in the case.

Analyses

We calculated the frequencies of positive responses to all RSQ and SIQ items as well as the frequencies of all demographic, clinical, and health care utilization characteristics for this patient population. To assess the validity of the individual RSQ items, we examined the agreement between individual RSQ items and suicidality as assessed by the criterion standard SIQ, with the chance corrected kappa statistic.³²

To assess the predictive ability of individual and combinations of RSQ screening questions, univariate and multivariable logistic regression models were constructed of the likelihood of being assessed as positively suicidal by the criterion standard SIQ. Models were estimated using the logistic procedure of SAS Version 6.12.³³ We evaluated all possible combinations of the 14 RSQ items. A positive response to any one of the questions on the RSQ constituted a positive screen for suicide risk. The predictive ability of each model was assessed using the c statistic, which represents the area under the receiver operating characteristic curve.^34,35

To identify the optimal combination of RSQ items for use in a brief screening tool, we calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and c statistic for all possible combinations of 4 RSQ items (the number of items at which little additional predictive value was gained with inclusion of more items). Because of the clinical significance and relative importance of not misclassifying suicidal youths as false-negatives, we also identified the proportion of suicidal children (as determined by the criterion standard SIQ), who would have been undetected by each combination of RSQ items; PPVs and NPVs were calculated for each combination.

	RESULTS

Top Abstract Methods Results Discussion References

The final sample consisted of 144 patients, 78 females and 66 males, with a mean age of 13.6 years, and a standard deviation of 2.5. Seventy-five percent of the sample was between 11 and 16 years of age. The sample was predominantly white (49%) and black (26%). The most common postevaluation diagnosis was some type of depressive disorder (see Table 1). The characteristics of the study population were found to be similar to the total population of children and adolescents visiting the ED for mental health concerns.

Agreement between individual RSQ items and suicidality as determined by the criterion standard SIQ was fair to poor,³² with kappas ranging from 0.54 to 0.02 (see Table 2). Table 3 presents the abilities of combinations of RSQ items to predict being classified as positive by the criterion standard SIQ. Little improvement in predictive ability was obtained beyond the inclusion of 4 RSQ items. For example, the c statistic for the most predictive 4-item model was 0.87 while the c statistic for a model containing all 14 RSQ items was 0.90. Table 3 also presents the sensitivity, specificity, PPV, and NPV of the most predictive models containing all possible numbers of RSQ items. Again, there was little to no improvement in either sensitivity or NPV with inclusion of >4 RSQ items.

Because little improvement in predictive ability, sensitivity, or NPV were gained beyond the addition of 4 RSQ items and because of the need to keep any ED screening tool brief, we focused on the psychometric properties of all 4-item combinations of RSQ items. The sensitivity, specificity, PPV, NPV, and c statistic of the 10 most predictive models containing 4 RSQ items are presented in Table 4. Responses to the combination of item 1 (current suicidal behavior), item 5 (past suicidal ideation), item 8 (past self-destructive behavior), and item 13 (current stressors), yielded the highest sensitivity (0.98), NPV (0.97), and overall prediction of suicidality as assessed by the criterion standard SIQ (c statistic = 0.87). The combination of 4 RSQ items that included item 1 (current suicidal behavior), item 5 (past suicidal ideation), item 9 (past thoughts of self-destructive behavior), and item 13 (current stressors), was observed to yield the next highest values of sensitivity (0.97), NPV (0.94), and predictive ability (c statistic = 0.869).

	DISCUSSION

Top Abstract Methods Results Discussion References

The suicide risk of many children and adolescents goes undetected in health care settings.^10,36 Because of the growing influx of pediatric patients with mental health problems into medical EDs and the diminishing availability of mental health specialty resources, the responsibility for detecting and properly managing suicide risk will increasingly fall on nonpsychiatric ED clinicians.¹⁷ These factors coupled with the ever-increasing time pressures on busy ED staff have made it imperative to provide nonmental health clinicians with practical and accurate tools that can help them rapidly evaluate suicide risk.

Results of this project suggest that nonmental health clinicians can successfully detect suicidality in children and adolescents seeking treatment in an ED by administering a brief 4-item screening tool. The 4 items selected by our analyses appear to have good content validity and they assess major facets of suicide risk: present and past thoughts of suicide, prior self-destructive behavior, and current stressors. Each has been identified by other studies to be an important risk factor for suicidal behavior.^23,37 In addition, good accuracy was demonstrated using 4 items, with only small additional increments in predictive ability with the addition of more items. Responses to these 4 items administered in the triage phase identified 98% of children at risk for suicide, as assessed by a much longer criterion standard instrument administered by a mental health clinician.

In addition to its accuracy and the brief time it takes to administer (<2 minutes), there are other features of this brief screening tool that makes it advantageous in ED settings. Evaluations of suicidality may be particularly stressful or avoided by clinicians who lack formal mental health training, do not have confidence in their psychological assessment and intervention skills, or are often uncomfortable treating this patient population.³⁸ However, with proper structured tools to guide them, nonmental health clinicians can increase their confidence and lower barriers to asking about suicidality. In poststudy focus groups, nurses reported a high level of satisfaction with the screening tool. Moreover, nurses who had been working in the ED before the tool was created reported a significant decrease in stress when managing psychiatric patients. Nurses reported that having the screening tool was much preferred to the previous method of judging by intuition when and how to ask about suicidal behavior (L. Horowitz, personal communication with ED nursing staff, February 1999).

As for patient and parent reactions to the screening tool, anecdotal information collected suggested a high degree of satisfaction with the suicidal symptom inquiry. For example, 1 adolescent female stated that being asked these questions allowed her to feel it was acceptable to begin discussing suicidal thoughts that she had kept to herself for years. In addition, parents of child patients often described their relief that a clinician was delving into a topic that they feared discussing with their children.

Findings from this project should be interpreted with the following limitations in mind. First, the sample size in which the screening tool was developed is relatively small. Thus, it will be important to replicate these results in a larger study. Second, it is possible that psychometric performance of the screening tool observed in our population of patients, arriving for primarily psychiatric problems in an urban pediatric teaching hospital, is not generalizable to other EDs or other clinical populations. For example, the prevalence of suicidality in our patient population was 0.44; it is not clear how the performance of the items either chosen for the screening tool, or not chosen, would vary in populations with different prevalences of suicidality. Again, confirmation of our findings through study of additional populations is needed. Third, although the criterion standard chosen for this project (the SIQ) has proven validity and reliability and identifies children who are in need of additional psychiatric evaluation, it primarily assesses suicidal ideation and may not necessarily be predictive of suicidal behavior. Finally, in our attempt to maximize the sensitivity of our screening tool, we may also have developed a tool that identifies children who are at relatively low risk as suicidal. Although the practical consequences of such a limitation include the possibility that some children will be needlessly subjected to additional evaluation and possibly intervention, they are outweighed by the consequences of failing to detect truly suicidal youths.

Despite potential limitations, the results of this project have significant clinical implications. Nonmental health clinicians were able to identify children and adolescents at risk for suicidality with a rapid screening tool as early as the triage phase of an ED visit. This detection allowed for increased monitoring of high-risk patients to prevent elopement and patient searches to remove dangerous items. Thus, early identification of suicidality can then lead to the timely initiation of multiple interventions, including those in the ED, additional inpatient or outpatient treatment, and interventions in the home or school. Early and accurate ascertainment of suicidality is clearly an important first step that will lead to improvements in the quality of treatment and ultimately the health outcomes of an extremely vulnerable pediatric population.

	ACKNOWLEDGMENTS

Dr Horowitz was supported by the Harvard Pediatric Health Services Research Fellowship Program, Grant T32 HS00063 from the Agency for Healthcare Research and Quality (AHRQ).

We gratefully acknowledge the assistance of Dr Donald Goldmann, Dr Charles Homer, Dr Anthony Spirito, and Kathleen Sultan. We would like to extend a very special thanks to Darcy Raches, Tara Eichner, Don Thompson, Alison Clapp, and the ED nurses of Children's Hospital, Boston.

	FOOTNOTES

Received for publication Jun 30 2000; accepted Dec 11, 2000.

Address correspondence to Lisa M. Horowitz, PhD, MPH, Department of Quality Improvement, Children's Hospital, 300 Longwood Ave, Boston, MA 02115. E- mail: horowitz{at}hub.tch.harvard.edu

	ABBREVIATIONS

ED, emergency department; RSQ, Risk of Suicide Questionnaire; SIQ, Suicide Ideation Questionnaire; PPV, positive predictive value; NPV, negative predictive value.

	REFERENCES

Top Abstract Methods Results Discussion References

1. US Public Health Service. The Surgeon General's Call to Action to Prevent Suicide. Washington, DC: US Public Health Service; 1999
2. Greenhill L, Waslick B Management of suicidal behavior in children and adolescents. Psychiatr Clin North Am 1997; 20:641-666 [CrossRef][Medline]
3. Centers for Disease Control and Prevention Suicide among children, adolescents, and young adults: United States, 1980-1992. MMWR Morb Mortal Wkly Rep. 1995; 44:289-291 [Medline]
4. Pfeffer C Normal children at risk for suicidal behavior: a two-year follow up. J Am Acad Child Adolesc Psychiatry 1988; 27:34-41 [CrossRef][Medline]
5. Schaffer D, Pfeffer C. Practice parameters for the assessment and treatment of children and adolescents with suicidal behavior. J Am Acad Child Adolesc Psychiatry. In press
6. Meehan P Attempted suicide among young adults: progress toward a meaningful estimate of prevalence. Am J Psychiatry 1992; 149:41-44 [Abstract/Free Full Text]
7. Centers for Disease Control and Prevention Youth Risk Behavior Surveillance: United States, 1997. MMWR Morb Mortal Wkly Rep 1998; 47:553
8. Blum R, Beuhring T, Shew M, Bearinger L, Sieving R, Resnick M The effects of race/ethnicity, income, and family structure on adolescent risk behaviors. Am J Public Health 2000; 90:1879-1884 [Abstract/Free Full Text]
9. Spicer R, Miller T Suicide acts in eight states: incidence and case fatality rates by demographics and method. Am J Public Health 2000; 90:1885-1891 [Abstract/Free Full Text]
10. Clark D Suicidal behavior in childhood and adolescence: recent studies and clinical implications. Psychiatr Ann 1993; 23:271-283
11. Frankenfield D, Keyl P, Gielen A, Wissow L, Werthamer L, Baker S Adolescent patients-healthy or hurting? Missed opportunity to screen for suicide risk in the primary care setting. Arch Pediatr Adolesc Med 2000; 154:162-168 [Abstract/Free Full Text]
12. Julien A. Children at risk stranded in the ER. Hartford Courant. 2000:1
13. Wong D. State cites shortfall in mental care for youths. Boston Globe. 1999:1
14. Schaffer D, Piacentini J. Suicide and suicide attempts. In: Rutter M, Hersov L, Taylor E, eds. Child and Adolescent Psychiatry. 3rd ed. London, United Kingdom: Blackwell Scientific Publications; 1994:407-424
15. Robinson J. Emergencies I. In: Robson KS, ed. Manual of Clinical Child and Adolescent Psychiatry. Washington, DC: American Psychiatric Press; 1994:185-211
16. Rauenhorst J Follow-up of young women who attempt suicide. Dis Nerv System 1972; 33:792-797
17. Press B, Kahn S Management of the suicidal child or adolescent in the emergency department. Curr Opin Pediatr 1997; 9:237-241 [CrossRef][Medline]
18. Rotheram-Borus M, Piacentini J, Van Rossem R, Treatment adherence among Latina female adolescent suicide attempters. Suicide Life Threat Behav 1999; 29:319-331 [Medline]
19. Spirito A, Brown L, Overholser J, Fritz G Attempted suicide in adolescence: a review and critique of the literature. Psychology Rev 1989; 9:335-363
20. Athey J, O'Malley P, Henderson D, Ball J. Emergency medical services for children: beyond lights and sirens. Profess Psychol: Res Pract. 1997;28:464-470
21. US Consumer Product Safety Commission. Hospital-Based Pediatric Emergency Resource Survey. Bethesda, MD: Division of Hazard and Injury Data Systems; 1997
22. Horowitz L, Schreiber M, Hare I, Walker V, Talley A. Psychological Factors in Emergency Medical Services for Children: Abstracts of the Psychological, Behavioral, and Medical Literature. 1st ed. Washington, DC: American Psychological Association; 1999
23. Reynolds W. Development of a semistructured clinical interview for suicidal behaviors in adolescents. Psychol Assess: J Consult Clin Psychol. 1990;2:382-390
24. Beck A, Brown G, Steer R, Dahlsgaard M, Grisham B Suicide ideation at its worst point: a predictor of eventual suicide in psychiatric outpatients. Suicide Life Threat Behav 1999; 29:1-8 [Medline]
25. White G, Murdock R, Richardson G, Ellis G, Development of a tool to assess suicide risk factors in urban adolescents. Adolescence 1990; 25:655-656 [Medline]
26. Baxter S, Chodorkoff B, Underhill R Psychiatric emergencies: dispositional determinants and the validity of the decision to admit. Am J Psychiatry 1968; 124:1542-1546 [Abstract/Free Full Text]
27. Beck A, Schuyler D, Herman I. Development of suicidal intent scales. In: Beck A, Resnik H, Lettieri D, eds. The Prediction of Suicide. Bowie, MD: Charles Press; 1974:45-56
28. Demaso D, Ross L, Beardslee W Depressive disorders and suicidal intent in adolescent suicide attempters. Dev Behav Pediatr 1994; 15:74-77 [Medline]
29. Schaffer D, Garland A, Gould M, Fisher P, Trautman P Preventing teenage suicide: a critical review. J Am Acad Child Adolesc Psychiatry 1988; 27:675-687 [Medline]
30. Reynolds W. Suicide Ideation Questionnaire. Odessa, FL: Psychological Assessment Resources; 1987
31. Reynolds W Assessment of suicidal intent in inner-city children and young adolescents: reliability and validity of the Suicide Ideation Question-Jr. School Psychol Rev 1999; 28:17-30
32. Fleiss J. Statistical Methods for Rates and Proportions. New York, NY: John Wiley; 1981
33. SAS Institute Inc. SAS User's Guide, Version 6.12. Cary, NC: SAS Institute Inc; 1998
34. Hanley J, McNeil B A method of comparing the areas under the receiver operating characteristic curves. Radiology 1983; 148:839-843 [Abstract/Free Full Text]
35. Hanley J, McNeil B The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143:29-36 [Abstract/Free Full Text]
36. McManus B, Kruesi M, Dontes A, Defazio C, Piotrowski J, Woodward P Child and adolescent suicide attempts: an opportunity for emergency departments to provide injury prevention education. Am J Emerg Med 1997; 15:357-360
37. Rotherham-Borus M Evaluation of suicide risk among youths in community settings. Suicide Life Threat Behav 1989; 19:108-119 [Medline]
38. Whitworth R Is your patient suicidal? Can Nurse 1984; 80:40-42 [Medline]