Abstract
Objective. To develop a brief screening tool that will allow emergency department (ED) staff to rapidly and accurately detect suicide risk in child and adolescent patients.
Design. Cross-sectional survey.
Participants. One hundred forty-four children and adolescents, mean age of 13.6 years, presenting to an urban university teaching hospital pediatric ED for primarily psychiatric reasons.
Data Collection. As part of a quality improvement initiative, we developed a 14-item screening survey (the Risk of Suicide Questionnaire [RSQ]) that was administered by a triage nurse to all pediatric mental health patients on admission to the ED. All patients were subsequently administered the 30-item Suicide Ideation Questionnaire (SIQ) by a mental health clinician, which served as the criterion standard assessment of suicidality. Other information collected included demographic and clinical characteristics.
Main Outcome Measures. Sensitivity, specificity, positive predictive value, negative predictive value (NPV), and area under the receiver operating characteristic curve for responses to individual and combinations of RSQ items, relative to determinations of suicidality by the criterion standard SIQ.
Results. Four of the items from the RSQ had a predictive c statistic of 0.87, a sensitivity of 0.98, and a NPV of 0.97. Little improvement in predictive ability was obtained by including other RSQ items (c statistic for the most predictive 4-item model = 0.87; cstatistic for the model containing all 14 items = 0.90). Among all possible combinations of 4 RSQ items, the combination of items inquiring about current suicidal behavior, past suicidal ideation, past self-destructive behavior, and current stressors yielded the highest sensitivity (0.98), NPV (0.97), and c statistic (0.87), as assessed by the criterion standard SIQ.
Conclusions. A brief 4-item screening tool can be used by nonmental health clinicians to accurately detect suicidality in children and adolescents who visit an ED. Early and accurate identification of suicidality is a critical first step that could lead to better treatment and improved health outcomes for children and adolescents with mental health concerns.
The rate of suicide among adolescents in the United States has tripled since the 1950s1,2 and now ranks as the third leading cause of death for this age group.3 In addition to completed suicides, suicidal ideation and suicidal attempts are an even greater problem. As many as 12% of children aged 6 to 12 and 53% of adolescents age 13 to 19 have suicidal thoughts4; 250 000 adolescents attempt suicide each year5; and 8% to 10% of all children in the United States attempt suicide at some point during their childhood.6 The Centers for Disease Control Youth Risk Behavior Surveillance System reported that in 1997, 20.5% of high school students had seriously considered attempting suicide, 15.7% had made a plan to attempt suicide, and 7.7% had made an actual suicide attempt.7
Despite its high prevalence,8,9 the risk of suicidal behavior in many children and adolescents is often undetected. Clark10 investigated adolescents in primary care settings and found that 83% of adolescent patients who had attempted suicide were not recognized as suicidal by their primary care physician. Frankenfield et al11 found that although 47% of primary care physicians reported having an adolescent patient who attempted suicide in the past year, less than half of the physicians surveyed reported that they routinely screened their patients.
Unrecognized suicidality in emergency department (ED) settings is an especially important problem for several reasons. First, increasing numbers of children and adolescents now present to hospital EDs with mental health concerns, primarily self-destructive behavior.1,12–16 Second, ED staff are increasingly being given the responsibility of triaging children and adolescents with mental health problems to crisis intervention and appropriate follow-up treatments.17,18 Finally, unrecognized suicidality in the ED is associated with substantial morbidity, potential mortality, and increased health care utilization and costs.19
Despite the importance of recognizing suicidality in the ED setting, providers with specialized training in recognizing and managing psychiatric crises are often unavailable.20 For example, only 24% of the pediatric hospitals in the United States have mental health specialty services available in the ED.21 In the absence of specialists, nonmental health clinicians are responsible for both detecting and managing mental health problems.17,22 This makes it imperative to have screening tools that can guide ED clinicians in the rapid and accurate detection of suicide risk.
Unfortunately, brief instruments to assess suicidality in children and adolescents in an ED setting are lacking. Some investigators have adapted full-length, adult scales for use with younger populations, but the psychometric characteristics of these measures have not been fully described.23,24 In addition, existing instruments may not be practical screening tools because of the time required for administration, which can range from 20 to 40 minutes.25Fewer than 15 minutes of total evaluation time are available before clinical decisions must be made in the majority of ED cases.26 Other suicide assessment instruments are not options for most EDs because they are designed to be administered by trained mental health specialists (ie, Suicidal Behavior Interview23) or require cumbersome computations by clinicians (ie, Suicide Intent Scale27).
The first aim of the present project was to develop a screening tool to assess suicide risk in children and adolescents. Given its intended use in the ED setting, we sought items that could identify children who were imminently at risk for self-destructive behavior,23,24,28 rather than items that predict future behavior.23,29 An additional aim was to identify the smallest number of items that could accurately identify suicidal youths to ensure that the screening tool is practical to administer and not burdensome to ED clinicians. Finally, among possible psychometric properties, we sought to develop a screening instrument with high sensitivity, given the relative importance of detecting children and adolescents at high risk and the potentially devastating consequences of not doing so. This project was part of a quality improvement initiative, and was developed as the first step of implementing a clinical practice guideline for patients at risk for suicide in the ED.
METHODS
Population
Participants consisted of 155 consecutive children and adolescents arriving in the ED of a major tertiary care teaching hospital in Boston between 1997 and 1998 with a chief complaint judged to be psychiatric in nature by the triage nurse. Parents were asked to give permission for their child to complete the gold standard survey. Five children judged to have severe cognitive impairments that interfered with completing the verbal assessment tools were excluded from the project. Four children were excluded because of missing data on either the screening tool or the gold standard assessments. One adolescent refused to participate and was excluded from the results.
Instruments
Risk of Suicide Questionnaire (RSQ)
We identified 14 potential screening questions from several sources, including published literature on adolescent suicide risk, interviews with senior mental health clinicians, and items from the Centers for Disease Control and Prevention Youth Risk Behavior Survey.7 Questions were reviewed and revised by a panel of mental health clinicians, health services researchers, and survey methodologists for use among children and adolescents presenting to an ED. The adapted items were then pilot-tested among several pediatric ED clinicians and mental health specialists, as well as a sample of child and adolescent psychiatric inpatients and nonpatients, for appropriateness, comprehensibility, and ease of administration. All items were phrased in the form of a question with possible responses of “yes,” “no,” or “no response.” If the child refused, then the accompanying parent or guardian was given the opportunity to answer.
Suicidal Ideation Questionnaire (SIQ)
The SIQ30 was used as a criterion standard to validate the RSQ. The SIQ is a 30-item questionnaire that assesses the frequency of suicidal thoughts in adolescents. Adolescents are asked to rate the frequency with which a cognition occurs on a 7-point scale ranging from “almost every day” to “never.” A cutoff score is used to judge the severity level of suicide ideation that requires additional psychiatric evaluation. For children below the 10th grade level, the SIQ-JR, similar to the SIQ but shorter in length (15 items), was administered. A total score of 41 or above was considered significant (31 or above for the SIQ-JR). The SIQ has demonstrated a high reliability (r = 0.97 [SIQ], r = 0.94 [SIQ-JR]) and evidence of validity.30,31
Other Covariates
We also collected information on sociodemographic characteristics (age, race, gender), clinical features (ie, diagnoses, previous suicide attempts, psychiatric history), and other health care utilization measures (ie, medications used, previous psychiatric hospital admissions).
Procedures
Each child/adolescent was administered all 14 potential screening items of the RSQ by a triage nurse in the ED. Once the patient was escorted to an examination room, the triage nurse paged a member of the psychology team, who administered all SIQ items while blind to the patient's RSQ results. Children were told that their answers would be shared with the team of staff members treating them (ie, pediatrician, nurse, mental health clinician). After assessing the patient, the psychology team reported the results to the ED staff members involved in the case.
Analyses
We calculated the frequencies of positive responses to all RSQ and SIQ items as well as the frequencies of all demographic, clinical, and health care utilization characteristics for this patient population. To assess the validity of the individual RSQ items, we examined the agreement between individual RSQ items and suicidality as assessed by the criterion standard SIQ, with the chance corrected kappa statistic.32
To assess the predictive ability of individual and combinations of RSQ screening questions, univariate and multivariable logistic regression models were constructed of the likelihood of being assessed as positively suicidal by the criterion standard SIQ. Models were estimated using the logistic procedure of SAS Version 6.12.33 We evaluated all possible combinations of the 14 RSQ items. A positive response to any one of the questions on the RSQ constituted a positive screen for suicide risk. The predictive ability of each model was assessed using the c statistic, which represents the area under the receiver operating characteristic curve.34,35
To identify the optimal combination of RSQ items for use in a brief screening tool, we calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and c statistic for all possible combinations of 4 RSQ items (the number of items at which little additional predictive value was gained with inclusion of more items). Because of the clinical significance and relative importance of not misclassifying suicidal youths as false-negatives, we also identified the proportion of suicidal children (as determined by the criterion standard SIQ), who would have been undetected by each combination of RSQ items; PPVs and NPVs were calculated for each combination.
RESULTS
The final sample consisted of 144 patients, 78 females and 66 males, with a mean age of 13.6 years, and a standard deviation of 2.5. Seventy-five percent of the sample was between 11 and 16 years of age. The sample was predominantly white (49%) and black (26%). The most common postevaluation diagnosis was some type of depressive disorder (see Table 1). The characteristics of the study population were found to be similar to the total population of children and adolescents visiting the ED for mental health concerns.
Demographic and Clinical Characteristics of the Population (n = 144)
Agreement between individual RSQ items and suicidality as determined by the criterion standard SIQ was fair to poor,32 with kappas ranging from 0.54 to 0.02 (see Table 2).Table 3 presents the abilities of combinations of RSQ items to predict being classified as positive by the criterion standard SIQ. Little improvement in predictive ability was obtained beyond the inclusion of 4 RSQ items. For example, thec statistic for the most predictive 4-item model was 0.87 while the c statistic for a model containing all 14 RSQ items was 0.90. Table 3 also presents the sensitivity, specificity, PPV, and NPV of the most predictive models containing all possible numbers of RSQ items. Again, there was little to no improvement in either sensitivity or NPV with inclusion of >4 RSQ items.
Chance Corrected Agreement Between Screening Items and the Criterion Standard Assessment of Suicide Risk
Predictive Abilities for Combinations of Screening Items
Because little improvement in predictive ability, sensitivity, or NPV were gained beyond the addition of 4 RSQ items and because of the need to keep any ED screening tool brief, we focused on the psychometric properties of all 4-item combinations of RSQ items. The sensitivity, specificity, PPV, NPV, and c statistic of the 10 most predictive models containing 4 RSQ items are presented in Table 4. Responses to the combination of item 1 (current suicidal behavior), item 5 (past suicidal ideation), item 8 (past self-destructive behavior), and item 13 (current stressors), yielded the highest sensitivity (0.98), NPV (0.97), and overall prediction of suicidality as assessed by the criterion standard SIQ (c statistic = 0.87). The combination of 4 RSQ items that included item 1 (current suicidal behavior), item 5 (past suicidal ideation), item 9 (past thoughts of self-destructive behavior), and item 13 (current stressors), was observed to yield the next highest values of sensitivity (0.97), NPV (0.94), and predictive ability (c statistic = 0.869).
Predictive Abilities for Combinations of Four Screening Items
DISCUSSION
The suicide risk of many children and adolescents goes undetected in health care settings.10,36 Because of the growing influx of pediatric patients with mental health problems into medical EDs and the diminishing availability of mental health specialty resources, the responsibility for detecting and properly managing suicide risk will increasingly fall on nonpsychiatric ED clinicians.17 These factors coupled with the ever-increasing time pressures on busy ED staff have made it imperative to provide nonmental health clinicians with practical and accurate tools that can help them rapidly evaluate suicide risk.
Results of this project suggest that nonmental health clinicians can successfully detect suicidality in children and adolescents seeking treatment in an ED by administering a brief 4-item screening tool. The 4 items selected by our analyses appear to have good content validity and they assess major facets of suicide risk: present and past thoughts of suicide, prior self-destructive behavior, and current stressors. Each has been identified by other studies to be an important risk factor for suicidal behavior.23,37 In addition, good accuracy was demonstrated using 4 items, with only small additional increments in predictive ability with the addition of more items. Responses to these 4 items administered in the triage phase identified 98% of children at risk for suicide, as assessed by a much longer criterion standard instrument administered by a mental health clinician.
In addition to its accuracy and the brief time it takes to administer (<2 minutes), there are other features of this brief screening tool that makes it advantageous in ED settings. Evaluations of suicidality may be particularly stressful or avoided by clinicians who lack formal mental health training, do not have confidence in their psychological assessment and intervention skills, or are often uncomfortable treating this patient population.38 However, with proper structured tools to guide them, nonmental health clinicians can increase their confidence and lower barriers to asking about suicidality. In poststudy focus groups, nurses reported a high level of satisfaction with the screening tool. Moreover, nurses who had been working in the ED before the tool was created reported a significant decrease in stress when managing psychiatric patients. Nurses reported that having the screening tool was much preferred to the previous method of judging by intuition when and how to ask about suicidal behavior (L. Horowitz, personal communication with ED nursing staff, February 1999).
As for patient and parent reactions to the screening tool, anecdotal information collected suggested a high degree of satisfaction with the suicidal symptom inquiry. For example, 1 adolescent female stated that being asked these questions allowed her to feel it was acceptable to begin discussing suicidal thoughts that she had kept to herself for years. In addition, parents of child patients often described their relief that a clinician was delving into a topic that they feared discussing with their children.
Findings from this project should be interpreted with the following limitations in mind. First, the sample size in which the screening tool was developed is relatively small. Thus, it will be important to replicate these results in a larger study. Second, it is possible that psychometric performance of the screening tool observed in our population of patients, arriving for primarily psychiatric problems in an urban pediatric teaching hospital, is not generalizable to other EDs or other clinical populations. For example, the prevalence of suicidality in our patient population was 0.44; it is not clear how the performance of the items either chosen for the screening tool, or not chosen, would vary in populations with different prevalences of suicidality. Again, confirmation of our findings through study of additional populations is needed. Third, although the criterion standard chosen for this project (the SIQ) has proven validity and reliability and identifies children who are in need of additional psychiatric evaluation, it primarily assesses suicidal ideation and may not necessarily be predictive of suicidal behavior. Finally, in our attempt to maximize the sensitivity of our screening tool, we may also have developed a tool that identifies children who are at relatively low risk as suicidal. Although the practical consequences of such a limitation include the possibility that some children will be needlessly subjected to additional evaluation and possibly intervention, they are outweighed by the consequences of failing to detect truly suicidal youths.
Despite potential limitations, the results of this project have significant clinical implications. Nonmental health clinicians were able to identify children and adolescents at risk for suicidality with a rapid screening tool as early as the triage phase of an ED visit. This detection allowed for increased monitoring of high-risk patients to prevent elopement and patient searches to remove dangerous items. Thus, early identification of suicidality can then lead to the timely initiation of multiple interventions, including those in the ED, additional inpatient or outpatient treatment, and interventions in the home or school. Early and accurate ascertainment of suicidality is clearly an important first step that will lead to improvements in the quality of treatment and ultimately the health outcomes of an extremely vulnerable pediatric population.
ACKNOWLEDGMENTS
Dr Horowitz was supported by the Harvard Pediatric Health Services Research Fellowship Program, Grant T32 HS00063 from the Agency for Healthcare Research and Quality (AHRQ).
We gratefully acknowledge the assistance of Dr Donald Goldmann, Dr Charles Homer, Dr Anthony Spirito, and Kathleen Sultan. We would like to extend a very special thanks to Darcy Raches, Tara Eichner, Don Thompson, Alison Clapp, and the ED nurses of Children's Hospital, Boston.
- ED =
- emergency department •
- RSQ =
- Risk of Suicide Questionnaire •
- SIQ =
- Suicide Ideation Questionnaire •
- PPV =
- positive predictive value •
- NPV =
- negative predictive value
REFERENCES
- Copyright © 2001 American Academy of Pediatrics
In this issue
Jump to section
Related Articles
Cited By...
- Validation and Feasibility of the ASQ Among Pediatric Medical and Surgical Inpatients
- Children and Young People-Mental Health Safety Assessment Tool (CYP-MH SAT) study: Protocol for the development and psychometric evaluation of an assessment tool to identify immediate risk of self-harm and suicide in children and young people (10-19 years) in acute paediatric hospital settings
- Evaluation and Management of Children With Acute Mental Health or Behavioral Problems. Part II: Recognition of Clinically Challenging Mental Health Related Conditions Presenting With Medical or Uncertain Symptoms
- Pediatric and Adolescent Mental Health Emergencies in the Emergency Medical Services System
- Children who harm themselves: development of a paediatric emergency department triage tool