BACKGROUND AND OBJECTIVE: The American Academy of Pediatrics called for action for improved screening of mental health issues in the emergency department (ED). We developed the rapid screening tool home, education, activities/peers, drugs/alcohol, suicidality, emotions/behavior, discharge resources (HEADS-ED), which is a modification of “HEADS,” a mnemonic widely used to obtain a psychosocial history in adolescents. The reliability and validity of the tool and its potential for use as a screening measure are presented.
METHODS: ED patients presenting with mental health concerns from March 1 to May 30, 2011 were included. Crisis intervention workers completed the HEADS-ED and the Child and Adolescent Needs and Strengths-Mental Health tool (CANS MH) and patients completed the Children’s Depression Inventory (CDI). Interrater reliability was assessed by using a second HEADS-ED rater for 20% of the sample.
RESULTS: A total of 313 patients were included, mean age was 14.3 (SD 2.63), and there were 182 females (58.1%). Interrater reliability was 0.785 (P < .001). Correlations were computed for each HEADS-ED category and items from the CANS MH and the CDI. Correlations ranged from r = 0.17, P < .05 to r = 0.89, P < .000. The HEADS-ED also predicted psychiatric consult and admission to inpatient psychiatry (sensitivity of 82% and a specificity of 87%; area under the receiver operator characteristic curve of 0.82, P < .01).
CONCLUSIONS: The results provide evidence to support the psychometric properties of the HEADS-ED. The study shows promising results for use in ED decision-making for pediatric patients with mental health concerns.
- ANOVA —
- analysis of variance
- CANS-MH —
- Child and Adolescent Needs and Strengths-Mental Health tool
- CDI —
- Children’s Depression Inventory
- CHEO —
- Children’s Hospital of Eastern Ontario
- CIW —
- crisis intervention worker
- ED —
- emergency department
- HEADS-ED —
- home, education, activities/peers, drugs/alcohol, suicidality, emotions/behavior, discharge resources
- ROC —
- receiver operating characteristic
What’s Known on This Subject:
The American Academy of Pediatrics prioritized detection of mental illness in children presenting to emergency departments (ED) by using standardized clinical tools. Only a minority of ED physicians indicate that they use evidence-based screening methods to assess mental health concerns.
What This Study Adds:
This study presents the psychometrics of the HEADS ED (home, education, activities/peers, drugs/alcohol, suicidality, emotions/behavior, discharge resources), a brief, standardized screening tool for pediatric EDs. This tool ensures key information is obtained for decision-making, determining acuity level, and areas of need.
Pediatric mental health presentations to the emergency department (ED) in Canada are increasing (15% from 2003 to 2006)1 mirroring increases in EDs within the United States.2–4 The ED has become a mental health “safety net”5 but is being stretched to its limit.6 It is not surprising that the Committee on Pediatric Emergency Medicine7 and the American Academy of Pediatrics5 identified as a priority the need to better detect mental illness in children presenting to the ED by using standardized clinical tools. Similarly, Habis and colleagues8 found that only a small minority (9%) of ED physicians indicate that they use evidence-based screening methods to assess mental health concerns; a majority of the physicians (62%) identify the lack of an available tool as a significant barrier. Such a tool would need to be brief, easily administered, and help guide clinical decision-making. Currently, there is no standard of practice or tool used to guide the assessment and disposition of mental health concerns for pediatric ED patients.2,7,9
Rather than developing a new assessment tool, the “HEADS interview” may be able to provide a relevant assessment platform for evaluating pediatric mental health problems in the ED. The HEADS is a mnemonic widely used in general medical outpatient settings to obtain a psychosocial history in adolescents.10,11 It was designed to be used by physicians as a way to organize the psychosocial history during routine adolescent visits. Although variations exist within the literature describing the HEADS mnemonic (eg, HEADDS, HEEADSSS), there is a commonality throughout. The mnemonic generally stands for key areas such as home, education, (eating), activities/ambition, drugs and drinking, sexuality, suicide and depression, safety.10–15 Its popularity is twofold; it eases into the more challenging personal questions, and the mnemonic allows it to remain memorable. Finally, because the HEADS mnemonic is already well known, it requires minimal physician training and can be efficiently completed in the ED.
The HEADS was modified to HEADS-ED (freely available on request) as a rapid mental health screening tool to be used in the ED setting. The tool contains 7 items (Home, Education, Activities and peers, Drugs and alcohol, Suicidality, Emotions and behaviors, Discharge resources) with an embedded scoring system with points associated for each variable (0 = no clinical action needed; 1 = needs clinical action but not immediately; and 2 = needs immediate clinical action). The HEADS-ED also includes discharge resources, because these resources are often a consideration when discharging someone with moderate mental health issues from the ED. Thus, the HEADS-ED includes the main domains of the HEADS but also expands it briefly to include discharge planning and a guided clinical severity/scoring system. Therefore, it provides structure and guides the clinician through the assessment/screening process.
The goal of this study is to evaluate the psychometric properties of the tool including reliability, construct, and criterion-related validity.
Recruitment of patients took place in the ED at the Children’s Hospital of Eastern Ontario (CHEO), in Ottawa, Ontario, Canada, a tertiary care children’s hospital with an annual ED census of 60 000 children, with ∼2200 mental health visits per year. The hospital serves a population of 1 100 300, with 13.1% consisting of visible minorities, and 10.8% considered to be of low income.16 Approximately two-thirds of mental health visits were seen by the crisis intervention workers (CIWs) and the remainder were seen by ED physicians. Patients with mental health concerns who do not require immediate medical or surgical intervention are directly triaged to the CIWs. The CIWs then assess the patients and are able to discharge them from the ED or refer them to the psychiatrist for consultation.
All children and youth presenting to the ED from March 1, 2011 to May 31, 2011 with a mental health concern seen by a CIW during the ED visit were included in the study. The study was approved by the hospital’s research ethics board.
The HEADS-ED Tool
This tool was designed by the multidisciplinary research team to match the “HEADS” interview as closely as possible while adapting it for ED use and adding a scoring component to assist with decision-making. Each letter stands for a specific component of the patient history. H is for home; E, education; A, activities and peers; D, drugs and alcohol; E, emotions and behavior; D, discharge resources. Three levels of scoring were denoted in this tool: 0, 1, and 2. These levels were developed from the communimetrics theory of measurement17 which emphasizes the use of nonarbitrary levels that translate immediately into action. The action levels for the HEADS-ED are designed to be consistent with triage decision-making in the ED: no action needed (0); action needed, but not immediately (1); and immediate action required (2). This tool was purposely designed to be straightforward and to not require additional training.
Child and Adolescent Needs and Strengths-Mental Health Tool
The Child and Adolescent Needs and Strengths-Mental Health tool (CANS-MH 3.0)18 integrates information concerning individual needs and strengths of children and youth with mental health challenges. This tool is being used in the CHEO crisis program and is useful for decision-making and communication. The CANS-MH 3.0 is known as a communimetric measure because individual items use anchors that define action levels: (0) no evidence: there is no need for action; (1) watchful waiting/prevention, this need should be monitored, or efforts to prevent it from returning or getting worse should be initiated; (2) action: an intervention is required because the need is interfering in some notable way with the individual’s, family’s, or community’s functioning; (3) immediate/intensive action: this need is either dangerous or disabling. Therefore, a score of a 2 or 3 would indicate a need for service. The CHEO crisis program uses a 23-item version with items key for ED decision-making (primarily related to the decision to admit or discharge the patient). The CANS-MH 3.0 has been shown to be reliable at the item level so the reliability of the tool is unaffected by selecting a subset of target items.19 The interrater reliability between CHEO crisis staff was established at 0.82 and yearly recertification is required. Interrater reliability by using vignettes across studies averages 0.74, whereas interrater reliability is 0.85 when raters use case records/current cases.18 The previous version of the CANS had demonstrated good validity19; it was significantly correlated (r = 0.63 to r = 0.72) with an independently assessed Child and Adolescent Functional Assessment Scale.20 In addition, the total scores on the dimensions of the tool reliably distinguished level of care received.21
Children’s Depression Inventory
The Children’s Depression Inventory (CDI)22 is a self-report measure of depressive symptoms for youth aged 7 to 17 years. Test-retest reliabilities vary depending on the time interval between assessments. Overall, the CDI demonstrates acceptable levels of stability (2 weeks, 0.82; 3 weeks, 0.74–0.83). This measure is well validated with good explanatory and predictive value and correlates well to other similar scales administered concurrently.22
For each patient triaged directly to the CIW, the CIW documented clinical information by using both the HEADS-ED tool and the CANS-MH 3.0. This process then allowed us to compare the brief HEADS-ED tool with the more comprehensive CANS-MH 3.0. Furthermore, participants over the age of 7 were asked to complete the CDI. To assess for interrater reliability, a research assistant trained and qualified to complete the CANS MH 3.0 reviewed the chart notes for a random set of 20% of the patients and completed the HEADS-ED tool.
The data were analyzed by using SPSS statistics version 19.0 (SPSS Inc, Chicago, IL). Descriptive statistics were obtained by frequency analysis and measures of central tendency. χ2 analysis was used to examine the proportion of patients referred for consult or admitted by HEADS-ED item and total scores. To test convergent validity, correlations were obtained between HEADS-ED categories and the CANS-MH 3.0. Analyses were performed for correlations of both individual and grouped items. For example, the HEADS-ED “home” history may be theoretically related to the CANS-MH 3.0 items: parent-child relational problems, safety, supervision, family functioning, and family strength; these items were correlated individually and as a group. In addition, correlations were obtained between total scaled score of the CDI and Subscale t score from the CDI to the related items from the HEADS-ED. Cross-tabs were used to establish sensitivity and specificity, and receiver operating characteristic (ROC) curves were analyzed to determine the predictive accuracy of the tool for screening for admission. Intraclass correlations were used to assess interrater reliability.
A total of 378 patients were seen in the crisis intervention program at the CHEO ED from March 1, 2011 to May 29, 2011. The crisis workers completed HEADS-ED and CANS-MH 3.0 tools for 313 patients (83% enrollment), leaving 65 patients without completed documentation. The mean age of the sample was 14.3 years (SD 2.63; range 4–17; median 15) with the majority being female n = 182 (58.1%). Of these patients, 181 (61%) reported seeing a mental health professional, and 161 (51.4%) were taking medications. No significant differences were found between those that received documentation and those that did not (age, F [13, 378] = 1.644; gender, F [1, 378] = 0.461, all P values were >.05).
Table 1 presents the frequency distribution of each item of the HEADS-ED tool for the sample. As can be seen from the table, the sample of youth is distributed throughout the tool. Those youth who present with suicidality and emotion/behavior concerns are more likely to need immediate action in comparison with problems with home, education, activities/peers, and also alcohol and drugs.
The overall interrater reliability analysis indicated strong agreement between raters (r = 0.785). Interrater reliability is preferred, since internal consistency is not relevant for communimetric measures because individual items measure different aspects of levels of action and the items are not chosen to necessarily correlate with one another.17Table 2 presents the results of the single-item reliability that ranges from good 0.57 to excellent 0.90.23
Concurrent validity was assessed by examining the relationship between the HEADS-ED, CDI, and CANS-MH 3.0. Tables 3 and 4 present the correlation between the measures. Because the HEADS-ED captures a broad range of items from the CANS-MH 3.0, we first combined those CANS-MH 3.0 items theoretically related to the individual HEADS-ED items (ie, combined CANS items school achievement and attendance scores) and compared them with the corresponding HEADS-ED item (ie, education). As can be seen in Table 3, the HEADS-ED items were significantly correlated with corresponding CANS scores.
In Table 4, the correlations between the HEADS-ED and CDI are presented. The HEADS-ED items of suicidality, activity/peers, and emotion/behavior were significantly correlated with the total CDI scaled score and subscales t score. The HEADS-ED suicidality item was also correlated with the youth’s report on the single CDI item (item 9 on the CDI) directly questioning the patient about their suicidal intentions (r = 0.57, P < .05).
We examined the predictive validity of the HEADS-ED by determining how well the tool predicted consultation for a full psychiatric assessment and subsequent admission to inpatient care. Table 5 presents the frequencies of the 7 items of the HEADS-ED broken down by the need for action (0, 1, 2) for each of the items. Of the total group of 313 youth, nearly half (47.8%) required a full psychiatric assessment, and 21.1% of these youth were ultimately admitted to inpatient care. χ2 analyses were then conducted on each of the items to examine whether there was a difference in the level of need for action and consultation and admission. With the exception of χ2 examining home environment and admission, significant χ2 analyses were obtained on all the remaining items for both requests for consultation and admission to hospital.
The 7 items were then combined to create a total HEADS-ED score (mean 5.4 [SD 2.5], 25th percentile = 4, 50th percentile = 5, 75th percentile = 7). The frequency distribution of the total scores broken down by those who were referred for psychiatric consult and those who were discharged from the ED is illustrated in Fig 1. It is not surprising that, as the total score of the HEADS-ED increases, the proportion of youth who obtain a full psychiatric assessment and admitted to hospital also increases. We then performed an analysis of variance (ANOVA) to determine whether there was a mean difference in scores for those referred for psychiatric consult. The ANOVA was significant F (1304) = 66.19, P < .000. The mean score for those that received a consult was 6.5 (SD = 2.4) and those that did not was 4.4 (SD = 2.1). A second ANOVA was performed to determine whether there was a mean difference in scores for those admitted and those discharged. This also was significant F (1, 305) = 86.56, P < .000. The mean score for those admitted was 7.73 (SD = 2.22; n = 62) and those discharged was 4.82 (SD = 2.19; n = 244).
Based on the analysis from the above ANOVA, we sought to determine if the use of a cutoff score of >7 would be useful to predict admission to hospital or discharge to the community. The use of cross-tab procedure results indicated a sensitivity of 48.5 and a specificity of 87.5. Although the specificity was high, the sensitivity was not adequate and would result in an inordinate number of youth being recommended for hospitalization. After expert review, we revised the decision model regarding hospitalization to suggest admission would not occur without significant levels of suicidal risk. Therefore, we set an algorithm of total HEADS-ED score of >7 and a suicidal risk factor of 2. Again, use of the cross-tab procedure results now indicated sensitivity of 81.8 and a specificity of 87.
Furthermore, to analyze the predictive validity of the HEADS-ED screening tool for admission decisions we analyzed the tool by using the ROC curve procedure.24 Results indicated a significant result with area under the ROC curve of 0.817, P < .01.
The inclusion of a standardized assessment tool in the ED to screen for mental health problems has been advocated by both the American Academy of Pediatrics and the Committee of Pediatric Emergency Medicine.5,25 The HEADS-ED is a rapid mental health screening tool that appears to have the potential to identify level of crisis in children in the ED setting. The result of this current study provides evidence of criterion, concurrent, predictive validity and interrater reliability.
The HEADS-ED is correlated with the youth’s ratings of depression and a more comprehensive clinician rating of mental health strengths and needs. Perhaps more importantly, the study also supports the predictive validity of the tool. The total score from the HEADS-ED indicated statistically and meaningfully different mean scores for those that were referred for a full psychiatric consultation (above the 50th percentile) and for those that were admitted (above the 75th percentile) to an inpatient psychiatric unit. The result of the ROC procedure also supports the validity of the tool for identifying the level of acuity of mental illness in children. The area under the curve indicates strong support,24 demonstrating that the tool had good detection of indicators of admission to inpatient psychiatry.
The HEADS-ED is the first measure to have been developed to date specifically for the ED pediatric mental health population. Other tools exist to assess self-harm risk in the ED,26 and there are still others designed to assess the multidimensional aspects of a crisis presentation and evaluate the patients’ risk,27 including the risk assessment matrix,28,29 which is probably one of the better known tools, although it was developed for adults. However, none of these tools have been successfully incorporated within the ED, and we suspect this is probably due to the extensive training and the lengthy administration time that is required.
The HEADS-ED has the advantage that, first, it uses the HEADS11 mnemonic which is familiar to many physicians and often taught in medical school. It is assumed that emergency physician and health clinicians already possess the skills to inquire about the items (eg, home environment) or level of need (supportive, conflictual, chaotic/dysfunctional) without needing an extensive “instruction manual” or supervised training. The items are clinically intuitive, and therefore do not require further training. These are the types of clinical queries that an ED physician would normally conduct as part of diagnostic assessment. The HEADS-ED is easy to administer, does not require scoring, but, as shown in this article, it has both clinical validity and utility. The tools help guide the mental health assessment as well as provide direction for follow-up services, for example, if the home environment is chaotic, then recommendation for family therapy or parent training can be suggested. Mental health services that are available within the hospital and community can easily be linked to each of the items allowing ED physicians to be even more specific in their recommendations (eg, a local community program that offers addiction services for youth). Our center is currently developing a web-based administration of the HEADS-ED linked electronically to hospital and community services.
The limitations of the current study, including the relatively small sample size drawn from 1 pediatric health center, are being addressed in current and future validation studies. ED physician ratings on the HEADS-ED will be compared with mental health specialist ratings and youth/parent reports of behavioral and emotional functioning. The HEADS-ED will also need to be evaluated in a multisite, pediatric and nonpediatric tertiary center study to increase the sample size and generalizability of the findings. We are optimistic that, given our current findings and the evidence in general on the HEADS,10,11 future results will provide further support of the HEADS-ED.
The HEADS-ED shows promise as a brief, easily administered standardized screening tool useful in directing the interview process to ensure that key information is obtained for decision-making, uncovering the level of crisis, and determining the level of treatment needed. It provides potential for use as a decisional tool to determine referral for psychiatric consultation, admission decisions, and guidance in the selection of services for patients discharged back to the community.
We would like to thank the CHEO crisis intervention team for their significant contribution to this project.
- Accepted March 27, 2012.
- Address correspondence to Mario Cappelli, PhD, Director of Mental Health Research, Children’s Hospital of Eastern Ontario, Ottawa, ON K1H 8L1 Canada. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by the Academic Health Science Centres Alternate Funding Plan Innovation Fund, the Royal Bank of Canada, and Children's Hospital of Eastern Ontario Foundations.
- Mahajan P,
- Alpern ER,
- Grupp-Phelan J,
- et al.,
- Pediatric Emergency Care Applied Research Network (PECARN)
- American Academy of Pediatrics, Committee on Pediatric Emergency Medicine; American College of Emergency Physicians and Pediatric Emergency Medicine Committee,
- Dolan MA,
- Mace SE
- Goldenring J,
- Cohen E
- Goldenring J,
- Rosen D
- Copperman SM
- Reitman D.
- Walling A
- ↵Dall K, Lefebvre M, Pacey M, Sahai V. Champlain LHIN: socio-economic indicators atlas. Ottawa, ON, Canada: Government of Ontario; 2006:26. Available at: www.champlainlhin.on.ca/lhinresearch.aspx?kmensel=e2f22c9a_72_206_132_4. Accessed April 23, 2012
- Lyons JS
- ↵Lyons JS, Bisnaire L, Greenham S, Dods J. The child and adolescent needs and strengths (CAN MH 3.0 manual). 2006. Available at: http://www.praedfoundation.org. Accessed April 23 2012
- Lyons JS
- Kovacs M
- Patel AS,
- Harrison A,
- Bruce-Jones W
- Copyright © 2012 by the American Academy of Pediatrics