OBJECTIVE: Suicidal youths are rarely identified in primary care settings. We describe here a care process that includes a computerized screen, colocated social workers, and a coordinated suicide-prevention team at a specialty mental health unit.
PATIENTS AND METHODS: Patients were 1547 youths aged 11 to 20 years seen in an urban primary care system during 2005 and 2006. We performed an observational study of services provided to youths who screened positive for suicidal ideation on a computerized behavioral health screen during visits to pediatric primary care clinics. Data included clinical records, provider notes, and patients' responses to the screen.
RESULTS: A total of 209 (14%) youths reported suicidal thought in the previous month. Suicidal thought was more common among girls, younger youths, substance users, depressed youths, youths who carried weapons, and those who had been in fights; 87% reported at least 1 other serious behavioral health problem. Social workers were able to triage 205 (98%) youths. Triage occurred on the visit day for 193 youths (94%). Mental health evaluations were recommended for 152 (74%) of the triaged youths. Of the 109 subjects referred to a clinic with records accessible for review, 71 (65%) received a mental health service within 6 months.
CONCLUSIONS: Pediatric primary care is a feasible setting in which to screen for suicidal youths and link them with mental health services. Youths who visit primary care clinics are willing to disclose suicidal ideation on a computerized screen. Youths who screen positive for suicide have many associated behavioral health needs. The use of information technology, colocated physician extenders, and a coordinated team on the mental health side can facilitate rapid, personal contact between the family and mental health service providers, and has the potential to overcome barriers to care for youths with suicidal ideation in the primary care setting.
WHAT'S KNOWN ON THIS SUBJECT:
Suicide is a leading cause of adolescent mortality. Screening for suicidal thought is safe and has acceptable psychometrics. Interventions to prevent suicide or treat depression can decrease suicidal behavior. Nevertheless, most primary care practices do not screen for suicidal thoughts.
WHAT THIS STUDY ADDS:
Screening youths for suicidal thought during a primary care visit by using a computerized system that links primary and specialty care can identify youths with many behavioral health risks and connect them to specialty mental health providers.
Suicide was the third leading cause of death for persons aged 10 to 19 years in the United States in 2005,1 comprising 11% of deaths. During 2007, 15% of US high school students reported that they had seriously considered suicide during the previous 12 months.2 Higher suicide rates have been associated with less access to health and mental health services in studies across the life span and in several countries.3,–,5 Primary care is important for suicide prevention, because primary care practices are often the most accessible professional sites in a community. More than 85% of insured youth and 75% of uninsured teenagers are seen in primary care in the United States annually.6 Six of 10 US persons aged 35 years or younger who completed suicide contacted a primary care provider (PCP) in the year before the suicide, and 1 in 5 contacted a PCP in the month before the suicide.7 PCPs are among the most trusted professionals, and the relationships that PCPs develop with their patients can engage patients in addressing chronic problems.8 Many youths report that they are more comfortable discussing risk-taking activity with their PCPs than with specialists.9 Screening adolescents for suicidal thought is safe10 and can be done with acceptable sensitivity and specificity.11 Finally, interventions to increase PCPs' capacity to assess for suicide risk, or to diagnose and treat depressive disorders, have successfully decreased suicidal thought and behavior.12,–,14 Thus, there is an opportunity in primary care to prevent youth suicide.
Despite this fact, most pediatric PCPs do not routinely ask about suicidal thoughts.15 Primary care clinicians are expected to perform many prevention tasks and may lack time to screen for suicidal thought. In previous work,16 however, we showed that computerized screens can be administered in the waiting room before primary care visits, which reduces the time it takes to screen.
Screening, however, is useful only if it is followed by effective care. Here we describe a primary care screening, triage, and referral process for youth with suicidal ideation and examine factors that might explain which at-risk youths received mental health services. We asked 4 questions: (1) Will adolescents report suicidal thought, knowing that a physician will read their answers? (2) What are the characteristics of patients who reported suicidal thought? (3) If an adolescent discloses suicidal thoughts on a primary care screen, how often did the youth later receive a mental health service (not including the screening and triage processes that constituted our intervention)? (4) Does a positive screen predict subsequent health care system contacts for suicidal ideation or behavior?
Data are from 1547 youths aged 11 to 20 years who completed screens on wireless tablet computers16,17 in the waiting rooms of 9 clinics of an urban primary care system from June 2005 through July 2006. The clinics were owned by the Nationwide Children's Hospital (NCH) of Columbus, Ohio, and served a largely minority population.
Youths were administered a screen that asked about substance use, depression, injury risk behaviors, and suicidal thought. Adolescents who were accompanied by a parent or guardian (if younger than 18 years) were approached by either clinic registration staff or research staff in the primary care waiting room and invited to participate in the study. Screening did not occur if the visit was solely for a procedure or emergency care. Recruitment rates for the clinic registration staff were unknown, because clinic workflow did not allow staff to log their unsuccessful approaches to possible recruits. Recruitment rates for the study's 3 research assistants ranged from 60% to 95%.
For those who were younger than 18 years, consent was obtained from the parent or guardian who accompanied the patient, and assent was obtained from the youths. Youths older than 18 years provided consent. We informed patients that information entered on the tablet would be available to both clinical and research staff. Once consent and assent were obtained, the adolescents completed the screen in the waiting room (or the examination room if they were called there before completion). The NCH's institutional review board approved this study.
The computerized screening system was designed not only to detect suicidal thinking but also to coordinate care between the primary care practice and a behavioral health network of the same pediatric health care system. A positive answer to the suicidal-thought question was reported to the PCP [Fig 1 (a)] before the PCP's visit with the family [Fig 1(b)]. The PCP discussed the issue with the family or referred the family to a colocated medical social worker [Fig 1(c)]. The PCP or medical social worker decided whether to refer the youth for mental health care [Fig 1(d)]. Sometimes, they judged that no action was required [Fig 1(e)]. Otherwise, the PCP or medical social worker called a suicide-prevention team of psychiatric social workers at the mental health provider (unless the youth was receiving care from another mental health provider) [Fig 1(f)]. The psychiatric social workers were given reduced clinical responsibilities to make time for on-call work with suicidal patients. A team member would triage the case, expedite the scheduling of a mental health visit for the family [Fig 1(g)], and, if possible, got the visit scheduled while the family was still in the primary care office [Fig 1(h)]. The triage assessment determined if the screen-positive youth was in acute danger of suicide and warranted emergency management. Those youths judged to be at emergent risk were referred for emergency evaluation. Youths who were judged to be at risk but not in need of emergent attention were referred for scheduled follow-up appointments. Occasionally, however, the report did not reach the PCP before he or she met the family [Fig 1(b)]. Therefore, the system also placed an alert [Fig 1(i)] on a secure Web page monitored by the suicide-prevention team. If the team noticed an alert and the primary care office had not contacted the team [Fig 1(j)], then a team member called the primary care office to alert it about the positive screen [Fig 1(k)]. Occasionally, the suicide-prevention team would meet families who had left the primary care office to triage the patient in the community.
The research intervention in this study was a computerized screening, triage, and referral process. We did not intervene to train how physicians responded to screen reports or alerts. All social workers had received training in suicide evaluation and prevention, but their practice was not based on a specific guideline or protocol and varied according to practitioner. Moreover, the change in the social workers' practice concerned their coordination with the primary care offices, their efforts to quickly contact families with positive screens, to triage the situation identified by the screen, and, if necessary, to make a quick referral. The goal was that a positive screen should result in a quick triage and referral, if possible, while the family was still in the primary care office.
Suicidal thought was assessed through a question from the Patient Health Questionnaire for Adolescents (PHQ-A)18: “Has there been a time in the past month when you have had serious thoughts about ending your life?” Items for the other domains of the screen (injury risk, depression, and substance use) were drawn from publicly available, validated measures. Injury risk was measured by using items from the Youth Risk Behavior Survey (YRBS).2 Depressive symptoms were assessed by using the Centers for Epidemiologic Studies Depression Scale for Children (CES-DC), a 20-item depression-screening tool.19,20 The CES-DC has acceptable internal reliability, reasonable test-retest reliability, and moderate concurrent validity for adolescents.21 Substance use was measured by using items from the Comprehensive Addiction Severity Index for Adolescents (CASIA).22
Records of Triage Decisions
The suicide-prevention team maintained a log that described the triage status of each patient who answered yes to the question about suicidal thought, which indicated whether a mental health visit was recommended. In a few cases, this log was incomplete, but we were able to determine the triage status of the patient from a questionnaire completed by the clinician after the visit with the family.
Records of Health Services Related to Suicide
We looked in the NCH's electronic health records for mental health services during the 6 months after the index visit. Although we missed records of mental health services at other institutions, NCH is the largest pediatric health provider in the region and by far the most likely source of specialty mental health services for youths in NCH-managed primary care practices. Finally, we looked for any record of a non–mental health service delivered at NCH that had a code indicating suicidal thought or behavior (Current Procedural Terminology codes E950.X or V62.84).
Figure 2 shows the flow of patients through the process of screening, triage, referral to mental health, and delivery of mental health services. Of the 1547 screened youths, data on the suicidal-thought question were missing for 44 youths (3%), usually because they did not complete the assessment before being called in to see the doctor. Table 1 provides demographic and clinical details about the 1503 youths who answered the suicidal-thought question.
Of these 1503 youths, 209 (14%) reported that they had had serious thoughts about taking their own lives in the previous month. Table 2 lists the factors associated with disclosure of suicidal thought. Patients who reported suicidal thought were more likely to be younger and female, to have used substances or carried weapons in the previous month, and to have been in a fight in the previous year. The strongest predictor, however, was symptoms of depression. When these factors were examined in a logistic regression, reports of suicide were associated with younger age (P < .001), fighting to injury (P = .002), and symptoms of depression (P < .0001). Overall, 182 of 209 (87%) youths who answered yes to the suicidal-thought question also reported at least 1 of the following problems: substance use; carrying a weapon; fighting that resulted in injury; or a CES-DC score of ≥16.
For 205 of the 209 (98%) youths who reported suicidal thought, the social workers spoke with the family and triaged the patient (for the remaining 4 youths we could not determine the disposition of the case). Among the 205 patients, 193 of the contacts (94%) occurred on the day of the primary care visit, 6 on the next day, and 3 within 5 days, and the dates of 3 triage contacts were missing.
On the basis of the triage, mental health evaluations were recommended for 152 of the 205 patients (74%). Table 3 lists the factors associated with recommendations. Patients with worse depression were more likely to be referred, and black youths were less likely to be referred. Referral was also significantly associated with fighting (odds ratio [OR]: 1.88 [95% confidence interval (CI): 1.09–3.24]; P = .024). Use of substances was not associated with referral. Finally, we estimated a logistic regression with referral as a dependent variable and depression, race, and fighting as covariates; depression (P < .001), white race (P = .013), and fighting (P = .026) were associated with a greater likelihood of referral.
Of the 152 patients recommended for mental health evaluations, 109 (72%) were referred to the NCH behavioral health clinics (the remaining 43 were referred to providers outside that system, and we did not determine if they subsequently received services). Of these 109 patients, 71 (65%) received a diagnostic or treatment service in the next 6 months. None of the demographic or clinical variables were associated with whether patients received mental health services.
Finally, how well did the initial screening question predict whether youths were seen later at NCH for a medical service with a Current Procedural Terminology code related to suicidal thoughts or behavior? We found that 10 of 1294 youths (0.8%) who answered no to our screening question and 7 of 209 patients who answered yes (3.4%) were seen at NCH during the 6 months after the index visit for a medical service with a code related to suicidal thought or behavior (OR: 4.50 [95% CI: 1.67–11.82]; P = .001).
Adolescents will disclose suicidal thoughts when screened in primary care, even when they know that the physician will see the answer. Nearly 1 in 6 of our patients who visited primary care offices reported suicidal thoughts in the previous month when asked the question on a computerized survey. This rate is comparable to results from anonymous surveys. During 2007, 15% of US high school students reported that they had considered attempting suicide in the previous year.23
Suicidal thought was common among depressed youths, as has been found in previous research.24,25 One in 20 youths had a CES-DC score of ≥40, and half of them reported suicidal thought in the previous month. Rates of suicidal thought were higher among those who were younger because of more suicidal thought among younger girls, as was reported in data from the 2007 Youth Risk Behavior Survey.23 Fighting was also associated with suicidal thought, similar to the Fordwood et al26 finding that externalizing behavior increased suicide attempts.
Second, we examined the clinical value of the suicidal-thought question. We found that 6 of 7 youths who answered yes to the question also acknowledged substance use, carrying weapons, participation in violence, or depression. This result suggests that a question about suicidal thought should be included in a comprehensive behavioral health screen, a strategy that is more efficient than mounting a screen for a single behavioral problem. A positive suicidal-thought screen result clearly identifies a youth at high risk, and it should be followed by a comprehensive behavioral health evaluation.27 Ideally, this would be done by a colocated social worker during the primary care visit, obviating the need for a referral to a mental health specialist. We believe that outside referral makes assessment and subsequent linkage with intervention less likely than if mental health services were integrated within primary care.28
Third, we examined the likelihood of successful linkage with mental health services for youths who disclose suicidal thought during a primary care visit. Even when appropriate treatment services exist, it is often difficult to connect a patient from a primary care setting to a specialty care service. We used Web-based computer services to screen the patient and to immediately inform both the PCP and a social worker affiliated with a specialty behavioral health clinic about the suicidal thinking. Using this technology, we successfully triaged almost all patients who reported suicidal thoughts, on the same day for the great majority of them. Screening is incomplete without a workflow that uses physician extenders to quickly engage the family in a discussion of the screen result.
When we asked what youth factors were associated with the patient receiving a referral, we found that black youth were less likely to be referred to specialty mental health services. It is unfortunate that we have no information about whether this was because the providers were less likely to recommend a mental health referral to the family or because black families were more likely to refuse the offer and the recommendation was subsequently not documented. In addition, although the youths' depression and involvement in violence played an important and appropriate role in driving positive recommendations, evidence of substance use apparently did not. This important risk factor should not be neglected in evaluations. Nevertheless, it is encouraging that among those patients referred to our hospital's mental health system, a substantial majority eventually received a service there. By comparison, in another study of primary care screening for emotional disorders, Campo et al29 found that among patients with a significant mood or anxiety disorder referred for mental health treatment at a primary care visit, only 35% received any mental health care in the subsequent 6 months, compared with 65% in this study. This result suggests that by facilitating rapid, personal contact between the family and a social worker representing a specialty clinic, we can overcome substantial barriers to accessing mental health services and deliver care to youths who report suicidal thoughts in primary care. The public health relevance of such efforts is highlighted by growing evidence that access to care is inversely correlated with rates of suicide and suicidal behavior in youth and across the life span.12,30
Finally, we found that disclosing suicidal thoughts during a primary care visit was associated with the youth presenting later at the emergency department or elsewhere with a diagnosis related to suicidal thought or a suicide attempt. This result indicates that a positive answer is a valid, albeit weak, index of risk for future suicidal thoughts or attempts.
In this study, we could not address whether our efforts to intervene with screen-positive youth reduced the risk of subsequent suicidal behavior. Because of ethical concerns, we did not randomly assign screen-positive patients to our intervention or usual care. Therefore, although we are confident that our system increased the rates of mental health evaluations of such patients, we do not have direct evidence. Second, our measure of suicidal thought or behavior after the primary care visit was whether a patient had a contact with our emergency department or other service with a diagnosis related to suicide. Suicidal patients who contacted other medical facilities or who had no medical contact would have been missed. We also did not document details of the discussions with families at triage, which limited our ability to interpret a disparity in the rates of referrals for black youths. Finally, our study was performed in an integrated delivery system with in-house social-work support that had direct access to a behavioral health network that could provide rapid triage services in both office and community settings. The level of referrals and the rate of services received may be different in stand-alone primary care clinics.
Adolescents disclosed thoughts about suicide to PCPs using a computerized screen administered while they waited for an office visit. Reports of suicidal thought were associated with many other significant behavioral health problems and markers of health risk, and with later suicidal thought or behavior. By using the computerized screening system to facilitate rapid contact with a social worker, it was often possible to triage the case and then connect at-risk patients to a specialty mental health care system. Future research should use randomized trials to test whether systems that integrate screening and treatment across care settings can reduce rates of suicidal thought and behavior.
This research was supported by the National Institute on Drug Abuse (Trial of Automated Risk Appraisal and Adolescents, grant 1 R01 DA018943-01 [principal investigator, Dr Kelleher]).
- Accepted January 4, 2010.
- Address correspondence to William Gardner, PhD, Department of Pediatrics, Ohio State University, Center for Innovation in Pediatric Practice, Research Institute at Nationwide Children's Hospital, 700 Children's Way, Columbus, OH 43023. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
Funded by the National Institutes of Health (NIH)
- PCP =
- primary care physician •
- NCH =
- Nationwide Children's Hospital •
- CES-DC =
- Centers for Epidemiologic Studies Depression Scale for Children •
- OR =
- odds ratio •
- CI =
- confidence interval
- 1.↵Centers for Disease Control and Prevention, National Center for Health Statistics. National Vital Statistics System. Available at: www.cdc.gov/nchs/nvss.htm. Accessed February 12, 2010
- 2.↵Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion. YRBSS: Youth Risk Behavior Surveillance System. Available at: www.cdc.gov/HealthyYouth/yrbs/index.htm. Accessed February 12, 2010
- Newacheck PW,
- Brindis CD,
- Uhler CU,
- Marchi K,
- Irwin CE
- Young PC,
- Shyr Y,
- Schork MA
- Stevens J,
- Kelleher KJ,
- Gardner W,
- et al
- Fendrich M,
- Weissman MM,
- Warner V
- Campo JV,
- Shafer S,
- Strohm J,
- et al
- Campo JV,
- Bridge J,
- Lucas A,
- et al
EKG Screening of All Athletes: Perhaps More Cost-effective Than We Thought: Until recently, it was believed that screening all athletes with an EKG was not cost-effective for detecting those at risk for sudden cardiac death compared to just doing a history and physical. Yet according to a recent article in The New York Times (Bakalar N, March 2, 2010), the Italian Ministry of Health has required screening for competitive athletes and in doing so reduced the number of cardiac deaths by 89% among athletes 14 to 35 years of age to a rate similar to those who are non-athletic in that age group. Drawing on this information, a study in the Annals of Internal Medicine (Wheeler MT, Heidenreich PA, Froelicher VF, Hlatky MA, Ashley EA. Cost-effectiveness of preparticipation screening for prevention of sudden cardiac death in young athletes. Ann Intern Med.2010;152:276–286) showed (with a computer simulation exercise) that using a history and physical but no EKG to screen saves .56 life years per 1000 athletes and costs about $111 per person. Whereas the addition of routine EKG screening saves 2.06 life years at an additional cost of $89 per athlete, including any further exams or treatments, making this comparable to the use of dialysis in cost per life-year saved and possibly justifying the implementation of mandatory EKG screening in athletes. Yet according to Dr Ashley, “That something is cost-effective does not mean that there is the money available to do it.”
Noted by JFL, MD
- Copyright © 2010 by the American Academy of Pediatrics