OBJECTIVES: Accurate recognition of pediatric concussion in the emergency department (ED) is important to ensure appropriate management for safe recovery. The study objective was to determine whether the Centers for Disease Control and Prevention’s Acute Concussion Evaluation (ACE) tools, modified for ED use, improved patient follow-up and post-injury behaviors.
METHODS: The original ACE tools (ACE, ACE Care Plan) were modified for ED use via Delphi methodology with an expert panel and implemented in 2 urban pediatric EDs for patients aged 5 to 21 years evaluated within 24 hours of a head injury. Pre- (February 2009 to July 2009) and post- (December 2009 to June 2010) implementation, patient phone surveys were conducted 1, 2, and 4 weeks after ED discharge. Reported rates of patient follow-up and recovery measures were analyzed. ED clinician adherence was assessed.
RESULTS: During the study, 164 patients were enrolled pre-implementation and 190 post-implementation. The mean patient age was 10.6 years (SD, 3.7); 65% were males, 49% were African American, and 46% were Caucasian. Post-implementation, 58% of patients received the modified ACE diagnostic tool and 84% received the modified ACE discharge instructions. Follow-up was improved at all time points (32% vs 61% at week 4; P < .001; odds ratio, 3.4; 95% confidence interval, 2.1–5.4). Post-implementation, parental recall of discharge instructions was significantly increased, patient’s mean total post-concussion symptom score was significantly higher, and report of return to normal activity was significantly longer.
CONCLUSIONS: The ACE tools, modified for ED use, were successfully implemented in the pediatric ED. Post-implementation, increased patient follow-up and improved recall of and adherence to ED discharge recommendations was demonstrated.
- ACE —
- Acute Concussion Evaluation
- ANOVA —
- analysis of variance
- CI —
- confidence interval
- DI —
- discharge instruction
- ED —
- emergency department
- OR —
- odds ratio
- PCSS —
- Postconcussion Symptom Scale
What’s Known on This Subject:
Concussions in youth are a common injury evaluated in the emergency department (ED). Early recognition and active management of this mild traumatic brain injury are important to safe recovery. Tools to assess and manage concussion in the ED are lacking.
What This Study Adds:
Acute Concussion Evaluation tools, modified for ED use, improved reported follow-up with primary care or concussion specialists and adherence to recommendations. Barriers to follow-up remain and the importance of ongoing outpatient management should be stressed.
Mild traumatic brain injury or concussion represents the predominant form of acquired brain injury (75%–90%) evaluated in US emergency departments (EDs) and accounts for an estimated 600 000 annual ED visits in children.1,2 Evidence shows those who suffer a concussion are at increased risk for sustaining another concussion and the effects can be cumulative.3–5 Returning to physical activities too soon after concussion can lead to disabling and even life-threatening outcomes.6–10 Additionally, school performance may be affected, necessitating specific accommodation plans.11,12 Early, active, and individualized concussion management is recommended, including initial rest followed by systematic return to activity.11,13
Most ED patients who have a head injury do not need immediate clinical interventions, and the majority are reassured and discharged. Thus, accurate initial assessments to make the diagnosis of concussion followed by clear recommendations for outpatient management are key. Historically, the acute evaluation and management of concussion has been inconsistent and outcomes largely unknown.14,15 In particular, grading systems dictating management are not validated and have not allowed adequate injury recognition or individualized recovery plans.16–18 Up-to-date clinical tools are needed in the ED setting to improve recognition of concussion and inform education given to patients and families to minimize the risk for prolonged recovery and functional problems.19
The Acute Concussion Evaluation (ACE) system published by the Centers for Disease Control and Prevention was developed to assist in identification and diagnosis in the office setting.20–22 The ACE interview tool includes inquiry regarding concussion injury characteristics, symptoms, signs, and risk factors for prolonged recovery. The ACE Care Plan tool was developed to guide management, including individualized recommendations for daily life, school, work, and return to sport. Despite the comprehensiveness and potential utility of these tools, in our survey of emergency physicians the majority felt the current format was not conducive to the unique ED challenges of clinical flow and time constraint.23 In the current study we hypothesized routine use of the ACE and ACE Care Plan, modified for the ED, would improve patient/family post-injury follow-up and behaviors.
We performed a pre- and post-implementation study of the use of a modified ACE and ACE Care Plan in the pediatric ED setting.
Setting and Patient Population
Patients were recruited from 2 EDs: Children’s National Medical Center and Children’s Hospital of Pittsburgh, both Level 1 trauma centers with volumes exceeding 70 000 annual visits. Eligible patients in both study phases were identified by review of daily ED chief complaint log, followed by chart review, and finally, phone survey confirmation. Inclusion criteria were age 5 to 21 years, presentation within 24 hours of injury, and ability to reach by phone. Exclusion criteria were conditions preventing assessment of concussion signs and symptoms or not allowing for phone follow-up. Parents were consented by phone by using a standardized script, and patients 11 years and older gave assent and answered the survey as able.
Instruments: Modified ACE-ED and ACE-ED Care Plan
The Office Version ACE and ACE Care Plan21 were modified for ED use via the Delphi method24 for consensus agreement by using an expert panel of emergency medicine physician leaders of the Pediatric Emergency Care Applied Research Network. This national network serves over 800 000 children annually and represents academic, community, urban, rural, general, and children’s hospitals serving a large, diverse population. The modified tools were named the ACE-ED and ACE-ED Discharge Instructions (ACE-ED DI).
The ACE-ED (diagnostic form, Supplemental Information A) was shortened to facilitate use and consists of 6 sections: (1) injury description (open-ended); (2) blunt trauma to the head (yes/no); (3) 4 key immediate injury characteristics (check-list); (4) 4 signs and 14 symptoms (check-list); (5) 2 key recovery risk factors (yes/no); and (6) concussion diagnosis (yes/no). Concussion was defined by positive diagnosis on ACE-ED. Instructions were provided to make the diagnosis if “yes” was answered to blunt trauma to the head (section 2) and if any key injury characteristic (section 3) or any sign or symptom (section 4) were present. This definition of concussion was used for patients in both study phases. The ACE-ED DI (DI, Supplemental Information B) was modified to a 3-page form. The first 2 pages provided instructions regarding expected symptoms and management of daily activities, school, and sport, with a final section on follow-up, including an advised appointment with the primary care provider within 3 days and concussion specialist if persistent symptoms. The third page, a detailed “Return to School Form,” included an individualized return to school and physical activity checklist and an education section reviewing key problems and symptoms school personnel should watch for, as well as potential school supports.
Educational Training Process for Implementation of ACE-ED Tools
The pre-implementation phase consisted of the existing routine care at each site. ED clinicians and nurses were trained in use of the ACE-ED tools for standardized concussion assessment, diagnosis, and discharge. Training was conducted over 4 months and included small group presentations, electronic mailings, and on-site posters. An ED research assistant was available at peak hours several days per week to assist staff during training to identify eligible patients and complete the ACE-ED and ACE-ED DI.
Based on different clinical flow paths and available charting systems, implementation of the ACE-ED was different between the 2 study sites:
Site 1 (Children’s National Medical Center)
The ACE-ED was available in paper and pencil format. Clinicians completed the ACE-ED for eligible patients and were prompted by the form to make the diagnosis and complete the ACE-ED DI. The ACE-ED was returned to the study investigator via a drop box in the ED.
Site 2 (Children’s Hospital of Pittsburgh)
The ACE-ED was incorporated into the electronic medical record and completed by the triage nurse. When a mandatory triage (forced field) question “Was there blunt trauma to head?” was answered “Yes,” the additional ACE-ED fields automatically opened and could be checked electronically. If any key injury characteristic or any 1 sign or symptom was checked, a brain icon was generated and appeared on the electronic patient tracking board, indicating the patient had a positive ACE-ED screen. This icon served as a prompt to the clinician to make the diagnosis and complete the ACE-ED DI.
Data Collection Tools
During both phases, phone surveys were conducted at 1 week (administered at 7–10 days), 2 weeks (administered at 14–20 days), and 4 weeks (administered at 28–35 days) after the ED visit. Standardized survey instruments for each week were developed for the study and recorded follow-up, barriers to follow-up, recall of ED DIs, adherence to recommendations, and time to return to normal activity, sport, and school. Patient’s self-reported recovery characteristics were recorded, including a validated 22-item Postconcussion Symptom Scale (PCSS), with each item scored on a scale of 0 to 6.25
The Institutional Review Board at each participating institution approved all study components.
A quantitative analysis comparing rates of follow-up and self-reported recovery measures was performed. Based on a projected 20% increase in follow-up with primary care provider/concussion specialist (primary outcome), with a power of 0.9 and α of 0.05, a sample size of 140 participants (70 from each site) was estimated for both phases. A descriptive analysis of clinician adherence with implementation of the ACE-ED tools was performed.
Descriptive statistics, Student’s t test, univariate Fishers exact, and χ2 analyses and a mixed-model analysis of variance (ANOVA; with pre- versus post-implementation condition as the between-subjects factor and the 3 post-injury time points as the within-subjects, repeated measures factor) were performed by using SPSS statistical software (IBM SPSS Statistics, IBM Corporation). Other variables (age group, study site, insurance type, gender, race) were also examined for possible inclusion. Patient outcome measures (PCSS, return to activity, return to sports) were compared between successive time points by using planned contrasts with a more conservative α level of 0.01 to correct for multiple comparisons. Post hoc tests with a Student-Newman-Keuls correction for multiple comparisons were used to compare return-to-activity rates between age groups.
During the study, 354 patients were enrolled: 164 during the pre-implementation phase (February 2009–July 2009) and 190 during the post-implementation phase (December 2009–June 2010). Overall, the mean patient age was 10.6 years (SD, 3.7), 65% were males, 46% of patients were Caucasian, and 49% were African American. There were no significant differences in the demographic characteristics between the pre- and post-implementation phases (Table 1). Race and insurance coverage varied, however, by site. The injury mechanisms did not differ significantly between phases. Falls (35%), followed by sports (27%), and struck by/against an object (18%) accounted for the majority of injuries. There were no significant differences between phases in patient history of loss of consciousness (15% vs 11%, P = .2) or Glasgow Coma Score of 15 (98% of patients). All patients were discharged from the ED. In phase 1, no patients were lost to follow-up at site 1 and 2% at site 2, and in phase 2, 25% were lost at site 1 and 20% at site 2. There were no significant differences in age, gender, race, injury mechanism, or insurance comparing these patients with their respective phase and site.
To assess clinician adherence with the ACE tools, patients were categorized as receiving the ACE-ED (diagnostic form), the ACE-ED DI, both, or neither. Overall, post-implementation, 58% (111/190) of patients received the ACE-ED and 84% (159/190) received the ACE-ED DI (Table 2). The ACE-ED was completed more often (75%) at site 2, electronically by triage nurses. There was no significant difference in ACE-ED DI use between sites.
Report of follow-up with the primary care provider or concussion specialist was significantly improved post-implementation at all time points (Table 3). Examination by site at week 4 found significant improvement at both sites with differed rates (site 1, 17%–53% vs site 2, 48%–68%). Overall, patients who had commercial insurance reported significantly higher follow-up than patients who had medical assistance; however, a significant increase was found for both (Table 4). There were no differences in reported follow-up rates based on gender or injury mechanism. The barriers to follow-up identified by parents did not vary by phase or site. Those who did not follow-up were asked to endorse all barriers. The predominant barriers were that parents did not think follow-up was necessary (73%) or felt the child improved (63%), with <4% reporting issues with scheduling, cost, insurance coverage, or transportation. The percentage of parents who reported they were not instructed to follow-up decreased from 27% pre-implementation to 13% post-implementation (P = .05; odds ratio [OR], 0.4; 95% confidence interval [CI], 0.2–1.0).
Parental recall of concussion symptom education and being given activity restrictions and sports recommendations was significantly increased post-implementation (Table 5). The largest improvement was in recall of instructions for school restrictions. Although more reported missed school post-implementation, the median number (2 days) was the same in both phases. Children post-implementation reported to receive greater supports than those pre-implementation (17% vs 4%; F = 19.5; P < .001; η2 = 0.06). Two-thirds of parents reported using the ACE-ED DI “Return to School” form.
Symptom reports were examined (Fig 1) for both phases over the 3 assessment points using a mixed-model ANOVA with pre-/post-implementation and age group as between-subjects factors, and the 3 time points (week 1, 2, and 4) as the within-subjects factor. Other potential factors including race, gender, and insurance status did not account for significant variance in the model and were excluded. There was a significant main effect of age group (F = 25.4; P < .001; η2 = 0.14) with younger children (age 5–9 years) reporting fewer symptoms than older children and adolescents. In addition, there was an overall main effect for pre-/post-implementation, with the pre-implementation group reporting significantly lower overall symptoms than the post-implementation group (F = 65.8; P < .001; η2 = 0.17). There was also an overall main effect of time (F = 187.4; P < .001; η2 = 0.37). Planned contrasts revealed the level of reported symptoms decreased substantially from week 1 to week 2 (P < .001; partial η2 = 0.14) and from week 2 to week 4 (P < .01; η2 = 0.03). These findings should be viewed within a significant interaction effect for pre-/post-implementation (F = 42.2; P < .001; η2 = 0.12). Planned contrasts reveal this difference to be between week 1 and week 2, with the post-implementation group reporting a higher initial level of symptoms at week 1 and a steeper decrease in symptoms than the pre-implementation group (P < .001; partial η2 = 0.14). Site was found to be a significant factor modifying symptom ratings over the 3 time points (F = 17.4; P < .001; η2 = 0.06), with site 2 rating symptoms significantly higher than site 1, the specific effect being most evident between weeks 1 and 2 (F = 23.1; P < .001; Eta2 = 0.07), with similar levels of symptom ratings between weeks 2 and 3.
Reports of return to normal activity and sport were examined (Figs 2 and 3) using the same mixed-model ANOVA methods. In the analysis of return to normal activity and sport, age group was included as a factor in the model. Significantly more children age 5 to 9 years returned to normal activity across the 3 time points relative to the older groups, ages 10 to 14 and 15 to 21 years (91% vs 68%; Student-Newman-Keuls P < .05). For return to sport, the 2 younger age groups returned sooner than the adolescent age group. For both return to activity and sport, there were main effects for time (F = 90.86; P < .001; η2 = 0.22 and F = 147.2; P < .001; η2 = 0.32, respectively) and phase (F = 34.85; P < .001; η2 = 0.10 and F = 22.8; P < .001; η2 = 0.07, respectively). From weeks 1 to 4, both pre-/post-implementation groups demonstrated significant increases in return to normal activity and sports, with higher return rates reported pre-implementation. There was no main effect of site or insurance status on return to normal activity or sport.
As the ED is one of the frontlines for youth who have a concussion who seek immediate care, accurate recognition, diagnosis, and management in this fast-paced clinical setting is critical. Lack of standardized concussion assessment tools feasible for ED use has created a challenge. In preliminary work by our group, a survey of pediatric ED physicians found little familiarity with the Centers for Disease Control and Prevention mTBI tool kit,26 yet most felt there was utility for these tools in their practice.23 In the current study, we were able to successfully implement modified ACE tools, the ACE-ED (diagnostic form) and ACE-ED DI, in 2 high-volume pediatric EDs. The ACE-ED DI was given to the majority of patients post-intervention. A key component to successful ED management of concussion is instilling the importance of ongoing outpatient management, and thus our primary outcome was to improve follow-up. We demonstrated improved reported follow-up post-implementation for all patients, including those who had different injury mechanisms. Improved follow-up to ensure recovery before return to activity for those with non-sport mechanisms is important, as those who do not participate in sports may not have the same motivation for follow-up, and those who do participate may not report the injury to coaches.
Despite the success of this implementation, 39% of patients still reported no follow-up after their ED visit for concussion. Of those who did not follow-up, 73% reported they did not think it was necessary and 63% that their child had improved. These barriers demonstrate a continued need for education on the importance of follow-up. Returning a child to activity and contact sport before full recovery may result in prolonged symptoms and risk for recurrent and more severe concussion.9,27 A systematic return to cognitive and physical exertion is needed for those who have both rapid and protracted recoveries.11,28,29 Previous research demonstrates outpatient follow-up post-concussion is associated with increased compliance with recommendations.30 Although parents did not cite financial barriers, we found those who had commercial insurance more often reported follow-up, which may reflect greater challenges for families who have socioeconomic barriers. Yet we also found the site with more medical assistance demonstrated greater increases in follow-up, suggesting the intervention discharge instructions may afford greater benefit in less advantaged patients who may have less awareness of concussion. This possibility further emphasizes the importance of concussion education and clear discharge instructions for families in the acute setting, stressing the need for ongoing management by a clinician who can objectively assess and manage recovery.
Post-implementation of the ACE-ED tools, families reported a better understanding of concussion. In addition, more reported they were given restrictions in activity and school, and school supports. Overall, total PCSS scores were increased post-implementation (largely in week 1) and time to return to activity and sport was longer, with a larger effect seen in adolescence. The symptom and post-implementation recovery curves in the current study more resemble curves in previous concussion studies using PCSS and pediatric ED populations.17,30,31 Our findings likely reflect both enhanced knowledge of the various manifestations of concussion and improved observance of ED discharge instructions. An alternative interpretation is patients were more significantly injured in the post-implementation phase. This is less likely, as overall, demographics, clinical features, and injury mechanisms did not differ between phases. An additional concern is the possibility of increasing patients’ level of disability post-concussion by emphasizing the negative expectation of symptoms, a clinical phenomenon called the nocebo response.32,33 Previous concussion research has shown education given with a positive message that recovery will occur is beneficial.19 Thus, patient education regarding possible symptoms coupled with positive reinforcement of likely recovery over the following days and weeks is recommended. As exact recovery timing cannot be predicted in the acute setting, a well-managed return to cognitive and physical exertion is needed as symptoms resolve.
Our study was unique in the implementation of clinical flow pathways of the ACE-ED (diagnostic form) differently in the 2 sites. The ACE-ED was more often completed electronically in triage, and this format allowed a patient tracking board icon to alert the clinician to a “positive ACE-ED screen.” As ED clinical flow is unique to an institution, careful consideration of local process is recommended to implement the ACE-ED tools. As only 58% of patients were screened with the ACE-ED, yet 84% received the ACE-ED DI, the DI intervention likely contributed more to the improved follow-up. Further research could explore alternative means of ACE-ED distribution to improve adherence, such as the parent/patient completing and reviewing with the clinician.
Implementing the ACE tools in 2 pediatric academic EDs may limit generalizability to other settings; however, the use of 2 different implementation strategies may allow a broader application. Site-specific effects were seen, although both sites and demographics benefited from the intervention. An inherent limitation of the pre-/post-implementation design is the possibility that differences seen have more to do with differences between 2 time points. Although we cannot account for changes in awareness of concussion in the population over time, overall demographics, clinical characteristics, and injury mechanisms did not differ. As our methods of final eligibility were determined by phone screening, we were not able to assess the true percent eligible and cannot determine how those not enrolled may have differed. An additional limitation is the possible over-diagnosis of concussion by using the ACE-ED definition, which allows diagnosis with only 1 sign or symptom. However, we believe inclusion with promotion of follow-up is best in the acute setting. Patient outcomes were limited by the bias of self-report. Standardized scripted serial surveys were used to minimize this effect. Despite these limitations, this study is strengthened by a large sample size, diverse demographic and injury mechanisms, and prospective, 2-center methodology.
The ACE tools for concussion diagnosis and discharge, modified for ED use, were successfully implemented in the pediatric ED setting. Post-implementation of the ACE-ED tools, increases in reported patient follow-up with a primary care provider or concussion specialist, and improvements in parent recall of concussion education and patient adherence to ED discharge recommendations were demonstrated.
- Accepted January 21, 2014.
- Address correspondence to Noel S. Zuckerbraun, MD, MPH, Associate Professor of Pediatrics, University of Pittsburgh School of Medicine, Department of Pediatrics, Division of Pediatric Emergency Medicine, Children’s Hospital of Pittsburgh of UPMC, 4401 Penn Avenue, Pittsburgh, PA 15224. E-mail:
Dr Zuckerbraun conceptualized, designed, and carried out the study, acquired data, carried out the initial analysis and interpretation of data, drafted the initial manuscript, and critically reviewed and revised the manuscript; Dr Atabaki conceptualized, designed, and carried out the study, acquired data, contributed to the interpretation of data, and critically reviewed and revised the manuscript; Drs Collins and Thomas conceptualized and designed the study and study instruments, contributed to the interpretation of data, and critically reviewed and revised the manuscript; Dr Gioia conceptualized and designed the study and study instruments, was the primary investigator on the grant funding the study, performed the initial and final analysis and interpretation of data, and critically reviewed and revised the manuscript; and all authors approved the final manuscript.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Centers for Disease Control and Prevention grant 5 U49 CE001385-02.
POTENTIAL CONFLICT OF INTEREST: Drs Collins and Gioia are authors of the Acute Concussion Evaluation; the other authors have indicated they have no potential conflicts of interest to disclose.
- ↵Collins MW, Lovell MR, Iverson GL, Cantu RC, Maroon JC, Field M. Cumulative effects of concussion in high school athletes. Neurosurgery. 2002;51(5):1175–1179; discussion 1180–1181
- Halstead ME,
- Walter KD,
- Council on Sports Medicine and Fitness
- ↵McCrory P, Johnston K, Meeuwisse W, et al. Summary and agreement statement of the 2nd International Conference on Concussion in Sport, Prague 2004. Clin J Sport Med. 2005;15(2):48–55
- Grubenhoff JA,
- Kirkwood M,
- Gao D,
- Deakyne S,
- Wathen J
- Ponsford J,
- Willmott C,
- Rothwell A,
- et al
- ↵Langlois JA. Heads up: facts for physicians about mild traumatic brain injury. 2010. Available at: www.cdc.gov/concussion/headsup/pdf/Facts_for_Physicians_booklet-a.pdf. Accessed July 10, 2013
- ↵Atabaki SGG, Zuckerbraun N, Palacios M, Grubbs R, Collins M. Practice patterns in emergency department management and follow-up of pediatric mild traumatic brain injury. In: International Brain Injury Associations Eighth World Congress on Brain Injury, 2010; Washington, DC: 2010;24:355
- ↵Fowles J. The Handbook of Futures Research. Westport, CT: Greenwood Press; 1978
- Gioia GA,
- Schneider JC,
- Vaughan CG,
- Isquith PK
- ↵Centers for Disease Control and Prevention. Heads up: brain injury in your practice, 2007. Available at: www.cdc.gov/ncipc/pub-res/tbi_toolkit/physicians/introduction.htm. Accessed July 10, 2013
- Boden BP,
- Tacchetti RL,
- Cantu RC,
- Knowles SB,
- Mueller FO
- Eisenberg MA,
- Andrea J,
- Meehan W,
- Mannix R
- Colloca L,
- Miller FG
- Copyright © 2014 by the American Academy of Pediatrics