External Validation of the PediBIRN Clinical Prediction Rule for Abusive Head Trauma
BACKGROUND AND OBJECTIVES: A 4-variable abusive head trauma (AHT) clinical prediction rule (CPR) for use in the PICU was derived and validated for children <3 years of age by the Pediatric Brain Injury Research Network (PediBIRN). We aimed to externally validate PediBIRN as designed (PICU only) as well as using broader inclusion criteria (admitted children with head injuries).
METHODS: This was a secondary analysis of a prospective multicenter study of pediatric head injuries at 5 Australian and New Zealand tertiary pediatric centers. Possible AHT was identified by clinician suspicion, epidemiology codes, or a high-risk group (<3 years of age, admitted, abnormal neuroimaging results). At 1 center, we additionally reviewed head injuries in the forensic database. We designated patients as positive for AHT, negative for AHT, or having indeterminate outcome after multidisciplinary review and applied the PediBIRN CPR, blinded to outcome, to PICU admissions only, and any head injury admissions. CPR accuracy was calculated by using 95% confidence intervals.
RESULTS: One hundred and forty-one patients were admitted with abnormal neuroimaging results. Twenty-eight (20%) were positive for AHT, 94 (67%) were negative for AHT, and 19 (13%) had indeterminate outcome. Excluding indeterminate cases, in the PICU (n = 28), the CPR was 100% (75%–100%) sensitive and 11% (0%–48%) specific; in all admitted patients (n = 141), sensitivity was 96% (82%–100%) and specificity of 43% (32%–53%).
CONCLUSIONS: This validation revealed high sensitivity and low specificity for PICU patients. Specificity was improved but moderate in a broader group of admitted head injury patients.
- AHT —
- abusive head trauma
- CI —
- confidence interval
- CPR —
- clinical prediction rule
- ED —
- emergency department
- PediBIRN —
- Pediatric Brain Injury Research Network
- TBI —
- traumatic brain injury
What’s Known on This Subject:
A 4-variable clinical prediction rule (CPR) for detecting abusive head trauma has been derived and validated in 14 American PICUs. It has not been validated outside the United States or assessed outside the PICU setting.
What This Study Adds:
We externally validated the CPR in an Australian and New Zealand data set. It performed with high sensitivity and lower specificity in children admitted to the PICU and in all head-injured admitted children.
Abusive head trauma (AHT) is the leading cause of traumatic death in infants and is responsible for two-thirds of child maltreatment fatalities.1,2 Approximately 20% of those who suffer AHT die, and more than half develop a disability of some degree.3,4 The American Academy of Pediatrics recommended defining AHT as “the constellation of cerebral, spinal, and cranial injuries that result from inflicted head injury to infants and young children.”5
It is often difficult for clinicians to distinguish between AHT and accidental trauma; nonspecific symptoms of brain injury such as vomiting and lethargy may also be attributed to other childhood diseases,6 and the caregiver may not provide a history of trauma or fabricate an explanation.7 For these reasons, AHT can be missed in the clinical setting, as described by Letson et al6 and Jenny et al,8 who showed that one-quarter to one-third of victims of AHT had been previously seen by clinicians providing an opportunity to make an earlier diagnosis. Missed diagnoses can have severe consequences because children may be sent back to an unsafe environment or may not receive adequate medical treatment. On the other hand, excessive child abuse investigation in children with accidental injuries puts emotional strain on the families involved, potentially exposes children to unnecessary radiation and investigations, and may be an inefficient use of scarce clinical resources. These diagnostic decisions can be stressful for clinicians.
Clinical prediction rules (CPRs) are evidence-based tools combining clinical features, history or results of investigations to predict diagnosis, and the prognosis or response to therapy.9,10 They are of most value when used to assist clinicians in making complex decisions when clinical stakes are high and intuition and experience may be of limited use.10
The 4 variables are easily available at admission to the PICU. If 1 or more are fulfilled, the child is categorized as higher risk and should undergo thorough evaluation for abuse (Table 1). As a screening tool for AHT, the aim was to have high sensitivity at the cost of specificity. In the validation study, the CPR performed with a sensitivity of 96% and a specificity of 43%. This study provided data on the performance of the CPR in American PICUs; however, it is not known how the CPR performs in different populations. Although designed for PICU use, it would be useful to know if the CPR can be applied to all young children admitted with abnormal neuroimaging results, because they are also at risk for AHT.13
We therefore set out to externally validate the PediBIRN 4-variable CPR for AHT in children <3 years of age admitted for management of symptomatic, acute, closed, traumatic, cranial, or intracranial injuries confirmed by neuroimaging; we considered the CPR (1) as it was designed (ie, for children admitted to the PICU) and (2) in a broader sample of all children admitted to the hospital.11
Study Design, Setting, and Patients
We performed a planned secondary subgroup analysis of children enrolled into a prospective multicenter observational study of children (0–18 years) with head injuries in 10 Australian and New Zealand pediatric emergency departments (EDs) between April 2011 and November 2014.14,15 All EDs are members of the Pediatric Research in Emergency Departments International Collaborative research network.16 The study was registered with the Australian New Zealand Clinical Trials Registry (identifier ACTRN12614000463673).
We obtained ethics approval from 5 participating sites (Australia: The Royal Children’s Hospital, Melbourne; the Monash Children’s Hospital, Clayton; and the Princess Margaret Hospital for Children, Perth; New Zealand: Starship Children’s Hospitals and KidzFirst Children’s Hospitals, Auckland) for additional medical record review of possible cases of AHT.
Full details of the primary study protocol are described in detail elsewhere.15 In short, all children and young people with head injury were enrolled by the treating ED clinician who collected clinical data, and a research assistant recorded ED and hospital management data after the visit. We collected injury and clinical variables, demographic and epidemiologic information, and information about neuroimaging, admission, and neurosurgery. In this study, we analyzed data from a subset of children <3 years of age with abnormal neuroimaging results indicating that a differential diagnosis of AHT should be considered.
Strategy to Identify Possible AHT Cases
We used several approaches to identify all possible AHT cases from the parent study and assessed them further using the following methods:
We evaluated an investigator-defined high-risk group for possible AHT, which included all children aged <3 years of age admitted to the hospital with a head injury and abnormal neuroimaging results;
We searched the database for positive responses to the question “Do you suspect nonaccidental injury (physical abuse of a child, not other assault)?” This was asked of ED clinicians in the clinical report forms. When the answers were “yes” or “unknown,” medical records were manually reviewed;
Research assistants assigned injury epidemiology codes on the basis of information recorded by the ED clinician and from medical record review; human intent codes other than nonintentional codes were reviewed; and
At The Royal Children’s Hospital, Melbourne (5372 [40.2%] of 13 371 patients enrolled at the 5 sites), we also accessed the database of the Victorian Forensic Pediatric Medical Service, the hospital child protection team, which we searched for children with head injury.
We then accessed the medical records of all possible AHT cases at 5 sites and abstracted relevant data, including predictor variables, outcomes, and eligibility criteria for the PediBIRN CPR.11
We used senior radiologists’ reports to determine the results of neuroimaging and used operative reports for patients who underwent neurosurgery.
Abnormal neuroimaging was defined as intracranial hemorrhage or contusion, cerebral edema, traumatic infarction, diffuse axonal injury, shearing injury, sigmoid sinus thrombosis, midline shift of intracranial contents or signs of brain herniation, diastasis of the skull, pneumocephalus,17 or skull fractures visible on cranial computed tomography or MRI scans.
A clinically important traumatic brain injury (TBI) was defined as per the Pediatric Emergency Care Applied Research Network17 as death from TBI, neurosurgical intervention for TBI (intracranial pressure monitoring, elevation of depressed skull fracture, ventriculostomy, hematoma evacuation, lobectomy, tissue debridement, or dura repair), intubation of >24 hours for TBI or hospital admission of ≥2 nights for TBI in association with abnormal neuroimaging results as defined above.
We used injury epidemiology codes based on Victorian Government codes18 that included activity, place, mechanism of injury, and human intent.
AHT was defined as the diagnosis of cranial or intracranial head injury (confirmed on neuroimaging), which was due to physical child abuse by parents or caregivers rather than neglect19 according to the decision of a multidisciplinary child protection team at the conclusion of their investigation and their consideration of the relevant social, forensic, and clinical features in the context of the presenting history, in accordance with the Australian and New Zealand standard child protection assessment processes. Non-AHT was defined as a witnessed accidental injury or an accidental injury confirmed by the decision of a multidisciplinary child protection team. Cases were categorized as AHT-positive or AHT-negative (non-AHT) by the study investigators on review of the multidisciplinary team records. Cases in which this categorization was not clear were deemed indeterminate. Any uncertainty in terms of category assignment on review of the records was arbitrated by the director of the Victorian Forensic Pediatric Medical Service (A.S.) on the basis of the forensic reports and medical records.
Application of the PediBIRN AHT CPR to Our Data Set
First, we applied the PediBIRN CPR to our data set blinded to AHT outcome using the original eligibility criteria: children <3 years of age with cranial or intracranial abnormality on neuroimaging admitted to the PICU for acute TBI.11 We then applied the CPR to a broader sample including all children <3 years of age admitted to the PICU or any other inpatient ward for the management of TBI confirmed by abnormal neuroimaging.
Patients for whom no neuroimaging was performed or neuroimaging results were normal were excluded. Patients presenting with injuries caused by a crash or unintentional injury involving a motor vehicle; who had preexisting brain malformation, disease, infection, or hypoxia-ischemia on neuroimaging; or who had open head injuries were also excluded, as in the original PediBIRN study.11
Data were entered into Epidata (The Epidata Association, Odense, Denmark) and were later entered into REDCap,20 and data were analyzed by using Stata 13 (Stata Corp, College Station, TX). Summary statistics were derived to describe total and subgroup characteristics; proportions and frequencies for categorical variables, and the median (interquartile range) for continuous variables. We calculated sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, and negative likelihood ratio, with 95% confidence intervals (CIs), excluding indeterminate AHT cases. To obtain a best-case and worst-case scenario in terms of indeterminate AHT cases, we also conducted 2 sensitivity analyses reclassifying indeterminate AHT cases as AHT-positive or as AHT-negative, respectively.
Of the 20 137 patients at 10 centers in the parent study, 13 371 (66%) patients presented at the 5 centers included in this secondary analysis. Of these patients, 5264 were <3 years old (39%), of whom 3038 (58%) were male. The medical records of 176 cases of children <3 years old with possible physical abuse–related head injuries were reviewed, and of these, 143 had abnormal neuroimaging results. Excluding 2 cases because of discharge from ED or preexisting brain malformation, 141 possible cases remained (2.7% of all <3-year-olds) (Fig 1).
Ninety-eight (70%) were aged <1 year, 82 (58%) were male, 28 (20%) were admitted to the PICU, 26 (18%) underwent neurosurgery, and 6 (4%) died (Table 2).
On the basis of the decision of the local multidisciplinary child protection team, patients were categorized as AHT-positive in 28 (20%), AHT-negative in 94 (67%), and AHT-indeterminate in 19 cases (13%). Among AHT-positive cases, bilateral or interhemispheric subdural hemorrhage was the most common predictor variable, whereas skull fractures other than an isolated, unilateral, nondiastatic, linear parietal skull fracture were more common among AHT-negative cases (Table 3).
The 28 (20%) patients admitted to the PICU met the original inclusion criteria of the PediBIRN CPR, with 13 (46%) of these judged to be AHT-positive. The PediBIRN CPR correctly identified 13 out of 13 AHT-positive cases (sensitivity = 100% [95% CI, 75%–100%]) and correctly categorized 1 out of 9 AHT-negative cases (specificity = 11% [0%–48%]) (Table 4).
Using broader inclusion criteria (any admission with abnormal neuroimaging), 141 cases were included; the CPR detected 27 out of 28 AHT-positive cases (sensitivity = 96% [82%–100%)) and correctly categorized 40 out of 94 AHT-negative cases (specificity = 43% [32%–53%]) (Table 4). Excluding the PICU cases from this group (ie, any admission with abnormal neuroimaging results outside the PICU), 113 cases were included and the CPR detected 14 out of 15 AHT-positive cases (sensitivity = 93% [68%–100%] and correctly categorized 39 out of 85 AHT-negative cases as lower risk (specificity = 46% [35%–57%]) (Table 4).
A total of 16 of 19 (84%) AHT-indeterminate cases were classified as higher risk by the PediBIRN CPR. When categorizing AHT-indeterminate cases as AHT-negative, specificity changed from 43% to 38% for all admitted patients, from 11% to 7% for PICU patients only, and from 46% to 43% for non-PICU patients. Sensitivity did not change (Table 4). When categorizing AHT-indeterminate cases as AHT-positive, sensitivity changed from 96% to 92% for all admitted patients, remained 100% for PICU patients only, and changed from 93% to 86% for non-PICU patients. Specificity did not change (Table 4).
Excluding the variable “skull fractures other than an isolated, unilateral, nondiastatic, linear parietal skull fracture” from the analysis, sensitivity and specificity remained unchanged for PICU patients. However, for all admitted patients, sensitivity decreased from 96% to 86% and specificity increased from 43% to 86%. For non-PICU patients, sensitivity decreased from 93% to 73% and specificity increased from 46% to 94%. These calculations were performed by excluding AHT-indeterminate cases.
In this external validation of the PediBIRN 4-variable AHT CPR in an Australian and New Zealand data set, the CPR revealed high sensitivity when it was used as designed in the PICU as well as when it was used with the broader inclusion criteria of admitted young patients with abnormal neuroimaging results. The specificity in the PICU group was lower than in the original validation (11% vs 44%), which may in part be due to case identification from the point of presentation to the ED in this study, which was used in place of recruitment directly from the PICU, the method used in the original study by Hymel et al.11 However, the prevalence of AHT among the PICU population was high (42.6%) in the PediBIRN validation study and 46% in our data set. One could argue that in a sample in which almost every second child has suffered head injuries due to abuse, all patients should be assessed for possible AHT.21 Indeed, most of the AHT-negative cases in the PICU population were categorized as higher risk by the PediBIRN CPR (8 out of 9).
The performance of the CPR in the broader sample comprising all head injured children <3 years of age admitted to the hospital with abnormal neuroimaging results or in the data set of ward patients (non-PICU) was similar to the performance of the original “PICU-only” data set in the validation study by Hymel et al11 (sensitivity: 96% vs 96% and specificity: 43% vs 43%). Therefore, the CPR may have a role in screening for AHT in all children <3 years of age admitted to the hospital with abnormal neuroimaging results (excluding motor vehicle crashes or unintentional injuries and preexisting abnormality on neuroimaging). In the 2 broader samples (all admissions  and non-PICU admissions ), the CPR categorized 54% (46 out of 85) and 57% (54 out of 94) of AHT-negative cases, respectively, as higher risk and would trigger a workup for possible abuse in these cases. One approach may be to workup all PICU patients with head injury and abnormal neuroimaging results for possible abuse and workup only patients categorized as higher risk by PediBIRN among patients admitted to the ward. High sensitivity in any setting where a CPR such as PediBIRN is used is an essential requirement. PediBIRN has a high sensitivity but a comparatively low specificity. Thus, the CPR will reduce missed cases of AHT, but possibly at the expense of overinvestigating, because it would recommend a high number of child protection investigations if applied to all children <3 years of age presenting with abnormal neuroimaging results. However, the PediBIRN investigators undertook a retrospective theoretical comparison between actual screening for AHT in the PICU and screening according to the PediBIRN CPR and showed that the application of the CPR could increase detection of AHT and the diagnostic yield of abuse evaluations, with marginally fewer patients undergoing evaluation for abuse.22
The Predicting Abusive Head Trauma CPR23,24 provides estimated probabilities of AHT for children <3 years of age with intracranial injury and combinations of clinical features at various points along the investigation process and, using a 50% probability cutoff (<50% classified as probable non-AHT and >50% classified as probable AHT), performs with a comparatively lower sensitivity of 72.3% and a much higher specificity of 85.7%. However, Predicting Abusive Head Trauma is an assistive CPR that does not recommend a direct course of action, whereas the PediBIRN states that all children <3 years of age with ≥1 of the 4 variables should undergo further investigation. A recent systematic review of 3 CPRs25 for AHT concluded that the PediBIRN had the highest methodological quality score, although it was recognized that each CPR was intended for use at different stages of the assessment pathway, that is, in the ED to screen for AHT,26 in the PICU to decide which cases to evaluate for abuse,11 or during an inpatient stay when investigations for AHT have already been performed to assess the likelihood of AHT.24
In the original derivation and validation studies, all cases that did not meet any of the proposed a priori definition criteria were categorized as nonabusive.11,12 In our study, the investigators categorized cases as AHT-positive, AHT-negative, or indeterminate after multidisciplinary child protection decisions; the treatment of indeterminate cases in which abuse could neither be confirmed nor excluded is a vexed issue and has been addressed through various definitions and approaches in the literature.23,26,27 On the basis of the methodology of identifying our study sample, we wanted to ensure that the AHT status was definitive and excluded the heterogenic group of indeterminate cases for the primary analysis of sensitivity and specificity. We undertook a sensitivity analysis in which indeterminate cases were either analyzed as AHT-positive or AHT-negative, and (importantly) this did not fundamentally change the CPR accuracy. Indeed, 84% of the indeterminate cases were predicted to be AHT-positive by PediBIRN, which is a desirable outcome because these cases should be investigated further.
The fact that the variable “any skull fracture other than an isolated, unilateral, nondiastatic, linear parietal skull fracture” was more common by a measure of statistical significance among AHT-negative cases than among AHT-positive cases could have accounted for the lower specificity and higher number of false-positives in this data set. In 41 out of 94 AHT-negative cases, the skull fracture was the only predictor variable present and resulted in the false-positive results. By excluding skull fractures as a predictor variable, specificity could be raised from 43% to 86% in admitted patients. In other studies, skull fractures were found to be nondiscriminatory, even when considering specific fracture types.23,28 In the original PediBIRN derivation study,12 a 3-variable CPR without this feature had a specificity of 60% compared with a specificity of 36% when included.
The study has a number of limitations. As patients were enrolled in the ED (except for the cohort from the forensic service at 1 of the sites), only 28 patients met the original inclusion criteria of being admitted to the PICU, which may have resulted in reduced precision around our point estimates. We did not capture cases directly admitted to the PICU on transfer from other centers, although trauma patients at most study hospitals were generally first assessed in the ED, regardless of origin and ultimate destination. We missed patients admitted to a ward or PICU with a presumed medical condition who were only later recognized as having suffered AHT. Excluding indeterminate cases for the primary calculation of sensitivity and specificity further reduced our numbers for primary analysis.
The majority of the predictor and outcome variables were collected prospectively; however, additional details of the variables required for the CPR were abstracted from medical records. The variable of respiratory compromise before admission in particular was not prospectively collected in the original data set and was extracted from medical records. Although it is likely that respiratory compromise would have been documented in the medical records, we do not know this for certain.
Although our study provides information about the use of the CPR in all admitted patients, it was not designed for this population and would need to be prospectively validated further for use in this population. PediBIRN should ultimately be tested in an impact analysis before its use in any population.
We categorized cases as AHT-negative, AHT-positive, and indeterminate on the basis of the decision of the local multidisciplinary child protection teams, whereas the original study authors used extracranial injuries suggestive of abuse as well as historical features; this information would have been difficult to obtain retrospectively. Outcome measures were based on thorough multidisciplinary and forensic assessment of the likelihood of AHT. When there was any doubt, cases were classified as indeterminate. At no time were the clinicians involved in case assessment aware of the PediBIRN CPR or the 4 factors within it. In addition, the research assistant applying PediBIRN was blinded to the case outcome. We took this approach to represent the clinical reality, in which decisions must be made on the balance of probability to protect children from future abuse, and as a methodological approach to minimize the risk of circular reasoning. Finally, in this study, we have not answered the question of what the optimal workup for suspected AHT should be, nor have we determined if the use of this CPR is the optimal approach for identifying which children should receive a “complete” workup. The PediBIRN CPR is a tool to assist clinicians in deciding whether to “launch (or to forgo) abuse evaluations” for AHT in a specific population of children with abnormal neuroimaging results; just like other CPRs, it should always be used in combination with clinical judgment.
The 4-variable AHT CPR derived and validated by the PediBIRN investigators revealed high sensitivity and lower specificity in an external data set when it was used as designed for PICU patients as well as when it was used in a broader group of admitted children <3 years old with abnormal neuroimaging results.
- Accepted February 13, 2018.
- Address correspondence to Franz E. Babl, MD, MPH, Emergency Research, Murdoch Children’s Research Institute, 50 Flemington Rd, Parkville, VIC 3052, Australia. E-mail:
This trial has been registered with the Australian New Zealand Clinical Trials Registry (identifier ACTRN12614000463673).
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Funded by grants from the National Health and Medical Research Council (project grant GNT1046727, Centre of Research Excellence for Pediatric Emergency Medicine GNT1058560), Canberra, Australia; the Murdoch Children’s Research Institute, Melbourne, Australia; the Emergency Medicine Foundation (EMPJ-11162), Brisbane, Australia; Perpetual Philanthropic Services (2012/1140), Australia; the Auckland Medical Research Foundation (3112011) and the A + Trust (Auckland District Health Board), Auckland, New Zealand; the Western Australia Health Targeted Research Funds 2013, Perth, Australia; and the Townsville Hospital and Health Service Private Practice Research and Education Trust Fund, Townsville, Australia; and supported by the Victorian Government’s Infrastructure Support Program, Melbourne, Australia. Dr Babl’s time was partially funded by a grant from The Royal Children’s Hospital Foundation and the Melbourne Campus Clinician Scientist Fellowship, Melbourne, Australia, and a National Health and Medical Research Council Practitioner Fellowship, Canberra, Australia. Dr Dalziel’s time was partially funded by the Health Research Council of New Zealand (HRC13/556).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Christian CW,
- Block R; Committee on Child Abuse and Neglect; American Academy of Pediatrics
- Babl FE,
- Lyttle MD,
- Bressan S, et al; PREDICT Research Network
- Kuppermann N,
- Holmes JF,
- Dayan PS, et al; Pediatric Emergency Care Applied Research Network (PECARN)
- Department of Health and Human Services
- Leeb RT,
- Paulozzi LJ,
- Melanson C,
- Simon TR,
- Arias I
- Hymel KP,
- Herman BE,
- Narang SK, et al; Pediatric Brain Injury Research Network (PediBIRN) Investigators; Pediatric Brain Injury Research Network PediBIRN Investigators
- Cowley LE,
- Morris CB,
- Maguire SA,
- Farewell DM,
- Kemp AM
- Pfeiffer H,
- Crowe L,
- Kemp AM,
- Cowley LE,
- Smith JAS,
- Babl FE
- Berger RP,
- Fromkin J,
- Herman B, et al
- Duhaime AC,
- Alario AJ,
- Lewander WJ, et al
- Kemp AM,
- Dunstan F,
- Harrison S, et al
- Copyright © 2018 by the American Academy of Pediatrics