Validation of the Pittsburgh Infant Brain Injury Score for Abusive Head Trauma
BACKGROUND: Abusive head trauma is the leading cause of death from physical abuse. Misdiagnosis of abusive head trauma as well as other types of brain abnormalities in infants is common and contributes to increased morbidity and mortality. We previously derived the Pittsburgh Infant Brain Injury Score (PIBIS), a clinical prediction rule to assist physicians deciding which high-risk infants should undergo computed tomography of the head.
METHODS: Well-appearing infants 30 to 364 days of age with temperature <38.3°C, no history of trauma, and a symptom associated with an increased risk of having a brain abnormality were eligible for enrollment in this prospective, multicenter clinical prediction rule validation. By using a predefined neuroimaging paradigm, subjects were classified as cases or controls. The sensitivity, specificity, and negative and positive predictive values of the rule for prediction of brain injury were calculated.
RESULTS: A total of 1040 infants were enrolled: 214 cases and 826 controls. The 5-point PIBIS included abnormality on dermatologic examination (2 points), age ≥3.0 months (1 point), head circumference >85th percentile (1 point), and serum hemoglobin <11.2g/dL (1 point). At a score of 2, the sensitivity and specificity for abnormal neuroimaging was 93.3% (95% confidence interval 89.0%–96.3%) and 53% (95% confidence interval 49.3%–57.1%), respectively.
CONCLUSIONS: Our data suggest that the PIBIS accurately identifies infants who would benefit from neuroimaging to evaluate for brain injury. An implementation analysis is needed before the PIBIS can be integrated into clinical practice.
- AHT —
- abusive head trauma
- ALTE —
- apparent life-threatening event
- AUC —
- area under the curve
- CHG —
- Ann & Robert H. Lurie Children’s Hospital in Chicago, IL
- CHP —
- Children’s Hospital of Pittsburgh of UPMC
- CI —
- confidence interval
- CPR —
- clinical prediction rule
- CPT —
- Child Protection Team
- CT —
- computed tomography
- ED —
- emergency department
- NPV —
- negative predictive value
- PIBIS —
- Pittsburgh Infant Brain Injury Score
- PPV —
- positive predictive value
- ROC —
- receiver operator characteristic curve
- SLC —
- Primary Children’s Hospital in Salt Lake City, Utah
What’s Known on This Subject:
Abusive head trauma (AHT) is the leading cause of death from physical abuse. Identification of AHT, particularly in its mild forms, is difficult; missing the diagnosis can result in increased morbidity and mortality.
What This Study Adds:
The Pittsburgh Infant Brain Injury Score may be able to assist physicians to decide whether an infant at increased risk for AHT would benefit from computed tomography of the head.
Proper diagnosis of mild AHT is difficult because caretakers rarely provide an accurate history,8 infants present with nonspecific symptoms, such as vomiting or fussiness, and physical examination is often normal.9–11 As a result, misdiagnosis is common and can have catastrophic medical consequences.11–13 In a landmark study,12 31% (54/173) of children diagnosed with AHT were evaluated previously by a physician for symptoms compatible with brain injury. A multicenter study 15 years later demonstrated a similar rate of missed diagnoses (M. Letson, MD, MEd, personal communication, 2016), suggesting that early, accurate diagnosis of AHT continues to be challenging.
Although AHT is the leading cause of morbidity and mortality from brain injury in infants, infants with atraumatic neurologic abnormalities, such as hydrocephalus or a brain tumor, and infants with traumatic injuries that are not due to abuse, can present with the same symptoms as infants with AHT. Timely diagnosis of these non-AHT–related brain abnormalities also can be difficult for the same reasons as early recognition of AHT can be difficult: physicians may not consider a brain abnormality as a cause of the infant’s symptoms.
Clinical prediction rules (CPR) are tools that quantify the contributions that components of the history, physical examination, and laboratory tests make toward a patient’s diagnosis.14 CPRs are particularly useful in diseases in which clinical stakes are high and clinical experience and intuition are insensitive.15
The Pittsburgh Infant Brain Injury Score (PIBIS) CPR was retrospectively derived based on data from 187 infants (150 without brain injury and 37 with mild AHT) who presented to a tertiary care children’s hospital for evaluation of nonspecific symptoms (R.P.B., unpublished data). A 5-step CPR derivation process was performed.16 Five predictor variables were identified: age ≥3 months, head circumference percentile >90%, serum hemoglobin <11.2 g/dL, abnormality on neurologic or dermatologic examination, and a previous emergency department (ED) visit for a high-risk symptom. The receiver operator characteristic (ROC) curve using these predictor variables showed an area under the curve (AUC) of 0.87 (95% confidence interval [CI] 0.80–0.95).
The current study was designed as a multicenter, prospective validation and refinement of the PIBIS CPR.
The protocol was approved by the institutional review boards at Children’s Hospital of Pittsburgh of UPMC (CHP), Primary Children’s Hospital in Salt Lake City, Utah (SLC), and Ann & Robert H. Lurie Children’s Hospital of Chicago (CHG). Enrollment at CHP started on October 1, 2006, SLC on June 1, 2010, and CHG on January 1, 2011. Consent was obtained except in cases of suspected abuse in which a waiver of informed consent was approved by all 3 institutional review boards.
Children were eligible if they were 30 to 364 days of age, well-appearing, and presented to a participating ED with a temperature <38.3°C, without a history of trauma and for evaluation of a symptom that is associated with an increased risk of AHT11,12 (Table 1). The only exclusion criterion was having a previously abnormal computed tomography (CT) scan of the head.
History of present illness, past medical history including previous ED visits, results of laboratory and radiologic testing, neurologic and dermatologic examination findings, serum hemoglobin, head circumference, and discharge diagnoses were collected. Neurologic and dermatologic examinations were recorded prospectively by the attending ED physician by using a checklist (Fig 1). If serum hemoglobin or neuroimaging were not done for clinical care, consent was obtained to participate in research. Due to increasing concern by treating physicians regarding radiation exposure, no research CTs were performed after May 1, 2012. No data about socioeconomic status or social history were collected.
Subjects were tracked by medical record review for 6 months after enrollment or up to 1 year of age, whichever came later. The goal of follow-up was to identify subjects with abnormal neuroimaging during the follow-up period and/or those who had neuroimaging performed to follow up on symptoms at enrollment.
Classification of Subjects
Subjects were classified as cases or controls based on the results of neuroimaging. Neuroimaging was classified dichotomously by using a paradigm developed a priori (Table 2). Controls included subjects with normal neuroimaging or no neuroimaging at enrollment and during follow-up. Cases included subjects with abnormal neuroimaging at enrollment or during follow-up. Cases were further classified into those with possible traumatic, probable/definite traumatic, and atraumatic abnormalities.
CTs and MRIs were interpreted as part of clinical care and by a study neuroradiologist (L.F.). When there was a difference between interpretations, a pediatric neurosurgeon (E.C.T.K.) reviewed the images and consensus was reached. SLC and CHG images were de-identified.
AHT diagnosis was defined as a brain injury that was assessed by each site’s hospital-based Child Protection Team (CPT) as being due to definite or probable, but not possible, abuse. Using the assessment of the CPT is a common way to define AHT in clinical research.1,12,18
Demographics, descriptive data, and site-to-site comparisons were calculated by using descriptive statistics. A P < .05 was considered statistically significant. Listwise deletion was used to handle missing data. All statistics were performed using IBM SPSS version 22.0 (IBM SPSS Statistics, IBM Corporation, Chicago, IL).
PIBIS Validation and Refinement
The 5 predictor variables identified in the retrospective derivation were reevaluated; variables with a nonsignificant log odds ratio and that failed to increase logistic regression model fit statistics when predicting group (case versus control) were dropped from further analysis. For the remaining predictor variables, the cut-points identified in the retrospective derivation, as well as several additional cut-points, were tested. Additional cut-points were examined to see whether minor changes would improve the odds ratio, model fit statistics, and/or accuracy of PIBIS. The additional cut-points were based on mathematical, rather than clinical, criteria. For example, the previous cutoff for hemoglobin was 11.3 g/dL; 11.1, 11.2, 11.4, and 11.5g/dL were, therefore, tested. Each predictor variable was then recoded as a binary value and logistic regression was used to evaluate the relationship between each binary predictor variable and group.
Weights were assigned to each variable based on the relative magnitude of the regression coefficients in the logistic regression and these were used to generate the PIBIS. The formulas developed with these weights were evaluated by using ROC curves.
From October 2006 to April 2014, 1040 subjects were enrolled: 801 at CHP, 138 at SLC and 101 at CHG. Mean (SD) age was 4.7 (3.1) months; 52% were boys and 78% were white. Fifty-nine percent came directly to the ED and 40% were transferred from a physician office (30%) or referring hospital (10%). Subjects presented due to seizurelike activity (25%), apparent life-threatening event (ALTE) (19%), vomiting (16%), fussiness (16%), scalp swelling (8%), bruise (6%), or a combination of symptoms (10%).
Classification of Subjects as Cases Versus Controls
Of the 1040 subjects, 826 (79%) were classified as controls and 214 (21%) as cases. There was no difference between sites in the proportion of cases. Overall, 507 (61%) of 826 of controls underwent head CT and/or MRI at enrollment or during follow-up. Eighty-three (10%) of the CTs were performed for research.
Among the 214 cases, 213 had neuroimaging at enrollment. The case without imaging at enrollment presented with seizures 2 weeks after enrollment; MRI showed chronic subdural hemorrhage. Of the 214 neuroimaging abnormalities, 166 (78%) were probable/definite trauma, 29 (13%) were possible trauma, and 19 (9%) were atraumatic abnormalities that included hydrocephalus (n = 6), cortical dysplasia (n = 7), tumors (n = 2), stroke (n = 1), arteriovenous malformation (n = 1), craniosynostosis (n = 1), and tuberous sclerosis (n = 1).
AHT was diagnosed in 109 subjects; all 109 were classified as probable/definite trauma by the paradigm in Table 2. There were also 51 subjects assessed by the CPT as having possible AHT; 28 had skull fractures without intracranial hemorrhage, 12 had skull fractures with an underlying hemorrhage, 5 had chronic subdural hemorrhages, and 5 had acute extra-axial hemorrhage with atypical clinical circumstances (eg, moderate/severe volume loss).
With the exception of children who presented with scalp swelling, in whom 75 (87%) of 86 were cases, a similar proportion of subjects with each presenting symptom was classified as cases: 35 (14%) of 260 infants with a seizure, 23 (12%) of 197 with an ALTE/apnea, 24 (15%) of 165 with fussiness, 19 (12%) of 161 with vomiting, and 13 (20%) of 65 with a bruise.
Subjects With Noncranial Abuse
Sixty-nine subjects without brain injury were diagnosed with noncranial abuse; 34 had abusive fracture(s) and 35 had abusive dermatologic injury.
Evaluation of Predictor Variables By Using Logistic Regression
The variables “a previous ED visit for a high-risk symptom” and “neurologic examination” had nonsignificant log odds ratio and failed to increase the logistic regression model fit statistics and were, therefore, removed from further analysis. The previously established cut-points for the other predictor variables were robust; the only variable for which there was a change in the cutoff relative to the derivation was head circumference, which decreased from 90% to 85%. When all 3 sites were included, the overall χ2 test was statistically significant (χ2 = 243.18, P < .001). Using the cut-points in the logistic regression yielded sensitivity and specificity that were nearly identical to the data in their continuous form (Table 3).
The 5-point PIBIS was computed by using the following weights: abnormal dermatologic examination (2 points), age ≥3.0 months (1 point), head circumference >85th percentile (1 point), and serum hemoglobin <11.2 g/dL (1 point) (Table 4). The ROC curve using these cutoffs had an AUC of 0.83 (95% CI 0.80–0.86) (Fig 2). The AUCs for each site were similar: CHP 0.83 (95% CI 0.80–0.86), CHG 0.82 (95% CI 0.72–0.91), and SLC 0.83 (95% CI 0.74–0.91).
Eighty-one percent of subjects had data available for all 4 CPR variables, 18% had 1 missing variable, and 1% had 2 missing variables. There was no difference between sites in terms of the proportion of subjects with full data (χ2 = 9.35, P = .53). There was a difference in the proportion of subjects with missing data by group: 3% of subjects with abnormal neuroimaging, 11% of subjects with normal neuroimaging, and 41% of subjects without neuroimaging had missing data.
Sensitivity, Specificity, and Negative and Positive Predictive Values
Sensitivity, specificity, negative predictive value (NPV) and positive predictive value (PPV) were calculated for the 862 subjects with complete data. The sensitivity for identification of abnormal neuroimaging at a score of 2 was 93.0% (95% CI 89.0%–96.0%). The specificity for detection of abnormal neuroimaging at a score of 2 was 53.0% (95% CI 49.0%–57.0%). The NPV of a score of <2 for detection of abnormal neuroimaging was 96.0% (95% CI 93.6%–97.9%), meaning that if a patient had a score of 0 or 1, the clinician could be 96% confident that the infant did not have a brain injury. The PPV of a score ≥2 for detection of abnormal neuroimaging was 39.0% (95% CI 34.8%–43.6%). There was no difference in the sensitivity, specificity, NPV or PPV among sites (Table 5).
Of the 69 subjects with noncranial abuse, 64 had data available for all 4 predictor variables and 89% (57/64) had a PIBIS score ≥2.
This multicenter prospective CPR validation demonstrates that 4 easily measureable clinical variables (dermatologic examination, age, head circumference, and serum hemoglobin) can help physicians assess which well-appearing high-risk infants might benefit from a head CT. The consistency in the data among the 3 sites and between the retrospective derivation and the prospective validation support the robustness of this CPR. As with all CPRs, PIBIS is meant to supplement, not replace, clinical judgment. PIBIS was designed to be used in well-appearing infants in whom brain injury may not be part of the initial differential diagnosis; it should not be used on infants needing emergent neuroimaging.
The goal of PIBIS is not to decrease the number of infants who undergo the screening test, head CTs in this case. Instead, the goal is to target head CTs to those infants who are most likely to have a positive result, an important goal given the growing concern about head CT use in young children.19 Consistent with this goal is that the proportion of infants who would undergo head CT by using PIBIS (55%) is almost identical to the proportion who underwent head CT as part of routine clinical care (59%). In the case of mild AHT in which published data suggest that clinical judgment has a sensitivity of ∼70%,12 our validation suggests that this sensitivity can be increased to >90% without an increase in head CT use. At the same time, the very high NPV when the PIBIS is 0 or 1 (96% [95% CI 93.6%–97.9%]) demonstrates that there is a cohort of low-risk infants who can be safely discharged without neuroimaging.
PIBIS is not designed to diagnose AHT, but to prompt the treating physician to consider the brain as a possible etiology of an infant’s symptoms. This is an important distinction because brain abnormalities, such as a brain tumor, can present with the same symptoms as AHT. That just >50% (109/214) of the children with abnormal neuroimaging had definite/probable AHT is evidence that AHT is just one of many conditions that may present with soft neurologic signs. At the time PIBIS is applied, the cause of the presenting symptom is not known; as a result, sensitivity, specificity, NPV, and PPV were calculated for brain abnormalities in general rather than for AHT in particular. Importantly, PIBIS appears to be able to identify infants with both atraumatic and traumatic abnormalities. If a head CT demonstrates abnormalities that may be consistent with AHT, then a full evaluation for AHT would need to be strongly considered. The relatively small number of patients with atraumatic conditions relative to the number of patients with AHT is consistent with the fact that the incidence of AHT is higher than that of brain tumors20 or hydrocephalus.21 The incidence of AHT in this study was higher than what we predicted when performing our initial sample size calculation. This was likely because the study took place over a period that included the Great Recession when the rate of AHT increased in multiple regions of the country, including western Pennsylvania, where most of the study subjects were enrolled.22,23
PIBIS was developed to maximize sensitivity with the goal of being able to identify >90% of well-appearing infants with AHT. Our data suggest that using PIBIS with a cutoff score of 2 would identify >90% of children with abnormal neuroimaging with a specificity of just >50%. Increasing the cutoff to 3 would decrease sensitivity to 81%, but would increase the specificity to 75%. Deciding whether to recommend a cutoff of 2 or 3 depends, in part, on the risk of neuroimaging. Although head CT is currently the gold-standard test to evaluate for brain injury in an ED setting and the one that was used in the current study, new data suggest that a rapid brain MRI may be a viable alternative.24–26 By eliminating radiation exposure, a rapid brain MRI would change the risk-benefit ratio and the PIBIS score at which one might obtain neuroimaging.
The fact that close to 90% of the subjects with noncranial abuse had a PIBIS score of ≥2 is related to the dermatologic variable. Only dermatologic abnormalities consistent with trauma, and not atraumatic findings, such as rashes, were included in this assessment. Studies have demonstrated high rates of abnormal head CTs in infants with bruising,13,27 which has led the American Academy of Pediatrics to recommend neuroimaging in nonmobile infants with noncranial abuse.28 PIBIS is, therefore, consistent with current clinical recommendations.
The lack of predictive value of “a previous ED visit for a high-risk symptom” and “neurologic examination” was surprising given their predictive strength in the derivation. We hypothesize that the lack of predictive value of the previous ED visit relates to recall bias in the retrospective derivation. Physicians routinely ask about previous ED visits when evaluating an infant for suspected abuse, but may not ask about this history for all infants presenting with nonspecific complaints. The lack of predictive value of the neurologic examination likely relates to the transient and subtle nature of neurologic abnormalities, such as fussiness.
We recognize that there is controversy about whether to dichotomize continuous predictor variables.29 Although accuracy is clearly important, ease of use is also critical for success of any CPR. For this reason, we dichotomized the predictor variables, similar to other studies.16,30,31
Because not all subjects had neuroimaging, some cases may have been misclassified as controls. By tracking all subjects during the follow-up period, we sought to decrease the possibility of misclassification. Furthermore, both CHP and SLC, in which 90% of the subjects were enrolled, are the only regional level I trauma centers; all children with suspected AHT in these regions would be evaluated at the participating hospital.
We cannot assess whether the study itself affected physicians’ practice. In most cases, the treating physician obtained consent to allow investigators to approach families and was, therefore, aware that the study was designed to develop a CPR for identification of brain injury. This may have prompted the treating physician to consider brain injury in the differential diagnosis and, therefore, order a head CT.
A final limitation is this was a convenience sample. During the study period, other children with similar symptoms were evaluated in the study EDs but were not approached for enrollment. There is no reason to believe, or are there data to suggest, that infants who come to an ED with nonspecific symptoms at 2 am, for example, are different from infants who come at 4 pm as it relates to the variables in PIBIS or to the likelihood of having a brain abnormality.
In conclusion, our study suggests that PIBIS can identify infants at increased risk for brain injury who should undergo neuroimaging. As with all CPR, implementation analysis is essential before incorporating PIBIS into clinical practice to determine whether PIBIS improves identification of AHT and/or changes the use of neuroimaging to screen for brain injury in the ED setting.
We thank Pamela Rubin, RN (Children’s Hospital of Pittsburgh of UPMC), Jane Flores, RN (Ann & Robert H. Lurie Children’s Hospital in Chicago), and Kammy Jacobson, RN (Primary Children’s Hospital) for data collection, and Lauren McCullaugh, MPH (Hofstra North Shore Long Island School of Medicine) for data analysis and interpretation.
- Accepted April 5, 2016.
- Address correspondence to Rachel P. Berger, MD, MPH, Department of Pediatrics, Division of Child Advocacy, Children’s Hospital of Pittsburgh of UPMC, 4401 Penn Ave, Pittsburgh, PA 15224. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: The study was funded by National Institutes of Health grant R01HD055986 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development. The National Institutes of Health was involved in the design of the study. Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: Dr Herman has served as a paid expert witness in cases of alleged physical abuse. The other authors have indicated they have no potential conflicts of interest to disclose.
COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2016-1190.
- Parks S,
- Annest J,
- Hill A,
- Karch D
- US Department of Health and Human Services
- Haviland J,
- Russell RI
- Morris MW,
- Smith S,
- Cressman J,
- Ancheta J
- Rubin DM,
- Christian CW,
- Bilaniuk LT,
- Zazyczny KA,
- Durbin DR
- Kharbanda AB,
- Taylor GA,
- Fishman SJ,
- Bachur RG
- Berger RP,
- Dulani T,
- Adelson PD,
- Leventhal JM,
- Richichi R,
- Kochanek PM
- Ostrom QT,
- Gittleman H,
- Liao P, et al
- Berger RP,
- Fromkin JB,
- Stutz H, et al
- Ho C,
- Yasrebi M,
- Kralik S,
- Hicks R,
- Harris M,
- Hibbard R
- Becker J,
- Kubal W,
- Hur S, et al
- Copyright © 2016 by the American Academy of Pediatrics