BACKGROUND AND OBJECTIVES: Abusive head trauma (AHT) may be missed in the clinical setting. Clinical prediction tools are used to reduce variability in practice and inform decision-making. From a systematic review and individual patient data analysis we derived the Predicting Abusive Head Trauma (PredAHT) tool, using multilevel logistic regression to predict likelihood of AHT. This study aims to externally validate the PredAHT tool.
METHODS: Consecutive children aged <36 months admitted with an intracranial injury, confirmed as abusive or nonabusive, to 2 sites used in the original model were ascertained. Details of 6 influential features were recorded (retinal hemorrhage, rib and long -bone fractures, apnea, seizures, and head or neck bruising). We estimated the likelihood of an unrecorded feature being present with multiple imputation; analysis included sensitivity, specificity, and area under the curve, with 95% confidence intervals (CIs).
RESULTS: Data included 133 non-AHT cases and 65 AHT cases, 97% of children were <24 months old. Consistent with original predictions, when ≥3 features were present in a child <36 months old with intracranial injury, the estimated probability of AHT was >81.5% (95% CI, 63.3–91.8). The sensitivity of the tool was 72.3% (95% CI, 60.4–81.7), the specificity was 85.7% (95% CI, 78.8–90.7), area under the curve 0.88 (95% CI, 0.823–0.926).
CONCLUSIONS: When tested on novel data, the PredAHT tool performed well. This tool has the potential to contribute to decision-making in these challenging cases. An implementation study is needed to explore its performance and utility within the child protection process.
- AHT —
- abusive head trauma
- CDR —
- clinical decision rule
- CI —
- confidence interval
- ICI —
- intracranial injury
- MICE —
- multiple imputation by chained equations
- nAHT —
- nonabusive head trauma
- NPV —
- negative predictive value
- PPV —
- positive predictive value
- PredAHT —
- Predicting Abusive Head Trauma
- RH —
- retinal hemorrhage
What’s Known on This Subject:
A previous multivariable statistical model, using individual patient data, estimated the probability of abusive head trauma based on the presence or absence of 6 clinical features: rib fracture, long-bone fracture, apnea, seizures, retinal hemorrhage, and head or neck bruising.
What This Study Adds:
The model performed well in this validation, with a sensitivity of 72.3%, specificity of 85.7%, and area under the curve of 0.88. In children <3 years old with intracranial injury plus ≥3 features, the estimated probability of abuse is >81.5%.
Many clinicians are involved in the care of infants and young children with head trauma, where the diagnosis of abusive head trauma (AHT)1 is often a consideration. Children with AHT present with many different combinations of clinical features, and there is ongoing controversy and debate about which clinical features support this diagnosis and clinicians’ ability to arrive at an evidence-based diagnosis.2,3 Clinicians who undertake child protection investigations of children presenting with head injury can meet resistance and face a barrage of criticism for even considering a diagnosis of AHT.4
Many areas of medicine use clinical decision rules (CDRs), which assist in “screening in” or “screening out” cases of a particular condition.5,6 These rules rely on the identification of a combination of clinical features from the history and examination, with or without associated investigations, to improve the accuracy of clinical decision-making, overcoming variability between clinicians. There are few CDRs in the child protection field, and the evidence for the diagnostic value of screening tools based on physical examination of children without previous suspicion of child maltreatment is weak,7 yet there is pressure to improve detection of such cases. A recently validated CDR with a high sensitivity of 96% (95% confidence interval [CI], 90%–99%) has been developed for the pediatric intensive care setting for nonvehicular traumatic brain injury to help exclude AHT.8,9 This approach has been questioned by some,10 on the grounds that the pretest probability of AHT was as high as 45%, in which case there is an argument that all children should be screened for the condition, and a CDR of this kind is not justified. Clearly, the optimal CDR in this setting would have high sensitivity and specificity without incurring extensive investigations that are not otherwise clinically warranted and may expose children to unnecessary doses of radiation.
With estimated annual incidence of AHT of 12.8 to 17 per 100 000 children <2 years old and 21 to 33.8 per 100 000 children <1 year old,11–14 many pediatricians may see only a handful of cases in their practice. Yet given the backlash physicians may face15 and expectations that those testifying in the courts must be able to support their opinions with scientific evidence,16,17 consolidating the scientific evidence into decision-making is paramount.
Although many high-quality studies have been conducted in this field,18–20 one of the challenges is obtaining a sufficient sample to identify discriminating features. Clinicians have tended to rely on a Bayesian approach to decision-making within this complex area, using existing information along with the pattern of injuries observed to obtain posterior probabilities of abuse.21,22 There has also been an increasing emphasis on going beyond the simple triad of injuries that is often cited when arriving at a diagnosis of AHT, namely retinal hemorrhage (RH), subdural hemorrhage, and encephalopathy.23 One major criticism of CDRs is how rarely they are developed appropriately, that is, derived by identifying appropriate elements from previous high-quality literature; applied to a relevant population with sufficient sample size to evaluate each element of the tool; validated on a novel data set in an appropriate multicenter population; and implemented in practice. Our original tool was derived from a rigorous systematic review of world literature.18 Thanks to the generous sharing of original data on 6 key clinical features from the highest-quality publications,24–29 we developed a tool to predict AHT, using a data set of 1053 cases of children with intracranial injury (ICI).30 We refer to this as the PredAHT tool. In the original study, we validated our model by partitioning the data into 5 random subsets and assessed the accuracy of our predictions by using cross-validation. However, external validation of a statistical model is important because, inevitably, regression techniques capture not only the signal but also some of the noise in putative relationships between potentially diagnostic features and observed etiology. Therefore, regression models tend to perform less well in sets of data other than the one in which their coefficients were estimated, where the noise component may be different. In the context of child protection, it is important to quantify any such degradation in performance and to determine the degree to which findings are generalizable to future cases of head injury. It cannot be assumed that even a well-derived tool will be successfully validated on a novel data set; a recent external validation exercise examining 3 screening tools for detecting psoriatic arthritis failed to demonstrate fully satisfactory performance of any of the proposed tools.31
This study aims to validate the previously developed PredAHT tool to provide a prediction of the probability of AHT in children aged <36 months presenting with an intracranial injury. This tool may assist clinicians in their discussions with social welfare, law enforcement, or other professionals involved in the child protection process.
We ascertained details from consecutive children aged <36 months admitted to 2 centers with an ICI confirmed as either abusive (AHT) or nonabusive (nAHT) in origin. Cases where the final outcome (AHT or nAHT) was unknown were classified as “indeterminate.” To accurately assess the predictive accuracy of the PredAHT tool, a known outcome is needed, and so indeterminate cases were excluded. Cases were ascertained from centers that were included in the original model derivation. The outcome for each case (AHT or nAHT) was recorded. We defined AHT as ICI where physical abuse had been confirmed as a cause. To minimize circularity (ie, the risk that decisions about abuse were based solely on the injuries in question), only confirmed cases of AHT were included, those ranked 1 or 2 for abuse, according to our previously published quality standard18 (Table 1). Importantly, higher-ranked cases included a thorough multidisciplinary assessment or court proceedings, social and historical factors beyond the presenting injury, or a perpetrator admission or independently witnessed abusive incident. The nonabused children had either witnessed accidental mechanisms or confirmed organic causes or had abuse excluded after child protection investigations.
Participants: Data Set 1 (Cardiff)
Data set 1 (Cardiff) consists of retrospective data collection from the case notes of 60 children aged <36 months needing hospital admission with an ICI (confirmed on neuroimaging) to the University Hospital of Wales, United Kingdom, between April 2005 and February 2012. ICI was defined as the presence of any combination of: extraaxial hemorrhage, diffuse or focal parenchymal injury, cerebral edema, cerebral contusion, hypoxic ischemic injury, or diffuse axonal injury.
Participants were identified from the hospital radiology database for children undergoing neuroimaging, supplemented by a search of the Pediatric Intensive Care database and Child Protection database. We excluded ineligible cases, defined as those with normal neuroimaging, age >36 months, an underlying structural abnormality or preexisting disease (hydrocephalus, cystic lesion or tumor, metabolic cause, malformation, or abnormal brain development), injuries caused by neglect, and birth injury. There were no “indeterminate” cases in this data set. Cases were anonymized, and data were extracted and entered into an Excel spreadsheet. Ethical approval was granted by the Research Ethics Committee for Wales on October 13, 2011.
Participants: Data Set 2 (Lille)
M. Vinchon and colleagues kindly provided prospective data on 143 anonymized cases from their large-scale prospective study of abusive and accidental head injuries in infants conducted at Lille University Hospital, France. Data collection has been ongoing since 2001, and the authors had originally provided us with a subset of their total data for use in our initial pooled analysis.30 For the purposes of this validation, all cases analyzed previously were omitted, and subsequent cases ascertained consecutively have been included, some of which have previously been published.24,32 All children <24 months of age referred alive to the neurosurgical department, the PICU, or the emergency department were included. We excluded 5 “indeterminate” cases from this data set because the outcome was not recorded, yielding a final number of 138 cases for this data set.
Data were collected for the 6 features of interest (Table 2). Each feature (RH, rib fracture, long-bone fracture, head or neck bruising, seizures, and apnea) was entered as 1 of 3 possible outcomes: present, absent, or missing data (“not recorded”). RHs were documented after a full indirect ophthalmologic examination, with pupils dilated, conducted by a pediatric ophthalmologist. Clinical details were recorded, and Retcam images were obtained on the majority. Details of skeletal imaging were obtained from the radiology database. Rib or long-bone fractures were deemed “absent” if appropriate imaging had been performed and no relevant fractures were seen or “missing” if the imaging had not been performed. Remaining details were obtained from the clinical record completed in the PICU. If apnea was asked about and absent in the initial history or not recorded during inpatient stay, it was deemed absent. Head or neck bruising was noted as “present” or “absent” if recorded as such in the case notes and was otherwise deemed as “missing.” Seizures included any documented seizures, from single seizures to status epilepticus. Overall, AHT was diagnosed in 33% of children (65/198). There were 16% of cases (31/198) without missing information; missing data are reported in Table 3.
Based on previous work, a sample size of 30 cases would give confidence limits of 15% to 20% around the values for sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).30 As sample size increases, these confidence limits decrease.
Because the presence or absence of the 6 clinical features was not always recorded in our validation data, imputation was needed to estimate the probability that a feature was present. These estimates can then be combined with knowledge of observed features in the regression model to determine an estimated probability of AHT for each child. It is important to note that because imputation is needed, the outcome of our validation exercise depends both on the quality of the regression model and the quality of the imputations.
We estimated the likelihood of an unrecorded feature being present by using multiple imputation by chained equations (MICE).33 This technique formally models each missing feature through regressions on all other variables, and it depends on a “missing at random” assumption. Informally, this means that the reason for particular features being unrecorded may depend on the recorded presence or absence of features for that child but not on any unrecorded presences or absences. MICE is becoming a standard strategy for dealing with missing information, but in our case it may slightly overestimate the chance of a missed or unrecorded feature that is in fact present. An alternative might be to assume unrecorded features were in fact present, but this seems likely to overestimate to an even greater degree. As before, we used a cutoff of >80% to define predicted AHT and <20% to define predicted nAHT.
Of the 198 children, 92% of those in data set 1 and 99% of those in data set 2 were aged <24 months. The proportion of AHT was 33% and was not significantly different between the 2 data sets; two-thirds of the children in each data set were boys (Table 3). The most commonly missing data from each data set related to long-bone or rib fractures; these features were recorded more often in abuse cases, because a skeletal survey may be inappropriate in nAHT cases. There were more data related to seizures in this validation data set than in the original derivation data set. Figure 1 details how the PredAHT tool performed, based on the predicted probabilities of abuse.
As can be seen, the model performed well. The model was correct for 82% (109/133) nAHT cases. Twelve cases where an accidental mechanism was independently confirmed were predicted as AHT. Among these cases, the presence of RH and apnea influenced the prediction of abuse.
The model was correct for 66% (43/65) AHT cases. Thirteen abuse cases were predicted as nAHT. Among these cases there were minimal positive findings, explaining the model’s prediction of nAHT, although a comprehensive child protection investigation confirmed abuse.
The size of the data points reflects the number of documented features (Fig 1). As can be seen, the cases with a predicted probability of 20% to 80% have fewer recorded features. Overall this validation using MICE is consistent with the original predictions, although the estimated probability of AHT when ≥3 of the 6 features are present is slightly lower, at 81.5% (95% CI, 63.3%–91.8%). The sensitivity of this tool with a cutoff of 50% is 72.3% (95% CI, 60.4%–81.7%), and the specificity is 85.7% (95% CI, 78.8%–90.7%). The PPV is 71.2% (95% CI, 59.4%–80.7%), and the NPV is 86.3% (95% CI, 79.5%–91.2%). The receiver operating characteristic curve displays the sensitivity and specificity of the tool given different cutoff values. We also display receiver operating characteristic curves corresponding to the 95% confidence limits for the area under the curve (Fig 2). The area under the curve is 0.88 (95% CI, 0.823–0.926).
This validation of the previously derived PredAHT tool to aid in the assessment of possible AHT has confirmed the original findings, namely that when ≥3 of these 6 features are present, the estimated probability for AHT is >81.5% (95% CI, 63.3%–91.8%). The model performed reasonably well, with a sensitivity of 72.3% (95% CI, 60.4%–81.7%), a specificity of 85.7% (95% CI, 78.8%–90.7%), a PPV of 71.2% (95% CI, 59.4%–80.7%), and an NPV of 86.3% (95% CI, 79.5%–91.2%). The strength of this work is that the validation has been conducted on a multicenter basis with data from 2 of the original 6 sites, the majority of which were prospectively obtained. The PredAHT tool is proposed as an assistive clinical prediction tool rather than a CDR, which would typically recommend a direct course of action.34 Prediction tools offer diagnostic or prognostic probabilities, through scores or algorithms, to assist with rather than dictate decision-making.34
Given the rarity of AHT for clinicians who are not child abuse specialists, it is important to be able to explicitly define which combination of clinical features carries a high probability of abuse when a clinical workup has been conducted. Knowledge of the results of this prediction tool may assist these clinicians in their discussions with child abuse specialists, in addition to facilitating discussions between child abuse specialists and social welfare, law enforcement, or other professionals involved in the child protection process. There are a number of points along the clinical pathway for infants and young children with head injury where one could use a CDR: to determine which children should undergo neuroimaging in the first instance,35 to exclude abuse when the results of the tool are negative,8,9 or to assist in determining the likelihood of abuse, once all the clinical information is known. The recently validated tool developed by the Pediatric Brain Injury Research Network investigators offers a high sensitivity (96%; 95% CI, 90–99) but at the expense of low specificity (43%; 95% CI, 35–50),9 whereas the PredAHT tool performs as a clinical prediction tool with a sensitivity of 72.3% (95% CI, 60.4–81.7) and a specificity of 86.3% (95% CI, 78.8–90.7) with a 50% cutoff applied. The sensitivity and specificity depend on the cutoff point applied to the probability scale. Each chosen cutoff point gives rise to different performance. If high sensitivity is desired, a lower cutoff point is chosen; for instance, to achieve a sensitivity >95%, a probability cutoff point of 9% could be selected (corresponding to a specificity of 45%). Alternatively, if high specificity is needed, a cutoff point >50% may be used. This is true not only of our tool but of tools in general; arbitrarily high sensitivity or specificity (but not both at once) can always be achieved if all (or none) of the patients are assigned to the relevant group.
Clearly it is vital that when any child with suspected AHT is assessed, all details of the case, including the precise pattern of ICI present, RH, and fractures identified, are carefully considered. However, developing a CDR that can account for the precise pattern of each of these possible variables would require thousands of children. Certain patterns of ICI have a stronger association with AHT, as demonstrated in an updated meta-analysis.36 It is increasingly clear that subdural hemorrhages in children <2 years old indicate a high probability of abuse, particularly when they are interhemispheric, multiple, over convexities, or bilateral.36 With increasing emphasis on the contribution of hypoxia and increasing use of MRI, it is unsurprising that the presence of hypoxic ischemic injury and diffuse axonal injury is also strongly associated with AHT.36 Likewise, the precise pattern of RHs identified has been used to derive a CDR.37 Although this study was based on a very small number of children with RHs (18 AHT, 8 nonabusive ICI) from a single center, it reiterated recognized associations with specific patterns of RH.38
A strength of the data underpinning the PredAHT tool is the standardization of the data entered into the analysis. All data contributed by M. Vinchon have been prospectively ascertained and include witnessed accidents and some cases of witnessed abuse. The Cardiff data were collected retrospectively, but all cases had a standardized assessment recorded in the case notes. Although a number of CDRs have been created for assisting in identifying children who may have been abused, few have been validated on a novel multicenter population.7,39 Many tools are generic and aim to screen all children attending an emergency department, as is mandated in Holland, but such tools have failed to perform well.7 It is clear from the literature that CDRs perform best when they are aimed at detecting a single condition and combine ≥3 features.7 A recent review of CDRs to detect which children with head injury should undergo computed tomography imaging highlighted that a CDR should include clinically reasonable variables,40 with minimal ambiguity. Multiple studies over the past 15 years have verified that the variables included in the PredAHT tool are clinically relevant, and when a “present or absent” classification is used, ambiguity is avoided.
The PredAHT tool should assist clinicians in reaching their decisions but will not confirm or exclude AHT in isolation. However, it is vital to record the presence or absence of each of the key clinical features, including apnea and seizures, to optimize decision-making. When features were missing, we imputed the data. Other imputation strategies include the (arguably “naive”) approach of deeming a feature absent if its presence or absence is unrecorded. This has the potential to bias results. Even in theory, it can only underestimate the prevalence of clinical features, and it seems likely to do so in practice. Another method is to group together similar children based on a shared presence or absence of features, which increases the number of cases with predicted probabilities of 20% to 80%. Other possibilities include imputing the missing data in abuse cases as present and the missing data in nonabuse cases as absent, representing a best-case scenario, or conversely imputing the missing data in abuse cases as absent and the missing data in nonabuse cases as present, representing a worst-case scenario. Missing information can be framed as a kind of measurement error, where our imputed values deviate from the truth in 1 direction or the other. Under some fairly mild assumptions about the nature of these errors, it follows that the performance of our tool would actually improve if this “measurement error” (missing data) could be eliminated. In short, missing data means that (for example) sensitivity and specificity are likely to be underestimated, not overestimated. It is on this conservative basis that we believe our conclusions to be justified. The different imputation strategies explored and their impact on the tool’s predicted probabilities are detailed in the Supplemental Information.
The limitations of this work include missing data, which is inevitable in this field because not all children undergo full skeletal survey or formal ophthalmology if there is no clinical indication to do so. Future impact evaluation should aim to maximize recorded items. To maximize its utility, the PredAHT tool relies on dichotomized data, but it is to be expected that clinicians would take into account more nuanced details available to them. Furthermore, the circularity of the model must be considered; in the absence of a gold standard test, we based the diagnosis of AHT not on clinical features alone but rather on a multidisciplinary assessment of all aspects of the case. However, unavoidably, this assessment will be influenced by the features included in the model.
Whereas Findley and colleagues suggest that “there continues to be very little objective guidance on how to distinguish between accidental, non-accidental and natural causes of findings,”3 we believe that this validation study assists in providing objective criteria to assist in making this distinction. In the era when the very diagnosis of AHT continues to be contested in both the public and legal domain, adequately validated CDRs provide more scientific evidence to support clinical decision-making. It is a criticism of CDRs, including those in widespread clinical use or even incorporated into national guidelines,41 that they are rarely validated or undergo clinical impact analysis.40 The PredAHT tool has been derived from a systematic review, supported by primary multicenter data, and now validated on a novel multicenter data set.
We thank Mathieu Vinchon for generously providing us with individual patient data for analysis in this validation study.
- Accepted May 26, 2015.
- Address correspondence to Laura Elizabeth Cowley, MSc, MBPsS, BSc, School of Medicine, Heath Park, Cardiff CF14 4XN, United Kingdom. E-mail:
Ms Cowley drafted the Methods section, produced the tables, revised the article, formatted the article, and contributed to overall editing; Dr Morris collected the data in data set 1 and entered all anonymized data into the database; Dr. Maguire had the initial study idea, drafted and revised the article, and contributed to the study design and overall editing; Dr Farewell designed and conducted all statistical analyses, drafted the statistical components of the article, produced the figures, and contributed to overall editing; Prof Kemp revised the article and contributed to overall study design and editing; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Christian CW,
- Block R,
- Committee on Child Abuse and Neglect,
- American Academy of Pediatrics
- Tuerkheimer D
- Findley KA,
- Barnes PD,
- Moran DA,
- Squier W
- ↵D v East Berkshire Community Health NHS Trust, MAK v Dewsbury Healthcare NHS Trust, RK v Oldham NHS Trust. UKHL 23 at paragraph 60. 2005
- Pickering A,
- Harnan S,
- Fitzgerald P,
- Pandor A,
- Goodacre S
- ↵Hymel KP, Armijo-Garcia V, Foster R, et al. Validation of a clinical prediction rule for pediatric abusive head trauma. Pediatrics. 2014;134(6). Available at: www.pediatrics.org/cgi/content/full/134/6/e1537
- Jayawant S,
- Rawlinson A,
- Gibbon F,
- et al
- ↵Tuerkheimer D. Anatomy of a misdiagnosis. The New York Times. September 20, 2010. Available at: www.nytimes.com/2010/09/21/opinion/21tuerkheimer.html. Accessed October 24, 2013
- Narang S
- ↵Royal College of Pathologists, Royal College of Paediatrics and Child Health. Sudden unexpected death in infancy: a multi agency protocol for care and investigation. September 2004. Available at: www.rcpath.org/Resources/RCPath/Migrated%20Resources/Documents/S/SUDI%20report%20for%20web.pdf. Accessed May 27, 2014
- Maguire S,
- Pickerd N,
- Farewell D,
- Mann M,
- Tempest V,
- Kemp AM
- Core Info. Neurological injuries. October 2014. Available at: www.core-info.cardiff.ac.uk/reviews/neurological. Accessed October 30, 2014
- Kemp AM,
- Jaspan T,
- Griffiths J,
- et al
- ↵Crown Prosecution Service. Non Accidental Head Injury Cases (NAHI, formerly referred to as Shaken Baby Syndrome [SBS]): Prosecution Approach. March 24, 2011. Available at: www.cps.gov.uk/legal/l_to_o/non_accidental_head_injury_cases/#a01. Accessed May 27, 2014
- Kemp AM,
- Stoodley N,
- Cobley C,
- Coles L,
- Kemp KW
- Hobbs C,
- Childs AM,
- Wynne J,
- Livingston J,
- Seal A
- Bechtel K,
- Stoessel K,
- Leventhal JM,
- et al
- Hettler J,
- Greenes DS
- van Buuren S,
- Groothuis-Oudshoorn K
- ↵Core Info. What neuroradiological features distinguish abusive from non-abusive head trauma? May 2014. Available at: www.core-info.cardiff.ac.uk/reviews/neurological/what-neuroradiological-features-distinguish-abusive-from-non-abusive-head-trauma. Accessed May 27, 2014
- Lyttle MD,
- Crowe L,
- Oakley E,
- Dunning J,
- Babl FE
- ↵National Institute for Clinical Excellence. Head injury. Triage, assessment, investigation and early management of head injury in children, young people and adults. January 2014. Available at: www.nice.org.uk/guidance/cg176/resources/guidance-head-injury-pdf. Accessed July 11, 2014
- Copyright © 2015 by the American Academy of Pediatrics