Abstract
OBJECTIVES: To determine if children identified by a predictive risk model as at “high risk” of maltreatment are also at elevated risk of injury and mortality in early childhood.
METHODS: We built a model that predicted a child’s risk of a substantiated finding of maltreatment by child protective services for children born in New Zealand in 2010. We assigned risk scores to the 2011 birth cohort, and flagged children as “very high risk” if they were in the top 10% of the score distribution for maltreatment. We also set a less conservative threshold for defining “high risk” and examined children in the top 20%. We then compared the incidence of injury and mortality rates between very high-risk and high-risk children and the remainder of the birth cohort.
RESULTS: Children flagged at both 10% and 20% risk thresholds had much higher postneonatal mortality rates than other children (4.8 times and 4.2 times greater, respectively), as well as a greater relative risk of hospitalization (2 times higher and 1.8 times higher, respectively).
CONCLUSIONS: Models that predict risk of maltreatment as defined by child protective services substantiation also identify children who are at heightened risk of injury and mortality outcomes. If deployed at birth, these models could help medical providers identify children in families who would benefit from more intensive supports.
- CI —
- confidence interval
- CPS —
- child protective services
- CYF —
- Child, Youth and Family
- ICD-10-AM —
- International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Australian Modification
- MSD —
- Ministry of Social Development
- NMDS —
- National Minimum Dataset
- PRM —
- predictive risk model
- SUID —
- sudden unexpected infant death
What’s Known on This Subject:
Administrative information available at birth can be used to effectively risk stratify large populations of children based on the likelihood of future maltreatment. It is unknown whether these children are also at risk for other adverse health and safety outcomes.
What This Study Adds:
Although a relatively small number of children experience injury hospitalizations and death during the first 3 years of life, these children are concentrated among the highest risk decile of a model built using birth records and predicting child protection involvement.
The prevalence of child maltreatment in the US population is considerable; by 18 years of age, more than 1 in 3 children have been investigated by child protective services (CPS) (37.4%),1 and 12.5% of all children have been substantiated as victims of maltreatment.2 In their research, authors have indicated that beyond any immediate harm caused, child maltreatment is also associated with a range of poor health outcomes, including behavioral risk factors3 and preventable death.4,5 Given high disparities in the prevalence of abuse and neglect by socioeconomic status,1,6 maltreatment may be an important indirect and direct contributor to health disparities over the life course. Support services that engage women early in their pregnancy and successfully prevent conditions that contribute to child abuse and neglect may therefore provide a vehicle for improved health and well-being.
The early identification of children at risk for maltreatment using linked administrative data and predictive risk models (PRMs) has been shown to be both theoretically possible and practically feasible.7–9 The richness of administrative data coupled with advances in technology mean that computerized PRMs can be deployed to cost-effectively screen entire populations of newborns. In previous research, authors have demonstrated that these models have good predictive accuracy in identifying children at heightened risk of experiencing abuse and neglect.7,8 Although ethical and practical use cases for PRM are still being developed, newborn screening could be used to improve targeted primary prevention by prioritizing those families with the greatest need of interventions and support.10,11
An important question, however, is whether children identified as at risk for maltreatment as defined by CPS involvement, are also at risk for other adverse health and safety outcomes. In particular, if CPS is a proxy for “surveillance,” then it might be that risk of CPS involvement is unrelated or even negatively related to more objective, adverse health outcomes. This has implications for preventive services. If children identified by a PRM for maltreatment are at risk for negative health outcomes beyond simply child protection involvement, these families should be prioritized for a broader swath of higher intensity preventive services, such as home visiting programs. By contrast, if children flagged by a maltreatment model are at no greater risk of other adverse health outcomes, then the justification for offering a broader range of preventive services is reduced.12,13
The objective of the current study is to test whether children in an overall birth cohort who are classified as at very high risk (top 10%) or high risk (top 20%) of substantiated maltreatment by CPS, also have a heightened risk of adverse health outcomes. By using a PRM similar to that developed by Wilson et al,7 we risk-scored all infants born in New Zealand in 2011. We then administratively followed those children until age 3 to determine the prevalence of injury and mortality outcomes for these children relative to other children in the birth cohort.
Methods
Data
Our study relied on the Integrated Child Dataset, which is a census of all live births in New Zealand between mid-2004 through the end of 2011, and linked to health, welfare benefits, CPS, and criminal justice registers (among others).14 This extensive data set was developed in 2014 by the Ministry of Social Development (MSD). Birth and death registrations came from the Department of Internal Affairs. The Ministry of Health provided health data from the following: (1) the National Minimum Dataset (NMDS) for hospitalization records, (2) the Program for the Integration of Mental Health Data, (3) the pharmaceuticals collection for mental health records, (4) the maternity collection for details on the birth weight and maternal age at the time of child’s birth, and (5) the mortality collection. Note that because inpatient care for children is universally provided and fully subsidized by the country’s public health system, the hospitalizations register offers almost complete coverage of injury-related hospitalizations. MSD incorporated CPS data from the Child, Youth and Family (CYF) register, along with welfare benefits records. Finally, the Department of Corrections provided criminal justice sentencing data. Supplemental Tables 3 and 5 in the Supplemental Information give details on the precise use of these registers in our study.
The 2011 birth cohort was defined as all children identified within 91 days of birth through either: (1) an official registration of that birth on the national register of births, or (2) inclusion in a main public benefit recorded in national welfare data. This methodology was estimated to cover 94% of all live-born children and resulted in linked records for ∼60 000 children in each annual birth cohort.14 Linkages were developed using probabilistic matching methodologies that incorporated identifying information including names and dates of birth of both children and parents. More details on the data linkages (and on the registers) can be found in the MSD’s report.14
We used the 2 latest birth cohorts available from MSD to define the study population. From all live births in 2010 and 2011, we then excluded children who were involved in Family Start, a home visiting program first introduced in 1998 in New Zealand, as such a program is likely to have effects on mortality outcomes.15,16 Among children born in 2010 and 2011, 2865 and 2475 children took part in the home visiting program, respectively. After excluding these children, the 2010 and 2011 cohorts consisted of 61 476 and 60 006 children, respectively.
Starting with children born in 2010 (referred to as “the 2010 birth cohort”), we first estimated each child’s probability of being substantiated as a victim of maltreatment by age 2. This probability depended on the child’s characteristics and family background (referred to as “predictors”). This first step allowed us to generate weights associated with each set of values for the predictors (referred to as “risk scores”). Second, applying the weights generated in the first step, we estimated for each child born in 2011 (referred to as “the 2011 birth cohort”) his or her probability of substantiated maltreatment by age 2. We then ranked children according to their maltreatment probability. Lastly, we estimated for children with the highest probability of substantiated maltreatment their mortality and injury rates by age 3.
Maltreatment Risk Score
The first step in our analysis consisted of predicting the probability that a child would have a substantiated allegation of maltreatment by age 2. Our definition of maltreatment included physical and emotional abuse, and neglect. Children who had only sexual abuse as a substantiated finding were excluded from the MSD data because they were too few to report because of Statistics New Zealand confidentiality rules. We estimated a single PRM for the 2010 birth cohort by using logistic regression and including factors at birth, which were earlier shown to be predictive of maltreatment.7,9,17,18 We included preterm birth (before 37 weeks’ gestation), infant’s sex (girl), and an indicator for high parenting demand, ie, more than 3 children in the family (or multiple birth children, or multiple children aged <2). We also included several maternal characteristics measured at the time of the child’s birth, such as age (younger than 25 or older than 35), marital status (single), receipt of public income support, history of a mental health or substance abuse, and criminal records in the last 5 years. Additionally, we included maternal and sibling history of childhood maltreatment allegations to CPS. Supplemental Table 3 in the Supplemental Information gives details on the definition of the predictor variables. All predictor variables were entered into the PRM as categorical variables. Supplemental Table 4 in the Supplemental Information lists the coefficients from the PRM of maltreatment at birth, which achieved an area under the receiver operating characteristic curve of 88% (95% confidence interval [CI]: 0.87 to 0.89).
From this logistic model regression that predicted the probability of substantiated maltreatment by age 2, we then assigned a risk score to each child in the 2011 birth cohort. We stratified births to identify those defined as very high risk 10% (also referred to as top 10%) and high risk 20% (also referred to as top 20%). The top 10% comprised 6009 children, whereas the top 20% included 12 096 children.
Outcome Variables
We examined adverse outcomes for children born in 2011 by using information available from MSD. We focused on measures of cause-specific infant mortality and injury hospitalizations using the International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Australian Modification (ICD-10-AM). Infant mortality was defined as overall mortality, with stratifications by inflicted injury deaths, unintentional injury deaths, and sudden unexpected infant death (SUID). We explored both postneonatal mortality (ie, death of an infant aged 29–365 days), and overall infant mortality (ie, death of a live-born infant before 365 days of life).
We also examined injury hospitalizations. We coded long bone fracture injuries by age 2 and intracranial injuries by age 1. Both measures are considered markers of inflicted injuries when observed among toddlers.19 We additionally risk-scored children falling into other hospitalization groups, including those with any injury hospitalization by age 3, as well as children experiencing maltreatment-related injury hospitalization by age 2, and those with an ambulatory sensitive hospitalization by age 3. Supplemental Table 5 in the Supplemental Information provides details on the definition of the outcome variables we examined based on the code set from the ICD-10-AM.
For both the subgroup of very high-risk children (top 10%) and the subgroup of high-risk children (top 20%), we estimated a log-linear model with robust standard errors for each outcome variable. From the exponentiated coefficients, we computed relative risk ratios for each outcome, defined as ratios of the incidence rates among children in the top 10% (and top 20%) over the equivalent rates for all other children in the cohort. We used Stata MP, version 14 (StataCorp, College Station, TX) for the statistical analyses.
Results
In Table 1, we present the prevalence of each predictor variable for children born in 2010 in the top 10% of the distribution of risk scores and for the whole birth cohort. Relative to the birth cohort overall, children identified by the PRM as at very high risk of substantiated maltreatment (ie, top 10%) were more likely to have a single mother (88.2% vs 23.0%), a mother <20 years of age (26.7% vs 6.1%), and to live in a family with high parenting demand (30.7% vs 6.6%).
Distribution of Predictors of Maltreatment at Birth Among Children Born in 2010 in New Zealand
Compared with the cohort overall, past or current CPS involvement in the family was more prevalent among children classified by the model as very high risk. More than 1 in 2 children had a mother who had been reported to CPS during childhood (53.8% vs 10.1%). Approximately 1 in 5 children at very high risk of substantiated maltreatment had older siblings who were referred to CPS in the year before the child’s birth (20.1% vs 2.1%). The prevalence of mothers receiving welfare was much higher among children at elevated risk of substantiated maltreatment (76.3% vs 13.9%). These mothers also had a higher probability of having served a criminal sentence in the 5 years before the child’s birth (25.6% vs 3.8%), and they were more likely to have a mental health record (36.6% vs 8.5%).
Results for children at high risk of substantiated maltreatment (ie, in the top 20% of the distribution of risk scores) are presented in Supplemental Table 6 in the Supplemental Information, and show similar patterns to children in the top 10%.
In Table 2, we summarize the early childhood injury and mortality outcomes for children born in 2011 and identified as at very high risk of maltreatment. We first report the number and rates (per 1000) of adverse outcomes for these children, followed by their relative risk ratios. Finally, we calculate the prevalence of children with risk scores in the top 10% who experienced the adverse outcome among all children exposed to that outcome.
Mortality and Injury Outcomes Among Children Born in 2011 in New Zealand and at Very High Risk (Top 10%) of Maltreatment
All relative mortality risk ratios were higher among children identified by our model at elevated risk of CPS substantiation. Overall, children in the highest 10% were 4.8 times more likely to die in infancy (95% CI: 3.2 to 7.2) compared with other children. The largest relative risk was observed for postneonatal mortality because of unintentional injuries. Children classified by the model as at very high risk had 9.9 times (95% CI: 4.2 to 23.3) the risk of other children, and accounted for 57.1% of all accidental deaths in the cohort. The relative risk ratio was 9.0 (95% CI: 3.9 to 20.7) for injury deaths overall, and 8.5 (95% CI: 4.4 to 16.5) for SUID. Children in the top 10% of the distribution of risk scores accounted for ∼43% and 50% of the deaths in these categories, respectively.
All hospitalization outcomes were also found to be greater among the very high-risk children. Overall, children in the top 10% accounted for 18.1% of all hospitalizations in early childhood for any type of injury, and were 2 times (95% CI: 1.8 to 2.2) more likely to be hospitalized than other children. Not surprisingly, 1 in 2 children hospitalized for a maltreatment-related injury by age 2 was at very high risk of maltreatment. The corresponding relative risk ratio was 9.6 (95% CI: 5.8 to 15.8) compared with the birth cohort overall. Hospitalization for long bone fractures by age 2 were 2.6 times (95% CI: 1.7 to 4.0) more likely to occur among very high-risk children.
Supplemental Table 7 in the Supplemental Information reports the same results reported in Table 2, but for the larger group of children identified as at high risk of substantiated maltreatment (ie, in the top 20%). Their relative risk ratios of mortality and hospitalization were similar to those of children in the top 10%. Overall, children in the top 20% were 4.2 times (95% CI: 2.9 to 6.2) more likely to die in infancy than other children. They also were 1.8 times more likely to be hospitalized (95% CI: 1.7 to 2.0). Fig 1 compares both groups graphically.
Postneonatal (death of an infant aged 29 days to 1 year) mortality outcomes among children born in 2011 in New Zealand and at very high risk (top 10%) and high risk (top 20%) of maltreatment. We applied Statistics New Zealand confidentiality rules to counts, which included the random rounding of all counts to base 3. Data come from the Integrated Child Dataset, a one-off integrated record of all live births between mid-2004 and the end of 2011 in New Zealand, with health, welfare benefits, child protection system, and justice registers. The data set was put together by the MSD in 2014.
Discussion
Determining the true prevalence of childhood abuse and neglect is inherently difficult because what constitutes maltreatment varies over time and by cultural norms.20,21 Although administratively recorded substantiations by CPS are a “noisy” and inexact approximation of maltreatment subject to detection bias,22,23 our findings document that CPS records can be used as the target variable for predictive models designed for use in child maltreatment prevention efforts. A risk model predicting substantiated maltreatment could identify children at heightened risk of a range of significant, adverse health outcomes. In particular, we found strong associations with unintentional and inflicted injury fatalities, as well as a strong association with SUID.24
This is not to say that substantiated maltreatment is necessarily the best measure to model in an effort to target children at elevated risk of later adverse health outcomes. Although the authors of a number of recent studies have suggested surveillance bias may be a less powerful dynamic in CPS involvement than previously thought,12,13,25 the research remains mixed.23 Rather, with our results, we simply suggest that factors predictive of substantiated maltreatment are highly correlated with factors that predict injury deaths and hospitalization. These factors are less correlated with more general postneonatal mortality or ambulatory sensitive hospitalization. With our findings, we suggest that PRMs trained and built around CPS outcomes appear to target those children at risk for the most serious forms of maltreatment.
The implications of having an administrative outcome that can be modeled without additional data collection and that is sensitive to the identification of children at highest risk of serious injury and fatality are important. In New Zealand, the home visiting program Family Start is designed and funded to target the 5% highest in-need families.18 In our cross-match between children enrolled in the home visiting program and those identified by the PRM, we found that only 43% of children who received home visiting fell into the top 10% of risk scores. One possible reason for this is that the home visiting program relied on crude admission criteria (eg, maternal age and poverty status). Another reason could be that the mothers the model identified as at high risk may be less willing to engage with home visiting programs. This is an important area for future research.
Identifying vulnerable children through the use of linked administrative data has considerable advantages, including the ability to look at full population cohorts with the power to detect differences in rare outcomes, such as infant mortality.26,27 That said, there are a number of limitations that must still be considered. In our study, the data linkages were probabilistic and some errors in matching clients were inevitable. Available data captured only information collected or generated in the process of administering government services, and included few direct measures of children’s well-being and parenting behaviors. Furthermore, our analysis was limited to children born in New Zealand and did not account for immigrant children who entered the country. Finally, we examined 1 birth cohort, and looked into outcomes in early childhood. Our findings may not be fully generalizable to other birth cohorts or to outcomes at older ages.
Conclusions
The availability of linked administrative records means that it is increasingly possible to use predictive models to stratify large populations and help target services to those children in families where the risk is greatest. The ability to use data to determine the level of services needed would help focus our most intensive and costly intervention efforts toward those families at greatest risk. In the current analysis, we document that although only a small subset of children classified as at high risk of maltreatment experience injury hospitalizations and death, those children who do go on to experience these events are overwhelmingly concentrated among the highest risk deciles. Our findings suggest that by designing effective preventive services and supports for children and families at high risk of substantiated maltreatment, there could be notable reductions in population level rates of injury and mortality.
Footnotes
- Accepted November 15, 2017.
- Address correspondence to Bénédicte Rouland, PhD, Auckland University of Technology, School of Economics, Private Bag 92006, Auckland 1142, New Zealand. E-mail: benedicte.rouland{at}aut.ac.nz
The results in this article are not official statistics; they have been created for research purposes from the Integrated Data Infrastructure (IDI), managed by Statistics New Zealand (NZ). The opinions, findings, recommendations, and conclusions expressed in this article are those of the authors, not Statistics NZ. Access to the anonymized data used in this study was provided by Statistics NZ in accordance with security and confidentiality provisions of the Statistics Act 1975. Only people authorized by the Statistics Act 1975 are allowed to see data about a particular person, household, business, or organization, and the results in this article have been confidentialized to protect these groups from identification. Careful consideration has been given to the privacy, security, and confidentiality issues associated with using administrative and survey data in the IDI. Further detail can be found in the Privacy impact assessment for the IDI available from www.stats.govt.nz.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2017-3469.
References
- Copyright © 2018 by the American Academy of Pediatrics
In this issue
