BACKGROUND AND OBJECTIVES: African American children are more than twice as likely to die after surgery compared with white children. In this study, we evaluated whether risk factors for death after surgery differ for African American and white children, and we also assessed whether race-specific risk stratification models perform better than non–race-specific models.
METHODS: The National Surgical Quality Improvement Program Pediatric Participant Use Data File contains clinical data on operations performed on children at participating institutions in the United States. Variables predictive of death within 30 days of surgery were analyzed for differences in prevalence and strength of association with death for both African American and white children. Classification tree and network analysis were used.
RESULTS: Network analyses revealed that the prevalence of preoperative risk factors associated with death after surgery was significantly higher for African American than for white children. In addition, many of the risk factors associated with death after surgery carried a higher risk when they occurred in African American children. Race-specific risk models provided high accuracy, with a specificity of 94% and a sensitivity of 83% for African American children and a specificity of 96% and a sensitivity of 77% for white children, and yet these 2 models were significantly different from each other.
CONCLUSIONS: Race-specific models predict outcomes after surgery more accurately compared with non–race-specific models. Identification of race-specific modifiable risk factors may help reduce racial disparities in surgery outcome.
- CI —
- confidence interval
- D30 —
- death within 30 days of surgery
- NSQIP —
- National Surgical Quality Improvement Program
- Pedi-PUF —
- Pediatric Participant Use Data File
What’s Known on This Subject:
Racial health disparities exist in surgical outcome. African American children are more than twice as likely to die after surgery compared with white children.
What This Study Adds:
The prevalence of preoperative risk factors and their association with postsurgical mortality were higher for African American than white children. Classification tree analyses on African American and white children separately were more accurate than non–race-specific risk models.
Racial and ethnic health disparities are pervasive in the United States.1,2 Life expectancy for African Americans in the United States lags behind that for white individuals, and African Americans experience greater mortality from cardiovascular disease, diabetes, cancer, and trauma.3–5 Racial and ethnic health outcomes disparities also exist among children. African American children are more likely to die after congenital heart surgery6 or neurologic procedures7,8 and are more likely to have complications related to appendicitis.9 African American children experience delays in getting placed on kidney transplant lists, wait longer periods on transplant lists,10,11 and are less likely to receive kidney transplants compared with white children.12,13
We previously developed a classification tree model for death within 30 days of surgery (D30) in children by using 6 preoperative conditions to stratify patients into subgroups with risks of D30 from <0.1% to >30%.14 This model performed as well as more complex models using hierarchical logistic regression, but the model systematically resulted in an underestimation of the risk of African American children while resulting in an overestimation of the risk for white children.14 This led us to consider whether race-specific models would be more accurate for predicting the risk for death after surgery for African American and white children, respectively. The purpose of this study was to develop race-specific models predictive of risk for D30 and to examine the performance characteristics of these models. This is a fundamentally different goal than that for models used to compare outcomes after surgery among African American compared with white children, adjusted for various comorbidities and other factors that might differ between the groups. With our study, we did not seek to assign or understand causation; rather, the goal of our research was to see if race-specific models could be used to identify patients at high risk for death after surgery more successfully than models in which all races were grouped together.
We used the National Surgical Quality Improvement Program (NSQIP) Pediatric Participant Use Data File (Pedi-PUF), a data file containing cases submitted to the American College of Surgeons NSQIP Pediatric to investigate and advance quality of care.15 We used Pedi-PUF data covering 2012–2014, which included data from 50 hospitals in 2012, 56 hospitals in 2013, and 64 hospitals in 2014. This data set included outcomes on a total of 183 233 surgical operations in children <18 years of age at the time of operation, including a total of 130 437 operations on white children and 23 263 operations on African American children. For model validation, we used the 2015 Pedi-PUF data set, which contained 84 056 surgical cases from 80 hospitals.
The Pedi-PUF data set contains over 300 perioperative variables including operation type (defined by the primary current procedural terminology code assigned by the operating surgeon), demographic variables, preoperative state (including preexisting comorbid conditions), medications, laboratory test results, intraoperative data including the length of the operation, blood loss, and specific intraoperative and postoperative occurrences. We limited our analysis to 15 preoperative variables (listed in Supplemental Table 5) that had in past studies been associated with death after surgery.14,16 The authors of one of these studies selected 7 variables as predictors of D30 of surgery in a logistic regression model. These variables were neonatal status, respiratory support, inotropic support, blood disorder, cerebrovascular injury, previous cardiac intervention, and work relative value unit for procedure.16 In our study, we did not include work relative value unit for procedure as a predictor because it is based on a subjective assessment of patients. Another study predicted D30 via a classification tree model using sepsis status, case type, do-not-resuscitate order status, inotropic support, ventilator dependency, oxygen support.14 Finally, we included transfusion and malignancy status as predictors because they were selected in the cross-validation steps in the same study. We defined our outcome of interest as D30. Further details about the risk factors can be found in the User Guide for the 2012 ACS NSQIP Pediatric Participant Use Data File (PUF).17
We used network analysis to perform descriptive analytics on the univariate and bivariate level. Network analysis is based on graph theory, the study of graphs formed by nodes (or vertices) and edges connecting the nodes.18,19 A network can be established among nodes (for example, individual users of Facebook), and edges can be defined between 2 nodes that are related (ie, friends on Facebook). Although nodes and the edges are enough to define a network, other additional information also can be represented in a network structure. For example, the size and color of nodes, and the thickness and color of the edges, are other properties that can reflect additional information and can help guide further, more formal quantitative research into how these characteristics interact. Although network analyses can be used for a range of questions, in this study, we used the network framework specifically to visualize the prevalence of risk factors and their association with D30 for children captured in NSQIP Pediatric.
Nodes were risk factors for the outcome, D30, whereas edges represented the type and the strength of relationship between these risk factors. Edges connecting 2 risk factors were the comorbidity of 2 connected risk factors; 2 risk factors were connected if they coexisted for at least 1 case. Four different node sizes represented the prevalence of risk factors (size 1: prevalence <1%, size 2: 1% < prevalence <5%, size 3: 5% < prevalence <10%, and size 4: prevalence >10%) and node color codes represented the risk of death (green: risk <1%, yellow: 1% < risk <5%, orange: 5% < risk <%10, and red: risk >10%). A similar color code and sizing was applied to edges; 4 different sizes of edge thickness represented the co-occurrence prevalence of 2 risk factors (size 1: prevalence <0.1%, size 2: 0.1% < prevalence <0.5%, size 3: 0.5% < prevalence <1%, and size 4: prevalence >1%) whereas the color codes represented the risk of death at co-occurrence of 2 risk factors (green: risk <1%, yellow: 1% < risk <5%. orange: 5% < risk <10%, red: 5% < risk <25%, and purple: risk >25%).
Classification Tree Analysis
Our multivariate approach was centered on classification tree analysis,20,21 which was used to predict the risk of D30. Tree-based predictive modeling tools such as classification trees have some important advantages over other predictive modeling approaches (such as logistic regression and support vector machines) in that they assist in identifying interactions between explanatory variables for discrete subgroups.20,22 Another advantage of classification trees is that the final model is intuitive and typically easy to understand, which is critical for the use of decision support tools in clinical settings. For these reasons, we used classification tree analyses to construct race-specific predictive models for D30.
We used the χ2 Automatic Interaction Detector algorithm to split parent nodes into children nodes by using the predictor yielding the minimum P value by χ2 test23 that is lower than the splitting criteria (0.05 in our case). In the χ2 Automatic Interaction Detector algorithm, Bonferroni-adjusted P values are used because the selection of predictors is a multiple testing task.24 The algorithm is terminated when there is no Bonferroni-adjusted P value lower than the determined significance level. To control overfitting, we implemented fivefold cross-validation on the Pedi-PUF 2012–2014 data set. The final tree represented is the one built on the full cohort of Pedi-PUF 2012–2014. We further conducted external validation of our classification models by using the Pedi-PUF 2015 data set.
The Pedi-PUF 2012–2014 data set included 183 123 surgeries with 621 deaths in the 30 days after surgery, corresponding to a risk of death of 0.0034, (95% confidence interval [CI]: 0.0031–0.0037). The risk of D30 among white children was 0.0027 (95% CI: 0.0024–0.0030); for African American children, the risk was 0.0060 (95% CI: 0.0051–0.0071). The odds ratio for death after surgery among African American compared with whites children was 2.22 (95% CI: 1.83–2.70).
On the basis of the unadjusted statistics summarized in Table 1, the prevalence of risk factors differed for African American and white children. The prevalence of ventilation, oxygen support, previous cardiac intervention, cerebrovascular injury, wound infection, hematologic disorder, inotropic support, transfusion, and neonatal status were significantly higher among African American children than among White children, whereas sepsis, malignancy, and emergent case were more common among white children.
Risk factors such as ventilation, oxygen support, wound infection, and neonatal status were significantly more common among African American children, and their association with death was significantly stronger. Additionally, even for risk factors that were more common among white children (sepsis, malignancy, and emergent case), or for risk factors that were similar in prevalence between African American and white children (bleeding disorder), the risk of death after surgery was significantly higher for African American children (Table 1).
The risk factor networks for African American and white children are shown in Fig 1. Node sizes for oxygen support and hematologic disorder were larger in the Fig 1A network, indicating their higher prevalence among African American children. In addition, the color of the ventilator node (red for African American children and orange for white children) shows that there was a stronger association with death among African American children on ventilator support in the 48 hours before surgery compared with white children. Similar observations were found for sepsis, neonate, and emergent cases.
The co-occurrence of 2 risk factors increases risk for D30 in all patients. This effect was frequently greater for African American children than for white children. For example, the color of the edge connecting inotropic support and hematologic disorder was purple for African American children, whereas the same connection was red for white children. In addition, the thickness of the edge connecting inotropic support and hematologic disorder is larger in Fig 1A network, implying that the co-existence of these 2 risk factors has higher association with death and is more prevalent in African American children. Similar results were seen for the joint effects of inotropic support-ventilator dependency, bleeding disorder-transfusion, hematologic disorder-sepsis, neonate-wound infection, and oxygen support-ventilator dependency. Moreover, in general, the prevalence of co-occurrences of risk factors were higher for African American children, as indicated by the thicker edges in the Fig 1A network, compared with those in the Fig 1B network.
To correct for the systematic underestimation of risk of death after surgery in African American children, we developed 2 separate risk stratification models for African American and white children, shown in Figs 2 and 3, respectively.
The classification trees developed for African American and white children differed substantially from each other, although some risk factors (ventilation, oxygen support, inotropic support, and emergent case) are present in both models. The risk of death associated with each of the terminal nodes are given in Tables 2 and 3 for African American and white children, respectively.
Model Comparison and External Validation
Finally, we used the 2015 Pedi-PUF data set for validation of tree-based classification models and also for comparison of race-specific models against non–race-specific models. When we compared the specificity and sensitivity of the new classification trees to the original tree, the race-specific models had better sensitivity with equal specificity (Table 4).
We developed an improved risk classification model that more accurately reflects risk for death after surgery among African American children and can be used to identify more African American children who are at risk for death after surgery. The separate classification trees built for African American and white children in Figs 2 and 3 reveal better estimates of risk for each terminal-node risk group. Although 4% improvement in sensitivity may seem a modest improvement from a statistical point of view, even a modest reduction in false-negatives for predicting mortality may have clinical importance. The finding that ventilation indicates that African American children are in the highest risk group is novel and demands further investigation.
We used network analysis to help visualize relationships among variables and how they impact racial disparities in preoperative risk factors for death after surgery. Although the number of white children was much higher than the number of African American children, we have obtained a more dense (larger nodes and thicker edges) network for African American children. This approach led to our second important finding, which was that the prevalence of most important factors identifying children at risk for dying after surgery was, in general, higher among African American children than among white children. For example, ventilator dependency was present among 4.3% of African American children compared with 2.5% of white children, whereas hematologic disorder was present among 6.2% among African American children and in only 2.6% of white children. Many of these risk factors also had a stronger association with death after surgery among African American children than among white children, and this was the case even for risk factors similar in prevalence between African American and white children. It may be that race-specific models differ by surgery type or surgical severity, and future researchers could address these topics.
There are both strengths and limitations to our study. NSQIP Pediatric is a high-quality, clinically abstracted database with data on a large number of concurrent procedures done in a variety of hospitals throughout the country. Findings based on this study should therefore be generalizable to similar operations performed on children in the United States. The Pedi-PUF is deidentified. Therefore, we cannot compare institutional completeness of data collection, nor evaluate differences in outcomes between institutions. It is possible that the NSQIP Pediatric member hospitals may not be representative of non–NSQIP Pediatric hospitals. Finally, the NSQIP Pediatric data set does not include information on social risk factors, which may also influence outcomes and differ among racial groups.
African American children have more preoperative risk factors, and these surgical risk factors are in general more strongly associated with death than those found in white children. A basic tenet of health equity is that African American families should receive accurate information on the surgical risks their children face, not risks based on analysis drawn from predominantly white children. Interventions to mitigate these risks will need to be tested within the context of race-specific risk strata to reduce the higher surgical mortality rate currently found in African American children.
- Accepted November 15, 2017.
- Address correspondence to Oguz Akbilgic, PhD, Le Bonheur Children's Hospital Research Center, 50 N Dunlap Ave, Suite 495R, Memphis, TN 38103. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Barr D
- Buno M,
- Mukherjee D,
- Elramsisy A,
- Nosova K,
- Lad S,
- Boakye M
- Kasiske BL,
- London W,
- Ellison MD
- Furth SL,
- Garg PP,
- Neu AM,
- Hwang W,
- Fivush BA,
- Powe NR
- Akbilgic O,
- Langham MRJ,
- Walter AI,
- Jones T,
- Huang EY,
- Davis RL
- American College of Surgeons
- Diestel R
- Wasserman S,
- Faust K
- Breiman L,
- Friedman JH,
- Olshen RA,
- Stone CJ
- Steinberg D,
- Colla P
- Hawkins D,
- Kaas G
- Copyright © 2018 by the American Academy of Pediatrics