Objective. To develop a mortality prediction score for retrieved neonates based on the information given at the first telephone contact with a retrieval service.
Methods. Data from the New South Wales Newborn and Pediatric Emergency Transport Service database were examined. Analysis was performed with the results for 2504 infants (median gestational age: 36 weeks; range: 24–43 weeks) who were <72 hours of age at the time of referral and whose outcome (neonatal death or survival) was known. The study population was divided randomly into 2 halves, the derivation and validation cohorts. Univariate analysis was performed to identify variables in the derivation cohort related to neonatal death. The variables were entered into a multivariate logistic regression analysis with neonatal death as the outcome. Receiver operator characteristic (ROC) curves were constructed with the regression model and data from the derivation cohort and then the validation cohort. The results were used to generate an integer-based score, the Mortality Index for Neonatal Transportation (MINT) score. ROC curves were constructed to assess the ability of the MINT score to predict perinatal and neonatal death.
Results. A 7-variable (Apgar score at 1 minute, birth weight, presence of a congenital anomaly, and infant’s age, pH, arterial partial pressure of oxygen, and heart rate at the time of the call) model was constructed that generated areas under ROC curves of 0.82 and 0.83 for the derivation and validation cohorts, respectively. The 7 variables were then used to generate the MINT score, which gave areas under ROC curves of 0.80 for both neonatal and perinatal death.
Conclusion. Data collected at the first telephone contact by the referring hospital with a regionalized transport service can identify neonates at the greatest risk of dying.
Provision of the most effective neonatal transport service requires accurate assessment of disease severity and prediction of prognosis, to facilitate appropriate triage and resource allocation. The transport process starts at the time the retrieval service receives the first call from the referring hospital; therefore, it is desirable to predict outcomes accurately at that point of contact. Unfortunately, scores developed for assessment of infants during the transport process have used data acquired after the retrieval team has arrived at the referring hospital.1,2
Prediction-of-mortality scores exist for the retrieved pediatric population and include the Pediatric Index of Mortality score3 and the pre-intensive care unit Pediatric Risk of Mortality,4 but such scores are also calculated from data obtained at the transport team’s first physical contact with the patient. In addition, neither of these scores is easily applicable to a neonatal population, because they are heavily dependent on assessment of the level of consciousness and pupillary signs. Other neonatal prediction scores, including the Berlin score,5 the Score for Neonatal Acute Physiology (SNAP),6 and the Clinical Risk Index for Babies (CRIB),7 were not developed to assess the outcomes of retrieved neonates and have factors that limit their use in such populations. The Berlin score requires classification of the degree of respiratory distress, and the SNAP has 16 variables and thus is time-consuming to calculate. Both the CRIB and SNAP use data collected over 12 hours and thus may reflect the effects of interventions rather than the underlying risk at an early time point. The National Institute of Child Health and Human Development network mortality prediction score8 and the CRIB II score9 are both based on information available shortly after birth but were developed for use among very low birth weight (VLBW) infants, whereas retrieved infants have a wide range of birth weights and gestational ages. Therefore, the aim of this study was to develop a new mortality prediction score for the retrieved neonatal population that was based on data collected at the time of the first call by the referring hospital, when resource allocation is decided. In addition, we wished to determine whether the score predicted death in the VLBW population more accurately than did gestational age or birth weight.
Data were obtained from the New South Wales Newborn and Pediatric Emergency Transport Service (NETS) database. NETS provides an integrated transport service for New South Wales and retrieves ∼1650 patients each year, 50% of whom are neonates. The retrieval process is started when a clinician from the referring hospital contacts NETS. During that call, the infant’s name, date and time of birth, medical history, and clinical data (gestational age, birth weight, gender, and Apgar scores at 1 and 5 minutes) are recorded (time of first call data). If deemed necessary, a retrieval team (specialist retrieval nurse and doctor) is then mobilized. After arrival at the referring hospital, the retrieval team collects additional data (time of first contact data). The infant is stabilized by the retrieval team and, when the team is ready to depart, another set of data is collected (time of stabilization data). The infant is then transported to the accepting intensive care unit, where a final set of data is collected (time of admission data). At each of the 4 time points, collection of the following data was attempted: heart rate, respiratory rate, fraction of inspired oxygen (Fio2), arterial partial pressure of oxygen (Pao2), arterial partial pressure of carbon dioxide (Paco2), pH, base excess, bicarbonate level, oxygen saturation, and ventilator settings. Outcome data (neonatal death or survival) were obtained from the Neonatal Intensive Care Unit Study (NICUS) database, which contains data on the outcomes of neonates admitted in New South Wales, Australia. Ethical approval for this study was obtained from Western Sydney Area Health Service, and permission to use the data from the NICUS database was obtained from NICUS and each of the local hospital consultants responsible for the NICUS database.
Data for infants who were <72 hours of age at the time of the first call, who had complete demographic data and blood gas data for 2 of the 4 time points, and whose outcomes were known were included in the analysis. The study population was divided randomly into 2 halves, ie, the derivation cohort and the validation cohort. The strategy described by Pollack et al10 was used to build the predictive model.11,12 The derivation cohort was used for model derivation, and the model’s accuracy was tested with the validation cohort.
Univariate analysis was used to determine whether age (in hours), gestational age, gender, birth weight, Apgar scores at 1 and 5 minutes, temperature, heart rate, respiratory rate, Fio2, intubation status, Pao2, Paco2, pH, base excess, bicarbonate level, oxygen saturation, presence of a congenital abnormality, oxygenation index, time between the different time points in the retrieval process, or total retrieval time differed (P < .2, with χ2 or Mann-Whitney analysis as appropriate) between the infants who died and those who survived. The variables that did differ were entered into a multivariate logistic regression analysis with forward stepwise entry, with death as the outcome. All of the variables that differed between the 2 groups at P < .2 were initially entered, and the least significant variables, identified by their logistic coefficient, odds ratio, and confidence intervals (CIs), were removed 1 at a time. At each stage, the resulting model was assessed with receiver operator characteristic (ROC) curves. Goodness-of-fit testing (with the Hosmer-Lemeshow goodness-of-fit test13) was used for both the derivation and validation cohorts. A P value of >.05 implied no significant difference between the observed and expected values, and the goodness of fit was considered acceptable.
With the logistic coefficients, odds ratios, and CIs, integer score points were assigned to each of the variables and a score was generated, the Mortality Index for Neonatal Transportation (MINT) score. Cutoff points for the individual variables were obtained by assessing the model cutoff points used in clinical practice and reported in the literature. ROC curves were constructed to determine the accuracy with which the MINT score predicted perinatal or neonatal death for the whole population and then for VLBW infants only. ROC curves were also constructed to assess whether gestational age or birth weight predicted death in the VLBW infant population.
During the study period (February 1992 to July 2001), 8806 infants were retrieved, of whom 6348 (72.1%) were <72 hours of age at the time of the first call (Table 1). Complete demographic and physiologic data were available for 3429 infants; of those 3429 infants, 2504 had outcome data recorded in the NICUS database and 302 died (12% mortality rate). The 2504 infants (1585 male patients) formed the study population. The median gestational age was 36 weeks (range: 24-43 weeks) and the birth weight was 2782 g (range: 520-6140 g). The median age at the time of the first contact with the retrieval service was 4.5 hours (range: 0-67.9 hours); an age of 0 was recorded when the referring hospital called the retrieval service before the infant was born.
The study population was significantly younger than the NETS population, because only infants <72 hours of age were included in the analysis (Table 1). The study patients also had significantly greater oxygen requirements and were more likely to be intubated at the time of the call.
The only significant difference between the derivation and validation cohorts was the proportion of infants who were intubated (Table 2). Univariate analyses indicated that postnatal age, gestational age, gender, birth weight, Apgar scores at 1 and 5 minutes, respiratory rate, intubation status, Pao2, pH, base excess, and presence of a congenital abnormality differed (P < .2) between infants who died and those who survived (data not shown).
From the multivariate logistic regression analysis, a 7-variable logistic regression model was generated (Table 3). The 7-variable model was shown to have the same area under the ROC curve as a model that contained all of the variables. Removal of additional variables resulted in a smaller area under the ROC curve; therefore, the 7-variable model was used. The equation used to generate the probability of death from the model is as follows: The probability of death (y) is given by the equation y = exp(logit)/[1 + exp(logit)]. Age is expressed in hours; for the presence of a congenital abnormality and intubation status, yes = 1.
The model generated areas under the ROC curve of 0.82 for the generation cohort and 0.83 for the validation cohort. The goodness of fit (Hosmer-Lemeshow test) for the derivation cohort was 0.364, and that for the validation cohort was 0.303.
With the information from the logistic regression analysis, the MINT score was derived (Table 4). The median MINT score for the study population was 3 (range: 0-33). Eighty per cent of infants with MINT scores of >20 died (Fig 1). The MINT score had an area under the ROC curve of 0.80 (95% CI: 0.76-0.83) for death in the perinatal period (first week after birth) and an area under the ROC curve of 0.80 (95% CI: 0.76-0.83) for death in the neonatal period (first month after birth) (Fig 2). For VLBW infants, the MINT score had areas under the ROC curves for perinatal and neonatal death of 0.69 (95% CI: 0.60-0.77) and 0.68 (95% CI: 0.60-0.76), respectively. Gestational age and birth weight had areas under the ROC curves of 0.64 (95% CI: 0.56-0.73) and 0.67 (95% CI: 0.59-0.76), respectively.
We generated and validated a prediction of mortality score (MINT score) for retrieved neonates that was based on data obtained at the first referral call. The MINT score exhibited performance similar to that of the Transport Risk Index of Physiologic Stability (TRIPS),1 which was generated with a similar population (the median gestational age of the study population used for generation of the TRIPS was 36 weeks and the birth weight was 2610 g). The areas under the ROC curves for perinatal and neonatal death for the TRIPS were 0.83 and 0.76 and those for the MINT score were 0.80 and 0.80, respectively. The TRIPS, however, is derived from data collected by a member of the transport team immediately after arrival at the referring hospital and immediately after arrival at the destination hospital.1 In contrast, the MINT score uses data collected when the referring hospital first contacts the transport team via telephone. This is a major advantage, because decisions are made at first contact regarding resource allocation. The MINT score has an additional advantage, ie, it is based on 7 objective data items; although the TRIPS comprises only 4 items, 1 is the response to noxious stimuli, which is subjective.
The study population differed from the total NETS population, because we included only infants for whom the transport process started at <72 hours of age (when most neonatal retrievals occur). We also included only infants with complete demographic data and blood gas data for at least 2 of the 4 time points. The study population differed significantly from the NETS population with respect to the proportion of patients intubated and their greater oxygen requirements. We were thus examining the sickest infants transported, as highlighted by the mortality rate of 12%, compared with the rate of 10% for the whole NETS population. A mortality prediction score would be most useful for the sickest infants. The MINT score, however, was developed with only 39% of the eligible cohort, and we recommend that it be prospectively validated before it is put into widespread use.
The MINT score comprises 7 variables, including the Apgar score at 1 minute. The 5-minute Apgar score, rather than the 1-minute score, has been considered to be more predictive of neonatal death.14 However, the National Institute of Child Health and Human Development mortality prediction model8 also used the 1-minute Apgar score. In this study, both 1- and 5-minute Apgar scores were available for all of the infants included and both were analyzed, but the 1-minute Apgar score performed better. The Pao2/Fio2 ratio is included in the SNAP as a measure of oxygenation.6 It was selected, rather than the oxygenation index, arterial-alveolar oxygen difference, or arterial-alveolar oxygen ratio, for the SNAP because it was statistically equivalent and avoided the need to determine mean airway pressure or concurrent carbon dioxide tension. In our study, we found that arterial oxygen tension, but not the inspired oxygen concentration or oxygenation index, was significantly related to death. A possible explanation for the difference was that more than one-fourth of the infants were receiving 100% oxygen at the time of the referral call. We excluded base excess from our score because of colinearity and because, although both pH and base excess were highly predictive, pH performed better. We included congenital abnormality in our analysis because congenital abnormalities are known to have mortality effects beyond those indicated by physiologic derangements.6 Indeed, the analysis demonstrated that the presence of a congenital abnormality was significantly associated with death, and that parameter was included in our model. Unfortunately, only the presence or absence of a congenital abnormality was recorded at the time of the referral call. Therefore, we cannot comment on whether weighting for the severity of the abnormality might have generated a more accurate score. A better standardized classification system must be available,15 however, before this can be appropriately investigated.
We have generated and validated an easy-to-use mortality prediction score for retrieved neonates. Such scores should not be used to ration health care; instead, a high score should be used to indicate the level or priority and the need for the most experienced transport team. All of the data used in the MINT score can be collected at the time of the first telephone contact by the referring hospital with the transport team. This score might be particularly useful because it could facilitate more effective triage.
- ↵Kanter RK, Edge WE, Caldwell CR, Nocera MA, Orr RA. Pediatric mortality probability estimated from pre-ICU severity of illness. Pediatrics.1997;99 :59– 63
- ↵Maier RF, Rey M, Metze BC, Obladen M. Comparison of mortality risk: a score for very low birthweight infants. Arch Dis Child Fetal Neonatal Ed.1997;76 :F146– F150
- ↵Richardson DK, Gray JE, McCormick MC, Workman K, Goldmann DA. Score for Neonatal Acute Physiology: a physiologic severity index for neonatal intensive care. Pediatrics.1993;91 :617– 623
- Copyright © 2004 by the American Academy of Pediatrics