PEDIATRICS Vol. 117 No. 4 April 2006, pp. e733-e742 (doi:10.1542/peds.2005-1853)
Assessment and Optimization of Mortality Prediction Tools for Admissions to Pediatric Intensive Care in the United Kingdom
a Intensive Care National Audit & Research Centre, London, United Kingdom
b Health Services Research, School of Health and Related Research, University of Sheffield, Sheffield, United Kingdom
c Birmingham Children’s NHS Trust, Birmingham, United Kingdom
d Royal Liverpool Children’s NHS Trust, Liverpool, United Kingdom
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
OBJECTIVE. To assess the Pediatric Risk of Mortality (PRISM, PRISM III-12, and PRISM III-24) systems and the Pediatric Index of Mortality (PIM and PIM2) systems for use in comparing the risk-adjusted mortality of children after admission for pediatric intensive care in the United Kingdom.
METHODS. All PICUs in the United Kingdom were invited to participate. Predicted probability of PICU mortality was calculated using the published algorithms for PIM, PIM2, and PRISM and compared with observed mortality. These scores, along with PRISM III-12 and PRISM III-24, whose algorithms are not published, were optimized for the United Kingdom.
RESULTS. Of 26 PICUs in the United Kingdom, 22 (85%) were recruited, and sufficient prospective data were collected from 18 (69%) units on 10197 (98%) of 10385 admissions between March 2001 and February 2002. All published tools were found to have poor calibration but provided good discriminatory power. After estimation of UK-specific coefficients, only PIM2, PRISM III-12, and PRISM III-24 had satisfactory calibration. All models provided good discriminatory power. Funnel plots for all of the recalibrated models indicated that the risk-adjusted mortality for all units was consistent with random variation.
CONCLUSIONS. PIM2, PRISM III-12, and PRISM III-24 all were found to be suitable for use in a UK PICU setting. All tools provided similar conclusions in assessing the distribution of risk-adjusted mortality in UK PICUs. It now is important that these tools be used to monitor outcome and improve the quality of pediatric intensive care within the United Kingdom.
Key Words: clinical score • intensive care • mortality prediction • outcome assessment • quality of care
Abbreviations: PRISM—Pediatric Risk of Mortality • PIM—Pediatric Index of Mortality • UK PICOS—United Kingdom Pediatric Intensive Care Outcome Study
The provision of intensive care to infants, children, and adults increased steadily during the latter half of the 20th century, with particular, rapid expansion during the 1970s and 1980s.1 In pediatric intensive care, the growth in activity has followed somewhat later and has been accompanied by little objective evaluation of quality of care provided or the effectiveness of specific treatments. Furthermore, evidence to support planned national infrastructure is limited when compared with the significant resources expended on pediatric intensive care.2,3
Within the United Kingdom, the equitable provision of pediatric intensive care, service structure, and rigorous evaluation of new and existing treatments all require methods to account for variation in case mix. Rigorously conducted randomized, controlled trials of sufficient statistical power are the "gold standard" research design to compare different aspects of medical practice. However, there are a number of situations in which random allocation is not possible for either ethical or logistic reasons.4 When randomization is impossible, the optimal research design consists of prospective studies that adjust outcomes for variation in risk factors between hospitals according to prespecified comparators.5,6 The term "risk adjustment" has been coined to describe the process of adjusting for such risk factors when comparing outcomes after intensive care.
Risk-adjustment tools that predict death in PICUs have become established in the past 20 years. The Physiologic Stability Index, published in 1984,7 has been updated twice, using data from North American PICUs, and renamed, first as the Pediatric Risk of Mortality (PRISM) and more recently as PRISM III.8,9 PRISM III currently provides the risk-adjustment tool for the United States based Pediatric Intensive Care Unit Evaluations (PICUEs) system, which provides comparative reports to participating units under a licensing arrangement.10 Various versions of the PRISM family of risk-adjustment tools have been used extensively in the United States and to some extent in the Netherlands to inform policy and organizational decisions.5,11
The alternative to the PRISM family of tools is the Pediatric Index of Mortality (PIM), which was updated recently to PIM2.12,13 PIM was published in 1997, based on admissions to PICUs in Australia and 1 in the United Kingdom. PIM2 was published in 2003 and included 15191 admissions from 10 PICUs in Australia and New Zealand and 4 PICUs in the United Kingdom. Although PIM was tested in 4 UK PICUs, none of the tools has been externally validated for general use in UK PICUs.14
Prediction tools must discriminate well between deaths and survivors and be well calibrated before they can be applied usefully to assess or standardize comparisons of PICUs or to correct for case-mix differences between groups in observational studies. Single-center studies in the United Kingdom and studies in other countries have generated conflicting evidence for the accuracy of predictions from both the PRISM and the PIM families of tools.11,15–18 Such single-center studies, often occurring several years after the publication of the tools, are likely to be unreliable because it is to be expected that risk-adjusted mortality will improve over time and that the scoring system hence will become decalibrated.19 For risk-adjusted mortality rates to be meaningful, epidemiologic studies must either use a tool that is derived from their own data or use the most recent recalibration of the chosen tool. Consequently, a joint working group of the Department of Health and the Medical Research Council concluded in 1997 that no risk-adjustment method had been validated sufficiently for general use in PICUs in the United Kingdom and stated that "work on patient risk adjustment is a high priority, as without better methods of risk adjustment, research on the effectiveness of IC/HDC (Intensive Care/High Dependency Care) will be severely limited."20 The United Kingdom Pediatric Intensive Care Outcome Study (UK PICOS), reported here, aimed to assess and optimize objectively the competing prediction tools in a large, prospective, and representative sample of admissions to UK PICUs.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
All PICUs in the United Kingdom were invited to participate in UK PICOS. Staff at participating units were asked to collect prospectively standardized information using dedicated software on consecutive admissions to their unit between March 5, 2001, and March 4, 2002. Data collection staff were trained to enhance standardization of data collection between units. Data were also validated both locally, using checks built into the data collection software, and at the central processing center, where extensive data validation and query resolution were conducted. At the end of the data collection period, all units were asked to provide an independent count, using departmental databases or admission books, of the number of admissions and deaths during the study period. Units that collected data on <80% of admissions were excluded in whole or were excluded in part when it was possible to identify clearly periods of satisfactory (
80%) coverage. The information that was collected on each admission included demographic details, the presence in the medical history of any of 15 significant events or premorbid conditions, the location before admission, the reason for admission to the PICU, measurements of the required physiologic parameters up to 24 hours after admission, outcome at discharge from the ICU, and eventual hospital discharge (status and length of stay).21 Patients who were already dead or had clinical signs of brain death on admission to the PICU were excluded from the study.
The mortality prediction tools under consideration were PIM, PIM2, PRISM, and PRISM III,8,9,12 all of which aim to predict mortality in the admitting PICU. PIM and PIM2 use information on 7 and 10 factors, respectively, to calculate the probability of death during the intensive care admission. These are gathered between first contact with a PICU doctor and up to 1 hour after admission to the PICU. PRISM uses measurements on up to 14 physiologic values during the first 24 hours after admission to the PICU. The PRISM III system can be broken down further into PRISM III-12, using measurements from up to 17 physiologic values during the first 12 hours after admission to a PICU, and PRISM III-24, using measurements from up to 17 physiologic values during the first 24 hours. In both PRISM and PRISM III, the physiologic values are used to form scores, which when combined with 2 (PRISM) or 8 (PRISM III) other factors, are used to generate predicted probability of mortality in the PICU.
Data Completeness
All tools assume that score components that are not recorded are normal. Data were collected to allow us to predefine "not applicable" to indicate when a tool component was appropriately not recorded, for example, no blood gases were taken and "missing" to indicate all other not recorded tool components. The rate of not applicable and missing tool components then was examined for each tool by PICU.
Statistical Analysis
Published Models
Models were applied to the data using coefficients and instructions that were published by the original authors of the tools and by personal communication with these authors for any necessary clarification. Access to the coefficients that were necessary to calculate a probability for PRISM III was refused by the main author; therefore, it could not be included in the initial stages of this evaluation (M.M. Pollack, Children’s National Medical Center, Washington, DC, personal communication, 2003). These published models were assessed by investigating their calibration and discrimination for PICU mortality.
Discrimination
Predictions from each model were assessed using the c index (area under the receiver operating characteristic curve) for discrimination. The c index is the probability of concordance between outcomes and predictions. In this study, it represents the probability that a randomly chosen patient who died will have a higher predicted probability of mortality than a randomly chosen patient who survived. Published c-index criteria suggest that 0.7 is acceptable, 0.8 is good, and 0.9 is excellent.22
Calibration
How well the predicted probabilities of mortality that were generated by the risk-adjustment models compared with the observed mortality was assessed using the Hosmer-Lemeshow 
2 test and Cox’s calibration regression.23–25 For each risk-adjustment model, for the Hosmer-Lemeshow goodness-of-fit test, patients were categorized into 10 groups according to quintiles of their associated predicted probability of mortality, and the observed and expected outcomes were compared using a 2 statistic. For the Cox regression model, the probability of mortality that was generated by each risk-adjustment model was transformed into a log odds of mortality. This then was fitted as an explanatory variable in a logistic regression model to predict mortality. Perfect calibration would be indicated by a model with a constant term of 0 and a slope term of 1. Significant differences from these values give a quantifiable indication of where the calibration of the models has failed.
Optimization of Current Models
The tools were optimized for the United Kingdom by reestimation of coefficients using logistic regression. In the case of PRISM and PRISM III, the score was entered into the regression rather than the individual component variables of the tool. Optimized models were assessed by random allocation of PICUs (stratified by annual admissions) into development and validation samples in a 2:1 ratio. Reestimation of coefficients was conducted in the development sample, and model predictions were assessed in the external validation sample using the discrimination and calibration measures described above. This approach avoids the bias that is inherent when the same data are used both to fit the model and to emulate the application of the tool at new PICUs.26,27 The process was repeated 5000 times to avoid basing conclusions on a single random split of the data that may have favored 1 of the models by chance. Summaries of the 5000 resulting discrimination and calibration statistics were calculated to guide the success of the optimization.
Funnel Plots
To illustrate the assessment of the published models and optimization of models, we produced a series of funnel plots of PICU mortality ratios by number of admissions along with corresponding 99.8% control limits.28 Funnel plots are a variation on control charts that identify units with unexpectedly high (above upper control limit) or low (below the lower control limit) mortality.29,30 They are becoming established as a superior alternative to league tables and ranks for the presentation of performance measures. Units with mortality ratios within the control limits are defined to have mortality consistent with random variation.
Mortality ratios were calculated by dividing the observed PICU mortality by an expected mortality. A funnel plot of crude mortality ratios first was produced using the mortality of the whole population of admissions as an expected mortality. Then 2 types of funnel plot were produced for each risk-adjustment method, first using the published coefficients to generate the expected mortality and second using the optimized coefficients to generate the expected mortality Ethical approval was granted for this study by the Trent Multi-Research Ethics Committee (MREC/99/4/046).
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Twenty-six PICUs were invited to participate in UK PICOS, and 22 (85%) agreed. The 4 units that did not take part all are based in London and admitted a median of 342 admissions in 2001 compared with a median of 452 in units that participated. Four units were excluded after the data collection period because of low or unverified coverage. In the remaining 18 units, data were collected on 10197 (98%) of 10385 admissions during the study period. The median age of included admissions was 1.4 years (interquartile range: 0.2–6.6 years), and 5772 (57%) were male (Table 1). Just more than one third (3813; 37%) were admitted to the PICU directly from the operating theater, a minority (622; 16%) of which were unplanned. The 3 most common reasons for admission directly from the operating theater were complex congenital cardiac abnormality (330; 8.7%), congenital ventricular septal defect (262; 6.9%), and kyphoscoliosis (259; 6.8%). In contrast, most of those who were admitted to the PICU other than directly from the operating theater were unplanned (5727 of 6384; 90%). The 3 most common reasons for these admissions were bronchiolitis (548; 8.6%), status epilepticus or uncontrolled seizure (548; 8.6%), and primary brain injury (278; 4.4%).
|
Of the 10197 admissions included in the analysis, 633 (6.2%) patients died in the admitting PICU. An additional 301 (3.0%) died before discharge from the hospital and 516 (5.4%) were lost to follow-up, making total hospital mortality 9.2%. There was wide variation between PICUs in both the characteristics and the outcome of patients who were admitted (Table 1).
Data Completeness
PIM2 and PRISM III excluded the most admissions from mortality prediction (7.1% and 8.3%, respectively), whereas PIM and PRISM have no explicit exclusion criteria (Table 2). Of eligible admissions, 42% (range: 13%–85% between PICUs) for PIM, 55% (range: 29%–97%) for PRISM, and 46% (range: 25%–86%) for PRISM III-12 and -24 required at least 1 assumption of normality as a result of a nonrecorded tool component’s being not applicable (Table 2). Also, 10% (range: 1%–48% between PICUs) for PIM, 80% (range: 29%–99%) for PRISM, and 80% (range: 28%–99%) for PRISM III-24 required at least 1 assumption of normality as a result of a nonrecorded tool component’s being missing (Table 2). However, the bulk of the missing tool components relating to PRISM and PRISM III were attributed to the Glasgow Coma Score component’s being missing in 77% of eligible admissions. Excluding the Glasgow Coma Score, 24% (range: 5%–56%) for PRISM and 23% (range: 1%–56%) for PRISM III-12 and -24 required at least 1 assumption of normality as a result of a nonrecorded tool component’s being missing (Table 2).
|
Published Models
Discrimination
The discrimination of the 3 published tools is shown in Table 2. All tools provided "good" (c index >0.8) discriminatory power.22 PIM2 (c index = 0.84) had significantly (P < .001) better discrimination than PIM (c index = 0.81) but not significantly (P = .105) better discrimination than PRISM (c index = 0.82).
Calibration
All models displayed significant lack of fit and therefore poor calibration according to the Hosmer-Lemeshow test. The Cox calibration regression found that PIM and PRISM tended to overestimate risk in high-risk admissions, and PIM2 tended to underestimate risk in low- and medium-risk admissions (Fig 1 and Table 2). PRISM also underestimated risk in very-low-risk (<1%) admissions. The slope estimate from Cox’s calibration regression was less than the ideal value of 1 for all tools, suggesting that predictions generally were exaggerated compared with observed risk.
|
Optimization of Current Models
The mean sample size for reestimation of model coefficients varied according to exclusion criteria, ranging from 6185 for PRISM III to 6751 for PRISM. The mean sample size for validating the models ranged between 3164 and 3443. The median discrimination and calibration statistics from the 5000 validation samples are shown in Table 2.
Discrimination
On average, all tools provided "good" (c index >0.8) discriminatory power.22 In the whole sample, the discrimination for both PRISM III-12 and PRISM III-24 (median c index = 0.88) was found to be significantly (P < .001 in all cases) higher than for PRISM (median c index = 0.85), PIM (median c index = 0.81), and PIM2 (median c index = 0.84).
Calibration
Using the Hosmer-Lemeshow test, we found a lack of calibration to be significant at the 5% level in only 25% of samples for PRISM III-24 compared with 32% of samples for PIM2 (P < .001). In the whole sample, the Hosmer-Lemeshow test indicated significant lack of fit for PIM (P = .03) and PRISM (P = .01) and adequate fit for PIM2 (P = .32), PRISM III-12 (P = .13), and PRISM III-24 (P = .37). All models displayed a slight tendency toward overfitting, indicated by the average slope estimate’s being <1 from Cox’s recalibration regression in validation data. Calibration plots for models that were reestimated and assessed in the whole sample are shown in Fig 2.
|
Funnel Plots
Figure 3 contains a funnel plot of the crude mortality ratio by number of admissions for each unit, which indicates that 1 unit seemed to have an unadjusted mortality ratio that was higher than expected. The funnel plots that were produced using the published models indicate that 1 unit had adjusted mortality that was lower than expected for PIM, 2 units had adjusted mortality that was higher than expected for PIM2, and 7 units had adjusted mortality that was lower than expected for PRISM, including 1 unit for which PIM had identified as having mortality that was higher than expected. The funnel plots for the recalibrated models indicate that the adjusted mortality for all units is consistent with random variation for all models.
|
5% chance of a unit’s falling outside the control limits, then the upper and lower control limits must represent not 95% confidence intervals but 99.7% (confidence intervals around a mortality ratio of 1 by number of admissions |
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Admissions to PICUs in the United Kingdom vary considerably in their characteristics, making adjustment for case mix imperative when comparing outcomes between different units. We compared all major published tools for predicting PICU mortality using a large, representative sample of consecutive admissions to 18 UK PICUs between March 2001 and February 2002. All published models (PIM, PIM2, and PRISM) displayed good discrimination but poor calibration. Including all models and after recalibration on UK data, only PIM2, PRISM III-12, and PRISM III-24 displayed adequate fit. In addition, both PRISM III-12 and PRISM III-24 demonstrated significantly higher discrimination than the other tools.
The data completeness seems to be higher for PIM and PIM2 than for PRISM, PRISM III-12, and PRISM III-24. However, this was found to be attributable primarily to low completeness of the Glasgow Coma Score associated with the various PRISM scores and does not seem to have affected the overall performance of these scores.
The overall calibration of PIM and PIM2 was better than PRISM when published coefficients were used. This is illustrated further by the corresponding funnel plots, in which PRISM identified 7 units as having adjusted mortality outside the control limits whereas PIM identified 1 and PIM2 identified 2. This is not surprising as PRISM is much older (published in 1988) and based on a relatively small sample (N = 2642) of patients in North America.8 If PRISM had shown perfect calibration, then this would have suggested that risk-adjusted mortality after admission to UK PICUs in 2001 was similar to that of North American PICUs in 1988. Miscalibration of PRISM in validation samples has been noted several times before.15,17,18 PIM was published in 1997, and PIM2 was published in 2003. This group recently performed a large study that compared PIM, PIM2, PRISM, and PRISM III in PICUs in Australia and New Zealand and concluded that PIM2 was the most accurate.31 For a tool to calibrate well in a validation sample, all factors that influence outcome must either be included in the model or have the same distribution in the validation sample as the sample used to develop the model. Differences between countries and over time make this second condition less likely. PICUs have been found to vary considerably in their structure, organization, and staffing across European countries and still can be anticipated to do so between the United Kingdom and Australia.2,32
This study reinforces the need to use a representative and large number of units when assessing such risk-adjustment methods in another country or health care system. A small or unrepresentative selection of units could lead to confusion between unit or overall system performance and the need to recalibrate the score. Even including all units may result in poor calibration, but this does not necessarily invalidate the scores as a simple recalibration of these scores can lead to their being useful and valid tools for that country or health care system.
The performance of PIM2, PRISM III-12, and PRISM III-24 using coefficients estimated in UK data were acceptable, although PRISM III-12 and PRISM III-24 displayed an advantage in terms of discrimination. The funnel plots of adjusted mortality ratios for all of the recalibrated models suggested that mortality in all of the units in this study was consistent with random variation and illustrates further the importance of recalibrating the models.
The use of physiologic readings up to 24 hours after admission has been criticized by the authors of PIM. They argued that such readings may be recognizing death, rather than predicting it, in the substantial proportion of children who die within the first 24 hours (29% of deaths in our data).12 In addition, quality-of-care differences may be masked because PICUs that mismanage care in the first 24 hours will seem to have a worse case mix.12,33 The development of the PRISM III-12 tool was motivated by these concerns.9 However, others have argued that by using admission values only, as in PIM and PIM2, there is an increased risk for observing nontypical point estimate values that misrepresent the true initial health status of the patient.34 The results of this study show that good discrimination and calibration are achievable without using information that is obtained after the first hour in the PICU. The amount of data to be collected is an important consideration in the design of any study; therefore, PIM2 may be valuable for most quality improvement and audit studies that require the minimum of data collection necessary to undertake the study. Larger scale research projects with funding to collect additional data could reasonably choose to use PRISM III variables with the provisos mentioned above.27,35 Whichever risk-adjustment method is chosen, units that wish to participate in comparative research studies or collaborative quality improvement efforts must ensure that participating centers not only collect the data for the risk-adjustment tool in a standardized manner but also that standardized reporting of outcomes are fed back to the units.
Any variation in risk-adjusted PICU mortality needs to be tempered by an assessment of variation in longer term outcome. A reduction in mortality is of dubious benefit if it is at the expense of increased severe morbidity. Standardized and reliable methods of measuring health status (or at least clinical status) are required for use in the United Kingdom.36 Currently, only the PRISM family of tools has been shown to have any relationship to longer term outcomes in the United States.37
|
CONCLUSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
This study confirms that risk-adjustment methods that are developed primarily in other countries require validation before being used to provide risk-adjusted outcomes of PICU mortality for units within a new health care setting. It is also important that the calibration of these tools be reassessed periodically to ensure their continued validity. Identifying suitable risk-adjustment tools is only a first step. It now is important that they be applied effectively to monitor outcome and improve the quality of pediatric intensive care within the United Kingdom.30
|
ACKNOWLEDGMENTS |
|---|
This study was funded by the UK Medical Research Council (G9900013). The study sponsor played no part in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.
UK PICOS Steering Group: Stephanie Black, Anthony Brady, Caroline Goldfrad, and Kathy Rowan, Intensive Care National Audit & Research Centre; Brigitte Colwell, John Brazier, Sam Jones, Chris McCabe, Jon Nicholl, Gareth Parry, and Khadija Rantell, Katherine Stevens, University of Sheffield; Neil Bennett and Charles Stack, Sheffield Children’s Hospital; David Hallworth, Royal Hospital for Sick Children, Glasgow; Philip Holland, General Infirmary at Leeds; Ian Murdoch, Guy’s & St Thomas Hospital, London; Gale Pearson, The Birmingham Children’s Hospital NHS Trust; Jane Ratcliffe, Royal Liverpool Children’s Hospital.
UK PICOS Study Group: Participating PICUs were based at the following NHS Trusts: Addenbrookes NHS Trust, Cambridge; Birmingham Children’s Hospital, Birmingham; Bristol Royal Hospital for Children, Bristol; Leeds General Infirmary, Leeds; Royal Liverpool Children’s NHS Trust (Alder Hey) Liverpool; Sheffield Children’s Hospital, Sheffield; Guy’s & St Thomas Hospital Trust, London; Royal Hospital for Sick Children, Glasgow; Royal Hospital for Sick Children, Edinburgh; Newcastle Upon Tyne Hospitals NHS Trust, Newcastle; Royal Manchester Children’s Hospital, Manchester; The John Radcliffe Hospital, Oxford; Great Ormond Street Hospital for Children, London; Royal Hospital for Sick Children, Belfast; Northern Ireland, Queen’s Medical Centre, Nottingham; Southampton General Hospital, Southampton; North Staffordshire Hospital NHS Trust, Stoke-on-Trent; University Hospitals of Leicester, Leicester; and University Hospital of Wales, Cardiff.
We thank the staff of participating units for their support and time in contributing to UK PICOS.
|
FOOTNOTES |
|---|
Accepted Oct 13, 2005.
Address correspondence to Gareth J. Parry, PhD, Department of Medicine, Children’s Hospital Boston, 300 Longwood Ave, Boston, MA 02115. E-mail: gareth.parry{at}childrens.harvard.edu
The authors have indicated they have no financial relationships relevant to this article to disclose.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
1. Gemke RJBJ. Outcome Assessment of Pediatric Intensive Care: Principles and Applications [doctoral thesis]. Utrecht, Netherlands: University of Utrecht; 1994
2. Pearson G, Shann F, Barry P, et al. Should paediatric intensive care be centralised? Trent versus Victoria. Lancet. 1997;349 :1213 –1217[CrossRef][Web of Science][Medline]
3. National Co-ordinating Group on Paediatric Intensive Care. Paediatric Intensive Care: A Framework for the Future. Leeds, United Kingdom: NHS Executive Leeds; 1997
4. Black N. Why do we need observational studies to evaluate the effectiveness of health care?
Br Med J. 1996;312
:1215
–1218
5. Pollack MM, Cuerdon TT, Patel KM, Ruttimann UE, Getson PR, Levetown M. Impact of quality of care factors on pediatric intensive care unit mortality.
JAMA. 1994;272
:941
–946
6. UK Neonatal Staffing Study Group. A prospective evaluation of patient volume, staffing and workload in relation to risk-adjusted outcomes in a random, stratified sample of all UK neonatal intensive care units. Lancet. 2002;359 :99 –107[CrossRef][Web of Science][Medline]
7. Yeh TS, Pollack MM, Ruttimann UE, Holbrook PR, Fields AI. Validation of a physiologic stability index for use in critically ill infants and children. Pediatr Res. 1984;2 :171 –179
8. Pollack MM, Ruttimann UE, Getson PR. Pediatric risk of mortality (PRISM) score. Crit Care Med. 1988;16 :1110 –1116[Web of Science][Medline]
9. Pollack MM, Patel KM, Ruttimann UE. PRISM III: an updated pediatric risk of mortality score. Crit Care Med. 1996;24 :743 –752[CrossRef][Web of Science][Medline]
10. Children’s National medical Center. PICUEs: The International Standard for PICU Evaluations of Quality and Efficiency of Care. Available at: www.dcchildrens.com/picues/. Accessed February 9, 2006
11. Gemke RJBJ, Bonsel GJ. Comparative assessment of pediatric intensive care: a national multicenter study. Crit Care Med. 1995;23 :238 –245[CrossRef][Web of Science][Medline]
12. Shann F, Pearson G, Slater A, Wilkinson K. Paediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med. 1997;23 :201 –207[CrossRef][Web of Science][Medline]
13. Slater A, Shann F, Pearson G. PIM2: a revised version of the Paediatric Index of Mortality. Intensive Care Med. 2003;29 :278 –285[Web of Science][Medline]
14. Pearson GA, Stickley J, Shann F. Calibration of the paediatric index of mortality in UK paediatric intensive care units.
Arch Dis Child. 2001;84
:125
–128
15. Bertolini G, Ripamonti D, Cattaneo A, Apolone G. Pediatric risk of mortality: an assessment of its performance in a sample of 26 Italian intensive care units. Crit Care Med. 1998;26 :1427 –1432[CrossRef][Web of Science][Medline]
16. Gemke RJBJ, van Vught AJ. Scoring systems in pediatric intensive care: PRISM III versus PIM. Intensive Care Med. 2002;28 :204 –207[CrossRef][Web of Science][Medline]
17. Wells M, Riera-Fanego JF, Luyt DK, Dance M, Lipman J. Poor discriminatory performance of the Pediatric Risk of Mortality (PRISM) score in a South African intensive care unit. Crit Care Med. 1996;24 :1507 –1513[CrossRef][Web of Science][Medline]
18. Goddard JM. Pediatric risk of mortality scoring overestimates severity of illness in infants. Crit Care Med. 1992;20 :1662 –1665[Web of Science][Medline]
19. Parry GJ, Tucker JS, Tarnow-Mordi WO. CRIB II: an update of the clinical risk index for babies score. Lancet. 2003;361 :1789 –1791[CrossRef][Web of Science][Medline]
20. MRC/DOH Working Party on Intensive Care. The Research Needs and Opportunities Relevant to the NHS. London, United Kingdom: Medical Research Council; 1997
21. Young JD, Goldfrad C, Rowan K. Development and testing of a hierarchical method to code the reason for admission to intensive care units: the ICNARC Coding Method.
Br. J Anaesth. 2001;87
:543
–548
22. Murphy-Filkins R, Teres D, Lemeshow S, Hosmer DW. Effect of changing patient mix on the performance of an intensive care unit severity-of-illness model: how to distinguish a general from a specialty intensive care unit. Crit Care Med. 1996;24 :1968 –1973[CrossRef][Web of Science][Medline]
23. Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests.
JAMA. 1982;247
:2543
–2546
24. Hosmer DW, Lemeshow S. Goodness of fit tests for the multiple logistic regression model. Commun Stat. 1980;A9 :1043 –1069
25. Cox DR. Two further applications of a model for binary regression.
Biometrika. 1958;45
:562
–565
26. Efron B. How biased is the apparent error rate of a prediction rule? J Am Stat Assoc. 1986;81 :461 –470[CrossRef][Web of Science]
27. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19 :453 –473[CrossRef][Web of Science][Medline]
28. Spiegelhalter D. Funnel plots for institutional comparison.
Qual Saf Health Care. 2002;11
:390
–391
29. Simpson JM, Evans N, Gibberd RW, Heuchan AM, Henderson-Smart DJ. Analysing differences in clinical outcomes between hospitals.
Qual Saf Health Care. 2003;12
:257
–262
30. Paediatric Intensive Care Audit Network. National Report 2003–2004. Universities of Leeds, Leicester, and Sheffield. Sheffield, United Kingdom: PICANet; 2005
31. Slater A, Shann F, for the ANZICS Paediatric Study Group. The suitability of the Paediatric Index of Mortality (PIM), PIM2, the Paediatric Risk of Mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand. Pediatr Crit Care Med. 2004;5 :447 –454[CrossRef][Medline]
32. Nipshagen MD, Polderman KH, DeVictor D, Gemke RJ. Pediatric intensive care: result of a European survey. Intensive Care Med. 2002;28 :1797 –1803[CrossRef][Web of Science][Medline]
33. Randolph AG. Paediatric index of mortality (PIM): do we need another paediatric mortality prediction score? Intensive Care Med. 1997;23 :141 –142[CrossRef][Web of Science][Medline]
34. Tarnow-Mordi WO, Tucker J, Parry GJ. Should paediatric intensive care be centralised? Lancet. 1997;350 :66 –67[Web of Science][Medline]
35. Kuhlthau K, Ferris TGG, Iezzoni LI. Risk adjustment for pediatric quality indicators.
Pediatrics. 2004;113
:210
–216
36. Keizer NF, Bonsel GJ, Gemke RJBJ. Health status prediction in critically ill children: a pilot study introducing standardized health ratios. Qual Life Res. 1997;6 :192 –199[Medline]
37. Ruttimann UE, Pollack MM, Fiser DH. Prediction of three outcome states from pediatric intensive care. Crit Care Med. 1996;24 :78 –85[CrossRef][Web of Science][Medline]
PEDIATRICS (ISSN 1098-4275). ©2006 by the American Academy of Pediatrics
CiteULike 
Connotea 
Del.icio.us 
Digg 
Facebook 
Reddit 
Technorati 
Twitter What's this?
This article has been cited by other articles:
|
|
 |
|
  C. Wang, G. Xie, B. Cheng, L. Du, L. Shi, L. Tan, Q. Shu, and X. Fang Performance of Pediatric Risk of Mortality, Pediatric Index of Mortality and PIM2 in term Chinese neonates J Trop Pediatr, December 17, 2009; (2009) fmp124v1. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. P. Marcin and M. M. Pollack Review of the Acuity Scoring Systems for the Pediatric Intensive Care Unit and Their Use in Quality Improvement J Intensive Care Med, May 1, 2007; 22(3): 131 - 140. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
