Context. Pediatric intensive care units (PICUs) have expanded nationally, yet few studies have examined the potential impact of regionalization and no study has demonstrated whether a relationship between patient volume and outcome exists in these units. Documentation of an inverse relationship between volume and outcome has important implications for regionalization of care.
Objectives. This study examines relationships between the volume of patients and other unit characteristics on patient outcomes in PICUs. Specifically, we investigate whether an increase in patient volume improves mortality risk and reduces length of stay.
Design and Setting. A prospective multicenter cohort design was used with 16 PICUs. All of the units participated in the Pediatric Critical Care Study Group.
Participants. Data were collected on 11 106 consecutive admissions to the 16 units over a 12-month period beginning in January 1993.
Main Outcome Measures. Risk-adjusted mortality and length of stay were examined in multivariate analyses. The multivariate models used the Pediatric Risk of Mortality score and other clinical measures as independent variables to risk-adjust for illness severity and case-mix differences.
Results. The average patient volume across the 16 PICUs was 863 with a standard deviation of 341. We found significant effects of patient volume on both risk-adjusted mortality and patient length of stay. A 100-patient increase in PICU volume decreased risk-adjusted mortality (adjusted odds ratio: .95; 95% confidence interval: .91–.99), and reduced length of stay (incident rate ratio: .98; 95% confidence interval: .975–.985). Other PICU characteristics, such as fellowship training program, university hospital affiliation, number of PICU beds, and children's hospital affiliation, had no effect on risk-adjusted mortality or patient length of stay.
Conclusions. The volume of patients in PICUs is inversely related to risk-adjusted mortality and patient length of stay. A further understanding of this relationship is needed to develop effective regionalization and referral policies for critically ill children.
- PICU =
- pediatric intensive care unit •
- PCCSG =
- Pediatric Critical Care Study Group •
- PRISM =
- Pediatric Risk of Mortality •
- EBMCC =
- Evidence Based Medicine in Critical Care •
- CI =
- confidence interval •
- OR =
- odds ratio •
- IRR =
- incident rate ratio
Pediatric intensive care units (PICUs) exhibit wide variation in cost and quality, similar to other fields of medicine.1–5 Surveys of PICUs reveal differences in structural and process factors6,7 and several studies have assessed these and other factors that might explain variations in outcomes.8–11 Others have noted the rapid expansion of PICUs and PICU beds that occurred during the 1980s.12 The expansion of PICUs and PICU beds may be related to variations in patient length of stay and PICU efficiency, creating concerns that too many PICUs and PICU beds exist.11,13 Despite these concerns, few studies have examined the potential impact of regionalization of PICU services,8,14–16 and no study has demonstrated a relationship between patient volume and outcomes in the PICU setting.10
Relationships between volume and outcomes are of special importance for regionalization policy.17–21 An inverse relationship between patient volume and outcomes suggests that regionalization may improve patient care quality17,22 or reduce the cost of care.21 Volume–outcome relationships have been studied in a number of different settings and have generally documented improved outcomes from greater volume.17–21,23–26
Two hypotheses to explain volume–outcome relationships are described in the literature. One hypothesis is that greater patient volume provides surgeons and other providers with more experience and leads to a practice-makes-perfect effect. Alternatively, the selective referral hypothesis suggests that higher quality institutions have developed community reputations resulting in greater numbers of referrals and, subsequently, increased patient volume.27 Both explanations have implications for regionalization policy. The practice-makes-perfect hypothesis implies that any mechanism that concentrates patients in high-volume institutions will improve outcomes. In contrast, the selective referral hypothesis still implies outcomes can be improved through regionalization, but only through the appropriate selection of hospitals. Both explanations are possible and have support in the literature. The general findings suggest that patient categories exhibiting more complex diagnoses are more consistent with a selective referral effect.20
There is reason to believe that volume–outcome relationships are possible in the setting of the PICU. PICUs generally care for heterogeneous groups of critically ill children whose survival depends on the quality of care received in the PICU.8,10 Because PICU patients are seriously ill and their care is often complicated, the potential for increased volume to be related to patient outcomes warrants further investigation.
This study examines volume–outcome relationships in the setting of the PICU. We specifically test whether an increase in the volume of patients is associated with improved mortality risk and shorter lengths of PICU stay. We also examine other quality-of-care factors using recent criteria developed by the Evidence Based Medicine in Critical Care Group.28
A prospective multicenter inception cohort design was used, with 16 PICUs that were members of the Pediatric Critical Care Study Group (PCCSG). In this study, all members of the PCCSG were invited to participate. Units were asked to contribute data on consecutive admissions over a 1-year period beginning in January 1993. Initially, 21 PICUs agreed to contribute data for this study, but 5 PICUs did not complete data collection. These units were excluded leaving a sample of 16 PICUs with observations on 11 106 patients. The study protocol was submitted to the human advisory research committee of the respective institutions for approval. The provision for informed consent was waived.
Information on the characteristics of each unit was provided by the PCCSG. These characteristics included the teaching status of the PICU, whether the unit had a fellowship training program, whether the hospital in which the unit was located had a separate step-down or monitored care unit, whether the unit was located in a children's hospital, and the number of beds in the unit. All of the units were headed by a board-certified pediatric intensivist, a factor known to be associated with improved outcomes.29 Patient level data included Pediatric Risk of Mortality (PRISM) score,30Pediatric Overall and Cerebral Performance scores as measures of functional status,31 primary and secondary diagnosis codes (International Classification of Diseases, Ninth Revision), surgical and trauma status, patient age, patient length of stay, and survival at PICU discharge. PICU volume was measured by the total number of admissions to the unit.
PICU outcomes were modeled using a 2-stage procedure. In the first stage, a clinical model was developed to adjust for severity and case-mix differences in mortality and length of stay across the 16 PICUs. The clinical models were developed through logistic and negative binomial regression models. Multiple independent predictors of mortality and length of stay were assessed and objectively reduced using backward elimination to form the best predictive models. PRISM scores were included using a quadratic specification to account for unequal variance in the scores between survivors and nonsurvivors.10
In the second stage, the impact of volume, university hospital affiliation, children's hospital, intermediate care unit, fellowship program, and number of PICU beds was examined on mortality risk and length of stay by adding these characteristics to the clinical model. In both the mortality risk and length of stay analyses, adjustments were made for clustered observations based on recent recommendations.32 Clustering occurs when observations (patients) are generated from different groups (PICUs). Observations within groups can no longer be considered independent because of within group correlation.33 Because the observations for this study were clustered within PICUs, a multilevel model based on generalized estimating equations was used to account for within and between PICU sources of variation.34
We tested several alternative models to evaluate patient length of stay across the 16 PICUs. Concerns with the multiplicative nature of log-transformed models35 and the inability to fit a geometric model with clustering resulted in the use of a negative binomial model. In this model, discrete measures of patient length of stay were created by rounding up. For example, all patients with short stays of <1 day were coded as having a stay of 1 day. It should be noted in this situation that the negative binomial is an a priori appropriate model, because it can be derived from assuming a Poisson or counting model, where the expected value of the mean itself has a random distribution (for patients of varying illness) of a (general) γ-distribution. Further, to generate a length of stay model with acceptable fit, outliers beyond the 98th percentile were excluded from the analysis.
The possibility of good model performance was initially evaluated for both models by determining whether the scaled deviance divided by the degrees of freedom was <1. Additionally, model performance for the logistic regression was examined using measures of discrimination and calibration.36,37 Discrimination measures the ability of a model to distinguish patients who die from patients who live based on the estimated probabilities. Discrimination was measured using the area under the receiver operating characteristic curve.38Calibration evaluates the correspondence between the estimated probabilities of mortality generated by the model and the actual mortality of the patient population. Calibration was measured by the Hosmer-Lemeshow goodness-of-fit test.39 Due to the clustering of observations within hospitals, discrimination and calibration were computed for each hospital, because the usual calculation of these statistics assumes that observations are distributed independently.
The results of the mortality model also were compared with a similar model that includes severely disabled outcomes as a negative outcome after recommendations from the Evidence Based Medicine in Critical Care (EBMCC) working group. It is recognized that mortality may not be a sufficient indicator of outcome.28,40 In particular, some have argued that reductions in mortality may come at the expense of increased morbidity.41 Thus, we tested whether our findings on mortality differed when severely disabled outcomes were included as poor outcomes.
Table 1 provides characteristics of the 16 PICUs. Because all of the units are members of the PCCSG, there is a high concentration of university-affiliated units (81%). The high concentration of university-affiliated units also is reflected in the size of the units. The median number of PICU beds in this sample is 14 and suggests that the average unit in this sample is larger than in previous investigations of PICU quality-of-care factors.10Volume ranges from a low of 147 patients to a high of 1378 patients. PICUs with fellowship training programs, affiliations with children's hospitals, and intermediate or step-down care units have nearly equal representation in the sample.
Table 2 provides characteristics of the patient population. There is considerable variation in patient length of stay, unadjusted mortality rates, surgical status, trauma status, and age across the 16 PICUs. The median length of stay and the average mortality rate are consistent with previous reports.7,10,11
Table 3 provides the results of the multivariate logistic regression analysis that estimates the relationship between volume and PICU mortality using adjustments for clustering of data by PICU. Both the PRISM scores and PRISM scores squared were significant and had coefficient estimates within a 95% confidence interval (CI) that correspond to a previous investigation of quality-of-care factors.10 Taking into account both PRISM terms, a change in the PRISM score from 10 to 20 increased the odds of a patient dying by a factor of 7.6. Coefficient estimates for the dichotomous indicators describing primary diagnoses also were within the 95% confidence limits of previous studies.
In the analysis of quality-of-care factors, university affiliation, fellowship program, children's hospital, number of beds, and intermediate care unit had no impact on mortality after adjustments for severity and case-mix differences. Only the volume variable remained significant after adjustments for clustering. Notably, an inverse relationship was demonstrated between patient volume and mortality (adjusted odds ratio [OR]: .95; 95% CI: .91–.99, for a volume change of 100 patients). In this sample of 16 PICUs, a 1 standard deviation change in volume equaled 341 patients.
For the logistic model, the scaled deviance divided by the degrees of freedom was well below 1 (scaled deviance/degrees of freedom = .22), which is one indication of good fit. Discrimination was good; the area under the receiver operating characteristic curve ranged from .77 to .96 with a median area of .91. Calibration was acceptable to good for the majority of the hospitals. The χ2statistic for the Hosmer-Lemeshow goodness-of-fit test was good (P > .20) for 11 hospitals, adequate (.10 <P < .20) for 2 hospitals, and poor for 3 hospitals (P < .10).
The relationship between volume and risk-adjusted mortality was examined further using functional outcome measures. In this analysis, the main outcome measure was coded to include severely disabled outcomes. When functional outcomes were included as an outcome, the shape of the relationship remained the same, but the estimated impact and statistical significance was weakened (OR: .99; 95% CI: .98–1.00).
Table 4 provides the multivariate regression results for the negative binomial model that predicts patient length of stay. The model has good fit (scaled deviance/degrees of freedom = .90). Many of the same clinical variables that predicted mortality in the logistic regression model also predicted length of stay in the negative binomial regression model. The total effect of the PRISM scores on length of stay was limited relative to the mortality model. A change in the PRISM score from 10 to 20 was associated with a 58% increase in PICU length of stay. Again, none of the structural characteristics of the PICUs with the exception of volume were significant after adjustment for clustering. Estimates of the relationship between volume and risk-adjusted length of stay indicated a small but statistically significant inverse relationship (incident rate ratio [IRR]: .98; 95% CI: .975–.985, for a change in volume of 100 patients).
This study provides the first documentation of an inverse relationship between patient volume and outcomes in the setting of the PICU. PICUs with greater volumes had reductions in both PICU mortality risk and PICU length of stay. Analysis of other quality-of-care factors had no impact on PICU mortality risk or patient length of stay after controlling for clustering across the 16 PICUs. Similar results were obtained when severely disabled outcomes were included as a poor outcome.
We are aware of only one other study that examined whether volume was associated with outcomes in the setting of the PICU. This study found a significant negative relationship between PICUs with residency teaching programs and risk-adjusted mortality, but volume was insignificant.10 A subsequent study found a positive relationship between fellowship programs in pediatric critical care and risk-adjusted mortality.42
The difference in findings between this study and the previous investigations may be related to differences in study designs. This study did not choose study sites randomly based on preselected quality-of-care factors. The nonrandom selection of PICUs led to a greater number of high-volume academic institutions included as study sites and may have created a sample selection effect. The large disparity in patient volume between PICUs may account for the finding of an inverse relationship between patient volume and outcomes. The findings also may be related to differences in sample size. Data for this study were collected for an entire year, regardless of the number of deaths that occurred at any given PICU. Attempts to disentangle differences based on sample size calculations are confounded by adjustments for clustering. Recent work has developed sample size calculations for randomized designs,43 but similar calculations are not available for generalized linear models based on observational designs.
There also were a number of similarities between this study and previous investigations. The contribution of the PRISM scores to mortality risk and the overall model performance are quite similar. Such findings are important because other investigations have found paradoxical associations attributable to the failure of the PRISM to account for premorbid illness.44 The more limited contribution of the PRISM score to length of stay is a consequence of shorter PICU stays by patients who die.40
The finding that an inverse relationship exists between patient volume and outcomes in the setting of the PICU, however, is limited by the inability to test competing hypotheses. We were unable to assess whether the estimated relationships provide evidence for or against either a practice-makes-perfect effect or a selective referral effect. Further, the data do not permit an analysis of specific types of illness or injury cared for in PICUs. Such an investigation requires a broader set of clinical variables and a larger database. The low rate of mortality in pediatric health services necessitates large samples to estimate statistically significant relationships.45
Our analysis of the relationship among volume, mortality risk, and length of stay in PICUs was guided by criteria from the EBMCC working group. The EBMCC working group asks 3 general questions of research articles. These questions include: 1) Were valid methods used?; 2) How should the results be interpreted?; and 3) Will the results help improve patient care? We discuss these criteria in relation to the study findings.
In response to the first question, the study demonstrates both strengths and weaknesses. The strengths include appropriate clinical measures from prospectively collected data to risk-adjust differences in PICU mortality and length of stay and a sufficient sample to test for a general relationship. We also tested our general findings using mortality outcomes against a similar model that included severely disabled outcomes. Finally, we used multilevel modeling to adjust for sample clusters across PICUs following the recommendations of the EBMCC working group.28,34 Despite these strengths, the study contains a number of weaknesses, including a lack of more comprehensive outcome measures and an inability to focus on care provided in the PICU. A more comprehensive set of outcomes would include known complications and survival to hospital discharge. However, such an analysis requires data on specific conditions and patient follow-up after PICU discharge. This information was not available. We also have no evidence that smaller PICUs were less likely to use appropriate therapies or more likely to use inappropriate therapies.46
The interpretation of the study findings is straightforward. A general relationship between volume and outcomes is documented that suggests that volume is an important factor, influencing both the quality and efficiency of care. A 1–standard deviation change in volume was associated with a 17% reduction in mortality risk and corresponds to a recent investigation examining volume–outcome relationships for acute myocardial infarction in elderly patients.47 The number of studies that have demonstrated such a relationship in settings where patient care is complex increases confidence in this finding. Critically ill children admitted to PICUs represent an extremely heterogeneous patient population. Pediatric intensivists often are confronted with rare life-threatening illnesses, where clinical experience plays an important role in determining outcome. Critically ill children also are likely to be transferred to tertiary care centers with specific expertise in pediatrics. The need for clinical experience and the ability to transfer patients are both consistent with volume–outcome relationships.
Finally, there is the question of whether the findings reported in this study will help to improve patient care. The new focus on quality in health care emphasizes the importance of comparing risk-adjusted outcomes. The idea is not to punish providers with poor outcomes but to provide information that can lead to quality improvement.48 This study provides additional information on the system of care for critically ill children. Providers can use this information in considering whether the system of care can be improved or whether patient care can be improved by referral to high-volume providers. Information on volume–outcome relationships has already affected systems of care. Studies documenting improved outcomes from increased patient volume have led some researchers to speculate that volume will become the proxy indicator for quality in health care and will result in the reorganization of systems to reflect this emphasis.49
The expansion of PICUs has increased concerns that too many units and beds exist and may be related to large variations in the efficiency of services.13 These concerns were not based on evidence of a relationship between volume and outcomes. This study provides the first evidence that volume is an important predictor of outcomes in the setting of the PICU. A further understanding of this relationship is needed to develop effective regionalization or referral policies for critically ill children. Such policies have the potential to improve both the quality and efficiency of pediatric critical care services.
This study was funded in part by Grant HS09055 from the Agency for Healthcare Research and Quality (formerly AHCPR) in collaboration with the Health Resources and Services Administration Maternal and Child Health Bureau.
- Received August 5, 1999.
- Accepted November 29, 1999.
Reprint requests to (J.M.T.) Center for Applied Research and Evaluation, Department of Pediatrics, 800 Marshall St, Little Rock, AR 72202-3591. E-mail:
- Hannan E,
- Racz M,
- Kavey R,
- Quaegebeur JM,
- Williams R
- Hadorn DC, Keeler EB, Rogers WH, Brook RH. Assessing the Performance of Mortality Prediction Models. Santa Monica, CA: RAND; 1993
- Hosmer DW, Lemeshow S. Applied Logistic Regression. New York, NY: John Wiley and Sons, Inc; 1989
- Van Etten P
- Copyright © 2000 American Academy of Pediatrics