Objective. To examine the relationship between annual provider (hospital and surgeon) volume of pediatric cardiac surgery and in-hospital mortality.
Design. Population-based retrospective cohort study using a clinical database.
Setting. The 16 acute care hospitals in New York with certificate of need approval to perform pediatric cardiac surgery.
Patients. All children undergoing congenital heart surgery in New York from 1992 to 1995.
Main Outcome Measures. Risk-adjusted mortality rates for various hospital and surgeon volume ranges. Adjustments were made for severity of illness using logistic regression.
Results. A total of 7169 cases were analyzed. After controlling for severity of preprocedural illness using clinical risk factors, hospitals with annual pediatric cardiac surgery volumes of fewer than 100 had significantly higher mortality rates (8.26%) than hospitals with volumes of 100 or more (5.95%), and surgeons with annual volumes of fewer than 75 had significantly higher mortality rates (8.77%) than surgeons with annual volumes of 75 or more (5.90%).
Conclusions. Both hospital volume and surgeon volume are significantly associated with in-hospital mortality, and these differences persist for both high-complexity and low-complexity pediatric cardiac procedures.
Numerous studies in the past 2 decades have documented significant inverse relationships between adverse outcomes for certain types of patients and the amount of experience providers have in treating those patients.1-8 Generally, patients have been identified according to the type of procedure they underwent or their medical condition (principal diagnosis), with surgical examples being more frequent. Also, provider volume has been measured both on the hospital and the physician/surgeon level, with some studies investigating both volume measures. For the most part, in-hospital or short-term mortality has been used as the measure of adverse outcome, although complications of treatment and hospital length of stay have also been used. The more sophisticated studies have attempted to investigate volume-outcome differences after having adjusted for patient severity of illness using various demographic and diagnostic indicators of severity. The reader is referred to the book by Luft et al9 for a thorough description and review of the methods used, assumptions made, procedures and medical conditions investigated, and current areas of research in this large body of work.
Only one study in the literature has investigated the relationship between adverse outcomes and the volume of pediatric cardiac surgery. Probably one of the reasons for this is that pediatric cardiac surgery involves a myriad of different procedures, involving different surgical challenges and entailing a wide range of risks to patients. Also, the annual volumes for each of these individual procedures tend to be quite low, even when accumulated across an entire state or large geographical region.
The purpose of this study is to examine the relationship between in-hospital mortality and provider (hospital and surgeon) volume for pediatric cardiac surgery in New York State between 1992 and 1995. This study extends the earlier study conducted by Jenkins et al1in a few respects. First, both hospital volume and surgeon volume are available as measures of provider volume, whereas Jenkins et al had access only to hospital volume. Second, clinical data from New York's Cardiac Surgery Reporting System (CSRS) are available for conducting risk-adjustments rather than having to rely on administrative data collected for other purposes. Third, in addition to investigating aggregate differences in risk-adjusted mortality for various provider volume groups, this study examines risk-adjusted mortality rate differences between provider volume groups for different levels of procedure complexity.
DATA AND METHODS
The database used in the study is the part of New York's CSRS dedicated to pediatric cardiac surgery. The CSRS was initiated in 1989 by the New York State Department of Health and its Cardiac Advisory Committee (CAC) for the purpose of improving quality of care in New York. The CAC is a group of cardiac surgeons, cardiologists, health services researchers, and consumers charged with advising the Department of Health on issues related to the quality of and access to cardiac care, and the prevention of coronary heart disease. The CSRS contains information on patient demographics, types of procedures performed, risk factors/diagnoses, complications of surgery, discharge status, and surgeon and hospital identifiers. The information is collected concurrently under the direction of each hospital's director of cardiac surgery. Completeness of registry data is assured by matching it with New York's administrative database, the Statewide Planning and Resource Cooperative System. Data accuracy is assured by conducting medical record reviews in each hospital.
The quality of coronary artery bypass graft surgery has been improved by identifying significant risk factors related to coronary artery bypass graft surgery, risk-adjusting adverse outcomes using these factors, computing risk-adjusted mortality rates for surgeons and hospitals, and providing this information to hospitals, surgeons, and the public.10,11 This has not been done to date for other procedures in the database (eg, valve surgery, pediatric cardiac surgery) primarily because these procedure groups are not as homogenous and have much lower volumes.
Although pediatric cardiac surgery data have been available since 1989, the set of risk factors in the database was expanded substantially in 1991. The data in this study comprises all pediatric cardiac surgery performed in New York State between 1992 and 1995 that is reportable in the state's pediatric cardiac surgery registry mentioned above, a total of 7169 procedures. These procedures are performed in the 16 hospitals in New York that have certificate of need approval.
The pediatric cardiac surgery report form that is part of the CSRS contains information on patient demographics (age, date of birth, sex, ethnicity, race); admission, procedure, and discharge dates; primary diagnosis and procedure codes; mode of cardiopulmonary bypass; weights at birth and at time of operation; number of previous open heart and closed heart operations; previous catheter interventions; 11 other risk factors; and a variety of complications after surgery.
The procedures used in the study were identified and defined by the CAC so as to accurately characterize the treatments obtained and the risks incurred by patients. They are identified using special codes created by the CAC that are similar to, but do not map on a one-to-one basis with, ICD-9-CM codes.
In CSRS, procedures are classified according to recommendations from the CAC. As mentioned above, one of the challenges in examining volume-outcome relationships in pediatric cardiac surgery is that there is a wide variety of procedures performed, with no one procedure having a sufficient volume in most databases to sustain complicated multivariate analyses. Consequently, the best analysis option is the one used by Jenkins et al,1 which is to combine procedures into groups that are as homogenous as possible with respect to patient severity of illness and to use the groups as severity measures (risk factors) in the risk-adjustment process.
The method used in this study to define complexity categories consisted of the following steps: (1) order the procedures according to in-hospital mortality rate from low to high, (2) look for natural breaks in the contiguous mortality rates so that between three and five groups could be identified (it was decided that this was a good range to use so that enough groups would be available for describing severity of illness differences while maintaining a sufficiently large number of cases in each group), (3) have the pediatric cardiac surgeon (J.Q.) and the two pediatric cardiologists (R.E.K. and R.W.) revise the categories so that similar procedures would be contained in the same category even if they had somewhat dissimilar mortality rates, and to reflect procedural complexity not necessarily captured by mortality rates in the database (perhaps because of low volumes), and (4) repeat steps 1 and 2. Four procedure complexity categories were identified.
The first step in the analysis consisted of calculating the frequencies and mortality rates associated with each of the demographic and diagnostic risk factors contained in New York's clinical pediatric cardiac surgery database. Later, these analyses were expanded to include frequencies and mortality rates for each of two hospital volume ranges (<100 pediatric cardiac procedures annually, 100 or more procedures annually). Significance of the various categorical variables (eg, sex, race, binary risk factors such as congestive heart failure) was determined using χ2 tests.
Next, a stepwise logistic regression model was constructed to determine which of the potential risk factors were significant predictors of in-hospital mortality, and how to predict mortality on the basis of those risk factors. The dependent variable was binary, with a 1 denoting in-hospital mortality and a 0 denoting a live discharge. The candidate independent variables were the demographic and diagnostic risk factors described above, and the four complexity categories. Independent variables were retained in the model if they were significant in the stepwise analysis (P < .05).
Complexity categories were tested in the model by using the group with the lowest complexity as a reference group, and the other three groups as binary risk factors in the model. Age was tested as a categorical variable by splitting it into ranges that were defined after examining their bivariate relationship with mortality. The ranges <7 days, 7 to 29 days, 30 to 89 days, 90 to 179 days, 180 to 359 days, and 360 days or more were used. Also, after a set of significant independent variables was identified, two-way interactions among these variables were tested in a new stepwise model. As a test of the adequacy of the complexity categories, observed and expected (using the model) mortality rates were compared for each of the complexity categories.
After the final statistical model was identified, hospital and surgeon volume measures associated with each case were computed as the number of pediatric cardiac procedures performed in that calendar year in the hospital the procedure was performed and by the surgeon performing the procedure, respectively. Then, risk-adjusted mortality rates were calculated for different hospital and surgeon volume groups in an attempt to determine which split that created high-volume and low-volume groups yielded the largest differential in risk-adjusted mortality rates between the two groups while maintaining reasonably large volumes in the two groups. Splits at 100 procedures annually in hospitals and 75 procedures annually for surgeons were obtained. As a confirmation that the two volume measures were significant predictors of mortality, they were also tested by adding them to the logistic regression model described above, and testing their significance in the model. Another test consisted of using the average 4-year hospital and surgeon volumes in lieu of annual volumes to determine if the conclusions changed substantially (they did not).
In computing risk-adjusted mortality rates, the first step consisted of computing the expected mortality rate for the volume groups by summing the predicted probabilities of death for each patient in the group using the logistic regression model, and then dividing by the number of patients. For each volume group, this rate was divided into the observed mortality rate (number of deaths/number of patients), and then multiplied by the overall mortality rate for all pediatric surgery patients to obtain the risk-adjusted mortality rate for the group. This rate represents the best estimate of what each hospital volume or surgeon volume group's mortality rate would have been if it had had an average severity of illness that was the same as that of the state as a whole. The rate for each group was then tested to determine if it was (statistically) significantly higher or lower than the statewide rate by calculating confidence intervals for risk-adjusted rates.12
The next set of analyses was aimed at determining if there were significant differences in risk-adjusted mortality rates for hospital volume groups and for surgeon volume groups by complexity category. Because the analysis used complexity categories rather than the entire data set (and this reduced the sample sizes), the relationship between each of the two provider volume measures and risk-adjusted mortality was examined separately rather than observing intersections of the volume measures. The purpose of these analyses was to determine if the relationship between mortality rate and provider volume was limited to the more complex procedures, or if it persisted across all complexity categories.
In the next set of analyses, four hospital volume/surgeon volume groups were obtained by splitting patients on the basis of their hospital and surgeon volumes (hospital volume <100/surgeon volume <75, hospital volume <100/surgeon volume >75, and so forth). Then, observed and risk-adjusted mortality rates were calculated for each intersection of hospital volume group and surgeon volume group to determine whether there were significant differences in risk-adjusted rates among the resulting four groups, and to determine if there were interaction effects between hospital volume and surgeon volume.
Table 1 presents the prevalence rates for various patient demographics and risk factors among pediatric cardiac surgery patients in New York between 1992 and 1995, as well as the in-hospital mortality rates for these groups of patients (last two columns, respectively). In addition, patients have been subdivided into the group who underwent cardiac surgery in hospitals with an annual pediatric cardiac surgery volume of <100 and hospitals with an annual pediatric cardiac surgery volume of 100 or more. Prevalence rates and mortality rates are reported for both of these groups. Note that if a hospital had a volume <100 in one year and >100 in another year, its patients are reported in different groups in the 2 years. It should also be noted that annual hospital volumes ranged from 19 to 379 cases, and annual surgeon volumes ranged from 1 to 364 cases.
The overall mortality rate for the 7169 patients was 6.75%, with a mortality rate of 7.52% for patients undergoing surgery in hospitals with annual pediatric cardiac surgical volumes of <100, and a mortality rate of 6.28% for patients undergoing surgery in hospitals with pediatric volumes of 100 or more. Patient age, sex, race, and all preoperative surgical risk factors available in the clinical database except number of previous open heart operations were found to be significantly (bivariately) related to inpatient mortality rate (see last column).
A comparison of the prevalence rates of demographics and risk factors between the two hospital volume groups demonstrates that hospitals with annual pediatric volumes <100 had significantly older patients, and a significantly lower percentage of patients of white race, and with previous open heart and closed heart operations. The lower volume hospitals also had a significantly lower percentage of patients with severe cyanosis or hypoxia, on a ventilator or inotropic support immediately before the operation, and with congestive heart failure. However, they had a significantly higher percentage of their patients who had an arterial pH <7.25 before the operation.
The Appendix presents, for each of the four complexity categories, each of the procedures contained in the category along with the frequency with which it was performed and the mortality rate of patients undergoing it in New York between 1992 and 1995. As the Appendix demonstrates, the distribution of patients in the four categories ranged from 12% in Category II to 48% in Category I. The mortality rates of the four categories were well distinguished from one another, with the respective rates being 1.39%, 4.48%, 10.97%, and 20.11%.
Table 2 presents the logistic regression model that predicts in-hospital mortality on the basis of demographics, available risk factors in the database, and the four procedure categories. As demonstrated in Table 2, the risk factors significantly (directly) related to mortality for pediatric cardiac surgery patients were age <90 days, age between 90 days and 1 year, severe cyanosis or hypoxia, arterial pH <7.25 preoperatively, significant extracardiac anomalies, pulmonary hypertension, and procedure complexity category. Except for age and complexity category, all of these factors are binary, and the odds ratios represent the odds of a patient with that risk factor dying in the hospital divided by the odds of a patient without the risk factor dying in the hospital, assuming that both patients have the same set of other significant risk factors. Thus, patients with severe cyanosis or hypoxia have odds of dying in the hospital that are 1.973 times the odds of patients without severe cyanosis or hypoxia dying in the hospital, all other risk factors being the same.
For age, the risk represented is in relationship to the reference (omitted) category, which is age >1 year. Thus, the odds of a patient <90 days old dying in the hospital are 2.317 times the odds of a patient older than 1 year dying in the hospital, if all other risk factors are identical.
The odds for each of the complexity categories in the model are relative to patients in Complexity Category I. Thus, for example, the odds of a patient in Complexity Category III dying in the hospital are 5.316 times the odds of a patient in Complexity Category I if all of both patients' other risk factors are the same.
Table 3 presents, for each of the four procedure complexity categories, a comparison of patients undergoing procedures in hospitals with annual pediatric cardiac surgery volumes of <100 with patients undergoing these procedures in hospitals with volumes of at least 100.
The comparisons include percentage of procedures, observed mortality, and risk-adjusted mortality for each of the two volume groups. The same information is reported for two surgeon volume groups (annual volume <75, annual volume 75 or more) in the bottom half of Table 3.
The percentage of procedures in each complexity category performed in hospitals with annual pediatric cardiac surgery volumes <100 was 42% for procedures in Complexity Category I, 33% for procedures in Complexity Category II, 34% for Complexity Category III, and 36% for Complexity Category IV. Thus, although there was a tendency for low volume hospitals to perform a larger share of procedures in the least complex group than in the other groups, there was no difference in the share of procedures performed in the highest three groups, which had observed mortality rates that ranged from 4.48% to 20.11%.
The overall risk-adjusted mortality rate for patients in hospitals with annual pediatric volumes <100 was 8.26%, which was significantly higher (P < .05) than the statewide rate of 6.75% (see the total group). The overall risk-adjusted mortality rate for patients in hospitals with volumes of 100 or more was 5.95%, which was significantly lower (P < .05) than the statewide rate. The higher hospital volume group had the lowest risk-adjusted mortality rate for each complexity category, and the respective differences between volume groups for cases in the four complexity categories were 4.11%, 3.61%, 2.99%, and 1.07%. Perhaps surprisingly, the smallest difference occurred for the highest risk patients.
The percentage of all patients undergoing procedures by the lower volume surgeon group (see bottom of Table 3) is 34% for the lowest complexity procedures, 34% for the highest complexity procedures, and 26% for each of the two middle complexity categories. Thus, there is not a strong tendency for the more complex cases to be assigned to the higher volume surgeons.
Table 3 also shows that the risk-adjusted mortality rate for patients of the lower volume surgeons was 8.77%, which was significantly higher than the statewide mortality rate (P < .05), and that the risk-adjusted rate for patients of the higher volume surgeons was 5.90%, which was significantly lower than the statewide mortality rate (P < .05). For each complexity category, patients of the lower volume surgeons had higher risk-adjusted mortality rates than patients of the higher volume surgeons. Also, the largest difference in risk-adjusted rates was for the lowest complexity category (10.23% vs 5.04%), and the next largest difference was for the second highest complexity category (8.78% vs 5.79%).
Table 4 presents, for each intersection of the two hospital volume groups (<100, ≥100) and the two surgeon volume groups (<75, ≥75), the number of patients undergoing pediatric cardiac procedures between 1992 and 1995 and the risk-adjusted mortality rate. Also included are the ranges for the number of hospitals and number of surgeons with annual volumes in each volume group. As indicated, the risk-adjusted mortality rate ranged from 5.45% for patients undergoing surgery in hospitals with annual volumes of at least 100 performed by surgeons with annual volumes of at least 75 to 8.94% for patients undergoing surgery in hospitals with annual volumes of <100 performed by surgeons with annual volumes of <75. These two rates were significantly lower and significantly higher, respectively, than the statewide rate (P < .05), and significantly different from one another (P < .05). Risk-adjusted mortality rates for combinations of low-volume surgeons and high-volume hospitals, and of high-volume surgeons and low-volume hospitals, were 8.47% and 7.45%, respectively.
The purpose of this study was to examine the relationship between the in-hospital mortality rate for pediatric cardiac procedures and the annual hospital and surgeon volumes for these procedures. However, there are many different pediatric cardiac procedures, each with its own complexity and mortality rate. Consequently, the strategy used to analyze the data was the one used by Jenkins et al,1which was to combine procedures into categories on the basis of their complexities/hospital mortality rates, and then to use these groups in addition to patient demographics and risk factors to risk-adjust mortality rates for different ranges of provider volumes. Two advantages of this study relative to the study by Jenkins et al1 are: 1) a clinical database was available to us for performing the risk-adjustments rather than having to rely on administrative data, and 2) annual surgeon volumes were available for testing the surgeon volume-mortality relationship in addition to the hospital volume-mortality relationship.
In observing the procedure categories, it is important to note that patient selection and the presence of important comorbidities undoubtedly account for differences in mortality between closely related procedures. As an example, patients with severe pulmonary stenosis associated with a dysplastic pulmonary valve are likely to undergo an open pulmonary valvotomy, whereas patients with pulmonary atresia, intact ventricular septum, and hypoplastic right ventricle may be more heavily represented in the group having a closed pulmonary valvotomy. Clearly, the hazard of death is greater in the latter group; consequently, the difference in mortality between open and closed pulmonary valvotomy is likely to be a function of the risk factors associated with the disease state rather than the technical features of the surgery.
Findings of the study were that annual hospital volume and annual surgeon volume were both significantly related to inpatient mortality rates, even after controlling for patient age and several clinical risk factors in addition to procedure complexity. The maximal differentiation in mortality rates between high- and low-volume providers was at 100 procedures annually for hospitals and 75 procedures annually for surgeons. However, in general, higher hospital volumes and higher surgeon volumes were associated with lower risk-adjusted mortality rates across all procedure volumes, so any decision to recommend minimum hospital or surgeon volumes for pediatric cardiac procedures should take into account this fact and practical considerations regarding the geographical distribution of pediatric centers.
It is also important to note that annual hospital volume and annual surgeon volume were independently related to risk-adjusted mortality rate, meaning that even if one of the two volume measures was high, mortality rates were lower when the other measure was high than when it was low. Patients undergoing procedures performed by higher volume surgeons (75 or more procedures annually) in higher volume hospitals (100 or more procedures annually) had the lowest risk-adjusted mortality rate of all groups (5.45%), which was significantly lower than the risk-adjusted mortality rate for patients undergoing procedures in hospitals with annual volumes <100 performed by surgeons with annual volumes <75 (8.94%).
Another important finding was that patients in higher volume hospitals and patients with higher volume surgeons experienced lower risk-adjusted mortality rates than patients with lower volume providers regardless of the complexity category of the procedure they had undergone. This was somewhat surprising because it had been hypothesized that mortality rate differences may be limited to higher risk procedures.
A caveat regarding the study is that, like the study by Jenkins et al,1 it was necessary to group procedures into complexity categories to perform the analyses because there were too many procedures (with too low volumes) to consider them separately. Thus, there is a possibility that lower volume providers performed a higher proportion of the more complex procedures within categories than higher volume providers, and were unfairly assessed. However, a perusal of the relative frequencies of procedures within categories by volume groups does not substantiate this concern.
Another potential bias could be introduced if there was a tendency for the higher volume providers to code patient risk factors more conscientiously than the lower volume providers, or if higher volume providers are coding risk factors that are not legitimate to have lower risk-adjusted mortality rates. However, no major coding problems have been found in data quality audits of pediatric forms conducted by the New York State Department of Health.
An important caveat regarding the interpretation of these results is that, as in other volume-mortality studies, it is difficult, if not impossible, to determine the direction of causality in the association between higher provider volumes and lower risk-adjusted mortality rates. One possible explanation is practice makes perfect, whereby a hospital or surgeon achieves better results by honing its skills on numerous patients; another possible explanation is the selective referral hypothesis, whereby the higher volume providers perform higher volumes because they are known to have superior outcomes. The former hypothesis suggests that it would be wise to encourage a system with as many high-volume providers and as few low-volume providers as possible, including the creation of high-volume providers by increasing the volumes of current low-volume providers. The latter hypothesis would suggest that attempts to alter the current distribution by increasing the volume of low-volume providers would probably not result in substantially better outcomes. For a thorough discussion of these alternative explanations, the reader is referred to the seminal book by Luft et al.9
The findings of this study have served to reinforce the findings of Jenkins et al1 by confirming the hospital volume-mortality relationship in another setting and by establishing that risk-adjusted mortality is also related to annual surgeon volume. Studies in other regions are recommended to help refine the identification of optimal provider volumes and to reexamine the volume-mortality relationship for various procedure complexity categories.
In New York, where access to clinical data is available, future initiatives will include attempts to assess provider quality of pediatric cardiac surgery by calculating provider-specific risk-adjusted mortality rates. Also, we will attempt to evaluate other important but more complex measures of quality such as medical versus surgical treatment of patients, and the timing and choice of procedures to be performed.
- Received May 30, 1997.
- Accepted October 16, 1997.
Reprint requests to (E.L.H.) University at Albany, State University of New York, Department of Health Policy, Management, and Behavior, One University Place, Rensselaer, NY 12144–3456.
- CSRS =
- New York Cardiac Surgery Reporting System •
- CAC =
- Cardiac Advisory Committee
- Jenkins KJ,
- Newburger JW,
- Lock JE,
- Davis RB,
- Coffman GA,
- Iezzoni LI
- Luft HS
- ↵Luft HS, Garnick DW, Mark DH, McPhee SJ. Hospital Volume, Physician Volume, and Patient Outcomes. Ann Arbor, MI: Health Administration Press; 1990
- ↵Breslow NE, Day NE. Statistical Methods in Cancer Research. New York, NY: Oxford University Press; 1991
- Copyright © 1998 American Academy of Pediatrics