Objective. To describe potential patient safety events for hospitalized children, examine associated factors, and explore impacts of safety events.
Methods. The newly released Patient Safety Indicators (PSIs), developed by researchers at the Agency for Healthcare Research and Quality to identify potential in-hospital patient safety problems using administrative data, were applied to hospital discharge data. All 5.7 million discharge records for children younger than 19 years from 27 states in the 2000 Healthcare Cost and Utilization Project were analyzed for PSI events. Prevalence of PSI events and associations with patient-level and hospital-level characteristics were examined. Multivariate regression adjusting for patient severity of illness was used to estimate impacts of safety events in terms of excess length of stay, charges, and in-hospital mortality.
Results. The prevalence of pediatric patient safety events is significant. PSI events occurred more frequently in the very young and those on Medicaid insurance, some of the most vulnerable hospitalized children. Regression analysis found that almost all PSIs are associated with significant and substantial increases in length of stay, charges, and in-hospital death. Using the estimates derived here and the actual number of cases identified in the 2000 data, we estimate that patient safety events incurred >$1 billion in excess charges for children alone in 2000.
Conclusions. Patient safety problems for hospitalized children occur frequently and with substantial impacts to our health care industry. Unmeasurable by this study are the additional “costs” and “burdens” of safety events that our patients are forced to handle. Additional work to describe and quantify better these outcomes in addition to ones measured here can help solidify the “business case” for patient safety efforts.
The Institute of Medicine reports To Err Is Human and Crossing the Quality Chasm: A New Health System for the 21st Century shined a national spotlight on preventable medical errors.1,2 Since the release of the reports, numerous entities have begun to actively tackle the patient safety problem from federal and state governments to private institutions and organizations.3–9 The Federal agency for patient safety, the Agency for Healthcare Research and Quality (AHRQ) of the Department of Health and Human Services, launched a $50 million initiative in patient safety research in 2001.3 As part of the efforts to address patient safety concerns, the AHRQ recently created and publicly released the Patient Safety Indicators (PSIs).10,11 Similar to the predecessor, AHRQ’s Healthcare Cost and Utilization Project Quality Indicators, the PSIs rely on administrative data and, as such, are indicators, not definitive measures, of patient safety concerns.12 The intent of these indicators is to provide a useful screening tool to identify processes of care that warrant additional institution-level evaluation from the patient safety perspective. The PSIs would enable institutions to identify a manageable number of medical records for closer scrutiny.
There are many unique issues that surround children’s health care, namely related to the 4 “Ds”: developmental change, dependence on adults, different disease epidemiology, and demographic characteristics.13 Each of these Ds directly relates to unique safety issues for children. For examples, developmental change can encompass issues such as the unique susceptibility of neonates to infections and the importance of other newborn issues such as kernicterus and detecting evolving cardiac abnormalities in the first few days of life. Comparably, dependence on adults relates to safety because children do not usually administer their own medications, have the understanding to question their own care, or typically serve as primary historians for medical complaints. Different disease epidemiology and demographics relate to safety in that children have unique illnesses that predispose them to unique safety events as compared with adults, as with birth trauma and screening for metabolic abnormalities, as well as high rates of living in poverty as compared with 18- to 64-year-old adults.14 Knowing this, we recently reported on patient safety event rates in 1997 for hospitalized children using a preliminary set of the PSIs to provide the first broad insight into the number of safety events for inpatient pediatrics.15 With the finalization of the PSIs, we take this important opportunity to provide a national picture of inpatient pediatric patient safety events for the most recent year of pediatric national data, 2000, and for the first time examine the impact of these events in terms of excess length of stay (LOS), charges, and risk of in-hospital mortality. This new patient impact analysis is key to helping institutions and policy makers understand the importance of patient safety for children. As institutions struggle to identify the “business case” for safety efforts, these data can serve as a starting point for this case.
A complete description of the development and testing of the PSI algorithms is available via the AHRQ.10 The development of the PSIs was based on the definition of patient safety in the To Err Is Human report, which is freedom from accidental injury as a result of medical care or medical errors. Medical errors were further defined, using the definition from the federal response to the Institute of Medicine report Doing What Counts as “the failure of a planned action to be completed as intended or use of a wrong plan to achieve an aim. Errors can include problems in practice, products, procedures, and systems.”4 This definition excludes acts that did not achieve their desired outcomes, as long as they were not the result of negligence, outcomes caused by the intrinsic properties of the underlying illness or additional patient comorbidities, and outcomes known to be risks of specific procedures. These exclusions distinguish the PSIs from other indicator systems designed to detect complications of care.16 Table 1 describes the characteristics of the individual PSI algorithms.
Analysis of PSI Events
Our analysis used 5.7 million hospital discharges for children 18 years of age or younger from 27 states in the 2000 Healthcare Cost and Utilization Project State Inpatient Databases.17 Patient-level variables included age (0–30 days, 31–365 days, 1–4 years, 5–9 years, 10–14 years, and 15–18 years), gender, and primary expected payer (Medicare, private, Medicaid, other, or uninsured). To account for severity of illness at admission for adjustment into regressions estimating impact of experiencing a PSI event, we used the Elixhauser method built into the PSI software to define 30 comorbid diseases that pertain to risk of death and expected resource consumption.18 The comorbid diseases were summarized into indices in a 2-step process. We first estimated the regression coefficients of the 30 comorbidities on LOS, charges, and in-hospital mortality, respectively, using all 5.7 million pediatric discharges and controlling for age, gender, insurance status, and diagnosis-related group (DRG) fixed effects. In the second step, the significant positive coefficients, which indicated independent effects of the presence of comorbid diseases on LOS, charges, and probability of death, were used as relative weights to summarize the comorbidities into indices. Preliminary analysis showed that the resulting 3 comorbidity indices were highly correlated (correlation coefficients >0.96). We recognized that the presence of a comorbid disease could shorten LOS and reduce charges by leading to an early death, resulting in negative coefficients in LOS and charge regressions contrary to the objective of the measures. On the basis of these preliminary analyses and considerations, we followed Charlson et al,19 Romano et al,20–22 and Ghali et al23 in choosing the index based on excess probability of death associated with comorbidities for our analysis. Each discharge record was then assigned an index value estimating overall patient severity of illness as a summation of these positive comorbidity associations with risk of in-hospital death.
Hospital-level variables were identified from the literature and were obtained either from the state databases or by linkage to the American Hospital Association’s Annual Survey of Hospitals Database, Fiscal Year 2000.24–28 These variables were hospital ownership, teaching status (identified by continuous variable of resident/bed ratio), nursing expertise (number of full-time and part-time registered nurses divided by number of full-time and part-time registered nurses plus full-time and part-time licensed practicing nurses), hospital location, total number of hospital beds, number of pediatric discharges, number of pediatric beds, and a technologic sophistication index (total number of the following items of equipment or facilities: open-heart surgery, organ transplantation capability, positron emission tomography, level 1 trauma center, and neonatal intensive care unit). This last hospital-level variable of technologic sophistication index was modified from the literature to reflect current health care options.24,25
To capture potential biases in detecting PSI events on the basis of coding practices, we also examined the effect of the number of diagnoses codes and procedure codes typically used by an institution (stratified into less than versus greater than or equal to the median number of codes for all institutions in the database). To examine variations among hospitals as a result of the types of patients cared for, we included 2 variables previously used in the literature to capture patient-level severity of illness: percentage of hospital beds in intensive care units and the proportion of inpatient surgical procedures to annual admissions.24,25
First, we examined the rates for each of the PSIs. Second, we examined the relationship between PSI events and patient and hospital characteristics using bivariate and regression analyses, including all patient and hospital characteristics explained in the preceding section. Last, we conducted bivariate and multivariate regression analyses to examine the impact of experiencing a PSI event in terms of patient LOS, charges, and risk of in-hospital mortality, again including all patient and hospital characteristics identified.
Nonparametric comparisons of medians for LOS and total charges were done using the Wilcoxon rank-sum tests because these data were not normally distributed. Comparisons of in-hospital mortality and bivariate associations between patient-level and hospital-level characteristics and PSI events were completed using χ2 tests. Multivariate logistic regression was used to examine associations with experiencing a PSI event controlling for other patient-level and/or hospital-level factors. We ran linear and logistic regressions to estimate excess outcomes attributable to patient safety events using SAS PROC GLM to adjust for hospital and patient variables. Multivariate logistic regression analyses yielded odds ratios (OR) and 95% confidence intervals (CI) for experiencing in-hospital death on the basis of whether a PSI event occurred. Multivariate linear regression analyses yielded excess days or dollars attributable to experiencing a PSI event. For these multivariate logistic regression analysis, the following variable categories were used as reference values: age 1 to 4 years, female gender, private insurance, not-for-profit ownership, nonteaching hospital status defined from resident/bed ratio, higher nursing expertise, urban hospital location, lower number of hospital beds, lower number of pediatric discharges, lower number of diagnosis or procedures codes recorded by institution, low percentage of beds in intensive care units, and low percentage of inpatient surgical volume. The technologic sophistication variable was coded as a numeric variable with each institution receiving 1 point for each of the 5 technologies present to a maximal score of 5. Analyses were done using SAS with a significance level of P < .05.29
Overall Rate of PSI Events for Children in 2000
Table 2 presents the rate of PSI events by PSI group for the 5.7 million pediatric discharges in the data for 2000. The event rates, per 10 000 pediatric discharges, range from <1 (in-hospital postoperative hip fracture and transfusion reactions) to high values of 68 (birth trauma), 103 (postoperative sepsis), 703 (failure to rescue), 1072 (obstetric trauma—vaginal without instrumentation), and 2152 (obstetric trauma—vaginal with instrumentation). In all of these cases, the patient who experienced the event was younger than 18 years, meaning that these obstetric data reflect events experienced by teenaged mothers. The actual number of cases for PSI events is substantial almost across the board with 16 of the 20 PSIs having raw rates of >100 pediatric cases across the country in 2000.
Factors Associated With PSI Events
Bivariate associations between patient-level and hospital-level characteristics and PSI events were examined for all PSI groups individually. Significant associations varied by type of PSI event. Logistic regression analysis examined these associations while controlling for other patient-level and hospital-level factors. Results of these analyses were the finding that the youngest hospitalized children (aged 0–30 days and aged 31–365 days) were consistently and significantly more likely to experience many PSI events than older children. For infants 0 to 30 days of age at discharge, compared with children 1 to 4 years of age, the OR of experiencing failure to rescue was 3.9 (95% CI: 3.4–4.4), infection as a result of medical care was 3.8 (95% CI: 3.2–4.4), postoperative hemorrhage/hematoma was 6.6 (95% CI: 4.6–9.2), postoperative pulmonary embolism/deep venous thrombosis was 1.6 (95% CI: 1.1–2.4), postoperative sepsis was 5.8 (95% CI: 3.7–9.0), and technical difficulty with medical care was 3.9 (95% CI: 3.2–4.8). For infants 31 to 365 days of age at discharge, compared with children 1 to 4 years of age, the OR of experiencing failure to rescue was 1.4 (95% CI: 1.1–1.6), infection as a result of medical care was 1.5 (95% CI: 1.3–1.6), postoperative hemorrhage/hematoma was 1.6 (95% CI: 1.1–2.3), postoperative respiratory failure was 1.7 (95% CI: 1.2–2.5), postoperative pulmonary embolism/deep venous thrombosis was 1.4 (95% CI: 1.1–2.0), postoperative sepsis was 2.0 (95% CI: 1.4–2.7), and technical difficulty with medical care was 1.3 (95% CI: 1.1–1.6).
Additional consistent findings from these regression examinations of associations with PSI events include that a child who had Medicaid primary insurance was significantly more likely, compared with private primary insurance, to experience death in low-mortality DRGs (OR: 1.4; 95% CI: 1.1–1.7), infection as a result of medical care (OR: 1.2; 95% CI: 1.1–1.4), postoperative respiratory failure (OR: 1.8; 95% CI: 1.4–2.3), and postoperative sepsis (OR: 1.8; 95% CI: 1.4–2.3). Birth trauma was significantly less likely to occur at teaching institutions compared with institutions without any residents employed (OR for institutions with less than median number of residents per bed compared with national average: 0.7; 95% CI: 0.6–0.7; OR for institutions with more than median number of residents per bed compared with national average: 0.6; 95% CI: 0.5–0.6). Birth trauma was also more likely to occur at institutions with lower nursing expertise compared with greater expertise (OR: 1.3; 95% CI: 1.2–1.3).
Last, almost all PSI events were associated with institutions that had greater technologic sophistication, possibly reflecting imperfect adjustment for patient severity of illness in our analyses or other factors related to greater technologic sophistication, such as linkage to academic medical centers, linkage to areas with lower socioeconomic status, and potentially more ill patients as a result of decreased access to care.
Impact of PSI Events on LOS, Charges, and Risk of In-Hospital Mortality
Bivariate associations between PSI events and LOS, charges, and in-hospital mortality revealed that, within clinical reason, experiencing a PSI event was significantly associated with increases in LOS, charges, and in-hospital mortality. The exceptions that made clinical sense include cases such as obstetric and birth trauma, which were not associated with increased LOS, charges, or in-hospital death but undoubtedly were associated with other untoward issues and outcomes for the patients involved.
Multivariate regression analyses to estimate the impact of experiencing a PSI event controlling for patient and hospital factors including patient severity of illness showed substantial impacts in all 3 measurable outcomes, namely LOS, charges, and in-hospital mortality (Table 3). In terms of LOS, nearly all PSI events produced significant increases in LOS ranging from relatively negligible amounts (0.2 days for birth trauma) to pronounced increases of 12 days for iatrogenic pneumothorax (±0.4 days), 16 days for postoperative physiologic and metabolic derangements (±1 day), 18 days for decubitus ulcers (±0.6 days), 26 days for postoperative sepsis (±0.6 days), and 30 days for infection as a result of medical care (±0.2 days). As expected, the impact in terms of charges closely mirrored the LOS impact and was substantial. All PSI events with significant increases in charges had excess charges in the range of $30 000 to $140 000 per each discharge experiencing a PSI event. Last, with respect to increased likelihood of in-hospital mortality, the majority of PSI events were associated with significant increases in in-hospital mortality compared with similar patients who did not experience a PSI event. These increases in in-hospital mortality were even apparent for PSI birth trauma, with an OR of 1.3 compared with infants who did not have trauma incurred at birth (95% CI: 1.1–1.6). The most substantial increases in in-hospital mortality risk were for iatrogenic pneumothorax (OR: 7.5; 95% CI: 5.0–11.4), postoperative sepsis (OR: 11; 95% CI: 6.9–17.5), postoperative physiologic/metabolic derangements (OR: 45.8; 95% CI: 21.7–93.0), and postoperative respiratory failure (OR: 76.6; 95% CI: 51.6–113.6).
Our analyses using the newly released and carefully developed PSIs show that patient safety events for hospitalized children in 2000 occurred in high numbers and that these events have significant associations with increased LOS, total charges, and risk of in-hospital mortality even after adjusting for patient severity of illness. These data represent the majority of states in the United States and encompass a near census of all pediatric hospitalizations in 2000. The magnitude of impact of safety events is easily extrapolated by taking our impact estimates per discharge and multiplying by the actual number of cases identified in these data alone to be >$1 billion dollars in excess charges for pediatric patients. Comparably, if one assumes that all deaths among patients who experience a PSI event are attributable to the PSI event, then the records in our analysis alone account for 4483 deaths among hospitalized children as a result of patient safety events. More so, we need to keep in mind that our PSI algorithms can detect only a small portion of the types of patient safety events that actually happen in hospitals. This means that the costs of patients’ safety events for children, in terms of both charges and deaths, is likely substantially greater than these estimates.
An equally important finding of this work is the strong association between experiencing a PSI event and various characteristics that clearly identify particularly vulnerable children, namely the very young and those on Medicaid insurance. Although our previous work also identified an association between PSI events and black race/ethnicity, our current data did not permit this important look into disparities in care. Also, consistent with our preliminary study, this analysis confirms a high rate of birth trauma and that these events tend to track with what could be described as likely smaller institutions with less resource of expertise.15
There are some inherent limitations in using the PSIs that rely on hospital administrative data. This is their strength and their most significant limitation. Electronic administrative data are readily available and inexpensive. However, these data provide limited clinical information and have known problems in coding accuracy, problems in coding variation, limited ability to risk adjust, and limited insight into timing of events.24–27 Given the careful work put into the PSIs, however, to specifically tailor inclusion and exclusion criteria to maximize the identification of events that groups of expert clinicians believed should “never happen,” the PSIs overcome a substantial proportion of this limitation on the basis of the data and provide real insight into the impact of patient safety events. A second limitation of the PSIs is the infrequency of the events that are, almost by definition, relatively rare. This issue underscores the appropriate use of the PSIs as institutional case-finding tools aimed at internal quality improvement as opposed to use for directly comparing individual institutions, especially in public reports that identify individual hospitals. In this study, we were able to overcome this limitation by using comprehensive pediatric discharge data from 27 states. Third, despite best efforts, the PSIs are likely imperfect in truly identifying only cases of compromised patient safety. There is considerable debate in the literature about whether patient safety events should focus on medical errors or medical injuries.30,31 It is most likely that events identified by the PSIs encompass the full spectrum here and capture medical errors without injury, medical errors with injury, and injury without medical errors. Using the PSIs for institutional quality improvement or to provide broad national snapshots as with this work minimizes the impact of this inherent limitation. Last, the PSIs are not a comprehensive catalog of all inpatient medical errors but instead a conservative list of potential errors that are amenable to detection with administrative data that can be a starting point for addressing the patient safety problem.
In summary, this first analysis of the national picture of the impact of patient safety events for children highlights the enormous burden of this problem. Although many organizations and institutions convene to discuss the “business case for patient safety,” this analysis serves as initial ground work to support that case by teasing out the impact on our health care industry and our patients.32 It cannot go without saying, however, that for our individual patients, be they large or small, the case for focus on patient safety can and should be made with an “n of 1” study, not requiring a link to increased LOS, charges, or in-hospital mortality. Furthermore, these “n of 1” studies should be studies of near-miss patient safety events as opposed to actual patient safety events, particularly with outcomes such as mortality. These important cases, ones with near-miss patient safety events, need to be recognized and valued into our traditional accounting mechanisms along with LOS, charges, and in-hospital mortality when we consider the “why” behind patient safety initiatives.
This research was supported by intramural research funding from the Agency for Healthcare Research and Quality.
Dr Miller completed this analysis while employed as the Acting Director of the Center for Quality Improvement and Patient Safety at the Agency for Healthcare Research and Quality. The authors of this article are responsible for its contents, including any clinical or treatment recommendations. No statement in this article should be construed as an official position of the Agency for Healthcare Research and Quality or the US Department of Health and Human Services.
- Received September 9, 2003.
- Accepted January 26, 2004.
- Address correspondence to Marlene R. Miller, MD, Johns Hopkins Children’s Center, CMSC 2-125, 600 N Wolfe St, Baltimore, MD 21287. E-mail:
- ↵Institute of Medicine. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999
- ↵Committee on Quality of Health Care in America, Institute of Medicine (ed). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001
- ↵Available at: www.ahrq.gov
- ↵Available at: www.quic.gov
- Available at: www.jcaho.org
- Available at: www.qualityforum.org
- Available at: www.npsf.org
- Available at: www.leapfroggroup.org
- ↵Available at: www.ismp.org
- ↵Agency for Healthcare Research and Quality. Patient safety indicators, version 2.1 (revision 1). Available at: www.qualityindicators.ahrq.gov/data/hcup/psi.htm. Accessed April 21, 2004
- ↵Miller MR, Battles J, Dougherty D. AHRQ’s pediatric patient safety agenda. Ambul Pediatr Assoc News.2003;43 :16
- ↵Forrest CF, Shipman SA, Dougherty D, Miller MR. Outcomes research in pediatric settings: recent trends and future directions. Pediatrics.2003;111 :171– 178
- ↵Fried VM, Prager K, MacKay AP, Xia H. Chartbook on Trends in the Health of Americans. Health, United States, 2003. Hyattsville, MD: National Center for Health Statistics; 2003
- ↵Miller MR, Elixhauser A, Zhan C. Patient safety events during pediatric hospitalizations. Pediatrics.2003;111 :1358– 1366
- ↵Agency for Healthcare Research and Quality. Guide to patient safety indicators. Available at: www.qualityindicators.ahrq.gov/downloads/pub/psi/psi_guide_rev1.pdf. Accessed April 21, 2004
- ↵Agency for Healthcare Research and Quality. HCUP kids’ inpatient database (KIDS). Available at: www.ahrq.gov/data/hcup/hcupkid.htm. Accessed April 21, 2004
- ↵Silber JH, Williams SV, Krakauer H, Schwartz JS. Hospital and patient characteristics associated with death after surgery. A study of adverse occurrence and failure to rescue. Med Care.1992;30 :615– 629
- ↵SAS Institute. SAS/SAT User’s Guide. Version 6, 4th ed. Cary, NC: SAS Institute; 1989
- ↵Layde P, Maas L, Teret S, et al. Patient safety efforts should focus on medical injuries. JAMA.2002;278 :1993– 1997
- ↵McNutt R, Abrams R, Aron D. Patient safety efforts should focus on medical errors. JAMA.2002;278 :1997– 2001
- ↵Agency for Healthcare Research and Quality. Business case for patient safety symposium. Available at: www.ahrq.gov/qual/ptsaftape.htm. Accessed April 21, 2004
- Copyright © 2004 by the American Academy of Pediatrics