



* University of Michigan Health System, Ann Arbor, Michigan
National Association of Children's Hospitals and Related Institutions, Alexandria, Virginia
Agency for Healthcare Research and Quality, Rockville, Maryland
|| Children's Hospital of Wisconsin, Wauwatosa, Wisconsin
| ABSTRACT |
|---|
|
|
|---|
Methods. The Agency for Healthcare Research and Quality PSI algorithms (version 2.1, revision 1) were applied to children's hospital administrative data (1.92 million discharges) from the NACHRI Aggregate Case Mix Comparative Database for 19992002. Rates were measured for the following events: complications of anesthesia, death in low-mortality diagnosis-related groups (DRGs), decubitus ulcer, failure to rescue (ie, death resulting from a complication, rather than the primary diagnosis), foreign body left in during a procedure, iatrogenic pneumothorax, infection attributable to medical care (ie, infections related to surgery or device placement), postoperative hemorrhage or hematoma, postoperative pulmonary embolism or venous thrombosis, postoperative wound dehiscence, and accidental puncture/laceration.
Results. Across the 4 years of data, the mean risk-adjusted rates of PSI events ranged from 0.01% (0.1 event per 1000 discharges) for a foreign body left in during a procedure to 14.0% (140 events per 1000 discharges) for failure to rescue. Review of International Classification of Diseases, Ninth Revision, Clinical Modification codes associated with each PSI category showed that the failure to rescue and death in low-mortality DRG indicators involved very complex cases and did not predict preventable events in the majority of cases. The PSI for infection attributable to medical care appeared to be accurate the majority of the time. Incident risk-adjusted rates of infections attributable to medical care averaged 0.35% (3.5 events per 1000 discharges) and varied up to fivefold from the lowest rate to the highest rate. The highest rates were up to 1.8 times the average.
Conclusions. PSIs derived from administrative data are indicators of patient safety concerns and can be relevant as screening tools for children's hospitals; however, cases identified by these indicators do not always represent preventable events. Some, such as a foreign body left in during a procedure, iatrogenic pneumothorax, infection attributable to medical care, decubitus ulcer, and venous thrombosis, seem to be appropriate for pediatric care and may be directly amenable to system changes. Evidence-based practices regarding those particular indicators that have been reported in the adult literature need to be investigated in the pediatric population. In their present form, 2 of the indicators, namely, failure to rescue and death in low-mortality DRGs, are inaccurate for the pediatric population, do not represent preventable errors in the majority of pediatric cases, and should not be used to estimate quality of care or preventable deaths in children's hospitals. The PSIs can assist institutions in prioritizing chart review-based investigations; if clusters of validated events emerge in reviews, then improvement activities can be initiated. Large aggregate databases, such as the NACHRI Case Mix Database, can help establish mean rates of potential pediatric events, giving children's hospitals a context within which to examine their own data.
Key Words: patient safety indicators Agency for Healthcare Research and Quality medical error National Association of Children's Hospitals and Related Institutions
Abbreviations: AHRQ, Agency for Healthcare Research and Quality PSI, patient safety indicator NACHRI, National Association of Children's Hospitals and Related Institutions DRG, diagnosis-related group ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification HCUP, Healthcare Cost and Utilization Project KID, Kids' Inpatient Database
In 1999, the Institute of Medicine defined patient safety as "freedom from accidental injury because of medical care or medical errors" and charged hospitals to create a culture of safety in their institutions.1 Studies of pediatric medication errors have been published2; however, there is a paucity of data on pediatric patient injuries in children's hospitals.
The Agency for Healthcare Research and Quality (AHRQ) was charged by Congress to promote patient safety and to reduce medical errors through research and collaborative partnerships. One result of this mandate was the development of the AHRQ patient safety indicators (PSIs), ie, software that can identify potential adverse events from International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes in hospital inpatient administrative data. The University of California, San Francisco-Stanford University Evidence-Based Practice Center, under a contract with AHRQ, developed the PSIs in a 5-stage process.3 First, the literature was reviewed to identify a list of candidate indicators. Second, clinical panels evaluated the face validity or "sensibility" of the indicators, as judged by practicing physicians. Third, ICD-9-CM coding experts were consulted to confirm that the definition of each indicator reflected the intended clinical situation. Fourth, an empirical analysis of the chosen indicators was conducted with the Healthcare Cost and Utilization Project (HCUP) data.4 Fifth, PSI software and documentation were produced for public release by AHRQ. Since its initial development, this software has undergone several modifications, including refinement of the indicator definitions. Therefore, in comparisons of PSIs, it is very important to note whether the versions of the PSIs are the same.
The indicators were risk adjusted on the basis of patient age, patient gender, and AHRQ comorbidity groups, as determined with AHRQ comorbidity software, which uses ICD-9-CM codes to assign up to 30 comorbidity groups (such as hypertension and diabetes mellitus) to patient records.5
Extensive work was published in 2003 by Miller et al,6 describing pediatric patient safety with the use of the PSIs (ß version) and administrative data from the 1997 HCUP.4 That study identified the high rates of birth trauma among newborns, with a rate of
1.5 cases per 100 births. Multivariate logistic regression analyses with the PSIs showed that patients with higher acuity exhibited higher rates of events. The study also demonstrated significant increases in length of stay and cost per case in relation to PSIs. The authors concluded "... that hospitalized children experience significant numbers of patient safety events, that these event rates are comparable to hospitalized adults, and that attention clearly needs to be paid to the unique event of childbirth."6(p1365) A follow-up study by Miller and Zhan7 with HCUP 2000 data confirmed the enormous burden of medical errors in pediatric care.
The HCUP database used by Miller et al6 is different from the National Association of Children's Hospitals and Related Institutions (NACHRI) database (Table 1). Specifically, the HCUP 2000 Kids' Inpatient Database (KID) includes a sample of pediatric discharges from community hospitals (short-term, nonfederal, general and specialty hospitals) drawn from 27 state inpatient databases on children
20 years of age (www.ahrq.gov/data/hcup). In comparison, the NACHRI Case Mix Database is a children's hospital-specific database (freestanding acute care and specialty children's hospitals and acute care children's hospitals located within larger institutions) that contains 100% of pediatric discharges from those institutions, excluding normal newborn and obstetric cases. Because of the differences in the databases and their potential effects on PSI rates, we considered it worthwhile to investigate the PSI algorithms specifically from a children's hospital perspective with the NACHRI Case Mix Database. Our objectives were (1) to apply the AHRQ PSI algorithms to the NACHRI Case Mix Database for 1999-2002, (2) to investigate PSI rates specifically in children's hospitals (with the major utility of the trend being to increase confidence in the data, because of the uniformity of the rates with time for each PSI), (3) to investigate any inadequacies of PSIs in relation to pediatric diagnoses, and (4) to express the data in such a way that individual children's hospitals could use the PSIs determined to be pediatric appropriate for comparison with their own data. In addition, we wished to use the data to set priorities for ongoing clinical investigation and to propose interventions.
|
| METHODS |
|---|
|
|
|---|
The AHRQ PSI software (version 2.1, revision 1) was used to analyze the data, and the resulting output (numerator, denominator, mean, observed rate, risk-adjusted rate, and smoothed rate) for each of the 11 PSIs with relevant pediatric volume was coupled with additional analyses with SPSS statistical software.8 Rates were measured for the following events: complications of anesthesia, death in a low-mortality diagnosis-related group (DRG), decubitus ulcer, failure to rescue (ie, death resulting from a complication rather than the primary diagnosis), foreign body left in during a procedure, iatrogenic pneumothorax, infection attributable to medical care (ie, infections related to surgery or device placement), postoperative hemorrhage or hematoma, postoperative pulmonary embolism or venous thrombosis, postoperative wound dehiscence, and accidental puncture/laceration.
The PSI software is very specific in defining the numerator and denominator for each indicator (see figure legends) and strictly limits the discharges that are considered for each patient safety event. For example, patients with paraplegia and quadriplegia are exempted from the decubitus measure, and patients with cancer and/or immunosuppression are excluded from the infection rate measure. The indicators are risk-adjusted on the basis of patient age, patient gender, and AHRQ comorbidity groups (ICD-9-CM codes are used to assign up to 30 comorbidity groups, such as hypertension and diabetes mellitus, to a patient record5). Depending on the covariate weight of these comorbidity groups, the rate of PSIs is adjusted up or down for each group analyzed. In this study, for example, the observed rate of decubiti was adjusted upward during the risk adjustment, in comparison with adult patients, who, because of diagnoses of vascular compromise and diabetes, are considered at higher risk for decubiti than are children.
The ICD-9-CM codes for each discharge identified as having a patient safety event were reviewed. This investigation of all of the principal and secondary diagnoses associated with each case was used to characterize complexities that influenced the indicators and allowed us to ascertain whether the patient safety events were potentially preventable.
| RESULTS |
|---|
|
|
|---|
Table 2 presents the numerator, denominator and observed and risk-adjusted rates of event per 1000 discharges for each PSI for the years 1999-2002. Because of growth in the database, the number of hospitals in the database is different for each year; however, we did not exclude hospitals for the aggregate analyses (the only exception being the follow-up analysis of infections attributable to medical care, for which we used the 43 hospitals that submitted data for the entire 4-year period).
|
|
|
With the high rate of infection incidents (>1000 incidents per year) allowing additional statistical analysis among hospitals, an analysis of ICD-9-CM codes indicating that many of these cases were potentially preventable, and national evidence-based literature regarding the prevention of device infection being available, a special examination of rates of infections attributable to medical care was performed for the 43 hospitals that provided data for each of the 4 years studied. A combined 4-year infection rate was calculated for each hospital and the overall sample.
The aggregated risk-adjusted rate for the 43 hospitals was 3.54 events per 1000 discharges (ie, 4469 cases with events in a total population of 1 252 768). As Fig 12 illustrates, 11 hospitals had rates that were significantly above the mean (P values ranging between .0001 and .038), and 19 hospitals had rates that were significantly below the mean (P values between .0001 and .046).
|
| DISCUSSION |
|---|
|
|
|---|
The NACHRI data demonstrated PSI rates that were relatively consistent for the 4 years, with aggregate means that did not vary significantly from year to year. Examination of diagnosis and procedure codes associated with each case led us to think that many of these pediatric patient safety events were preventable, although the complexities and characteristics of these pediatric cases might be different from those for events in adult populations. For example, the risk-adjusted rate of decubiti was surprisingly high for children (17.2 events per 1000 discharges), and decubiti were noted frequently for children with poor perfusion, ie, those undergoing extracorporeal membrane oxygenation or cardiac patients who might require adrenergic agents to support their blood pressure, with resultant poor perfusion of the skin. Perhaps different methods to protect against decubiti in this population need to be investigated, compared with methods used for children without perfusion problems. Our goal should be that no child experiences decubiti. Three of the indicators, ie, infections attributable to medical care (ie, associated with procedures or device placements), postoperative pulmonary embolism and venous thrombosis, and decubitus ulcers, are subjects of evidence-based best-practice reports in the adult literature and are highlighted by AHRQ12 as events that should be investigated, with institution of standardized protocols.
Perhaps more importantly, examination of diagnosis and procedure codes associated with identified patient safety events led us to think that, in their current form, some of the PSIs are not sensitive for pediatric use, do not predict preventable events the majority of the time, and should not be used to compare institutions or judge the adequacy of care in children's hospitals. The majority of cases that the AHRQ software identified as deaths in low-mortality DRGs had primary diagnoses of pneumonia, seizures, or abdominal procedures. However, review of the associated ICD-9-CM codes for those cases showed that several patients who were listed as dying as a result of pneumonia actually had cystic fibrosis, some patients with seizures actually died as a result of terminal metabolic disease, and children who died as a result of abdominal procedures often had severe congenital defects, such as gastroschisis. For the failure to rescue indicator, the majority of cases identified by the software were very complex cases for which death was nonpreventable. A combination of diagnoses (eg, acute renal failure, respiratory failure, sepsis, and cardiac arrest) listed with the primary diagnosis (eg, congenital heart disease, leukemia, chronic renal failure, or lupus), rather than a preventable complication, often caused the death.
Our major goal was to identify events that are potentially preventable and to perform clinical redesign when evidence-based literature is available regarding improvement protocols. The infection rate variance analysis demonstrated hospital rates up to 5 times the lowest hospital rate and 1.78 times the mean rate. This variation may indicate that we can investigate the hospitals with the highest and lowest rates to identify best practices, which can be used when appropriate. For example, for the indicator of infections attributable to procedures and device placements, there is randomized, controlled trial evidence that rates of line infections can be reduced significantly with the use of antibiotic-coated central lines,13,14 preparation of the patient's skin with chlorhexidine instead of betadine,15,16 and the use of maximal barrier precautions (ie, sterile gloves, mask, long-sleeved gown, and full-sized sterile drapes) during insertion.17 Although these procedures have not been studied among pediatric patients in a randomized, controlled trial, a number of children's hospitals have already put some of these measures into place, on the basis of adult literature reports, and have observed decreases in infection rates (M. Miller, Johns Hopkins Hospital, personal communication, 2004). The goal should be to use these data to improve children's care. It is hoped that future findings can demonstrate progress toward decreasing the mean number of events observed and decreasing the variance among hospital performances.
PSI events are relatively rare in children's hospitals; therefore, adequate analysis of true variance among hospitals is difficult. As Miller et al previously noted, this underscores the appropriate use of the PSIs as "institutional case-finding tools aimed at internal quality improvement, as opposed to use for directly comparing individual institutions especially in public reports."6(p1364)
| CONCLUSIONS |
|---|
|
|
|---|
In their current form, the PSIs death in low-mortality DRGs and failure to rescue are inaccurate in relation to the pediatric population, do not predict preventable events the majority of the time, and should not be used to compare institutions, to judge adequacy of care, or to calculate potentially preventable deaths in children's hospitals. Research focused on improving indicators of this type by making them more sensitive for pediatric use is critically important and is one of the goals of future work.
|
|
|
|
|
|
|
|
|
| ACKNOWLEDGMENTS |
|---|
We also acknowledge the state data organizations that participate in the HCUP 2000 KID: Arizona Department of Health Services; California Office of Statewide Health Planning and Development; Colorado Health and Hospital Association; ConnecticutChime, Inc; Florida Agency for Health Care Administration; GeorgiaAn Association of Hospitals and Health Systems; Hawaii Health Information Corp; Iowa Hospital Association; Kansas Hospital Association; Kentucky Department for Public Health; Maine Health Data Organization; Massachusetts Division of Health Care Finance and Policy; Maryland Health Services Cost Review Commission; Missouri Hospital Industry Data Institute; New Jersey Department of Health and Senior Services; New York State Department of Health; North Carolina Department of Health and Human Services; Oregon Association of Hospitals and Health Systems; Pennsylvania Health Care Cost Containment Council; South Carolina State Budget and Control Board; Tennessee Hospital Association; Texas Health Care Information Council; Utah Department of Health; Virginia Health Information; Washington State Department of Health; West Virginia Health Care Authority; and Wisconsin Department of Health and Family Services.
| FOOTNOTES |
|---|
Reprint requests to (A.S.) University of Michigan Health System, C201 Med Inn Building, Box 0825, 1500 E Medical Center Dr, Ann Arbor, MI 48109-0825. asedman{at}umich.edu
The views expressed in this article are those of the authors and do not necessarily reflect those of the Agency for Healthcare Research and Quality or the US Department of Health and Human Services.
Conflict of interest: Dr Sedman is a paid medical advisor for the National Association of Childrens Hospitals and Related Institutions.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
K. M. McDonald, S. M. Davies, C. A. Haberland, J. J. Geppert, A. Ku, and P. S. Romano Preliminary Assessment of Pediatric Health Care Quality and Patient Safety in the United States Using Readily Available Administrative Data Pediatrics, August 1, 2008; 122(2): e416 - e425. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. P. Kronman, M. Hall, A. D. Slonim, and S. S. Shah Charges and Lengths of Stay Attributable to Adverse Patient-Care Events Using Pediatric-Specific Quality Indicators: A Multicenter Study of Freestanding Children's Hospitals Pediatrics, June 1, 2008; 121(6): e1653 - e1659. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. C. Scanlon, J. M. Harris II, F. Levy, and A. Sedman Evaluation of the Agency for Healthcare Research and Quality Pediatric Quality Indicators Pediatrics, June 1, 2008; 121(6): e1723 - e1731. [Abstract] [Full Text] [PDF] |
||||
![]() |
O. J. Benavidez, K. Gauvreau, P. D. Nido, E. Bacha, and K. J. Jenkins Complications and Risk Factors for Mortality During Congenital Heart Surgery Admissions Ann. Thorac. Surg., July 1, 2007; 84(1): 147 - 155. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. A. Mark, D. W. Harless, and W. F. Berman Nurse Staffing and Adverse Events in Hospitalized Children Policy Politics Nursing Practice, May 1, 2007; 8(2): 83 - 92. [Abstract] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||