An Intervention to Decrease Narcotic-Related Adverse Drug Events in Children's Hospitals
OBJECTIVES. Narcotic-related adverse drug events are the most common adverse drug events in hospitalized children. Despite multiple published studies describing interventions that decrease adverse drug events from narcotics, large-scale collaborative quality improvement efforts to address narcotic-related adverse drug events in pediatrics have not been described. The purpose of this study was to evaluate collaborative-wide narcotic-related adverse drug event rates after a collection of expert panel–defined best practices was implemented.
METHODS. All 42 children's hospitals in the Child Health Corporation of America were invited to participate in the Institute for Healthcare Improvement–style quality improvement collaborative aimed at reducing narcotic-related adverse drug events. A collection of interventions known or suspected to reduce narcotic-related adverse drug events was recommended by an expert panel, with each site implementing ≥1 of these best practices on the basis of local need. Narcotic-related adverse drug event rates were compared between the baseline (December 1, 2004, to March 31, 2005) and postimplementation periods (January 1, 2006, to March 31, 2006) after an a priori–defined intervention ramp-up time (April 1, 2005, and December 31, 2005). Secondary outcome measures included constipation rates and narcotic-related automated drug-dispensing-device override percentages.
RESULTS. Median narcotic-related adverse drug event rates decreased 67% between the baseline and postimplementation time frames across the 14-site collaborative. Constipation rates decreased 68.9%, and automated drug-dispensing-device overrides decreased from 10.18% to 5.91% of all narcotic doses administered.
CONCLUSIONS. Implementation of ≥1 expert panel–recommended interventions at each participating site resulted in a significant decrease in narcotic-related adverse drug events, constipation, and automated drug-dispensing-device overrides in a 12-month, 14-site children's hospital quality collaborative.
Adverse drug events (ADEs) in pediatric inpatients are common, costly, and occasionally life-threatening or fatal. In a recently published study that reviewed 960 inpatients from 12 children's hospitals,1 the mean rate of ADEs in US children's hospitals was 11.1 per 100 admissions with 22.4% classified as preventable. This extrapolates to a mean of 174 preventable ADEs per children's hospital per year. In addition, Kaushal et al2 found that 7% of pediatric inpatient ADEs were “fatal or life-threatening.” Using an estimated cost of $4685 per preventable ADE,3 these preventable ADEs resulted in direct costs of $909644 per children's hospital per year.
The national implications of these rates are striking; extrapolating from US hospital statistics4 an estimated 160840 preventable ADEs and 11260 preventable fatal or life-threatening events at an estimated cost of $753.5 million occur in US pediatric inpatients annually. Interventions aimed at decreasing preventable ADEs in pediatrics are needed to reduce the serious public health and social consequences of this harm to our nation's most vulnerable patient population.
To understand better the ability of interventions to reduce or eliminate ADEs in pediatric inpatients, we studied narcotic-related ADE rates in the 14-site Reducing Narcotic-Related Adverse Drug Events in Children improvement collaborative. Narcotics were specifically targeted because this medication class is high risk,5 high priority,6 frequently used in inpatient children (data from Child Health Corporation of America's [CHCA's] Pediatrics Health Information Systems, an administrative database from 38 freestanding, academic, not-for-profit, tertiary care pediatric hospitals in the United States, suggest that 41% of the 363689 discharges from 38 children's hospitals in 2004 were provided at least 1 dose of a narcotic)7 and is associated with 33% to 51% of pediatric ADEs.1,2 The objective of this study was to evaluate the effect of a combination of evidence-based interventions on narcotic-related ADE rates in hospitalized children during the course of a 12-month, 14-site quality improvement collaborative project.
Fourteen freestanding children's hospitals participated in a 12-month improvement collaborative project entitled the Reducing Narcotic-Related Adverse Drug Events in Children improvement collaborative (Table 1). Inclusion criteria required participation for the entire 12-month project period and a minimum requirement of at least 50% of monthly data submitted. Sites were excluded from this analysis when they joined after April 1, 2005, or when they contributed data for <6 of the 12 months of the collaborative. This project was sponsored by the CHCA (Shawnee Mission, KS), a business alliance of 42 children's hospitals. The project was open to all 42 freestanding children's hospitals affiliated with the CHCA as of January 1, 2005, and was funded by a grant from the Agency for Healthcare Research and Quality and CHCA Research and Development funds. The CHCA obtained institutional review board (IRB) approval for data management and analysis from an external IRB (Western IRB, Olympia, WA), and each participating site obtained local IRB approval or waiver before project participation.
The project design and content were developed by a multidisciplinary pediatric expert panel that consisted of invited staff from participating hospitals and external subject matter experts (7 clinical and 4 improvement/process experts [see Acknowledgments]). This panel developed a comprehensive, pediatric-specific “change package” of evidence-based practices organized into the broad categories of (1) narcotic use, (2) medication systems, (3) medication reconciliation, and (4) culture of safety. The change package (available at www.chca.com) was based on high-risk medication change packages previously developed by the Institute for Healthcare Improvement (IHI).8 Because the change package was broad in scope, 4 primary areas for active collaboration were selected. First, constipation, known to be a frequent narcotic-associated ADE,1 was addressed by recommending the routine proactive use of laxatives and stool softeners when narcotics are prescribed. This type of order, called a “corollary order,” has been shown in the literature to decrease the frequency of medication-related errors of omission.9 Second, overrides of automated medication dispensing devices, known to increase the risk for medication errors,10 were targeted with a recommendation to limit override opportunities. Third, withdrawal symptoms, a significant ADE with ineffective weaning of medication doses after extended narcotic use, was targeted for improvement by recommending a standardized weaning protocol. Finally, ADEs that occur at the high-risk transfer process11 were addressed by recommending consistent medication reconciliation strategies.11,12
The IHI collaborative quality improvement model,13 including meeting structure (3 face-to-face learning sessions), communication strategies (monthly conference calls, active Listservs, and available content and quality experts), and monthly data submission and reporting, was used to facilitate quality improvement in this collaborative. Each site assembled a multidisciplinary team and designated a site leader and a senior leader. Hospitals learned improvement methods and strategies that were based on the Model for Improvement,14 which emphasizes small tests of change (plan-do-study-act), as well as specific sustain and spread strategies. Hospitals were also free to apply other improvement methods, such as Lean15 or Six Sigma.16 During the collaborative, site teams made numerous process improvements and reported key outcome and process measures on a Web-based data repository. Full transparency in the form of unblinded data and open sharing of successes and barriers was a critical requirement for participation. The effort was coordinated by central staff who were employed by the CHCA. These CHCA staff members provided feedback and formal reports to sites monthly, planned and implemented learning sessions, provided content oversight and process management for the collaborative, communicated progress to site-specified senior leaders on an ongoing basis, and produced final deliverables describing the results of the project. All site members had access to a project-specific Web site and listserv coordinated by the CHCA. The baseline phase occurred between December 1, 2004, and March 31, 2005, and the “implementation phase” of the project spanned 12 months from April 1, 2005, to March 31, 2006. For determination of the effects of the collaborative on outcomes, results were reported on a quarterly basis; the collaborative defined a priori that the final results would be reported on the basis of the final quarter of the project, January 1, 2006, through March 31, 2006.
Patients were eligible for inclusion when they were pediatric inpatients discharged between December 1, 2004, and March 31, 2006; had a minimum 24-hour length of stay; and received at least 1 dose of a narcotic medication. Patients were selected and classified on the basis of discharge date, regardless of when the narcotic medications were administered. ADEs were identified using a modified trigger tool specifically designed for narcotic ADE detection, a method developed by the IHI17–19 and adapted for pediatric patients by the CHCA.1,20 The trigger tool detection method was used in a consistent manner throughout the course of the study. ADE data were collected for 20 randomly selected eligible patient discharges per site per month. Chart reviewers were instructed to focus on 8 triggers that typically are associated with narcotic ADEs (Table 2) and spend a maximum of 20 minutes per chart reviewed as recommended by the IHI trigger tool method.17,21 Sites collected information on narcotics dispensed from automated medication dispensing devices, both as total narcotic doses dispensed and the number of doses dispensed via an override. An override was defined as any process that allowed the pharmacist order verification step to be bypassed so that nurses can obtain medications before a pharmacist review.
An ADE (harm) was defined as an injury, large or small, caused by the use of a drug, including both preventable and nonpreventable events.1,21,22 This project did not include harm from nonuse of narcotics (“error of omission”), such as unrelieved pain. All events classified as categories E through I of the National Coordinating Council for Medication Error Reporting and Prevention Index for Categorizing Errors23 were included.
The primary outcome measure was narcotic-associated ADE rates (expressed as either per 1000 narcotic patient-days or per 1000 narcotic doses). Secondary outcomes included constipation rates (per 1000 narcotic patient-days or per 1000 narcotic doses) and percentage of narcotics dispensed via an automated medication dispensing device using an override (doses overridden divided by total doses dispensed). Most sites did not actively pursue withdrawal-related ADEs or medication reconciliation related specifically to narcotics; therefore, no outcomes related to these interventions were systematically collected. The estimated number of ADEs averted was calculated for each site and for the collaborative as a whole by using the difference between the baseline ADE rate and the rates reported during each month of the collaborative. Collaborative-wide targets were established for each measure; teams were encouraged to customize and/or set additional targets to meet individual site goals.
Sites were instructed to collect monthly baseline data for the period December 1, 2004, through March 31, 2005, and to report project data on a monthly basis for 12 months (April 1, 2005, through March 31, 2006). Monthly site data were reported to the CHCA via a secure Web-based data repository hosted by the IHI. Hospitals used either standardized data collection forms developed by the CHCA or processes developed at the individual institution to translate chart review and electronic data to monthly results.
The primary outcome measure for this project was the rate of narcotic-related ADEs per 1000 narcotic doses or per 1000 narcotic days. Sites were encouraged to use the denominator most appropriate for their particular site. To combine results from hospitals that submitted the measure in terms of doses and days, we computed, for each hospital, a percentage change from baseline for each of the postbaseline quarters. We then used the Wilcoxon signed-rank test to determine whether the percentage change was different from 0. A similar strategy was used to determine the statistical significance of the percentage change in constipation rates. To assess whether a significant change occurred from baseline to postimplementation time period for override percentages, we used the nonparametric Wilcoxon rank-sum test. We hypothesized that the primary and secondary outcome measure rates would be reduced between baseline and the postimplementation time frame. Because of the variation in the specific interventions implemented by each site and the variable time frames for implementation, the rates of successful implementation of any given improvement practice were not calculated.
The total number of ADEs averted during the project was computed for the sample population on the basis of the difference between the baseline rate and rates reported during each month of the collaborative. The number of ADEs averted for the entire 14-site collaborative was extrapolated from the project sample population (20 per month per site) to the entire 14-site hospital population.
The data for the measures were nonnormally distributed. Thus, measures were aggregated on a quarterly basis by using medians with interquartile ranges. Missing data were imputed by using the previous data point carried forward.24 The reported P values are 2-sided, and P < .05 was considered significant. All analyses were performed by using SAS 9.1 (SAS Institute, Inc, Cary, NC).
Forty-two children's hospitals in the CHCA were invited to participate in the Reducing Narcotic-Related Adverse Drug Events in Children improvement collaborative, 17 of which agreed to participate. Two of these 17 hospitals were excluded from data analysis on the basis of suboptimal data submission (1 site submitted 5 months of data, the second submitted 0 months of data), 1 hospital was excluded on the basis of late entrance into the collaborative (July 2005), and 1 hospital declined release of its data for publication. Thus, 14 children's hospitals met all inclusion requirements; 13 sites composed the final group for analysis (Table 1).
Narcotic-related ADE rates, measured either per 1000 narcotic doses or per 1000 narcotic days, decreased 67% between baseline and postimplementation time frames (n = 13; P < .001; Table 3). Statistically significant reductions in rates were noted in quarters 2 and 3 as well as 4 (the postimplementation time frame). The median rate at baseline was 41.0 narcotic-related ADEs per 1000 narcotic doses (n = 10 sites) and 164.9 narcotic-related ADEs per 1000 narcotic days (n = 3). Median constipation rates increasingly declined each quarter as well, with a statistically significant reduction of 69% seen in the postimplementation time frame (n = 13; P < .005; Table 4). The median rate at baseline was 14.1 constipation events per 1000 narcotic doses (n = 10 sites) and 36.1 constipation events per 1000 narcotic days (n = 3). Median override rates declined significantly each quarter, with the postimplementation time frame reduced from 10.2% to 5.9% (n = 10 [42%]; P < .001; Table 5). Three sites submitted <50% of override data and thus were excluded from this outcome analysis on the basis of our a priori–defined inclusion criteria.
On the basis of these results, we extrapolated the difference between the absolute number of expected (baseline) ADEs and the actual ADEs during the implementation and postimplementation time frames to estimate that 14594 ADEs were averted during the full 12-month intervention time frame in the 13 hospitals that submitted ADE data.
The goal of the Reducing Narcotic-Related Adverse Drug Events in Children improvement collaborative was to improve narcotic-related medication safety in participating children's hospitals. Narcotic-related ADE rates decreased 67% between the baseline and postimplementation time frames. The constipation subset of ADEs decreased significantly as well, as did the percentage of narcotic overrides. We believe this to be the first report of a pediatric-specific medication safety collaborative to demonstrate a significant reduction in narcotic-related ADEs.
This study suggests that a large-scale collaborative project, focused on reducing the incidence of narcotic-related ADEs in children's hospitals, is feasible and can have important public health consequences. This is of particular importance because, to our knowledge, no published studies have shown significant reductions in ADEs in large-scale pediatric collaboratives. We believe that this collaborative was successful at reducing narcotic-related ADEs for several reasons, including the availability of a robust performance improvement infrastructure at the CHCA to facilitate this work, the willingness of each site to share openly data and practices (transparency), and a public commitment to remaining on schedule with assigned tasks (accountability). Each of these success factors is thought to be a crucial component of an effective collaborative.25
On the basis of these results, we believe that important reductions in morbidity and health care costs are possible if our change package to reduce narcotic-related ADEs in hospitalized children is introduced nationwide. Between 33% and 51% of all pediatric ADEs are caused by narcotics,1,2 and an estimated 160840 preventable ADEs and 11260 preventable fatal or life-threatening ADEs occur annually in pediatric inpatients in the United States. Because ADEs in the pediatric population are frequently associated with narcotics, our experience suggests that a significant amount of the total ADE burden in children could be eliminated with universal implementation of our change package.
This study had several limitations. First, the pre–post intervention study design limits our ability to assign causality between the intervention and the primary outcome of decreased narcotic-related ADE rates. Evidence supporting that this decrease is related to the implementation of the change package is reflected in attainment of the collaborative-wide goal of decreasing constipation, a major subset of ADEs resulting from narcotic administration in pediatric inpatients. In pre–post studies, there is no way to determine the effect of secular trends on outcomes; however, we are not aware of any clear nationwide interventions implemented during the course of our project that could account for large-scale improvements at these 14 geographically and demographically diverse children's hospitals. Hence, although we believe that these improvements were the direct result of collaborative implementation of the change package, it is possible that other, unmeasured and unrecognized factors could have contributed.
Second, there is the risk for underreporting of narcotic-related ADEs after implementation of the intervention. Local site data were not verified by study personnel for accuracy, and data collectors were not blinded to study time frames. If postimplementation narcotic-related ADEs were underreported, then this would create a measurement bias that would exaggerate the results. We believe that this is unlikely because a standard trigger tool method17–21 was used at the beginning of the project and used in a consistent manner throughout by consistent personnel at each site.
Third, we did not evaluate compliance with the change package, limiting our ability to link directly the intervention with the outcome. Measuring compliance with the change package would have been challenging, particularly in light of the heterogeneity among site practices at baseline, and was beyond the scope of this pragmatic quality improvement project. Fourth, we did not attempt to evaluate the impact of each of the individual components of the change package. Because our goal was to improve maximally patient safety of inpatient children, rapid implementation of change package components relevant to individual sites rather than stressing compliance to specific components offered the best opportunity to do so. Fifth, this project was open only to children's hospitals in the CHCA, which by definition are freestanding. These results therefore may not be applicable to pediatric inpatients in an adult hospital or a pediatric hospital within an adult hospital, potentially rendering our national estimates of effect inaccurate. Sixth, we did not formally evaluate the risks associated with implementing components of the change package because of our belief that none of the interventions recommended was likely to cause harm. Finally, the baseline data collection period was short and thus potentially inaccurate in terms of true baseline collaborative-wide ADE burden. In an effort to increase the stability of the baseline period, we defined the baseline period a priori to be 4 months in duration. This short baseline was factored into the statistical analysis, the result being a statistically significant decrease in narcotic-related ADEs collaborative-wide with 95% confidence.
Narcotic-related ADEs in pediatrics are frequent, expensive, and occasionally life-threatening or fatal. As part of a children's hospital patient safety initiative, 14 freestanding children's hospitals collaboratively implemented various interventions from an expert-constructed change package of multiple interventions to decrease narcotic-related ADEs in hospitalized children. Coincident with these interventions, we saw a decrease in narcotic-related ADEs of 67% across the collaborative. Broad use of these interventions could significantly reduce morbidity and the costs of care associated with narcotic-related ADEs in hospitalized children. Future studies to explore the implementation of these and similar interventions in pediatric inpatients who are cared for outside freestanding children's hospitals is warranted.
This study was funded by Agency for Healthcare Research and Quality grant 1 U18 HS013698-01.
We thank Stuart Levine, PharmD, Jane Taylor, EdD, Lloyd Provost, MS, Spence Byrum, Lee Flowers, Matt Hall, PhD, Jane Roessner, PhD, and Frank Davidoff, MD, for contributions to this collaborative effort resulting from participation on the expert panel (Levine, Taylor, Provost, Byrum, and Flowers), guidance with the statistical analysis (Hall), and thoughtful review of the draft manuscript (Roessner and Davidoff).
- Accepted June 17, 2008.
- Address correspondence to Paul J. Sharek, MD, MPH, Lucile Packard Children's Hospital, 700 Welch Rd, No. 227, Palo Alto, CA 94304. E-mail:
The authors have indicated they have no financial relationships relevant to this article to disclose.
What's Known on This Subject
ADEs occur at the rate of 15.7 per 1000 patient-days in hospitalized children. Narcotics are the most frequent drug class associated with these harms, resulting in 51% of ADEs in hospitalized children.
What This Study Adds
To our knowledge, this study is the first to reveal a significant collaborative-wide reduction in narcotic-related ADE rates in hospitalized children after implementation of a simple “change package” of best practices.
- ↵Takata G, Mason W, Takatoma C, Logsdon T, Sharek PJ. Development, testing, and findings of a pediatric-focused trigger tool to identify medication related harm in US children's hospitals. Pediatrics.2008;121 (4). Available at: www.pediatrics.org/cgi/content/full/121/4/e927
- ↵HCUP Kids' Inpatient Database (KID). Healthcare Cost and Utilization Project (HCUP). Rockville, MD: Agency for Healthcare Research and Quality; 2003. Available at: www.hcup-us.ahrq.gov/kidoverview.jsp. Accessed March 20, 2008
- ↵Institute of Safe Medication Practices. ISMP's list of high-alert medications. Available at: www.ismp.org/Tools/highalertmedications.pdf. Accessed March 20, 2008
- ↵Institute for Healthcare Improvement. 5 million lives campaign. Available at: www.ihi.org/IHI/Programs/Campaign. Accessed March 20, 2008
- ↵Child Health Corporation of America. Pediatric Health Information System (PHIS) Database. Shawnee Mission, KS: Child Health Corporation of America; 2004
- ↵Leape LL, Kabcenell A, Berwick DM, Roessner J. Reducing Adverse Drug Events: Breakthrough Series Guide. Boston, MA: Institute for Healthcare Improvement; 1998
- ↵Overhage JM, Tierney WM, Zhou XH, et al. A randomized trial of corollary orders to prevent errors of omission. J Am Med Inform Assoc.1997;4 (5):364– 375
- ↵Oren E, Shaffer ER, Guglielmo BJ. Characteristics of antimicrobial overrides associated with automated dispensing machines. Am J Health Syst Pharm.2002;59 (15):1445– 1448
- ↵Joint Commission on Accreditation of Healthcare Organizations. National patient safety goals, 2005. Available at: www.jointcommission.org/PatientSafety/NationalPatientSafetyGoals/05_hap_npsgs.htm. Accessed March 20, 2008
- ↵Institute for Healthcare Improvement. The Breakthrough Series: IHI's Collaborative Model for Achieving Breakthrough Improvement. IHI Innovation Series White Paper. Boston, MA: Institute for Healthcare Improvement; 2003. Available at: www.ihi.org/IHI/Results/WhitePapers/TheBreakthroughSeriesIHIsCollaborativeModelforAchieving+BreakthroughImprovement.htm. Accessed March 20, 2008
- ↵Langley GL, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. San Francisco, CA: Jossey-Bass Publishers; 1996
- ↵Institute for Healthcare Improvement. Going Lean in Health Care. IHI Innovation Series White Paper. Cambridge, MA: Institute for Healthcare Improvement; 2005. Available at: www.ihi.org/IHI/Results/WhitePapers/GoingLeaninHealthCare.htm. Accessed March 20, 2008
- ↵Rozich JD, Haraden CR, Resar RK. Adverse drug event trigger tool: a practical methodology for measuring medication related harm. Qual Saf Health Care.2003;12 (3):194– 200
- Resar RK, Rozich JD, Classen DC. Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care.2003;12 (suppl 2):ii39– ii45
- ↵Resar RK, Rozich JD, Simmonds T, Haraden CR. A trigger tool to identify adverse events in the intensive care setting. J Comm J Qual Patient Saf.2006;32 (10):585– 590
- ↵Sharek PJ, Horbar JD, Mason W, et al. Adverse events in the neonatal intensive care unit: development, testing, and findings of a NICU-focused trigger tool to identify harm in North American NICUs. Pediatrics.2006;118 (4):1332– 1340
- ↵Institute for Healthcare Improvement. IHI Trigger Tool for Measuring Adverse Drug Events. IHI Innovation Series White Paper. Cambridge, MA: Institute for Healthcare Improvement; 2004. Available at: www.ihi.org/IHI/Topics/PatientSafety/MedicationSystems/Tools/Trigger%20Tool%20for%20Measuring%20Adverse%20Drug%20Events%20(IHI%20Tool). Accessed March 20, 2008
- ↵National Coordinating Council for Medication Error Reporting and Prevention. Taxonomy of medication errors. Available at: www.nccmerp.org/pdf/indexColor2001-06-12.pdf. Accessed March 20, 2008
- ↵Little RA, Rubin DB. Statistical Analysis With Missing Data. New York, NY: Wiley Publishing; 1987
- ↵Kilo CM. Improving care through collaboration. Pediatrics.1999;103 (1 suppl E):384– 393
- Copyright © 2008 by the American Academy of Pediatrics