Sustaining and Spreading the Reduction of Adverse Drug Events in a Multicenter Collaborative
OBJECTIVES: Adverse drug events (ADEs) occur more frequently in pediatric patients than adults. ADEs frequently cause serious harm to children and increase the cost of care. The purpose of this study was to decrease ADEs by targeting the entire medication-delivery system for all high-risk medications.
METHODS: Thirteen freestanding children's hospitals participated in this ADE collaborative. An advisory panel developed a change package of interventions that consisted of standardization of medication-ordering (eg, consensus-based protocols and order sets and high-alert medication protocols), reliable medication-dispensing processes (eg, automated dispensing cabinets and redesign of floor stock procedures), reliable medication-administration processes (eg, safe pump use and reducing interruptions), improvement of patient safety culture (eg, safety-culture changes and reduction of staff intimidation), and clinical decision support (eg, increase ADE detection and redesign care systems). ADE rates were compared from the 3-month baseline period to quarters of the 12-month intervention phase. ADE rates were categorized further as opioid related and other medication related.
RESULTS: From baseline to the final quarter, the collaborative resulted in a 42% decrease in total ADEs, a 51% decrease in opioid-related ADEs, and a 41% decrease in other medication ADEs.
CONCLUSION: A pediatric collaborative that targeted the medication-delivery system decreased the rate of ADEs at participating institutions.
Children are at greater risk than adults of experiencing medication-related harm. The rate of potential adverse drug events (ADEs) found in 1 study was ∼3 times that of hospitalized adults.1 Authors of previous studies have estimated that ADEs occur at a rate of 2.3 to 11.2 ADEs per 100 pediatric inpatients.1,–,4 Up to 35% of ADEs in these studies were classified as serious or life-threatening.1,2 ADEs are also expensive to the health care system and estimated to cost from $2000 to $8750 per ADE.5,–,7
Pediatric hospitals affiliated with Child Health Corporation of America (CHCA) (Shawnee Mission, Kansas) developed and evaluated a trigger tool to detect ADEs in pediatric inpatients. In this study, 11.1 ADEs occurred per 100 patients.3 To address harm related to opioids, CHCA designed a multidisciplinary Reducing Narcotic-Related ADEs collaborative that reduced the ADE rate by 49%, prevented an estimated 13 478 ADEs, and showed a net cost savings of more than $2.6 million.8
The goals of the Adverse Drug Event Sustain and Spread (ADESS) collaborative were to ensure the sustained prevention of opioid-associated ADEs achieved in the previous work and to spread the improvements through a 2-pronged approach: (1) improve core medication processes for all medications; and (2) ensure the safe use of high-risk medications throughout the medication-delivery system. In this collaborative, participating hospitals worked in the following areas: standardization of medication-ordering; reliable medication-dispensing and administration; improvement of patient safety culture; and clinical decision support. The goal of this collaborative was to reduce overall ADEs by 50% in the 13 participating pediatric hospitals during a 12-month period.
Thirteen freestanding US children's hospitals participated in a 12-month performance-improvement collaborative to reduce ADEs in pediatric hospitals. The ADESS collaborative was sponsored by CHCA and open to all 43 CHCA member hospitals. CHCA's previous Reducing Narcotic-Related ADEs collaborative of 14 hospitals8 concluded ∼2 years before the initiation of the ADESS collaborative. Six hospitals participated in both collaboratives. In addition to general medication processes that affect the safety of all medications, we focused on 4 high-risk medication types: opioids; anticoagulants; insulin; and total parenteral nutrition. The target population for the ADESS collaborative was all inpatient units (including ICUs) and the emergency department. Perioperative areas and outpatient services were excluded.
A multidisciplinary advisory panel that consisted of pediatric pharmacists, nurses, physicians, and quality-improvement experts developed the project. Using the previous CHCA Reducing Narcotic-Related ADEs collaborative as a reference point and recent published evidence, the panel developed the aim, measurement strategy, and change package of recommended interventions through a series of telephone conferences.
The change package focused on several key components of the medication process (ordering, dispensing, and administration), as well as patient safety culture and clinical decision support. The change package focused on improving process reliability, critical thinking, and interdisciplinary cooperation of pharmacists and nurses. The change package also focused on clinical decision-making by physicians, related to standardization of ordering through order sets and clinical scoring tools. Basic medication-safety practices such as prohibiting the use of nonstandard abbreviations and use of weight-based dosing were assumed to be widely implemented and, thus, not targeted by this collaborative. Medication reconciliation, technology-dependent solutions, staffing, construction, and remodeling were not considered appropriate for this 1-year improvement effort and were excluded from the change package.
Each participating hospital determined if institutional review board (IRB) review was needed and, if appropriate, obtained local IRB approval or waiver before project initiation. Data were collected in compliance with measure specifications developed by the advisory panel using standardized forms developed by CHCA. Hospitals submitted monthly aggregate-level data to CHCA via a Web-based data repository administered by the Institute for Healthcare Improvement.
Interdisciplinary cooperation was a key strategy throughout the project. Each hospital assembled a multidisciplinary team to drive the improvement efforts. Depending on the hospital, the team could include pharmacists, nurses, quality professionals, physicians, clinical educators, laboratory or information technology staff, and accreditation and risk-management staff. Physician engagement varied from being minimal at some hospitals to providing guidance and support for nursing and pharmacy improvements at others and to implementing changes in physician practices. Each team designated team and senior leaders to oversee the project and to facilitate cooperation across disciplines.
The model for improvement used by the Institute for Healthcare Improvement was the methodologic basis for this collaborative.9,10 The model for improvement is based on the iterative use of small tests of change (plan-do-study-act [PDSA] cycles) and subsequent spread of successful change ideas. Hospitals were free to apply additional improvement methods used, such as Lean or Six Sigma. Principles and tools from reliability science were emphasized throughout the project, as were strategies for spreading and maintaining improvements in participating hospitals after conclusion of the collaborative.
Teams were instructed to select a broad range of change-package topics for implementation, but individual teams were not expected to implement the entire package. The change strategies of the standardization of medication-ordering, reliable medication-dispensing and -administration processes, patient safety culture, and clinical decision support with key changes and examples are listed in Table 1.
To facilitate sharing of change ideas and strategies among teams, CHCA facilitated 4 learning sessions (2 virtual, 2 in-person), monthly conference calls, and individual team coaching. Hospitals were encouraged to connect 1-on-1 with other teams as needed. CHCA staff also provided a Listserv and collaborative Web page.
Measurement and Reporting
The aim of the ADESS collaborative was to reduce overall ADEs by 50%. The primary outcome measure was ADEs per 1000 patient-days, measured in a random sample of 20 patients per month in each hospital by using the trigger-tool chart-review method.3 The triggers are listed in Table 2. The rate of opioid-related ADEs found during the chart review was tracked separately, because opioids were the most common source of medication-related harm in CHCA hospitals.3 Two optional medication-specific outcome measures were also developed: anticoagulation laboratory values out of range per 100 anticoagulant days and rescue doses for insulin-associated hypoglycemia per 100 insulin days. Process measures included the rate of automated dispensing device overrides for opioids (percentage of opioids dispensed from the automated dispensing device that were dispensed via override) and pump-programming accuracy (percentage of all pump-programming events that were completed correctly, including a complete double-check as required).
Hospitals collected and submitted data monthly during the study period (July 1, 2008 to June 30, 2009). Data from the preceding 3 months (April 1, 2008 to June 30, 2008) were collected from each hospital as baseline data. The analysis included hospitals that submitted data for at least the baseline period and the final 3 months of the project. Pump-programming accuracy data were not available from the baseline data period, so hospitals were asked to begin collecting data immediately after the collaborative began. Hospital-level data were unblinded and could be viewed by all participating teams.
Each hospital reported a self-assessment score of their progress (score between 1 and 5, based on a scale developed by the Institute for Healthcare Improvement) and provided a narrative progress report to CHCA through the Web-based data repository on a monthly basis. The progress report included descriptions of change efforts over the previous month, barriers, successes, lessons learned, and next steps. The CHCA collaborative leader produced hospital-specific and aggregate reports based on these submissions, including an assessment score (using the same scale noted above) and personalized feedback for each team. CHCA communicated team progress with senior leaders on an ongoing basis and produced final reports that described the results of the project.
At the conclusion of the ADESS collaborative, each hospital was asked (through a Web survey [Survey Monkey, Palo Alto, CA]) which portions of the change package they implemented or improved on in their existing processes.
Rates for all measures are presented from baseline and each quarter of the collaborative with exact 95% Poisson confidence intervals (CIs). Changes in rates between baseline and the intervention period were assessed by using exact person-year tests. The 12 hospitals that provided baseline data were included in this analysis. Comparisons were made at the hospital level and aggregated to the collaborative level. We performed comparisons for all drugs combined and then stratified the analyses according to opioid versus nonopioid drugs. Ten hospitals provided data according to type of drug and were included in the stratified analysis. We performed comparisons of hospitals in both the Reducing Narcotic-Related ADEs and ADESS collaboratives and hospitals only in the ADESS collaborative.
An estimate of total ADEs averted was computed by calculating the difference between the baseline to actual rates throughout the intervention period. The number of ADEs averted for the collaborative was extrapolated from the sample population (20 per site per month) to the entire collaborative hospital population.
Secondary outcomes measures were optional and only collected by hospitals working on changes in applicable areas. Thus, they were reported by only a maximum of 4 hospitals. Analyses of rates followed the procedures described above. Proportional measures were compared by using χ2 statistics.
All analyses were performed by using SAS 9.2 (SAS Institute, Cary, NC), and a P value of <.05 was considered statistically significant.
Twelve of the 13 participating hospitals reported complete ADE-rate data. One hospital did not submit baseline data. Among these 12 hospitals, the ADE rate (Fig 1) declined by 42% in the final quarter of the project (22.4 ADEs per 1000 patient-days [95% CI: 19.2–26.2]) compared with the baseline (38.6 ADEs per 1000 patient-days [95% CI: 33.9–43.9]) (P < .001). Using actual rates for each quarter, we estimated that 5843 net ADEs were averted during the 12-month period across 12 hospitals. Six hospitals had a decrease in ADEs, and 6 hospitals had a net increase in ADEs (Fig 2). Two hospitals had lower ADE rates in the final quarter than at baseline, but higher rates during the middle of the project resulted in a net increase in ADEs over the entire intervention period.
For the 6 hospitals that participated in both the Reducing Narcotic-Related ADEs and ADESS collaboratives, the ADE rate declined by 53% from a baseline of 29.2 ADEs per 1000 patient-days (95% CI: 23.5–36.4) to 13.8 ADEs per 1000 patient-days in the final quarter (95% CI: 10.3–18.5) (P < .0001) (Fig 3). The 6 hospitals that only participated in the ADESS collaborative decreased their ADE rate by 36% from a baseline of 46.4 ADEs per 1000 patient-days (95% CI: 39.6–54.3) to 29.6 ADEs per 1000 patient-days in the final quarter (95% CI: 24.7–35.5) (P = .0003). When compared, the baseline rate of 46.4 ADEs per 1000 patient-days in the ADESS-only group was higher than the baseline of 29.2 ADEs per 1000 patient-days for the both-collaboratives group (P < .0006). The final rate of 29.6 ADEs per 1000 patient-days in the ADESS-only group was also higher than the rate of 13.8 ADEs per 1000 patient-days in the group that participated in both collaboratives (P < .0001).
For the 10 hospitals that reported opioid-related ADEs separately, these events declined 51% (26.3 ADEs per 1000 patient-days [95% CI: 22.2–31.2]) to 13.0 ADEs per 1000 patient-days (95% CI: 10.3–16.2) in the final quarter of the project compared with baseline (Fig 4) (P < .001). The opioid-related ADE rate began with higher baseline ADE rates and also had a larger decrease in relative rates compared with the ADE rate for all other medications, which declined 41% from 15.5 ADEs per 1000 patient-days (95% CI: 12.4–19.3) to 9.1 ADEs per 1000 patient-days (95% CI: 7.0–11.9) (P = .003). The estimated number of opioid-related ADEs averted for the 10 hospitals that submitted this measure was 2744.
Other measures defined by the advisory panel were specific to anticoagulants, insulin, and automated dispensing overrides. Because not all hospitals worked in these change areas, only 3 to 4 hospitals reported these metrics (Table 3). Improvement was not observed over time, but observed rates across the entire project are included to provide general information about rates in children's hospitals. No hospitals reported pump-programming accuracy during the first 3 months of the collaborative, so those data are not reported.
Ten of the 13 hospitals responded to the postcollaborative survey of the components of the change package that were implemented, and the results are listed in Table 4.
The goal of the ADESS collaborative was to decrease total ADEs by 50%. Several hospitals decreased their ADE rates by >50%. The collaborative as a whole decreased ADEs by 42%. The 6 hospitals that participated in the previous Reducing Narcotic-Related ADEs and this collaborative had a 53% decrease in ADEs. Opioid-related ADEs decreased by 51%, and all other ADEs decreased by 41%. Hospitals that did not have a decrease in ADEs started below the mean postintervention ADE rate for the ADESS collaborative. Additional improvements for these hospitals may have been difficult to demonstrate without reviewing a larger number of charts to increase the power of the study.
A wide variety of change strategies helped to achieve these results across the participating hospitals. One of the most commonly implemented interventions was reducing interruptions and distractions during medication administration and, in some hospitals, during other key parts of the medication process such as medication preparation in the pharmacy and medication cabinet-stocking on the units by pharmacy technicians. Frequent interruptions during the medication-administration process have been associated with an increased risk of errors.11 Interruptions and distractions were reduced by a combination of visual cues12 (eg, caution tape and signage around the medication-dispensing device), staff practices13 (eg, handing off beepers and alarms to another caregiver during medication administration), and physical changes14 (eg, moving the automated dispensing machine to a quiet location).
Another common change area was improving the reliability and effectiveness of independent double-checks.15,16 Policies and definitions for double-checks were clarified (eg, defining which medications and components should be double-checked in different situations, such as what is checked when starting a new infusion versus at shift change)17 and ensuring that double-checks were truly independent.18 Nationally accepted standard definitions were absent or inadequate and should be developed. Hospitals also implemented checklists to improve the reliability of the double-check process.19 Careful selection of medications and processes for double-checks will prevent errors.16,18,20
Other key change areas included improving pump guardrail settings,21 standardizing ordering through protocols and corollary orders,22,23 ensuring that laboratory findings are communicated and reviewed in a timely fashion to appropriately manage the selection and dose of medications, and reducing the risky practice of removing medications from automated dispensing cabinets via override and thus bypassing safety checks.24
The ADESS collaborative provided anticoagulant- and insulin-related rates and process measures for automated dispensing unit override for opioids that can serve as background data for pediatric hospitals.
As far as we are aware, our quality-improvement collaborative is the first to demonstrate that a pediatric multicenter collaborative targeting the whole continuum of medication safety can reduce ADEs. We also believe that we are the first to report the spread of a collaborative-based effort to reduce narcotic-related ADEs across pediatric hospitals: the 51% reduction in narcotic-related ADEs in the ADESS collaborative is comparable to the 67% decrease demonstrated in the previous CHCA Reducing Narcotic-Related ADEs collaborative. In addition, 6 hospitals were able to sustain their performance improvement efforts from the Reducing Narcotic-Related ADEs collaborative through the ADESS collaborative. The 6 hospitals that participated in both the ADESS and Reducing Narcotic-Related ADEs collaboratives began with lower baseline ADE rates and ended with lower ADE rates at the end of the collaborative when compared with the 6 hospitals that only participated in the ADESS collaborative.
The limitations of this collaborative are similar to those of the CHCA Reducing Narcotic-Related ADEs collaborative.8 First, the ADESS collaborative consisted of freestanding children's hospitals, so the results may not be generalizable to pediatric care provided in general hospitals or pediatric hospitals within general hospitals. Second, we did not determine the impact of individual change strategies, although we did determine which components of the change package were implemented by the participating hospitals. Third, we used a before-and-after design to evaluate the impact of the collaborative, which makes it more difficult to demonstrate the causality between the intervention and outcomes. Although there may have been other confounders, we believe that the improvement over time was due in large part to the hospitals' interventions over the course of the collaborative. A randomized study of the different hospitals with different implementation change packages would be a stronger study design, but it would lack the interactive and collaborative learning style possible in the ADESS collaborative. Fourth, the period for the collaborative was 1 year. If we had studied the effect of our interventions over a longer period, we potentially could have demonstrated a larger improvement. Fifth, the trigger-tool method is subject to sampling bias and may not have detected all of the ADEs. However, all of the participating institutions underwent standardized education on the use of the trigger-tool method; thus, there was uniform application of the tool across all sites, and the rigor of the tool provides greater certainty that the measure was providing reliable estimates of ADE rates.
Pediatric patients are at high risk for ADEs. A performance-improvement collaborative of 13 pediatric institutions that used a broad change strategy targeting all phases of the medication process decreased ADEs significantly by 42% (a 41% decrease in nonopioid ADEs and a 51% decrease in opioid ADEs). Implementing different components of the change package, these freestanding pediatric hospitals improved the safety of their patients. This collaborative benefited from lessons learned from a previous collaborative, and the lessons learned from this collaborative can, in turn, be spread to other hospitals that care for children to reduce harm to this vulnerable population.
- Accepted April 21, 2011.
- Address correspondence to Eric Tham, MD, MS, Department of Pediatrics, University of Colorado School of Medicine, 13123 E 16th Ave, B251, Aurora, CO 80045. E-mail:
Drs Tham, Calmes, Poppy, and Eliades, Ms Schlafly, Dr Namtu, Ms Smith, Dr Vitaska, Ms McConnell, Dr Potts, Ms Jastrzembski, Ms Logsdon, and Drs Hall and Takata provided substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; drafting of the article or revising it critically for important intellectual content; and approval of the version to be published.
FINANCIAL DISCLOSURE: Dr Takata reported owning 100 shares of Pfizer stock and 100 shares of Amgen stock, which he sold at the start of this study; the other authors have indicated they have no financial relationships relevant to this article to disclose.
- ADE —
- adverse drug event
- CHCA —
- Child Health Corporation of America
- ADESS —
- Adverse Drug Event Sustain and Spread
- CI —
- confidence interval
- Takata GS,
- Mason W,
- Taketomo C,
- Logsdon T,
- Sharek PJ
- Takata GS,
- Taketomo CK,
- Waite S
- Aspden P
- Sharek PJ,
- McClead RE Jr.,
- Taketomo C,
- et al
Institute for Healthcare Improvement. The Breakthrough Series: IHI's Collaborative Model for Achieving Breakthrough Improvement. Boston, MA: Institute for Healthcare Improvement; 2003
- Langley GJ
- Cohen MR,
- Anderson RW,
- Attilio RM,
- Green L,
- Muller RJ,
- Pruemer JM
Institute for Safe Medication Practices. Santa checks his list twice: shouldn't we? Available at: www.ismp.org/newsletters/acutecare/articles/20091217.asp. Accessed April 18, 2011
Institute for Safe Medication Practices. The virtues of independent double checks: they are worth your time! Available at: www.ismp.org/newsletters/acutecare/articles/20030306.asp. Accessed April 18, 2011
- White RE,
- Trbovich PL,
- Easty AC,
- Savage P,
- Trip K,
- Hyland S
- Overhage JM,
- Tierney WM,
- Zhou XH,
- McDonald CJ
- Copyright © 2011 by the American Academy of Pediatrics