A Multicenter Collaborative Approach to Reducing Pediatric Codes Outside the ICU
OBJECTIVES: The Child Health Corporation of America formed a multicenter collaborative to decrease the rate of pediatric codes outside the ICU by 50%, double the days between these events, and improve the patient safety culture scores by 5 percentage points.
METHODS: A multidisciplinary pediatric advisory panel developed a comprehensive change package of process improvement strategies and measures for tracking progress. Learning sessions, conference calls, and data submission facilitated collaborative group learning and implementation. Twenty Child Health Corporation of America hospitals participated in this 12-month improvement project. Each hospital identified at least 1 noncritical care target unit in which to implement selected elements of the change package. Strategies to improve prevention, detection, and correction of the deteriorating patient ranged from relatively simple, foundational changes to more complex, advanced changes. Each hospital selected a broad range of change package elements for implementation using rapid-cycle methodologies. The primary outcome measure was reduction in codes per 1000 patient days. Secondary outcomes were days between codes and change in patient safety culture scores.
RESULTS: Code rate for the collaborative did not decrease significantly (3% decrease). Twelve hospitals reported additional data after the collaborative and saw significant improvement in code rates (24% decrease). Patient safety culture scores improved by 4.5% to 8.5%.
CONCLUSIONS: A complex process, such as patient deterioration, requires sufficient time and effort to achieve improved outcomes and create a deeply embedded culture of patient safety. The collaborative model can accelerate improvements achieved by individual institutions.
- AHRQ HSOPS —
- Agency for Healthcare Research and Quality’s Hospital Survey on Patient Safety Culture
- CHCA —
- Child Health Corporation of America
- CI —
- confidence interval
- MET —
- medical emergency team
- RRT —
- rapid response teams
Serious harm resulting from failures in patient care, although relatively rare at individual institutions, is a significant public health care issue. Analysis of data collected by the Child Health Corporation of America (CHCA, Shawnee Mission, KS) from 2003 to 2006 revealed events involving “failed escalation of care” accounted for 16% of all events reported by the 19 contributing hospitals (Children’s Hospitals Advancing Patient Safety, unpublished shared sentinel event database, 2003-2006).1 Although infrequent in pediatric inpatients, cardiopulmonary arrest requiring cardiac compressions for pulselessness has a survival rate of only 27%; survival increases to 80% if patient deterioration is recognized earlier. Additional analysis of event characteristics uncovered missed opportunities to recognize and intervene before cardiopulmonary arrest. These data together with published research demonstrating that adult patients experience signs of deterioration hours before they arrest1,2 suggest that recognizing deterioration earlier in a patient’s hospital course could reduce the frequency of codes and improve outcomes.
Two recent important strategies to prevent codes have been the use of early warning systems (or in pediatrics, pediatric early warning systems) and the development of rapid response teams (RRTs; also called medical emergency teams [MET]). Early warning systems are intended to improve recognition of patient deterioration with the use of an easily applied scoring system to more objectively identify changes in patient status.3–6 RRTs first emerged in the early 1990s in Australia and typically comprise ICU physicians, nurses, and respiratory therapists who bring critical care expertise to the bedside of a patient in a general care unit.
In addition to effective clinical interventions, understanding factors influencing an organization’s patient safety culture is important in reducing medical errors.7 Evidence from other high-reliability industries, such as aviation, shows an association between a positive safety culture and reduced errors.8 The Agency for Healthcare Research and Quality’s Hospital Survey on Patient Safety Culture (AHRQ HSOPS)9 assesses hospital staff perception of patient safety issues, medical error, and event reporting. In 2007, the lowest positive response categories within pediatric units were “Nonpunitive Response to Error” (43% positive responses) and “Handoffs and Transitions” (45% positive responses),10 highlighting the need for improvement in these areas.
This report describes CHCA’s multidisciplinary improvement collaborative of 20 children’s hospitals, which implemented a suite of prevention, detection, and correction strategies on targeted inpatient units with the stated aim of reducing the number of inpatient pediatric cardiopulmonary arrests (referred to in this article as codes) by 50% and improving the culture of patient safety scores by 5 percentage points in each of 3 key domains (Nonpunitive Response to Error, Handoffs and Transitions, and Communication Openness, chosen as a third category that best fit with our change package).
Twenty CHCA hospitals participated in a 12-month improvement project to “eliminate codes and associated mortality on inpatient units” (Group A). This collaborative was offered to all 42 CHCA owner hospitals. Each participating hospital determined if internal review board assessment was necessary and, if appropriate, obtained local approval or waiver before project initiation. The purpose of this collaborative was to establish reliable systems that rescue the deteriorating patient, focusing on 3 key change areas: prevention, detection, and correction. See Supplemental Information for change package used during the collaborative. Each hospital assembled a multidisciplinary team with designated administrative and/or physician sponsors. Each team identified “target” units (typically 1–3 units per hospital) from among noncritical care inpatient units, emergency department/emergency departments, operating rooms, and ICUs. Four hospitals focused on all noncritical care units during the project.
The collaborative was designed by a multidisciplinary pediatric advisory panel, including participating hospital staff and external subject matter experts. A comprehensive, pediatric-specific change package of practices with evidence supporting their efficacy, low risk of harm, and feasibility of implementation and measurement was developed. The interventions focused on change in practice to improve prevention, detection, and intervention of the deteriorating patient (Table 1). Three categories of changes with increasing complexity were identified. Foundational changes were relatively simple to implement and recommended to be put into practice early in the collaborative (eg, implementing “SBAR,” or Situation, Background, Assessment, Recommendation). Midlevel changes, such as developing a RRT, were implemented as the foundational changes were accomplished. Advanced changes (eg, family activation of the RRT) were considered more complex and were generally implemented once several other change types had been achieved. Teams were instructed to select a broad range of change package elements for implementation.
Approach and Organization of the Collaborative
The collaborative process was based on the Model for Improvement, which emphasizes small tests of change (Plan-Do-Study-Act), as developed by Associates in Process Improvement and adopted by the Institute for Healthcare Improvement.11 The process included a well-defined aims statement, 3 face-to-face learning sessions, communication strategies (eg, monthly conference calls, collaborative listserv, and project Web page), and monthly data submission. Hospitals were free to apply additional improvement methods used in their facilities, such as Six Sigma.12 On a monthly basis, participants reported their project measures to CHCA through a secure Web-based data repository hosted by Institute for Healthcare Improvement.
The aims of the collaborative were to reduce the rate of codes by 50%, double the days between codes, and improve the patient safety culture scores by 5 percentage points in the target units. The primary outcome measure was reduction in codes per 1000 patient days, with the secondary outcomes being the days between codes and change in patient safety culture scores (see Supplemental Information). Three process measures were also collected: RRT response time compliance, RRT activations per 1000 patient days, and RRT activation before a code. A goal of 95% compliance with RRT response time compliance was established. Each hospital set its own expected response time, typically 15 minutes.
Data Reporting to the Collaborative
Hospitals agreed to collect and submit monthly data during the study period of July 1, 2007 through June 30, 2008. Additionally, baseline data from the preceding 12 months (July 1, 2006–June 30, 2007) regarding codes and unplanned transfers to a higher level of care and from the preceding 3 months (April 1, 2007–June 30, 2007) for the process measures were collected from each hospital as baseline. Approximately 18 months after the collaborative action period concluded, a postcollaborative survey was conducted to collect an additional 12 months of code data. Twelve of the original 20 hospitals submitted postcollaborative data (Group B). Definitions of a “code” and an “unplanned transfer” were not standardized across institutions; each institution used their existing definitions throughout the data collection period.
Patient safety culture was measured via the AHRQ HSOPS and focused on 3 domains: (1) communication openness, (2) handoffs and transitions, and (3) nonpunitive response to error. The survey was conducted 3 times during the project: at the beginning of the project, midproject, and at the conclusion of the action period. Although only 3 domains were analyzed for the collaborative, teams were instructed to conduct the entire survey to minimize potential bias.
Data are expressed as median with interquartile ranges and 95% confidence intervals. Mann-Whitney was used for nonparametric tests with an exact P < .05 considered statistically significant. Paired t test was used for AHRQ survey data, with an exact P < .05 considered statistically significant. Analyses were performed utilizing Minitab version 15 (Minitab, State College, PA).
Before joining the project, hospitals’ efforts related to patient deterioration focused primarily on various preventive practices, review of hospital cases, staff training, RRT, and improvement in the chain of command process (Table 1). Some of the most widely implemented change areas during the collaborative were in the use of staff training and competency in recognition of deterioration and response algorithms. The most dramatic change implemented was the use of pediatric early warning system, starting out in no hospitals and implemented in 92% of hospitals within 12 months of the end of the collaborative period. Table 1 describes when practices were initially implemented; many practices that were in place before the collaborative began were actively refined or improved during the collaborative, such as established mock code processes.
The primary outcome measure was reduction in codes outside the ICU per 1000 patient days. The change in median code rate did not reach statistical significance for Group (difference in rate 0.01, 95% confidence interval [CI]: –0.05, 0.16, P = .284; Fig 1, Table 2). For Group B, the decrease in median code rate was statistically significant from baseline performance to action period performance (difference in rate 0.10, 95% CI: 0.00–0.31, P = .039; Fig 2, Table 2). Group B had a higher pooled baseline median code rate than the 8 teams not reporting postcollaborative data, although this did not reach statistical significance (P = .066); Group B also had a lower pooled median code rate during the action period, although this again did not reach statistical significance (P = .399).
Although 75% of the hospitals in each of these groups began the collaborative with an existing RRT, there were differences noted between Groups A and B in RRT implementation during the project. In Group B, 100% had a RRT in place by the end of the action period. At the conclusion of the action period in the group of 8 hospitals not reporting postcollaborative data, no additional hospitals had implemented a RRT.
Patient safety culture scores improved in all 3 targeted domains of the AHRQ HSOPS for the 14 hospitals (70%) that conducted the survey (Fig 3). When comparing the baseline and final surveys, the domains improved between 4.5 and 8.5 percentage points compared with the collaborative goal of 5 percentage points. The only statistically significant improvement was seen in “nonpunitive response to error” (39% positive response baseline, 47% positive response postcollaborative, P = .02). The remainder of the survey improvements were not statistically significant.
Our report of this multicenter collaborative process improvement initiative to reduce codes demonstrates mixed results and failed to achieve the a priori goal of a 50% reduction in codes after 1 year. Across all 20 hospitals, a modest 3% decrease in the median code rate was realized during the 1-year implementation period (Table 2). Interestingly, the variability of the code rates across the collaborative dramatically increased during the action period, suggesting that some institutions saw meaningful improvements but others did not. In an effort to explore this variability and obtain follow-up data, Group B was evaluated separately, and its median code rate declined by 24.4% (interquartile range: 0.10–0.54; P = .04) during the action period and 34.1% from baseline (interquartile range: 0.02–0.63; P = .06) during the following year.
Convincing data exist that survival to discharge for inpatient codes is dismal.13 It seems nearly as convincing that the rates of such events can be reduced, and the most dramatic evidence has been published in the context of RRT implementation in several children’s hospitals. The inpatient code rate declined by 59% (0.27–0.11 per 1000 patient days) at Cincinnati Children’s Hospital Medical Center after implementation of a medical emergency team.14 At Lucile Packard Children’s Hospital, a 71.7% reduction in code rate was realized following RRT implementation; this was accompanied by an 18% reduction in hospital-wide mortality.15 At the Royal Children’s Hospital in Melbourne, Australia, the inpatient “preventable” code rate also declined by 65%, accompanied by a dramatic 45% drop in total hospital mortality, after implementation of a MET.16
The variability of our results is predictable for several reasons. First, each hospital was starting from a different place along a continuum of existing systems, and each then implemented different elements of the change package to varying degrees. In addition, each hospital used their own internal definition of a “code”; depending on the definition, some may have been more preventable than others. The interventions employed were process improvement/systems changes and were likely implemented with varying levels of zeal and acceptance at different institutions. Indeed, the collaborative monthly progress reports ranged from the equivalent of barely getting started to successful completion of all the institution’s goals and objectives. Tempting as it may be to try to correlate institutional progress with the collaborative change package implementation success and outcomes (ie, change in code rates), the variability across institutions was too great to perform such analysis. Likely attributable to each institution’s own change in safety culture during the period of analysis, no consistent pattern of change package implementation was seen in either the top- or bottom-performing hospitals. Certainly significant changes were seen in 1 of the AHRQ survey questions targeted for improvement during this collaborative, whereas the other 2 domains trended toward improvement. This represents important improvement in a relatively short time frame.
Multicenter collaborative process improvement initiatives have demonstrated meaningful results for focused interventions. Pronovost et al demonstrated near elimination of catheter-associated bloodstream infections across 108 ICUs in Michigan.17 Miller et al showed a 43% reduction in catheter-associated bloodstream infections in a 29 PICU collaborative.18 Similarly, Wirtschafter et al were able to reduce central line–associated bloodstream infections by 25% across 13 neonatal ICUs.19 In addition, CHCA has a track record of successful multicenter process improvement collaboratives, demonstrating reduction in catheter-associated bloodstream infections in 26 PICUs20 and narcotic-related adverse drug events across 14 hospitals.21 As complex and difficult to eradicate as these infections and adverse drug events have been, the pathophysiology and the implementation strategies involved in these collaboratives were relatively straightforward. Without intent to diminish the tremendous results, all involved “bundles” of behaviors, short checklists, or discrete process improvements to reduce the risk of infections or errors. Assuming compliance with these behaviors can be monitored and enforced, particularly during the active study phase, the results can be observed in a short time frame.
Preventing codes, in comparison, is far more complicated, and our change package introduced interventions that represent upheavals in care paradigms. Indeed, some of these care paradigm changes are truly cultural shifts. A nurse “going over the head” of a resident who is not responding to a deteriorating patient properly and calling the attending or activating an RRT on his or her own is an example of cultural shift. When residents acknowledge they need help in the middle of the night and call the attending for advice, this, too, represents a cultural change. However, culture change in any field, particularly health care, takes time and a concerted and consistent focus. The implementation of a new, multifactorial way of monitoring patients cannot possibly be performed consistently and effectively in a short time frame, even for 1 institution, not to mention 12 or 20.
As noted earlier, the 1 systems change process that has, at least in before and after retrospective studies, been shown to reduce codes is the RRT. Of more than passing interest, then, must be the recognition that 75% of our participating institutions reported an RRT in place at the beginning of the collaborative. Depending on how effective such implementations were at each center, it is possible that little additional “code-reducing” behaviors could be exacted from these hospitals. On the other hand, just having an RRT in place does not mean it is being used effectively. The 1 cluster randomized controlled trial of an MET (in adults) was unable to show a crude difference in code rates between hospitals randomized to develop a MET and those without one.22 However, when data from this study were retrospectively analyzed, it was clear that as the number of MET calls increased, inpatient code rates decreased.23 Many hospitals assigned to implement a MET simply were not calling the team as often as necessary.
Allowing extrapolation from the median code rate change from the group of 12 hospitals during the postcollaborative period, for a hypothetical average-sized children’s hospital with 225 licensed beds and 50 000 patient days per year, this would represent 7 to 8 codes averted annually. If the mortality rate after inpatient pediatric codes is ∼75%,1 then 5 to 6 fewer deaths per year might be expected for such an institution. Of course, this is speculative but entirely consistent with the existing literature on prevention of codes and mortality.
Measuring 1 aspect of patient safety culture, the improvement seen in the AHRQ HSOPS was modest but in most domains met the goal of 5 percentage point increase recommended by AHRQ. A culture of patient safety has been clearly linked to improved patient outcomes including recently in the highest risk environment of the ICU. For each 10% reduction in “ICU perceptions of management” percent-positive score, the odds ratio for hospital mortality was 1.24 (95% CI: 1.07–1.44; P = .005) across 30 adult ICUs.24
Our time frame merits discussion in reference to changing patient safety culture, because sufficient time must be allowed to enable deeply embedded culture to change. There is evidence to support this notion with regard to the effects of the RRT. Santamaria et al report on code rates at a single institution up to 4 years after MET introduction.25 Their code rates did not drop significantly (45%) until 2 years after MET implementation but then declined an additional 42% after 2 more years. This may explain the results of our collaborative for the 12 hospitals that submitted data for an additional year. Of course, it may also be that these hospitals were fundamentally different in their ability to implement elements of our change package successfully; being interested and able to submit data 1 year later may be evidence of such a commitment.
Other limitations exist. This was not a randomized controlled trial. There were no data monitors or site visits to ensure compliance. Different institutions implemented different interventions, in all likelihood with differing levels of enthusiasm. However, the process improvement model, as noted earlier, has produced results in other critical areas of inpatient care that dwarf those of scientifically unassailable studies of new medications and expensive devices.
Certainly included in the role of health care providers is the goal of continually improving the care we provide our patients. The ultimate goal is preventing errors, such as “failed escalation of care.” A multifactorial process, such as codes after patient deterioration, likely requires a multidimensional approach to prevention. Changing patient safety culture is an ongoing process that is closely tied to improved patient outcomes. Not only is measuring changes in rates of relatively rare events challenging, maintaining sustained control of such improvements requires continual attention.
- Accepted October 10, 2011.
- Address correspondence to Leslie W. Hayes, MD, Children’s Hospital of Alabama, 1600 7th Ave South, ACC Suite 504, Birmingham, AL 35233. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
- Reis AG,
- Nadkarni V,
- Perondi MB,
- Grisi S,
- Berg RA
- Pronovost PJ,
- Weast B,
- Holzmueller CG,
- et al
- ↵AHRQ. Surveys on Patient Safety Culture. Available at: www.ahrq.gov/qual/patientsafetyculture. Accessed January 6, 2012
- ↵AHRQ Hospital Survey on Patient Safety Culture. 2007 Comparative Database Report. Available at: www.ahrq.gov/qual/hospsurveydb/hospsurveydb1.pdf. Accessed January 6, 2012
- ↵Institute For Healthcare Improvement. How to improve. Available at: www.ihi.org/knowledge/Pages/HowtoImprove/default.aspx“www.ihi.org/knowledge/Pages/HowtoImprove/default.aspx Accessed January 6, 2012
- ↵Schoeder RG. 2000. Six Sigma quality improvement: what is Six Sigma and what are the important implications? In: Proceedings of the Fourth Annual International POMS Conference; August 27–September 1; Seville, Spain
- Donoghue AJ,
- Nadkarni VM,
- Elliott M,
- Durbin D,
- American Heart Assocation National Registry of Cardiopulmonary Resuscitation Investigators
- Brilli RJ,
- Gibson R,
- Luria JW,
- et al
- Miller MR,
- Griswold M,
- Harris JM II,
- et al
- Huang DT,
- Clermont G,
- Kong L,
- et al
- Copyright © 2012 by the American Academy of Pediatrics