Implementation of a Multicenter Rapid Response System in Pediatric Academic Hospitals Is Effective
OBJECTIVES: This is the first large multicenter study to examine the effectiveness of a pediatric rapid response system (PRRS). The primary objective was to determine the effect of a PRRS using a physician-led team on the rate of actual cardiopulmonary arrests, defined as an event requiring chest compressions, epinephrine, or positive pressure ventilation. The secondary objectives were to determine the effect of PRRSs on the rate of PICU readmission within 48 hours of discharge and PICU mortality after readmission and urgent PICU admission.
METHODS: A PRRS was developed, implemented, and evaluated in a standardized manner across 4 pediatric academic centers in Ontario, Canada. The team responded to activations for inpatients and followed patients discharged from the PICU for 48 hours. A 2-year, prospective, observational study was conducted after implementation, and outcomes were compared with data collected 2 years before implementation.
RESULTS: After PRRS implementation, there were 55 963 hospital admissions and a team activation rate of 44 per 1000 hospital admissions. There were 7302 patients followed after PICU discharge. Implementation of the PRRS was not associated with a reduction in the rate of actual cardiopulmonary arrests (1.9 vs 1.8 per 1000 hospital admissions; P = .68) or PICU mortality after urgent admission (1.3 vs 1.1 per 1000 hospital admissions; P = .25). There was a reduction in the PICU mortality rate after readmission (0.3 vs 0.1 death per 1000 hospital admissions; P = .05).
CONCLUSION: The standardized implementation of a multicenter PRRS was associated with a decrease in the rate of PICU mortality after readmission but not actual cardiopulmonary arrests.
WHAT'S KNOWN ON THIS SUBJECT:
Single-center studies have revealed the effectiveness of pediatric rapid response systems, including decreases in cardiopulmonary arrests, respiratory arrests, and the incidence of preventable cardiac arrests.
WHAT THIS STUDY ADDS:
This study from the Ontario Pediatric Critical Care Response Team Collaborative is unique in that it is the first multicenter, prospective, observational study describing the implementation of a pediatric rapid response system using a physician-led medical emergency team.
Delayed treatment of evolving critical illness in pediatric ward patients is a source of preventable morbidity and mortality. Late treatment of clinical deterioration is common in hospitalized patients1 and results in greater severity of illness at ICU admission and increased ICU mortality.2,3 Use of rapid response systems is a strategy recommended to improve patient outcomes through early identification and management, by specialized teams of critical care trained professionals, of those ward patients whose condition is deteriorating.4 Although the only adult, randomized, multicenter, rapid response system trial cast doubt on its effectiveness,5 single-center rapid response system studies have demonstrated improved outcomes in adult6 and pediatric5,7,–,13 hospitals. Similarly, a systematic review and meta-analysis of rapid response teams showed a reduction in cardiopulmonary arrests outside of PICUs and hospital mortality.14
Acutely ill children have an innate ability to compensate, making evolving critical illness difficult to recognize and diagnose. Unplanned PICU admission is associated with increased risk of mortality. Mortality in children admitted to the PICU from hospital wards is 1.65 times higher than children admitted from emergency departments.15 Similarly, children readmitted to the PICU during the same hospitalization are at greater risk of death compared with those with a single PICU admission.16,17 Finally, cardiopulmonary arrest in children is associated with poor outcome (15%–48.7% survival to hospital discharge).18,–,21
In 2006, the Ministry of Health and Long-term Care (MOHLTC) in Ontario, Canada, funded the development and implementation of a pediatric rapid response system (PRRS) at 4 academic pediatric hospitals as part of their critical care strategy to improve access and quality of critical care resources.22 This multicenter, prospective, observational study from the Ontario Pediatric Critical Care Response Team Collaborative describes the implementation of a PRRS using a physician-led medical emergency team (MET). Ontario has a population of >13 million people, and all critically ill children requiring PICU admission are admitted in 1 of 5 pediatric centers. The objectives of the present study were to determine the effect of a PRRS on clinically important outcomes in children and to describe MET activity.
A prospective observational study design was used to collect data during this 2-year study. Prospective data were compared with data collected retrospectively for the 2 years before PRRS implementation. Our objective was to describe MET activity and determine the effect of the PRRS on hospital code blue events, PICU readmission rate, all-cause hospital mortality, and PICU mortality after urgent PICU admission and PICU readmission. Approval was obtained from the research ethics board at each site.
The MOHLTC funded the 4 level 3 academic pediatric hospitals in Ontario to develop and implement the PRRS. Level 3 hospitals are defined as capable of providing the highest level of care.23 Each hospital is university affiliated, accredited for training by the Royal College of Physicians and Surgeons of Canada, and has a pediatric critical care fellowship program. All PICUs across the 4 sites are closed; accredited pediatric intensivists are responsible for all management and care decisions with subspecialty services providing consultation only. No in-house hospitalists are present 24 hours/day, 7 days/week at any center. There were no other major system level changes during the study period.
The standardized development and implementation of the PRRS occurred in 3 phases. During the 6 months of phase 1 (May 1, 2006, to October 31, 2006), the multidisciplinary site leaders reached consensus regarding delivery of the PRRS service and developed promotion and education strategies. In addition, a data collection tool was developed. The PRRS was piloted during phase 2 (October 31, 2006, to January 29, 2007), Monday through Friday, 8:00 pm to 4:00 pm, during which time user groups completed satisfaction surveys used to refine the team's delivery of service. The data collection tool was also used and MET providers suggested revisions to the tool. On January 29, 2007, phase 3 began with the PRRS available 24 hours/day, 7 days/week.
Delivery of Service
Our PRRS was composed of a physician-led MET and included a PICU physician (PICU attending and fellow/resident during the day and a PICU fellow/resident overnight with attending backup), a critical care nurse, and a respiratory therapist. The RN MET providers were dedicated to the team with no other patient care responsibilities, day and night. The MD MET providers were dedicated to the team during the day but at night were also responsible for PICU patients. The MET was activated by calling a dedicated pager for any ward patient, and it replaced PICU consultations for inpatients. The MET did not respond to calls regarding patients in the emergency department, operating room, postanesthetic care unit, or NICU. The PRRS was 2-tiered, and MET activations and code blue calls were distinct. The MET was also part of the code blue team.
Any health care provider (HCP) could activate the MET day or night if his or her patient met calling criteria. All centers used the age-specific, physiologic criteria described by Tibballs et al9 excluding cardiac or respiratory arrest (Table 1). At each hospital, family members could activate the MET through their primary nurse or physician; at 1 test center, families were permitted to activate the team directly. The patient's primary physician was called at the time of MET activation. When activated, the MET arrived at the patient's bedside within 5 minutes, and their assessment involved participation of the HCP who activated the team and the patient's primary nurse and physician. The MET determined whether the patient required PICU admission. Patients who required PICU admission remained in the care of the MET until PICU admission. For patients who remained on the ward, the MET provided suggestions regarding diagnostic investigations, medical interventions, and scheduled follow-up visits that monitored the patient's progress with the primary medical team. Patients were followed up by the MET until the primary medical team and the MET agreed that the patient no longer required such follow-up.
The MET also followed up patients discharged from the PICU for 48 hours, with visits occurring once in 24 hours or more frequently, if requested by the HCP on the ward.
Data regarding hospital admissions and mortality were extracted from hospital administrative databases. PICU data were extracted from PICU administrative databases that collect PICU admissions, readmissions, mortality, and Pediatric Risk of Mortality (PRISM) III scores contemporaneously. Data were extracted from October 31, 2004, for 2 years before the PRSS implementation and from January 29, 2007, for 2 years after the PRRS implementation. Data from pilot activity during phase 2 were excluded.
A standardized data collection tool and database were developed by a group of experts to track MET activity. MET activity and outcome definitions were defined in a standardized way across all 4 sites (Table 2). For all activations, the indication, time, and activating HCP were recorded. For all MET activity, the patient's disposition, PICU survival, and PRISM III score was collected. All code blue event activity was collected, including MET involvement before the event and interventions required during the event.
Our primary outcome measure was the rate of total code blue events, defined as any activation of the code blue system. In our hospitals, the code blue system is activated if, in the judgment of an HCP, immediate assistance from the resuscitation team or equipment is required. Outcome of these code blue system activations includes both actual and near cardiopulmonary arrests. Actual cardiopulmonary arrests require treatment with chest compressions or positive pressure ventilation for >30 seconds or intravenous epinephrine. Near cardiopulmonary arrests do not require these treatments.
Before the implementation of the PRRS, 2 centers did not collect data regarding treatment received during code blue events; therefore, these data were not used in the subgroup analysis of actual and near cardiopulmonary arrests.
The secondary outcome measures included PICU readmissions and PICU mortality after urgent PICU admissions, and PICU readmission. All-cause hospital mortality data were also collected.
Across the 4 centers, there were 140 urgent PICU admissions without MET involvement. These patients were admitted into the PICU directly by the intensivists during the early part of phase 3. These patients were included in the analysis as urgent PICU admissions.
Descriptive statistics were calculated for the PRISM III scores. Student's t test was used to compare the PRISM III scores of unplanned PICU admissions before and after PRRS implementation. Poisson regression analysis was used to compare the rates of total code blue events, actual and near cardiopulmonary arrests, and PICU readmissions within 48 hours of PICU discharge, all-cause hospital mortality, and mortality after urgent PICU admission and PICU readmission. Adjustment was made for potential differences between hospitals by including “hospital” as a variable in the regression model. When significant differences were identified, additional exploratory analysis assessed for interactions between hospitals as well as for before and after effects. Analyses were performed using SAS 9.1 (SAS Institute Inc, Cary, NC). Relative rate ratios (RRRs) are reported with 95% confidence intervals (CIs). A P value of ≤.05 was considered statistically significant.
There were 55 469 hospital admissions and 7068 PICU admissions for the 2 years before PRRS implementation. During the 2 years after PRRS implementation, there were 55 963 hospital admissions and 7227 PICU admissions. Severity of illness was compared between the 2 eras for all patients admitted into the PICU from the wards using PRISM III scores. During the era before PRRS implementation, the mean PRISM III score was 8.1 (SD: 6.6; median: 6 [interquartile range: 3–11]); after PRRS implementation, the mean PRISM III score was 7.8 (SD: 6; median: 6 [interquartile range: 3–11]). The rate of PICU admission from the wards did not change significantly before compared with after PRRS implementation —17 versus 18 per 1000 hospital admissions (RRR: 0.7 [95% CI: 0.6–0.9]); P = .19).
The MET received 2476 new activations and followed up with them for 6230 visits. There were 44 new activations per 1000 hospital admissions. Overall, 30% of all activations led to an unplanned PICU admission (Table 3).
The main indications for activating the MET were respiratory (46%), cardiovascular (21%), HCP concern (18%), neurologic (11%), and other (3%). Nurses were the most common health care providers to activate the MET (57%), followed by physicians (37%), respiratory therapists (2%), family (1%), and other (3%). The MET was most commonly activated between 7:00 am and 7:00 pm (55%). The MET followed up 7300 PICU discharges for 48 hours (15 031 visits).
The rate of total code blue events significantly decreased between the 2 eras: 4 events before versus 3 events after PRRS implementation (RRR: 0.71 [95% CI: 0.61–0.83]; P < .0001). There was a significant decrease in the rate of near cardiopulmonary arrests: 3.4 before compared with 1.9 events after PRRS implementation (RRR: 0.54 [95% CI: 0.52–0.57]; P < .0001) (Table 4). There was no difference in the rate of actual cardiopulmonary arrests: 1.9 before and 1.8 events after PRRS implementation (RRR: 0.95 [95% CI: 0.76–1.96]; P = .68). After PRRS implementation, 41% of all code blue events had MET involvement before the event. Of those, 35% were actual cardiopulmonary arrests and 48% were near cardiopulmonary arrests.
Significant differences were found between hospitals with respect to PICU readmission rate. Subsequent analysis found a significant interaction between hospitals and before and after effect. Two centers demonstrated a significant decrease, 1 center demonstrated a significant increase, and the fourth center showed no change in the rate of PICU readmission within 48 hours of discharge in the after-PRRS era compared with the before-PRRS era.
There was a significant decrease in the PICU mortality rate after PICU readmission, with 0.3 death before PRRS implementation compared with 0.1 death after PRRS implementation (RRR: 0.43 [95% CI: 0.17–0.99]; P < .05) (Table 4). There was no change in the rate of PICU mortality after urgent PICU admission between the 2 eras: 1.3 before PRRS implementation compared with 1.1 deaths after PRRS implementation (RRR: 0.9 [95% CI: 0.7–1.0]; P = .25) (Table 4). There was no change in the rate of all-cause hospital mortality between the 2 eras: 10 before PRRS to 9.6 after PRRS implementation (RRR: 0.97 [95% CI: 0.83–1.12]; P = .65). Hospital was included as a variable in the regression model to guarantee that any before and after outcome differences were not because of hospital variations. No interaction between hospitals and before and after effect was found during the regression analysis.
This multicenter, prospective, observational study from the Ontario Pediatric Critical Care Response Team Collaborative describes the implementation of a PRRS using a physician-led MET. This unique multicenter study revealed an association between implementation of a PRRS and pediatric outcomes.
To date, several single-center studies have demonstrated the effectiveness of PRRS. Some researchers have shown a significant decrease in cardiopulmonary arrests,11,–,13 and Hunt10 reported a significant decrease in respiratory arrests. Most recently, Tibballs and Kinney8 published a follow-up report of the 2005 article of Tibballs et al,9 revealing a significant decrease in the incidence of “preventable cardiac arrests” and all-cause hospital mortality after the implementation of a PRRS but no significant reduction in the rate of cardiac arrests. All of these single-center studies have demonstrated improvement in some aspect of patient care, but the data are difficult to interpret because of the small numbers and the variation of outcomes measured.
Our study found that after PRRS implementation, there was a 46% reduction in the rate of near cardiopulmonary arrests but no change in actual cardiopulmonary arrest rate. This is the first report, to the best of our knowledge, which describes using the MET to follow-up patients discharged from the PICU. The introduction of the MET was associated with a 57% reduction in PICU mortality after PICU readmission, but it had no effect on mortality after urgent PICU admission. This suggests that, for high-risk patients, there may be a protective effect of being followed up by the MET.
The changing hospital culture is a major challenge to effective rapid response system implementation.6,24 The activation rate of our MET was higher than previously published studies, with 40 vs 2.8 to 11.9 activations per 1000 hospital admissions.9,–,12 This widespread adoption of the PRRS was likely the result of extensive promotion and education strategies described elsewhere.25 Only 30% of patients seen by the MET after an activation were admitted into the PICU compared with 42% to 57% quoted by others.9,11,12 Our lower PICU admission rate suggests that the PRRS was activated for less acutely ill children. Compared with other studies examining single-tiered systems,9,–,11 our PRRS was 2-tiered and encouraged early activation for 2 reasons. First, every MET activation was seen as an opportunity for continued promotion and “on-the-spot” teaching of HCPs. Second, given the dose response (MET activation rate) necessary to improve outcomes described by DeVita et al,6 we used broad calling criteria in an attempt to achieve the appropriate dose. Despite these broad calling criteria and activation rate, only 40% of children had MET activations preceding a code blue event. A pediatric early warning score26,27 with mandatory MET activation may be a more reliable method of identifying children at risk of an actual cardiopulmonary arrest. Conversely, the decrease in near cardiopulmonary arrest in our study may be due to HCPs activating the MET instead of the code blue system during the era after PRRS. This use of the MET instead of the code blue team allowed the resources of the code blue system to be used more appropriately.
Our study limitations are a consequence of studying a system-level intervention. Although there were no other major system changes during our study period, the improved outcomes may be due to the “side effects” of a PRRS and MET. The impact the MET had on HCP education and consequently improved recognition of evolving critical illness cannot be disregarded. Second, as in all other PRRS studies, our data reporting and definitions are inconsistent with the rest of the literature. Data in the PRRS literature are reported with a variety of denominators, including 1000 patient-days, 1000 patient discharges, and total hospital admissions. Similarly, outcome definitions, such as code blue events and preventable arrest, are not universal. Ideally, PRRS outcomes should be reported in a standardized format, such as the Utstein style revised in 2007 by Peberdy et al.28 Finally, it is difficult to draw any conclusions with regard to the effect of PRRS on PICU readmission 48 hours after discharge because the PRRS seemed to impact readmission differentially at different sites. However, this novel use of a PRRS did demonstrate a reduction in mortality after PICU readmission.
This multicenter study revealed an association between implementation of a PRRS and pediatric outcomes. This is the first report to describe using the MET to follow-up patients discharged from the PICU, and the results suggest that, for high-risk patients, being followed up by the MET may be protective.
The Ontario MOHLTC provided funding for the development and implementation of the PRRS. Also, Ms Gaiteiro was funded by the Ontario MOHLTC to produce biannual progress reports for the MOHLTC. The Ontario MOHLTC was not involved in data collection, analysis, or writing of the manuscript, and it did not have access to the study data.
We thank the Critical Care Secretariat and Robert McKay, PhD, Julie Trpkovski, RN, and Bernard Lawless, MD, MHSc, CHE, FRCSC.
- Accepted March 15, 2011.
- Address correspondence to Afrothite Kotsakis, MD, FRCP(C), Hospital for Sick Children, Department of Critical Care Medicine, 555 University Ave Toronto, Ontario, Canada M5G 1X8. E-mail:
Drs Kotsakis and Lobos had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Dr Parshuram performed major revisions and acted as the database development and database administrator. Provincial managers were Dr Bohn and Ms Gaiteiro. Data reporting was conducted by Ms Berdan, Helena Frndova, MEng, and Wendy Seidlitz, RN, BScN, MSc. Statistical analysis was conducted by Mr Atenafu, MSc, and interpretation of data was performed by Drs Kotsakis and Lobos.
Study concept and design and data collection were performed by the Ontario Pediatric Critical Care Response Team Collaborative; Drs Kotsakis, Lobos, Gilleland, Mohseni-Bod, Parshuram, Bohn, Singh; April Price, MD, FRCP(C), Jonathan Costello, MB, MRCPI, FCEM, MBA; and Gavin Morrison, MSc, MRCP; Christa Ramsay, RRT; Karen Laidlaw, RN, MN; Bonnie Molinski, RN, BScN; Janet VanLuttikhuizen, RN; Jennifer Watson, RN, BScN, CNCCP; Wendy Seidlitz, RN, BScN, MSc, and Ms Gaiteiro.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
- MOHLTC —
- Ministry of Health and Long-term Care
- PRRS —
- pediatric rapid response system
- MET —
- medical emergency team
- HCP —
- health care provider
- PRISM —
- Pediatric Risk of Mortality
- RRR —
- relative rate ratio
- CI —
- confidence interval
- Kause J,
- Smith G,
- Prytherch D,
- Parr M,
- Flabouris A,
- Hillman K
- Kanter RK,
- Edge WE,
- Caldwell CR,
- Nocera MA,
- Orr RA
- DeVita MA,
- Braithwaite RS,
- Mahidhara R,
- Stuart S,
- Foraida M,
- Simmons RL
- Tibballs J,
- Kinney S,
- Duke T,
- Oakley E,
- Hennessy M
- Hanson CC,
- Randolph GD,
- Erickson JA,
- et al
- Reis AG,
- Nadkarni V,
- Perondi MB,
- Grisi S,
- Berg RA
Ministry of Health and Long-term Care. Intensivist-led critical care response teams. Available at: www.health.gov.on.ca/english/providers/program/critical_care/ccrt_intens.html. Accessed July 2, 2010
Ministry of Health and Long-term Care. Critical care information system. Available at: www.health.gov.on.ca/english/providers/program/critical_care/cct_infosystem.html. Accessed July 2, 2010
- Peberdy MA,
- Cretikos M,
- Abella BS,
- et al
- Copyright © 2011 by the American Academy of Pediatrics