BACKGROUND AND OBJECTIVES: Few studies have demonstrated improvement in adherence to Pediatric Advanced Life Support guidelines for severe sepsis and septic shock. We sought to improve adherence to national guidelines for children with septic shock in a pediatric emergency department with poor guideline adherence.
METHODS: Prospective cohort study of children presenting to a tertiary care pediatric emergency department with septic shock. Quality improvement (QI) interventions, including repeated plan-do-study-act cycles, were used to improve adherence to a 5-component sepsis bundle, including timely (1) recognition of septic shock, (2) vascular access, (3) administration of intravenous (IV) fluid, (4) antibiotics, and (5) vasoactive agents. The intervention focused on IV fluid delivery as a key driver impacting bundle adherence, and adherence was measured using statistical process control methodology.
RESULTS: Two-hundred forty-two patients were included: 126 subjects before the intervention (November 2009 to March 2011), and 116 patients during the QI intervention (October 2011 to May 2013). We achieved 100% adherence for all metrics, including (1) administration of 60 mL/kg IV fluid within 60 minutes (increased from baseline adherence rate of 37%), (2) administration of vasoactive agents within 60 minutes (baseline rate of 35%), and (3) 5-component bundle adherence (baseline rate of 19%). Improvement was sustained over 9 months. The number of septic shock cases between each death from this condition increased after implementation of the QI intervention.
CONCLUSIONS: Using QI methodology, we have demonstrated improved adherence to national guidelines for severe sepsis and septic shock.
- ED —
- emergency department
- IQR —
- interquartile range
- IV —
- LOS —
- length of stay
- PALS —
- Pediatric Advanced Life Support
- PIM2 —
- Pediatric Index of Mortality Score, version 2
- QI —
- quality improvement
- SPC —
- statistical process control
Severe sepsis and septic shock are associated with high rates of mortality in children (10%), and death rates are even higher among those with comorbid conditions (12%).1,2 The American Heart Association, in 2010, updated the Pediatric Advanced Life Support (PALS) septic shock guidelines, although the standards for sepsis care have remained essentially unchanged for more than a decade.3–5 Five algorithm components include timely recognition of septic shock and vascular access within 5 minutes, and antibiotic and vasoactive agent delivery within 60 minutes.5 The fifth PALS time point recommends 60 mL/kg of intravenous (IV) fluids administered within 60 minutes of meeting the definition of septic shock.5,6
In adults, timely fluid resuscitation and antibiotic administration, components of early goal-directed therapy, have been shown to decrease mortality.7,8 Pediatric studies in the community, intensive care, and international settings have demonstrated that timely fluid resuscitation is associated with decreased morbidity and mortality.9–12 Three recent studies conducted in tertiary pediatric emergency department (ED) settings have demonstrated barriers to the care of children with sepsis, resulting in poor adherence to the rapid administration of IV fluids, vasoactive agents, and antibiotics.13–15
Preliminary Institutional Study
From November 2009 to March 2011, we conducted a prospective cohort study at Boston Children’s Hospital ED to evaluate adherence rates to the PALS guidelines for children with septic shock.15 We observed that adherence to all algorithm time points as a bundle was low (19%), with particularly low adherence to timely administration of IV fluids (37%) and vasoactive agents (35%).
Adjusting for severity of illness at presentation using the Pediatric Index of Mortality score, version 2 (PIM2),16 we observed in previous work that patients who received 60 mL/kg of IV fluid within 60 minutes had a 57% shorter hospital length of stay (LOS), and a 42% shorter ICU LOS. We also observed that adherence to the 5-step bundle was associated with a 57% shorter hospital LOS and a 59% shorter ICU LOS.15
There have been 2 studies in a tertiary care pediatric ED that examined adherence to national guidelines for patients with sepsis after a quality improvement (QI) intervention.13,14 These studies demonstrated improved time to delivery of 20 mL/kg of fluids but not 60 mL/kg of fluids per the PALS recommendation. Based on our obser vation that a minority of patients with septic shock at our institution received timely IV fluids, and that rapid fluid administration and bundle adherence were associated with improved outcomes, we instituted a QI intervention to improve adherence to PALS guidelines.
Study Design and Setting
This was a prospective cohort QI study for patients presenting to the ED with septic shock between November 2009 and March 2011 (preintervention period) and October 2011 and May 2013 (QI period). The ED treats ∼60 000 patients per year and has 15 pediatric emergency medicine fellows, 120 rotating pediatric residents, and 90 emergency medicine residents annually. This study was approved by the institutional review board at Boston Children’s Hospital.
Planning the Intervention
QI interventions were first initiated in October 2011. The improvement team consisted of 5 physicians from the pediatric ED and medical ICU, 3 nurses, 1 pharmacist, 1 information technology specialist, and 3 research assistants. A Pareto diagram was developed to determine the components of the bundle that were the most integral to improving adherence to the PALS sepsis guideline (Fig 1). We focused on the timely administration of IV fluid as a primary driver of bundle adherence, as adherence to this parameter was particularly low. An Ishikawa fishbone diagram was then developed with input from all stakeholders involved in the delivery of fluid, including nurses, clinical assistants, the phlebotomy team, pharmacists, and physicians (Fig 2). It was determined that use of an IV pump for fluid delivery was a major contributor to poor adherence, as IV pump characteristics limit the administration of 60 mL/kg of IV fluid within 60 minutes to children weighing ≤16 kg.17
Several plan-do-study-act cycles were implemented over the course of the intervention by using varying levels of reliability.18 Reliability is measured as the inverse of the system’s failure rate; a system that has a defect rate of 1 in 10, or 10%, performs at a level of 10−1 (a level 1 system).
The first cycle of interventions included 10−1 reliability measures. Level 1 systems have no common process and focus on training and retrospective reminders.18 We educated all stakeholders through every-other-month educational meetings, hospital-wide Internet-based learning modules, and skills days for nursing staff, emphasizing use of an IV fluid pressure bag. Weekly e-mails were sent to individual providers involved in the care of a specific patient with septic shock, detailing adherence to the 5 components of the sepsis bundle and providing recommendations if there were any observed barriers to timely care. Additionally, for each patient seen, a 10-question electronic survey was completed, allowing the caregiver to give feedback regarding barriers to care. Finally, midway through the intervention, we conducted a second department-wide staff meeting where group feedback was elicited and reference materials were revised.
Level 2 reliability strategies were incorporated during the first and subsequent plan-do-study-act cycles. Level 2 systems have intentionally designed tools that aim to error-proof a system, ensuring ease of performing in the desired standard way.18 The PALS algorithm with local modifications was posted prominently in the resuscitation rooms, charting areas, office areas, and break rooms. The algorithm, including details of care, specific definitions of septic shock, and dosing of medications, was included in a pocket card that was created and distributed to all members of the health care team. We also incorporated a standardized order set into the electronic medical record to assist providers at the bedside. This order set included guidance on IV fluid bolus volume and infusion time, recommended laboratory tests, and antibiotic and vasoactive agent choices. Every time the order set was used, a lightning bolt symbol automatically appeared on the ED tracking board, notifying nurses and ancillary staff of a patient likely to require additional personnel and resources (which became known as “firing the bolt”). This lightning bolt also helped to address a nursing concern over limited resources by alerting the charge nurse to redirect additional nursing resources to the bedside. The bolt also alerted the pharmacist that all medications and fluids were needed emergently for that patient.
Level 3 reliability strategies included redesigning the process through which a child with septic shock was managed. A large clock, the “shock clock,” was placed in all resuscitation areas, and a portable clock was located in the physician charting area, designed to be used in any nonresuscitation room. We instructed physicians or nurses to start the shock clock for any patient with concern for severe sepsis or septic shock. This visual cue allowed providers and parents to be aware of the 60-minute time goal to complete all the components of the algorithm. The shock clock was large and centrally located in the resuscitation areas, to allow for maximal visibility by the clinical staff during a critical care process. Based on feedback from physicians, a smaller clock was later introduced, to allow caregivers to feel more comfortable bringing it into the room.
Planning the Study of the Intervention and Evaluation
We evaluated the effectiveness of our interventions on a monthly basis, by using 5 process, 3 outcome, and 3 balancing measures. The process measures included adherence to the time goals for the 5 components of the algorithm. The primary outcome measure was adherence to the total algorithm bundle (all 5 components), and the secondary outcome was change in rates of mortality. Balancing measures included ED length of stay (for all patients) to evaluate whether the redirection of personnel and resources affected ED throughput. We also analyzed cases for which the lightning bolt was fired inappropriately (ie, for a patient not meeting our case definition for septic shock) and thus resources may have been excessively diverted.
Each month, patients with septic shock were identified by using admission logs from the ED to the ICUs, intermediate care unit, and bone marrow transplant and oncology units. In our institution, all patients who have septic shock are admitted to 1 of these units. Patients were classified as having either severe sepsis or septic shock by using definitions from the 2005 International Pediatric Sepsis Consensus Conference.19 Measures were then manually abstracted from the electronic medical record as described in our previous study.15 Appropriate recognition, within 5 minutes from the definition of septic shock, was determined by documentation of severity of illness, transfer to an ED resuscitation room, additional IV placement, or second fluid bolus initiation. Abstraction of time points was duplicated in 10% of patients by another physician to ensure reliability.15
We tracked all measures by using statistical process control (SPC) charts to allow for immediate detection of special cause variation that could be investigated. Control limits were set at 3 SDs from the mean. Multivariate logistic regression, controlling for severity of illness using the PIM2 score, was used to determine whether the use of an appropriate fluid-delivery apparatus (pressure bag, rapid-infuser, or manual push) was associated with improved adherence to the guidelines.16 We also ascertained how often laboratory tests were obtained to establish the diagnosis of septic shock, such as coagulation profiles, lactic acid, and liver function tests. χ2 tests were used to compare categorical variables. The t-test and Wilcoxon rank sum test were used to compare normally and non-normally distributed continuous data respectively; P < .05 was considered statistically significant. The Statistical Package for the Social Sciences, version 21.0 (IBM SPSS Statistics, IBM Corporation, Chicago, IL) and Stata version 11 (Stata Corp, College Station, TX) were used for statistical analysis.
Our cohort consisted of 242 patients: 126 patients with septic shock before the intervention, and 116 patients during the QI intervention. Overall, 47 (19%) had severe sepsis and 195 (81%) ultimately developed septic shock (Table 1). The median ED LOS for included children was 4 hours and 10 minutes (interquartile range [IQR] 2 hours 46 minutes to 5 hours 56 minutes). Thirty-one percent of patients met the definition for severe sepsis or septic shock at the time of triage in the ED. Among all patients, the median time to development of severe sepsis or septic shock was 52 minutes (IQR 0–140). There were no differences between the baseline and QI intervention cohorts with regard to PIM2 score (10.3 vs 8.2, P = .06), percent with comorbidities (60.0% vs 61.2%, P = .85), or age (9.2 vs 12.3 years, P = .14).
Percent adherence to process measures, including fluid and vasoactive agent delivery, is detailed in SPC charts (Fig 3). There was a significant improvement, initially attaining 80% adherence for the administration of IV fluids in February 2012, and 60% adherence for the administration of vasoactive agents in April 2012. These initial improvements were demonstrated after feedback e-mails and the process of “firing the bolt” were initiated. Further improvements were noted after the second group meeting generated the idea for use of a smaller, more portable shock clock and revised pocket reference cards. By August 2012, adherence to fluid and vasoactive agent time goals reached 100%. Adherence to these measures remained near 100% for the last 9 months of the study period. Median time to administration of IV fluid decreased from 83 minutes (IQR 43–145) in the baseline group, to 33 minutes (IQR 0–68) in the QI group. Additionally, median time to delivery of vasoactive agents decreased from 90 minutes (IQR 51–164) to 35 minutes (IQR 14–86). Adherence to other process measures is detailed in Table 2.
Adherence to the total PALS bundle showed initial significant improvement to 50% in December 2011 (Fig 4A), and reached 100% by September 2012. Sustainability was demonstrated for the last 9 months of the study period. Use of the appropriate fluid-delivery apparatus was associated with adherence to the fluid guidelines (odds ratio 4.8 [confidence interval 2.3–10.1]) and the total bundle (odds ratio 4.4 [confidence interval 2.1–9.1]). The number of cases of septic shock between each death from this condition increased, demonstrating significant outcome improvement (Fig 4B).
Overall ED LOS for all patients did not increase over the course of the study period. The septic shock pathway and “firing of the bolt” was inappropriately initiated in patients who did not meet criteria for septic shock in fewer than 5 cases per month on average.
Although national guidelines for pediatric septic shock have existed for more than a decade, several studies have shown that they have not been translated into practice.11–13 One ICU-based study conducted in the United Kingdom demonstrated 38% adherence to 2002 PALS guidelines, citing inexperience and a delay in the administration of vasoactive agents until central venous catheter placement as the major barriers to guideline adherence.11 Two pediatric EDs have attempted to improve care of patients with sepsis through QI interventions.13,14 One ED focused on improving awareness and early recognition of children with septic shock in addition to alleviating a barrier of limited resources.13 They demonstrated an improvement in the time to administration of IV fluids and antibiotics.13 Despite these gains, they fell short of guideline recommendations, with fluids and antibiotics being delivered after the 1-hour goal. Another pediatric ED also engaged in a successful QI endeavor, citing difficulties with recognition but also difficulties with vascular access as major barriers.14 This group demonstrated improved delivery of 20 mL/kg of IV fluids and antibiotics within 3 hours, but did not address the PALS recommendation to administer 60 mL/kg IV fluid and antibiotics within 60 minutes of recognition.
We demonstrated improved adherence to all 5 components of the PALS septic shock algorithm. We learned from lessons identified in previous studies and from the detailed stakeholder-driven knowledge gained at our own institution regarding the barriers of implementing the PALS sepsis guidelines.13–15 Rapid IV fluid administration was the most common barrier to guideline adherence and we observed that this was due to the use of an IV pump in most cases. Much of our intervention focused on educating clinicians regarding the limitations of this method of fluid administration. As predicted from reliability theory, education alone was not effective in improving our process.18 Level 2 and 3 reliability strategies, such as “firing the bolt” and individual feedback e-mails were needed to achieve statistical improvement in all process and outcome measures. By the end of the study period, as evident from the SPC chart (Fig 4B), there was an increase in the number of cases of septic shock between each death, possibly associated with our improvement measures. This will need to be followed over time to determine if there is sustained reduction in mortality. More than half of the patients did not meet the definition of septic shock on arrival in the ED, but rather, had their condition evolve over the course of their ED stay. This demonstrates the importance of continual monitoring of vital signs and frequent reassessment of a patient’s clinical status throughout their ED course to be able to identify more subtle presentations of sepsis.
Through continual assessment and feedback from clinicians, we became aware of physician hesitancy in using the larger shock clock, particularly at a time when a child needed acute care and resources. Based on this feedback, we introduced a smaller clock, yet still placed it in a strategically apparent location, ultimately resulting in dramatically improved measure adherence. We also recommended that the clock count forward rather than being a “countdown” clock, because of parental concern and perception of time “running out.” However, when we accommodated providers’ concern further by allowing caregivers to not use the clock, but rather use a sepsis time-out at the bedside (whereby the time of definition of septic shock was verbally announced), we saw a decrease in adherence to all measures. Within a month, we again returned to use of the shock clock and subsequently returned to previous levels of adherence. This highlights the fluid nature of improvement, and the continual need for feedback mechanisms and real-time monitoring of process-improvement efforts.
After detailed analysis of preimprovement practice and a Pareto diagram, we were able to focus on the key interventions that drove bundle adherence, rather than diverting resources toward remediating all barriers. From the SPC charts (Figs 3A and 4A), we confirmed our hypothesis that use of the appropriate fluid-delivery apparatus was indeed significantly associated with fluid adherence and thus bundle adherence; the fluid-adherence SPC chart strongly mirrors that of total bundle adherence. This was supported through traditional logistic regression analysis as well.
We also aimed to minimize the impact on other aspects of health care delivery by using balancing measures. We recognized that allocation of resources toward a child with sepsis could affect the delivery of care for other children in the ED. We observed no change in ED LOS during the implementation of this QI intervention. In addition, there was minimal use of the septic shock pathway for patients who did not meet sepsis criteria throughout the study period.
Feasibility is a key component of a successful QI intervention. This study, although having the support of the institution, did not require significant additional resources or personnel. We engaged nursing staff already available in the ED to achieve timely care for patients with septic shock. All educational outreach, feedback, and data collection were performed by existing frontline workers. This has positive implications for institutions where resources may be limited.
This work has several limitations. Definitions of septic shock were based on International Sepsis Consensus Conference guidelines.18 These can be cumbersome to apply in real time at the patient’s bedside and in record review to evaluate for guideline adherence. However, these are the most rigorous criteria that exist for children with sepsis, and other strategies for identification, such as International Classification of Diseases coding, have underestimated the true prevalence of septic shock.1,20 Second, without randomization, we were unable to ascertain whether improvements over time were due solely to our interventions. Regardless, institutional change is likely multifactorial, requiring multiple system-level changes not amenable to a simple randomized control design. Third, based on our initial work, which showed good performance in recognition of septic shock, we decided not to focus on timely recognition. However, through overall education and awareness, we were still able to show improvements in this regard. Finally, our study ended after demonstrating 9 months of sustainability. QI interventions are sustainable only if embedded into standard practice and thus we will continue to monitor our progress to ensure continued improvement.
Through implementation of a QI initiative, we were able to improve the care of patients with septic shock in a pediatric ED. Focus on a single key driver and use of reliability theory to inform our interventions was successful in improving adherence to all components of PALS guidelines. This process was possible without additional resources and has proved sustainable over time.
We thank John Andrea and Kaitlin Morris for their dedicated efforts in data collection for this project.
- Accepted January 22, 2014.
- Address correspondence to Raina Paul, MD, Wake Forest University Baptist Medical Center, Department of Emergency Medicine, Medical Center Boulevard, Winston-Salem, NC 27103. E-mail:
Dr Paul conceived and designed the study, supervised the conduct of the trial and data collection, undertook acquisition of data of included patients and managed the data, including quality control, drafted the manuscript, including all revisions, and takes responsibility for the paper as a whole; Dr Melendez conceived and designed the study as well as supervised the conduct of the trial and data collection, undertook acquisition of data of included patients and managed the data, including quality control, assisted with drafting of the manuscript, including all revisions, and approved the manuscript as submitted; Dr Stack conceived and designed the study and undertook acquisition of data of included patients and managed the data, drafted the manuscript, including all revisions, and approved the manuscript as submitted; Dr Capraro conceived and designed the study and undertook acquisition of data of including patients and managed the data, assisted with drafting of the manuscript, including all revisions, and approved the manuscript as submitted; Dr Monuteaux provided statistical advice on study design and analyzed the data, assisted with drafting of the manuscript, including all revisions, and approved the manuscript as submitted; and Dr Neuman conceived and designed the study as well as supervised the conduct of the trial and data collection, undertook acquisition of data of included patients and managed the data, including quality control, assisted with drafting of the manuscript, including all revisions, and approved the manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by an internal Program from Patient Safety and Quality Grant at Boston Children’s Hospital.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Zaritzky AL,
- Nadkarni VM,
- Kickey RW,
- et al
- Ralston M,
- Hazinski MF,
- Zaritsky AL,
- et al
- Dellinger RP,
- Levy MM,
- Carlet JM,
- et al.,
- International Surviving Sepsis Campaign Guidelines Committee,
- American Association of Critical-Care Nurses,
- American College of Chest Physicians,
- American College of Emergency Physicians,
- Canadian Critical Care Society,
- European Society of Clinical Microbiology and Infectious Diseases,
- European Society of Intensive Care Medicine,
- European Respiratory Society,
- International Sepsis Forum,
- Japanese Association for Acute Medicine,
- Japanese Society of Intensive Care Medicine,
- Society of Critical Care Medicine,
- Society of Hospital Medicine,
- Surgical Infection Society,
- World Federation of Societies of Intensive and Critical Care Medicine
- Han YY,
- Carcillo JA,
- Dragotta MA,
- et al
- Carcillo JA,
- Kuch BA,
- Han YY,
- et al
- Inwald DP,
- Tasker RC,
- Peters MJ,
- Nadel S,
- Paediatric Intensive Care Society Study Group (PICS-SG)
- Oliveira CF,
- Nogueira de Sá FR,
- Oliveira DS,
- et al
- Cruz AT,
- Perry AM,
- Williams EA,
- et al
- ↵Larsen GY, Mecham N, Greenberg R, et al. An emergency department septic shock protocol and care guideline for children initiated at triage. Pediatrics. 2011;127(6). Available at: www.pediatrics.org/cgi/content/full/127/6/e1585
- ↵Slater A, Shann F, Pearson G, et al. PIM2: a revised version of the Paediatric Index of Mortality. Intensive Care Med. 2003;29(2):29:278–285
- ↵Nolan T, Resar R, Haraden C, Griffin FA. Improving the Reliability of Health Care. IHI Innovation Series white paper. Boston, MA: Institute for Healthcare Improvement; 2004
- Odetola FO,
- Gebremariam A,
- Freed GL
- Copyright © 2014 by the American Academy of Pediatrics