Implementation Methods for Delivery Room Management: A Quality Improvement Comparison Study
- Henry C. Lee, MD, MSa,b,
- Richard J. Powers, MDc,
- Mihoko V. Bennett, PhDa,b,
- Neil N. Finer, MDd,
- Louis P. Halamek, MDa,
- Courtney Nisbet, RN, MSa,b,
- Margaret Crockett, MS, CNS, NNP-BCe,
- Kathy Chance, MDf,
- David Blackney, RCP, RRT-NPSe,
- Connie von Köhler, RNC, MSN, CPHQg,
- Paul Kurtin, MDh, and
- Paul J. Sharek, MDb,i,j
- Divisions of aNeonatal & Developmental Medicine and
- iGeneral Pediatrics, Department of Pediatrics, Stanford University, Palo Alto, California;
- bCalifornia Perinatal Quality Care Collaborative, Palo Alto, California;
- cDivision of Neonatology, Department of Pediatrics, Good Samaritan Hospital, San Jose, California;
- dDivision of Neonatology, Department of Pediatrics, University of California San Diego, San Diego, California;
- eSutter Medical Center, Sacramento, California;
- fCalifornia Children’s Services, Sacramento, California;
- gMiller Children’s Hospital Long Beach, MemorialCare Health System, Long Beach, California;
- hRady Children’s Hospital, San Diego, California; and
- jCenter for Quality and Clinical Effectiveness, Lucile Packard Children’s Hospital, Palo Alto, California
BACKGROUND: There is little evidence to compare the effectiveness of large collaborative quality improvement versus individual local projects.
METHODS: This was a prospective pre-post intervention study of neonatal resuscitation practice, comparing 3 groups of nonrandomized hospitals in the California Perinatal Quality Care Collaborative: (1) collaborative, hospitals working together through face-to-face meetings, webcasts, electronic mailing list, and data sharing; (2) individual, hospitals working independently; and (3) nonparticipant hospitals. The collaborative and individual arms participated in improvement activities, focusing on reducing hypothermia and invasive ventilatory support.
RESULTS: There were 20 collaborative, 31 individual, and 44 nonparticipant hospitals caring for 12 528 eligible infants. Each group had reduced hypothermia from baseline to postintervention. The collaborative group had the most significant decrease in hypothermia, from 39% to 21%, compared with individual hospital efforts of 38% to 33%, and nonparticipants of 42% to 34%. After risk adjustment, the collaborative group had twice the magnitude of decrease in rates of newborns with hypothermia compared with the other groups. Collaborative improvement also led to greater decreases in delivery room intubation (53% to 40%) and surfactant administration (37% to 20%).
CONCLUSIONS: Collaborative efforts resulted in larger improvements in delivery room outcomes and processes than individual efforts or nonparticipation. These findings have implications for planning quality improvement projects for implementation of evidence-based practices.
- quality improvement
- neonatal resuscitation
- very low birth weight
- CPAP —
- continuous positive airway pressure
- CPQCC —
- California Perinatal Quality Care Collaborative
- IHI —
- Institute for Healthcare Improvement
- NRP —
- Neonatal Resuscitation Program
- QI —
- quality improvement
- VLBW —
- very low birth weight
What’s Known on This Subject:
Quality improvement (QI) studies generally do not account for concurrent trends of improvement and it is difficult to distinguish the impact of a multihospital collaborative QI project without a contemporary control group.
What This Study Adds:
A multihospital collaborative QI model led to greater declines in hypothermia and invasive ventilation rates in the delivery room compared with an individual NICU QI model and NICUs that did not participate in formal QI activities.
Recent updates of the Neonatal Resuscitation Program (NRP)1 and other evidence emphasize the importance of teamwork and safety climate in effective implementation of complex tasks.2–6 Adopting evidence-based changes has been enhanced by participation in collaborative quality improvement (QI) projects, where changes are bundled together and implemented by teams from different centers sharing common strategies.7,8
Collaborative QI, based on the Institute for Healthcare Improvement’s (IHI) model,9 has been successful in settings such as the California Perinatal Quality Care Collaborative (CPQCC).6,7–10 This model brings together teams from different institutions, providing structured learning with face-to-face meetings, Web-based discussions, electronic mailing lists, expert panel access, and unblinded outcome and process data display, with incremental learning through reporting of successes, barriers, and lessons learned.
In 2010, CPQCC planned a collaborative QI project to improve management of high-risk deliveries, with a best practice bundle constructed from published evidence and guidelines for temperature and respiratory management for very low birth weight (VLBW; birth weight <1500 g) infants.1,2,11–17 Specific interventions included strategies to avoid hypothermia; establishment of lung volume in the least-invasive manner; and supporting teamwork with checklists, briefings, and debriefings.
Although the collaborative model has been successful, significant barriers to participation occur, including costs, personnel availability, and limitations on number of sites. Therefore, CPQCC constructed a novel single-site QI model (NICU QI) not requiring interaction with other sites. In this arm, the same bundle was provided, but implementation was directed by individual centers.
The purpose of this study was to prospectively test 2 different QI methods (collaborative QI and NICU QI) against a third group of nonparticipants. This comparison would minimize the influence of biases frequently seen in QI study designs that compare the intervention over time against baseline data without controls.18–21 A comparison group of nonparticipants can account for secular, concurrent trends in practices. Furthermore, comparison with a group also pursuing QI with a differing strategy would limit selection bias of participant motivation. Our hypothesis was that there would be incremental increases in improvement from nonparticipants to NICU QI to collaborative QI.
Study Design and Setting
This was a prospective longitudinal cohort study of a collaborative QI model compared with a single-site QI model (NICU QI) and a nonparticipant population when implementing an evidence-based practice bundle for delivery room management. The CPQCC collects clinical data prospectively for infants born at 132 California hospitals by using an expanded version of the Vermont Oxford Dataset.22,23 Participation in collaborative QI was offered to all CPQCC members and open to the first hospitals that responded until spaces were filled. Remaining hospitals were offered participation in NICU QI. The institutional review board at Stanford University reviewed and approved a waiver of consent for this project.
The collaborative QI group adhered to the IHI model previously used by CPQCC.7–9 An expert panel developed an evidence-based change package and corresponding metrics. The group participated in 3 face-to-face learning sessions: at collaborative launch, 4 months, and 12 months. Other components included monthly webcasts to share progress and barriers, an electronic mailing list, submission of unblinded outcome and process data to a Web-enabled database exclusively for members, and expert panel guidance. The entire group’s data were presented in monthly meetings and each center had real-time data access at all times. A key aspect of implementation involved teamwork training in a readiness bundle, including preresuscitation checklist, briefing, and debriefing. There was an administrative charge for joining this group in addition to costs for travel to several 1-day learning sessions, paid for by each hospital.
The elements of the NICU QI group were based on a design intended to provide centers with the change package and framework for implementation without interaction with other centers. NICU QI centers were given the same change package and metrics grid as Collaborative QI and were instructed to implement all interventions, report the same metrics monthly via electronic forms (not visible to other sites), and to meet with local QI experts at least quarterly. NICU QI centers were encouraged to seek practical assistance from internal QI experts and, if necessary, outside content experts. Table 1 lists key characteristics distinguishing the Collaborative and NICU QI groups.
This group included CPQCC hospitals that were not in the other groups. They were not required to collect data other than that already being collected for CPQCC.
Eligible patients for analysis included inborn patients with birth weight ≤1500 g or gestational age between 22 and 29 weeks. As hospitals were not randomly assigned groups, we restricted the analyses to hospitals that had at least 10 eligible infants born during each of the 3 study periods to have similar comparison groups.
The expert panel reviewed multiple sources for potential best practices, including the NRP and randomized controlled trials demonstrating optimal outcomes with specific delivery room strategies targeting VLBW infants. The interventions in the change package (Table 2) included (1) maintaining normal core body temperature (36.5–37.5°C), by equipment use (radiant warmer, polyethylene wrap, chemical blankets, caps) and raising ambient room temperature, (2) monitoring of oxygen saturation within 2 minutes, (3) emphasizing continuous positive airway pressure (CPAP) over intubation and prophylactic surfactant, and (4) using checklists, briefings, and debriefings.
The primary outcome, percentage of VLBW newborns with hypothermia, was selected because (1) evidence correlates hypothermia with increased risk of death and morbidities such as intraventricular hemorrhage,24,25 (2) first temperature is a standard metric collected by all CPQCC members, and (3) hypothermia prevention has been a consistent target of QI.26 Secondary outcomes included frequency of delivery room CPAP (goal of increasing frequency), and intubation and surfactant administration in the delivery room (goal of reducing frequency, which may help to prevent bronchopulmonary dysplasia).27 The CPQCC data set was the source of measurement; these data are collected by trained abstractors by using standard definitions and methods.23
Balancing measures included (1) hyperthermia (temperature >37.5°), (2) the need for epinephrine and chest compressions, and (3) pneumothorax rates. These were selected to assess the possibility that a narrow focus on improving temperature control, maximizing CPAP, and reducing intubation could inadvertently lead to increased resuscitative efforts.
Outcome variables were compared across 3 time periods: baseline (infants born June 2010 to May 2011), intervention (June 2011 to May 2012), and postintervention (June 2012 to May 2013). The baseline period occurred before any intervention. The intervention period encompassed the time from the first Learning Session to 12 months afterward, during which there were monthly meetings among the collaborative QI group, 3 face-to-face Learning Sessions, and active group E-mail dialogue. The postintervention time frame encompassed the 12 months after Learning Session 3. There was a final virtual Learning Session during the midpoint of this period to review the status of the initiative. Temperature was categorized as hypothermic (<36.5°) versus not. The temperature recorded is the first temperature recorded on NICU admission and must be recorded within 1 hour after birth. Temperature was missing for 1% of records. Rates were compared by using the χ2 test.
Multivariable analyses were conducted at the patient level. The main predictor variable was time period. We accounted for the following risk adjustment variables: NICU-eligible patient volume (categorized in terciles), birth weight, gender, maternal age, race, multiple gestation, small for gestational age, congenital anomaly, receipt of antenatal steroids, and mode of delivery. Birth weight and small for gestational age were used rather than gestational age alone as predictors along with the hospital of care being accounted for as a random effect, by using methods similar to those described previously.7,28,29
We determined the statistical significance of the observed heterogeneity of outcome change over time by incorporating study period and study group interactions. A P < .05 for the interaction term would indicate evidence of study group effect.
Ninety-five CPQCC member hospitals (20 collaborative QI, 31 NICU QI, 44 nonparticipant hospitals) were eligible. In these 95 hospitals, there were 12 528 eligible inborn infants (collaborative QI hospitals, n = 4625; NICU QI, n = 3229; nonparticipants, n = 4674). There were 37 CPQCC hospitals excluded (collaborative QI 4, NICU QI 4, nonparticipants 29) for not meeting the minimum number of 10 eligible infant deliveries for each of all 3 study periods. Baseline characteristics by group are shown in Table 3. For the collaborative QI group, the 3 face-to-face Learning Sessions had an average of 105 participants.
All 3 groups had decreased hypothermia rates over time (Fig 1A). The collaborative QI group had the largest decrease in prevalence of hypothermia from 39.2% at baseline, 27.7% during the intervention period, and 20.7% during the postintervention period (P < .0001 compared with baseline for intervention and postintervention). The NICU QI and nonparticipant groups also had decreased prevalence of hypothermia (Fig 1A). Hyperthermia increased in the collaborative QI group from 5.9% in the baseline period to 8.8% in the postintervention period (P = .0015), but did not change in other groups (Fig 1B).
CPAP use (Fig 1C), and CPAP without intubation (Fig 1D) increased from baseline to postintervention for all groups. Delivery room surfactant decreased for collaborative QI (36.6% baseline to 19.5% postintervention, P < .0001) and NICU QI (18.8% baseline to 12.4% postintervention, P < .0001), but did not change in nonparticipants (17.6% baseline to 16.1% postintervention, P = .26, Fig 1E). Delivery room endotracheal intubation decreased for collaborative QI (53.3% to 39.6%, P < .0001) and NICU QI (43.8% to 36.2%, P = .0003) but not for nonparticipants (43.3% baseline to 40.0% postintervention, P = .06, Fig 1F).
In mixed effects multivariable logistic regression analyses, there was general improvement across all groups in temperature and respiratory management (Fig 2). However, there was a larger degree of improvement for collaborative QI compared with NICU QI and nonparticipants, in reduction of hypothermia, intubation, and surfactant use. The greater improvement seen in the collaborative QI group compared with the NICU QI and nonparticipant groups was significant based on test for heterogeneity and by nonoverlapping confidence intervals for the collaborative QI group compared with the other groups. The NICU QI group had a more significant improvement in reduction of surfactant use compared with nonparticipants.
For both unadjusted rates and in multivariable models, there was no change in cardiac compressions, epinephrine use, or pneumothorax across time periods for all groups.
Of the collaborative QI group, 90% of NICUs saw an improvement in hypothermia rates from baseline to postintervention (defined as any decrease in hypothermia rate), compared with 61% of NICU QI centers and 59% of nonparticipant centers. CPAP use increased for 90% of collaborative QI, 71% for NICU QI, and for 64% of nonparticipants. Ventilation by endotracheal tube decreased for 75% of collaborative QI centers, 68% of NICU QI centers, and 64% of nonparticipants. A similar number of centers across the 3 groups (65% to 70%) had an increase in hyperthermia rates.
Our goals were to (1) compare the effectiveness of the IHI’s collaborative QI model with a single-site QI model in which both groups were supplied with the same change package, metrics grid, and data-tracking requirements over the same time frame, and (2) to compare both of those models to a nonparticipant group. For the first goal, the difference in the 2 models was the participation of the collaborative QI sites in a “community of practice” in which each site had access to expert panel members, comparative data, and each other, whereas the NICU QI group had none of those elements. For the second goal, the nonparticipants did not have the change package, metrics grid, data-tracking recommendations, or the community of practice. By comparing the 2 models with nonparticipants, we were able to demonstrate the incremental effect of each QI model, while accounting for secular trends.
Patients in all 3 groups had significant reductions in hypothermia, surfactant use, and delivery room intubation and significant increases in CPAP use. The frequency of surfactant administration in the collaborative QI was higher at baseline than nonparticipants, suggesting that the collaborative QI group had more room to improve. In fact, the postintervention rate for collaborative QI remained higher than both the baseline and postintervention rates of the other 2 groups. The lower rate of surfactant use at baseline for nonparticipants may have been 1 reason that these NICUs were not as interested in joining either QI project.
A previous single-center study of improving delivery room management for preterm infants had similar goals and led to reduced hypothermia and oxygen use, but not a decrease in intubations.30 That study did not have a nonparticipant group for comparison. A similar single-center study found a reduction in hypothermia (defined in that study as temperature <36°) from 55% to 6%.31 It is difficult to compare our groups with theirs because of differing definitions and their much higher baseline rate of hypothermia. Comparing the 3 groups in our study allowed us to control for secular trends toward better delivery room temperature regulation and establishment of lung volumes during resuscitation, even without QI participation. We speculate that nonparticipant hospital improvements resulted from generic knowledge dissemination, partly due to release of the 2011 NRP guidelines. Interaction through state meetings, conferences, and other communications could have also spread to this group, a potential benefit of belonging to an informal community of practice.
We were able to identify the incremental effect of a collaborative approach compared with similar QI efforts outside of a formal collaborative environment. Specifically, although collaborative QI participation resulted in greater reductions in hypothermia and intubation, there was not a significant incremental effect in NICU QI compared with nonparticipants. There were increased trends toward improvements in several outcomes when comparing NICU QI hospitals to nonparticipants for crude rates (Fig 1), but only surfactant use remained prominent in multivariable analyses (Fig 2). As the interventions used by both groups relied on a common toolkit, the processes related to intercenter collaboration may have led to the significantly greater degree of improvement seen for the collaborative QI group.
Of note, the collaborative QI group had an increase in hyperthermia. The 8.8% incidence of hyperthermia in the collaborative group compared with 4.1% in both NICU QI and nonparticipants represents a potentially concerning risk of temperature control elements. We did not find an association of decreased hypothermia to increased hyperthermia (P = .47) when categorizing the collaborative QI into terciles of performance. Careful attention to avoiding excessive warming is necessary when pursuing QI in this area. Analyses of other balancing measures (epinephrine use and chest compressions in the delivery room, and pneumothorax) were reassuring.
Reports of multicenter collaborative QI projects in adult and pediatric medicine settings have demonstrated improvements over time when comparing outcomes at the end of the collaborative with baseline.32–36 A limitation of these studies is the lack of prospective controls. Some reports have incorporated simultaneous control groups, including cluster randomization protocols.37–39 Although the studies by Horbar et al38 and Scales et al39 reported significant improvements in primary outcomes based on process implementation, the large multicenter NICHD trial by Walsh et al37 failed to demonstrate significant differences in the primary outcome of survival without bronchopulmonary dysplasia despite wide adoption of practices. The study of collaborative multicenter practice may have some advantages to traditional randomized controlled trials in understanding strategies for effective QI.40
This study was subject to several limitations. We did not actively collect process metrics for the NICU QI or nonparticipant populations. However, surrogate measures of engagement were as follows. The average number of individuals on the leadership team for each NICU was 3.8 for collaborative QI and 2.7 for the NICU QI hospitals (P = .01), and all members of the collaborative QI group entered extranet data for 100% of the months during the intervention, whereas only 22 of the 35 NICU QI hospitals submitted electronic data forms (of those centers, forms were submitted for 92% of the months). Study groups were not selected randomly, but by voluntary enlistment on a first-come basis, introducing potential bias related to degree of commitment, interest in the topic, and general culture for QI at individual centers. Nonparticipants that did not volunteer for either group would be expected to have less intrinsic motivation. Centers with more motivation and QI organization may have better results. The collaborative QI hospitals generally had higher patient volume (Table 3), which may signify more baseline resources to perform QI, and perhaps a greater opportunity to refine practices due to more patients. We do not yet know the sustainability of the intervention beyond the 12-month postintervention period.
In this study, sites participating in a formal IHI style collaborative QI initiative had improved delivery room outcomes compared with both a formal single-site QI initiative, and nonparticipation. The interaction and exchange of experiences between sites within a QI collaborative project may have led to measurable gains in outcomes as compared with sites that implemented a similar change package locally without interaction with other sites. We hypothesize that building a community of practice could be 1 reason for the incremental improvement seen for collaborative QI over single-site QI. Our collaborative project also included and emphasized the use of checklists and debriefing to improve team performance and communication. This may have facilitated the ability to learn and adapt from the simultaneous experience of teams from similar organizations, leading to greater success. We also have shown the importance of tracking outcomes in nonparticipants to adjust for concurrent trends in clinical practice when reporting results of QI projects. Without the prospective data collection from this group, mistaken conclusions regarding efficacy of both QI methods may have occurred.
We thank the participating CPQCC member hospitals and the members of the Perinatal Quality Improvement Panel (http://cpqcc.org/about_us/pqip_panel).
- Accepted August 20, 2014.
- Address correspondence to Henry C. Lee, MD, Division of Neonatal and Developmental Medicine, Stanford University, 750 Welch Rd, Ste 315, Palo Alto, CA 94304. E-mail:
Dr Lee conceptualized and designed the study, and drafted the initial manuscript; Dr Powers conceptualized and designed the study, coordinated and supervised the study, and critically reviewed the manuscript; Dr Bennett contributed to the study design, developed statistical models for the study, coordinated and supervised data analyses, and critically reviewed the manuscript; Drs Finer and Halamek contributed to study design, served on the expert panel that guided the collaborative quality improvement arm, and critically reviewed the manuscript; Ms Nisbet contributed to study design, coordinated the expert panel, participated in data collection, and critically reviewed the manuscript; Ms Crockett, Drs Chance and Kurtin, Mr Blackney, and Ms von Köhler contributed to the intervention design, served on the expert panel that guided the collaborative quality improvement arm, and critically reviewed the manuscript; Dr Sharek contributed to study design, led the expert panel of the quality improvement arm, coordinated and supervised the study team, and critically reviewed the manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Supported by grant K23HD068400, Eunice Kennedy Shriver National Institute of Child Health and Human Development, and a grant from the Neonatal Resuscitation Program (American Academy of Pediatrics). The sponsors had no active role in relation to study design, data collection, analysis and interpretation, or manuscript preparation and review. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Eunice Kennedy Shriver National Institute of Child Health and Human Development, National Institutes of Health, or the Neonatal Resuscitation Program. Funded by the National Institutes of Health (NIH).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Copyright © 2014 by the American Academy of Pediatrics