OBJECTIVE: Poor communication and teamwork may contribute to errors during neonatal resuscitation. Our objective was to evaluate whether interns who received a 2-hour teamwork training intervention with the Neonatal Resuscitation Program (NRP) demonstrated more teamwork and higher quality resuscitations than control subjects.
METHODS: Participants were noncertified 2007 and 2008 incoming interns for pediatrics, combined pediatrics and internal medicine, family medicine, emergency medicine, and obstetrics and gynecology (n = 98). Pediatrics and combined pediatrics/internal medicine interns were eligible for 6-month follow-up (n = 34). A randomized trial was conducted in which half of the participants in the team training arm practiced NRP skills by using high-fidelity simulators; the remaining practiced with low-fidelity simulators, as did control subjects. Blinded, trained observers viewed video recordings of high-fidelity–simulated resuscitations for teamwork and resuscitation quality.
RESULTS: High-fidelity training (HFT) group had higher teamwork frequency than did control subjects (12.8 vs 9.0 behaviors per minute; P < .001). Intervention groups maintained more workload management (control subjects: 89.3%; low-fidelity training [LFT] group: 98.0% [P < .001]; HFT group: 98.8%; HFT group versus control subjects [P < .001]) and completed resuscitations faster (control subjects: 10.6 minutes; LFT group: 8.6 minutes [P = .040]; HFT group: 7.4 minutes; HFT group versus control subjects [P < .001]). Overall, intervention teams completed the resuscitation an average of 2.6 minutes faster than did control subjects, a time reduction of 24% (95% confidence interval: 12%–37%). Intervention groups demonstrated more frequent teamwork during 6-month follow-up resuscitations (11.8 vs 10.0 behaviors per minute; P = .030).
CONCLUSIONS: Trained participants exhibited more frequent teamwork behaviors (especially the HFT group) and better workload management and completed the resuscitation more quickly than did control subjects. The impact on team behaviors persisted for at least 6 months. Incorporating team training into the NRP curriculum is a feasible and effective way to teach interns teamwork skills. It also improves simulated resuscitation quality by shortening the duration.
WHAT'S KNOWN ON THIS SUBJECT:
Breakdowns in teamwork and communication are detrimental to neonatal resuscitations. A teamwork and error curriculum incorporated into the Neonatal Resuscitation Program may improve teamwork and the quality of resuscitations.
WHAT THIS STUDY ADDS:
The 2-hour teamwork and error curriculum added to the Neonatal Resuscitation Program improved teamwork frequency and shortened the duration of simulated resuscitations immediately after training. Differences in teamwork behavior frequency persisted for at least 6 months.
The Neonatal Resuscitation Program (NRP) teaches caregivers how to resuscitate newborns in the delivery room.1 It includes instruction about life-saving therapies such as ventilation and chest compression and other beneficial treatments such as drying, warming, and stimulation2; however, the quality of resuscitations is often suboptimal,3,–,5 a result in part of poor teamwork and communication.6,7 There is broad consensus from expert groups that measuring and improving teamwork will help to improve the quality of health care.8,–,10 Interventions to improve teamwork and reduce deviations from the NRP could have a significant impact on the quality of care.
Researchers have recommended teaching team behaviors during simulated neonatal resuscitations,11,12 developing definitions specific for this care process,7 and demonstrating that they can be reliably measured.4 The existing NRP course, taught to most caregivers in the United States who care for newborns, focuses on teaching the technical aspects of neonatal resuscitation with little attention (1 page of the manual) paid to communication and teamwork. Adding teamwork instruction to NRP, based on crew resource management, may improve communication, teamwork, and the overall quality of neonatal resuscitation.
This study incorporated team training into the NRP curriculum and used simulation to evaluate the effectiveness of the training and the NRP performance quality. We also compared NRP with low-fidelity skills stations with NRP with high-fidelity skills stations within the team training group.
The study was conducted in the Surgical and Clinical Skills Center at the University of Texas Medical School at Houston, a 14000-ft2 facility completed in 2006. The resuscitation scenarios took place in two 400-ft2 simulation theaters, equipped with cameras and microphones. The NRP curriculum consists of 9 modules that cover specific skills. After completing each module, the participants practiced the skills on either low-fidelity or high-fidelity mannequins, depending on the group assignment. SimBaby mannequins (Laerdal Medical Corp, Stavanger, Norway) were used in the high-fidelity skills stations. These mannequins have simulated heart tones, breath sounds, pulses, and cries.
All incoming interns for pediatrics, combined pediatrics and internal medicine, family medicine, emergency medicine, and obstetrics and gynecology who began training in June 2007 or June 2008 and had not previously completed NRP certification were eligible for participation in the study. Participants who gave consent were randomly assigned (by Ms Williams using a random-number generator) to 1 of 3 groups: (1) standard NRP with low-fidelity skills stations (control); (2) NRP with low-fidelity skills stations and team training (LFT); or (3) NRP with high-fidelity skills stations and team training (HFT; Fig 1). Participants were randomly assigned to 3- or 4-person teams with other members from the same study group. They remained with their teams throughout the lectures and skills stations and completed the final resuscitation simulation (megacode) in their assigned teams. The institutional review board approved all study procedures.
A teamwork curriculum was created to advance the following learning objectives: (1) to recognize the frequency, types, and causes of errors, both in medicine in general and in neonatal resuscitation specifically, and (2) to understand and practice team behaviors. The teamwork and human error component was presented to LFT and HFT groups before the standard NRP course. The instructor for the teamwork training component (Mr Taggart) was an expert in team training with experience in both aviation and health care. The teamwork curriculum lasted for 2 hours and consisted of (1) information about human error, including limitations of human performance and the epidemiology of error in medicine and neonatal resuscitation; (2) examples of specific communication behaviors (information sharing, inquiry, assertion, verbalizing intentions, workload management, vigilance, and leadership) used to prevent and manage error (Table 1); (3) other methods for improving communication (using standard terminology, increasing clarity, repeating information, and sharing a mental model) and the SBAR model (situation, background, assessment, and recommendation); and (4) customized video clips and role-playing to illustrate teamwork behaviors. NRP instructors were certified and experienced in teaching NRP and were not involved in study design, data collection, or analysis. Two instructors were present during each megacode simulation: 1 instructor managed the simulator and the flow of the scenario, and the second instructor acted as the facilitator (reading the scenario and answering designated questions). Instructors were told not to deviate from the scenarios and not to give any hints to the teams.
Baseline Resuscitation Simulations
On completion of NRP lectures and skills stations, all participants (including teams who practiced skills stations on a low-fidelity mannequin) performed a megacode on a SimBaby mannequin (high-fidelity) with their assigned team. All teams were given a brief introduction to the simulator before the scenarios. Each participant led a resuscitation scenario assisted by their team members. Three standardized resuscitation scenarios were developed for the study (Appendix, which is published as supporting information at www.pediatrics.org/content/full/125/3/539). The scenarios differed in 2 key components: 1 scenario was a precipitous delivery, so the team did not have time to prepare their equipment before the infant was born, and 2 scenarios required volume administration when the infant did not respond to epinephrine. A debriefing occurred after each of the resuscitation scenarios in which the instructors reviewed the management of the case. Video recordings were edited to begin when the instructor started the scenario prompt and end immediately after the team indicated that the infant was ready to be transferred to a NICU.
Table 1 describes the teamwork behaviors that were measured in the study and were based on previous research.4,7,13,–,16 Observers received training in teamwork behavior assessment for ∼50 hours each. Teamwork observers, blinded to intervention status, scored resuscitation videos for the occurrence of teamwork behaviors by using Noldus Observer XT 7.0 (Noldus Information Technology, Wageningen, Netherlands). Ten videos, randomly distributed throughout the data sets, were scored by both observers to assess interrater reliability.
Teamwork observers coded the specific behaviors each time they occurred during the resuscitations. Five behaviors were scored as discrete verbalizations (sharing information, inquiry, assertion, teaching/advising, and evaluation of plans). Vigilance and workload management were scored as state behaviors, with a start and end time for each occurrence of the behavior (Table 1).
NRP Performance Observations
The NRP Megacode Assessment Form (Advanced) was used as a template for scoring NRP performance.1 Performance observers were blinded to participant team training status. Their training consisted of ∼40 hours each (10 meetings and independently scoring 8 videos) during the 6-month training period. During meetings, the observers discussed scoring discrepancies with an investigator (Ms Williams), and scoring rules were refined when necessary. Like the teamwork observers, performance observers met monthly while scoring the data set for recalibration. Twenty videos were randomly distributed throughout both observers' data sets to evaluate interrater reliability.
Noldus Observer software was also used to facilitate the recording of observations for NRP performance assessment. Performance observers scored each NRP step every time the step occurred during the resuscitation. Some steps occurred only once per resuscitation, but some steps, such as providing positive pressure ventilation, occurred multiple times within a resuscitation. Observers recorded a score for each NRP step every time it was performed or should have been performed. Scores were assigned according to the NRP Megacode Assessment form, which indicates a score of 0 when a step is omitted; 1 when the step is performed incorrectly, incompletely, or out of order; and 2 when the step is correctly performed. All steps on the Megacode Assessment Form were scored, including preparing and administering epinephrine and preparing and inserting the umbilical venous catheter (UVC). Technique for UVC insertion was not scored because proper technique was difficult to perform on the SimBaby. UVC insertion was scored as an error only when it was incorrectly omitted or performed at the incorrect time.
Because hints from instructors could potentially influence simulation outcomes, performance observers recorded the occurrence of instructor hints during the resuscitation simulations. Hints were defined as “comments from the instructor giving some indication of the step the team should perform.” Examples of hints are, “What questions do you want to ask me?” and, “You should consider intubation.” Performance observers also recorded the occurrence of “teaching moments” imposed by the instructors. If the instructor stopped the scenario for >5 seconds to explain a concept, then those times were marked for later removal from the total resuscitation time. Whereas hints were scored as discrete events, teaching moments were scored as states (continuing for some duration). Teaching moments often included hints, but hints frequently occurred without stopping the resuscitation for 5 seconds.
We measured 3 teamwork outcomes: teamwork event rate, vigilance percentage, and workload management percentage. The teamwork event rate was calculated by summing the number of scored teamwork events (sharing information, inquiry, assertion, teaching/advising, and evaluation of plans) and dividing by the total resuscitation time (in minutes). Vigilance and workload management percentages were calculated by summing the total time the team demonstrated each behavior and dividing by the total resuscitation time.
We analyzed 2 measures of NRP quality: performance score and resuscitation duration. The performance score was calculated by averaging the scores (ranging from 0 to 2) for each NRP step (some of which occurred multiple times). Those scores were summed and divided by the total possible score (2 × the number of steps that should have been performed). When a step was not indicated for the specific resuscitation scenario (eg, meconium aspiration), that step was not scored by the observers and it was not included in the denominator for performance calculation. This produced a measure of performance percentage ranging from 0% to 100% for each resuscitation. A second outcome for this study was the time required to complete the resuscitation, another important component of neonatal resuscitation.1 The total duration for each resuscitation was calculated from the start of the instructor's reading of the scenario to the team's statement that the infant should be transferred to the NICU. When any teaching moments occurred during the simulation, the total teaching time was subtracted from the resuscitation duration.
Follow-up Resuscitation Simulations
Interns in pediatrics and combined pediatrics and internal medicine (n = 43) were eligible to return for a follow-up megacode 6 months after the NRP training course. The other interns did not perform follow-up because of scheduling conflicts. Without additional instruction, participants completed a full megacode scenario with team members from the same study arm but not necessarily from the original team. Follow-up scenarios were the same 3 scenarios as baseline resuscitations. Follow-up resuscitations were randomly mixed into the data set so that teamwork and quality observers were blinded to baseline or follow-up status.
Differences were calculated between the study outcome measures scored by each observer for the reliability resuscitation scenarios, according to the method described by Bland and Altman.17 Mean differences between observers are presented for normally distributed variables, and median differences are presented for nonparametric data. For teamwork event rates, the average difference between the 2 observers was 0.12 events per minute, and the difference ranged from −4.2 to 2.4 events per minute (r = 0.85, P = .002). The median difference for vigilance was 0%, and the difference ranged from 0% to 2% (r = 0.58, P = .079). The median difference for workload management was 0%, ranging from −4% to 41% between observers (r = 0.23, P = .523). Performance score difference averaged 4%, ranging from −14% to 17% difference for the 20 reliability videos for the NRP observers (r = 0.68, P = .001). Resuscitation duration (without teaching times) had an average difference of 11 seconds; the difference ranged from −16 to 85 seconds (r = 0.99, P < .001).
We hypothesized that teams who received team training would demonstrate higher rates of teamwork behaviors and higher NRP performance scores during their megacodes than would control subjects. One-way analysis of variance models were calculated to compare teamwork rates, NRP performance, and resuscitation duration between the control subjects and the 2 team training groups. Bonferroni test of multiple comparisons was calculated when statistically significant differences were found between groups. Kruskal-Wallis equality-of-proportions rank tests were calculated (including ties) to compare vigilance and workload management percentages between groups because these outcomes were not normally distributed. Teamwork outcomes were also compared between the LFT and HFT groups. We expected that no differences would be found between these 2 groups. Follow-up megacodes were compared for all team training participants (LFT and HFT) versus control subjects to increase power. Only pediatrics and combined pediatrics and internal medicine interns were eligible to return for follow-up. T tests were calculated to compare normally distributed outcome measures at follow-up (teamwork rate, performance, and duration), and Wilcoxon rank-sum tests were used to compare measures that were not normally distributed (vigilance and workload management percentages). Statistical significance was assessed at α = .05. All presented P values are 2-sided.
Intervention Effects on Teamwork
Teamwork outcomes in all groups are included in Table 2. Teamwork event rates differed significantly among the 3 groups (P < .001). The HFT group had significantly higher teamwork event rates than did control subjects (P < .001). Teamwork event rates did not significantly differ between the LFT group and the control subjects (P = .198). With the exception of teaching and evaluation of plans, which were rarely observed in any group, this pattern was consistent for all teamwork event behaviors (Table 2).
There were significant differences between the groups with respect to workload management (P < .001) but not vigilance (P = .454). Teams who received teamwork training managed the resuscitation workload better than did control subjects. The LFT group managed the workload for a median of 100% of the resuscitation, versus 96.5% for control subjects (P < .001). The HFT group's median workload management was also 100% of the resuscitation (control comparison P < .001). The effect of team training on vigilance is less clear. All teams maintained vigilance for at least 95% of the resuscitation scenario.
Intervention Effects on Quality
There was no evidence of an association between the teamwork training intervention and NRP performance. The average performance scores were 72% for control subjects, 74% for LFT teams, and 72% for HFT teams, and none of the differences between groups were significant at the .05 level (P = .654). Resuscitation duration was significantly different among the 3 groups (P < .001). Control subjects took an average of 10.6 minutes to complete the resuscitation, compared with 8.6 minutes for LFT teams (control comparison P = .040) and 7.4 minutes for HFT teams (control comparison P < .001). Intervention teams completed the resuscitation an average of 2.6 minutes faster than control subjects, a time reduction of 24% (95% confidence interval: 12%–37%). Figure 2 compares specific resuscitation time points between control subjects and team training groups (combined LFT and HFT).
Low-fidelity Versus High-fidelity Skills Practice
HFT teams demonstrated more teamwork event behaviors than did LFT teams (P = .004). HFT teams were not significantly more vigilant (P = .923) or more likely to manage the workload (P = .244) than LFT teams. We found no evidence that HFT teams differed from LFT teams in NRP performance (P = 0.999) or resuscitation duration (P = .452).
Interns who received team training demonstrated more frequent teamwork behaviors in the 6-month follow-up megacodes than did control subjects (mean: 11.8 vs 10.0 behaviors per minute; P = .030). We found no evidence that trained participants maintained more vigilance (median: 100% [control subjects] vs 100% [intervention]; P = .951) or workload management (median: 100% [control subjects] vs 100% [intervention]; P = .549) than did control subjects. There were no significant differences in NRP performance at follow-up (mean: 68.9% [control subjects] vs 69.6% [intervention]; P = .742) or in resuscitation duration, although the intervention groups had shorter duration resuscitations as they did immediately after training (mean: 9.3 minutes [control subjects] vs 8.3 minutes [intervention]; P = .314).
Interns whose NRP course included a brief teamwork curriculum used more frequent teamwork behaviors and completed simulated resuscitations more quickly than did interns who received NRP without team training. The effect of the intervention on team behaviors persisted for at least 6 months. This is only the second blinded, randomized trial of team training to be conducted with trainees,15 and it is the first study in any setting to show a positive impact on both team behaviors and a measure of quality (resuscitation time).
We expected that participants who received team training would demonstrate more frequent teamwork behaviors during their megacodes because the training focused on methods for improving communication, situational awareness, and task distribution, which directly correlate to the behavior outcomes for this study. We also found that teams who received team training took less time to complete the resuscitation scenarios. This may have resulted from an increase in team behaviors such as assertiveness or inquiry in team-trained groups: they were more willing to ask questions of each other or to make suggestions about the resuscitation procedure. When the newborn was not responding to the current strategy, teams who received team training were more likely to suggest a new tactic immediately. Several important milestones of resuscitation occurred more quickly among the teams who received team training (Fig 2). It is interesting that the only randomized, controlled trial of team training among nontrainees also found a positive impact on measures of time for care processes but not other patient outcomes.18 Shorter resuscitations in team-trained groups also explained the increase in teamwork behavior rate for those groups. Because the team training did not focus on task performance, it may not be surprising that the training did not improve the performance of NRP skills.
Within the team training group, half of the participants practiced skills stations on low-fidelity mannequins and the other half practiced skills on high-fidelity mannequins. We did not expect to find differences between these groups in teamwork behaviors or resuscitation quality during the final megacodes (all using high-fidelity mannequins); however, teams who practiced NRP skills on high-fidelity mannequins exhibited more frequent teamwork behaviors during the final resuscitation megacodes. These results are consistent with previous thinking about the importance of high-fidelity simulation for teaching teamwork.11,12
During 6-month follow-up assessments, team-trained groups continued to demonstrate more frequent teamwork than did control subjects. We were unable to detect any differences in vigilance and workload management or resuscitation quality at follow-up. The ceiling effects for vigilance and workload management as well as the reduced follow-up sample size may have contributed to the lack of significant differences at follow-up.
A guiding principal of this research was to build on and integrate our research into the current NRP curriculum, not to create a stand-alone team training program. This is advantageous because with only slight modification, the widely accepted standard curriculum would become the delivery vehicle to provide team training to large numbers of health care providers. Even with the team training, we remained able to complete the NRP training in 1 day, as had been the norm at our institution. We also built on existing NRP practices by using the NRP's Megacode Assessment Form and scoring method for 1 of our primary outcome measures.
The study has several methodologic strengths that are rarely found in team training studies and educational research. Participants were randomly assigned to control and intervention arms to control for confounding factors, and observers were blinded. A recent review of team training programs for medical students and interns found no previous randomized trials.19 Outcome measurements were performed by highly trained individuals who made reliable assessments. Teamwork behaviors and the megacode performance measures were recorded by separate observers, further reducing bias. Additional measures were collected to evaluate the potential effects of hints from instructors on resuscitation outcomes. The frequency of hints did not differ significantly between the study groups at baseline (P = .498) or follow-up (P = .646). Finally, the behaviors that we measured were previously validated in that they are correlated with measures of NRP quality.7
Although the participants for this study were not multidisciplinary teams, many residency programs conduct NRP training in a similar manner to the study protocol. Questions remain about whether similar training will be effective for other types of caregivers. Generalizability might be limited if other sites were not able to re-create the team training content; however, it used commonly available methods knowledge.
The team training intervention was originally conceived as a curriculum to reduce errors, but the results of this study do not indicate error reduction in the intervention group. We found no differences in the NRP performance score between control subjects and participants who received team training. The human error portion of the training may have had less impact on the participants than the team behavior components. A teamwork curriculum with less emphasis on error might be explored in future research.
Our intervention differs from other team training curricula in 2 important ways. The most widely known courses are TeamSTEPPS20 and those delivered by private consultants. These courses are much longer than ours, and they often do not focus on a specific care process such as neonatal resuscitation. It may be that brief and focused curricula, such as ours, are most effective. Alternatively, the 2 approaches may be complementary. Over time, medical schools and health care organizations may implement several types of team training courses in multiple contexts. This may be necessary to train properly and then maintain effective teamwork skills in caregivers.
It is feasible to add team training to the NRP curriculum, and it results in improved teamwork and a marked reduction in the time required to complete a simulated megacode. The impact of the intervention on team behaviors persisted for at least 6 months. Our results should prompt educators and the NRP curriculum committee to include brief team training and human error content in NRP courses. These findings may also be generalizable to other courses, such as Advance Cardiac Life Support, Advanced Trauma Life Support, and Pediatric Advanced Life Support.
This study was supported by a grant from the dean's office of the University of Texas Medical School at Houston and National Institutes of Health grant UL1 RR024148 (CTSA).
- Accepted September 9, 2009.
- Address correspondence to Eric J. Thomas, MD, MPH, 6410 Fannin St, UPB 1100, Houston, TX 77030. E-mail:
FINANCIAL DISCLOSURE: Mr Taggart works as a team training consultant; the other authors have no financial relationships relevant to this article to disclose.
- NRP =
- Neonatal Resuscitation Program •
- HFT =
- high-fidelity training •
- LFT =
- low-fidelity training •
- UVC =
- umbilical venous catheter
- Kattwinkel J
- Sinclair JC,
- Bracken MB
- Tyson JE
- Carbine DN,
- Finer NN,
- Knodel E,
- Rich W
- Falck AJ,
- Escobedo MB,
- Baillargeon JG,
- Villard LG,
- Gunkel JH
- Thomas EJ,
- Sexton JB,
- Helmreich RL
- Kohn LT,
- Corrigan JM,
- Donaldson MS
- Bogner MS
- Helmreich RL,
- Schaefer HG
- Halamek LP,
- Kaegi DM,
- Gaba DM,
- et al
- Wiener EL,
- Kanki BG,
- Helmreich RL
- Helmreich RL,
- Foushee HC
- Williams AL,
- Lasky RE,
- Dannemiller JL,
- Andrei A,
- Thomas EJ
- Clancy CM,
- Tornberg DN
- Copyright © 2010 by the American Academy of Pediatrics