In Situ Simulation Training for Neonatal Resuscitation: An RCT
OBJECTIVES: High-fidelity simulation is an effective tool in teaching neonatal resuscitation skills to professionals. We aimed to determine whether in situ simulation training (for ∼80% of the delivery room staff) improved neonatal resuscitation performed by the staff at maternities.
METHODS: A baseline evaluation of 12 maternities was performed: a random sample of 10 professionals in each unit was presented with 2 standardized scenarios played on a neonatal high-fidelity simulator. The medical procedures were video recorded for later assessments. The 12 maternities were then randomly assigned to receive the intervention (a 4-hour simulation training session delivered in situ for multidisciplinary groups of 6 professionals) or not receive it. All maternities were evaluated again at 3 months after the intervention. The videos were assessed by 2 neonatologists blinded to the pre-/postintervention as well as to the intervention/control groups. The performance was assessed using a technical score and a team score.
RESULTS: After intervention, the median technical score was significantly higher for scenarios 1 and 2 for the intervention group compared with the control group (P = .01 and 0.004, respectively), the median team score was significantly higher (P < .001) for both scenarios. In the intervention group, the frequency of achieving a heart rate >90 per minute at 3 minutes improved significantly (P = .003), and the number of hazardous events decreased significantly (P < .001).
CONCLUSIONS: In situ simulation training with multidisciplinary teams can effectively improve technical skills and teamwork in neonatal resuscitation.
- CEP —
- Continuous Educational Program
- CI —
- confidence interval
- ILCOR —
- International Liaison Committee on Resuscitation
- TPS —
- team performance score
- TS —
- technical score
What’s Known on This Subject:
High-fidelity simulation improves individual skills in neonatal resuscitation. Usually, training is performed in a simulation center. Little is known about the impact of in situ training on overall team performance.
What This Study Adds:
In situ high-fidelity simulation training of 80% of a maternity’s staff significantly improved overall team performance in neonatal resuscitation (technical skills and teamwork). Fewer hazardous events occurred, and delay in improving the heart rate was shorter.
Because 1% of newborns require resuscitation in the delivery room, high-quality neonatal resuscitation should be possible in each maternity of a perinatal network. High-quality neonatal resuscitation can be defined according to the guidelines published by the International Liaison Committee on Resuscitation (ILCOR).1 Maintaining the professional’s neonatal resuscitation skills is a real challenge in level 1 (without a neonatal unit) and level 2 (with an intermediary care neonatal unit) maternities because these professionals have little opportunity to practice neonatal resuscitation. Furthermore, the entire staff’s performance depends not only on the skills of individuals but also on their ability to face unexpected resuscitations with any member of the staff. To achieve and maintain high-quality neonatal resuscitation, in situ training offers a good opportunity for training staff in their own environment.2
Since 2009, a Continuous Educational Program (CEP) based on high-fidelity (HF) simulation was introduced in our Perinatal Network AURORE (27 maternities: 45 000 annual births) to improve the staff competency in neonatal resuscitation. This CEP was an in situ group training focusing on the multidisciplinary staff (midwives, pediatricians, anesthetists, and nurses) working in the delivery room. A teaching session, performed for 12 professionals, included the theoretical knowledge of neonatal resuscitation, technical skills, and application of the ILCOR algorithm. The CEP had similar goals to those of the American Academy of Pediatrics’ Helping Babies Breathe program.3 Between April 2009 and June 2011, >750 professionals participated in this program.
The results of this CEP led us to believe that overall performance of the staff could be improved if staff members were trained in situ and over a short period because each member would acquire the same background and follow the same algorithm. Although several studies have shown that in situ simulation training improves the individual skills of professionals4,5 and the quality of care,6–10 to our knowledge, there are no data regarding the impact of in situ training on global performance in neonatal resuscitation .
Therefore, we aimed to evaluate whether an in situ high-fidelity simulation-based group-training program would improve the efficacy of the overall staff performance in neonatal resuscitation.
Study Design and Randomization
The study was designed as a multicenter randomized controlled trial.
Only Level 1 and Level 2 maternities of the AURORE Perinatal Network fulfilling the following criteria were eligible: >1000 annual births, previous participation in the CEP before June 2011, and a commitment to not participate in any other training program during the study period.
In the 12 maternities included in the study, a first evaluation (evaluation 1) was performed. Then the 12 maternities were stratified by the level of care and the number of annual births. The name of each maternity was hidden in a sealed opaque envelope and maternities were blindly selected by an individual not involved in the study to have their personnel trained (intervention group) or not trained (control group). After the intervention, a second evaluation of the 12 maternities was performed (Evaluation 2; Fig 1).
In each maternity, a sample of 10 professionals (1 pediatrician and 9 midwives) were randomly selected from the list of the staff and then divided into 5 pairs by the unit manager according to their availability during the day of evaluation. Anonymity was ensured by wearing masks, gowns, and caps and focusing the cameras on their hands. Each pair of professionals had to perform 2 evaluation scenarios successively, acting as a leader (at the head of the manikin) or a helper (at the side of the manikin), and the pair changed positions in the second scenario. Before starting the scenarios, they were briefed on what the manikin could do and were allowed to manipulate it. The scenarios were specifically designed for evaluation 1, differed from those used in intervention, and were not followed by a debriefing. Scenario 1 consisted of neonatal resuscitation of an asphyxiated newborn with clear amniotic fluid, and scenario 2 consisted of neonatal resuscitation of an asphyxiated newborn with meconium-stained amniotic fluid.
The Sim New B (Laerdal Medical, Stavanger, Norway) simulator was programmed to react automatically, step by step, to the actions of the team, without any intervention from the instructors. When the team performed effectively and followed the algorithm, the manikin status improved step-by-step within 5 minutes (tone from “limp” to “movement,” color from “cyanosis” to “pink,” heart rate from 40 per minute to 130 per min, and respiratory rate from 0 per minute to 40 per min). Because there was no debriefing after the evaluation scenarios, we chose to avoid the “death” of the simulator, which was considered potentially traumatic to the professionals. Therefore, when the team performed ineffectively, the manikin status improved from one step to the next step after a predetermined time to allow full recovery only within eight minutes.
The intervention consisted of a 4-hour in situ training session on the Sim New B high-fidelity simulator. The instructors traveled from the organizing center to the various maternities with the simulator and video equipment. Because most maternities had a single neonatal resuscitation room likely to be needed at any time, the delivery room’s environment was re-created in a nearby separate room. The instructors used the maternity’s own resuscitation equipment and radiant warmer. All the participants were dressed in gowns, masks, caps, and gloves.
Each session was dedicated to a group of 6 professionals from the local staff (midwives, pediatricians, anesthetists, or nurses). In each maternity, as many sessions as necessary were provided to train at least 80% of the staff within a period of 1 month.
The resuscitation algorithm, adapted from ILCOR 2010,1 was first reviewed with the professionals. The simulation session had the following objectives: (1) perform appropriate manual ventilation and evaluate its efficacy; (2) apply the algorithm until 10 minutes after birth, either with clear or with meconium-stained amniotic fluid; and (3) improve teamwork.
The instructors used a panel of 9 manually driven training scenarios. Each training scenario had to be performed by 2 professionals. This allowed each professional to practice 2 or 3 training scenarios during the session and to train as a leader as well as a helper. Each training scenario lasted 2 to 10 minute and was followed by a 10- to 30-minute debriefing. The debriefing focused on technical skills (stimulation, bag and mask ventilation, thoracic compressions, and intubation), application of the algorithm (compliance with the guidelines and timing), and non-technical skills (specific roles of the leader and helper and communication between them). During the intervention, there was no discussion or debriefing of the 2 scenarios used in Evaluation 1.
This second evaluation was conducted 3 months after the intervention with exactly the same design as evaluation 1. A new sample of 10 randomly selected professionals was evaluated using the same 2 scenarios used in evaluation 1.
The scenario analyses were performed at 3 months after the end of Evaluation 2. We assessed the technical and team performances as well as the resuscitation outcome. The 2 evaluation scenarios were automatically recorded on the Laerdal Debrief Viewer software that simultaneously saved the video of the scenario as recorded by a webcam, the actions performed by the professionals, and the vital signs of the manikin. A second video from a different angle was recorded using an HD camera.
Each scenario analysis comprised reviews of the Debrief Viewer (with its own video recording) and the HD camera video recordings by 2 neonatologists, separately. The neonatologists were blinded to the study group and to the time of evaluation. The order of the analyses was randomly selected by the Department of Methodology and Statistics (Random list, including random blocks of 12 videos).
The analysis was based on 2 score sheets filled out by reviewing neonatologists, heart rate recorded at 3 and 5 minutes, and the number of hazardous events.
A technical score (TS) sheet adapted from a published checklist11 assessed the technical skills (ventilation, chest compression, intubation, tracheal aspiration if needed) and compliance with the ILCOR 2010 guidelines. The 30 items varied slightly between scenario 1 and scenario 2, according to the medical situation and were scored 0 = no or 1 = yes. They were scored only if the ILCOR algorithm was correctly followed with an appropriate timing.
A team performance score (TPS) sheet was assessed using the Team Emergency Assessment Measure (TEAM),12 with items from 1 to 11 scored as 0 to 4 points (maximum total score: 44 points). The 12th item, titled “Global Performance Appreciation,” was separately assessed and scored as 1 to 10 points (max score: 10 points). The scoring sheets had been previously tested for feasibility with videos recorded in 2 maternities not involved in the study.
Immediately after their independent scenario analysis, the 2 neonatologists compared their TS and TPS. In case of a discrepancy for any item, they reviewed the videos together to determine the optimal score and validate the final sheet. Only these final sheets were used for statistical analysis. For TPS, the κ assessment for interrater reliability was 0.62 (CI 95%: 0.57–0.67).
To evaluate the efficiency of neonatal resuscitation, the heart rate of the manikin at 3 and 5 minutes was recorded in the Laerdal Debrief Viewer. Heart rates >90 per minute at 3 minutes and >130 per minute at 5 minutes were considered a result of efficient neonatal resuscitation.
The occurrence of hazardous events was determined from the videos. In scenario 1, 4 hazardous events were defined: “the team did not realize that the baby was not correctly intubated,” “intubation maneuver exceeding 60 seconds,” “wrong evaluation of the heart rate inducing wrong decisions,” and “traumatism or traumatic action.” In scenario 2, 5 hazardous events were defined: the previously mentioned 4 events plus “ventilation before or without tracheal aspiration.” The maximum number of possible hazardous events was calculated for each scenario as the number of hazardous events of the scenario × the number of scenarios analyzed.
The study was submitted to the Ethics Committee of Lyon that approved the study as a noninterventional study (Comité de Protection des Personnes Sud-Est IV Lyon, 2011/09/01). Individual consents to participate and to be filmed were obtained from all the professionals involved in the study. After the end of evaluation 2, the same training intervention was performed in the Control Group.
TSs and TPSs were expressed as medians and ranges. The TSs were analyzed separately for each scenario because the items varied between scenario 1 and scenario 2, whereas the TPSs were jointly analyzed for the 2 scenarios. The median TSs and TPSs values of the intervention and control groups were compared using the nonparametric Mann-Whitney test.
Hazardous events and heart rates, considered as binary variables, were compared with Fisher’s exact test.
All statistical tests were 2-sided. P < .05 was considered statistically significant. All analyses were performed by using SPSS (Statistics 18.0; SPSS Inc, Chicago, IL).
Among the 19 level 1 and 2 maternities previously trained by the AURORE Network, 6 did not meet the eligibility criteria (4 because of <1000 annual births and 2 because they started the CEP after June 2011) and 1 maternity was closed (Fig 1). Therefore, 12 maternities were recruited, the study was presented to the staff of each maternity, and the consent obtained from the hospital’s managers. The characteristics of the intervention and control maternities and of the professionals involved are described in Table 1.
Evaluation 1 was conducted between January 1, 2012 and February 9, 2012 and included 116 (42%) professionals (midwives and pediatricians); 4 were missing in the control group because of absenteeism on the day of evaluation. Thus, 116 scenarios were recorded. The randomization was performed on the February 10, 2012. The intervention was performed in 6 maternities between March 8, 2012 and April 14, 2012. In the intervention group, 202 professionals (84%–100% of the delivery room staff) received the intervention, and 34 sessions were necessary with an average of 6 per maternity.
Evaluation 2 was performed a median of 107 days (minimum–maximum 61–126) after the intervention and involved 114 professionals because 6 were missing in the control group due to absenteeism on the day of evaluation. Overall, 70% of the professionals randomly selected for evaluation 2 had not participated in evaluation 1. Thus, 114 scenarios were recorded.
The 230 scenarios were reviewed between January 1, 2013 and February 2, 2013. Before the training sessions, the median TS was not significantly different between the Control and Intervention groups, either for scenario 1 (median [minimum–maximum]; 17.5 [15.8–20.1] vs 18.3 [16.9–21.0], P = .42) or for scenario 2 (17.7 [15.6–20.8] vs 17.9 [17.0–19.2], P = .75). After the training sessions, the median TS was significantly higher in the Intervention group than in the Control group for Scenario 1 (17.4 [15.6–19.5], vs. 24.4 [18.7–26.6], P = .01) and Scenario 2 (17.5 [15.3–19.6] vs. 22.7 [21.3–25.0], P = .004; Fig 2).
Before the training sessions, the median TPS was similar between the control and intervention groups (20.0 [13.5–24.0] vs 21.1 [15.4–25.6], P = .64). After the training sessions, it was significantly higher in the intervention group than in the control group (19.9 [13.3–25.0] vs 31.1 [20.8–36.8], P < .001; Fig 3). The Global Performance appreciation, which was similar between the 2 groups in evaluation 1 (3.75 [2.0–5.0] vs 3.8 [2.6–5.2], P = .73) was significantly higher in the intervention group than in the control group in evaluation 2 (6.7 [3.4–8.3] vs 19.9 [13.3–25.0], P = .001).
The achievement of heart rates >90 per minute at 3 minutes and >130 per minute at 5 minutes was significantly higher in the intervention group than in the control group (Table 2); furthermore, the number of hazardous events decreased by 38% in the intervention group but did not change in the control group (Table 3).
Our study demonstrated significant improvements in the technical skills and teamwork of staff members in initial neonatal resuscitation in the delivery room after a single in situ high-fidelity simulation training session.
The effectiveness of simulation in improving skills and teamwork in medicine is now well documented.7,13–15 A randomized controlled study in the Netherlands demonstrated an improvement in team performance in obstetrical management after training in a simulation center.16 In situ simulation allows a better identification of systems issues8,9 or safety threats10,17 and has been shown to improve the management of deteriorating patients.18 In situ training is less developed than training in a simulation center probably because it is more difficult to organize particularly when maternities are far from the reference center; however, it may be highly relevant in improving medical staff performance.10 In situ training and evaluation was feasible in our network because the simulator and all the equipment could be transported in a regular car; moreover, all the maternities were located <200 km from the organizing center. We could not evaluate whether such a program including almost all the staff members and performed in a simulation center over a short period would have produced the same results.
Simulation is effective in enhancing the neonatal resuscitation performance of individuals19,20 and teamwork behavior,21 and there is a correlation between technical and non-technical skills.22 The evaluation of the specific impact of in situ training on improvement in teamwork needs to be assessed by prospective randomized trials.23 The fact that the training period in each maternity lasted <4 weeks in our study may have played an important role.24 Indeed, the same algorithm and messages were experienced by the whole staff over a short period, giving them confidence in their colleagues’ competence and helping them to follow the guidelines. In the present trial, we observed a highly significant improvement in teamwork although our training was not focused on Crew Resource Management.25
Our study has certain limitations. There may have been a selection bias for the professionals. A sample of 10 professionals to evaluate a staff may seem too small, but this number represents ∼40% of the professionals (Table 1). The experience of working in the delivery room of the professionals selected in evaluation 1 was slightly higher in the intervention group (Table 1); however the scores of the 2 groups were similar (Figs 2 and 3). At the time of Evaluation 2, the selected professionals in the 2 groups did not differ in terms of number of working years in the delivery room or previous exposure to the CEP. The use of the simulator for evaluation 1 could have introduced a bias because it could have given some training to the professionals before evaluation 2, although evaluation 1 was not followed by a debriefing. However, we observed no differences between evaluation 1 and evaluation 2 in the control group. Another limitation is that the evaluation of neonatal resuscitation performance by using videos may have been limited by the observers’ subjectivity.26,27 To reduce this effect, we chose to have the video recordings viewed and evaluated separately by 2 neonatologists. Because our aim was to evaluate the short-term effects of the intervention, the mean delay between the intervention and the evaluation was 3 months. In fact, several studies have reported that the improvement in individual technical skills is no longer significant at 6 months after an intervention.28
Our study evaluated the efficiency of resuscitation performed on a manikin; however, the impact in real-life neonatal resuscitation remains unknown. In other words, the study reached the third level of Kirkpatrick’s model to evaluate training programs.29 Additional studies are needed to evaluate whether this beneficial effect is long lasting and whether the resulting improvements can be translated into better practices and better outcomes for neonates or other patients (Kirkpatrick’s fourth level).
The current study demonstrated the feasibility of in situ high-fidelity simulation as a useful training tool for a regionalization program. We observed a significantly positive effect of in situ simulation training on multidisciplinary teams in both technical skills and teamwork in neonatal resuscitation in a sample of maternity units of a Regional Perinatal Network. These encouraging results should foster the development of other in situ training programs in obstetrics and other specialties that use multidisciplinary teams.
We thank the professionals of the 12 maternities involved in the study: Centre Hospitalier Ardèche-Méridionale (Aubenas), Clinique du Beaujolais (Arnas), Centre Hospitalier Fleyriat (Bourg-en-Bresse), Hôpital Privé Drôme-Ardèche (Guilherand-Granges), L’Hôpital Nord-Ouest (Villefranche sur Saône), Clinique Natécia (Lyon), Clinique St Vincent de Paul (Bourgoin-Jallieu), Hôpital St-Joseph-St-Luc (Lyon), Centre Hospitalier de Ste Foy les Lyon (Ste Foy les Lyon), Clinique Mutualiste de l’Union (Vaulx en Velin), Clinique du Val d’Ouest (Ecully), and Centre Hospitalier L. Hussel (Vienne). We also thank the Regional Health Agency (ARS Rhône-Alpes), which supports AURORE perinatal Network.
- Accepted May 27, 2014.
- Address correspondence to Sophie Rubio-Gurung, MD, Réseau Périnatal Aurore, Hôpital de la Croix-Rousse, 103 grande rue de la Croix-Rousse, F-69004, Lyon, France. E-mail:
Dr Rubio-Gurung conceptualized the study, was implicated in Evaluations 1 and 2, supervised the scenario analysis, and drafted the initial manuscript; Dr Putet conceptualized the study with Dr Rubio-Gurung, supervised the training, participated to Evaluations 1 and 2, supervised the scenario analysis, and revised the manuscript; Dr Touzet supervised the statistical analysis and critically reviewed the manuscript; Drs Gauthier-Moulinier, Jordan, Beissel, Labaune, and Blanc performed the training of maternity staff members and the scenario analyses; Mrs Amamra performed the initial statistical analysis; Mrs Balandras coordinated the interventions and the evaluations in the maternity units; Dr Rudigoz critically reviewed the manuscript; Dr Colin supervised the statistical methodology of the study; Dr Picaud supervised and critically reviewed the manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: The study was fully funded by the French Ministry of Health (Research Program on Quality of Care PREQHOS).
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Kattwinkel J,
- Perlman JM,
- Aziz K,
- et al
- Kurosawa H,
- Ikeyama T,
- Achuff P,
- et al
- Walker ST,
- Sevdalis N,
- McKay A,
- et al
- Patterson MD,
- Geis GL,
- Falcone RA,
- LeMaster T,
- Wears RL
- Rovamo L,
- Mattila MM,
- Andersson S,
- Rosenberg P
- Riem N,
- Boet S,
- Bould MD,
- Tavares W,
- Naik VN
- Sørensen JL,
- Van der Vleuten C,
- Lindschou J,
- et al
- Schmutz J,
- Manser T
- Carbine DN,
- Finer NN,
- Knodel E,
- Rich W
- Thomas EJ,
- Williams AL,
- Reichman EF,
- Lasky RE,
- Crandell S,
- Taggart WR
- Kirkpatrick DL,
- Kirkpatrick JD
- Copyright © 2014 by the American Academy of Pediatrics