BACKGROUND AND OBJECTIVES: Graduate medical education faces challenges as programs transition to the next accreditation system. Evidence supports the effectiveness of simulation for training and assessment. This study aims to describe the current use of simulation and barriers to its implementation in pediatric emergency medicine (PEM) fellowship programs.
METHODS: A survey was developed by consensus methods and distributed to PEM program directors via an anonymous online survey.
RESULTS: Sixty-nine (95%) fellowship programs responded. Simulation-based training is provided by 97% of PEM fellowship programs; the remainder plan to within 2 years. Thirty-seven percent incorporate >20 simulation hours per year. Barriers include the following: lack of faculty time (49%) and faculty simulation experience (39%); limited support for learner attendance (35%); and lack of established curricula (32%). Of those with written simulation curricula, most focus on resuscitation (71%), procedures (63%), and teamwork/communication (38%). Thirty-seven percent use simulation to evaluate procedural competency and resuscitation management. PEM fellows use simulation to teach (77%) and have conducted simulation-based research (33%). Thirty percent participate in a fellows’ “boot camp”; however, finances (27%) and availability (15%) limit attendance. Programs receive simulation funding from hospitals (47%), academic institutions (22%), and PEM revenue (17%), with 22% reporting no direct simulation funding.
CONCLUSIONS: PEM fellowships have rapidly integrated simulation into their curricula over the past 5 years. Current limitations primarily involve faculty and funding, with equipment and dedicated space less significant than previously reported. Shared curricula and assessment tools, increased faculty and financial support, and regionalization could ameliorate barriers to incorporating simulation into PEM fellowships.
- INSPIRE —
- International Network for Simulation-Based Pediatric Innovation, Research, and Education
- PEM —
- pediatric emergency medicine
What’s Known on This Subject:
Simulation-based education is increasing but its use in pediatric emergency medicine (PEM) fellowships has not been recently documented. Previous studies identified barriers including equipment and space, but growth of simulation centers and equipment has been widespread.
What This Study Adds:
Simulation is widely used in PEM fellowships, and current barriers include faculty and learner time, implementation of best practices in simulation; equipment is less significant. Future work should focus on curriculum and evaluation development, aligning with the milestones.
Safe and effective pediatric patient care requires well-trained providers with a broad set of clinical skills. Published objective assessments of pediatric residents’ and fellows’ skills raise concerns about the quality of pediatric training in certain domains.1–3 Pediatric resident and pediatric emergency medicine (PEM) fellow success at pediatric intubation has been documented at less than 50% first-attempt success.4 Deficiencies in time-sensitive high-stakes and low-frequency clinical skills have the potential to negatively impact patient outcomes (examples: cardiac arrest, intubation, trauma). Reductions in trainees exposure to these procedures have been demonstrated in pediatric residency and fellowship training programs.5,6 This has been attributed to reductions in the total number of pediatric critical care events, duty hour limitations, reduced tolerance for medical errors, and increased attending presence. This problem is most challenging for acute care specialty training programs such as PEM.
Simulation affords training programs opportunities for skill development in a safe environment.7 Additionally, simulation provides a forum for standardized assessments that will be required in the Accreditation Council on Graduate Medical Education Next Accreditation System.8 The evidence supporting simulation-based medical education has dramatically increased in the past decade.9,10 A 2011 systematic review of health care simulation analyzed 609 studies and revealed that “technology-enhanced simulation training in health professions education is consistently associated with large effects for outcomes of knowledge, skills, and behaviors and moderate effects for patient related outcomes.”10 A pediatric specific analysis by Cheng et al9 noted large effects in comparison with no intervention across 57 studies in pediatrics.
A previous survey of PEM fellowships’ simulation use identified barriers including lack of simulation equipment, dedicated space, and faculty skill sets to implement effective simulation-based education. In 2008, 37% of responding PEM fellowships did not use high-fidelity simulation.11 Regional “boot camps,”12 in-situ simulation, and faculty training were identified as possible solutions. This study aims to describe the current use of simulation and barriers to its implementation in PEM fellowship programs.
Survey Development and Administration
A 50-question survey was developed by PEM simulation experts in collaboration with representatives from the American Academy of Pediatrics Section on Emergency Medicine Program Directors Committee. A group of PEM program directors and simulation experts used consensus methods to develop the survey, which was subsequently reviewed for face and content validity and pilot tested. The primary aim of this survey was to describe the use of simulation in PEM fellowships in the 2011–2012 academic year, including applications of simulation, barriers to its use and possible solutions. The goal of this survey is to inform future national efforts to develop PEM fellowship simulation.
The survey was sent via a Web-based survey tool (www.surveymonkey.com; SurveyMonkey, Inc, Palo Alto, CA) and was distributed to the American Academy of Pediatrics Section on Emergency Medicine Program Directors e-mail list, as well as to US and Canadian program directors listed in the 2012 PEM fellowship programs listing in Pediatric Emergency Care.13 An initial e-mail and 4 follow-up e-mails over 6 months (May to October 2012) were sent to the 73 program directors, as well as associate directors when available. Program directors were asked to fill out the survey themselves, or to forward to their faculty member most knowledgeable about PEM fellowship simulation activities. The results of the 2008 survey on this topic were used with permission for comparison.
The survey (Supplemental Material) assessed the following: (1) current use of simulation including specific content areas such as medical decision-making, technical skills, and teamwork skills; (2) plans for future use of simulation; (3) simulation resource availability including curricula, equipment, trained faculty, and funding; and (4) barriers to use. The initial version of the survey was developed and substantially revised from on the previous survey from Eppich et al.11
At the start of the survey simulation-based training was defined broadly to include a range of tools or techniques used to recreate reality, including the following: task or procedural trainers, simple mannequins that support bag-mask ventilation, high-fidelity simulators that can be programmed to respond to medical interventions, and simulated patients (actors who portray patients or family members).
Data Analysis/Institutional Review Board
Responses were anonymous and data were presented in aggregate. Participation in the study implied consent. Baylor College of Medicine Institutional Review Board approval was obtained.
Sixty-nine out of 73 programs (95%) responded to the survey. Sixty-seven (97%) reported by using simulation-based training for PEM fellows, and the remaining 2 programs planned to begin in 1 to 2 years. Respondents highly valued simulation training, agreeing or strongly agreeing that they would allocate additional resources for simulation because it teaches procedural skills (96%), teaches complex medical decision-making (93%), improves ability to function as a team leader in multidisciplinary teams (92%), improves ability to assess competency leading resuscitations (92%), and improves ability to function as a team member in multidisciplinary teams (88%). Over one-third of programs revealed that their fellows spend more than 20 hours annually in simulation. Not all 67 respondents who use simulation answered all questions, perhaps due to the length of the survey. As such, the denominators in Tables 1, 2, and 3 reflect the number who responded to each question.
PEM fellowship simulation activities are taught by a wide range of educators, most commonly including PEM, pediatric critical care, and general emergency medicine faculty (Table 1). Funding sources vary with 21% having no funding and 57% not incurring direct costs to the fellowship program (Table 1). Physical resources, including both simulators and simulation laboratory space, are widely available to PEM fellowships (Table 2).
When asked which shared resources would be most useful in expanding simulation in their fellowship, validated procedural assessment tools were ranked most highly (69%), followed by a PEM major procedures curriculum (63%). More than 50% also identified the following as very useful: a case-based curriculum meeting PEM core competency requirements, case-based assessment tools, validated teamwork assessment tools, and faculty development tools. Availability of simulation experts for consultation was considered least useful; however, 39% still ranked this as extremely useful.
Table 3 and Fig 1 illustrate specific uses of simulation in PEM fellowships. Although many programs currently teach a variety of procedures by using simulation, fewer programs have specific curricula or assessment tools for these skills. PEM fellows have also taken on the role as teachers using simulation (51/66 = 77%). Fellows incorporate simulation into teaching residents (96%), medical students (53%), nurses (47%), and other fellows (29%). Fellows have conducted simulation-based research as their scholarly activity at one-third of programs, and 56% of the programs perform simulation-based research within their division.
Although 97% of PEM fellowships use simulation, only 36% have formalized fellowship simulation curricula (Table 4). Of those with a curriculum, 83% include overall goals and objectives for the fellowship, but only 9 (38%) have a teamwork and communication curriculum. In addition, 6 programs (24%) reveal having debriefing curricula. Although curricular integration is considered an important component of effective simulation-based training,7 only 14 (58%) of PEM fellowships with simulation curricula reveal that their simulation curriculum is integrated into their overall curriculum.
One-third of programs revealed that there were no barriers to simulation. The remaining programs noted barriers that included faculty protected time (49%), faculty experience with simulation (39%), support for nonphysician staff attendance (35%), lack of an established curriculum (32%), and the ability to schedule simulation center time (26%). Physical space for simulation (21%) and equipment (20%) were less commonly cited barriers.
Regional boot camps for fellows are used by 32% of programs, and 18% of programs hope to use them in the future. Twenty-nine percent are able to provide a local boot camp. Barriers to regionalization include financial support (26%), regional availability (29%), and clinical scheduling (15%).
Our results document the nearly ubiquitous use of simulation in PEM fellowships. This widespread use has also been noted in emergency medicine residency programs and neonatology fellowships.14,15 The 97% of programs using simulation in the current study is a substantial increase from only 63% of programs described in a previous study in 2008.11 Compared with this previous study, the absolute number of programs using simulation has increased for each application since 2008. However, because of the larger denominator of programs using simulation in 2012, with a wider range of applications, the percentage of programs using each application has decreased.
PEM fellowships now use simulation broadly, to teach medical decision-making, technical skills, and teamwork and communication. Simulation is also being employed as a tool to assess PEM fellow competency in both technical and nontechnical skills. PEM programs continue to value simulation for procedural training and increasingly recognize the value of simulation for complex decision-making skills and team training. Although respondents highly valued simulation because it improves teamwork skills (up to 92% for team leadership), 58% reported specific opportunities for teamwork or crisis resource management training. We hypothesize that this may reflect the multifocal nature of many simulation curricula, which often focus simultaneously on cognitive, technical, and behavioral skills. Thus, although teamwork may be an important objective in most team simulations, fellowship directors may not have reported it because it is only 1 part of a simulation focusing on medical or traumatic events. Although teamwork may be seen as important, it does appear that fewer programs have dedicated curricula, highlighting an opportunity for future curriculum development.
Current PEM fellowships reveal increased access to simulation space and equipment, including high-fidelity mannequins. The number of programs with access to high-fidelity mannequins and simulation laboratories has substantially increased since 2008, when 74% of programs not using simulation revealed that lack of equipment was a primary barrier, and 68% revealed a lack of dedicated space.11 Currently, 89% have a simulation laboratory at their institution. Ongoing limitations include the need for more faculty development and protected time, consistent with evidence from emergency medicine residencies.15 Additional limitations include lack of written curricula and validated assessment tools. Simulation and educational experts must continue to advocate for simulation funding, including support for faculty time to rigorously develop, implement, and evaluate curricula and assessment tools. Incorporating the Accreditation Council on Graduate Medical Education competencies and milestones into these assessments would facilitate trainee evaluation in the Next Accreditation System.8,16 PEM is 1 of the first subspecialties to transition to the Milestones Project. The Milestones Project has involved PEM leaders identifying unique skills essential for achieving competence in their field. These subcompetencies are contained within 6 core competency domains common to all specialties and are not just skills or knowledge based. For each subcompetency, 4 to 5 performance levels have been defined, consisting of milestones, measurable attributes or behaviors, marking the learner’s progression toward mastery of each skill. Simulation and clinical assessment tools must be developed and validated to describe clear expectations for the performance so that educators can guide trainees to develop their skills. Without objective assessment methods, we risk graduating physicians who lack the skills necessary to provide high quality and safe patient care.
Pediatrics is currently at the forefront of collaborative simulation based research and education. The International Network for Simulation-Based Pediatric Innovation, Research, and Education (INSPIRE; www.inspiresim.com) is a collaboration of researchers, clinicians, and educators from across the globe, who work collaboratively to share their expertise and energy related to pediatric simulation. This pediatric simulation research network aims to improve pediatric medical care by answering research questions pertaining to pediatric resuscitation, technical skills, behavioral skills, and simulation-based education. Within INSPIRE there is a specific working group focusing on the subspecialty of PEM. The network’s ongoing and completed studies cover topics that have been identified in this survey including the following: a description of best practices in simulation-based research,9 the development of a framework for procedural skills training,17 work on debriefing in PEM fellowship programs,18,19 and regional simulation-based PEM training courses. Content created by members has been shared within the network and through sites such as MedEdPortal.20–23 Results from this survey should further inform the direction of these efforts within PEM.
PEM fellows increasingly teach by using simulation, but debriefing curricula are lacking in most fellowships. Simulation is also increasingly a part of PEM educational research (44% in 2008, 56% currently) including PEM fellows’ scholarly projects (33% currently). As such, PEM faculty will be called upon to mentor fellows in simulation-based educational research. Faculty and fellow development is critical to the future development of PEM simulation expertise. Shared curricula on simulation instruction, debriefing, and educational research, as well as programs at national meetings, such as the National PEM Fellows Conference, would improve simulation education and availability of simulation-trained educators over time. In addition, simulation curricula, when they exist, largely do not integrate with the program’s fellowship curriculum. Future efforts should focus on curricular development and integration into the broader fellowship curriculum, to optimize learning and assessment within the Next Accreditation System.16
In 2008, 45% of program directors believed that regionalization would increase simulation use.11 Currently, one-third of programs send their fellows to regional boot camps and 18% would like to in the future, remaining barriers include cost, regional availability, and clinical coverage. Programs able to provide simulation locally may be less likely to participate in regional programs. Thus, although regional solutions may play a role, they may not be effective for all programs, and may be less needed as local simulation resources become more widespread.
Our study has several limitations. A high response rate makes us confident in our results; however, nonrespondents may differ from respondents. However, even if all nonresponding programs do not use simulation, there has still been a dramatic increase in simulation use in PEM fellowships, and shift in the current challenges facing simulation. Although our survey was validated for face and content validity, and based upon previous studies, it has not been quantitatively validated.11 Finally, PEM program directors or their delegates may not have had full knowledge of all fellowship simulation activities, such as on outside rotations. In addition, we did not explore the use of crisis resource management and team training programs at the granular level of the case content. This information could help to describe where these skills are being taught in simulation curricula and if they are being reinforced on an ongoing basis across these curricula.
Almost all PEM fellowships currently use simulation-based medical education, including high-fidelity simulation. Future efforts should include development and validation of simulation-based curricula and assessment tools across institutions, as well as a focus on faculty development, to increase high-quality simulation-based education at every PEM fellowship. In addition, with now ubiquitous use, future studies should evaluate simulation quality and outcomes including evidence of fellows’ learning and patient-based outcomes.
Simulation is now ubiquitous in PEM fellowship programs and is being applied to a wide spectrum of training topics. The barriers to using simulation described in the past are decreasing and the use of simulation is increasing. Shared curricula and assessment tools, increased faculty and financial support, and regionalization for programs with less local resources could further ameliorate barriers to PEM fellowship simulation.
The authors thank Sweta Bhargava for assistance with organizing references for this manuscript and assisting with the submission. The authors also thank the contributions of members of INSPIRE who have helped to shape this project.
- Accepted April 28, 2015.
- Address correspondence to Cara B. Doughty, MD, MEd, Baylor College of Medicine, 6621 Fannin St A2210, Houston, TX 77030. E-mail:
Drs Doughty, Kessler, and Auerbach conceptualized and designed the study, designed the data collection instruments, and drafted the initial manuscript; Drs Zuckerbraun, Stone, and Reid designed the data collection instruments, interpreted the results, and drafted the manuscript; Drs Kennedy and Nypaver reviewed and revised the data collection instrument and drafted the manuscript; and all authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Gaies MG,
- Landrigan CP,
- Hafler JP,
- Sandora TJ
- Falck AJ,
- Escobedo MB,
- Baillargeon JG,
- Villard LG,
- Gunkel JH
- Cheng A,
- Auerbach M,
- Hunt EA,
- et al
- Johnson L,
- Mu T,
- Sawyer T
- ↵Auerbach M, Chang T, Fein D, et al. A comprehensive infant lumber puncture novice procedural skills training package: an INSPIRE simulation-based procedural skills training package. MedEdPORTAL Publications; 2014. Available at: https://www.mededportal.org/publication/9724. Accessed May 3, 2015
- Auerbach M, Chang T, Krantz A, et al. Infant lumbar puncture: POISE pediatric procedure video. MedEdPORTAL Publications; 2011. Available at: https://www.mededportal.org/publication/8339. Accessed May 3, 2015
- Reid J, Stone K. Pediatric emergency medicine simulation curriculum: hypovolemic shock. MedEdPORTAL Publications; 2013. Available at: https://www.mededportal.org/publication/9452. Accessed May 3, 2015
- ↵Reid J, Stone K. Pediatric emergency medicine simulation curriculum: septic shock. MedEdPORTAL Publications; 2013. Available at: https://www.mededportal.org/publication/9639. Accessed May 3, 2015
- Copyright © 2015 by the American Academy of Pediatrics