 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ELECTRONIC ARTICLE |
* Department of Pediatrics, University of Washington School of Medicine, Seattle, Washington
University of Utah School of Medicine, Salt Lake City, Utah
Children’s Hospital and Regional Medical Center, Seattle, Washington
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Design. The resuscitation skills of all 39 pediatric R1’s in 2 university-based training programs were assessed immediately before and after completion of a PALS course just before starting residency. Independent observers scored and timed resident performance of bag mask ventilation, endotracheal intubation, intraosseous access, and defibrillation. Scores before and after the PALS course were compared. Four independent observers scored 4 residents’ videotaped skills performance. Observers’ scores for each resident were compared.
Results. Successful performance improved for bag mask ventilation from 62% to 97% after the PALS course; for successful endotracheal intubation, from 64% to 90%; for successful intraosseous needle placement, from 54% to 92%; and for successful defibrillation, from 77% to 97%. Interobserver reliability was high for continuous and noncontinuous variables.
Conclusions. New residents demonstrated significant acquisition of pediatric resuscitation skills immediately after completion of the PALS course. The skills performance checklist has excellent interobserver reliability and is a useful tool for evaluation of other training venues.
Key Words: resuscitation • education • advanced life support • training
Abbreviations: AHA, American Heart Association • PALS, pediatric advanced life support • BVMV, bag valve mask ventilation • ACLS, advanced cardiac life support
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Advanced resuscitation courses teach both cognitive knowledge and psychomotor skills. Assessment of students’ cognitive knowledge acquisition is accomplished by written examinations for which specific pass grades are established. On the other hand, assessment of complex psychomotor skills, such as intubation and bag valve mask ventilation (BVMV), is less structured and allows for greater instructor variability in assessing students’ competency.2 Furthermore, there are no standard, objective, validated, uniformly accepted methods of assessing competency in these technical skills. Reliable, objective measures are needed.
In a previous study, we noted that pediatric residents who had completed a PALS course demonstrated superb cognitive knowledge of resuscitation but variable skills performance 1- to 2-years posttraining.3 Although the Neonatal Resuscitation Program improved both knowledge and skill performance of students, the AHA PALS course has been shown only to improve acquisition of cognitive knowledge among Taiwanese physicians and nurses.4,5,6 There have been no studies evaluating the acquisition of resuscitation skills in the AHA PALS course.
We sought to determine the interobserver reliability of the performance skills checklist that we had developed to assess resident performance of bag mask ventilation, intubation, defibrillation, and intraosseous vascular access.3 In addition, we wanted to determine the acquisition rate of performance skills as a test of the PALS course’s teaching effectiveness and to identify lack of proficiency in steps of performance skills to determine aspects of the PALS course that need greater emphasis.
|
METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
After giving informed consent, each resident completed a questionnaire. In the questionnaire, residents rated their level of comfort with pediatric resuscitation on a scale of 1 (not at all comfortable) to 10 (extremely comfortable), the number of mock resuscitations and actual resuscitations they had attended, the number of resuscitation skills they had completed, as well as previous participation in an AHA PALS or advanced life support course (advanced cardiac life support [ACLS]). Actual resuscitations were defined as situations where intubation, bag mask ventilation, administration of resuscitation medications, or defibrillation was required.
Each resident completed 2 written case scenarios and identified 3 rhythm strips to assess their knowledge base. Residents were then asked to perform 4 skills, including BVMV, tracheal intubation, intraosseous line placement, and defibrillation. Each resident was asked to perform these skills on a Laerdal (Laerdal Medical Corporation, Wappingers Falls, NY) infant mannequin with inflatable lungs, a mannequin head with an intubatable airway, and a mannequin with an artificial tibia for intraosseous line placement. The study was conducted in the pediatric emergency department at Children’s Hospital and Regional Medical Center and the Education Skills Laboratory at Primary Children’s Hospital Medical Center. Equipment available for residents’ use was standardized for both institutions.
Performance was scored using a standardized checklist by the observers. The observers’ protocol, standardized checklist of scoring, and timing the steps and endpoint for each skill, have been previously described.3 Observers were PALS instructors, trained in the study protocol, and had not met the residents involved before the assessment. Observers noted whether the resident achieved the defined endpoint for each skill and whether the resident completed necessary steps for each skill. For example, for endotracheal intubation skill, the endpoint was visualization of the tube in the trachea while necessary steps included selection of the correct endotracheal tube size and correct laryngoscope blade size. If the endpoint was achieved, the skill was documented as successfully completed. Completing the necessary steps was not required for successful completion of the skill. At the completion of skills, observers gave the resident feedback about his or her performance.
Residents then attended a 2-day AHA PALS course. During the final hour of the course, the residents were asked to complete the same evaluation of cognitive knowledge and to perform the same 4 skills in the same manner as previously described.
To evaluate interobserver reliability/validity of the scoring system, we made a video recording of a convenience sample of 5 pediatric residents performing each of the 4 psychomotor skills immediately after completing the PALS course. Four independent observers trained in the protocol (2 from Seattle and 2 from Salt Lake City) watched the video and scored and timed each of the residents’ skills performances using the checklist. None had been instructors for this course. There were a total of 42 tasks on the checklist for skills performance of all 4 skills. We compared observers’ scores for the 42 tasks using the kappa statistic for multiple observers when the score was categorical and using the intraclass correlation coefficient when the score was a continuous variable, such as time to complete a task.7,8
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Of the 22 new residents from the University of Washington and 17 new residents from the University of Utah, all 39 agreed to participate in the study. None had previously taken a PALS course and 24/38 (63%) had previously taken ACLS. There was no statistically significant difference between residents from the 2 programs with respect to previous training. Before taking the PALS course, new residents reported a mean level of comfort with resuscitation skills of 1.9 (95% confidence interval: 1.6–2.3) on a 10-point scale. After completion of the course, the mean level of comfort with resuscitation increased to 6.2 (95% confidence interval: 5.5–6.9).
For the knowledge-based questions, responses improved after the course but the differences were not significant. All (100%) residents recognized the shock scenario before (39/39) and after (29/29) the course; 72% (28/39) recognized the potential respiratory failure scenario before the course and 83% (24/29) after; 100% recognized sinus bradycardia before (39/39) and after (29/29); 74% (29/39) recognized ventricular fibrillation before and 93% (27/29) after; and 67% (26/39) recognized supraventricular tachycardia before and 83% (24/29) after the course. All residents passed the written PALS examination at the end of the course.
Participation in the PALS course resulted in a significant increase in the number of residents who achieved the defined endpoint for each of the resuscitation skills (Table 1). Improvement occurred for each of the tasks and ranged from a 27% increase in the number who were able to discharge the defibrillator to a 71% increase in the number who were able to place an intraosseous needle in the mannequin’s tibia. The time required to complete resuscitation skills showed a trend toward improvement for all skills except endotracheal intubation.
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
Importantly, we demonstrated the interobserver reliability of our checklist scoring assessment of resuscitation skills. We believe its reliability is based on the use of a 2-point, "pass" or "fail", scale rather than a more expanded scale in the observation of very specific, objective steps in each skill. Halamek1 noted that many pediatric residency programs require documentation of skills with the expectation that the residents will become proficient in them. However, Halamek also noted the lack of uniformly accepted methods of assessing skill competency leaves this responsibility to hospitals and training programs.1 Our checklist addresses this lack of standard objective methods of assessing competency in these clinical skills. In addition, the reliability of this tool makes it a useful instrument for research purposes.
The usefulness of the skills checklist is not limited to assessing competency post-PALS training. There is growing interest in alternative strategies and methods for teaching advanced life support. The skills checklist could also be used to evaluate the resuscitation skills of experienced practitioners or made part of the AHA PALS renewal course to identify individual students’ needs. A key advantage of the checklist is that it allows the student and the instructor to identify specific tasks or steps that were either forgotten or represent a motor skill problem for the student. It also provides the student an impetus to coordinate all tasks because time to complete tasks is an outcome measure. By focusing on individual students’ specific needs, training could be more efficient.
Our study is limited in that we only assessed performance of complex sequences of psychomotor skills immediately after completion of a training course. Studies of other complex psychomotor resuscitation skills, especially cardiopulmonary resuscitation, show that these psychomotor skills are difficult to acquire and that it is critical to prove they are acquired despite completion of accepted training courses.10 Therefore, we thought it was key to determine first whether these skills were learned during the PALS course and to determine which aspects, or subcomponent steps, were not learned.
We broke down the skills to very basic steps to allow the identification of specific areas of weakness. Notably, for 10 of the 22 subcomponents tested, the percentage of residents correctly performing the subcomponents remained less than 90%, even after completing the course. The subcomponents that were performed the least were performance of BVMV without a leak, checking the pulse after defibrillation, checking that the suction was on, and checking the BVMV was functioning before intubation. Remarkably, these subcomponents were also the least likely performed by White’s cohort of residents 1 to 2 years posttraining.3 These steps were not emphasized in the PALS text or any of the stations in the 1994 PALS course, but were only mentioned in the PALS instructor manual checklist that residents did not review. This suggests that the problem rested with the lack of emphasis or teaching of the step in the course rather than any learning difficulty on the part of the residents. For example, many residents completed the endpoints of the skills but had difficulty turning on the defibrillator or the suction. During the PALS course, these are often already turned on for the student. All of the subcomponents but 2 are mentioned in the provider’s manual. However, these subcomponents are not in a checklist format for the provider and could be easily missed. In addition, instructors may not have identified the subcomponent steps in the teaching stations. Thus, we identified the need to provide the full sequence of steps to residents as part of their learning materials and to provide more hands on practice during the course.
We did not evaluate residents’ use of the skills, recognition of indications for the skills, promptness of initiation of the skills, or the ultimate outcome measure, improved performance during an actual resuscitation and improved survival. One study from the United Kingdom showed outcome from prehospital pediatric cardiac arrest in an emergency medical services system was not affected by the institution of pediatric life support courses.11 However, ACLS training has been shown to improve survival immediately after resuscitations but not long-term survival.12 Given the extremely low survival rate of pediatric resuscitations and the many factors that determine patient outcome, survival may be a poor measure of the efficacy of pediatric resuscitation training. A better outcome measure would be early recognition of respiratory distress and failure.
We did not attempt to test retention of skills performance. In the year before our study, White evaluated the resuscitation skills performance among pediatric residents of all levels in the Seattle program and showed the residents had variable success in performing these skills 1- to 3-years post-PALS course. Not surprisingly, the percentage who performed tasks correctly in White’s cohort was lower or the same in almost all endpoints and subcomponents. This is not surprising, given that all studies of performance after other advanced life support courses, such as ACLS and the Neonatal Resuscitation Program, show major declines in retention of learned skills 1 year after the course.13,14 Additional resuscitation training efforts should concentrate on efficient ways to retain skills.
|
CONCLUSION |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
|
ACKNOWLEDGMENTS |
|---|
|
FOOTNOTES |
|---|
Reprint requests to (L.Q.) Emergency Services CH 04, Children’s Hospital and Regional Medical Center, 4800 Sand Point Wy NE, Seattle, WA 98105. E-mail: lquan@chmc.org
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMETHODSRESULTSDISCUSSIONCONCLUSIONREFERENCES |
|---|
This article has been cited by other articles:
 |
|
 |
|
  J. M. Gerard, A. J. Scalzo, S. P. Laffey, G. Sinks, D. Fendya, and P. Seratti Evaluation of a Novel Web-Based Pediatric Advanced Life Support Course Arch Pediatr Adolesc Med, June 1, 2006; 160(6): 649 - 655. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2001 American Academy of Pediatrics. All rights reserved. |
||
|
||
