 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
* Department of Pediatrics, University of Texas Health Science Center at San Antonio, San Antonio, Texas
Department of Pediatrics, College of Medicine, University of Oklahoma, Oklahoma City, Oklahoma
University of Texas Health Science Center at San Antonio, San Antonio, Texas
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Methods. Indication for intubation, number of attempts, and achievement of success were recorded by the respiratory therapist present for the procedure. Each intubation was scored according to the attempt on which intubation was successful. Indications for intubation were categorized as respiratory failure, delivery room resuscitation, and meconium-stained amniotic fluid. Competency was defined as a successful intubation occurring on the first or second attempt 
80% of the time. Intubation scores were compared between residents at various stages of training and analyzed by multivariate logistic regression analysis for significance. Comparisons were then performed to determine percentage success with confidence intervals. We also surveyed previous graduates of the training program not included in the observations for this study and asked them to indicate how frequently they perform intubation in current practice and to assess their own competence in the procedure.
Results. A total of 449 resident procedures were observed during the study periods: 192 by postgraduate year 1 (PGY-1) residents, 126 by PGY-2 residents, and 131 by PGY-3 residents. A total of 35% (160 of 449) of intubation procedures were never successful by pediatric house officers. Intubation was successful on the first or second attempt for 50% of PGY-1 residents (95% confidence interval [CI]: 42.6–56.8), 55% of PGY-2 residents (95% CI: 46–63.5), and 62% of PGY-3 residents (95% CI: 53.9–70.7). The third-year residents exhibited a significantly higher likelihood of performing a successful intubation compared with first-year residents. The first-year residents in 1998–1999 showed no improvement by their third year in 2000–2001. Surveys were sent to 56 graduates of our residency program (1998–2000). Completed surveys were received from 31 (66%) of 47. A total of 71% of the respondents are practicing general pediatrics, and 36% attend deliveries or perform intubations. A total of 87% reported that their level of confidence with endotracheal intubation was good or excellent after completion of residency training.
Conclusions. We provide objective and subjective data concerning the proficiency of pediatric residents in performing neonatal endotracheal intubation. None of our resident groups met the specified definition of technical competence, although there was improvement with advancing training level in bivariate analyses. However, graduates of our training program felt confident with their intubation skills in contrast to our objective findings. As exposure to these important skills becomes limited, methods to ensure attainment of technical competency during training may need to be redefined.
Key Words: residency education • intubation • neonatal resuscitation
Abbreviations: NRP, neonatal resuscitation program • RRC, Residency Review Committee • NICU, neonatal intensive care unit • PGY, postgraduate year • MSAF, meconium-stained amniotic fluid • CI, confidence interval • OR, odds ratio
Guidelines of the Neonatal Resuscitation Program (NRP) developed by the American Academy of Pediatrics dictate that people who are trained in basic skills of resuscitation should be present at every delivery. In addition, individuals who are able to perform a complete resuscitation, including endotracheal intubation, should be immediately available.1 The World Health Organization estimates that 
20% of the 5 million infant deaths each year are attributable to birth asphyxia, so skillful resuscitation could potentially affect 1 million infants per year.1,2
In many settings, the general pediatrician is responsible for managing delivery room resuscitation and stabilizing infants in distress. Therefore, gaining knowledge and experience in neonatal resuscitation and acquiring proficiency in neonatal endotracheal intubation are important skills for the pediatrician to acquire. Attainment of these skills should be emphasized during pediatric residency training.
In 1996, the Accreditation Council for Graduate Medical Education and the Residency Review Committee (RRC) for Pediatrics revised requirements for pediatric residency education. The 1996 guidelines state that neonatal intensive care curricula must be structured to teach residents to perform delivery room resuscitation and stabilization of infants. Residents are required to learn procedural skills appropriate for the general pediatrician, including endotracheal intubation. Experience should be graduated so that residents build and maintain skills throughout their training program.3 However, these training guidelines also have restricted the amount of intensive care experience and exposure to neonatal resuscitation obtained by pediatric residents. A maximum of 6 months of intensive care is allowed, including both neonatal and pediatric intensive care rotations. Three blocks of neonatal intensive care are required but can be in nurseries classified as either level II or level III. Night and weekend coverage while on other rotations is included in this maximum period, with 200 hours considered the equivalent of 1 month. For residents who desire additional experience, 1 elective block is allowed.3 Therefore, it is possible for a resident to complete pediatric training with only 3 rotations in neonatal intensive care, with variability in the acuity of patient care.
There are no data from the period before the RRC guideline changes, so the impact of the changes on developing competency in procedures such as endotracheal intubation can no longer be assessed directly. The goal of this study was to provide objective and subjective data evaluating the proficiency of pediatric residents in performing neonatal endotracheal intubation under the current guidelines. This information can provide a basis of comparison for any future changes and allows us to speculate about whether the previous changes were warranted.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
All residents had received neonatal resuscitation training and participated in an intubation skills workshop at the beginning of their internship year. The intubation skills workshop consisted of a review of the procedure and necessary equipment, with practice in an animal laboratory under the guidance of neonatal faculty. Postgraduate year 1 (PGY-1) and PGY-2 residents rotated in the term newborn nursery for 1 month and attended all routine deliveries. During the newborn nursery rotation, PGY-2 residents had a supervisory role in the delivery room. NICU experience consisted of 2 months for PGY-1 residents, 1 month for PGY-2 residents, and 1 month for PGY-3 residents. Residents attended all high-risk deliveries during their intensive care experience.
There are 35 beds in the NICU at University Hospital with approximately 3000 deliveries per year. Respiratory therapists are present for all endotracheal intubations, both in the delivery room and in the NICU. Data were collected and recorded by the respiratory therapist who was present for each intubation attempt. Information recorded included the date, birth weight of the infant, indication for intubation, number of intubation attempts by each individual, and whether the attempt was successful. Indications for intubation were categorized as respiratory failure, delivery room resuscitation, and meconium-stained amniotic fluid (MSAF). All intubation attempts performed for respiratory failure were performed in the NICU. The number of previous attempts by other individuals were documented. Success was determined by physical examination and radiographic evaluation when available. Transcutaneous blood gas monitoring was not used to determine success for this study. For MSAF, successful intubation was determined subjectively by the individual performing direct laryngoscopy. When the resident was unable to achieve success or the infant became unstable, intubation was performed by a neonatology fellow or the attending neonatologist. Usually, a resident was given at least 2 intubation attempts with a maximum of 4 attempts allowed.
Our study objective was to determine whether residents are able to achieve competency in endotracheal intubation during their training period; however, defining competency is difficult as there are no standard objective measures. Most of the published reports evaluating technical competency are found in the gastroenterology literature.4–6 For this study, the definition was adapted from Jowell et al,4 who evaluated the number of supervised endoscopic retrograde cholangiopancreatography procedures performed by gastroenterology fellows that were required to achieve technical competence. They believed that the likelihood of performing a successful procedure is a better indicator of proficiency than the total number of procedures performed per se, so they defined competency in endoscopic retrograde cholangiopancreatography procedures as successful completion of the procedure in 80% of attempts.4 Because there is no precedent for our specific procedure, we adapted the definition of Jowell et al to define competency in endotracheal intubation. We scored intubation attempts as follows: 4 = success on the first attempt, 3 = success on the second attempt, 2 = success on the third attempt, 1 = success on the fourth attempt, 0 = no success. Competency was defined as a score of 3 or 4 for 80% of intubation attempts.
The percentage distributions of successful intubations and potentially confounding clinical characteristics were estimated across each of the resident training level groups. For assessing differences in prevalence across the subgroups, 95% confidence intervals (CIs) were generated for each estimate. Subgroups with 95% CIs that did not overlap were considered to have exhibited statistically significant differences. Logistic regression was used to assess the association of the explanatory variables (level of resident training, indication for intubation, and number of previous attempts at intubation) with the dichotomous response variable successful intubation as previously defined. The results were analyzed by multivariate logistic regression analysis to determine significance, then compared via percentage success and odds ratios (ORs) with CIs.
Scores were also evaluated for a cohort of residents who participated in the study during their PGY-1 and PGY-3 years to assess for improvement during residency training. In addition to receiving NRP training with an intubation workshop during internship, this cohort of residents participated in a 1-month pediatric procedures elective rotation during which they performed supervised intubations under the guidance of pediatric anesthesiology.
An additional goal of this study was to evaluate the perceptions of recent graduates of our training program about their intubation skills after completion of training and their experience with endotracheal intubation in clinical practice. We sent a mail survey to graduates from the 1998–2000 academic years with questions to determine their area of practice, opinions of intubation skills before completion of residency, and frequency of endotracheal intubation in the practice setting. We did not ask the graduates to evaluate their own competence in intubation according to our study definition; rather, they were asked to indicate whether their "confidence" with their own intubation skills at completion of residency training was excellent, good, or poor.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
|
Examination of the association between indication for intubation and resident training level shows that PGY-1 residents performed more intubations for the indication categorized as respiratory failure than PGY-2 and PGY-3 residents. This elevated rate was statistically significant only in the comparison of first- to second-year residents. No statistically significant differences in the proportions of the 2 other indications for intubation (delivery room resuscitation, MSAF) were exhibited across resident training level.
Table 1 further shows that the frequency of success in intubations that were presumably more difficult because they had already been attempted by another clinician increased in a stepwise manner according to resident training level. In fact, examination of the 95% CIs associated with these estimates shows that PGY-1 residents exhibited a statistically significantly decreased proportion of difficult intubations compared with PGY-2 and PGY-3 residents. This result ought to be expected in a residency training program, in which first-year trainees are usually offered the first attempt at procedures.
Table 2 represents the results of a logistic regression model used to examine the influence of training level and successful intubation on the dichotomous response variable with simultaneous adjustment for indication for intubation, whether previous attempts had been made, and the date of the intubation. Consistent with the bivariate analysis, the model shows that there is a stepwise increase in predicting successful intubation with each year of resident training, but unlike the bivariate analysis, the third-year residents demonstrated a statistically significantly higher likelihood of performing a successful intubation compared with the PGY-1 reference group. In fact, PGY-3 residents were twice as likely as PGY-1 residents to perform an intubation successfully after adjustment for potential confounding variables.
|
Despite an apparent difference in the number of successful intubations in the second half of the training year compared with the first half of the year, the difference was not statistically significant. Although not shown in Table 1, there was also no statistically significant change in intubation success rate based on infant birth weight.
For the cohort of residents evaluated during both their PGY-1 and PGY-3 years, there were 103 intubations recorded during the PGY-1 academic year and 61 intubations recorded during the PGY-3 academic year. Successful intubation on the first or second attempt was achieved by 60% of the group when they were PGY-1 residents (95% CI: 46.2–65.2) as compared with 75% when they were PGY-3 residents (95% CI: 63.2–84.6). This group also did not meet our definition of technical competence and did not show statistically significant improvement from their first to third year of residency training. Individual skill varied, but there was an inconsistent number of procedures performed by individuals within the cohort to draw any specific conclusions regarding skill level.
To determine the perception of pediatric residents about their intubation skills after completion of residency and their incorporation of these skills into clinical practice, surveys were sent to 1998–2000 graduates of our pediatric residency training program and surveys were returned by 66% (31 of 47) of graduates. Seventy-one percent are practicing general pediatrics, and 36% attend deliveries and/or perform endotracheal intubation.
A total of 87% of all respondents reported that their level of confidence with intubation skill was good or excellent after completion of residency training. Among surveyed graduates, those who currently perform intubations in their clinical practice exhibited a higher proportion of self-reported perceived competence than those who did not currently perform intubations. As evidenced by overlap of the 95% CIs associated with these estimates, this difference was not statistically significant.
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
80% success on first or second intubation attempt. Intubation skills have been shown to be used frequently by the general pediatrician, especially in more rural communities. In 1995, Feigin et al7 surveyed 597 pediatricians in Texas communities of different sizes to determine practice variation. They found that 70% of pediatricians surveyed take care of newborns and provide some degree of level II care including stabilization of critical infants for transfer to a tertiary facility. Physicians in more rural communities (population <100 000) were significantly more likely to provide intermediate and intensive care than those in urban communities. Of the pediatricians surveyed, 81% of those practicing in rural areas perform endotracheal intubation as compared with 51% practicing in urban areas.7 The latter figure is closer to our own estimate of 36% provided by the survey of graduates of our training program. The proportion of pediatricians who intubate neonates varies by locale and represents large numbers of patients. These results emphasize the importance of acquiring skills necessary for successful resuscitation of newborns during residency.
There is scant literature about what constitutes technical competency, and the information available varies among training programs in different medical specialties. Many training programs seem to be designed to determine technical competency after observation and/or performance of a certain number of procedures. Our findings suggest, however, that the number of procedures needed to become competent cannot be established a priori, and competency is better defined by each physician’s clinical knowledge, judgment, motivation, and interpersonal and technical skills. Thus, a more individualized approach adapted to the resident’s skill level and goals for future practice may help to improve development of proficiency.8
According to cognitive learning theory, there are 5 basic principles that help to ensure the attainment of procedural competence. First, content should be relevant to the individual’s ultimate goal, in this case providing quality patient care. Second, the outcome gained by correctly performing the procedure becomes a motivating force for continued learning. The knowledge should be placed in a familiar framework, such as the delivery room. The individual should practice by active involvement, because repetition allows for skill refinement. Finally, self-evaluation tools are necessary for continued use of the procedure.8 These important principles provide an excellent framework for developing a teaching approach that both is motivating for residents and ensures development and maintenance of procedural skills.
Direct observation of a procedure by experienced practitioners provides one of the best measures for assessment of proficiency. However, it has been shown that supervision of residents in the delivery room in many training programs is performed by upper-level residents.9 Supervision by those with more experience assumes good role models and that their attributes can be incorporated into the behavior of the trainee,9,10 but our data suggest that the skill of the more senior person may be only marginally better than the junior in many cases. It might be appropriate to question the time-honored tradition of "see one, do one, teach one." In our training program in San Antonio, mandatory in-house call for attending neonatologists and neonatal fellows was implemented in 2001. Neonatal faculty are now present in the hospital at all times and are available to supervise intubations and attend high-risk deliveries as necessary. Although not measured in the study, it seems that providing more direct supervision in the delivery room and the NICU has improved our awareness of residents’ skill level and has provided the neonatal faculty more opportunity to take an active role in teaching procedural skills and identifying those who require assistance.
There are flaws in our study in ensuring the accurate assessment of successful intubation. First, it is possible that not every intubation attempt was captured during the 2-year study period. Although inevitable, there is no reason to suspect bias in attempts that were captured versus those that were not captured. Second, determining whether an intubation was successful was not always confirmed, and required some clinical judgment from the respiratory therapist. In the case of MSAF, the determination was solely by report of the individual performing intubation. Even with these challenges, however, we sought to perform this study in the "real world" setting. Procedures such as endotracheal intubation are performed in the stressful environment of treating a critically ill infant, and this setting is difficult to simulate. Therefore, we elected not to use the techniques used by others of assessing skill in mock codes or skills workshops. These venues provide excellent education but may not demonstrate a practitioner’s true skill level.
The results of our study carry striking implications for residency training. Traditionally, resident training in neonatal resuscitation is initiated with participation in a formalized course in neonatal resuscitation such as NRP or pediatrics advanced life support, followed by graduated responsibility in the delivery room and the NICU. It is known that optimal acquisition and retention of knowledge and skills by adults is achieved by active participation and that repetition is very important to reinforce skills.10 However, there is significant variation in the experience with endotracheal intubation obtained by individual trainees because of time constraints. Therefore, a standardized approach to resident education with a formalized method of assessment and feedback could help to ensure the development and maintenance of skill throughout residency training. Methods of formalized assessment that could be effective in reinforcing skills include skill workshops and delivery room scenarios using manikins or animal labs. Simulation-based training systems have been used and enthusiastically received,10 but any system should be consistent and repeated periodically through training, given the variable nature of resident’s exposure to clinical rotations.
In summary, our study provides exploratory data describing the proficiency of pediatric residents in performing neonatal endotracheal intubation after the revision of RRC guidelines that result in limited intensive care exposure during residency training. Our study results indicate the need to include objective evidence in decisions to change the format of residency training. Additional changes have been recently ordered by the RRC that will limit hours of residency training. These new guidelines were implemented apparently without evidence-based evaluation of the impact that can be expected. With limitations on the amount of time that residents spend in critical care, approaches to residents’ training may need to be reexamined to ensure the development of technical competence. Primarily relying on the total number of procedures performed may not be a sufficient measure to ensure attainment of adequate skills. Possible approaches to teaching could include emphasizing successful completion instead of number of procedures performed, providing supervised and individualized learning with feedback, and developing new guidelines and evidence-based standards to aid in assessing proficiency.
|
ACKNOWLEDGMENTS |
|---|
We thank the respiratory therapy department at University Hospital, San Antonio, Texas, for assistance with data collection. We also thank the pediatric residents at University of Texas Health Science Center at San Antonio for participation in the study.
|
FOOTNOTES |
|---|
Reprint requests to (J.H.G.) Department of Pediatrics, University of Texas Health Science Center, Mail Stop 7812, 7703 Floyd Curl Dr, San Antonio, TX 78229. E-mail: gunkel{at}uthscsa.edu
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
This article has been cited by other articles:
 |
|
 |
|
  M. G. Gaies, C. P. Landrigan, J. P. Hafler, and T. J. Sandora Assessing Procedural Skills Training in Pediatric Residency Programs Pediatrics, October 1, 2007; 120(4): 715 - 722. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E M Dempsey, F Al Hazzani, D Faucher, and K J Barrington Facilitation of neonatal endotracheal intubation with mivacurium and fentanyl in the neonatal intensive care unit Arch. Dis. Child. Fetal Neonatal Ed., July 1, 2006; 91(4): F279 - F282. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. P.F. O'Donnell, C. O. F. Kamlin, P. G. Davis, and C. J. Morley Endotracheal Intubation Attempts During Neonatal Resuscitation: Success Rates, Duration, and Adverse Effects Pediatrics, January 1, 2006; 117(1): e16 - e21. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. A. Murphy and L. P. Halamek Educational Perspectives: Simulation-based Training in Neonatal Resuscitation NeoReviews, November 1, 2005; 6(11): e489 - e492. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2003 American Academy of Pediatrics. All rights reserved. |
||
|
||
