Objective. To assess the accuracy of pediatric residents in recognizing the physical examination findings of acute otitis media (AOM) and otitis media with effusion (OME), technical competence to perform tympanocentesis, and knowledge of guideline-recommended antibiotic management of AOM.
Methods. A total of 383 pediatric residents from various programs in the United States viewed 9 different video-recorded pneumatic otoscopic examinations of tympanic membranes during a continuing medical education course. The ability to differentiate AOM, OME, and normal was ascertained. A mannequin of a child was used to assess technical proficiency of performing tympanocentesis on artificial tympanic membranes. A series of questions was posed regarding appropriate, pathogen-directed, second-line antibiotic selection for AOM.
Results. The average ± standard deviation correct diagnosis on the otoscopic video examination was 41% ± 16% (range: 19%–70%; median: 38%) by pediatric residents, tympanocentesis was optimally performed by 89%, and appropriate antibiotic therapy for drug-resistant Streptococcus pneumoniae was selected by 78% and appropriate therapy for β-lactamase-producing Haemophilus influenzae was selected by 74%.
Conclusions. According to this video-based examination, pediatric residents misdiagnose OME frequently. Pediatric residents have the skills to be trained to perform tympanocentesis. Approximately 75% of pediatric residents have knowledge of the appropriate antibiotics to select for treatment of resistant AOM pathogens. Interactive instruction with simulation technology may enhance skills and lead to improved diagnostic accuracy and treatment paradigms.
Otitis media (OM) is 1 of the most common conditions encountered in community pediatrics practice,1 but there are no data on the adequacy of pediatric residency training for its diagnosis and management. In a recent article in Pediatrics, Steinbach and Sectish2 described the results of a survey of 144 pediatric training programs that found that approximately half had any formalized education in OM diagnosis and treatment and most of those included only infrequent lectures. Because acute otitis media (AOM) is the leading indication for antibiotic use in community practice,3 because there are diagnostic criteria distinguishing AOM (for which antibiotics are indicated) from OM with effusion (OME, for which antibiotics may be appropriately deferred),4–7 and because there are guidelines for judicious antibiotic use,8 the need for structured, consistent training of pediatric residents perhaps should be considered.
Most often pediatric residents learn how to diagnose and treat OM on a case-by-case basis.2 Usually this occurs without double-headed otoscopes or otomicroscopes to allow the instructor to ascertain whether the trainee recognizes the features of the otoscopic examination being taught. Furthermore, in a recent study of OM diagnostic accuracy, it was found that practicing pediatricians (the teachers of most pediatric residents) may not know themselves the differentiating features of AOM and OME, because their misdiagnosis frequency may be approximately 50%.9 In other words, the teachers may be passing on inaccurate information during the education process.
A continuing medical education (CME) workshop program was launched in January 1999 and conducted in >200 locations across the United States from 1999 through 2001. A total of 383 pediatric residents attended the workshops, along with 2190 academic- and community-based practicing pediatricians. They were shown video footage of otoendoscopic examinations that were either normal or from patients with AOM or OME. Their skills in performing tympanocentesis—as advocated by the Centers for Disease Control and Prevention’s (CDC’s) Drug-Resistant Streptococcus pneumoniae Therapeutic Working Group,10 the American Academy of Pediatrics,11 and several recognized authorities12–15—was assessed using a simulation mannequin model. In addition, their knowledge about appropriate antibiotic selection for AOM based on guidelines and recommendations10,11 for treatment of drug-resistant S pnemoniae and Haemophilus influenzae was tested.
A CME-accredited workshop, “Improving Outcomes in Acute Otitis Media,” was conducted in 240 US locations from January 1999 to December 2001 as previously described.9 Several locations were at approved pediatric residency training programs; the programs were diverse, including larger and mid-sized as well as academic and community based. Some pediatric residents attended at American Academy of Pediatrics annual chapter meetings and others at community- and hospital-based locations so that the geographic representation was broad. Data on the years of training were not collected. Each course participant answered questions with a wireless digitized personal keypad electronically linked to a laptop computer, creating the database on which this report is based. Each keypad was identified with an individual throughout the examination. Pre- and postcourse multiple-choice questions about the diagnostic features of AOM and OME, the indications for tympanocentesis, and appropriate antibiotic selection for AOM were asked and recorded. The course had 3 assessment components: diagnosis accuracy, tympanocentesis performance, and antibiotic management.
Description of Diagnostic Session
Knowledge of the distinguishing features of AOM and OME were assessed by examination based on video footage obtained with an otoendoscope. For familiarizing the pediatric residents with the expectations of the examination, 2 examples were shown: 1) a typical normal tympanic membrane (TM; translucent, slightly gray, fully mobile on pneumatic otoscopy, and with no evidence of middle ear effusion) and 2) a typical TM from a patient with AOM (bulging, red, opaque, and diminished mobility to pneumatic otoscopy with a middle ear effusion). Then, 9 sequential 30-second video clips of otoscopic examinations were shown: in still frame (approximately 10 seconds), then with pneumatic otoscopy (10 seconds), and then again still frame (10 seconds). Each video was projected onto a large screen in a classroom-style setting. All cerumen had been removed from the external auditory canal before filming. The pediatric residents were instructed to record the findings in a structured format and then reach a conclusion as to 1 of 4 possible diagnoses: AOM, OME, retracted TM but otherwise normal without effusion, and normal. The correct diagnosis had been established by consensus among an expert panel based on reviewing the video plus tympanometry and tympanocentesis findings. The video clip had usual and ordinary findings consistent with each diagnosis. As much time as necessary was afforded for each participant in the courses to reach a conclusion and record the answer. Different faculty taught the workshops (see Appendix) but the presentations were standardized; the material was scripted to minimize potential differences in emphasis or information. There was teaching during the video session so that participants’ understanding about the criteria to differentiate the diagnoses had the potential to improve as the workshop proceeded.
Description of the Tympanocentesis Session
A mannequin model was developed. The mannequin consisted of a plastic external shell head and shoulders made from a mold with an external pinnae and auditory canal. The caliber, length, and angulation of the external auditory canal were designed to the anatomic specifications simulating a 1- to 2-year-old child. The ear canal material was slightly spongy, flesh-colored, and moderately resistant to needle puncture. A cartridge consisting of 4 simulated TMs was inserted into the head portion of the mannequin. A sliding mechanism produced positioning of the artificial TM at an angulation that was anatomically correct. The TM was engineered to simulate the appearance and feel of a pop when a tympanocentesis needle is used for penetration. Behind the TM, artificial pus was placed in the inferior portion of the simulated middle ear space. This was recommended as the target for tympanocentesis. The anterior inferior quadrant was suggested as the preferred location (out of concern for safety and the provision of increased depth before the posterial wall of the middle ear space would be encountered during the aspiration procedure). The superior half of the simulated middle ear space contained a red dye. This was an area of designated avoidance out of concern for possible injury to the malleus, incus, or stapes. At a depth of approximately 5 mm, a second membrane was present, which prevented access to a second chamber that contained a blue dye. The depth between the TM surface and the second chamber was anatomically correlated with the depth of the middle ear space in a 1- to 2-year-old child. Thus, the pediatric resident was instructed to insert the tympanocentesis needle into the inferior half of the simulated TM with preference to the anterior inferior quadrant and advised to avoid the superior half of the TM completely. Last, the pediatric resident was advised that if the tympanocentesis needle was advanced too far, then a blue dye would indicate that event. Instruction was provided on hand positioning during the procedure. An operating head on a hand-held otoscope was used. For this simulation, a 20-gauge, 3.5-inch-long spinal needle was attached to either a 1- or 3-mL syringe. The needle was bent approximately one third from the hub at 45° to 90° to allow advancement of the needle via the exterior auditory canal to the TM without blockage of visualization through the operating scope magnifying lens. Each TM cartridge contained 4 artificial TM/middle ear effusion discs to allow 4 attempts during the training session. Each pediatric resident was allowed up to 15 minutes to complete the 4 tympanocentesis procedures; they were evaluated on the basis of their success with each of the 4 test discs. There were times when the mannequins did not function optimally or time constraints did not allow 4 attempts; these results were included when at least 2 attempts occurred. The grading tool did not include assessment of change in skill during the tympanocentesis attempts.
Antibiotic Management Session
A Powerpoint presentation emphasized the CDC’s Drug-Resistant Streptococcus pneumoniae Working Group guideline as an appropriate treatment paradigm10 with a broadened acceptance of several additional antibiotics as among the acceptable second-line choices for persistent and recurrent AOM. Pharmacokinetic/pharmacodynamic prescribing principles and results of double tympanocentesis trial results were explained and described, respectively. Compliance features as an integral component of antibiotic selection were discussed.
Pre- and Post-Test Course Testing
At the beginning and end of the program, pediatric residents were asked questions relevant to the most reliable, predictive TM findings for AOM, the indications for tympanocentesis (ear tap), and appropriate antibiotics for treatment of AOM caused by drug-resistant S pneumoniae and β-lactamase–producing H influenzae. At the end of the course, pediatric residents were asked about the effectiveness of the AOM diagnosis interactive video, tympanocentesis mannequin lab, and antibiotic lecture and whether they would change the way they diagnosed AOM, OME, and normal ears and whether they intended to begin performing tympanocentesis.
For comparing the pediatric residents’ performance with that of practicing pediatricians, the comparison of proportions test was used and P ≤ .05 was considered significant.
A total of 383 pediatric residents participated in the workshop; 211 participated at hospital settings where pediatric residents were in training. For comparison, the responses and performance of 2190 practicing pediatricians to be reported elsewhere (M Pichichero, unpublished results) are provided here. The sequence of otoendoscopic examinations, a description of the findings, and the correct diagnoses are shown in Table 1. There were 5 video examinations from patients with OME (ears 1, 2, 5, 6, and 8), 3 from patients with retracted TMs but no effusion (ears 3, 7, and 9), and 1 patient with classic AOM (ear 4). Ear 5 was a classic glue ear (thickened TM, completely immobile with pneumatic otoscopy, with thick amber fluid in the middle ear space). Ear 6 had a hypermobile deep retraction pocket in the superior posterior quadrant.
The average ± standard deviation (SD) correct diagnoses during the otoendoscopic video examinations by pediatric residents was 41% ± 16 (range: 19%–70%; median: 38%). For practicing pediatricians, it was 51% ± 11 (range: 29%–72%; median: 49%; P < .001; difference: 10%; confidence interval [CI]: 4%–15%; Table 2).
In Table 3, the diagnosis made by the pediatric residents (and practicing pediatricians) is shown for all 9 ears. None of the 9 examination ears was completely normal, and pediatric residents recognized the absence of normality accurately 83% to 99% (mean ± SD: 92% ± 5%; median: 93%) of the time compared with 89% to 100% (95 ± 4%; median: 98%) for practicing pediatricians (P = .024; difference: 3%; CI: 0.05%–5%). For the distinction between AOM (antibiotics appropriate) and OME/retracted TM but otherwise normal and normal (antibiotics not necessary), pediatric residents overdiagnosed AOM from 6% to 58% of the time (34 ± 23%; median: 37%) compared with practicing pediatricians (7% to 51%; 26 ± 19%; median: 21%), respectively (P < .001; difference: 8%; CI: 3%–13%). When the correct diagnosis was OME, residents more often diagnosed AOM (46% ± 20%; median: 53%) than pediatricians (35% ± 18%; median: 40%; P < .001). When the correct diagnosis was retracted, otherwise normal TM, residents more often diagnosed AOM (14% ± 9%; median: 13%) than pediatricians (11% ± 3%; median: 12%; P < .001).
Performance on the tympanocentesis procedure for pediatric residents is shown in Table 4. Optimally performed procedures involved needle insertion in the anterior or posterior-inferior quadrant without blue dye; this was accomplished by 89% of the pediatric residents (and 86% of the practicing pediatricians).
Before the course began, 78% of pediatric residents chose a recommended antibiotic after amoxicillin failure for drug-resistant S pneumoniae, and at the end of the course there was an increase to 95%; this 17% (CI: 14%–20%) improvement was significant (P < .001). For practicing pediatricians, the results were 80% and 96%, respectively, and this 16% (CI: 14%–18%) improvement was significant (P < .001). Similarly, before the course, 74% of pediatric residents chose a recommended antibiotic for β-lactamase–producing H influenzae, and at the end of the course there was an increase to 90%; this 16% (CI: 11%–21%) improvement was significant (P < .001); for practicing pediatricians the results were 73% and 93%, respectively, and this 20% (CI: 18%–22%) improvement was significant (P < .001; Table 5).
Pre- and Postcourse Responses
The percentage who gave the correct answer to questions asked regarding the most reliable TM findings for AOM and indications for tympanocentesis in a pre- and postcourse test is shown in Table 6. Residents and pediatricians showed acquisition of knowledge for both questions as evidenced by a higher percentage of correct answers (P < .001) for all comparisons, both groups.
At the completion of the workshop, pediatric residents responded to questions about the effectiveness of the diagnosis video, tympanocentesis mannequin lab, and antibiotic lecture as a learning tool. Their responses are shown in Table 6. The residents were also asked whether they would change their diagnosis approach to AOM, OME, and normal ears because of course participation and whether they would like to begin to perform tympanocentesis because of course participation; their responses are shown in Table 6.
Pediatric residents and practicing pediatricians perform otoscope examinations every day as part of routine and sick child care. Their skills and accuracy are infrequently, if ever, assessed in any formal way during training or in practice. Differentiation of AOM from OME has become more critical in an era of rising antibiotic resistance among AOM bacterial pathogens. For testing pediatric residents’ diagnostic accuracy, otoendoscopic video recorded examinations were presented. Pediatric residents made a correct diagnosis on average 41% of the time compared with practicing pediatricians, who on average were 51% correct. OME and a retracted TM without associated middle ear effusion were frequently misdiagnosed as AOM.
AOM and OME are disease entities in the OM continuum. There is often a transition between OME and AOM, and the 2 conditions at times may be indistinguishable from each other diagnostically.16 It is sometimes difficult to determine visually whether an effusion in the middle ear space is purulent (consistent with AOM) or nonpurulent (consistent with OME). Misdiagnosis also may occur when a patient who is experiencing a viral upper respiratory infection develops nasopharyngeal and eustachian tube congestion, causing the TM to become distorted (retracted) and thereby misleading the practitioner to conclude that AOM or OME is present.
The course participants did not have the advantage of a specific history of symptoms relating to the physical examination findings displayed in the otoendoscopic videos. However, not all patients with otalgia have AOM. Symptoms that predict AOM are neither sensitive nor specific.17,18 Children with a retracted TM but no effusion may feel discomfort, and those with an upper respiratory tract infection may develop a middle ear effusion (OME) that does not progress to AOM, yet they may display ear tugging and sleeplessness.
The otoendoscopic videos gave a 2-dimensional image, and a better depth of field may have improved diagnostic accuracy. Only 9 test ears were shown; none was normal, and only 1 was AOM. However, before each otoendoscopic examination, the external auditory canal was completely cleared of cerumen, appropriately performed pneumatic otoscopy was included as recommended,19,20 the video was shown for 30 seconds, and as much time as necessary was allowed for participants to reach a conclusion. As a referent group, otolaryngologists who were taking the same video examination generally recognized the correct diagnosis approximately 80% of the time.9 Pediatric residents considered the color of the TM as a key diagnostic sign (based on the precourse responses to most reliable diagnostic criteria for AOM), although it has been shown that the color of the TM does not correlate with pathogen isolation from middle ear effusions.5,20 A red TM with normal TM position and mobility has a predictive value of 7% for AOM.5,20 The position of the TM is key. For both AOM and OME, there is fluid in the middle ear space and TM mobility may be diminished with pneumatic otoscopy examination. Patients with AOM have a TM under positive (full or bulging) pressure, and bacterial pathogens are isolated >90% of the time.21 Patients with OME have a TM under negative pressure (retracted) or no pressure (neutral position), and bacterial pathogens are isolated <10% of the time.21 Only by combining an assessment of TM position, color, and mobility can a 99% predictive value of AOM be achieved when compared with findings at tympanocentesis.5,20
As noted by Steinbach et al,21,22 there is a paucity of formalized resident education regarding otitis media diagnosis and management and no training in tympanocentesis. In a recent study, 27 pediatric residents examined 43 of the same patients (and 86 ears) as a pediatric otolaryngologist and an audiologist using tympanometry. Only a slight to moderate correlation was found between the pediatric residents’ diagnosis, the otolaryngologists’ diagnosis, and tympanometry.22 Few other educational programs and evaluations of residents or practitioners in OM diagnosis have been published.23–25
In 1999, a report was issued from the Drug-Resistant Streptococcus pneumoniae Therapeutic Working Group convened by the CDC regarding the management of AOM.10 Diagnostic tympanocentesis with culture and susceptibility testing of isolates was recommended. The CDC guideline10 and the American Academy of Pediatrics Red Book11 state that tympanocentesis may be necessary to guide the choice of therapy in difficult AOM cases. Performing a tympanocentesis was thought to be particularly important for children who have received several courses of antimicrobial therapy and therefore are more likely to harbor a multiply-resistant bacteria. Other prominent authorities have advised tympanocentesis for the child who is highly febrile, toxic, in acute pain, immunosuppressed, or experiencing persistent or recurrent AOM.12–15 With consideration of an increasing recognized need for the performance of tympanocentesis by properly trained/certified pediatricians, a mannequin has been designed to facilitate training. Pediatric residents demonstrated a high level of skill in performing this procedure on the simulation model. This skill should be added to the program requirements mandated by the Residency Review Committee in Pediatrics.
Antibiotic selection for treatment of AOM is complex.12–14 Amoxicillin, in a double dosage at 80 to 100 mg/kg/d, has been recommended in recent guidelines as first-line therapy.10,11 For persistent AOM (clinical failure on therapy or failure within 1 month of completion of a treatment course) and recurrent AOM, antibiotic selection should take into consideration a higher likelihood of drug-resistant S pneumoniae and H influenzae. Guidelines, primarily based on pharmacokinetic and pharmacodynamic antibiotic principles and results of double tympanocentesis trials, have suggested a group of preferred agents and a group of agents with which anticipated efficacy might be lower. When this knowledge was tested during the training workshop, pediatric residents (and pediatricians) chose recommended antibiotics as first and second choices approximately 70% of the time in a pretest and approximately 95% of the time after completion of the course.
A total of 97% of pediatric residents who attended the workshop said that it would change their diagnostic approach to OM, 49% indicated an interest in applying their new skill in tympanocentesis, and approximately 20% showed knowledge acquisition in the area of antibiotic treatment. Ninety percent or more of the pediatric residents found the otoendoscopic video, mannequin lab, and antibiotic lecture to be highly effective learning tools. The active interaction used in the workshops is a more effective teaching format in producing acquisition and retention of knowledge in the education arena compared with standard didactic sessions.26
The long-term outcome was not evaluated among the pediatric residents who attended the workshops. However, after participation in the course, a cohort of pediatricians from 1 community had their medical records monitored for diagnostic codes for AOM, OME, and eustachian tube dysfunction/otalgia compared with a cohort of pediatricians from the same community who did not attend the course.27 The AOM diagnosis code during the 16-month follow-up after CME course participation was used in 70% of OM diagnoses among CME attendees versus 89% among non-CME attendees, whereas the OME diagnosis code was used in 28% and 11%, respectively; this was significant (P = .003).27 A change to more frequent use of guideline-recommended antibiotics also was shown in that follow-up study.
Introduction of a structured, standardized training module in pediatric residency training programs for instructions in the diagnosis and management of OM seems to be needed. An outcomes assessment of such a module would fit well with the call by the Accreditation Council for Graduate Medical Education for competency-based educational outcomes of training.28
Workshop faculty: Gerson Aronovitz, MD; Itzhak Brook, MD; Jeffrey Blumer, PhD, MD; Michael Biavati, MD; Stan Block, MD; Janet Casey, MD; Steve Chartrand, MD (deceased); G. Scott Giebink, MD; Manford Gooch, MD; Christopher J. Harrison, MD; James A. Hedrick, MD; Candice Johnson, MD; Jerome D. Klein; Colin Marchant, MD; Michael Pichichero, MD; Michael Poole, MD; Sam McLinn, MD; Stephen Pelton, MD; Lawrence Sher, MD; and Sylvan Stool, MD.
- Received April 22, 2002.
- Accepted July 2, 2002.
- Reprint requests to (M.E.P.) Elmwood Pediatric Group, University of Rochester Medical Center, 601 Elmwood Ave, Box 672, Rochester, NY 14642. E-mail:
The author is a faculty member of the University of Rochester, the continuing medical education accreditor of the programs described in this paper, and a co-chairman and a principal of Outcomes Management Educational Workshops, the organizer of the programs.
- ↵Steinbach WJ, Sectish TC. Pediatric resident training in the diagnosis and treatment of acute otitis media. Pediatrics.2002;109 :404– 408
- ↵Stool W, Berg A, Berman S, et al. Otitis Media With Effusion in Young Children: Clinical Practice Guideline. Rockville, MD: Agency for Health Care Policy and Research, Public Health Service, US Department of Health and Human Services; 1994
- ↵Paradise JL, Rochette HE, Colborn DK, et al. Otitis media in 2253 Pittsburgh-area infants: prevalence and risk factors during the first two years of life. Pediatrics.1997;99 :318– 332
- ↵Dowell SF, March SM, Phillips WR, et al. Otitis media: principles of judicious use of antimicrobial agents. Pediatrics.1998;101(suppl) :165– 171
- ↵Pickering L, ed. 2000 Red Book Committee Report. Elk Grove Village, IL: American Academy of Pediatrics; 2000
- ↵Block SL. Tympanocentesis: why, when, how. Contemp Pediatr.1999;16 :103– 127
- ↵Pichichero ME, Casey JR. Head and neck infections (otitis, mastoiditis, and sinusitis). In: Baddour L, Gorbach S, eds. Therapy of Infectious Diseases. Philadelphia, PA: Harcourt Health Sciences. 2003. In press
- ↵Cavanaugh RM Jr. Pediatrician and the pneumatic otoscope: are we playing it by ear? Pediatrics.1989;84 :362– 364
- ↵Karma PH, Sipila MM, Kayaja MJ, Penttila MA. Pneumatic otoscopy and otitis media: the value of different tympanic membrane findings and their combinations. In: Lim DJ, Bluestone CD, Klein JO, Nelson JD, Ogra PL, eds. Recent Advances in Otitis Media: Proceedings of the Fifth International Symposium. Burlington, Canada: Decker; 1993:41–45
- ↵Steinbach WJ, Sectish TC, Benjamin DK, Chang KW, Messner AH. Pediatric residents’ clinical diagnostic accuracy of otitis media. Pediatrics.2002;109 :993– 998
- ↵Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA.1999;282 :867– 874
- ↵Pichichero ME. A physician education intervention influenced prescribing for otitis media. J Manage Care Pharm.2002;8 :141– 145
- ↵Accreditation Council for Graduate Medical Education. Outcome Project. Available at: www.acgme.org. Accessed May 2001
- Copyright © 2002 by the American Academy of Pediatrics