How Helpful Is Pneumatic Otoscopy in Improving Diagnostic Accuracy?
* Uniformed Services University, Bethesda, Maryland
San Antonio Military Pediatric Center, Lackland Air Force Base, Texas
University of Pittsburgh School of Medicine and Children’s Hospital of Pittsburgh, Pittsburgh, Pennsylvania
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Background. Pneumatic otoscopy is believed to be helpful in optimally assessing the presence or absence of middle ear effusion (MEE). Although expert clinicians teach the importance of this diagnostic skill to trainees, evidence exists that many pediatric providers do not typically perform pneumatic otoscopy.
Objective. To determine if the otoscopic accuracy within a group of clinicians improves with the pneumatic assessment when compared with the static assessment using videotaped otoendoscopic examinations (VOEs).
Methods. Residents and faculty from 2 pediatric training programs served as subjects. All viewed a set of 50 video otoscopic examinations of tympanic membranes (TMs) from a validated VOE developed previously for training purposes. The video displays each TM in a static presentation and then in a pneumatic (mobile) presentation, followed by a final static presentation. Each subject first viewed the initial static presentation of each TM and responded "yes/no" to the presence of MEE, and then viewed the pneumatic presentation of the same TM and again responded "yes/no" to the presence of MEE. We compared the accuracy of assessment for both the static and the pneumatic tests.
Results. Thirty-four pediatric residents and 6 clinical faculty participated. Accuracy (percent of total test items correct) on the pneumatic test was uniformly greater than accuracy on the static test. The mean absolute improvement in the accuracy from the static test (61%) to the pneumatic test (76%) was 15% (95% confidence interval [CI] = 12%–18%). The mean relative improvement in accuracy from the static test to the pneumatic test was 26% (95% CI = 19%–32%). Higher accuracy on the VOE was associated with greater absolute (r = 0.57) and greater relative (r = 0.47) improvement. The mean relative improvement in sensitivity and specificity from static viewing to pneumatic viewing was 24% (95% CI = 15%–33%) and 42% (95% CI = 27%–58%), respectively.
Conclusions. Using a video otoendoscopic test, we found that accurate identification of both the presence and the absence of MEE improved after pneumatic assessment of TM mobility. Providers who were more accurate at otoscopy, defined by higher video total test scores, benefited more from the pneumatic component than providers with lower scores.
Key Words: pneumatic otoscopy • otitis media • education • clinical competence
Abbreviations: MEE, middle ear effusion • TM, tympanic membrane • VOE, videotaped otoendoscopic examination • PGY, postgraduate year
Pneumatic otoscopy is believed to be helpful in optimally assessing the presence or absence of middle ear effusion (MEE).1–6 We use the term MEE to include MEEs found in both acute otitis media and otitis media with effusion. Pneumatic otoscopy describes the delivery of both positive and negative pressure through the otoscope using a pneumatic bulb to elicit information regarding the condition of both the tympanic membrane (TM) and the middle ear space. The normal TM with ambient middle ear pressures will show visually detectable mobility during otoscopy with minimally applied pressures after an airtight seal is obtained between the speculum and external auditory canal. The degree of alteration in TM mobility depends on the amount of MEE, degree of negative or positive pressure alteration in the middle ear space from an ambient state, and/or the changes present on the TM (eg, thickening, tympanosclerosis, atrophic areas). Therefore, it is valuable for the clinician to know to what degree the TM moves relative to the normal state, rather than noting merely the presence or absence of movement. Except in the presence of air-fluid interfaces or bubbles, the degree of mobility of the TM is one of the most important otoscopic findings in determining whether MEE is present.6,7
Expert otoscopists and academicians stress the importance of assessing mobility along with the color, position, and translucency of the TM to both practitioners and trainees.3,4,6,8–10 However, there is evidence that many providers who take care of children do not typically perform pneumatic otoscopy, even in children with a history of middle ear disease.11,12 Although such assessment of TM mobility is considered optimal for accurate diagnoses, we are not aware of any report that validates the benefits of pneumatic otoscopy compared with otoscopic assessment alone. Therefore, the objective of this study was to assess whether the pneumatic otoscopic component of a standardized tool, the videotaped otoendoscopic examination (VOE), improved accuracy of diagnosing MEE.13
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Participants were prospectively enrolled at 2 pediatric residency training programs, the San Antonio Military Pediatric Center, San Antonio, Texas, and the National Naval Medical Center, Bethesda, Maryland. Neither program had a standardized training program in otoscopy, but relied primarily on patient encounters under supervision along with additional didactic sessions. The study participants attended VOE testing/training sessions (with exception of the 2 pediatric otolaryngologists) as part of Institutional Review Board-approved protocols at both institutions assessing providers’ otoscopic interpretive skills using VOEs. In May 2000, all the participants attending the sessions at both institutions agreed to additionally record their pre- (static test) and post-pneumatic (pneumatic test) assessments of MEE status of the 50 ears shown on the VOE. The test answer sheets used by participants had no identifying information other than level of training.
The VOE test consists of a 5-second still image (static component) of each TM and then a 10-second full-motion view where the TM is exposed to positive and negative pressure (pneumatic component), followed by a final 5-second still image. On the VOE, this sequence is repeated for 50 different TMs of varying degrees of diagnostic difficulty. For each TM, the clinician answered either "yes," if he or she believed MEE was present, or "no," if he or she believed MEE was absent after, first, the static component and, then, the pneumatic component of the VOE. The clinicians were not asked to distinguish acute otitis media from otitis media with effusion since clinical information was not provided.
Of the 3 VOEs available, volume 1 was used because it has previously been validated to differentiate Expert from Novices in otoscopy and because it was shown to correlate well with direct otoscopic findings.14 The Novice group’s mean total score was 68%, whereas the Expert group’s mean total score was 93%. Test answers were determined by an expert panel’s consensus diagnosis of MEE status (tympanometry and myringotomy data were available for most ears).14,15
To determine accuracy, the static and pneumatic assessments of each participant were compared with the VOE test answers. The maximum score on the test was 100%. Sensitivity and specificity results for both the static and pneumatic tests were also determined. Paired Student t tests were used to assess the differences in the study group’s mean total scores, sensitivity, and specificity on the static test, compared with the pneumatic test. We also determined each participant’s absolute improvement (pneumatic test–static test) as well as the group’s mean absolute improvement in total score, sensitivity, and specificity from the static test to the pneumatic test. To better understand the impact of pneumatic otoscopy over static evaluation alone on an individual clinician’s performance, we calculated each participant’s relative improvement ([pneumatic test–static test]/static test) and the group’s mean relative improvement in total score, sensitivity, and specificity. Pearson correlation analysis was used to assess the association of the absolute and relative improvement with otoscopic accuracy. We used pneumatic total test scores, rather than static total scores or training level, to define higher accuracy, given that all participants’ pneumatic scores were higher than their static scores and because VOE performance has previously been validated to distinguish more skilled otoscopists.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Forty clinicians—34 pediatric residents (13 postgraduate year [PGY] 1, 15 PGY-2, 6 PGY-3) and 6 clinical faculty (4 pediatricians and 2 pediatric otolaryngologists)—participated in the VOE sessions that included recording of both static and pneumatic assessments. The accuracy (percentage of test items correct) of all participants’ pneumatic tests was greater than their static tests. The mean total score, sensitivity, and specificity were significantly higher after pneumatic assessment than after static assessment (P < .001; Fig 1). Table 1 shows the mean absoluteimprovement and the mean relative improvement in participants’ accuracy from the static test to the pneumatic test. Themean absolute increase from static to pneumatic performance was 15% improvement in total score, 12% improvement in sensitivity (accurate detection of ears with MEE), and 18% improvement in specificity (accurate detection of effusion-free ears). The mean relative improvement from static to pneumatic performance was 26% improvement in total score, 24% improvement in sensitivity, and 42% improvement in specificity. Figures 2 and 3 demonstrate with scatter plots for individual participants that increasing absolute improvement and increasing relative improvement, respectively, correlated with higher pneumatic test total scores. A higher degree of accuracy was associated (using Pearson correlations) with greater absolute improvement (r = 0.57) and relative (r = 0.47) improvement.
|
The group’s mean pneumatic score is significantly higher than group’s mean static score (P < .001, paired Student t test).
|
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
The present study, based on otoscopic assessment using a VOE, suggests that pneumatic assessment does indeed improve the ability to detect both MEE-containing and MEE-free ears. Clinicians who are more skilled at visual assessment, as determined by higher total scores on the VOE, demonstrated even greater improvement in accuracy following pneumatic assessment than clinicians with lower accuracy on visual assessment. Presumably, the more skilled otoscopists have learned how to better interpret the additional information regarding MEE status provided by the pneumatic component of the examination. Because MEE is usually found concurrently in acute otitis media, this study suggests that clinicians who perform pneumatic otoscopy might be able to more accurately identify MEE-containing ears in acute otitis media and otitis media with effusion as well as MEE-free ears. Future research will be necessary to confirm such an improvement in diagnostic accuracy in the clinical setting attributable to performance of pneumatic otoscopy. Our findings have potentially important implications regarding appropriate antibiotic utilization and other therapeutic decisions. Furthermore, improving diagnostic accuracy may be even more significant in light of recent evidence suggesting that many pediatricians and pediatric residents may have suboptimal otoscopic interpretive skills.16,17
The limited evidence available suggests that many providers of pediatric care do not routinely perform pneumatic otoscopy. Nelson11 reported that only 42% of pediatricians surveyed always performed pneumatic otoscopy on patients with a history of otitis media. Of the 67 pediatricians surveyed in the late 1980s, 21% reported never performing pneumatic otoscopy "even if the patient had a history of any form of OM." In a more recent survey of physicians concerning management of upper respiratory infections, 25% of pediatricians and 46% of family physicians reported "never or rarely" performing pneumatic otoscopy.12 Furthermore, 33% of the family practitioners in that survey, who acknowledged it was essential to perform pneumatic otoscopy, self-reported that they did not routinely do it. In contrast, Steinbach et al16 recently reported that use of pneumatic otoscopy among residents was 79%.
One of the limitations of our study is the design of the VOE in which the static presentations were sequentially followed by the pneumatic presentations of the same ear. Static accuracy may have been influenced by the pneumatic assessment, or vice versa. In addition, given the superior view provided of the TM on the VOE, this test may overestimate true otoscopic skills. However, because a superior view of the TM should have equally improved the static as well as the pneumatic accuracy, we do not feel that the VOE overestimates the benefits of pneumatic otoscopy. If the VOE is associated with a higher static accuracy than that expected in the clinical setting, then the potential Absolute Improvement and, even more so, the potential Relative Improvement demonstrable attributable to assessment of TM mobility are reduced. When the TM can only be partially visualized through a moving child’s cerumen-filled external auditory canal, a clinician in the office who can discern degrees of mobility has a distinct advantage over the provider who simply has a partial, static view of the TM. Most participants in this study were pediatric residents rather than more experienced practitioners. However, because the VOE has been validated to differentiate the more experienced, skilled otoscopists and because those with higher test scores (more skilled otoscopists) had greater improvement from the static to the pneumatic presentations, it may be reasonable to assume that the overall trend of increased accuracy associated with pneumatic otoscopy would be found among medical providers with greater clinical experience.
To our knowledge, this study is the first to quantitatively validate the benefits of pneumatic otoscopy in improving clinicians’ otoscopic accuracy. Those clinicians who were more skilled otoscopists benefited more from the pneumatic assessment than those who were less skilled. Given current concerns regarding injudicious antibiotic use, along with growing evidence suggesting that some providers’ have suboptimal otoscopy skills, educators and expert clinicians must consider renewing efforts to promote the benefits of performing pneumatic otoscopy as the standard of care and to provide an educational milieu to enhance this fundamental skill.
|
ACKNOWLEDGMENTS |
|---|
We thank Joseph O. Lopreiato, MD, National Naval Medical Center, for assistance with VOE testing and manuscript review; Jerome O. Klein, MD, Boston University School of Medicine, for suggesting this study; Cara H. Olsen, Biostatistics Consulting Center, Uniformed Services University, for help with the statistical analysis; and Alejandro Hoberman, MD, for manuscript review.
|
FOOTNOTES |
|---|
Received for publication Dec 18, 2002; Accepted Mar 3, 2003.
Reprint requests to (W.S.J.) Department of Pediatrics, Uniformed Services University of the Health Sciences, 4301 Jones Bridge Rd, Bethesda, MD 20814-4799. E-mail: wjones{at}usuhs.mil
The opinions or assertions contained herein are the private ones of the authors and are not to be construed as official or reflecting the views of the Department of Defense, United States Air Force, or the Uniformed Services University. One of the authors (P.H.K.) is co-owner of the copyright of the Videotaped Otoendoscopic Examination Series, which was funded in part by an educational grant from SmithKline Beecham.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
- Bluestone CD, Klein JO. Otitis Media in Infants and Children. 3rd ed. Philadelphia, PA: WB Saunders; 2000:124–134
- Finitzo T, Friel-Patti S, Chinn K, Brown O. Tympanometry and otoscopy prior to myringotomy: issues in the diagnosis of otitis media. Int J Pediatr Otolarygnol.1992; 24 :101 –110[CrossRef][Web of Science][Medline]
- Pelton SI. Otoscopy for the diagnosis of otitis media. Pediatr Infect Dis J.1998; 17 :540 –543[CrossRef][Web of Science][Medline]
- Dowell SF, Marcy SM, Phillips WR, et al. Otitis media—principles of judicious use of antimicrobial agents.
Pediatrics.1998; 101(suppl)
:165
–171
[Abstract/Free Full Text] - Mains BT, Toner JG. Pneumatic otoscopy: a study of inter-observer variability. J Laryngol Otol.1989; 103 :1134 –1135[Web of Science][Medline]
- Kaleida PH. The COMPLETES exam for otitis. Contemp Pediatr.1997; 14 :93 –101
- Karma PH, Penttila MA, Sipila MM, Kataja MJ. Otoscopic diagnosis of middle ear effusion in acute and non-acute otitis media. I. The value of different otoscopic findings. Int J Pediatr Otolaryngol.1989; 17 :37 –49[CrossRef][Web of Science][Medline]
- Kaleida PH, Stool SE. Assessment of otoscopists’ accuracy regarding middle-ear effusion.
Am J Dis Child.1992; 146
:433
–435
[Abstract/Free Full Text] - Silva AB, Hotaling AJ. A protocol for otolaryngology—head and neck resident training in pneumatic otoscopy. Int J Pediatr Otolaryngol.1997; 40 :125 –131[CrossRef][Web of Science][Medline]
- Eavey RD, Santos JI, Arriaga MA, et al. An educational model for otitis media care field-tested in Latin America. Otolarnygol Head Neck Surg.1993; 109 :895 –898[Web of Science][Medline]
- Nelson JD. Management of chronic suppurative otitis media: a survey of practicing pediatricians. Ann Otol Rhinol Laryngol.1988; 97(suppl 131) :26 –28
- Watson RL, Dowell SF, Jayaraman M, Keyserling H, Kolczak M, Schwartz B. Antimicrobial use for pediatric upper respiratory infections: reported practice, actual practice, and parent beliefs.
Pediatrics.1999; 104
:1251
–1257
[Abstract/Free Full Text] - Kaleida PH, Hoberman A, Smith CG. Videotaped otoendoscopic examinations in clinical education and research [abstract]. Pediatr Res.1992; 31 :123A
- Kaleida PH, Hoberman A, Smith CG. Videotaped otoendoscopic examinations in clinical education and research: 1995 Update [abstract]. Arch Pediatr Adolesc Med.1995; 149(suppl) :117
- Kaleida PH, Hoberman A, Smith CG. Assessment and enhancement of otoscopic diagnostic accuracy using videotaped otoendoscopic examinations. Abstracts of the Sixth International Meeting on Recent Advances in Otitis Media, June 1995
- Steinbach WJ, Sectish TC, Benjamin DK, Chang KW, Messner AH. Pediatric residents’ clinical diagnostic accuracy of otitis media.
Pediatrics.2002; 109
:993
–997
[Abstract/Free Full Text] - Pichichero ME, Poole MD. Assessing diagnostic accuracy and tympanocentesis skills in the management of otitis media.
Arch Pediatr Adolesc Med.2001; 155
:1137
–1142
[Abstract/Free Full Text]
PEDIATRICS (ISSN 1098-4275). ©2003 by the American Academy of Pediatrics
CiteULike 
Connotea 
Del.icio.us 
Digg 
Facebook 
Reddit 
Technorati 
Twitter What's this?
This article has been cited by other articles:
|
|
 |
|
  D. A. Varrasso Otitis Media: The Need for a New Paradigm in Medical Education Pediatrics, October 1, 2006; 118(4): 1731 - 1733. [Full Text] [PDF]
|
|
|
|
 |
|
 Acute Otitis Media: More on Diagnosis and Treatment Journal Watch Pediatrics and Adolescent Medicine, December 15, 2003; 2003(1215): 2 - 2. [Full Text]
|
|
|
|
 |
|
 Otitis Media: More on Diagnosis and Treatment Journal Watch Dermatology, October 28, 2003; 2003(1028): 7 - 7. [Full Text]
|
|
|
|
 |
|
 Otitis Media: More on Diagnosis and Treatment Journal Watch (General), September 12, 2003; 2003(912): 5 - 5. [Full Text]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
