How Helpful Is Pneumatic Otoscopy in Improving Diagnostic Accuracy?

METHODS

Participants were prospectively enrolled at 2 pediatric residency training programs, the San Antonio Military Pediatric Center, San Antonio, Texas, and the National Naval Medical Center, Bethesda, Maryland. Neither program had a standardized training program in otoscopy, but relied primarily on patient encounters under supervision along with additional didactic sessions. The study participants attended VOE testing/training sessions (with exception of the 2 pediatric otolaryngologists) as part of Institutional Review Board-approved protocols at both institutions assessing providers’ otoscopic interpretive skills using VOEs. In May 2000, all the participants attending the sessions at both institutions agreed to additionally record their pre- (static test) and post-pneumatic (pneumatic test) assessments of MEE status of the 50 ears shown on the VOE. The test answer sheets used by participants had no identifying information other than level of training.

The VOE test consists of a 5-second still image (static component) of each TM and then a 10-second full-motion view where the TM is exposed to positive and negative pressure (pneumatic component), followed by a final 5-second still image. On the VOE, this sequence is repeated for 50 different TMs of varying degrees of diagnostic difficulty. For each TM, the clinician answered either “yes,” if he or she believed MEE was present, or “no,” if he or she believed MEE was absent after, first, the static component and, then, the pneumatic component of the VOE. The clinicians were not asked to distinguish acute otitis media from otitis media with effusion since clinical information was not provided.

Of the 3 VOEs available, volume 1 was used because it has previously been validated to differentiate Expert from Novices in otoscopy and because it was shown to correlate well with direct otoscopic findings.¹⁴ The Novice group’s mean total score was 68%, whereas the Expert group’s mean total score was 93%. Test answers were determined by an expert panel’s consensus diagnosis of MEE status (tympanometry and myringotomy data were available for most ears).^14,15

To determine accuracy, the static and pneumatic assessments of each participant were compared with the VOE test answers. The maximum score on the test was 100%. Sensitivity and specificity results for both the static and pneumatic tests were also determined. Paired Student t tests were used to assess the differences in the study group’s mean total scores, sensitivity, and specificity on the static test, compared with the pneumatic test. We also determined each participant’s absolute improvement (pneumatic test–static test) as well as the group’s mean absolute improvement in total score, sensitivity, and specificity from the static test to the pneumatic test. To better understand the impact of pneumatic otoscopy over static evaluation alone on an individual clinician’s performance, we calculated each participant’s relative improvement ([pneumatic test–static test]/static test) and the group’s mean relative improvement in total score, sensitivity, and specificity. Pearson correlation analysis was used to assess the association of the absolute and relative improvement with otoscopic accuracy. We used pneumatic total test scores, rather than static total scores or training level, to define higher accuracy, given that all participants’ pneumatic scores were higher than their static scores and because VOE performance has previously been validated to distinguish more skilled otoscopists.

RESULTS

Forty clinicians—34 pediatric residents (13 postgraduate year [PGY] 1, 15 PGY-2, 6 PGY-3) and 6 clinical faculty (4 pediatricians and 2 pediatric otolaryngologists)—participated in the VOE sessions that included recording of both static and pneumatic assessments. The accuracy (percentage of test items correct) of all participants’ pneumatic tests was greater than their static tests. The mean total score, sensitivity, and specificity were significantly higher after pneumatic assessment than after static assessment (P < .001; Fig 1). Table 1 shows the mean absoluteimprovement and the mean relative improvement in participants’ accuracy from the static test to the pneumatic test. Themean absolute increase from static to pneumatic performance was 15% improvement in total score, 12% improvement in sensitivity (accurate detection of ears with MEE), and 18% improvement in specificity (accurate detection of effusion-free ears). The mean relative improvement from static to pneumatic performance was 26% improvement in total score, 24% improvement in sensitivity, and 42% improvement in specificity. Figures 2 and 3 demonstrate with scatter plots for individual participants that increasing absolute improvement and increasing relative improvement, respectively, correlated with higher pneumatic test total scores. A higher degree of accuracy was associated (using Pearson correlations) with greater absolute improvement (r = 0.57) and relative (r = 0.47) improvement.

• Download figure
• Open in new tab
• Download powerpoint

Fig 1.

Performance of the 40 participants on videotaped otoendoscopic examination: static accuracy versus pneumatic accuracy. †The group’s mean pneumatic score is significantly higher than group’s mean static score (P < .001, paired Student t test).

• Download figure
• Open in new tab
• Download powerpoint

Fig 2.

Correlation of the 40 participants’ pneumatic test scores on the videotaped otoendoscopic examination with percent Absolute Improvement. Every participant’s pneumatic test score was higher than his/her static test score. Higher pneumatic test scores were correlated with greater Absolute Improvement (r = 0.57, Pearson correlation). Absolute Improvement = (Pneumatic Test Score − Static Test Score).

• Download figure
• Open in new tab
• Download powerpoint

Fig 3.

Correlation of the 40 individual participants’ pneumatic test scores on the videotaped otoendoscopic examinations with percent Relative Improvement. Every participant’s pneumatic test score was higher than his/her static test score. Higher pneumatic test scores were correlated with greater Relative Improvement (r = 0.47, Pearson correlation). Relative Improvement = (Pneumatic Test Score − Static Test Score)/Static Test Score.

DISCUSSION

The present study, based on otoscopic assessment using a VOE, suggests that pneumatic assessment does indeed improve the ability to detect both MEE-containing and MEE-free ears. Clinicians who are more skilled at visual assessment, as determined by higher total scores on the VOE, demonstrated even greater improvement in accuracy following pneumatic assessment than clinicians with lower accuracy on visual assessment. Presumably, the more skilled otoscopists have learned how to better interpret the additional information regarding MEE status provided by the pneumatic component of the examination. Because MEE is usually found concurrently in acute otitis media, this study suggests that clinicians who perform pneumatic otoscopy might be able to more accurately identify MEE-containing ears in acute otitis media and otitis media with effusion as well as MEE-free ears. Future research will be necessary to confirm such an improvement in diagnostic accuracy in the clinical setting attributable to performance of pneumatic otoscopy. Our findings have potentially important implications regarding appropriate antibiotic utilization and other therapeutic decisions. Furthermore, improving diagnostic accuracy may be even more significant in light of recent evidence suggesting that many pediatricians and pediatric residents may have suboptimal otoscopic interpretive skills.^16,17

The limited evidence available suggests that many providers of pediatric care do not routinely perform pneumatic otoscopy. Nelson¹¹ reported that only 42% of pediatricians surveyed always performed pneumatic otoscopy on patients with a history of otitis media. Of the 67 pediatricians surveyed in the late 1980s, 21% reported never performing pneumatic otoscopy “even if the patient had a history of any form of OM.” In a more recent survey of physicians concerning management of upper respiratory infections, 25% of pediatricians and 46% of family physicians reported “never or rarely” performing pneumatic otoscopy.¹² Furthermore, 33% of the family practitioners in that survey, who acknowledged it was essential to perform pneumatic otoscopy, self-reported that they did not routinely do it. In contrast, Steinbach et al¹⁶ recently reported that use of pneumatic otoscopy among residents was 79%.

One of the limitations of our study is the design of the VOE in which the static presentations were sequentially followed by the pneumatic presentations of the same ear. Static accuracy may have been influenced by the pneumatic assessment, or vice versa. In addition, given the superior view provided of the TM on the VOE, this test may overestimate true otoscopic skills. However, because a superior view of the TM should have equally improved the static as well as the pneumatic accuracy, we do not feel that the VOE overestimates the benefits of pneumatic otoscopy. If the VOE is associated with a higher static accuracy than that expected in the clinical setting, then the potential Absolute Improvement and, even more so, the potential Relative Improvement demonstrable attributable to assessment of TM mobility are reduced. When the TM can only be partially visualized through a moving child’s cerumen-filled external auditory canal, a clinician in the office who can discern degrees of mobility has a distinct advantage over the provider who simply has a partial, static view of the TM. Most participants in this study were pediatric residents rather than more experienced practitioners. However, because the VOE has been validated to differentiate the more experienced, skilled otoscopists and because those with higher test scores (more skilled otoscopists) had greater improvement from the static to the pneumatic presentations, it may be reasonable to assume that the overall trend of increased accuracy associated with pneumatic otoscopy would be found among medical providers with greater clinical experience.

To our knowledge, this study is the first to quantitatively validate the benefits of pneumatic otoscopy in improving clinicians’ otoscopic accuracy. Those clinicians who were more skilled otoscopists benefited more from the pneumatic assessment than those who were less skilled. Given current concerns regarding injudicious antibiotic use, along with growing evidence suggesting that some providers’ have suboptimal otoscopy skills, educators and expert clinicians must consider renewing efforts to promote the benefits of performing pneumatic otoscopy as the standard of care and to provide an educational milieu to enhance this fundamental skill.