The article on pacifier use and acute otitis media (AOM) by Niemelä et al1 underscores an interesting and important point given the incidence of otitis media and its impact on families, economics, and antibiotic resistance patterns. Nevertheless, the results of the study may have been biased in several ways.
First, parents provided informed consent in this unblinded study. Knowledge that the relationship between pacifier use and the incidence of AOM was being studied and the general belief that pacifiers are not recommended by physicians might have influenced families to underreport pacifier use.
In addition, the clinic nurses may have introduced error. Nurses from intervention clinics were aware of the study goals and underwent a training session on “the harmful effects of both AOM and prolonged pacifier use” and how to best counsel parents to limit pacifier use. Nurses from control clinics, however, received training about only “the possible harmful effects of AOM” and were informed that local epidemiology patterns were being studied. Nurses in this group, therefore, were not “true controls.” They may have had a heightened awareness of otitis with a focus on recording every occurrence and thus influenced parents when instructing them to fill out symptom diaries.
Most importantly, the diagnosis of AOM was made predominantly by parents. Previous studies have shown that symptomatology does not always correlate with AOM, especially in this age group. Only 67.7% of children younger than 2 years old have any ear-related symptoms,2 and, among children <1 year old, as many as 46% are completely asymptomatic.3 Even among children <2 years old who are symptomatic, the positive predictive value of parental suspicion is only 49%, although the negative predictive value is 9%.4 In addition to the inherent inaccuracy of parental diagnosis, parents may have been influenced by their knowledge of the study goals.
This article correctly stresses the importance of and need for preventive measures, given the increasing frequency of otitis media. Many parents and child advocacy groups currently seek such remedies through the Internet, where information is often truncated or reinterpreted. Thus, we must be especially cautious to acknowledge possible biases in results and interpretations, particularly when suggesting interventions in this era of the World Wide Web.
To the Editor—
Niemelä et al conducted a clinic-based, group-randomized trial to test whether office counseling about pacifiers was effective at reducing pacifier use and the rate of acute otitis media (AOM).1 The authors found the incidence of AOM to be 29% lower among children at the intervention clinics. As the medical costs of treating AOM are estimated at over $5 billion annually in the United States,2 even a small reduction in AOM episodes has the potential for tremendous cost savings, lower antibiotic use, and less antibiotic resistance.
We applaud the goals of their study, but wish to raise several questions about the design and analysis. The authors chose to randomize their study by clinic rather than by patient, a feature that must be accounted for in data analysis to obtain valid P values and confidence intervals. Thus, the observations on each individual child are not truly “independent,” as some degree of correlation or “clustering” is expected among children in the same clinic. When the clustered nature of data is ignored, the statistical significance of the results is overstated, often markedly so.3,4 It would be important to know whether the effect of the intervention would remain statistically significant if cluster analysis was applied.
Second, the observed effect seems larger than one might reasonably predict. If pacifier use increases the risk of AOM by a relative risk of 1.4, as reported in an earlier study,5 it is unlikely that lowering continuous pacifier use by 21% could lead to a 29% reduction in AOM rates.
Third, it does not appear that patients were analyzed according to the “intention to treat” principle, unraveling the benefits of initial clinic randomization. If patients were allowed to “drop out” of the study for poor compliance, compliance and dropout rates may not by the same in the control and intervention groups, thus biasing the outcome.
Last, the authors measured AOM using a parent-reported symptoms sheet. It is possible that parents in the intervention group may have felt sheepish about reporting otitis when they knew they were “supposed to” limit pacifier use. Selective underreporting of AOM in the experimental group may have exaggerated the benefit of the intervention. It might have been possible to avoid this bias by ascertaining AOM from medical record review, particularly if the physician diagnosing AOM were blinded to the study.
We appreciate that Niemelä et al have tackled an important subject in the clinical setting, and hope they will revisit their data to examine whether limiting continuous pacifier use really can reduce otitis media.
- ↵Niemelä M, Pihakari O, Pokka T, Uhari M, Uhari M. Pacifier as a risk factor for acute otitis media: a randomized, controlled trial of parental counseling. Pediatrics.2000;106(3) :483– 488
- ↵Bondy J, Berman S, Glazner J, Lezotte D. Direct expenditures related to otitis media diagnoses: extrapolations from a pediatric medicaid cohort. Pediatric.000;105(6). Available at:s http://www.pediatrics.org/cgi/content/full/105/6/e72
To the Editor—
We read with interest the study by Niemelä et al.1 The authors are to be commended for mounting a randomized trial of an intervention to reduce the burden of this common pediatric complaint. However, we have a number of methodologic concerns about the study which, we believe, seriously jeopardize the conclusions.
First, this was a cluster- or group-randomized trial and should have been powered and analyzed as such. The unit of allocation was the clinic. Because subjects within clinics are more alike in their outcomes than are subjects between clinics, observations of clustered data should account for this correlation. Depending on the degree of intracluster correlation, the effective sample size might be much closer to 14 (the number of clinics) than to 484 (the number of individuals for whom follow-up data were available).2
To their credit, the authors paired clinics before randomization, a maneuver that might reduce heterogenity between clinics in the analysis.3 In addition, they collected baseline data about the subjects that could have been used to adjust for differences between the study groups that account for some of the correlation in outcomes.4 Nevertheless, failure to address the clustered design of the trial during analysis will typically underestimate the confidence interval of the result. This suggests that the marginally significant intervention effect (P = .047) would be nonsignificant in a properly conducted analysis.
In addition, we were concerned that the assessment of outcome was by parent diary. It was unclear to us if parents, who presumably knew the purpose of the study and the premise of the intervention, were asked to report on both pacifier use and incidence of acute otitis media (AOM). Do the reported results include parent-reported episodes of AOM or only those diagnosed by a physician?
Turning to analytic strategies, we were surprised to see that episodes of otitis per unit time at risk (a rate) was modeled using linear regression. This is a discrete outcome (a count) and should be analyzed as such using a suitable technique, eg, Poisson or negative binomial regression.5 Linear regression makes assumptions about the distribution of the outcome (that it is normally distributed and could include negative numbers, for example) that are untenable in this case.
Finally, the authors do not address the lack of congruence between the theoretical basis of their intervention and the results they report. The intervention was designed to reduce pacifier use in children older than 6 months which would, it was hypothesized, lead to a reduction in the incidence of AOM. The authors do report a decrease in pacifier use among 6- to 10-month-old children, and inspection of their Fig 2 suggests a corresponding decrease in the cumulative incidence of AOM. The intervention did not, however, result in a change in pacifier use among children older than 10 months, but a decline in the incidence of AOM is nevertheless noted in this age group as well. This finding suggests that any true decrement in the diagnosis of AOM among children in the intervention clinics was not mediated by changes in patterns of pacifier use.
To the Editor—
Niemelä et al1 have suggested that pacifier use is a preventable risk factor for acute otitis media (AOM) in children. This may be so; however, we believe that there are a number of unresolved problems with their methodology.
In particular, we note that AOM “diagnosis” was made by parents and that there is no mention of the number of episodes verified by otolaryngologists or MDs.
In addition, negative comments on the use of pacifiers have been perpetrated by the interviewing nurses—not one positive attribute of pacifier use, such as calming and satisfying sucking needs has been suggested to the parents. Despite this, intervention apparently did not reduce the incidence of pacifier use, merely shifted it to sleep time.
However, we do not find it surprising that when a mother is confronted with these negative pressures and told among other things that a) a pacifier “increases the incidence of AOM” and b) “pacifiers should be limited to the moments when the child is falling asleep after the age of 6 months,” she will subsequently report to the same health personnel that she took the advice and only used the pacifier at sleep time and that the incidence of AOM has reduced.
We have searched the literature and are unable to find any other studies that have suggested that it is possible to change an infant’s pacifier sucking habits from continuous to just sleep time—certainly not on the scale reported in the Niemelä study.
We sincerely thank Drs Wellington and Hall; Ebel, Feinglass, and Koepsell; Johnston and Grossman; and Weiss and Nowak for their interest on our article.1 Although we found the criticism relevant, there were some misunderstandings resulting from our unclear writing. The diagnoses of acute otitis media (AOM) were not made by the parents. Neither were they based on the symptoms of the children. Their own physicians made all the AOM diagnoses using pneumatic otoscopy. The physicians were asked by the parents to write the diagnoses down on the follow-up diaries, which were then returned to us. The diagnoses were done and reported in the same way in both intervention and control groups.
The problem concerning the validity of the data about the sucking habits is complicated. In our open cohort study, the parents may have felt guilty for not being able to avoid the use of a pacifier, which may have affected the reporting of pacifier use. In the control group we suspect that even if some of the parents may have been aware of the harmful effects of pacifier use, the reporting bias was not remarkable. This would cause an exposure misclassification bias in the study groups. This is a possibility that cannot be excluded from our results.
We did not count poor compliance as dropouts but analyzed the patient according to “intention to treat” principle.
Because the age of the child (in months) affects both the susceptibility to AOM, to the use of a pacifier, and to the other risk factors of AOM, the analyses are complicated, and the cluster analysis according to our cluster randomization could introduce extra biases in our results. Furthermore, as the children contributed different lengths of time to the trial and changed their sucking habits during the follow-up, to avoid imbalances in randomization we decided to analyze the data as should be done in open cohort trials by calculating the events per person months at risk.2 Our conclusions are supported with the results in linear regression model, where the counseling had significant effect on the occurrence of AOM/person month at risk when the age of the child was included as a covariate in the model.
- Copyright © 2002 by the American Academy of Pediatrics