BACKGROUND: American Academy of Pediatrics guidelines for acute otitis media (AOM) allow for children meeting certain criteria to undergo watchful waiting (WW). The cost-effectiveness of this policy has not been evaluated in the United States.
METHODS: A retrospective review of a random selection of 250 patients ≤18 years old with AOM in the emergency department of a tertiary care children’s hospital was used to characterize current practice of AOM management. These data were incorporated into a decision-analytic cost-utility model comparing the cost-effectiveness of implementing WW to current practice. The primary outcome was the incremental cost-effectiveness ratio (ICER) expressed in 2015 USD per disability-adjusted life year (DALY) averted from a societal perspective. Multiple sensitivity analyses were conducted.
RESULTS: From this cohort, chart review confirmed 247 actually had AOM on physical examination. Of these, 231 (93.5%) were prescribed antibiotics, 7 (2.8%) underwent WW, and 9 (3.6%) were sent home without an antibiotic prescription. When American Academy of Pediatrics criteria for WW were applied to this population, 104 patients (42.1%) met conditions for immediate antibiotic prescription, and 143 patients (57.9%) qualified for WW. In our modeled scenario, for every 1000 patients with AOM, implementing WW yielded 514 fewer immediate antibiotic prescriptions and 205 fewer antibiotic prescriptions used, averting 14.3 DALYs, and saving $5573. The preferability of WW over current practice proved highly robust to sensitivity analysis.
CONCLUSIONS: WW for AOM management is cost-effective. Implementing WW may improve outcomes and reduce health care expenditures.
- AAP —
- American Academy of Pediatrics
- AOM —
- acute otitis media
- DALY —
- disability-adjusted life year
- ED —
- emergency department
- WW —
- watchful waiting
What’s Known on This Subject:
Acute otitis media is a leading diagnosis for antibiotic prescription. Recent American Academy of Pediatrics guidelines allow for a watchful waiting approach for children based on age and symptom severity, but the cost-effectiveness of these recommendations is unknown.
What This Study Adds:
This study examines the cost-effectiveness of American Academy of Pediatrics guidelines for watchful waiting compared with current practice for acute otitis media management in an urban emergency department. Implementing watchful waiting results in improved health outcomes and cost savings.
Acute otitis media (AOM) affects ∼50% to 85% of children before age 3 and remains one of the leading diagnoses for antibiotic prescription.1 Recent data suggest that physicians in the United States prescribe antibiotics for AOM 95% of the time despite 2004 American Academy of Pediatrics (AAP) guidelines allowing for a watchful waiting (WW) option, in which there is an observation period of 48 to 72 hours before antibiotic initiation to permit spontaneous resolution of symptoms.1–4 These recommendations were further revised in the 2013 AAP guidelines, which stratify treatment with antibiotics based on age and symptom severity for uncomplicated cases of AOM in healthy children. Children of any age with otorrhea or severe symptoms, defined as moderate or severe otalgia, otalgia ≥48 hours, or temperature ≥39°C, should receive immediate antibiotics. Children without severe symptoms aged 6 to 23 months should receive immediate antibiotics if they have bilateral AOM, whereas children ≥2 years old may still undergo WW regardless of laterality. In children whose family requests immediate antibiotic therapy despite qualifying for the WW approach or in those in whom follow-up is not assured, immediate initiation of antibiotic therapy after diagnosis is recommended for best clinical outcomes.
These recommendations by the AAP are based on a Cochrane Review compiling data from 13 randomized controlled trials that demonstrated antibiotics, when compared with placebo, initially reduced the number of abnormal tympanometry findings, resulted in earlier resolution of pain symptoms, decreased tympanic membrane perforations, and halved contralateral otitis episodes.1 Yet in the long-term, antibiotics did not alter the rate of abnormal tympanometry or the occurrence of serious complications.1 In trials comparing immediate antibiotic use with a WW approach, there was no difference in pain resolution, tympanometry findings at 1 month, tympanic membrane perforations, AOM recurrence, or rate of serious complications. Of note, rates of serious complications, such as mastoiditis and meningitis, are low in AOM, which makes differences between antibiotic and WW groups difficult to compare.1,5–8 However, treatment with antibiotics also significantly increases the rate of adverse events, such as vomiting, diarrhea, and rash.1
The prevalence of AOM renders it a significant health care utilization burden, resulting in an increase of $314 per child of outpatient health care costs, 2 more office visits, 0.2 more emergency department (ED) visits, and 1.6 more prescription refills per year when compared with children without AOM.9 Although the AAP guidelines for AOM offer a WW option, a survey of primary care doctors before and after the 2004 AAP guidelines were published indicated little change in clinical practice.10 Current practice thus appears to favor immediate antibiotic prescription for AOM. Although reducing the rate of antibiotic prescriptions may save on prescription costs and antibiotic side effects, this must be balanced against costs to the parents, such as time lost from work, cost of additional analgesia, and repeat office visits or phone calls. With these added costs, it is unclear whether the WW approach would be cost-effective to society.
In this study, the current practice of antibiotic prescription for AOM in a pediatric ED was characterized. The term “current practice” is used throughout to describe the management of AOM in this pediatric ED. Data from this population were used to create a decision-analytic model to evaluate the cost-effectiveness of current practice in the management of AOM when compared with the WW option described in the AAP guidelines. Decision-analytic models, used frequently in cost-effectiveness research, assess alternative management strategies under conditions of uncertainty.
In this study, 250 patients aged ≤18 years who were diagnosed with AOM in the ED of a tertiary care, freestanding, urban children’s hospital were randomly selected based on International Classification of Diseases, Ninth Revision, codes from April 2014 to January 2015. The institutional review board approved this study. Data were abstracted from the electronic medical record using a standard collection form. Basic demographic data such as patient age and clinical information on signs and symptoms were collected, including duration of pain, temperature at time of presentation, physical examination of the tympanic membranes, and pain score (FACES and Face, Legs, Activity, Cry, Consolability scales).11,12 The assumption was made that all patients with documented physical examination findings suggestive of AOM had AOM. Stringent diagnostic criteria, which would be ideal in all AOM studies, were impossible to verify in this retrospective review. Data were also recorded on treatment plan, specifically, the content of the discharge instructions, including any discussion or instructions on WW, education on diagnosis and management, and any antibiotic prescriptions provided.
The same single reviewer who abstracted the data from the electronic medical record used the AAP guidelines to identify patients who qualified for WW based on age, pain severity and duration, temperature, and physical examination. An independent reviewer verified the accuracy of data abstraction and application of the WW option on a random sample of 10% of the cohort. Data abstraction was repeated until interreviewer concordance was 100% (κ = 1).
Using data from the chart review and from the literature, a decision-analytic cost-utility model of a hypothetical cohort of 1000 children with AOM was constructed (Fig 1). The costs and disability-adjusted life years (DALY) were calculated for the “current practice” arm and compared against an ideal reference scenario in which all patients qualifying for WW received WW. Comorbidities were not assessed because the vast majority of this cohort was previously healthy. Model parameters are shown in Table 1. Direct and indirect costs of care (ie, antibiotics, office visits, hospitalizations, missed work) were derived from the literature, Medicaid data, and costs at this institution. Pain relief and analgesia from treatment were factored in as indirect costs of care.
The primary outcome was the incremental cost-effectiveness ratio (ICER), measured from the societal perspective and reported in 2015 US$ per DALY averted. The modeled time horizon included the beginning of AOM illness to the direct consequences of the disease. Future DALYs were discounted at 3% per year.14 Costs were not discounted due to the short time horizon of the illness.
Sensitivity and Uncertainty Analyses
To account for uncertainty in parameter values, 1-way sensitivity analyses were performed across the plausible ranges of values for all variables subject to nontrivial uncertainty. In an alternate scenario, universal WW was compared with a policy of universal antibiotic prescription and to no antibiotic prescription for AOM. In the no-antibiotic arm, the patient has to return for an office visit to obtain antibiotics if AOM fails to resolve spontaneously. Probabilistic uncertainty analysis, in which all parameters were simultaneously randomly varied across 10 000 Monte Carlo iterations, was used to estimate the probability of cost-effectiveness of the WW option versus current practice. Analyses were conducted in TreeAge Pro 2011 (TreeAge Software, Williamston, MA).
Current Practice of Antibiotic Prescription
From the original cohort of 250 patients, 3 patients were excluded from the analysis because they did not have AOM based on chart review of their physical exam findings. Of the remaining 247 patients, the median age was 3 years (range: 6 months–18 years) with 135 males (54.7%). In this population, 210 (85%) patients were previously healthy presenting with only AOM, and 13 (5.3%) had coexisting conditions that would affect antibiotic decision (persistent fever, initiation of antibiotics by outside provider, urinary tract infection, conjunctivitis), and 24 (9.7%) had chronic comorbidities (eg, asthma, recurrent AOM, prematurity).
In this population, 231 (93.5%) patients were prescribed antibiotics, 7 (2.8%) were advised to undergo WW, and 9 (3.6%) were sent home without an antibiotic prescription. When reviewing each patient’s clinical signs and symptoms alongside the AAP guidelines for AOM management, 104 patients (42.1%) met conditions for immediate antibiotic prescription, and 143 patients (57.9%) were eligible for WW. When patients met AAP guidelines criteria for immediate antibiotics, current practice in this ED prescribed antibiotics 100% of the time. However, when AAP guidelines suggested that WW was an option, current practice in this ED instituted WW only 4.9% of the time. Of those who were advised to undergo WW or sent home without an antibiotic prescription, all met AAP guidelines criteria for WW.
Cost-effectiveness of Current Practice Compared With AAP Guidelines for WW
In the decision-analytic model, implementing WW for all those who met criteria yielded lower costs and averted more DALYs than current practice. For every 1000 patients with AOM, adopting WW yielded 514 fewer immediate antibiotic prescriptions and 205 fewer antibiotic courses ultimately taken, averting 14.3 DALYs and saving $5573. In this model, using WW when clinically appropriate was thus a dominant strategy, meaning it was superior to current practice by generating fewer DALYs at a lower cost.
One-way sensitivity analyses showed that the dominance of implementing WW over current practice was preserved across the entire range of values considered for each of these parameters. Probabilistic uncertainty analysis found that implementing WW dominated current practice 99.94% of the time (Fig 2), costing less and incurring fewer DALYs. In 0.06% of cases, WW was equivocal because it either cost less but incurred more DALYs (0.05%) or cost more but incurred fewer DALYs (0.01%) compared with current practice. WW implementation was never dominated by current practice. In summary, only 6 of our 10 000 simulations generated a result in which current practice was either less costly or more effective than WW. When using a per-DALY willingness-to-pay threshold equal to the per capita gross domestic product of the United States ($54 630),19,20 a commonly used threshold for assessing the cost-effectiveness of interventions, adopting WW was preferred over current practice 100% of the time.10
In an alternate scenario comparing the methods of universal antibiotics, no antibiotics, and WW, WW dominated the other approaches. In a cohort of 1000 individuals, a no-antibiotic approach would be expected to cost $15 659 more and generate an additional 0.4 DALYs compared with WW. In the same cohort, a universal antibiotic approach would cost $9724 more and generate 27.8 more DALYs. Sensitivity analysis across a range of values for the parameters in this scenario maintained the same dominance pattern.
This cost-effectiveness analysis demonstrated that from the societal perspective, using the WW option when consistent with AAP guidelines for AOM management is a dominant strategy compared with current practice, associated with lower total costs and more averted DALYs. These results were maintained across a number of sensitivity analyses.
The cost-effectiveness of a WW approach was further explored through an alternative scenario that unbundled and compared 3 possible treatment strategies for all patients with AOM (universal antibiotic prescription, universal WW, and no antibiotics) with a finding of dominance of WW over the other strategies.
Although antibiotics do reduce duration of pain due to AOM, pain resolves after 1 day of symptoms in 60% of patients even without antibiotics, and the DALYs incurred do not outweigh DALYs from the adverse effects of antibiotics.1 Moreover, sequelae of AOM, namely, mastoiditis and meningitis, are exceedingly rare, occurring at probabilities of 0.00037 and 0.00008, respectively; the risks of developing these complications have not been shown to be affected by antibiotics at the individual level.1,21 Because numerous studies agree that there are no significant differences in rates of serious sequelae from AOM between placebo and antibiotic arms, no difference in probabilities was incorporated in the primary cost-utility model.1,5–8 Nevertheless, when higher probabilities of mastoiditis were modeled in patients not receiving antibiotics, the dominance of WW persisted over current practice. Moreover, AAP guidelines do not recommend a universal no-antibiotic approach because many studies have shown a high prevalence of bacteria in the middle ear effusion (70%–90%).22,23 Although many children will spontaneously clear the bacterial infection, in randomized controlled trials comparing antibiotics to placebo, antibiotic therapy was especially beneficial for symptom resolution in bilateral AOM, AOM with otorrhea, and in children <2 years old.13,17,24 Withholding antibiotics on a population level could lead to a reemergence of suppurative complications such as mastoiditis, which was twice as common before antibiotics.25 These studies demonstrate the benefits of risk stratifying based on symptom severity and age, as recommended by the guidelines, for determining which children should receive immediate antibiotics versus a WW option. The goal of WW is thus to balance antibiotic stewardship with prevention of the rare, but serious, complications of AOM.
Despite using sensitivity analysis, there were limitations to this study. One major limitation is the assumption that the parameter values were the same for every patient regardless of age or symptom severity. This suggests, for example, that the probabilities of cure and of complications from AOM are the same regardless of patient characteristics. Studies have shown a greater difference in symptom resolution between those receiving antibiotics versus placebo in younger patients and in those with more severe AOM symptoms.7,26 These findings suggest that using different parameter values for patients of different ages and symptom severity would more accurately model a real-world scenario. However, many of these values are not differentiated in the literature and could not be accurately depicted in this model.
Another limitation of this study was the use of some parameter values specific to the population in a single ED, such as the probability of receiving antibiotics and the probability of undergoing WW for AOM. These parameter values may not be generalizable to the entire United States. In sensitivity analysis, variation of parameters specific to the study population did not alter the dominance of implementing WW over current practice. Additionally, the feasibility of a WW approach may be different in an ED setting, where the encounters are episodic, compared with a primary care setting, where the physicians are familiar with the families and can provide continuity of care.
Use of data from the literature and retrospective review in a single ED as parameters in the decision-analytic model may also lead to uncertainty about the accuracy of certain parameter values. A previous review by Pichichero et al27 found that many earlier trials evaluating antibiotic management of AOM were based on lax diagnostic criteria for AOM. When possible, parameter values used in our model were obtained from randomized controlled trials implementing stringent diagnostic criteria or from meta-analyses compiling data from multiple trials. As such, the study by Tähtinen et al,8 which confirmed AOM diagnosis with strict pneumatic otoscopic examination criteria and clinical symptomatology, was used for the probability of cure with antibiotics, probability of cure without antibiotics, and probability of adverse events with and without antibiotics parameters. In the retrospective chart review, physical examinations were reviewed in the chart to check laterality of AOM, but stringent diagnostic criteria could not be used to exclude the diagnosis of AOM. Similarly, the data could underestimate the degree of WW used in current practice because there may be cases in which WW was implemented but not documented in the chart. Despite these limitations, retrospective data were only used in this study to characterize current practice and to determine the percentage of patients who met criteria for a WW option. All of these parameters were varied in sensitivity analyses across reasonable ranges.
Another parameter value, the probability of cure without antibiotics after symptoms failed to resolve within 48 hours, was not found in the literature. As a best estimate of this parameter, the probability of cure without antibiotics was used. This was found to be 55.1% in a randomized controlled trial by Tähtinen et al, which compared antibiotics to placebo for AOM. Because this value was not studied in the literature, this parameter varied widely in sensitivity analysis, from cure rates as low as 5%, which is the prevalence of viruses causing AOM, to as high as 86.6%, which is the upper limit of cure rate with antibiotics.22 Varying this parameter did not alter the dominance of implementing WW over current practice.
Antibiotic resistance patterns were also not modeled. Studies have suggested that the immediate use of antibiotics result in more Streptococcus pneumoniae resistance, both when compared with placebo8 and to a WW group.26 In countries with accepted policies of low antibiotic use for AOM, there is also a lower prevalence of resistant pathogens in the nasopharynx.28 Moreover, recent research suggests negative consequences of early and frequent antibiotics on a child’s microbiome and subsequent risk for developing certain diseases.29 Modeling antibiotic resistance patterns and the negative consequences of antibiotics on the microbiome would further improve the cost-effectiveness of WW as an option described by the AAP guidelines.
A cost-effectiveness study conducted of 512 children in Canada, which used post hoc results from a randomized controlled trial comparing amoxicillin to placebo, used the patients from the placebo group who required antibiotics as the WW group. This study found that the amoxicillin group was more cost-effective than the WW group.30 The increased cost of the WW group was due to increased office visits for pain or fever. The original trial, however, was not designed to evaluate WW, and the placebo group patients who required antibiotics constitute a selected subgroup that is unlikely to provide a valid proxy for a randomly assigned WW group (some of whom would not require antibiotics). The results also evaluated effectiveness using treatment failure rather than DALYs or quality-adjust life years, so adverse effects from antibiotics were not modeled. Other cost-effectiveness analyses of AOM management were published before the 2004 AAP AOM guidelines and did not evaluate a WW approach.15,31–33 More recently, a study by Coco evaluated the cost-effectiveness of (1) routine antibiotics; (2) no antibiotics, in which parents have to return for an office visit if symptoms fail to resolve in 48 to 72 hours; and (3) delayed prescription, in which parents are given an antibiotic prescription to be used if symptoms fail to resolve in 48 to 72 hours, similar to our study’s WW option.16 In that study, WW was found to be less costly but less effective than routine antibiotics. However, Coco’s analysis evaluates only mastoiditis and not hearing loss or meningitis as possible sequelae of AOM. Additionally, Coco modeled nonattendance, in which parents do not bring their children with AOM to the physician for evaluation and assumed different rates of nonattendance for each group, with WW having a 37% rate but routine antibiotics having only a 6% rate. Coco’s findings were sensitive to these nonattendance rates, and although we do not model nonattendance because our analysis begins once a child is brought to a physician, it is worth noting that the use of differential rates of nonattendance as a function of future treatment assignment is conceptually problematic, as a parent’s decision to seek care cannot depend on the (unknown to the parent at the time) treatment that the physician would recommend should care be sought.
Ultimately, WW is an option for AOM management in patients meeting criteria outlined in the AAP guidelines. These criteria apply to healthy children with uncomplicated AOM with access to close follow-up. Providers following the WW option by giving parents a safety-net prescription should educate parents to fill the prescription immediately if symptoms do not improve after 2 to 3 days. Physicians identify the main impediments to the WW option as parental reluctance (83.5%) and the cost and difficulty of managing patients who failed to improve (30.9%).10 Yet studies evaluating parental satisfaction found no difference between antibiotic and WW groups.18,26 These studies demonstrate that further education of physicians and parents would be expected to facilitate improved WW implementation.
This study demonstrates that implementing WW for AOM management, when consistent with the AAP guidelines, is cost-effective from a societal perspective. This suggests that although the WW approach requires additional patient follow-up and increasing use of this option would likely require additional provider and parental education, appropriate use of the strategy could simultaneously reduce health expenditures, improve health outcomes, and be cost-saving to society.
- Accepted January 23, 2017.
- Address correspondence to Danica Liberman, MD, MPH, Division of Emergency and Transport Medicine, Children’s Hospital Los Angeles, 4650 Sunset Blvd, Mailstop #113, Los Angeles, CA 90027. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- American Academy of Pediatrics Subcommittee on Management of Acute Otitis Media
- Siegel RM,
- Kiely M,
- Bien JP, et al
- Damoiseaux RA,
- van Balen FA,
- Hoes AW,
- Verheij TJ,
- de Melker RA
- Hoberman A,
- Paradise JL,
- Rockette HE, et al
- Vernacchio L,
- Vezina RM,
- Mitchell AA
- Rovers MM,
- Glasziou P,
- Appelman CL, et al
- Coco AS
- The World Bank
- Heikkinen T,
- Chonmaitree T
- McCormick DP,
- Chonmaitree T,
- Pittman C, et al
- Gaboury I,
- Coyle K,
- Coyle D,
- Le Saux N
- Branthaver B,
- Greiner DL,
- Eichelberger B
- Copyright © 2017 by the American Academy of Pediatrics