PEDIATRICS Vol. 108 No. 1 July 2001, pp. 1-7

Reducing Antibiotic Use in Children: A Randomized Trial in 12 Practices

Jonathan A. Finkelstein, MD, MPH^*, Robert L. Davis, MD, MPH^{, §}, Scott F. Dowell, MD, MPH, Joshua P. Metlay, MD, PhD^¶, Stephen B. Soumerai, ScD^*, Sheryl L. Rifas-Shiman, MPH^*, Margaret Higham, MD^#, Zachary Miller, MD^§, Irina Miroshnik, MS^*, Alex Pedan, PhD^**, and Richard Platt, MD, MS^*,

From the ^* Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, Boston, Massachusetts;  University of Washington, Seattle, Washington; ^§ Group Health Cooperative, Seattle, Washington;  Centers for Disease Control and Prevention, Childhood and Respiratory Diseases Branch, Atlanta, Georgia; ^¶ Veterans Administration Medical Center, Philadelphia, Pennsylvania; ^# Tufts University, Medford, Massachusetts; ^** Vasca, Inc, Tewksbury, Massachusetts; and  Channing Laboratory, Boston, Massachusetts.

	ABSTRACT

Top Abstract Methods Results Discussion References

Objective.  To test whether an educational outreach intervention for families and physicians, based on the Centers for Disease Control and Prevention (CDC) principles of judicious antibiotic use, decreases antimicrobial drug prescribing for children younger than 6 years old.

Setting.  Twelve practices affiliated with 2 managed care organizations (MCOs) in eastern Massachusetts and northwest Washington State.

Patients.  All enrolled children younger than 6 years old.

Methods.  Practices stratified by MCO and size were randomized to intervention or control groups. The intervention included 2 meetings of the practice with a physician peer leader, using CDC-endorsed summaries of judicious prescribing recommendations; feedback on previous prescribing rates were also provided. Parents were mailed a CDC brochure on antibiotic use, and supporting materials were displayed in waiting rooms. Automated enrollment, ambulatory visit, and pharmacy claims were used to determine rates of antibiotic courses dispensed (antibiotics/person-year) during baseline (1996-1997) and intervention (1997-1998) years. The primary analysis (for children 3 to <36 months and 36 to <72 months) assessed the impact of the intervention among children during the intervention year, controlling for covariates including patient age and baseline prescription rate. Confirmatory analyses at the practice level were also performed.

Results.  The practices cared for 14 468 and 13 460 children in the 2 study years, respectively; 8815 children contributed data in both years. Sixty-two percent of antibiotic courses were dispensed for otitis media, 6.5% for pharyngitis, 6.3% for sinusitis, and 9.2% for colds and bronchitis. Antibiotic dispensing for children 3 to <36 months old decreased 0.41 antibiotics per person-year (18.6%) in intervention compared with 0.33 (11.5%) in control practices. Among children 36 to <72 months old, the rate decreased by 0.21 antibiotics per person-year (15%) in intervention and 0.17 (9.8%) in control practices. Multivariate analysis showed an adjusted intervention effect of 16% in the younger and 12% in the older age groups. The direction and approximate magnitude of effect were confirmed in practice-level analyses.

Conclusions.  A limited simultaneous educational outreach intervention for parents and providers reduced antibiotic use among children in primary care practices, even in the setting of substantial secular trends toward decreased prescribing. Future efforts to promote judicious prescribing should continue to build on growing public awareness of antibiotic overuse.  Key words:  antibiotics, prescribing, physician behavior change.

Antibiotic resistance among common pathogenic bacteria in communities has been identified as an emerging threat to public health.¹ The Centers for Disease Control and Prevention (CDC) has identified drug-resistant Streptococcus pneumoniae (DRSP) as a particular threat, and has recommended increased surveillance, risk factor identification, and promotion of judicious antimicrobial prescribing.² The SENTRY hospital-based antimicrobial surveillance program reported high rates of DRSP, with 27.8% and 16.0% showing intermediate- and high-level penicillin resistance, respectively.³ Rates of DRSP in children as high as 41% have also been reported.⁴

The selective pressures on organisms to develop antibiotic resistance are many, but high rates of antibiotic use by humans is a major contributing factor.^5,6 Approximately 110 million antimicrobial courses are prescribed in the United States each year, with the highest rates of use by young children.⁷ We have previously reported rates of 2 to 3 antibiotic prescriptions per child per year in practices affiliated with 2 managed care organizations (MCOs).⁸

Observational⁹ and intervention^10,11 studies from other countries have supported a link between rates of antibiotic prescribing and resistance in communities. The CDC has developed, and other child health organizations have endorsed,¹² Principles of Judicious Use of Antimicrobial Agents aimed at reducing overuse of antibiotics in children.^13,14 There is some evidence that such nationwide campaigns, in combination with publicity in the professional and lay press, have already resulted in some reduction in antibiotic prescribing for children.⁸

Changing patterns of antibiotic use through national campaigns is possible, as demonstrated by a number of studies outside the United States.^10,11,15 However, in more decentralized medical systems, such as that found in the United States, attempts to change physician behavior have had mixed results. Educational materials, without behaviorally oriented reinforcing strategies, are unlikely to change prescribing.^16,17 Targeted educational materials and face-to-face visits to prescribers known as "academic detailing" have been more effective in reducing prescribing of contraindicated or marginally effective therapies.^18,19 Moreover, perception of parental expectation for antimicrobial agents seems to be responsible for substantial overprescribing.^20-22 It is, therefore, likely that reducing prescribing for children will require changes in the behavior of parents as well as their physicians.

We tested the effect of mailings to parents and waiting room materials developed by the CDC, in combination with small group educational sessions with physicians, on antimicrobial prescribing for children younger than 6 years old. We undertook this trial in 12 practices of 2 MCOs, whose data systems were used to analyze rates of antimicrobial prescribing.

	METHODS

Top Abstract Methods Results Discussion References

Design

In this prospective trial, practices were randomly assigned to intervention or control groups. Data from automated claims were analyzed retrospectively for the baseline (preintervention) year (December 1, 1996-November 30, 1997) and intervention year (December 1, 1997-November 20, 1998). The primary outcome measure was the rate of antibiotic courses dispensed to children (antibiotics per person-year) in intervention and control practices. The study was designed to detect a 10% decrease in the rate of antibiotic dispensing attributable to the intervention with 80% power and 2-sided type 1 error of 5%.

Setting

Twelve practices affiliated with 2 MCOsHarvard Pilgrim Health Care (HPHC) in eastern Massachusetts and Group Health Cooperative (GHC) in northwest Washington Statewere studied. The HPHC sites were suburban group practices contracting with the MCO, whereas the GHC sites were multispecialty clinics of an integrated group-model health system. Of the 6 GHC practices, 2 were urban and 4 were suburban or in small towns; no rural practices were included. At HPHC, pharmacy claims for Medicaid enrollees were paid directly by Medicaid and unavailable; HPHC Medicaid members were, therefore, excluded from analysis. Medicaid enrollees at GHC were included; a previous study documented that 11.6% of 2-year-old GHC members were Medicaid enrollees.²³ Although there may have been attention to antibiotic prescribing at some of the practices before the initiation of the randomized trial, there were no relevant plan-wide initiatives at either MCO.

Randomization

Sites were stratified by size (<500 vs >500 person-years of observation for children 3 to <72 months of age), separately for HPHC and GHC. They were then ranked in decreasing order of antibiotic courses prescribed using available data from 1994 to 1996 (before the baseline study year). Practice pairs with the most similar baseline prescribing rates were randomized to intervention or experimental groups.

Subjects

We included in our analysis children 3 months to <72 months of age enrolled in the health plans and receiving pharmacy benefits for at least 3 months during the study period December 1, 1996 to November 30, 1998. We excluded children enrolled in the health plan according to membership data, but who had no record of any ambulatory visits (including well-child care) and no antimicrobial dispensing during the study period. We specified 2 age groups before the analysis: 3 to <36 and 36 to <72 months, because the incidence of otitis media,²⁴ patterns of antibiotic prescribing⁸ and the approach to testing and treatment of febrile illness^25-27 differ in older and younger preschool children.

Educational Intervention

The 1-year targeted educational intervention, begun in the fall of 1997, was designed to change both physician and parental behavior to decrease unnecessary antimicrobial use. Six 1-page evidence-based summaries corresponding to the diagnoses targeted by the CDC's initiative to promote judicious antibiotic prescribing¹³ were created in the format of "academic detailing" summaries¹⁹ (available from the CDC). Endorsement by both CDC and local MCOs was designed to increase the credibility of key messages. In October or November of the intervention year, a practicing pediatrician "peer leader," trained at a CDC workshop, led an initial 90-minute small group educational session with members of the practice. Topics included a discussion of the general problem of antibiotic resistance and potential ways to prevent overuse of antibiotics. Peer leaders focused particularly on differentiating acute otitis media from otitis media with effusion, because prescribing for otitis media is particularly frequent, and on the benefit of clinicians in a practice using similar diagnostic criteria for respiratory tract infections. At approximately the same time, each family receiving care at intervention practices was mailed a copy of the CDC-produced pamphlet entitled "Your Child and Antibiotics," with a cover letter signed by their pediatricians. Key messages to parents were reinforced by educational materials (additional CDC pamphlets and posters) in the waiting and examination rooms of the intervention practices. Study coordinators restocked the experimental sites with educational materials throughout the intervention year. Approximately 4 months after the initial session, the peer leaders visited the intervention sites again to reinforce the recommendations for judicious antibiotic use and to present feedback (bar graphs of practitioner and practice-level antibiotic prescribing rates) from the previous year. No feedback was given to physicians of control practices.

Data Sources

Antibiotic courses dispensed for each child were extracted from automated pharmacy claims data, which include records of both original prescriptions and refills. These claims are submitted by pharmacies for payment when a medication is actually dispensed and include the drug dispensed and the date. We excluded antituberculosis drugs, antihelminthics, topical antimicrobials, and other antimicrobial agents rarely used in ambulatory pediatric care.

Each agent dispensed was linked, when possible, to the most recent ambulatory encounter, ascertained from automated ambulatory claims records.⁸ Encounter diagnoses were grouped according to the following International Classification of Diseases-Ninth Revision codes and their subcodes: otitis media, pharyngitis (including tonsillitis and scarlet fever), sinusitis, bronchitis, pneumonia, upper respiratory infection/common cold, other viral, and a composite group (nonrespiratory bacterial) including other focal bacterial, skin/soft tissue, and urinary tract infections. Remaining diagnoses, including well-child care visits, were categorized as "other." Encounters for which >1 diagnosis was recorded were assigned a primary diagnosis giving priority to a respiratory tract illness and, when present, to a potential bacterial source (eg, if both common cold and otitis media were coded, the latter was designated primary). For this analysis we use the general category "otitis media" to include both acute otitis media and otitis media with effusion, because these conditions are not reliably separable in retrospective, claims-based analyses.

Analysis

The primary goal was to compare the rates of antibiotic courses dispensed per person-year in experimental and control practices. Eligible observation time (in person-years) was determined for each child as the period of membership, in the age group of interest, during the baseline and intervention years. Because some children entered or left the practice, or "aged out" of the cohort in the middle of a year, person-time may have been <1 year for any individual. Unadjusted dispensing rates in control and intervention groups were compared using negative binomial regression, which accounts for the observed overdispersion of rates compared with the Poisson distribution.

In our primary analysis, we assessed the adjusted intervention effect for children enrolled during both years of observation. Because some children crossed the boundary between age groups during the study period, we assigned each child to the 3 to <36 month or 36 to <72 month age group based on their age at the start of the intervention year (December 1, 1997). We used generalized estimating equations²⁸ to predict the antibiotic prescription rate for each individual child in the intervention year, adjusting for each patient's antibiotic use in the baseline year, age, and MCO. Usual regression techniques assume that each observation (in this case, the rate of antibiotic use by each child) is independent. Generalized estimating equations is a regression technique that can account for nonindependence, or clustering, of individuals. Our model accounted for the fact that the prescription rates of children in each practice site are correlated. This model also assumed the relationship between the mean and the variance of the outcome variable was the same as that of a negative binomial distribution.

We also performed a practice-level (N = 12) analysis using linear regression to predict the antibiotic prescription rate in the intervention year, adjusting for the baseline prescription rate in each practice. Because this practice-level analysis did not adjust for clustering or patient level covariates, it was expected to provide a conservative estimate of the intervention effect. All analyses used SAS Software, Version 8 (SAS Institute, Inc, Cary, NC). Results were declared significant if 2-tailed P values were <.05.

	RESULTS

Top Abstract Methods Results Discussion References

A total of 14 468 and 13 460 patients were identified in each of the study years, respectively. Patients in MCO A represented 37% of the study sample and contributed an average of 0.90 person-years of observation in each study year. Patients in MCO B contributed 63% of the study sample and an average of 0.83 person-years in each study year. The characteristics of the 12 practice sites are displayed in Table 1. The practice sites in MCO A had no family physicians on their staffs, but more nurse practitioners than MCO B. 

Otitis media accounted for the majority of antibiotic courses dispensed in both age groups (62.1%) (Fig 1). Other frequent diagnoses associated with prescribing were pharyngitis (6.5%), sinusitis (6.3%), nonrespiratory bacterial infections (3.8%), and pneumonia (2.9%). Antibiotic dispensing linked with viral illnesses such as colds (5.3%) and bronchitis (3.9%) accounted for a total of 9.2% of all antibiotics dispensed in these practices. An additional 9.2% of antibiotics were linked with "other" diagnoses, including "well-child care" visits at which the reason for the prescription was not separately coded.

View larger version (19K): [in this window] [in a new window]   Fig. 1.   Distribution of primary diagnoses linked to an antimicrobial dispensing for all children during both study years.

The rate of antibiotic prescribing in the baseline year varied considerably among the enrolled practices. Practices in MCO B had substantially lower baseline dispensing rates than practices in MCO A (P < .001); and, across both MCOs, the practices randomly selected for intervention status had lower baseline rates than the control sites (P < .001). For children 3 to <36 months (Table 2A), the rate of antimicrobial prescribing decreased by 0.41 antibiotics per person-year (18.6%) in the intervention practices, and 0.33 antibiotics per person-year (11.5%) in the control practices (P < .001). For children 36 months to <72 months, antibiotic use decreased by 0.21 antibiotics per person-year (15.0%) in the intervention group and 0.17 antibiotics per person-year (9.8%) in the control group (P < .001) (Table 2B).

The primary analysis included the 8815 patients who were present in both the baseline and intervention years, and allowed us to control for individual-level covariates such as age and baseline year prescribing; the model also accounted for clustering among patients within practice groups. The data showed a relative intervention effect of 16% (95% confidence interval [CI]: 8%-23%) in patients 3 to <36 months and 12% (CI: 2%-21%) in those 36 to <72 months, beyond the decrease in the control practices (P < .01).

The secondary practice-level analysis, including all children, provided a similar result. For children 3 to <36 months, the regression model showed an intervention effect of 0.23 fewer (95% CI: 0.08-0.39, P < .01) antibiotics per person-year. Adjusting for the baseline rate, there was a relative decrease of 17.4% in the experimental group compared with 11.4% in the control group. For the older children, the intervention effect was 0.13 (95% CI: 0-0.27, P = .06) antibiotics per person-year, with relative decreases of 14.4% in the intervention group and 9.3% in the control group. Finally, we performed an analysis that removed from the model, in turn, the data from each practice site and confirmed that our findings were not the result of extreme results from a single practice site.

	DISCUSSION

Top Abstract Methods Results Discussion References

Antibiotic prescribing decreased in these primary care practices from 1997 to 1998, even in the absence of targeted intervention. This trend in "control" practices, which in magnitude exceeds the crude intervention effect, may suggest substantial success of initiatives to promote more prudent use of antibiotics by the CDC and other public health officials,¹³ professional organizations,¹² and coverage in the lay press. Our multivariate analysis showed a significant additional effect attributable to the intervention of 16% in the younger age group and 12% in the older age group, apparent when we adjusted for nonequivalence of baseline prescribing rates and other potential confounders. These results suggest additional benefit of such direct intervention and reinforce the need for randomized, controlled trials in actual practice settings to assess the incremental value of interventions in the presence of strong secular trends.

The intervention effect is in the range of that seen in other educational outreach studies.²⁹ Our program was based on the body of literature suggesting that dissemination of literature alone would not be enough to change physician prescribing behavior,^16,17 and was designed to be replicable (eg, low-cost, limited time burden on clinicians) for large MCOs. We used small group meetings with a physician peer leader, and distribution of 1-page summaries of key messages consistent with principles of "academic detailing."¹⁹ The intervention also included simultaneous mailings to all families in these practices and supplemental waiting-room materials designed to reduce parental demand for unnecessary antibiotics that has been described by others.^20,21

A number of studies undertaken outside of the United States have shown promise in reducing antibiotic prescribing, and even antimicrobial resistance.^10,11,15 Some of these interventions have included nationwide campaigns or changes in regulations. Gonzales et al³⁰ were successful in decreasing antibiotic use for adults with acute bronchitis by 40%. That trial, in a large group model practice setting, targeted a single condition and intervened in a preselected single site. Although treatment rates for acute bronchitis have also been shown to be high in children,^8,31 in the practices studied here bronchitis accounted for a small fraction of overall antibiotic use.

Although we have no independent confirmation of the diagnoses assigned by physicians in these practices, only a small minority of prescriptions seem to be written for inappropriate indications (eg, colds, bronchitis). Of prescriptions linked to a diagnosis, otitis media accounted for 62% of antibiotic courses prescribed. These include antibiotics given for otitis media with effusion, which is not recommended for initial treatment for this condition.³² We also cannot rule out the possibility that physicians use the diagnosis of otitis media to justify antibiotic treatment of nonspecific respiratory symptoms. If otitis media is overdiagnosed, it would quite likely account for the majority of any antibiotic overuse in these practices.

The strong secular trend in these data are consistent with previous data from these 2 MCOs.⁸ Such decreases may be the result of both changes in parental demand for antibiotics and physician prescribing practices. The CDC produced guidelines for the judicious use of antibiotics in pediatric practice during the study period. These were endorsed by a number of professional groups, including the American Academy of Pediatrics, and were published in full as a supplement to Pediatrics.¹³ Physicians in the control practices were, therefore, exposed to many of the same core messages as those in our intervention. The differences between the practices of the 2 MCOs, previously reported,⁸ is also striking. Possible explanations include differences in actual disease incidence, care-seeking behavior, or thresholds in antibiotic prescribing by providers. Because antibiotic dispensings were not linked with individual providers, we cannot assess possible differences in prescribing by physician specialty. We also did not separately identify dispensings for antibiotic prophylaxis for recurrent acute otitis media or prevention of urinary tract infections. It is possible that regional variation in the use of antibiotics for prophylaxis might affect overall rates, but we believe this effect would make only a small contribution to the differences observed here.

Our results should be interpreted in light of the small number of practices in the study and the substantial variation among them in baseline rates, as well as the limitations of analyses using automated claims data. Automated pharmacy dispensing data routinely collected by these MCOs undercount antibiotics paid for out-of-pocket by patients or given directly as samples in the office. Our data on diagnoses are dependent on accurate coding by physicians themselves. Finally, the lower baseline rates in the practices randomized to the control arm may have limited our ability to detect an effect of our intervention because of a "floor effect." Our random assignment of practices was not effective in equalizing baseline prescribing rates across the study arms because of the small number of units randomized and the use of prescribing data for matching that antedated the baseline study year. We used both patient-level and practice-level techniques to account for these differences, and the finding of an intervention effect was consistent. The result of each of the limitations noted would bias our result toward the null hypothesis or cause us to underestimate the intervention effect.

It remains an open question what level of decrease in antibiotic use would translate into measurable reduction in the prevalence of resistant respiratory tract bacteria in the United States. There is also a lack of information about the optimal threshold for antibiotic use for children that balances treatment benefit with the potential harm of development of resistance in both individuals and communities. Many indications for antibiotic use in children, such as otitis media and sinusitis, are clinical diagnoses with substantial attendant uncertainty and low (but not zero) risks of complications. For otitis media, where the rate of spontaneous resolution is high, physicians in some countries have concluded that initial treatment is not indicated for selected patients.^33-35 The principles of judicious antibiotic use developed by the CDC¹³ articulate condition-specific guides for physicians, but continued measurement will be needed to assess their effect on rates of antibiotic prescribing and resistance.

We believe that it is unlikely that a single intervention will result in a dramatic, sustained drop in antibiotic use for children in this country. Rather, a more gradual change in prescribing rates resulting from continued focus on patient education and physician behavior change may be the best long-term solution to the problem of antibiotic overuse. These data suggest some impact already from national campaigns and increased public awareness and a modest, but real, additional benefit of patient mailings and provider education. What types of interventions might move us to the next level of judicious antibiotic use? Continuing to educate parents about the natural history of viral illnesses in children could certainly decrease parental pressure on physicians to prescribe unnecessary antibiotics. The role that child care center policies play in encouraging unnecessary antibiotic use should be further elucidated.^36,37 However, given the high proportion of antibiotic use for otitis media, we believe that renewed focus on the use of strict criteria for its diagnosis and treatment may have the greatest impact on overall rates of use.

	ACKNOWLEDGMENTS

This study was supported by the Centers for Disease Control and Prevention through task order #200-95-0953 to the Association of Community Health Plans.

We thank Susan Chu, PhD, and Benjamin Schwartz, MD, for invaluable assistance in the early stages of this project; Ann Zavitkovsky and David Rabinowice for their work on Group Health data; and, most importantly, the providers and patients of the 12 practices that participated in this trial.

	FOOTNOTES

Received for publication Sep 6, 2000; accepted Nov 9, 2000.

Address correspondence to Jonathan A. Finkelstein, MD, MPH, Department of Ambulatory Care and Prevention, Harvard Medical School and Harvard Pilgrim Health Care, 126 Brookline Ave, Suite 200, Boston, MA 02215. E-mail: jonathan_finkelstein{at}hphc.org

	ABBREVIATIONS

CDC, Centers for Disease Control and Prevention; DRSP, drug-resistant Streptococcus pneumoniae; MCO, managed care organization; HPHC, Harvard Pilgrim Health Care; GHC, Group Health Cooperative; CI, confidence interval.

	REFERENCES

Top Abstract Methods Results Discussion References

1. Tenover FC, Hughes JM The challenges of emerging infectious diseases . JAMA 1996; 275:300-304
2. Jernigan DB, Cetron MS, Breiman RF Minimizing the impact of drug-resistant Streptococcus pneumoniae (DRSP): a strategy from the DRSP working group. JAMA 1996; 275:206-209
3. Doern GV, Pfaller MA, Kugler K, Freeman J, Jones RN Prevalence of antimicrobial resistance among respiratory tract isolates of Streptococcus pneumoniae in North America: 1997 results from the SENTRY antimicrobial surveillance program . Clin Infect Dis. 1998; 27:764-770
4. Hofmann J, Cetron M, Farley MM, The prevalence of drug-resistant Streptococcus pneumoniae in Atlanta. N Engl J Med 1995; 333:481-486
5. Schwartz B, Bell DM, Hughes JM Preventing the emergence of antimicrobial resistance. A call to action by clinicians, public health officials, and patients. JAMA 1997; 278:944-945
6. Guillemot D, Carbon C, Balkau B, Low dosage and long treatment duration of beta-lactam. Risk factors for carriage of penicillin-resistant Streptococcus pneumoniae. JAMA 1998; 279:365-370
7. McCaig LF, Hughes JM Trends in antimicrobial drug prescribing among office-based physicians in the United States. JAMA 1995; 273:214-219
8. Finkelstein JA, Metlay J, Davis RL, Rifas S, Dowell SF, Platt R Antimicrobial use in defined populations of infants and young children. Arch Pediatr Adolesc Med 2000; 154:395-400
9. Magee JT, Pritchard EL, Fitzgerald KA, Dunstan FD, Howard AJ Antibiotic prescribing and antibiotic resistance in community practice: retrospective study, 1996-8. Br Med J 1999; 319:1239-1240
10. Stephenson J Icelandic researchers are showing the way to bring down rates of antibiotic-resistant bacteria. JAMA 1996; 275:175
11. Seppala H, Klaukka T, Vuopio-Varikila J, The effect of changes in the consumption of macrolide antibiotics on erythromycin resistance in group A streptococci in Finland. N Engl J Med 1997; 337:441-446
12. American Academy of Pediatrics. Judicious use of antimicrobial agents. In: Pickering LK, ed. 2000 Red Book: Report of the Committee on Infectious Diseases. 25th ed. Elk Grove Village, IL: American Academy of Pediatrics; 2000:647-649
13. Dowell SF, Marcy SM, Phillips WR, Gerber MA, Schwartz B Principles of judicious use of antimicrobial agents for pediatric upper respiratory tract infections. Pediatrics 1998; 101:163-165
14. Dowell SF, Schwartz B Resistant pneumococci: protecting patients through judicious use of antibiotics. Am Fam Phys 1997; 55:1647-1654
15. Fujita K, Murono K, Yoshikawa M, Murai T Decline of erythromycin resistance of Group A streptococci in Japan. Pediatr Infect Dis J 1994; 13:1075-1078
16. Davis DA, Thomson MA, Oxman AD, Haynes B Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995; 274:700-705
17. Soumerai SB, Mujumdar S, Lipton HL. Evaluating and improving physician prescribing. In: Strom BL, ed. Pharmacoepidemiology. 3rd ed. Chichester, England: John Wiley and Sons; 2000:483-504
18. Avorn J, Soumerai SB Improving drug therapy decisions through educational outreach: a randomized controlled trial of academically-based "detailing." N Engl J Med. 1983; 308:1457-1463
19. Soumerai SB, Avorn J Principles of educational outreach (`academic detailing') to improve clinical decision making. JAMA 1990; 263:549-556
20. Mangione-Smith R, McGlynn EA, Elliott MN, Krogstad P, Brook RH The relationship between perceived parental expectations and pediatrician antimicrobial prescribing behavior. Pediatrics 1999; 103:711-718
21. Bauchner H, Pelton SI, Klein JO Parents, physicians, and antibiotic use. Pediatrics 1999; 103:395-401
22. Britten N, Ukoumunne O The influence of patients' hopes of receiving a prescription on doctors' perceptions and the decision to prescribe: a questionnaire survey. BMJ 1997; 315:1506-1510
23. Davis RL, Lieu TA, Mell LK, Impact of the change in polio vaccination schedule on immunization coverage rates: a study in two large Health Maintenance Organizations. Pediatrics 2001; 107:671-676
24. Teele DW, Klein JO, Rosner B, Greater Boston Otitis Media Study Group Epidemiology of otitis media during the first seven years of life in children in Greater Boston: a prospective, cohort study. J Infect Dis 1989; 160:83-94
25. Harper MB, Fleisher GR Occult bacteremia in the 3-month-old to 3-year-old age group [review]. Pediatr Ann 1993; 22:484-493
26. Baraff LJ Management of the febrile child: a survey of pediatric and emergency medicine residency directors. Pediatr Infect Dis J 1991; 10:795-800
27. Baraff LJ, Bass JW, Fleisher GR, Practice guideline for the management of infants and children 0 to 36 months of age with fever without source. Pediatrics 1993; 92:1-12
28. Diggle PJ, Liang KY, Zeger SL. Analysis of Longitudinal Data. New York, NY: Oxford University Press Inc; 1994:264
29. O'Brien T, Oxman AD, Davis DA, Haynes RB, Freemantle N, Harvey EL. Educational outreach visits: effects on professional practice and health care outcomes (Cochrane Review). In: The Cochrane Library. Oxford, England; 2000
30. Gonzales R, Steiner JF, Lum A, Barrett PH Decreasing antibiotic use in ambulatory practice: impact of a multidimensional intervention on the treatment of uncomplicated acute bronchitis in adults. JAMA 1999; 281:1512-1519
31. Nyquist AC, Gonzales R, Steiner JF, Sande MA Antibiotic prescribing for children with colds, upper respiratory tract infections, and bronchitis. JAMA 1998; 279:875-877
32. Dowell SF, Marcy SM, Phillips WR, Gerber MA, Schwartz B Otitis media-principles of judicious use of antimicrobial agents. Pediatrics 1998; 101:165-171
33. van Buchem FL, Peeters MF, van't Hof MA Acute otitis media: a new treatment strategy. BMJ 1985; 290:1033-1037
34. Froom J, Culpepper L, Jacobs M, Antimicrobials for acute otitis media? A review from the International Primary Care Network. BMJ 1997; 315:98-102
35. Damoiseaux RAMJ, van Balen FAM, Hoes AW, Verheij TJM, de Melker RA Primary care based randomised, double blind trial of amoxicillin versus placebo for acute otitis media in children aged under 2 years. Br Med J 2000; 320:350-354
36. Schwartz B, Giebink GS, Henderson FW, Reichler MR, Jereb J, Collet J Respiratory infections in day care. Pediatrics 1994; 94:1018-1020
37. Skull SA, Ford-Jones EL, Kulin NA, Einarson TR, Wang EL Child care center staff contribute to physician visits and pressure for antibiotic prescription. Arch Pediatr Adolesc Med 2000; 154:180-183