A Multicenter Collaborative to Improve Care of Community Acquired Pneumonia in Hospitalized Children
BACKGROUND AND OBJECTIVES: The Value in Inpatient Pediatrics Network sponsored the Improving Care in Community Acquired Pneumonia collaborative with the goal of increasing evidence-based management of children hospitalized with community acquired pneumonia (CAP). Project aims included: increasing use of narrow-spectrum antibiotics, decreasing use of macrolides, and decreasing concurrent treatment of pneumonia and asthma.
METHODS: Data were collected through chart review across emergency department (ED), inpatient, and discharge settings. Sites reviewed up to 20 charts in each of 6 3-month cycles. Analysis of means with 3-σ control limits was the primary method of assessment for change. The expert panel developed project measures, goals, and interventions. A change package of evidence-based tools to promote judicious use of antibiotics and raise awareness of asthma and pneumonia codiagnosis was disseminated through webinars. Peer coaching and periodic benchmarking were used to motivate change.
RESULTS: Fifty-three hospitals enrolled and 48 (91%) completed the 1-year project (July 2014–June 2015). A total of 3802 charts were reviewed for the project; 1842 during baseline cycles and 1960 during postintervention cycles. The median before and after use of narrow-spectrum antibiotics in the collaborative increased by 67% in the ED, 43% in the inpatient setting, and 25% at discharge. Median before and after use of macrolides decreased by 22% in the ED and 27% in the inpatient setting. A decrease in asthma and CAP codiagnosis was noted, but the change was not sustained.
CONCLUSIONS: Low-cost strategies, including collaborative sharing, peer benchmarking, and coaching, increased judicious use of antibiotics in a diverse range of hospitals for pediatric CAP.
- AAP —
- American Academy of Pediatrics
- ANOM —
- analysis of means
- CAP —
- community acquired pneumonia
- ED —
- emergency department
- ICAP —
- Improving Care in Community Acquired Pneumonia
- IDSA —
- Infectious Diseases Society of America
- IQR —
- interquartile range
- PIDS —
- Pediatric Infectious Diseases Society
- QIDA —
- Quality Improvement Data Aggregator
- QuIIN —
- Quality Improvement Innovation Networks
Pneumonia is one of the most common infections in childhood and a leading indication for pediatric hospitalization in the United States.1 However, substantial variation in disease management is evident across hospitals, resulting in care that is inefficient or, worse, ineffective.2–4 Eliminating unnecessary or unproven therapies and emphasizing evidence-informed best practices is critical to optimizing care. This, among other practices, includes eliminating antibiotic use in those unlikely to benefit, as well as limiting exposure when antibiotics are needed to the most narrow-spectrum agents likely to be effective.
A consensus guideline for the management of pneumonia in children published jointly in 2011 by the Pediatric Infectious Diseases Society (PIDS) and Infectious Diseases Society of America (IDSA) emphasized the need for judicious antibiotic use, both to improve individual outcomes and to slow the impact of antimicrobial resistance.5 To date, changes in antibiotic use patterns in accordance with the guideline have been modest in most US hospitals studied.6,7 Several studies highlight successful local guideline implementation efforts,7–10 although most of these efforts were limited to single institutions.
In this study, we sought to assess the impact of a multicenter learning collaborative, Improving Care in Community Acquired Pneumonia (ICAP), on care for children who require hospitalization for pneumonia, with the overarching goal of increasing compliance with the PIDS/IDSA guideline. Over a period of 1 year across the collaborative, we sought to: (1) increase the overall usage of narrow-spectrum antibiotics for children with pneumonia by 50%; (2) decrease the overall usage of macrolides for children with pneumonia by 50%; and (3) increase judicious antibiotic use for children with pneumonia by improving diagnostic specificity.
This project was sponsored by the Value in Inpatient Pediatrics Network, part of the American Academy of Pediatrics (AAP) Quality Improvement Innovation Networks (QuIIN) and was approved by the AAP Institutional Review Board. Written informed consent was obtained from the team leader at each site. Local teams handled institutional review board approvals as deemed necessary by each institution. No protected health information or patient identifiers were collected for the project and sites were de-identified in any public presentation of data.
An open call for participation was conducted via the AAP Section on Hospital Medicine listserv as well as the QuIIN listserv. To be considered for inclusion, each site was required to obtain institutional approval, develop a multidisciplinary improvement team of at least 3 members, and provide care for ≥20 community acquired pneumonia (CAP) admissions annually. All types of hospitals, including community hospitals, free-standing and non–free standing children’s hospitals, and university- and nonuniversity-affiliated hospitals were recruited for the project.
Planning the Metrics and Intervention
The project was a 1-year collaborative that included educational webinars, a project listserv, and individual site coaching by e-mail and telephone. Project planning began with a 2-day planning group meeting. The planning group set project goals and operationalized consensus metrics derived from the PIDS/IDSA CAP guideline. The following primary goals were established: increase use of narrow-spectrum antibiotics, reduce use of macrolides, and increase judicious use of antibiotics by increasing diagnostic specificity of CAP. To operationalize judicious antibiotic use in CAP, the expert group decided to focus on the codiagnosis of asthma and pneumonia; specifically, the goal was to address patients with an asthma exacerbation that are also misdiagnosed with a bacterial pneumonia and started on antibiotics.
The project leadership created a change package related to the measures and this served as the primary intervention for the project. The change package consisted of: (1) examples of evidence-based pathways and order sets; (2) a toolkit for developing an antibiotic stewardship program; (3) examples of effective communication tools for promoting behavior change; and (4) slide sets for use in educational initiatives.
Additional individualized project guidance for sites was provided on a regular basis by a preassigned expert coach. Each coach was assigned 3 to 4 sites based on area of expertise and the sites’ stated priorities. Coaches were provided with the specific goals of assigned sites as well as access to site performance data. Eight educational webinars were conducted over the project period with topics including quality improvement methodology, clinical evidence surrounding optimal CAP management, and strategies to promote practice change. During the intervention phase, group aggregate performance feedback was reviewed and individual sites presented local data and experience with change efforts. In addition, progress reports were submitted by sites with each data collection period. The AAP maintained an online project workspace with access to project materials, webinar recordings, individual site performance data and group aggregate data (https://www.aap.org/en-us/professional-resources/quality-improvement/Quality-Improvement-Innovation-Networks/Pages/Value-in-Inpatient-Pediatrics-Network-Projects.aspx).
All data were collected by chart review for 3 quarters of each year (summer was excluded due to low volume of CAP) for 2014 and 2015. Encounters for children 3 months to 18 years of age hospitalized (inpatient or observation status) with an International Classification of Diseases, Ninth Revision discharge diagnosis code in any position for pneumonia (481, 482.0, 482.2–0.42, 482.89–0.9, 485, 486) and who received antibiotics for the treatment of CAP were considered for inclusion. Children with chronic, comorbid conditions predisposing them to severe or recurrent respiratory illnesses (eg, genetic, congenital, chromosomal, neuromuscular, or neurodevelopmental abnormalities), those requiring intensive care, mechanical ventilation, or a pleural drainage procedure, and those transferred to or from another hospital were excluded. The first 20 encounters meeting project criteria in each quarter (or all meeting criteria if <20 encounters) were included. The teams initially reviewed patient charts from the preproject period (cycles 1–3) to establish a baseline and then reviewed postintervention data monthly (cycles 4–6). The cycles in this project refer to time periods; specifically, baseline cycles included: cycle 1 (September to November 2013), cycle 2 (December 2013 to February 2014), and cycle 3 (March 2014 to May 2014); and intervention cycles included: cycle 4 (September to November 2014), cycle 5 (December 2014 to February 2015), and cycle 6 (March 2015 to May 2015). June to August was excluded in both the baseline and intervention data collection cycles because of the lower incidence of pneumonia hospitalizations during that time.
Web-based data collection was accomplished by using the AAP Quality Improvement Data Aggregator (QIDA), which created run charts that allowed participants to see their own real-time performance compared with group aggregate performance (which was also presented during the webinars). One designated team member entered data into QIDA, and all core team members had the ability to view data. All data collected in reference to project metrics were based on compliance with the metrics at the chart level.
Narrow-spectrum antibiotics were defined as amoxicillin, penicillin, or ampicillin only. Macrolides included erythromycin, clarithromycin, or azithromycin. Codiagnosis of asthma was determined by administration of steroids and β-agonist therapy in addition to antibiotics for CAP.
Methods of Evaluation
Each site had access to run charting on their performance benchmarked by group aggregate performance throughout the project. Pre- and postproject surveys were administered to capture baseline site characteristics, self-reported knowledge, attitudes, and behaviors surrounding CAP management, and site-specific goals. Local team progress was tracked qualitatively through quarterly narrative progress reports, which provided information on timing and the types of interventions attempted, challenges encountered, and perceived successes.
Data were analyzed at the collaborative and individual hospital levels. During the project, simple run charts were available to sites along with comparisons to group means. At the collaborative level, continuous variables were summarized by using median and interquartile ranges (IQRs) due to nonnormal distributions. Wilcoxon rank-sum tests were used to compare hospital use before and after the project intervention. Generalized estimating equations with robust standard errors were used to assess the association of ICAP implementation on antibiotic use at the hospital level while accounting for clustering. This analysis yielded comparable results to the bivariate analyses presented in the results section.
Analysis of means (ANOM) was also used to analyze project results for individual metrics. ANOM is an established quality improvement statistical method for performing multiple comparisons.11 ANOM was used to compare cycle means to the overall group mean with the goal of determining if the variation between cycles was due to common-cause variation. We chose ANOM over statistical process control because the low volume of CAP in most of our sites resulted in fewer data points than would be required for evaluating change based on statistical process control. ANOMs provide for hypothesis testing by using 3-σ control limits adjusted for the number of comparisons made. All bars crossing the upper or lower control limits are deemed to have differed from the overall project mean for that measure at the 3-σ level. Statistical analyses were performed by using Stata version 13.0 (Stata Corp, College Station, TX).
Fifty-four hospitals applied for project participation and 53 were accepted based on meeting stated minimum requirements in their applications. Fifty-two hospitals were in the United States, and 1 was in Pakistan. Five hospitals (9%) failed to complete the full 6 cycles of data entry and were excluded from the final analysis, yielding 48 hospitals in the final analysis. Table 1 shows demographic information for the 47 hospitals in the United States. Over 40% self-identified as community hospitals; a majority were in the South, in urban settings, and provided care for >50% of patients with public insurance. The hospital in Pakistan self-identified as a free-standing children’s hospital and reported caring for >300 patients with CAP annually.
A total of 3802 charts were reviewed for the project, 1842 (48%) during preintervention cycles 1 through 3 and 1960 (52%) during postintervention cycles 4 through 6. Length of stay for hospitalized patients with CAP remained consistent between the baseline and postintervention periods (P > .05); median baseline length of stay was 3 days (IQR, 2–4 days) before and 2 days (IQR, 2–3) after intervention. Aggregate before and after results summarizing rates of compliance with each measure are presented as site medians and IQRs in Table 2.
Narrow-Spectrum Antibiotic Use
The overall proportion of patients receiving narrow-spectrum antibiotics increased over the continuum of care for the hospitalized patient. Median before and after use of narrow-spectrum antibiotics in the collaborative increased by 67% in the emergency department (ED), 43% in the inpatient setting, and 25% at the time of discharge. For each postintervention cycle, rates of narrow-spectrum antibiotics in both the inpatient and discharge settings exceeded 3-σ above the overall project mean. In the ED setting, narrow-spectrum antibiotic use approached the control limit in cycle 4 and exceeded 3-σ above the overall project mean in cycles 5 and 6 (Fig 1).
Macrolide use in the collaborative decreased in both the ED and inpatient setting. Median before and after use of macrolides in the collaborative decreased by 22% in the ED and by 27% in the inpatient setting. In the ED, use of macrolides decreased toward 3-σ control limits below the overall project mean in cycles 4 and 5 and below the control limit in cycle 6. For inpatient settings, use of macrolides did not decrease in cycle 4, but improved to below the control limits in cycles 5 and 6 (Fig 2).
Diagnostic Specificity of CAP
Our aim to increase diagnostic specificity of CAP was operationalized by focusing on the patients who are codiagnosed with both CAP and asthma. Our goal of decreasing concomitant diagnosis of CAP and asthma met with inconsistent results. There was a decrease in the measure to an aggregate rate below the 3-σ control limit in cycle 5, but this was not sustained into cycle 6. (Fig 3)
Site-specific change for narrow-spectrum antibiotics across the 3 settings (ED, inpatient, and discharge) is presented in Fig 4. Although there was improvement at the collaborative level on each of these measures, not all sites demonstrated improvement over the project period. Specifically, 3 sites did not improve in the ED setting, 4 sites did not improve in the inpatient setting, and 8 sites did not improve at discharge (Fig 4).
This improvement collaborative met 2 of the 3 project goals; increasing narrow-spectrum antibiotic use and reducing macrolide use for CAP at participating hospitals; however, no sustained change in CAP and asthma codiagnosis was noted. The success of the ICAP collaborative in terms of antibiotic selection is notable because it mobilized diverse hospital-based groups from 26 states and 2 countries in an entirely voluntary project to improve management of CAP. Using low-resource strategies, this virtual collaborative shared a toolkit that included sample order sets, clinical practice guidelines, informational presentations, as well as coaching, and was modeled after the strategy used in the Value in Inpatient Pediatrics Network Bronchiolitis Quality Improvement Project.12 Similar to the Bronchiolitis Quality Improvement Project, the ICAP collaborative recruited community hospitals with the aim of dissemination of best practices in sites that may not have easy access to other pediatric-specific quality improvement resources.
Although the publication of a national consensus guideline is an important step in standardizing care and improving quality, much work remains to implement guideline recommendations at the local level, as demonstrated by previous studies reporting on the wide variation in management of CAP in differing types of hospitals.3,13–16 One single-center quality improvement intervention performed before the publication of the PIDS/IDSA guideline demonstrated that implementation of a local guideline increased inpatient narrow-spectrum prescribing from 2% to 44%.9 A multicenter assessment of the impact of local clinical practice guidelines in 41 children’s hospitals found that narrow-spectrum prescribing occurred at a rate of 24% in hospitals without a guideline versus 46% in hospitals with a guideline.13 An intensive quality improvement intervention in a large free-standing children’s hospital combined a clinical practice guideline and an antibiotic stewardship program and achieved an increase in narrow-spectrum usage from 13% to 63% in the inpatient setting.8 By comparison, ICAP promoted similar strategies and achieved similar final rates of narrow-spectrum prescribing: 44% in the ED and 63% in the inpatient setting.
One notable success of the ICAP collaborative was the ability to demonstrate improvement in judicious antibiotic use in the ED setting. One single-center project, employing an intensive quality improvement strategy at a large children’s hospital, increased appropriate prescribing to 100% in both the ED and inpatient settings.10 Others have shown that the use of narrow-spectrum antibiotics is lowest in the ED and increases with inpatient care and, subsequently, discharge.17,18 The ICAP results mirrored these findings. ICAP also included several sites that achieved 100% compliance with narrow-spectrum usage in each of the settings measured. Although narrow-spectrum antibiotic use was lowest in the ED setting, nearly all ICAP hospitals (45 of the 48 sites) demonstrated improvement in this setting, suggesting benefit from the strategy of forming multidisciplinary teams that engage ED physicians.
The second success of the ICAP collaborative was the reduction in inpatient and, to a lesser extent, ED macrolide use. Macrolide antibiotics are typically used to cover for Mycoplasma pneumoniae, a bacterium estimated to cause ∼7% to 10% of inpatient CAP,19 Additionally, it is unclear whether macrolide antibiotics improve outcomes in patients with confirmed M pneumoniae.20 The baseline data from ICAP revealed more than twice as much empirical treatment than prevalence estimates suggest would have been appropriate. Data from ICAP cycles 5 and 6 revealed a decrease in macrolide use to a rate more consistent with prevalence estimates.
The third area of interest in this project was diagnostic specificity, which we addressed by focusing on the concomitant diagnosis of asthma with CAP, colloquially known as “asthmonia.” In a study using administrative data from multiple children’s hospitals, 43% of patients hospitalized with CAP had an asthma codiagnosis.21 However, the same study showed wide regional variation in rates of such a codiagnosis, ranging from 12.4% to 34.8%.21 Given the fact that radiography may not be a true gold standard for CAP diagnosis due to high rates of atelectasis in pediatrics and literature documenting variation in chest radiography interpretation,22 the expert group for this project felt that diagnostic specificity could likely be significantly improved. Furthermore, ED physicians and hospitalists disagreed on the presence of CAP in asthma at rates >50% in at least 1 study,18 with the hospitalists frequently discontinuing antibiotics started in the ED setting. Such disagreements occur because opacities on chest radiographs are variably interpreted as atelectasis or infiltrates, with diagnostic uncertainty prevailing around the presence or absence of bacterial infection. Our experience with this metric supports the idea that the variation observed indicates potentially unnecessary antibiotics use in asthma, although this is an area that warrants significant additional scrutiny.
An important limitation of this study is the absence of data integrity controls at the individual sites. Participating sites performed their own chart reviews and entered their data into the online data collection system. Although the project provided technical support for data collection and entry, there was no way of ensuring that the data entered were correctly abstracted from the patient charts, leaving open the possibility that social desirability bias affected the results. However, as we report, there were sites that did not show improvement. In terms of generalizability, individuals who chose to participate in this voluntary collaborative are motivated to engage in systems change. Although the sites varied in their demographics, any voluntary project will be generalizable only with an engaged and motivated team. Another limitation is the short duration of the project and, hence, the lack of ability to assess the sustainability of change with the current dataset. Finally, the small number of data points and lack of a control group limited our ability to explicitly examine the temporal relationship of the intervention and rule out the impact of ongoing secular trends toward more judicious antibiotic use. However, our analysis is based on showing change that are 3-σ from the overall project mean, which is a conservative approach to asserting change from the preintervention cycles.
This voluntary, multisite quality improvement collaborative using low-resource strategies demonstrated a significant increase in the use of narrow-spectrum antibiotics and a reduction in macrolide usage. ICAP focused on diverse hospitals and therefore has the potential to be generalizable to the wide range of hospitals where the majority of children are hospitalized in the United States.
We thank members of the project planning group and project coaches: Drs Eric Biondi, Matthew Garber, Michael Koster, Joanna Leyenaar, Michelle Marks, Russell McCulloh, Angela Myers, Joanne Nazif, Jason Newland, Natalia Paciorkowski, Kavita Parikh, Mary Ann Queen, Ricardo Quinonez, Shawn Ralston, Samir Shah, Emily Thorell, Ilana Waynik, and Derek Williams. We also thank the 53 hospitals that participated in ICAP (listed below) and their teams’ efforts and commitment to improve care for children hospitalized with pneumonia. Lastly, we thank the AAP QuIIN staff: Faiza Wasif, MPH, Elizabeth Rice-Conboy, MS, Keri Thiessen, Med, and QIDA Program Manager, Kristen Gerage, for their project and data management support.
ICAP participants: AnMed Health Women's and Children's Hospital; Anne Arundel Medical Center; Augusta Health; Bakersfield Memorial Hospital; Baystate Children’s Hospital; Blank Children's Hospital; Cardinal Glennon Children's Hospital; Carilion Roanoke Memorial Hospital; Children's Healthcare of Atlanta at Scottish Rite; Children's Hospital of Illinois at OSF St Francis Medical Center; Children's Hospital of San Antonio; Children's Hospital of the University of Illinois; Children’s Memorial Hermann Hospital; Chippenham Medical Center; Dell Children’s Medical Center; East Tennessee Children's Hospital; Elmhurst Hospital Center; Hackensack University Medical Center; Johns Hopkins Bayview Medical Center; Kosair Children's Hospital; Lehigh Valley Health Network; Levine Children's Hospital; Lucile Packard Children's Hospital at Stanford; Maimonides Infants and Children's Hospital; Mary Washington Hospital; Memorial Children's Hospital; Metrowest Medical Center; Mission Children's Hospital; New York‐Presbyterian Morgan Stanley Children's Hospital, Columbia University Medical Center; Nishtar Hospital, Multan (Pakistan); Northwestern Lake Forest Hospital; Ochsner Hospital for Children; Our Lady of the Lake Children's Hospital; Peyton Manning Children's Hospital at St. Vincent; Primary Children's Hospital; Progress West Hospital; Rush University Medical Center; Sacred Heart Medical Center and Children's Hospital; UCLA Medical Center, Santa Monica; Scottsdale Healthcare Shea Medical Center; Shand's Children’s Hospital; Silver Cross Hospital; Sparrow Hospital Children’s Center; St Luke's University Hospital; Stormont‐Vail HealthCare; SUNY Downstate Medical Center; Texas Children's Hospital; University of Kentucky; Upstate Medical University; Vermont Children's Hospital; WakeMed; WVU Children's Hospital; and Yakima Valley Memorial Hospital.
- Accepted September 6, 2016.
- Address correspondence to Kavita Parikh, MD, MSHS, Children’s National Health System, 111 Michigan Ave NW, Washington, DC 20010. E-mail:
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: This project was funded in part by the American Academy of Pediatrics Friends of Children Fund.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Bourgeois FT,
- Monuteaux MC,
- Stack AM,
- Neuman MI
- Florin TA,
- French B,
- Zorc JJ,
- Alpern ER,
- Shah SS
- Bradley JS,
- Byington CL,
- Shah SS, et al; Pediatric Infectious Diseases Society and the Infectious Diseases Society of America
- Ross RK,
- Hersh AL,
- Kronman MP, et al
- Williams DJ,
- Edwards KM,
- Self WH, et al
- Newman RE,
- Hedican EB,
- Herigon JC,
- Williams DD,
- Williams AR,
- Newland JG
- Ralston SL,
- Garber MD,
- Rice-Conboy E, et al; Value in Inpatient Pediatrics Network Quality Collaborative for Improving Hospital Compliance with AAP Bronchiolitis Guideline (BQIP)
- Parikh K,
- Hall M,
- Mittal V, et al
- Milner TL,
- McCulloh R,
- Koster M,
- Biondi E,
- Hill V,
- Ralston S
- Coon ER,
- Maloney CG,
- Shen MW
- Biondi E,
- McCulloh R,
- Alverson B,
- Klein A,
- Dixon A,
- Ralston S
- Wilson KM,
- Torok MR,
- Localio R, et al; Pediatric Research in Inpatient Settings (PRIS) Network
- Copyright © 2017 by the American Academy of Pediatrics