Improvement Methodology Increases Guideline Recommended Blood Cultures in Children With Pneumonia
BACKGROUND AND OBJECTIVE: A national evidence-based guideline for the management of community-acquired pneumonia (CAP) in children recommends blood cultures for patients admitted with moderate to severe illness. Our primary aim was to increase ordering of blood cultures for children hospitalized with CAP from 53% to 90% in 6 months. The secondary aim was to evaluate the effect of obtaining blood cultures on length of stay (LOS).
METHODS: At a tertiary children’s hospital, interventions to increase blood cultures focused on 3 key drivers and were tested separately in the emergency department and inpatient units by using multiple plan-do-study-act cycles. The impact of the interventions was tracked over time on run charts. The association of ordering blood cultures and LOS was estimated by using linear regression models.
RESULTS: Within 6 months, the percentage of patients admitted with CAP who had blood cultures ordered increased from 53% to 100%. This change has been sustained for 12 months. Overall, 239 (79%) of the 303 included patients had a blood culture ordered; of these, 6 (2.5%) were positive. Patients who had a blood culture did not have an increased LOS compared with those without a blood culture.
CONCLUSIONS: Quality improvement methods were used to increase adherence to evidence-based national guidelines for performing blood cultures on children hospitalized with CAP; LOS did not increase. These results support obtaining blood cultures on all patients admitted with CAP without negative effects on LOS in a setting with a reliably low false-positive blood culture rate.
Community-acquired pneumonia (CAP) is a common cause of hospitalization in children. Blood cultures are the most widely available diagnostic tool for the identification of bacterial pathogens in CAP. In 2011, the Pediatric Infectious Diseases Society (PIDS) and the Infectious Diseases Society of America (IDSA) published a guideline for the management of CAP in children older than 3 months. This guideline recommended that blood cultures be obtained in all children requiring hospitalization for moderate to severe presumed bacterial CAP, and for patients whose symptoms progress or who clinically deteriorate after initiation of antibiotic therapy.1 Despite the consensus recommendation by the PIDS/IDSA, previous work has demonstrated variation in blood culture performance in children hospitalized with CAP.2
Though bacteremia is relatively uncommon,3–6 the PIDS/IDSA guidelines recommend blood cultures because etiology (ie, bacterial versus viral) cannot be readily discerned from clinical presentation or radiographic findings.1 The epidemiology of CAP has changed with an increase in the prevalence of CAP-associated complications, particularly among young children.7 Children who develop CAP-associated complications may initially present with uncomplicated CAP.5 In these patients, a blood culture obtained before antibiotic initiation may be the only opportunity to identify the causative pathogen, because pleural drainage may not always be necessary and pleural fluid cultures are frequently negative.8,9
Guidelines at our institution before the PIDS/IDSA publication recommended obtaining blood cultures only for hospitalized patients with severe, resistant, or “other unusual forms of CAP.”10 This local guideline emphasized the limited utility of blood cultures when antibiotics were administered before obtaining the specimen. At our institution, before the national guideline publication, blood cultures were obtained in only 57.7% of hospitalized patients with a bacteremia rate of ∼6.3% and a “false-positive”/contamination rate of 1.3%.5 To standardize management, we assembled a multidisciplinary team to increase the percentage of patients admitted with CAP who had a blood culture obtained within 24 hours of admission from 53% to 90% within 6 months. A secondary aim examined the effect of obtaining more blood cultures on length of stay (LOS), given concerns reported in the literature that increasing the number of blood cultures will lead to increased LOS.3,11–14
Cincinnati Children’s Hospital Medical Center (CCHMC) is a free-standing, tertiary-care, academic medical center with 523 inpatient beds. This study was not considered human subjects research by the institutional review board.
Planning the Intervention
The improvement team consisted of 4 pediatric hospital medicine (HM) physicians (Drs Shah, Thomson, Statile, and White), 2 pediatric emergency medicine (EM) physicians (Drs Murtagh Kurowski and Iyer), 2 pediatric infectious diseases physicians (Drs Rebecca Brady and Shah), a clinical epidemiologist (Dr Ambroggio), and a research assistant (Ms Sheehan). Process maps were created to outline the current processes for obtaining blood cultures on patients admitted to hospital medicine (HM) from the CCHMC emergency department (ED) or from an outside medical facility (Fig 1). Providers were encouraged, when possible, to obtain the blood culture before antibiotic administration; however, to encourage universal blood cultures, the timing of antibiotic administration was not included on process maps. The team used the maps to guide a failure mode and effects analysis15,16 and constructed a key driver diagram to depict the theory behind our aim (Fig 2). Key drivers were as follows: shared knowledge of the guideline across all providers; efficient, reliable, and accurate ordering of diagnostic tests; and provider knowledge of group and individual performance.
Before designing interventions, we conducted a survey to identify how providers decide when to order blood cultures for patients admitted with CAP. This survey comprised 3 clinical scenarios used to elicit the respondent’s presumptive diagnosis and plan for testing (Supplemental Material). Additionally, the survey included 2 questions that asked providers to identify factors that influenced their decision to obtain or not obtain a blood culture in a patient admitted with CAP. This paper survey was anonymously administered to a random sample of practitioners including attending physicians, fellows, residents, and nurse practitioners. Surveys were completed by 39% of HM and 40% of emergency medicine (EM) providers. The most common reasons for not obtaining a blood culture were related to the clinician: questioning the applicability of the guideline to their patient, perceiving a low severity of illness, or anticipating the patient could be treated with oral antibiotics. Although the survey was not an intervention, it allowed us to target our educational interventions at the concerns expressed by providers.
Interventions were designed to increase collection of blood cultures for children admitted with CAP. Multiple plan-do-study-act cycles17 were conducted to test interventions developed to address key drivers (Fig 2).
In October 2012, the team began intensive education of all HM and EM providers, including residents, around the guideline recommendation for obtaining blood cultures. The recommendation was presented at HM and EM division faculty meetings. Additionally, a short presentation of local data about positive blood culture and contamination rates was developed and distributed to all providers by e-mail and displayed in workspaces. E-mails also included education on “true-positive” blood cultures during this project and laboratory processes for rapidly differentiating true-positive blood cultures from contaminants.
To ensure a consistent knowledge base among current and future providers, an educational learning module was created to provide case-based education on the guideline recommendations.
Identification and Mitigation/Education (Intervention)
In November 2012, daily review of inpatient orders for all newly admitted CAP patients with real-time mitigation of failures began. Study team HM physicians called inpatient providers caring for eligible patients who did not have a blood culture ordered to remind them of the recommendation. These calls were discontinued when a “best practice alert” (BPA) was integrated into the electronic medical record (EMR). Identification and education was provided to EM providers via e-mail when failures were noted. Positive feedback was also given to providers when blood cultures were appropriately ordered, an approach termed “identify and congratulate.”
EMR Changes (Intervention)
The team worked with a specialist for our EMR system (Epic Systems Corporation, Verona, WI) to incorporate higher reliability interventions to assist physicians in obtaining blood cultures.18,19 First, the inpatient CAP admission order set was revised to include a blood culture order if not obtained before admission. Then, the “history and physical examination” template for CAP was revised to reflect the recommendation to obtain a blood culture within the plan. Finally, a BPA was developed to notify providers in the ED if a blood culture was not ordered for a patient with a disposition of “admit” and a diagnosis of CAP. The BPA appears with a preselected order to obtain a blood culture if 1 has not already been ordered within the past 24 hours. The BPA was designed to act as an electronic decision support with the preferred action as the default.
We defined moderate/severe CAP as any patient requiring admission because children with mild illness are not typically hospitalized. We acknowledge that some children may have been admitted for reasons not traditionally associated with moderate/severe disease, such as an inability to tolerate oral antibiotics or mild hypoxemia. However, we felt that these initial findings may be early signs of more severe disease and therefore should be included in our population. Patients were eligible for inclusion if they were admitted to the hospital with a primary or secondary International Classification of Diseases, Ninth Revision (ICD-9) discharge diagnosis code of pneumonia (480.0-2, 480.8-9, 481, 482.0, 482.30-2, 482.41-2, 482.83, 482.89-90, 483.8, 484.3, 485, 486, and 487.0) or effusion/empyema (510.0, 510.9, 511.0-1, 511.8-9, and 513).20 To avoid inclusion of patients with suspected viral CAP in the study population, patients who did not receive antibiotics and patients with a primary diagnosis of asthma, viral CAP, or bronchiolitis were excluded.
Success was defined as ordering a blood culture, at CCHMC or an outside medical facility, within 24 hours of presentation to CCHMC. Although our preference was to have blood cultures drawn before antibiotic administration, the timing of antibiotics relative to the collection of the blood culture did not affect the definition of success, because the team’s aim was to achieve universal blood cultures on all admitted patients with CAP. To determine a baseline percentage of blood cultures obtained, data of patients admitted with CAP between April and September 2012 were retrospectively collected. Although the PIDS/IDSA guideline was published in August 2011, our interventions did not begin until October 2012. During the intervention period, medical records of eligible patients were reviewed daily to ensure patient eligibility and to confirm a blood culture order. Reviewer consensus defined “success” or “failure.” Blood culture results were verified 5 days after initial collection. Run charts were updated weekly during the intervention period and continually displayed in the ED offices. They were also communicated every 1 to 2 months at HM and EM staff meetings. Covariates collected from chart review included the following: age, presence of a complex chronic condition,21 Emergency Severity Index version 4 Triage Level,22 ICU transfer, supplemental oxygen given, and complete blood cell (CBC) count or chest radiograph (CXR) performed. These characteristics were selected a priori as clinically important factors influencing the care of patients with CAP. LOS was defined in hours as difference between admission and discharge date and time.
We constructed 2 run charts to track the impact of our interventions. The first was an aggregate measure of all blood cultures ordered from either an outside medical facility, the ED, or on the inpatient units. The second run chart tracked only blood cultures ordered in the ED to more effectively assess the impact of our interventions in the ED setting, where most blood cultures in our institution are ordered.
Continuous variables were described by using medians and interquartile ranges (IQRs) and compared across groups by using Wilcoxon rank-sum test given presence of outliers and nonnormal distributions. Categorical variables were described by counts and frequencies and compared by using the χ2 test.
Multivariable linear regression analysis was performed to assess the independent effect of obtaining a blood culture on LOS while adjusting for covariates. As LOS was nonnormally distributed, we logarithmically transformed these values. The resulting β-coefficients were back-transformed to reflect the percent change in LOS incurred between subjects who did and those who did not have a blood culture obtained.23 Covariates were included in the final model because of their clinical and biological relevance to the outcome of LOS (eg, age, ICU transfer, and oxygen requirement) or if they had a P value < .1 on univariate analysis (eg, CBC count performed). All analyses were performed with SAS version 9.3 (SAS Institute, Inc, Cary, NC), and P values < .05 were considered statistically significant.
Blood cultures were performed in 239 (79%) of 303 patients admitted to the HM service between April 2012 and June 2013 with a diagnosis of CAP. Median age was 3.0 years (IQR: 1– 6) and 46% were boys. Only 3% (n = 10) of patients were transferred to the ICU (Table 1).
The median proportion of patients who had blood cultures obtained at baseline was 53% (Fig 3 A and B). As a result of improvement efforts, the median shifted to 100% by the beginning of November 2012 and was sustained for 12 months.
In unadjusted analysis, patients who had a blood culture performed were more likely to have a CBC count performed and a longer LOS than patients who did not have blood cultures (Table 1). However, in covariate-adjusted models, there was no statistical difference in LOS between patients who did and did not have a blood culture performed (Table 2).
Of all blood cultures performed on patients admitted with CAP during the study period, 6 (2.5%) were positive. Three blood cultures were considered true-positives: Streptococcus pneumoniae, methicillin-resistant Staphylococcus aureus (MRSA), and Klebsiella oxytoca. The patient with MRSA presented with persistent fever after several doses of amoxicillin, but was well-appearing in the ED. The blood culture obtained on admission identified MRSA and allowed for appropriate broadening of antibiotic therapy to clindamycin on hospital day 2. The other 3 blood cultures were considered false-positives: Staphylococcus saccharolyticus, coagulase-negative Staphylococcus species, and viridans group Streptococcus. Only 2 of 3 patients with a true-positive blood culture had a pleural effusion identified on CXR, and only 1 of those was present on the initial CXR. Because of the small number of cases, no risk factors for blood culture positivity were identified.
We used improvement methodology to sustainably increase the percentage of children admitted with CAP who have a blood culture performed from 53% to 100% in 6 months. Differences in LOS between patients with and without blood cultures were not significant. This improvement to a median of 100% was sustained for 12 months after the completion of the project although wider variation was noted in the ED-specific run chart after completion of this project. We hypothesized that this increased variation was due to the introduction of additional projects in the ED.
Three groups of interventions allowed us to impact our key drivers and obtain our goal of rapidly increasing the rate of blood cultures for patients admitted with CAP: (1) education; (2) identification and education/mitigation; and (3) EMR changes. Reliability principles have previously been used to decrease the rate of system failure in health care.24 Interventions can be classified by their level of reliability (LOR), which translates to the expected failure rate that can be achieved with the intervention (for example, a LOR 1 intervention should yield a 10−1 failure rate).24 In our study, this translated to decreasing the number of patients admitted for CAP without a blood culture performed in the first 24 hours of their visit. We started with a LOR 1 intervention, education and training, which laid the groundwork necessary for our higher level interventions to succeed. Progressing to LOR 2 interventions (identification and education/mitigation) allowed us to achieve our goal rate for blood cultures. The LOR 3 interventions (EMR changes) were essential to redesigning our system; providing the ability to scale back on time-intensive interventions and sustain improvement. Although electronic decision support should theoretically yield a failure rate of 10−3, we were unable to achieve this low of a failure rate in our system. This is likely because of the specific context of our setting, including alert fatigue from multiple existing BPAs.
Since the publication of the PIDS/IDSA guidelines, several studies have addressed the controversy regarding the utility of blood cultures in children with CAP.3,5,12,25,26 Heine et al3 described 330 patients seen at a pediatric hospital for CAP, of which 40% were discharged from the ED and 60% were admitted. Blood cultures were performed in 47% of the patients in their cohort and 3.2% grew pathogenic bacteria.3 Based on their results, the authors advocated institutional guidelines for limiting blood cultures to a subset of pediatric patients admitted with CAP.3 The PIDS/IDSA guideline recommends blood cultures in all patients with moderate/severe CAP, which our team interpreted to include all patients requiring admission for CAP. The utility of routine blood cultures in children admitted with CAP is that children with bacteria not covered by typical first-line antibiotics (ampicillin at our institution) would be identified earlier. Blood cultures may also provide the only opportunity to identify the causative bacterium in children with subsequent clinical deterioration. Better methods of identifying the cause of CAP or those at risk for deterioration may result in a more targeted approach to blood cultures in the future.
Our study was designed to evaluate if improvement methodology could be used to rapidly increase the percentage of patients admitted with CAP who have a blood culture performed without an unintended increase in LOS. We did not find a significant difference in LOS when 100% of patients admitted with CAP had blood cultures performed. Central to the success of this project is the low rate of blood culture contamination in our population (1%), which is substantially lower than rates (3.2%–3.9%) reported in other studies evaluating the utility of blood cultures in children with CAP.3,5,12 Our study suggests that blood cultures can be obtained in all pediatric patients admitted with CAP in a setting with a low rate of blood culture contamination without fear of increased LOS.
We identified a true-positive blood culture rate of 1.3% that, although similar to previous studies, is substantially lower than our preintervention rate (6.3%). This difference may be due to our approach of obtaining blood cultures on all patients rather than selectively in patients perceived to be more severely ill.3,4,6,27–30 This may be a closer estimate of the true rate of bacteremia in admitted patients with CAP.
There are several limitations to this study. First, because this project was performed at a single institution, results may not be generalizable to other institutions. The CCHMC has a strong history of improvement work and a robust infrastructure supporting improvement efforts including easy access to timely data, ability to construct and interpret statistical process control charts, and a culture of continuous improvement. We hypothesize that these contextual factors assisted in the rapidity of uptake and sustainability of our improvement. A quality improvement methods approach, however, could be adapted and used at other institutions to increase the rate of blood cultures in children admitted with CAP. Secondly, we identified eligible patients, those admitted with suspected bacterial CAP, by ICD-9 codes, which could bias our population. We limited potential bias by using a previously established ICD-9 algorithm for patient identification and verifying eligibility through chart review.20,31 Although we attempted to exclude patients with suspected viral CAP on chart review, the degree of suspicion for bacterial CAP is based on clinical judgment, because currently there is no highly sensitive and specific test to differentiate bacterial and viral etiologies. It is possible that inclusion of patients with viral CAP in our study population artificially decreased our rate of bacteremia. Finally, the current study was not designed to address all possible unintended negative consequences of universal blood cultures in patients admitted with CAP (eg, increased cost because of extraneous laboratory tests), but this will be important in future studies.
The PIDS/IDSA CAP guideline provided recommendations for evaluation and management, but adherence to this guideline has been variable.3,5,25,26,32,33 Our study highlights that blood cultures can be routinely performed in children admitted for CAP without increasing LOS in a setting with a low false-positive blood culture rate.
The authors thank Dr Rebecca Brady, Dr Camille Graham, and Joshua Courter for their support and content expertise during this project.
- Accepted December 1, 2014.
- Address correspondence to Eileen Murtagh Kurowski, MD, MS, Division of Emergency Medicine, Cincinnati Children’s Hospital Medical Center, 3333 Burnet Ave, MLC 2008, Cincinnati, OH 45229-3039. E-mail:
Dr Murtagh Kurowski participated in the design of the study, developed the data collection criteria, carried out the statistical analysis, performed chart reviews, designed and executed interventions, and drafted the initial manuscript; Dr Shah participated in the design of the study, participated in the design of the interventions, developed the data collection criteria, and reviewed all drafts of the manuscript; Drs Thomson and Statile participated in the design of the study, performed chart reviews, designed and executed interventions, and reviewed all drafts of the manuscript; Ms Sheehan participated in the design of the study, performed chart reviews, carried out statistical analysis, and reviewed all drafts of the manuscript; Dr Iyer participated in the design of the study, participated in the design of the interventions, supervised the statistical analysis, and reviewed all drafts of the manuscript; Dr White participated in the design of the study, participated in the design of the interventions, developed the data collection criteria, and reviewed all drafts of the manuscript; Dr Ambroggio participated in the design of the study, developed the data collection criteria, performed chart reviews, carried out the statistical analysis, executed interventions, and reviewed all drafts of the manuscript; and all authors approved the final manuscript for submission.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: Drs Ambroggio and Thomson were supported by funds from the National Research Service Award T32HP10027-14, and Dr Shah was supported by funds from the National Institute of Allergy and Infectious Diseases K01A173729.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Bradley JS,
- Byington CL,
- Shah SS,
- et al.
- Heine D,
- Cochran C,
- Moore M,
- Titus MO,
- Andrews AL
- Lee GE,
- Lorch SA,
- Sheffler-Collins S,
- Kronman MP,
- Shah SS
- Langley G,
- Nolan K,
- Nolan T,
- Norman C,
- Provost L
- Nolan T,
- Resar R,
- Haraden C,
- Griffin FA
- Feudtner C,
- Christakis DA,
- Connell FA
- ↵Gilboy N, United States Agency for Healthcare Research and Quality. Emergency Severity Index (ESI). Version 4 Implementation Handbook: A Triage Tool for Emergency Department. AHRQ publication number 12-0014. Rockville, MD: Agency for Healthcare Research and Quality; 2012
- Vittinghoff E,
- Glidden DV,
- Shiboski SC,
- McCulloch CE
- Nolan TW,
- Resar R,
- Haraden C,
- Griffin F
- Parikh K,
- Davis AB,
- Pavuluri P
- Williams DJ
- ↵Ambroggio L, Thomson J, Murtagh Kurowski E, et al. Quality improvement methods increase appropriate antibiotic prescribing for childhood pneumonia. Pediatrics. 2013;131(5). Available at: www.pediatrics.org/cgi/content/full/131/5/e1623
- Newman RE,
- Hedican EB,
- Herigon JC,
- Williams DD,
- Williams AR,
- Newland JG
- Copyright © 2015 by the American Academy of Pediatrics