Accuracy of Hospital Administrative Data in Reporting Central Line–Associated Bloodstream Infections in Newborns
OBJECTIVES: Central line-associated bloodstream infections (CLABSIs) are a significant source of morbidity and mortality in the NICU. In 2010, Medicaid was mandated not to pay hospitals for treatment of CLABSI; however, the source of CLABSI data for this policy was not specified. Our objective was to evaluate the accuracy of hospital administrative data compared with CLABSI confirmed by an infection control service.
METHODS: We evaluated hospital administrative and infection control data for newborns admitted consecutively from January 1, 2008, to December 31, 2010. Clinical and demographic data were collected through chart review. We compared cases of CLABSI identified by administrative data (International Classification of Diseases, Ninth Revision, Clinical Modification 999.31) with infection control data that use national criteria from the Centers for Disease Control and Prevention as the gold standard. To ascertain the nature possible deficiencies in the administrative data, each patient's medical record was searched to determine if clinical phrases that commonly refer to CLABSI appeared.
RESULTS: Of 2920 infants admitted to the NICU during our study period, 52 were identified as having a CLABSI: 42 by infection control data only, 7 through hospital administrative data only, and 3 appearing in both. Against the gold standard, hospital administrative data were 6.7% sensitive and 99.7% specific, with a positive predictive value of 30.0% and a negative predictive value of 98.6%. Only 48% of medical records indicated a CLABSI.
CONCLUSIONS: Our findings from a major children’s hospital NICU indicate that International Classification of Diseases, Ninth Revision, Clinical Modification code 993.31 is presently not accurate and cannot be used reliably to compare CLABSI rates in NICUs.
- BSI —
- bloodstream infection
- CDC/NHSN —
- Centers for Disease Control and Prevention and National Healthcare Safety Network
- CLABSI —
- central line-associated bloodstream infection
- CVC —
- central venous catheter
- ICD-9-CM —
- International Classification of Diseases, Ninth Revision, Clinical Modification
- KID —
- Kids’ Inpatient Database
- PICC —
- peripherally inserted central catheter placement
- UVC —
- umbilical venous catheter placement
Bloodstream infections (BSIs) are common in NICUs, occurring in 5% to 42%1,2 of very low birth weight infants. BSIs are associated with a substantial increase in the risk of death,1 poor neurodevelopmental outcomes3 and longer, more costly hospitalizations.1,2 Given the strong association between BSIs and the use and dwell time of central venous lines,1,4,5 decreasing the incidence of central line–associated bloodstream infections (CLABSIs) has emerged as a prominent quality improvement target.
To incentivize hospitals to reduce CLABSIs in adults, in 2008, Medicare initiated a nonpayment policy for hospitalizations with a documented hospital-acquired condition, including CLABSI.6 The Patient Protection and Affordable Care Act of 2010 mandated that Medicaid, the payer for almost one-half of all newborns in the United States each year, adopt a similar nonreimbursement policy for charges associated with CLABSI care.7 Consequently, NICUs will have a strong financial incentive to minimize CLABSIs.
State Medicaid programs have flexibility in determining when a CLABSI has occurred, including the option to use hospital administrative claims data. For the nonpayment policy to have its intended quality-improving effect, CLABSI must be accurately coded. Previous efforts to estimate the accuracy of hospital administrative data for hospital-acquired conditions have chiefly focused on adults,8–11 with few studies including data for children,12,13 and have generally found administrative data to be an inaccurate source of information when compared with clinical data.
No study has focused on accuracy of administrative data for CLABSI in neonates, nor has the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code specific for CLABSI been evaluated in any population. We sought to evaluate the accuracy of administrative data compared with infection control–confirmed CLABSI at a high-volume NICU over several years, to understand the potential consequences of Medicaid nonpayment policies in the NICU setting.
Study Design and Setting
Newborns were identified as having CLABSI using (1) hospital administrative or (2) by Infection Control and Epidemiology review at the University of Michigan Health System for hospital stays beginning on or after January 1, 2008, and ending by December 31, 2010.
Hospital Administrative Data
Newborns were identified as having CLABSI in the hospital administrative record if the ICD-9-CM diagnostic code of 999.31 (infection due to a central venous catheter) appeared in any diagnostic field of the discharge abstract. Although the Centers for Medicare and Medicaid Services have not released coding guidelines for 999.31, our center has developed a guideline, as follows. Hospital billing staff identify first if a central venous line was in place when a bloodstream infection was documented. If yes, then hospital billers determine if the CLABSI was present on admission or acquired. Importantly, hospital billers may code for a CLABSI only if it is documented in the clinical record and may not use other sources of information, including laboratory data. In our analysis, we included any patient with 999.31 coded if their hospital stay began in the NICU, even if their location changed before discharge. Of note, our hospital administrative record limits the number of total diagnostic fields to 26.
Infection and Epidemiology Data
Blood cultures were obtained on patients in the NICU at the discretion of treating clinicians, and were processed in the microbiology laboratory using BacT/Alert (BioMérieux, Marcy-l'Etoile, France). CLABSIs were verified using the Centers for Disease Control and Prevention and National Healthcare Safety Network (CDC/NHSN) definition by infection control personnel assigned to the NICU.14 Cases were ascertained through review of microbiology laboratory records, medical records, and infection control records using the TheraDoc (Hospira, Inc, Lake Forest, IL) computer system.
After identification of patients, clinical and demographic data were collected through chart review and from data gathered from previous reviews conducted as part of our center’s participation in the Vermont Oxford Network. To improve our understanding of the potential limitation of algorithms used by hospital billing staff to identify CLABSIs, we identified 22 phrases to enhance identification through medical record review. These phrases that commonly refer to CLABSI were agreed on by author consensus (see Appendix) and each patient chart was queried using the Electronic Medical Record Search Engine to determine if any of these phrases appeared in any patient note. The Electronic Medical Record Search Engine is a validated, Web-based tool used to search for patient-specific information in the electronic medical record based on the University of Michigan Clinical Data Repository.15
A study-specific dataset was created with data from infection control, hospital billing, Vermont Oxford Network, and from chart review. Characteristics relevant to newborns were identified, including gestational age, birth weight, gender, race, transfer status, mode of delivery, and multiple gestations. In addition, comorbidities common to high-risk newborns were identified, including intraventricular hemorrhage, periventricular leukomalacia, presence of patent ductus arteriosus, and necrotizing enterocolitis. Data regarding mortality were also collected.
To externally validate our findings, we evaluated the frequency of the code 999.31 in the Kids’ Inpatient Database (KID) from 2009. KID is part of the Healthcare Cost Utilization Project of the Agency for Healthcare Research and Quality. The dataset is nationally representative and samples 80% of pediatric discharges and 10% of uncomplicated births. Discharges are weighted based on the sampling scheme to permit inferences for a nationally representative population.
In 2009, KID contained de-identified information for 7.4 million weighted discharges from 4121 hospitals in 44 states. The Healthcare Cost Utilization Project provides additional details to facilitate statistical analyses that permit national inferences.16 Newborns were first identified as a hospital birth occurring at discharge hospital or another hospital (to include transferred newborns). Using administrative ICD-9-CM diagnostic and procedure codes, descriptive variables identified were umbilical venous catheter placement (UVC; 38.92), peripherally inserted central catheter placement (PICC; 38.93) and infection due to a central venous catheter (999.31).
Descriptive statistics were generated. Measures of predictive performance were calculated, comparing hospital administrative data to infection control reporting as the gold standard. Statistical comparisons were performed by using Stata version 12.0 (Stata Corp., College Station, TX). The institutional review board of the University of Michigan Hospital and Health Systems approved this study.
Of 2920 patients admitted to the NICU from January 2008 through December 2010, a total of 52 patients were identified as having a CLABSI by either infection control review or by hospital administrative data. This translates to a rate of 18 CLABSIs per 1000 patients.
Most patients with CLABSI were girls (58%), white (78%), born at University of Michigan Health System (71%), born by Caesarean delivery (67%), and singletons (79%). Median birth weight was 935 g and median gestation age was 27 weeks. This high-risk cohort of newborns suffered many comorbidities of prematurity including patent ductus arteriosus (62%), any intraventricular hemorrhage (44%), necrotizing enterocolitis (31%), and periventricular leukomalacia (8%). The overall mortality rate was 14% (Table 1).
Predictive Characteristics of Hospital Administrative Data
Among 52 patients in the study cohort, 42 were identified by infection control only, 7 by hospital administrative data only, and 3 by both infection control and hospital administrative data. When compared with infection control reporting as the gold standard, hospital administrative data had sensitivity of 6.7%, specificity of 99.7%, positive predictive value of 30% and negative predictive value of 98.6% (Table 2).
It has been suggested that a high number of diagnoses on a hospital discharge abstract might be associated with inaccurate coding.17 Given that our selected patient cohort was complex with multiple comorbid conditions, we sought to evaluate if the total number of diagnoses in the discharge abstract might be associated with an increased likelihood of inaccurate coding. Among our cohort, the maximum number of discharge diagnoses was 26, with a mean of 20. There was no statistical association between the total number of diagnoses and the reliability of hospital administrative data (data not shown).
Twenty-five (48%) medical records for CLABSI cases contained clinician documentation that described the infection as line associated, such as “line infection” (20), “line sepsis” (9), “catheter associated” (5), or “infected line” (1). In other cases, clinician documentation for CLABSI-positive cases described the microorganism responsible for the bacteremia or described the clinical situation more generally as “late-onset sepsis” (Table 3).
Comparing Frequency of CLABSI to Frequency in a National Dataset
To provide external validity to our findings, we compared our data with the KID, a nationally representative administrative dataset. In 2009, there were 4.1 million newborn discharges recorded in KID. In the KID cohort, 97 400 had a UVC or PICC placed (2.4%) and 891 children with these catheters were reported as having a CLABSI: a line infection rate of 0.9%. In 2009, there were 298 patients in our institution with a UVC or PICC of 942 total NICU admissions (32%) and 11 children with catheters were reported as having CLABSIs by infection control: a rate of 3.7%.
Within our institution, hospital administrative data using ICD-9-CM code 999.31 do not accurately reflect clinically identified CLABSIs. This finding is consistent with previous studies that found administrative data to be an inaccurate source of data for central venous line insertion,13 pneumonia,9 health care–associated infections among children12 and adults,11 methicillin-resistant Staphylococcus aureus,10 and catheter-associated urinary tract infections.8
Administrative data remain an attractive source of information for hospital-acquired conditions, because they are readily available. However, administrative data were not intended for this purpose; hospital coding staff are not trained in infection control processes and are limited by a physician’s description of the infection and its cause in medical record documentation. Additionally, physicians have little incentive to accurately identify CLABSI, might be limited by lack of knowledge of CLABSI definitions, or might simply use different terminology when referring to a CLABSI, as evident in our chart review. One possible solution is to have infection control place an alert in a patient’s medical record if a CLABSI is detected. This might alleviate any ambiguity in physician documentation and could improve the accuracy of hospital administrative data.
Adding to the possible sources of inaccuracy of hospital administrative data are that hospitals may typically only use a defined number of diagnostic fields in a discharge abstract. For instance, the maximum number of diagnostic fields allowed by the Centers for Medicare and Medicaid Services is 9 (although this will soon increase to 25). Given this constraint, hospitals might prioritize the highest paid diagnoses first. This could cause errors of omission, in which low or nonpaying diagnoses (like CLABSI) might have lower priority.17 Our study did not find this effect, possibly because of insufficient power or because our hospital administrative data included a maximum number of 26 diagnoses. Errors of omission may be more likely to occur in administrative datasets such as the KID, which allows a maximum of 25 diagnostic fields but to which some states report fewer fields on a routine basis.18
Although our study confirms that administrative data do not accurately represent actual CLABSI, some states rely upon these data for public reporting.19 In addition, the Agency for Healthcare Research and Quality regularly publishes quality metrics, including one for CLABSI in neonates that relies on administrative data.20 Concerns about the accuracy of administrative data may lead to changes in policy. In fact, in 2011 Medicare began using CDC/NHSN data for hospital quality data published on the public Web site hospitalcompare.gov,21 a trend that many states are following.19
An additional challenge is that public reporting of CLABSIs, whether from CDC/NHSN or administrative data, is not yet risk-adjusted, which might not allow for fair comparisons of institutional differences. For instance, extremely low birth weight infants are at higher risk of CLABSI than very low birth weight infants.1 Given this, if a hospital had a disproportionate number of extremely low birth weight infants, they might appear to have an unusually high CLABSI rate than a hospital with a low proportion of very low birth weight infants. Appropriate methods of risk adjustment may allow for more accurate institutional comparisons.
Our study is limited given that all the data are obtained from 1 center, limiting generalizability. However, our analysis of concurrent KID data regarding children with intravascular catheters (overwhelmingly in the NICU population) suggests that undercoding of CLABSI is a system-wide problem. The national rate of 0.9% that we identified in 2009 is far lower than previous reported rates of 13% to 20%1,2,22 and below our center’s reported rate that same year of 3.7%. Furthermore, analysis of one center’s medical records in detail permits greater understanding of why administrative data have errors of omission.
Another possible limitation to our study is that infections could have been missed by both infection control and administrative coding. This possibility is mitigated by our automated blood culture system, which automatically triggers an infection control service review. Last, misspellings may have occurred in the medical record limiting our ability to detect their CLABSI in our medical record review.
The desired goal of incentivizing NICUs to reduce CLABSI through payment mechanisms would not reliably occur if these policies reply on administrative data for neonatal discharges. Payment policies must rely on more accurate data by mandating reporting to CDC/NHSN and requiring institutions to ensure that their administrative data match these data.
The authors acknowledge Gwen Blackford for her input on this project.
- Accepted December 20, 2012.
- Address correspondence to Stephen Patrick, MD, MPH, MSc, 8-621 C&W Mott Hospital, 1540 E. Medical Center Dr, Ann Arbor, MI 48109-4254; E-mail:
Dr Espinosa's current affiliation is Pediatric Infectious Diseases, University of Louisville, Louisville, Kentucky.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
- Stoll BJ,
- Hansen N,
- Fanaroff AA,
- et al
- Payne NR,
- Carpenter JH,
- Badger GJ,
- Horbar JD,
- Rogowski J
- Stoll BJ,
- Hansen NI,
- Adams-Chapman I,
- et al.,
- National Institute of Child Health and Human Development Neonatal Research Network
- Sengupta A,
- Lehmann C,
- Diener-West M,
- Perl TM,
- Milstone AM
- Aronsky D,
- Haug PJ,
- Lagor C,
- Dean NC
- Schaefer MK,
- Ellingson K,
- Conover C,
- et al
- ↵Yang L, Mei Q, Zheng K, Hanauer DA. Query Log Analysis of an Electronic Health Record Search Engine. Annual Symposium Proceedings Archive, 2011(2011): p. 915–924
- ↵Healthcare Cost and Utilization Project (HCUP). Introduction to the KID: 2009, HCUP KID Related Reports. Rockville, MD: Agency for Healthcare Research and Quality; 2011
- Jhung MA,
- Banerjee SN
- ↵HCUP Kids’ Inpatient Database (KID). Healthcare Cost and Utilization Project (HCUP). 2009. Available at: www.hcup-us.ahrq.gov/kidoverview.jsp. Accessed February 29, 2012
- ↵Neonatal Quality Indicators: Neonatal Blood Stream Infections NQI #3. 2009. Version 4.1. Available at: www.qualityindicators.ahrq.gov/Downloads/Software/SAS/v41A/TechSpecs/NQI%2003%20Neonatal%20Blood%20Stream%20Infection.pdf. Accessed February 29, 2012
- Pyrek KM.,
- Public reporting of infections and the CLABSI mandate.
- Copyright © 2013 by the American Academy of Pediatrics