Challenges to Measuring the Quality of Low-Risk Newborns

With rising health care costs in the United States and other countries, there is a growing interest in measuring and improving the value of health care. In addition to programs such as the Choosing Wisely initiative, which identifies wasteful or unnecessary medical tests,¹ pediatrics and perinatal medicine have seen the development of various measures of care quality. Most of these measures assess outpatient practices, such as immunization adequacy or the appropriate use of well care, in which all children are included regardless of their underlying medical complexity.² For inpatient practice, however, the available metrics tend to focus on the care of more complex patients, such as the very low birth weight infant. Although this group of patients is important, given their high risk of adverse short- and long-term outcomes and their inherent high treatment costs, metrics focused on the very low birth weight infant (eg, reducing central line–associated bloodstream infections) may not provide insight into the quality of care received by the >85% of low-risk newborns born at term each year.

The results of Sebastiao et al³ help to fill the knowledge gap about the care received by this understudied group of infants. Using data from 124 Florida hospitals with >100 deliveries annually, they found that “unexpected complications” after delivery were not unusual, with a rate of 37 complications per 1000 deliveries. These complications were not distributed equally among the 124 delivery hospitals studied. Instead, these rates ranged from 6.7 per 1000 births in the hospital with the lowest rate, to 98.6 per 1000 births in the hospital with the highest rate (a 14.7-fold difference). This between-hospital variation was driven primarily by differences in hospital characteristics, such as hospital level, volume, percentage of Medicaid deliveries, and geographic location. Using this information to improve care, however, requires us to understand what this quality measure can and cannot assess for low-risk patients.

The measure used in the study of Sebastiao et al³ is a composite measure of unexpected newborn complications developed by the California Maternal Quality Care Collaborative. Composite measures are a useful method of combining potential measures of quality that occur rarely into more stable and reliable metrics. Composite metrics, however, have several challenges when applied to an individual hospital. First, we must understand how the various components of the composite measure are combined. As with the unexpected newborn complication measure, many composite measures treat each component equally, with patients triggering the measure if they have any one of a list of outcomes. For this particular measure, a serious complication would be any infant with complications such as death, Apgar score ≤3 at 5 or 10 minutes, sepsis, birth trauma, hypoxia/asphyxia, shock, or intraventricular hemorrhage. It also includes a neonatal transfer to another hospital for any reason and the need for continuous positive airway pressure. As with other composite measures in neonatology, these specific complications have both different short- and long-term outcomes in infants (eg, a transfer of a hypoglycemic infant is very different from that of an infant with sepsis or an infant who has died), and these various measures likely assess different aspects of the overall care delivered to laboring women and their newly delivered infant.^4,5 As a result, hospitals with the same rate of “severe complications” could trigger this metric through an unusually high rate of neonatal death, or a high rate of neonatal transfer. The root causes for such a difference are likely to be substantially different. To further optimize the outcomes of these low-risk deliveries, hospitals, providers, and patients should examine what measures resulted in a hospital achieving the unexpected complication rate that is reported.

The second challenge is the inclusion of measures of hospital utilization in a measure of unexpected complications, specifically the classification of neonatal transfer as a “severe complication.” Perinatal regionalization policies are designed to match hospital capabilities to the needs of a patient.⁶ However, our measures of care focus entirely on the hospital, not the larger systems of perinatal care. For rural women, or women residing in areas without nearby access to a hospital with a high-volume/high-level delivery service, a term low-risk delivery at a local, lower level hospital may be appropriate. In this case, a neonatal transfer would be totally appropriate if a condition arose that the delivery hospital was uncomfortable with or unable to manage but otherwise did not reach the level of illness severity suggested by other outcome measures in this complication category. Although a postdelivery transfer is obviously disruptive to families, it is likely more palatable to families who otherwise may have to travel several hours, while the mother is in active labor, to reach a high-level delivery hospital. A similar case can be made for the prolonged length of stay measure in the “moderate complication” category, which could be elevated for a variety of nonmedical indications, including maternal complications of pregnancy or clearance by child protective service agencies. In both cases, hospitals may be unfairly penalized for their geographic location, their position within larger regional structures of perinatal care that overall improve outcomes of infants, and potentially their patient case-mix. It is also a challenge to compare hospitals for which their elevated rates of complications result from hospital utilization versus a defined medical complication.

Finally, it is difficult to interpret hospitals with very low rates of a measure that in general cannot reach 0%.⁷ At first glance, having the lowest rate possible may seem desirable, as infants did not experience any of these outcomes that suggest an unwanted medical complication. However, we could imagine that hospitals, attempting to reach a rate of zero, may change their practices to avoid triggering such measures, potentially resulting in an early discharge of a patient or a late transfer of care.⁸ Other hospitals may truly be achieving very low rates on this measure but through changes in obstetric practice such as high rates of cesarean delivery. Generally, composite measures include such unwanted changes in practice into the composite to minimize such practice change, such as including neonatal readmission rates when using length of stay in a measure to ostensibly reduce the likelihood that providers will discharge patients too early. Without examining these potential practice changes together with the complication measure, we do not have a sense of what high-performing hospitals may be doing to achieve such a low rate on this metric.

Thus, the research by Sebastiao et al³ provides extremely important insights into the quality and variability of care provided to the vast majority of deliveries in the United States. The paucity of metrics makes such measurement very difficult, particularly when 1 single measure may only provide limited insights into the overall quality of care provided to low-risk populations.⁴ The relative rareness of these outcomes in these low-risk populations, however, provides an additional barrier toward understanding the drivers of such variation, particularly when measures of health care use are combined with more tangible measures of illness severity. Continued development and understanding of the drivers of improved performance in measures of the low-risk population will be critical to improving the value of health care in the United States.

• Copyright © 2017 by the American Academy of Pediatrics