The Challenges of Addressing Pediatric Quality Measurement Gaps

In this issue of Pediatrics, the article by House et al¹ provides a summary of pediatric quality measures currently available in national databases as well as an assessment of how comprehensively these measures represent common pediatric issues. The authors found a preponderance of process measures that have not been outcome-validated. They also found a dearth of measures for some of the most common and/or costly conditions, and relatively few health outcome measures. These findings are not unexpected and are unlikely to change without a paradigm shift in how quality measures are developed and implemented in health care systems across the United States.

The first step in quality measure development is to review the evidence and/or clinical practice guidelines related to the topic under consideration.² Guidelines represent “state-of-the-art” thinking about how conditions should be diagnosed, treated, and, for chronic conditions, managed over time. Thus, when they exist for a given condition, clinical practice guidelines are usually the main source used to develop quality measures. When guideline recommendations are based on low-quality evidence, which is often the case in pediatrics,³⇓⇓⇓^–7 the quality measures they inform will also be based on this same level of evidence. For such quality measures, a critical step in their development is establishing their validity. This goal can be accomplished by examining how scores on the measures relate to improved health and/or health care outcomes after care was received. However, given the lack of available pediatric outcomes that occur with reasonable frequency or that are sensitive to changes in care processes or structures over relatively short periods of time, pediatric quality measure developers often face difficulty in accomplishing this step. A potential solution is to only assess aspects of care supported by high-levels of evidence, but this approach will then provide an extremely narrow assessment of pediatric health care quality.

Quality measurement is often driven by what we can easily assess given the data we have rather than what we wish to assess related to the most common and costly pediatric conditions. For example, there have been calls to develop measures assessing the quality of care coordination for children with medical complexity,⁸ a high-cost population of pediatric patients.⁹ Twenty such measures were developed by one of the Pediatric Quality Measures Program Centers of Excellence,⁸ and 8 of these measures went on to be endorsed by the National Quality Forum in 2016 supporting the rigorous process that was used in their development.¹⁰ Why, therefore, are these measures not in common use? Mainly because they are parent-reported process measures that rely on survey administration, making them resource-intensive to implement. This area of care is unfortunately one that cannot be meaningfully assessed by using readily available data sources such as administrative claims or electronic health records (EHRs). Until there is a willingness to invest in routine electronic survey data collection on a broad scale, measuring quality in areas of care in which consumers are the best source of the information will remain limited.

The same can be said for common conditions (eg, bronchiolitis) in which implementation of guideline-based quality measures often requires resource-intensive manual abstraction of medical records data from the EHR. Structured EHR data fields that provide automated access to clinical information are extremely limited, and even when they do exist, they often do not provide valid information regarding the care that was rendered.¹¹ Until there are advances in natural language processing and/or the availability of a more comprehensive set of validated, structured EHR data fields for such quality measures, this situation again is unlikely to change.

The infrequent occurrence of many health care outcomes (including hospital readmissions, emergency department return visits, and mortality in children) make their use as meaningful pediatric quality measures challenging at best. If instead we choose to measure health-related quality of life or functional status,^12,13 we face problems of feasibility of implementation and sensitivity to change. These measures require surveys for assessment, making them less feasible to implement on a wide scale. In addition, most children are healthy and primarily receive preventive health services (eg, immunizations), making it difficult to detect meaningful changes in these measures over time. We might assess missed school days or school performance as key pediatric health outcome measures; however, access to educational databases is highly restricted for privacy reasons, making such measurements infeasible on a routine basis.

In short, there is a tension between what we would like to measure and what we are able to measure, given the current data systems and resources available for quality assessment. Without the development of more sophisticated electronic data systems that afford routine collection of both patient- and parent-reported measures, EHRs that include a more comprehensive set of fields that can be validly queried, and information-sharing across silos, we will continue to have many gaps in pediatric quality measurement.

• Copyright © 2017 by the American Academy of Pediatrics