Measuring Quality of Pediatric Care: Where We’ve Been and Where We’re Going

• quality measurement
• quality improvement
• health services research

It is a special privilege to receive an award that honors Doug Richardson, who was my attending physician during residency. He showed extraordinary dedication to his patients, his colleagues, and his trainees. I thought I would share my thoughts on quality measurement here not only because the care that Doug gave was of the highest quality but also because he believed in applying measurement to the delivery of health care^1,2 and he encouraged me to view health services research as fundamental to improving children’s health.

First, I will discuss why we measure quality. Then I will describe where the field was 2 decades ago when I first started working in it. I will then move on to where we are now, and finally, where I hope to see us go in the future.

So, to begin, what do we use quality measurement for? First, quality measurement can help us improve quality. If we don’t know how we’re doing, we don’t know where we need to do better. Study after study has shown that measuring quality can lead to improving it.^3–5 Quality measurement can also make us more accountable, as in public reporting and pay-for-performance, which are becoming more common around the country.⁵ Quality measurement can inform accreditation and certification processes.⁵ The Joint Commission, for example, has incorporated quality measures into its accreditation process and has found evidence suggestive that these measures might be leading to improvement.⁶ And quality measurement can answer critical research questions. It can elucidate disparities by race, ethnicity, income, and geography,⁷ and it can document the effects of policy changes and innovations, such as the development of Accountable Care Organizations.

Back when I was first starting out, I worked on an analysis of published quality-of-care studies for all age groups from the 1980s and 1990s.⁸ The publications showed a consistent pattern of large gaps between the care people should receive and the care they did receive. For example, about half were not receiving recommended preventive care, about 30% were not receiving recommended acute care, and about 40% were not receiving recommended chronic care.⁸ A substantial amount of inappropriate and useless care was also being delivered. As a pediatrician, what stood out to me was how few studies on pediatric quality had been conducted. One of the only studies was by Barbara Starfield, MD, MPH, who showed that around 30–40% of children with asthma did not receive various components of standard care, such as getting an annual flu shot.⁹

I updated this study a few years later to provide the evidence review for the Institute of Medicine (IOM) report, Crossing the Quality Chasm: A New Health System for the 21st Century.³ The report made the bold statements that health care in the United States was not based on the best scientific knowledge and that the health care system was not adequately addressing quality.³ The report helped galvanize the field of quality measurement across the United States and beyond.^10,11

My evidence review again found little work on pediatric quality of care,¹² and the report made scant mention of children.³ A standard explanation for why there was limited development of pediatric quality measures was that the federal government wasn’t funding development because publicly insured children don’t cost it nearly as much as adults do. In addition, though, I think there are some challenges in measuring pediatric quality that have contributed to a lag in developing measures.¹³ First, the low prevalence of many pediatric conditions means that it can be particularly complicated to develop measures beyond those for common conditions like asthma and attention-deficit/hyperactivity disorder. Second, measures need to be age-specific and developmentally appropriate. So, a measure on response to abnormal laboratory values or on proper medication dosing needs to be tied to age and weight, which adds complexity to measure design. Also, much of what we do is prevention, aimed at helping our patients stay healthy decades later, so key outcomes may be too far in the future to measure directly. We may do a great job of helping keep a diabetic child’s blood glucose under control, but any effect on her eyesight is likely years away. Furthermore, ethical and funding constraints have limited the number of randomized trials and rigorous studies on pediatric care, leaving us with a thinner evidence base on which to build quality measures.^14,15 In addition, young children cannot report on their own experience of care, so we turn to parents for answers, but parents aren’t always aware of all that’s happened, particularly when their child is hospitalized.

A number of studies took on these challenges and advanced the field of pediatric quality measurement in the decade after the Quality Chasm report. I’ll give a few examples from among studies that I am particularly familiar with. Several studies showed that a number of children were not receiving basic elements of care, such as standard anticipatory guidance.^16,17 In fact, as Stella Yu, ScD, MPH, documented, many were not even receiving their well visits and dental visits.¹⁸ Rita Mangione-Smith, MD, MPH, published a national study that showed that half of indicated pediatric outpatient care was not provided,¹⁹ and Jason Wang, MD, PhD, developed a measure set for very low birth weight children and used it to show that the vast majority were not receiving key elements of follow-up care.^20,21 That’s just a sampling of all that was published during the first decade of the 2000s; and across the board, studies found substantial gaps in care similar to adult findings.

A decade after the Quality Chasm report, which barely mentioned children, enough work had been done to fill an entire IOM report on pediatric health and health care quality.⁴ The report acknowledged that there had been good progress in developing measures, but it concluded that the United States was still unable to provide timely, comprehensive, valid, and reliable information on quality for children.

This report had been mandated by the Children’s Health Insurance Program (CHIP) Reauthorization Act of 2009, which sought to ramp up the federal investment in pediatric quality. In 2011, in response to this legislation, the Agency for Healthcare Research and Quality and the Centers for Medicare and Medicaid Services established the Pediatric Quality Measures Program; the program funded 7 centers of excellence to increase the number of measures for use across the nation, with a special emphasis on use by state Medicaid and CHIP programs.²² The measures will be in the public domain and available to payers, providers, and consumers. The centers are developing measures on several dozen topics, covering a wide range of issues, from medical errors to newborn care, from mental health treatment to antibiotic prescribing.²³ This represents the largest investment in pediatric quality measurement to date, and if all goes well, it will provide a major leap forward in our ability to measure pediatric quality around the country.

With that foundation, what does the future look like? As technology improves and as we learn from experience, quality measurement will improve as well. The Internet will make it easier, faster, and cheaper to collect patient and family experience data. More versatile and accurate electronic health records will enhance quality measurement, because medical records have much richer information than do claims data,²⁴ which is especially important for controlling for severity of illness. Currently, we must sometimes wait months or longer for measures to be collected and analyzed. Automation drawing on electronic health records and claims data will enable us to get feedback faster. All-payer claims databases are being developed in many states.²⁵ These will prove to be invaluable for collecting data across multiple insurers, practices, and hospitals. We all seem to think that our patients are sicker and more complex. We need to put all patients’ information in a unified data set so that we can case-mix adjust and create a level playing field.

Furthermore, pediatric hospitals and practices are finding that creating registries and quality collaboratives can enable joint measurement plans, benchmarking across sites, and common strategies for improvement. Inflammatory bowel disease, for example, is already benefiting from the ImproveCareNow Network.²⁶

Also looking to the future, researchers and clinicians are working on improving how we convey quality information to patients and families and how we can support them in using this information. This area is expected to move forward rapidly in the coming years, especially with the Patient-Centered Outcomes Research Institute infusing substantial funding into the field.

We are in a period of transition. Pediatric quality measurement is still young. Some things we try will work, and some won’t. This can be hard and frustrating. If we do it right, though, we can make a real difference. What will quality measurement look like when we get to the other side of this transition period? Here are my thoughts on what it will consist of if we get it right:

• Quality measures will be appropriate and designed for children rather than ones that were designed for adults and perhaps tweaked a bit. Indicator sets will no longer cover one small aspect of care, with a clinic, for example, focusing all of its efforts on follow-up visits or on immunizations, to the neglect of other important areas. Indicators will be balanced across key dimensions, such as preventive, acute, and chronic care, as well as transitions. They will also cover both process and outcome measures, including patient experience measures, along with structure measures, where relevant.

• Each insurance contract will use a national consensus-approved set of core measures that are standardized across insurers and states, thereby reducing the institutional burden of responding to multiple measure sets. At the same time, institutions will have the freedom to voluntarily adopt internal quality improvement measures to drive rapid cycle improvement in areas of local concern.

• Measurement will be routine and, as I mentioned above, automated and thereby more affordable. On the subject of affordability, our system will consider the cost of collecting and reporting each measure and its likely impact on quality when judging the value of measures so that we don’t squander our limited resources.

• All measures and their specifications will be publicly available so that we can scrutinize them and determine whether they are valid. Similarly, the results of quality measurement will be made available to consumers so that they can make informed choices.

• Measurement will cover issues that patients and families care about. They care not only about whether we gave them the right medicine but also whether we told them about possible side effects and taught them how to administer the medicine at home.

• If we are succeeding in creating an effective system to measure quality, all of us in health care will embrace the idea of measurement. No one likes being measured, but we need to move measurement past making us feel like we’re being beaten with a stick to a future in which we consider it a tool that is critical to our efforts to provide even better care.

I want to come back to Doug for the final way in which we’ll know if quality measurement is achieving its potential, because I think it’s what he would have cared about the most. Quality measurement only matters if it’s leading to better health care for children and their families, and ultimately to better health. When we get past the details of benchmarks and pay-for-performance and case-mix adjustment, that’s what it is all about: doing a better job for children. As we seek to achieve this goal, I hope we can live up to the standard set by Doug Richardson.

• Copyright © 2015 by the American Academy of Pediatrics