Skip to main content

Advertising Disclaimer »

Main menu

  • Journals
    • Pediatrics
    • Hospital Pediatrics
    • Pediatrics in Review
    • NeoReviews
    • AAP Grand Rounds
    • AAP News
  • Authors/Reviewers
    • Submit Manuscript
    • Author Guidelines
    • Reviewer Guidelines
    • Open Access
    • Editorial Policies
  • Content
    • Current Issue
    • Online First
    • Archive
    • Blogs
    • Topic/Program Collections
    • AAP Meeting Abstracts
  • Pediatric Collections
    • COVID-19
    • Racism and Its Effects on Pediatric Health
    • More Collections...
  • AAP Policy
  • Supplements
  • Multimedia
    • Video Abstracts
    • Pediatrics On Call Podcast
  • Subscribe
  • Alerts
  • Careers
  • Other Publications
    • American Academy of Pediatrics

User menu

  • Log in
  • My Cart

Search

  • Advanced search
American Academy of Pediatrics

AAP Gateway

Advanced Search

AAP Logo

  • Log in
  • My Cart
  • Journals
    • Pediatrics
    • Hospital Pediatrics
    • Pediatrics in Review
    • NeoReviews
    • AAP Grand Rounds
    • AAP News
  • Authors/Reviewers
    • Submit Manuscript
    • Author Guidelines
    • Reviewer Guidelines
    • Open Access
    • Editorial Policies
  • Content
    • Current Issue
    • Online First
    • Archive
    • Blogs
    • Topic/Program Collections
    • AAP Meeting Abstracts
  • Pediatric Collections
    • COVID-19
    • Racism and Its Effects on Pediatric Health
    • More Collections...
  • AAP Policy
  • Supplements
  • Multimedia
    • Video Abstracts
    • Pediatrics On Call Podcast
  • Subscribe
  • Alerts
  • Careers

Discover Pediatric Collections on COVID-19 and Racism and Its Effects on Pediatric Health

American Academy of Pediatrics
Special ArticleThe 2014 Douglas K. Richardson Award for Perinatal and Pediatric HealthCare Research Address

Measuring Quality of Pediatric Care: Where We’ve Been and Where We’re Going

Mark A. Schuster
Pediatrics April 2015, 135 (4) 748-751; DOI: https://doi.org/10.1542/peds.2014-3082
Mark A. Schuster
Division of General Pediatrics, Department of Medicine, Boston Children’s Hospital, and Harvard Medical School, Boston, Massachusetts
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Info & Metrics
  • Comments
Loading
Download PDF
  • quality measurement
  • quality improvement
  • health services research

It is a special privilege to receive an award that honors Doug Richardson, who was my attending physician during residency. He showed extraordinary dedication to his patients, his colleagues, and his trainees. I thought I would share my thoughts on quality measurement here not only because the care that Doug gave was of the highest quality but also because he believed in applying measurement to the delivery of health care1,2 and he encouraged me to view health services research as fundamental to improving children’s health.

First, I will discuss why we measure quality. Then I will describe where the field was 2 decades ago when I first started working in it. I will then move on to where we are now, and finally, where I hope to see us go in the future.

So, to begin, what do we use quality measurement for? First, quality measurement can help us improve quality. If we don’t know how we’re doing, we don’t know where we need to do better. Study after study has shown that measuring quality can lead to improving it.3–5 Quality measurement can also make us more accountable, as in public reporting and pay-for-performance, which are becoming more common around the country.5 Quality measurement can inform accreditation and certification processes.5 The Joint Commission, for example, has incorporated quality measures into its accreditation process and has found evidence suggestive that these measures might be leading to improvement.6 And quality measurement can answer critical research questions. It can elucidate disparities by race, ethnicity, income, and geography,7 and it can document the effects of policy changes and innovations, such as the development of Accountable Care Organizations.

Back when I was first starting out, I worked on an analysis of published quality-of-care studies for all age groups from the 1980s and 1990s.8 The publications showed a consistent pattern of large gaps between the care people should receive and the care they did receive. For example, about half were not receiving recommended preventive care, about 30% were not receiving recommended acute care, and about 40% were not receiving recommended chronic care.8 A substantial amount of inappropriate and useless care was also being delivered. As a pediatrician, what stood out to me was how few studies on pediatric quality had been conducted. One of the only studies was by Barbara Starfield, MD, MPH, who showed that around 30–40% of children with asthma did not receive various components of standard care, such as getting an annual flu shot.9

I updated this study a few years later to provide the evidence review for the Institute of Medicine (IOM) report, Crossing the Quality Chasm: A New Health System for the 21st Century.3 The report made the bold statements that health care in the United States was not based on the best scientific knowledge and that the health care system was not adequately addressing quality.3 The report helped galvanize the field of quality measurement across the United States and beyond.10,11

My evidence review again found little work on pediatric quality of care,12 and the report made scant mention of children.3 A standard explanation for why there was limited development of pediatric quality measures was that the federal government wasn’t funding development because publicly insured children don’t cost it nearly as much as adults do. In addition, though, I think there are some challenges in measuring pediatric quality that have contributed to a lag in developing measures.13 First, the low prevalence of many pediatric conditions means that it can be particularly complicated to develop measures beyond those for common conditions like asthma and attention-deficit/hyperactivity disorder. Second, measures need to be age-specific and developmentally appropriate. So, a measure on response to abnormal laboratory values or on proper medication dosing needs to be tied to age and weight, which adds complexity to measure design. Also, much of what we do is prevention, aimed at helping our patients stay healthy decades later, so key outcomes may be too far in the future to measure directly. We may do a great job of helping keep a diabetic child’s blood glucose under control, but any effect on her eyesight is likely years away. Furthermore, ethical and funding constraints have limited the number of randomized trials and rigorous studies on pediatric care, leaving us with a thinner evidence base on which to build quality measures.14,15 In addition, young children cannot report on their own experience of care, so we turn to parents for answers, but parents aren’t always aware of all that’s happened, particularly when their child is hospitalized.

A number of studies took on these challenges and advanced the field of pediatric quality measurement in the decade after the Quality Chasm report. I’ll give a few examples from among studies that I am particularly familiar with. Several studies showed that a number of children were not receiving basic elements of care, such as standard anticipatory guidance.16,17 In fact, as Stella Yu, ScD, MPH, documented, many were not even receiving their well visits and dental visits.18 Rita Mangione-Smith, MD, MPH, published a national study that showed that half of indicated pediatric outpatient care was not provided,19 and Jason Wang, MD, PhD, developed a measure set for very low birth weight children and used it to show that the vast majority were not receiving key elements of follow-up care.20,21 That’s just a sampling of all that was published during the first decade of the 2000s; and across the board, studies found substantial gaps in care similar to adult findings.

A decade after the Quality Chasm report, which barely mentioned children, enough work had been done to fill an entire IOM report on pediatric health and health care quality.4 The report acknowledged that there had been good progress in developing measures, but it concluded that the United States was still unable to provide timely, comprehensive, valid, and reliable information on quality for children.

This report had been mandated by the Children’s Health Insurance Program (CHIP) Reauthorization Act of 2009, which sought to ramp up the federal investment in pediatric quality. In 2011, in response to this legislation, the Agency for Healthcare Research and Quality and the Centers for Medicare and Medicaid Services established the Pediatric Quality Measures Program; the program funded 7 centers of excellence to increase the number of measures for use across the nation, with a special emphasis on use by state Medicaid and CHIP programs.22 The measures will be in the public domain and available to payers, providers, and consumers. The centers are developing measures on several dozen topics, covering a wide range of issues, from medical errors to newborn care, from mental health treatment to antibiotic prescribing.23 This represents the largest investment in pediatric quality measurement to date, and if all goes well, it will provide a major leap forward in our ability to measure pediatric quality around the country.

With that foundation, what does the future look like? As technology improves and as we learn from experience, quality measurement will improve as well. The Internet will make it easier, faster, and cheaper to collect patient and family experience data. More versatile and accurate electronic health records will enhance quality measurement, because medical records have much richer information than do claims data,24 which is especially important for controlling for severity of illness. Currently, we must sometimes wait months or longer for measures to be collected and analyzed. Automation drawing on electronic health records and claims data will enable us to get feedback faster. All-payer claims databases are being developed in many states.25 These will prove to be invaluable for collecting data across multiple insurers, practices, and hospitals. We all seem to think that our patients are sicker and more complex. We need to put all patients’ information in a unified data set so that we can case-mix adjust and create a level playing field.

Furthermore, pediatric hospitals and practices are finding that creating registries and quality collaboratives can enable joint measurement plans, benchmarking across sites, and common strategies for improvement. Inflammatory bowel disease, for example, is already benefiting from the ImproveCareNow Network.26

Also looking to the future, researchers and clinicians are working on improving how we convey quality information to patients and families and how we can support them in using this information. This area is expected to move forward rapidly in the coming years, especially with the Patient-Centered Outcomes Research Institute infusing substantial funding into the field.

We are in a period of transition. Pediatric quality measurement is still young. Some things we try will work, and some won’t. This can be hard and frustrating. If we do it right, though, we can make a real difference. What will quality measurement look like when we get to the other side of this transition period? Here are my thoughts on what it will consist of if we get it right:

  • Quality measures will be appropriate and designed for children rather than ones that were designed for adults and perhaps tweaked a bit. Indicator sets will no longer cover one small aspect of care, with a clinic, for example, focusing all of its efforts on follow-up visits or on immunizations, to the neglect of other important areas. Indicators will be balanced across key dimensions, such as preventive, acute, and chronic care, as well as transitions. They will also cover both process and outcome measures, including patient experience measures, along with structure measures, where relevant.

  • Each insurance contract will use a national consensus-approved set of core measures that are standardized across insurers and states, thereby reducing the institutional burden of responding to multiple measure sets. At the same time, institutions will have the freedom to voluntarily adopt internal quality improvement measures to drive rapid cycle improvement in areas of local concern.

  • Measurement will be routine and, as I mentioned above, automated and thereby more affordable. On the subject of affordability, our system will consider the cost of collecting and reporting each measure and its likely impact on quality when judging the value of measures so that we don’t squander our limited resources.

  • All measures and their specifications will be publicly available so that we can scrutinize them and determine whether they are valid. Similarly, the results of quality measurement will be made available to consumers so that they can make informed choices.

  • Measurement will cover issues that patients and families care about. They care not only about whether we gave them the right medicine but also whether we told them about possible side effects and taught them how to administer the medicine at home.

  • If we are succeeding in creating an effective system to measure quality, all of us in health care will embrace the idea of measurement. No one likes being measured, but we need to move measurement past making us feel like we’re being beaten with a stick to a future in which we consider it a tool that is critical to our efforts to provide even better care.

I want to come back to Doug for the final way in which we’ll know if quality measurement is achieving its potential, because I think it’s what he would have cared about the most. Quality measurement only matters if it’s leading to better health care for children and their families, and ultimately to better health. When we get past the details of benchmarks and pay-for-performance and case-mix adjustment, that’s what it is all about: doing a better job for children. As we seek to achieve this goal, I hope we can live up to the standard set by Doug Richardson.

Acknowledgments

I thank Debanjan Pain, AB, and Cassandra J. Thomson, AB, for research assistance and Paul J. Chung, MD, MS, Jonathan A. Finkelstein, MD, MPH, and Katherine D. Vestal, MPH, for comments on drafts of the manuscript.

Footnotes

    • Accepted December 29, 2014.
  • Address correspondence to Mark A. Schuster, MD, PhD, Division of General Pediatrics, Boston Children’s Hospital, 300 Longwood Ave, Boston, MA 02115. E-mail: mark.schuster{at}childrens.harvard.edu
  • Dr Schuster drafted the manuscript and approved the final manuscript as submitted.

  • FINANCIAL DISCLOSURE: The author has indicated he has no financial relationships relevant to this article to disclose.

  • FUNDING: No external funding.

  • POTENTIAL CONFLICT OF INTEREST: The author has indicated he has no potential conflicts of interest to disclose.

References

  1. ↵
    1. Richardson DK,
    2. Gray JE,
    3. McCormick MC,
    4. Workman K,
    5. Goldmann DA
    . Score for Neonatal Acute Physiology: a physiologic severity index for neonatal intensive care. Pediatrics. 1993;91(3):617–623pmid:8441569
    OpenUrlAbstract/FREE Full Text
  2. ↵
    1. Richardson DK,
    2. Corcoran JD,
    3. Escobar GJ,
    4. Lee SK
    . SNAP-II and SNAPPE-II: simplified newborn illness severity and mortality risk scores. J Pediatr. 2001;138(1):92–100pmid:11148519
    OpenUrlCrossRefPubMed
  3. ↵
    1. Institute of Medicine, Committee on Quality of Health Care in America
    . Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001
  4. ↵
    1. Institute of Medicine, Committee on Pediatric Health and Health Care Quality Measures
    . Child and Adolescent Health and Health Care Quality: Measuring What Matters. Washington, DC: National Academies Press; 2011
  5. ↵
    1. Chassin MR,
    2. Loeb JM,
    3. Schmaltz SP,
    4. Wachter RM
    . Accountability measures—using measurement to promote quality improvement. N Engl J Med. 2010;363(7):683–688pmid:20573915
    OpenUrlCrossRefPubMed
  6. ↵
    1. Williams SC,
    2. Schmaltz SP,
    3. Morton DJ,
    4. Koss RG,
    5. Loeb JM
    . Quality of care in U.S. hospitals as reflected by standardized measures, 2002-2004. N Engl J Med. 2005;353(3):255–264pmid:16034011
    OpenUrlCrossRefPubMed
  7. ↵
    Agency for Healthcare Research and Quality. Disparities in healthcare quality among racial and ethnic groups: selected findings from the 2011 National Healthcare Quality and Disparities Reports. Available at: www.ahrq.gov/research/findings/nhqrdr/nhqrdr11/minority.html. Accessed December 4, 2014
  8. ↵
    1. Schuster MA,
    2. McGlynn EA,
    3. Brook RH
    . How good is the quality of health care in the United States? 1998. Milbank Q. 2005;83(4):843–895pmid:16279970
    OpenUrlCrossRefPubMed
  9. ↵
    1. Starfield B,
    2. Powe NR,
    3. Weiner JR,
    4. et al
    . Costs vs quality in different types of primary care settings. JAMA. 1994;272(24):1903–1908pmid:7990241
    OpenUrlCrossRefPubMed
  10. ↵
    1. Dentzer S
    . Still crossing the quality chasm—or suspended over it? Health Aff (Millwood). 2011;30(4):554–555pmid:21471471
    OpenUrlFREE Full Text
  11. ↵
    1. Fernandopulle R,
    2. Ferris T,
    3. Epstein A,
    4. et al
    . A research agenda for bridging the ‘quality chasm’. Health Aff (Millwood). 2003;22(2):178–190pmid:12674420
    OpenUrlAbstract/FREE Full Text
  12. ↵
    Schuster MA, McGlynn EA, Pham CB, Spar M, Brook RH. The quality of health care in the United States: a review of articles since 1987. In: Institute of Medicine, Committee on Quality of Health Care in America, ed. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001:231–308
  13. ↵
    1. Forrest CB,
    2. Simpson L,
    3. Clancy C
    . Child health services research: challenges and opportunities. JAMA. 1997;277(22):1787–1793pmid:9178792
    OpenUrlCrossRefPubMed
  14. ↵
    1. Schuster MA,
    2. Asch SM,
    3. McGlynn EA,
    4. Kerr EA,
    5. Hardy AM,
    6. Gifford DS
    . Development of a quality of care measurement system for children and adolescents: methodological considerations and comparisons with a system for adult women. Arch Pediatr Adolesc Med. 1997;151(11):1085–1092pmid:9369869
    OpenUrlCrossRefPubMed
  15. ↵
    Institute of Medicine, Committee on Palliative and End-of-Life Care for Children and Their Families. Directions for research. In: Field MJ, Behrman RE, eds. When Children Die: Improving Palliative and End-of-Life Care for Children and Their Families. Washington, DC: National Academies Press; 2003:350–391
  16. ↵
    1. Schuster MA,
    2. Regalado M,
    3. Duan N,
    4. Klein DJ
    . Anticipatory guidance for parents of infants and toddlers: are they getting what they need? In: Halfon N, McLearn KT, Schuster MA, eds. Child Rearing in America: Challenges Facing Parents of Young Children. New York, NY: Cambridge University Press; 2002:320–344
  17. ↵
    1. Kogan MD,
    2. Schuster MA,
    3. Yu SM,
    4. et al
    . Routine assessment of family and community health risks: parent views and what they receive. Pediatrics. 2004;113(6 suppl):1934–1943pmid:15173464
    OpenUrlPubMed
  18. ↵
    1. Yu SM,
    2. Bellamy HA,
    3. Kogan MD,
    4. Dunbar JL,
    5. Schwalberg RH,
    6. Schuster MA
    . Factors that influence receipt of recommended preventive pediatric health and dental care. Pediatrics. 2002;110(6). Available at: www.pediatrics.org/cgi/content/full/110/6/e73pmid:12456940
    OpenUrlAbstract/FREE Full Text
  19. ↵
    1. Mangione-Smith R,
    2. DeCristofaro AH,
    3. Setodji CM,
    4. et al
    . The quality of ambulatory care delivered to children in the United States. N Engl J Med. 2007;357(15):1515–1523pmid:17928599
    OpenUrlCrossRefPubMed
  20. ↵
    1. Wang CJ,
    2. McGlynn EA,
    3. Brook RH,
    4. et al
    . Quality-of-care indicators for the neurodevelopmental follow-up of very low birth weight children: results of an expert panel process. Pediatrics. 2006;117(6):2080–2092pmid:16740851
    OpenUrlAbstract/FREE Full Text
  21. ↵
    1. Wang CJ,
    2. Elliott MN,
    3. McGlynn EA,
    4. Brook RH,
    5. Schuster MA
    . Population based assessments of ophthalmologic and audiologic follow-up in children with very low birth weight enrolled in Medicaid: a quality-of-care study. Pediatrics. 2008;121(2). Available at: www.pediatrics.org/cgi/content/full/121/2/e278
  22. ↵
    Agency for Healthcare Research and Quality. Pediatric Quality Measures Program (PQMP) Centers of Excellence grant awards. 2014. Available at: www.ahrq.gov/policymakers/chipra/pubs/pqmpfact.html. Accessed December 4, 2014
  23. ↵
    Agency for Healthcare Research and Quality. Pediatric Quality Measures Program: quality measures. 2014. Available at: www.ahrq.gov/policymakers/chipra/factsheets/factsheets2.html. Accessed December 4, 2014
  24. ↵
    1. Tang PC,
    2. Ralston M,
    3. Arrigotti MF,
    4. Qureshi L,
    5. Graham J
    . Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: implications for performance measures. J Am Med Inform Assoc. 2007;14(1):10–15pmid:17068349
    OpenUrlAbstract/FREE Full Text
  25. ↵
    All-Payer Claims Database Council. Interactive state map report. Available at: www.apcdcouncil.org/state/map. Accessed December 4, 2014
  26. ↵
    1. Crandall W,
    2. Kappelman MD,
    3. Colletti RB,
    4. et al
    . ImproveCareNow: the development of a pediatric inflammatory bowel disease improvement network. Inflamm Bowel Dis. 2011;17(1):450–457pmid:20602466
    OpenUrlCrossRefPubMed
  • Copyright © 2015 by the American Academy of Pediatrics
PreviousNext
Back to top

Advertising Disclaimer »

In this issue

Pediatrics
Vol. 135, Issue 4
1 Apr 2015
  • Table of Contents
  • Index by author
View this article with LENS
PreviousNext
Email Article

Thank you for your interest in spreading the word on American Academy of Pediatrics.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Measuring Quality of Pediatric Care: Where We’ve Been and Where We’re Going
(Your Name) has sent you a message from American Academy of Pediatrics
(Your Name) thought you would like to see the American Academy of Pediatrics web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Request Permissions
Article Alerts
Log in
You will be redirected to aap.org to login or to create your account.
Or Sign In to Email Alerts with your Email Address
Citation Tools
Measuring Quality of Pediatric Care: Where We’ve Been and Where We’re Going
Mark A. Schuster
Pediatrics Apr 2015, 135 (4) 748-751; DOI: 10.1542/peds.2014-3082

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Share
Measuring Quality of Pediatric Care: Where We’ve Been and Where We’re Going
Mark A. Schuster
Pediatrics Apr 2015, 135 (4) 748-751; DOI: 10.1542/peds.2014-3082
del.icio.us logo Digg logo Reddit logo Twitter logo CiteULike logo Facebook logo Google logo Mendeley logo
Print
Download PDF
Insight Alerts
  • Table of Contents

Jump to section

  • Article
    • Acknowledgments
    • Footnotes
    • References
  • Info & Metrics
  • Comments

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • Pediatricians Participation in Quality Improvement at the Time of Enrollment in MOC
  • The Landscape of Quality Measures and Quality Improvement for the Care of Hospitalized Children in the United States: Efforts Over the Last Decade
  • The Creation of a Pediatric Hospital Medicine Dashboard: Performance Assessment for Improvement
  • Google Scholar

More in this TOC Section

  • Enrolling Minors in COVID-19 Vaccine Trials
  • Perspectives on Race and Medicine in the NICU
  • Islamic Beliefs About Milk Kinship and Donor Human Milk in the United States
Show more Special Article

Similar Articles

Subjects

  • Administration/Practice Management
    • Quality Improvement
    • Administration/Practice Management

Keywords

  • quality measurement
  • quality improvement
  • health services research
  • Journal Info
  • Editorial Board
  • Editorial Policies
  • Overview
  • Licensing Information
  • Authors/Reviewers
  • Author Guidelines
  • Submit My Manuscript
  • Open Access
  • Reviewer Guidelines
  • Librarians
  • Institutional Subscriptions
  • Usage Stats
  • Support
  • Contact Us
  • Subscribe
  • Resources
  • Media Kit
  • About
  • International Access
  • Terms of Use
  • Privacy Statement
  • FAQ
  • AAP.org
  • shopAAP
  • Follow American Academy of Pediatrics on Instagram
  • Visit American Academy of Pediatrics on Facebook
  • Follow American Academy of Pediatrics on Twitter
  • Follow American Academy of Pediatrics on Youtube
  • RSS
American Academy of Pediatrics

© 2021 American Academy of Pediatrics