This Article Abstract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My File Cabinet Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via CrossRef Citing Articles via Google Scholar Google Scholar Articles by Dougherty, D. Articles by Simpson, L. A. Search for Related Content PubMed PubMed Citation Articles by Dougherty, D. Articles by Simpson, L. A. Related Collections Miscellaneous Social Bookmarking                         What's this?

PEDIATRICS Vol. 113 No. 1 January 2004, pp. 185-198

SUPPLEMENT ARTICLE

Measuring the Quality of Children’s Health Care: A Prerequisite to Action

Denise Dougherty, PhD and Lisa A. Simpson, MB, BCh, MPH, FAAP

From the Child Health Agency for Healthcare Research and Quality, Rockville, Maryland

	ABSTRACT

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
Objective. To assess the availability and use of quality measures for children’s health care, highlight promising developments, and develop recommendations for future action steps by the child health quality measurement and improvement fields, pediatrics, and the national quality of care enterprise generally.

Study Design. Two-day invitational expert meeting, informed by 3 commissioned articles.

Results. Quality of care for children is far less than optimal. A number of measures are available for measuring children’s health care quality on a regular basis, although measures are scarce at least in many areas (eg, pediatric patient safety, end-of-life-care, mental health care, oral health care, neonatal care, care for school-aged children, and coordination of care). Many of the available measures are not being applied regularly to measure the quality of children’s health care; barriers to implementation include lack of an information infrastructure that is child- and quality-friendly and lack of public support for improving children’s health care quality. To improve the availability and use of quality measures for accountability and improvement, meeting participants recommended that at least 4 activities be national priorities: 1) build public support for quality measurement and improvement in children’s health care; 2) create the information technology infrastructure that can facilitate collection and use of data; 3) improve the reliability, validity, and feasibility of existing measures; and 4) create the evidence base for measures development and quality improvement.

Conclusions. Although substantial progress has been made in the development of quality measures and the implementation of quality-improvement strategies for children’s health care, interest in quality of care for children lags behind that for adult conditions and disorders. Making significant progress will require not only sustained attention by those concerned about improving children’s health and health care but also activities to build a broad base of support among the public and key health care decision-makers.

---

Key Words: quality of care • public opinion • effectiveness research

Abbreviations: AHRQ, Agency for Healthcare Research and Quality • SCHIP, State Children’s Health Insurance Program • CAHMI, Child and Adolescent Health Measurement Initiative • NICHQ, National Initiative for Children’s Health Care Quality • AAP, American Academy of Pediatrics • IOM, Institute of Medicine • NQF, National Quality Forum • QuIC, Quality Interagency Coordination • AMA, American Medical Association • CAHPS, Consumer Assessment of Health Plans Survey • ADHD, attention-deficit/hyperactivity disorder • ICU, intensive care unit • HEDIS, Health Plan Employer Data and Information Set • IT, information technology • NIH, National Institutes of Health • NHQR, National Healthcare Quality Report • NHDR, National Healthcare Disparities Report • CHCS, Center for Health Care Strategies • HRSA, Health Resources and Services Administration • NGC, National Guidelines Clearinghouse • CPD, continuous professional development • PBRN, primary care practice-based research network

All too often, the quality of children’s health care falls short of what it could be. Indeed, extensive quality problems have been documented across all sectors of health services for children and adolescents. For example, problems persist in asthma care, well-child and adolescent care, immunization rates for children and adolescents, sexually transmitted disease screening for adolescents, and in the basic respect health care providers give to their patients and families.^1–4 We know about the presence of these and other quality problems because measures have been developed to monitor them. However, there persists in the child health community a pervasive sense that there are many other problems in children’s health care delivery that are not being adequately measured and monitored.⁵ Without adequate documentation of the full scope and nature of quality problems for children and adolescents, it is likely that efforts directed at improving quality will not yield the results hoped for or will fail to occur in the first place.

To assess the nation’s progress in developing and implementing measures of children’s health care quality, the Agency for Healthcare Research and Quality (AHRQ) and numerous cosponsors convened an invitational meeting in February 2002. The goal of the meeting was to review and assess the state of the science and practice of quality measurement for children’s health care, identify gaps, and develop priorities for future investments by funders, both public and private.

	CURRENT CONTEXT

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
Before the meeting, we were well aware of the substantial progress in children’s health care quality measurement and improvement since the first event focusing on children’s health care quality held in May 1997.⁶ In 1997, quality of care was not a prominent theme nationally, and quality of care for children had not been well-defined. In a June 1997 article on the future of child health services research, the need to measure and improve quality for children was not a theme.⁷ Since then, key developments in children’s health care quality include federal efforts to develop quality measures for Medicaid and the State Children’s Health Insurance Program (SCHIP)⁸; initiation of the Child and Adolescent Health Measurement Initiative (CAHMI)⁹; the National Initiative for Children’s Health Care Quality (NICHQ); leadership at the presidential and other levels of the American Academy of Pediatrics (AAP)¹⁰; the adoption of Maintenance of Certification by the American Board of Pediatrics with an emphasis on assessment and improvement of quality of care in practice as a requirement for ongoing certification; the adoption of the concept of core competencies for physicians in training by the Accreditation Council for Graduate Medical Education, which includes competency in practice-based learning and quality improvement; and multiple sources of support for the development and/or refinement of children’s quality measures and implementation of quality-improvement strategies in real-world health care settings.

Across the age spectrum, the need to improve quality of care for Americans has become a broadly accepted national theme with the publication of reports from a Presidential commission,⁵ the Institute of Medicine (IOM),^11,12 coalitions of health care purchasers,¹³ and foundations.^2,14,15 National interest in monitoring and improving quality spurred Congress to give the former Agency for Health Care Policy and Research a new name—AHRQ—that reflected its new mission to improve quality. The National Quality Forum (NQF) was created as a new public-private–sector organization and joined by numerous professional societies, employers, consumer advocacy organizations, as well as major public purchasers of care.¹⁶ Within the federal government, the Quality Interagency Coordination (QuIC) Task Force, led by AHRQ, coordinates quality-measurement and -improvement efforts across multiple agencies.¹⁷ With few exceptions, these broader national efforts have not yet focused on children.

	MEETING OVERVIEW

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
On the first day of the meeting, participants (Table 1) heard the findings of 3 articles commissioned for the meeting. These presentations were followed by discussion among the 4 groups of key participants (and potential participants) in the emerging national focus on quality: first, the organizations and individuals who develop measures (eg, National Committee on Quality Assurance and Joint Commission on Accreditation of Healthcare Organizations); health care providers and plans, in their roles as both the subjects and potential users of quality measures for accountability and/or improvement (eg, the AAP, the American Academy of Family Physicians, the American Medical Association [AMA], and the Alliance of Community Health Plans); users of quality measures (eg, child advocates, purchasers of health care, and quality-improvement organizations); and experts in children’s health care, quality measures, or both. There of course is some overlap among these groups. For example, professional societies not only respond to requirements for measurement but also develop their own measures¹⁸ and sponsor quality-improvement initiatives. Child advocacy groups have developed their own sets of measures.^19,20 Some meeting attendees represented the pediatric perspective on quality; others offered a diverse set of viewpoints from the wider world of quality. On day 2 of the meeting, funding organizations discussed the points and recommendations made on day 1.

View this table: [in this window] [in a new window]   TABLE 1. Meeting Participants

 
This article has 3 sections. First, we summarize and comment on the 3 commissioned articles and the recommendations made by the meeting participants for further action. Meeting participants were not asked to come to consensus on any issues or to express a majority view; accordingly, this document does not necessarily reflect the precise views of any or all of the organizations and individuals who participated in the discussion. Second, we highlight areas in which progress seems likely given current investments by public and private funders. Third, we propose a number of actions in which pediatricians can engage that will contribute to continued improvements in the quality of children’s health care.

	QUALITY MEASURES FOR CHILDREN’S HEALTH CARE: A CATEGORIZATION USING THE IOM FRAMEWORK

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
Beal et al²¹ conducted a broad literature search for quality measures for children’s health care and used the IOM conceptual framework for quality to characterize the availability of measures.¹² The IOM uses a dual conceptualization of 6 "quality domains" (safety, effectiveness, timeliness, patient-centeredness, efficiency, and equity) and 4 "consumer perspectives" (staying healthy, getting better, living with illness, and coping with end of life) (Fig 1). Because a subsequent IOM report on quality reporting recommended that efficiency measures be omitted from consideration because few or none have been developed, Beal et al did not include efficiency in their analysis.²² They assessed additional criteria that the IOM recommended be applied to thinking about individual quality measures and quality measures as a group, such as the balance and comprehensiveness of the set of available quality measures for children.

View larger version (22K): [in this window] [in a new window]   Fig 1. IOM conceptual framework for quality.

 
To use the IOM framework for analytic purposes, Beal et al deconstructed the 19 child measurement sets they identified as available for current use. Most of the measurement sets were developed before publication of the IOM framework, and many measurement sets’ original conceptualizations did not map completely to the IOM quality framework categories. This state of affairs made necessary an intensive and iterative process to try to "fit" previous measures and items within those measures into the new IOM framework. For example, the Consumer Assessment of Health Plans Survey (CAHPS) may be considered in its entirety a measure of patient-centeredness because it focuses on patient experiences of care, but it also includes items related to timeliness of care.²³ When subscales corresponding to the IOM quality categories were available (eg, CAHPS and Promoting Healthy Development Survey²⁴), they were used in the Beal et al analysis; when they were not, individual items were examined and placed in the appropriate categories. It is critical to note that the availability of the resulting 396 "measures" as defined by Beal et al does not mean that 396 measures are currently being applied to children’s health care. Shaller’s article²⁵ suggests that only relatively few measurement sets come to mind when policy makers and others are asked what measures they use to accomplish specific objectives.

The finding of Beal et al that over half of the measures they categorized came under the effectiveness domain is encouraging from many perspectives. This emphasis on effectiveness of care fits well with national and international efforts to practice evidence-based health care.^26–30 Quality measures in the effectiveness domain can assess the extent to which children are receiving evidence-based health care. For example, the Rand QA Tools System addresses whether children get appropriate diagnosis, treatment, and follow-up for a number of conditions including acne, asthma, and attention-deficit/hyperactivity disorder (ADHD).³¹ Broadly speaking, measures of patient-centeredness and timeliness are both relatively well represented at one third each. In the face of concerns about medical errors, the relative paucity of measures in the patient safety domain is troubling (14%). Beal et al determined that assessing equity with available measures could be difficult, because so few measures contain mechanisms to identify children at risk for inequitable care (eg, by race, income, or presence of a chronic condition).

From the consumer/patient perspective, the Beal et al analysis results were surprising. Fully 40% of the categorized measures apply to the "getting better" or acute illness domain. However, these measures are included predominantly in measurement sets that do not seem to be in widespread use (eg, Ambulatory Care Medical Audit Demonstration Project; Beth Israel Hospital Complications Screening Program; Multi-Dimensional Assessment of Parental Satisfaction; Joint Commission on Accreditation of Healthcare Organizations Indicator Measurement System; and Rand QA Tools System). On the other hand, they are available for those interested in measuring care for children with acute illnesses. Another 24% were categorized as being earlier in the health care pipeline, to the "staying healthy" or preventive care perspective, which may be appropriate because prevention is a cornerstone of children’s health care. Children with special health care needs may face proportionately more quality challenges than healthy children,³² and therefore the availability of sound measures is particularly important. Only 17% of measures were categorized as applying to children living with chronic illness or disability. Very troubling is the complete lack of available measures for children at the end of life in the face of documented quality problems for this group of children.³³

	RISK ADJUSTMENT

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
The second article related to children’s quality measures was by Kuhlthau et al.³⁴ The authors found that, compared with risk adjustment for financial purposes, adjusting or stratifying quality measures for risk has been an understudied area. Risk adjustment is relevant to measurement criteria of validity, feasibility, and equity and deserves a special focus. Willingness of plans to accept individuals from low-income and other potentially high-need groups of children may be affected if appropriate risk adjustments are not developed.³⁴ On the other hand, risk adjustment should not have the effect of freeing plans and providers from the responsibility to meet the needs of high-risk and vulnerable populations of children. After conducting a careful review of risk-adjustment and stratification approaches in child health care quality measures, the authors of this article made 3 observations.

First, they observed that risk adjustment is more important for quality measures based on child outcomes and population-based data than for quality measures that make use of process of care. Many quality measures that rely exclusively on process may not require risk adjustment. Second, risk-adjustment measures for neonatal and pediatric intensive care unit (ICU) outcomes are sufficiently well developed to support comparative studies of neonatal and pediatric ICU quality. Their use in comparing ICU outcomes in both settings raises practical and political barriers but few technical barriers. Third, because risk-adjustment techniques could seem to lessen important subpopulation differences in quality, quality measures should be stratified, not adjusted, by important patient characteristics such as race. This observation has implications for the cost of collecting quality data (due to increased sample sizes) as well as reporting of quality data (due to increased complexity).

	STATE OF THE PRACTICE

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
The conclusion of Beal et al, that many quality measures may be available for measuring quality as defined by the IOM framework, leads naturally to the question of whether available measures are being used and, if not, why not? Shaller²⁵ conducted 4 sets of semistructured interviews of key informants and summarized the results in an article commissioned for the meeting. Shaller interviewed 40 opinion leaders from 4 of the groups represented at the meeting: 1) funders of quality-measure development and implementation; 2) developers of quality measures; 3) users of quality measures (eg, purchaser coalitions, Medicaid and SCHIP plan directors, and child health advocacy groups); and 4) the health plans and front-line providers in their roles as subjects and users of quality measures. (As with the invitees, there is some overlap among the groups.) Six major topics were covered (to different degrees) in each interview. A remarkable conclusion to be drawn from Shaller’s interviews is the consistency of responses. All 4 groups expressed a desire for a robust set of standard measures that could serve multiple audiences and objectives including quality improvement, consumer choice, and accountability. The quality measures known to the interviewees overall were CAHPS and the Health Plan Employer Data and Information Set (HEDIS), followed by 2 CAHMI measures and special topic studies to support specific quality-improvement applications (eg, asthma, ADHD, medication errors, lead screening, and prenatal care). Three areas of progress in the use of quality measures for children were mentioned most frequently: 1) growing consensus and collaboration among diverse stakeholder groups; 2) increasing collection and use of specific measures; and 3) early documentation of tangible results. In addition to the HEDIS, CAHPS, and CAHMI measures that were cited as frequently used, the Rand QA Tools System was cited as a success by funders and developers of measures.

A number of issues and challenges remain according to those interviewed by Shaller. Primary among them is the difficulty in generating sustained interest and support in child health quality-measurement development and implementation. According to Shaller’s respondents these difficulties are due to the complexity of quality-measurement issues, competing funding priorities in the face of limited funds, and the lack of clear and compelling evidence that quality measurement and improvement actually result in better outcomes for children. Building public support and making the case for quality measurement and improvement in children’s health care were 2 of the top priorities for future action and funding noted by Shaller’s diverse set of respondents. Other implications drawn from the interviews were the need for greater coordination among funders and developers of measures and improvement initiatives and a need for those concerned about children’s health care quality to integrate themselves into larger, national quality efforts.

	MEETING DISCUSSION AND RECOMMENDATIONS FOR FUTURE ACTION: FOCUS ON IMPLEMENTATION

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
The meeting discussion added depth to the set of observations made in the commissioned articles and added several critical new points. An overwhelming sense of the group was the imperative to implement existing measures. From that flowed a discussion of what actions would be required to foster and promote implementation. A set of recommendations, developed by the meeting cosponsors, was given to the meeting participants, who were asked to score each proposal. The results presented here are the rankings obtained for actions that would result in the greatest improvements in children’s health care quality and were therefore recommended for national action. Table 2 summarizes the top 4 recommended steps.

View this table: [in this window] [in a new window]   TABLE 2. Highest-Scoring Recommendations for National Action to Improve Children’s Health Care Quality

 
Creating the Requisite Information Infrastructure
The highest-priority recommendation for national action was to create the requisite information technology (IT) infrastructure for quality measurement in children’s health care and across all health care sectors. Clearly, IT in children’s health care and health care generally is woefully underdeveloped.³⁵ The need for building an infrastructure became even clearer with the public health emergencies of fall 2001, in particular the anthrax attacks and threats. Without IT, providers, patients, health plans, and others will remain mired in a paper-based system for reporting and improving quality that is outdated, outmoded, extremely inefficient, and even dangerous. Providers and plans are likely to continue to resist the call to measure quality because it is such an extra burden and one that apparently results in few obvious improvements in care or in their lives. Participants recognized that IT alone cannot solve all the problems of quality measurement and improvement. However, constructed with quality in mind, IT could help. Experience has shown, however, that systems that work for children’s health care and quality would not necessarily be a natural byproduct of mainstream IT systems. For example, current computerized physician order entry systems typically are not designed to include weight- and body-mass-based measurements, an essential part of preventing medication errors in children’s health care. Special attention must be paid to the needs of children and their health care.^36,37

Build Public Support for Quality Measurement and Improvement in Children’s Health Care
Health care and quality organizations that are not focused entirely on child issues spoke of the demands placed on them around quality. The marketplace most often requires that quality concerns be combined with concerns about high costs. Therefore, activities to develop quality measures and quality-improvement strategies most often focus on those topics that are of high prevalence or high cost and can result in foreseeable benefits to those who most obviously pay the health care bills. Very often, children’s health care issues are left out of the equation. For example, massive national efforts have focused on measuring and attempting to improve quality for diabetes care.^38,39 Although childhood diabetes is of increasing concern as American children show more signs of obesity, the costs of children’s diabetes care have not been documented; in all likelihood, they would pale in comparison to health care and costs for adults with diabetes.⁴⁰

Meeting participants discussed whether future efforts to build support for children’s health care quality initiatives should focus on the business case per se. This debate is both practical and philosophical. Children’s health care is a relatively small part of overall national health care expenditures, and in any business setting (eg, employer-purchaser, large health plan, and even a state Medicaid program), a comparison of child and adult costs would make adult care issues a higher priority. Many meeting participants expressed concern that improving children’s health care quality not be thought of only as a potential short-term money-saver. Children do represent the future, and they are not just "little adults."⁷ They depend on the adults—employers, health plans, providers, and government officials—who are making health care policy decisions that affect their lives. Participants did feel that it was important to make the case for investing in children’s health care quality. Such a case could take what is actually the standard cost-effectiveness perspective. Such a perspective includes direct and indirect (eg, lost productivity) costs and benefits, over the long term, from a societal perspective.⁴¹ Although examples of costs and benefits of interventions for children’s health have been developed by using the societal perspective,⁴² meeting participants seemed to think that a compelling case for focusing on children’s health care quality had not yet been made. They concluded that making it would be an important ingredient in building public support for future investments in children’s health care quality. In addition to laying out the costs and benefits of investing in children’s health care quality, participants felt that it will be important to effectively frame the message and target it to the appropriate decision-makers.

Improve Reliability, Validity, Feasibility, and Utility of Existing Measures for Children’s Health Care
The third-highest recommendation of the meeting participants was to make improving the reliability, validity, feasibility, and utility of existing measures for children’s health care a national priority. The development of children’s health care quality measures is a challenging enterprise.⁴³ In terms of overall score, this was far below that of the IT and public support recommendations. It does reflect concerns in the group about which measure to use (ie, if there are measures for similar topics, which measure is most reliable and valid?). In addition, it reflects concerns about the feasibility of using particular measures in everyday practice; for example, survey measures have been held to greater scrutiny than measures using readily available administrative data.

Conduct Research to Provide Underlying Scientific Evidence (eg, Effectiveness and Cost-Effectiveness of Services)
This recommendation was rated not far behind the recommendation to improve reliability, validity, etc of the available measures and is related to it. One of the leaders in children’s health care quality improvement, who participated in the meeting, noted how difficult it was to develop quality-improvement strategies and measures to assess their effectiveness in the absence of evidence about the effectiveness of specific preventive, treatment, and management interventions. Five years before the meeting, during the early stages of their development of the Rand QA Tools System, the Rand team identified a similar problem.⁴⁴ They noted that they were much more likely to find good evidence for women’s health care services than for children’s services. Development and implementation of quality measures and improvement strategies on important clinical topics will be stymied without good evidence that what providers and plans are being measured on makes a difference to their patients’ health.

	PROMISING DEVELOPMENTS AND REMAINING CHALLENGES

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
Given the priority actions recommended by the participants at the meeting to promote implementation of quality measures for accountability and improvement, it is interesting to highlight areas in which there are promising developments and others in which little progress seems to be occurring and warrant serious consideration.

Table 3 shows gap areas using primarily the recommendations of meeting participants as a guide and notes promising developments and remaining challenges. In addition, we agree with the conclusion of Beal et al that some new quality measures are needed, and we include development of new quality measures in selected areas as a gap area in Table 3.

View this table: [in this window] [in a new window]   TABLE 3. Selected Gap Areas, Promising Developments, and Remaining Challenges

 
The first gap area is research to provide underlying scientific evidence. Although much progress has been made in so-called outcomes research, much of what is done in clinical practice and in the organization of care is not undergirded by sufficient evidence. Outcomes research is research on the effects of clinical and organizational interventions on patient outcomes. This situation usually becomes evident each time that efforts to synthesize the scientific literature on the effectiveness of children’s health care services are undertaken. For example, evidence syntheses on children’s topics from the AHRQ-supported Evidence-Based Practice Centers typically end with a lengthy and compelling research agenda.⁴⁵ The scope of outcomes research funded by federal agencies does not fully cover the full range of children’s development; the predominant focus has been on neonatal and infant health.⁴⁶

AHRQ has had a longstanding interest in developing the evidence base for clinical services for children^46–48 as well as comparable evidence on organizational management strategies. A challenge for the development of additional evidence is the relative imbalance in this nation’s investment in health services research versus biomedical research. This imbalance is evident when one considers that in 2002, AHRQ’s budget of $300 million was 1.27% of the $23.6 billion National Institutes of Health (NIH) budget or 0.24% of the amount spent annually on health care.⁴⁹ Given other priorities, only 15% of the AHRQ budget has been devoted to children’s studies. NIH focuses primarily on discovery, playing an essential role in understanding the etiology of disease, developing new interventions, and testing their efficacy in rigorous clinical trials. However, NIH-supported clinical trials generally involve only a few, carefully selected participants; results for those participants may not be applicable to the broader child population likely to use the treatments, and the outcomes measured rarely include broader, patient-centered outcomes or cost dimensions. The proposed National Children’s Study, being organized by the National Institute of Child Health and Human Development, the Environmental Protection Agency, and the Centers for Disease Control and Prevention, with participation from other agencies (including AHRQ) may provide an unprecedented opportunity to study what works and what doesn’t in health care for children.⁵⁰ Other research models for enhancing the evidence base for children’s health services include the Children’s Oncology Group, a nationwide multicenter effort to develop and test new interventions for children with cancer.⁵¹ The Children’s Oncology Group is supported by the National Cancer Institute and has been enormously successful in reducing childhood cancer mortality rates.⁵²

The second gap area is to improve the reliability, validity, feasibility, and utility of existing measures for children’s health care. The article by Beal et al was not able to address fully the reliability, validity, feasibility, and utility of existing measures for children’s health care. According to the IOM, scientifically sound measures are not only reliable and valid using standard criteria from measurement science, but the measures should have an underlying evidence base that demonstrates their relationship to positive health care outcomes (if they are process measures), processes of care under the control of the health care system (if they are outcome measures), or both (if they are structural measures).^22,53 Beal et al found that the published literature on children’s quality measures varied in how extensively criteria for scientific soundness were assessed. The development of criteria for developing and assessing measures would help, but there is no obvious source of funds to help measurement developers improve existing measures.

Although the meeting participants did not include new measure development as a priority focus, it is clear from the Beal et al and Shaller articles and other sources that gaps remain despite recent investments. Dental and mental health care are areas in desperate need of good measures because they affect so many children at vulnerable ages. The stakeholders interviewed by Shaller expressed a need for measures for adolescents and on mental health. Beal et al noted the need for more measures for newborns and school-aged children. Examples of where other measures are needed include other IOM domains (equity and efficiency), settings other than hospitals and health plans, and coordination of care for children to reduce problems associated with "hand-offs." A new consumer-focus domain for child well-being and healthy development may be an appropriate addition to the IOM framework; few quality measures have been developed in this area. Risk-adjustment strategies also need to be developed in conjunction with development and testing of new measures.³⁴ Finally, most measures have been developed for purposes of accountability; whether more attention should be paid to developing common measures for improvement purposes is an open question.

The second-highest ranked recommendation of the group was to build public support for efforts to measure and improve children’s health care quality. Several activities hold promise for making progress (Table 4). AHRQ, working with Department of Health and Human Services-wide and public input, is leading the development of 2 new reports to Congress, the National Healthcare Quality Report (NHQR) and the National Healthcare Disparities Report (NHDR).^54,55 These reports will provide an opportunity for national benchmarking on quality measures and eventually the development of benchmarking data at the state level, both of which will support the application of quality measures for children. Although it is unlikely that the NHQR will go into great depths on any one population, the NHDR addresses disparities for all the populations listed in AHRQ’s authorizing legislation, including children.⁵⁶ These annual reports can also serve as a rallying point and common language for stakeholders in child health to monitor progress on improving children’s health care quality.

View this table: [in this window] [in a new window]   TABLE 4. Selected Initiatives in Children’s Healthcare Quality Improvement

 
In addition, stimulated by the February 2002 discussions, a meeting was held early in 2003 to discuss how children’s health care quality issues might be framed, the state of the science for making cases by using business case and societal case paradigms, and leverage points that could be used to bring to scale effective quality-improvement efforts.

The federal government has responded to the need for leadership in quality in many different ways.⁵⁷ In response to the report from the President’s Commission on Consumer Protection and Quality in the Health Care Industry, the QuIC Task Force was established in 1998.¹⁷ This broad initiative has brought together 11 federal agencies including the Department of Health and Human Services, the Department of Defense, and the Veterans Administration to address quality systematically. The goal of the QuIC Task Force is to ensure that all federal agencies that purchase, provide, study, or regulate health care services are working in a coordinated way toward the common goal of improving the quality of care. Substantial progress has been achieved in the areas of patient safety, diabetes, and depression. However, to date, there has been no specific focus on children despite the fact that, together, these federal programs oversee or provide for health care for nearly 25 million children (S. Simone, oral communication, 2002; F. Stewart, MD, written communication, 2002).^58–59 Additional opportunities for leadership in health care quality are reflected in several recent IOM reports.^60–62 Leadership by Example⁶⁰ includes a focus on children, recommending that standardized measures for SCHIP and Medicaid be developed and that those who provide care under those programs report on those measures and, eventually, be paid differentially for higher quality. Perhaps the most promising IOM report from the child perspective is Priority Areas for National Action; of the 20 priority areas for immediate national action recommended by the IOM Committee, 9 are applicable to children’s health care: care coordination, self-management, health literacy, asthma, children with special health care needs, diabetes, immunization, major depression, medication management, and obesity.⁶² IOM recommendations do not always result in national action, however. The IOM is holding a January 2004 summit to develop national strategies for improving care in the 20 priority areas. Thus, it remains to be seen whether (and to what extent) future efforts will address children’s quality challenges. It is likely that the conclusion of the participants at the February 2002 meeting will be relevant for some time to come: we must do a better job of making the case for investing in children’s health care quality!

A fifth gap area is creating the requisite information infrastructure. Promising developments in IT for children’s health include NICHQ’s use of a web-based audit-and-feedback system for providers participating in its learning collaboratives and AAP’s implementation of Education in Quality Improvement for Pediatric Practice. A recent California HealthCare Foundation report illustrates that a variety of ITs can contribute meaningfully to improved health care quality "today."⁶³ In the wake of September 11th, AHRQ-supported evidence reports and web-based tools provide guidance on using IT in the face of public health emergencies.⁶⁴ Because the evidence for some IT approaches in health care is still sparse, AHRQ continues to support research on medical informatics tools for use in health care, including children’s health care.

The most prominent barriers to widespread implementation of IT across the spectrum of patients and health care settings seem to be lack of available capital, lack of outside funding, and lack of commitment and public demand. Concerted public and private action is needed to overcome these barriers. Harnessing the power of private purchasers, the Leapfrog Group has established a set of "leaps" intended to serve as solutions to improve hospital systems that could break down and harm patients. One of these leaps is focused on the purchase and deployment of computerized physician order entry by hospitals.⁶⁵ The public sector role in the development and deployment of IT in health care has expanded with the likely availability of funds to support IT research and implementation in federal fiscal year 2004.⁶⁶ The adoption and promotion of IT standards, including a common data dictionary that would help benchmarking and coordination of disparate efforts, is a key step toward more widespread adoption. Standards sets are currently available for the laboratory and pharmacy sectors, which require endorsement by major payers such as Medicare. Others require further development. Within this overall approach, there is need for attention to the requirements of quality measurement and improvement and the requirements of children’s health care.

	THE FINAL CHALLENGE: IMPLEMENTATION FOR IMPROVEMENT

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
Given the progress detailed above, implementation of quality measures for accountability and improvement should advance over the coming years.

Accountability and improvement can overlap, according to the Strategic Framework Board of the NQF. In one "pathway to improved quality," quality measures used for monitoring and accountability can drive out poor quality providers through nonselection and create demand for actions to improve both the organization and delivery of health care services.⁶⁷ In another pathway to improved quality, key health care actors realize the need for change without being held publicly accountable and voluntarily participate in efforts to change care.

Either path benefits immensely from the availability of scientifically sound, feasible quality measures, but knowledge about performance is only one step toward improving quality. Quality improvement is a complex process that seems to require a careful combination of leadership, evidence-based tools, training, environmental supports, and time.⁶⁸

Several important initiatives in children’s health care quality improvement include the Vermont Oxford Network for neonatal care improvement,⁶⁹ the NICHQ, the Center for Health Care Strategies (CHCS), the Health Resources and Services Administration (HRSA) Health Disparities collaboratives (jointly with the Institute for Healthcare Improvement), the Robert Wood Johnson Foundation programs (CHCS, Pursuing Perfection, and Rewarding Results), and AHRQ-funded research on the science of improvement. The California HealthCare Foundation is also supporting improvement efforts. NICHQ has had learning collaboratives in asthma, ADHD, preventive services, children in foster care, and children with special health care needs.⁷⁰ CHCS’s Best Clinical and Administrative Practices Program has included focus areas including improving birth outcomes, improving preventive care services for children, achieving better care for asthma, improving managed care for children with special needs, enhancing child development services in Medicaid managed care, and improving maternal and newborn health care.⁷¹ HRSA is supporting quality-improvement efforts in its Community Health Centers;⁷² children were included in some of the projects. HRSA’s Maternal and Child Health Bureau is supporting a learning collaborative effort aimed at achieving the "medical home" for children with special health care needs.⁷³ In addition to work with individual providers and Medicaid and SCHIP managed care plans, NICHQ, CHCS, and AHRQ get their messages out to larger numbers of providers via conferences and organized networks.

In its role as a research and quality agency, AHRQ has led the way in supporting development and rigorous testing of various models of clinical and organizational quality improvement⁷⁵ as well as supporting evaluations of others’ quality improvement initiatives.^76,77 Current research efforts include projects to improve care in the neonatal ICU, quality of care for newborns with jaundice, a number of projects aimed at improving children’s asthma care, several on improving adolescent preventive services including one on chlamydia screening, and tests of the impact of providing evidence at the point of ambulatory pediatric care.^74,75 AHRQ-supported research has learned something about what does and doesn’t work to increase delivery of pediatric preventive services.^78–80 Research and evaluation remain critical because the ideal model for quality improvement is not yet known and may vary by setting, population, condition, and other factors.⁸⁰

In addition to supporting the development of new tools to measure quality, developing and disseminating a range of evidence-based tools for quality is a key element of AHRQ’s efforts at facilitating improvement.⁸¹ Three examples are the Evidence Reports from the AHRQ-supported Evidence-Based Practice Centers, the National Guidelines Clearinghouse (NGC), and the National Quality Measures Clearinghouse. AHRQ’s Evidence Reports, syntheses of available high-quality evidence on clinical and other topics, have been used by professional societies such as the AAP to develop clinical practice guidelines on topics such as ADHD and neonatal hyperbilirubinemia.⁸² NGC is a web-based compilation of clinical practice guidelines developed with the participation of the AMA and the Association of American Health Plans.⁸³ The NGC includes almost 500 guidelines that either focus exclusively on or include recommendations for children (J. Slutsky, PA, MSPH, oral communication, 2002). The National Quality Measures Clearinghouse, which became available on the web in winter 2003, will promote widespread access to quality measures by the health care community and other interested individuals.⁸⁴ AHRQ’s Child Health Toolbox, currently on the web, provides information about quality measurement and its purposes as well as information about, and links to, selected available quality measures for children’s health care.⁸⁵ The Child Health Toolbox is geared toward the needs of states.

	WHAT CAN PEDIATRICIANS DO?

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
In the words of Donald Berwick, MD (CEO, Institute for Healthcare Improvement), "We envision a system of care in which those who give care can boast about their work, and those who receive care can feel total trust and confidence in the care they are receiving."⁸⁶ Goode et al⁸⁷ add, "The medical profession has powerful levers to create change. And even modest change can lead to substantial improvement."

Although the chasm in quality of care for children and others needs systems-based solutions,¹² it takes people to change systems. As leaders in children’s health care, pediatricians can play several critical roles in improving quality. We see at least 3: they can commit to improvement in their own practices, participate in the larger quality enterprise, and engage in health services research.

Commit to Improvement
Opportunities to commit to making improvements in the daily practice of care are increasing, and pediatricians are increasingly responding to incentives. For example, continuing medical education is beginning to evolve into continuous professional development (CPD), a much broader view of physician activities that are directed at improving physician performance and ultimately clinical care.⁸⁸ New tools to help support CPD include the AAP’s PediaLink and Education in Quality Improvement for Pediatric Practice.^89,90 More recently, the AAP joined in a partnership with NICHQ and the American Board of Pediatrics to engage in partnerships with 5 state AAP chapters in a systems- and evidence-based collaborative learning to improve ADHD care.⁹¹ These professional partners will work with the national family-based organization Children and Adults with Attention Deficit Disorder to ensure an appropriate child and family focus to improvement efforts. The effort is being supported by AHRQ under its Partnerships for Quality cooperative agreement initiative.⁹²

The AAP’s involvement in the Partnership for Quality initiative reflects its understanding that a commitment to quality improvement requires more than simply adhering to new rules for recertification and continuing medical education/CPD. According to Dr Charles Homer⁹³ in his speech to the AAP meeting in fall 2000, "Committing to improvement requires a perspective of humility—that we as practitioners can be better, that our current modes of work and care are not as good as they might be." Dr Homer suggested that individual practitioners might take 6 steps to begin to improve quality and help restore the public’s confidence in health care: 1) make an individual commitment to making the care one delivers tomorrow better than the care one gave today; 2) learn about improvement methods either by reading the work of seminal leaders in the field of quality improvement (eg, Donald Berwick or Paul Batalden) or attending quality-improvement workshops at national meetings or AAP chapter meetings; 3) examine one’s own practice experience, share the data with partners and staff, and then measure again; 4) involve patients by finding out what they really think about care or what ideas they have to make care better; 5) involve colleagues to generate ideas and maintain momentum; and 6) try, try, and try—improvement requires change. A recent conference on the role and responsibility of physicians to improve patient safety held jointly by AHRQ and the American Board of Internal Medicine Foundation developed 10 recurring themes and opportunities relevant to physician roles in quality improvement in general.⁸⁷ Some of these are already being implemented by the children’s health care community (eg, create awards for individuals, groups, and societies⁹⁴). Although a commitment to improvement may begin with humility and require hard work, Dr Homer and his colleagues have found that providers who do get engaged in quality improvement report that they are experiencing "joy" for the first time in a long time.⁹⁵

Participate in the Larger Quality Effort
Quality of care is on the national agenda, and pediatricians have many unrealized opportunities to be at the tables at which national debates occur. Although several pediatricians are engaged in the CAHMI effort as Executive Committee and Advisory Committee members, the CAHMI includes mostly people very sympathetic to the child perspective in quality. What continues to be in short supply are individuals and organizations that can give voice to children’s issues and concerns in the midst of broader quality debates. One prominent example would be the NQF. By all reports, it took heroic efforts to ensure there was a meaningful discussion of hospital measures for children and children’s hospitals at the NQF. Action by the National Association of Children’s Hospitals and Related Institutions and other child advocates resulted in the Leapfrog Group presenting its hospital survey results differently for children’s hospitals and other hospitals.⁹⁶

Participation in these and other efforts will mean that those who care deeply about children’s health care or their own interests may face searching questions from those with other priorities. Pediatricians could be ready with both compelling data and compelling stories to tell of how improvements in quality have made a difference to them and their patients. In addition, participants should be prepared to work through disagreement and compromise within the child health community. A quality measure that looks attractive to a consumer advocate or ambulatory care physician may not look so attractive to someone in a hospital setting.

It’s been quite some time since children were told to be "seen and not heard."⁹⁷ Perhaps it is time for pediatricians to take to heart the new approach to participation in family life in this important policy area and be themselves "seen and heard." The potential price of anything less is too great: children’s issues will be neither seen nor heard, children’s health care needs will be misunderstood, and quality measures and improvement strategies may be irrelevant or even harmful to children and the providers who serve them.

Engage in Health Services Research
The cornerstone of improved health care quality is evidence-based medicine. To improve the evidence base for children’s health care, pediatricians should strongly consider getting more engaged in research, in particular health services research. Especially with the growth of AHRQ’s primary care practice-based research networks (PBRNs), opportunities for practitioners based outside of academic medical centers are increasing.⁹⁸ In 2001–2002, AHRQ supported 3 pediatric PBRNs. In 2002–2003, 7 PBRN planning grants were made available to pediatric-focused PBRNs. The PBRNs give providers an opportunity to help shape research questions of importance to them and participate in research design, data collection, and publication to inform other practice-based colleagues. Providers who practice in integrated delivery systems or managed care organizations might look into whether they participate in AHRQ’s integrated delivery system research network initiative.^99,100 Good information developed at the practice level can provide the basis for new quality measures and for quality-improvement efforts. Indeed, to foster this very linkage and exchange, AHRQ recently sponsored a meeting, Putting Practice Into Research, to examine strategies to better integrate the research enterprise with the practice environment.¹⁰⁰

	CONCLUSION

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 
Substantial progress has been made in the development of quality measures and the implementation of quality-improvement strategies for children’s health care. However, interest in quality of care for children lags behind that for adult conditions and disorders. Children may be the focus of many well-intentioned statements, but improving their health care and health often takes a back seat to more immediate concerns such as reducing major immediate health care costs. Making significant progress will require not only sustained attention by those concerned about improving children’s health and health care but also specific activities to build a broad base of support among the public and key health care decision-makers.

	FOOTNOTES

 
Received for publication May 13, 2003; Accepted Oct 2, 2003.

Reprint requests to (D.D.) Agency for Healthcare Research and Quality, 540 Gaither Rd, Rockville, MD 20850. E-mail: ddougher{at}ahrq.gov

Dr. Simpson’s present address is ACH Guild Endowed Chair, Child Health Policy, University of South Florida, St Petersburg, Florida. E-mail: las{at}ichp.edu

The views expressed in this article do not necessarily reflect those of the Agency for Healthcare Research and Quality or the US Department of Health and Human Services.

	REFERENCES

TOP ABSTRACT CURRENT CONTEXT MEETING OVERVIEW QUALITY MEASURES FOR CHILDREN'S... RISK ADJUSTMENT STATE OF THE PRACTICE MEETING DISCUSSION AND... PROMISING DEVELOPMENTS AND... THE FINAL CHALLENGE:... WHAT CAN PEDIATRICIANS DO? CONCLUSION REFERENCES

 

1. National Committee on Quality Assurance. The State of Health Care Quality 2002: Industry Trends and Analysis. Washington, DC: National Committee on Quality Assurance; 2002
2. The Commonwealth Fund. Quality of Health Care in the United States: A Chartbook. New York, NY: The Commonwealth Fund; 2002
3. Agency for Healthcare Research and Quality. MEPS Statistical Brief no. 3: Children’s Health Care Quality. Rockville, MD: Agency for Healthcare Research and Quality; 2002
4. Finkelstein JA, Lozano P, Farber H, Miroshnik I, Lieu TA. Underuse of controller medications among Medicaid-insured children with asthma. Arch Pediatr Adolesc Med.2002; 156 :562 –567[Abstract/Free Full Text]
5. The President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry. Quality First: Better Health Care for All Americans. Washington, DC: US Government Printing Office; 1997
6. Halfon N, McGlynn EM, Steinwachs D, Valentine W, Eds. Improving the Quality of Healthcare for Children: An Agenda for Research: A Special Supplement to HSR–Articles from an Invitational Conference. Malden, MA: Health Research Educational Trust by Health Administration Press; 1998
7. Forrest C, Simpson L, Clancy C. Child health services research: challenges and opportunities. JAMA.1997; 277 :1787 –1793[Abstract/Free Full Text]
8. Centers for Medicare and Medicaid Services. Framework for annual report of State Children’s Health Insurance Programs under Title XXI of the Social Security Act. Available at: http://www.cms.gov/schip/chartn99.pdf. Accessed August 29, 2002
9. Foundation for Accountability. Child and Adolescent Health Measurement Initiative. Available at: http://www.facct.org/facct/site/generic/generic/home?action=viewLink&id=2. Accessed August 29, 2002
10. Berman S. Improving children’s health care: an interview with Charles Homer. Jt Comm J Qual Improv.2002; 28 :139 –147[Medline]
11. Institute of Medicine. To Err is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999
12. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001
13. Midwest Business Group on Health. Reducing the Cost of Poor Health Quality Through Responsible Purchasing Leadership. Chicago, IL: Midwest Business Group on Health; 2002
14. California HealthCare Foundation. Consumers and Health Care Quality Information: Need, Availability, Utility. Oakland, CA: California HealthCare Foundation; 2001
15. The Commonwealth Fund. Room for improvement: patients report on the quality of their health care. Available at: http://www.cmwf.org/programs/quality/davis improvement bn 534.asp. Accessed May 15, 2002
16. Kizer KW. Establishing health care performance standards in an era of consumerism. JAMA.2001; 286 :1213 –1217[Abstract/Free Full Text]
17. Quality Interagency Coordination Task Force. Available at: http://www.quic.gov. Accessed August 29, 2002
18. American Medical Association Physician Consortium for Performance Improvement. Introduction to Physician Performance Measurement Sets. Chicago, IL: American Medical Association; 2001
19. Family Voices. What do families say about health care for children with special health care needs. Available at: http://www.familyvoices.org/information/caring.htm. Accessed December 9, 2003
20. Krauss MW, Gulley S, Leiter V, et al. The Family Partners Project: Report on a National Survey of the Health Care Experiences of Families of Children with Special Health Care Needs. Boston, MA: Brandeis University and Family Voices; 2000
21. Beal AC, Co JPT, Dougherty D, et al. Quality measures for children’s health care. Pediatrics.2004; 113 :199 –209[Abstract/Free Full Text]
22. Institute of Medicine Committee on the National Quality Report on Health Care Delivery. Envisioning the National Health Care Quality Report. Washington, DC: National Academy Press; 2001
23. Agency for Healthcare Research and Quality. Child Health Toolbox–CAHPS. Rockville, MD: Agency for Healthcare Research and Quality; 2002
24. Bethell C, Peck C, Schor E. Assessing health system provision of well-child care: the Promoting Healthy Development Survey. Pediatrics.2001; 107 :1084 –1094[Abstract/Free Full Text]
25. Shaller D. Implementing and using quality measures for children’s health care: perspectives on the state of the practice. Pediatrics.2004; 113 :217 –227[Abstract/Free Full Text]
26. Eisenberg JM. Globalize the evidence, localize the decision: evidence-based medicine and international diversity. Health Aff.2002; 21 :166 –168[Abstract/Free Full Text]
27. Barton S. Clinical Evidence Pediatrics. London, UK: BMJ Publishing Group; 2002
28. Moyer VA, Elliott EJ, Davis RL, et al. Evidence Based Pediatrics and Child Health. London, UK: BMJ Books; 2000
29. Sackett DL, Straus SE, Richardson WS, et al. Evidence-Based Medicine: How to Practice and Teach EBM. 2nd Ed. London, UK: Churchill Livingstone; 2000
30. Christakis DA, Davis R, Rivara FP. Pediatric evidence-based medicine: past, present and future. J Pediatr.2000; 136 :383 –389[CrossRef][Web of Science][Medline]
31. McGlynn EA, Damberg CL, Kerr EA, Schuster MA. Quality of Care for Children and Adolescents: A Review of Selected Clinical Conditions and Quality Indicators. Santa Monica, CA: Rand; 2000
32. Kuhlthau K, Walker DK, Perrin JM, et al. Assessing managed care for children with chronic conditions. Health Aff.1998; 17 :42 –52[Abstract]
33. Institute of Medicine Committee on Palliative and End of Life Care for Children and Their Families. When Children Die: Improving Palliative and End-of-Life Care for Children and Their Families. Washington, DC: National Academy Press; 2002
34. Kuhlthau K, Ferris TG, Iezzoni LI. Risk adjustment for pediatric quality indicators. Pediatrics.2004; 113 :210 –216[Abstract/Free Full Text]
35. Bates D. The Quality Argument for Information Technology in Healthcare. BMC Medical Informatics and Decision Making. Available at: www.biomedcentral.com/1472-6947/2/7. Accessed October 23, 2003
36. Weitzman M, Shiffman R. Information technology and the future of child health care: a revolution is occurring. Arch Pediatr Adolesc Med.2001; 155 :990 –991[Free Full Text]
37. Johnson K. Barriers that impede the adoption of pediatric information technology. Arch Pediatr Adolesc Med.2001; 155 :1374 –1379[Abstract/Free Full Text]
38. Texas Medical Foundation. Diabetes quality improvement program. Available at: http://www.tmf.org/hcqip/diabetes.html. Accessed December 9, 2003
39. Fleming BB, Greenfield S, Engelgau MM, Pogach LM, Clauser SB, Parrott MA. The Diabetes Quality Improvement Project: moving science into health policy to gain an edge on the diabetes epidemic. Diabetes Care.2001; 24 :1815 –1820[Free Full Text]
40. Barlow SE, Dietz WH, Klish WJ, Trowbridge FL. Medical evaluation of overweight children and adolescents: reports from pediatricians, pediatric nurse practitioners, and registered dietitians. Pediatrics.2002; 110 :222 –228[Abstract/Free Full Text]
41. Gold MR, Siegel JE, Russell LB, Weinstein MC, Eds. Cost-Effectiveness in Health and Medicine. New York, NY: Oxford University Press; 1996
42. Koplan JP, Schoenbaum SC, Weinstein MC, Fraser DW. Pertussis vaccine—an analysis of benefits, risks and costs. N Engl J Med.1979; 301 :906 –911[Abstract]
43. Palmer RH, Miller MR. Methodologic Challenges in Developing and Implementing Measures of Quality for Child Health Care. Ambul Pediatr.2001; 1 :39 –52[CrossRef][Web of Science][Medline]
44. Schuster MA, Asch SM, McGlynn EA, Kerr EA, Hardy AM, Gifford DS. Development of a quality of care measurement system for children and adolescents. Arch Pediatr Adolesc Med.1997; 151 :1085 –1092[Abstract/Free Full Text]
45. Agency for Healthcare Research and Quality. Evidence-Based Practice. Available at www.ahrq.gov/clinic/epcix.htm. Accessed September 2, 2002
46. US Department of Health and Human Services. Pediatric Outcomes Research in the U.S. Department of Health and Human Services: A Report to Congress. Washington, DC: US Department of Health and Human Services; 1998
47. Dougherty D, Miller M, Lefkowitz D. AHRQ enhances data collection on children’s health care quality. Ambulatory Pediatric Association Newsletter.2001; 38 :23 –24
48. Forrest CB, Shipman S, Dougherty D, Miller M. Outcomes research in pediatric settings: recent trends and future directions. Pediatrics.2003; 111 :171 –178[Abstract/Free Full Text]
49. Cowan CA, McDonnell PA, Levit KR, Zezza MA. Burden of health care costs: businesses, households, and governments, 1987–2000. Health Care Financ Rev.2002; 23 :131 –159.
50. National Institute for Child Health and Human Development, National Institute for Environmental Health Sciences, Centers for Disease Control and Prevention, Environmental Protection Agency. The National Children’s Study. Available at: http://www.nationalchildrensstudy.gov. Accessed August 30, 2002
51. Children’s Oncology Group. Children’s Oncology Group. Available at: http://www.childrensoncologygroup.org. Accessed September 2, 2002
52. Glockler-Ries L. Childhood cancer mortality. Available at: http://seer.cancer.gov/publications/childhood/mortality.pdf. Accessed November 25, 2002
53. Donabedian A. Evaluating the quality of medical care. Milbank Q.1966; 44 :166 –206[Web of Science]
54. Clancy CM. Testimony on AHRQ’s Role in Eliminating Racial and Ethnic Disparities in Health Before the Subcommittee on Criminal Justice, Drug Policy, and Human Resources, Committee on Government Reform United States House of Representatives. Rockville, MD: Agency for Healthcare Research and Quality; 2002
55. Institute of Medicine. Guidance for the National Healthcare Disparities Report. Washington, DC: National Academy Press; 2002
56. Agency for Healthcare Research and Quality. National Healthcare Disparities Report Fact Sheet. Rockville, MD: Agency for Healthcare Research and Quality; 2002
57. Cubanski J, Kline J. Improving Health Care Quality: Can Federal Efforts Lead the Way? New York, NY: The Commonwealth Fund; 2002:7
58. Health Care Financing Administration. A Profile of Medicaid. Baltimore, MD: Health Care Financing Administration; 2000
59. Social Security Administration Office of Policy. Annual Statistical Supplement, 2000. Baltimore, MD: Social Security Administration; 2000:1–4
60. Institute of Medicine. Leadership by Example: Coordinating Government Roles in Improving Health Care Quality. Washington, DC: National Academy Press; 2002
61. Institute of Medicine. Fostering Rapid Advances in Health Care: Learning from Systems Demonstrations. Washington, DC: National Academy Press; 2002
62. Institute of Medicine. Priority Areas for National Action: Transforming Health Care Quality. Washington, DC: National Academy Press; 2003
63. California HealthCare Foundation. Crossing the Chasm With Information Technology: Bridging the Quality Gap in Health Care. Oakland, CA: California HealthCare Foundation; 2002:33
64. Agency for Healthcare Research and Quality. Responding to Bioterrorism: AHRQ Helps Clinicians, Health Systems, and Policymakers—Practical Science-Based Advice Emerging from AHRQ’s Research. Rockville, MD: Agency for Healthcare Research and Quality; 2002
65. The Leapfrog Group. Computer Physician Order Entry (CPOE). Available at: http://www.leapfroggroup.org/consumer_intro2.htm#CPOE. Accessed September 29, 2002
66. Agency for Healthcare Research and Quality and National Institutes of Health, National Library of Medicine. Available at: www.ahrq.gov/fund/grantix.htm. Accessed December 10, 2003
67. The National Quality Forum. A National Framework for Healthcare Quality Measurement and Reporting: A Consensus Report. Washington, DC: National Forum for Health Care Quality Measurement and Reporting; 2002
68. Grol R. Improving the quality of medical care: building bridges among professional pride, payer profit, and patient satisfaction. JAMA.2001; 286 :2578 –2585[Abstract/Free Full Text]
69. Horbar JD, Rogowski J, Plsek PE, et al. Collaborative quality improvement for neonatal intensive care. Pediatrics.2001; 107 :14 –22[Abstract/Free Full Text]
70. National Initiative for Children’s Healthcare Quality. Initiatives. Available at: http://www.nichq.org/initiatives. Accessed September 2, 2002
71. Center for Health Care Strategies. Managed care best practices: best clinical and administrative practices. Available at: http://www.chcs.org/ManagedCare/bcap.html. Accessed September 2, 2002
72. Health Resources and Services Administration Bureau of Primary Health Care. The Quality center. Available at: http://bphc.hrsa.gov/quality/default.htm. Accessed September 2, 2002
73. National Initiative for Children’s Healthcare Quality. Medical home learning collaborative. Available at: http://www.nichq.org. Accessed December 9, 2003
74. Shafer MA, Tebb KP, Pantell RH, et al. Effect of a clinical practice improvement intervention on chlamydial screening in adolescent girls. JAMA.2002; 288 :2846 –2852[Abstract/Free Full Text]
75. Agency for Healthcare Research and Quality. Improving Quality in Child and Adolescent Health—Research and Tools: Fact Sheet. Rockville, MD: Agency for Healthcare Research and Quality; 2002:12
76. Agency for Healthcare Research and Quality. Changing Practices, Changing Lives: Assessing the Impacts of the HRSA Health Disparities Collaboratives. Rockville, MD: Agency for Healthcare Research and Quality; 2002
77. Agency for Healthcare Research and Quality. Evaluations of Demonstrations: Rewarding Results. Rockville, MD: Agency for Healthcare Research and Quality; 2002
78. Bordley WC, Margolis PA, Stuart J, Lannon C, Keyes L. Improving preventive service delivery through office systems. Pediatrics.2001; 108(3) . Available at: http://www.pediatrics.org/cgi/content/full/108/3/e41
79. Hillman A, Ripley K, Goldfarb N, Weiner J, Nuamah I, Lusk E. The use of physician financial incentives and feedback to improve pediatric preventive care in Medicaid managed care. Pediatrics.1999; 104 :931 –935[Abstract/Free Full Text]
80. Ferris TG, Dougherty D, Blumenthal D, Perrin JM. A report card on quality improvement for children’s health care. Pediatrics.2001; 107 :143 –155[Abstract/Free Full Text]
81. Clancy CM, Simpson LA. Looking Forward to Impact: Moving Beyond Serendipity. Health Services Research; 2002
82. American Academy of Pediatrics, Subcommittee on Attention-Deficit/Hyperactivity Disorder and Committee on Quality Improvement. Clinical practice guideline: treatment of the school-aged child with attention-deficit/hyperactivity disorder. Pediatrics.2001; 108 :1033 –1044[Abstract/Free Full Text]
83. Agency for Healthcare Research and Quality. National Guideline Clearinghouse. Available at: www.guideline.gov. Accessed December 9, 2003
84. Agency for Healthcare Research and Quality. National Quality Measures Clearinghouse. Available at: http://www.qualitymeasures.ahrq.gov. Accessed August 3, 2003
85. Agency for Healthcare Research and Quality. Child Health Toolbox. Rockville, MD: Agency for Healthcare Research and Quality; 2002
86. Berwick D. Institute for Healthcare Improvement. Available at: http://www.ihi.org. Accessed September 2, 2002
87. Goode LD, Clancy CM, Kimball HR, Meyer GM, Eisenberg JM. When is "good enough"? The role and responsibility of physicians to improve patient safety. Acad Med.2002; 77 :947 –951[Web of Science][Medline]
88. Sectich TC, Florianai V, Badat MC, Perelman R, Bernstein HH. Continuous professional development: raising the bar for pediatricians. Pediatrics.2002; 110 :152 –156[Free Full Text]
89. American Academy of Pediatrics. PediaLink CME. Available at: http://www.pedialink.org/pedialink/index.cfm. Accessed September 2, 2002
90. American Academy of Pediatrics. Education in quality improvement for pediatric practice. Available at: http://www.eqipp.org. Accessed September 2, 2002
91. Agency for Healthcare Research and Quality. Research on Child and Adolescent Health New Starts—Fiscal Year 2002. Rockville, MD: Agency for Healthcare Research and Quality; 2003
92. Agency for Healthcare Research and Quality. Partnerships for Quality RFA. Rockville, MD: Agency for Healthcare Research and Quality; 2002
93. Homer CJ. Quality in pediatrics: a progress report for the American Academy of Pediatrics. Pediatrics.2001; 108 :1362 –1364[Free Full Text]
94. American Academy of Pediatrics. Quality improvement and management. Available at: http://www.aap.org/visit/qualityimp.htm. Accessed January 6, 2003
95. Shaw J. NICHQ Update. Paper presented at: Child and Adolescent Health Measurement Advisory Committee Meeting; May 30–31, 2002; Elk Grove Village, IL
96. The Leapfrog Group. Leapfrog Group Hospital Survey Results Site. Available at: http://www.healthgrades.com/leapfrog/index.cfm?fuseaction+mod&modtype=LeapFrogSurvey&modact=LeapFrogSurvey search. Accessed September 2, 2002
97. Hulbert A. The century of the child. Wilson Q.1999; 23 :14 –29
98. Agency for Healthcare Research and Quality. Fact Sheet: Primary Care Practice-Based Research Networks. Rockville, MD: Agency for Healthcare Research and Quality; 2001
99. Agency for Healthcare Research and Quality. Fact Sheet: Integrated Delivery System Research Network (IDSRN)—Field Partnerships to Conduct and Use Research. Rockville, MD: Agency for Healthcare Research and Quality; 2002
100. Fraser I, Lanier D, Hellinger F, Eisenberg JM. Putting practice into research. Health Serv Res.2002; 37 :xiii –xxvi[Web of Science][Medline]
101. Kaplan-Sanoff M. Healthy steps: delivering developmental services for young children through pediatric primary care. Infants Young Child.2001; 13 :69 –76
102. Minkovitz C, Strobino D, Hughart N, Scharfstein D, Guyer B. Healthy Steps Evaluation Team. Early effects of the Healthy Steps for Young Children program. Arch Pediatr Adolesc Med.2001; 155 :470 –479[Abstract/Free Full Text]
103. Pelletier H, Abrams M. The North Carolina ABCD Project: a new approach for providing developmental services in primary care practice. Available at: http://www.cmwf.org/programs/child/pelletier_nashpabcd.pdf. Accessed October 7, 2002
104. Gausche M, Lewis RJ, Stratton SJ, et al. Effect of out-of-hospital pediatric endotracheal intubation on survival and neurological outcome: a controlled clinical trial. JAMA.2000; 283 :783 –790[Abstract/Free Full Text]
105. Cifuentes J, Bronstein J, Phibbs CS, Phibbs RH, Schmitt SK, Carlo WA. Mortality in low birth weight infants according to level of neonatal care at hospital of birth. Pediatrics.2002; 109 :745 –751[Abstract/Free Full Text]
106. Tilford JM, Simpson PM, Green JW, Lensing S, Fiser DH. Volume-outcome relationships in pediatric intensive care units. Pediatrics.2000; 106 :289 –294[Abstract/Free Full Text]
107. Kradjan WA, Schulz R, Christensen DB, et al. Patients’ perceived benefit from and satisfaction with asthma-related pharmacy services. J Am Pharm Assoc.1999; 39 :658 –666[Medline]
108. Miller MR, Elixhauser A, Zhan C, Meyer GM. Patient safety indicators: using administrative data to identify potential patient safety concerns. Health Serv Res.2001; 36 :110 –132[Medline]
109. Miller MR, Elixhauser A, Zahn C. Patient safety events during pediatric hospitalizations. Pediatrics.2003; 111 :1358 –1366[Abstract/Free Full Text]
110. National Quality Forum. A National Framework for Healthcare Quality Measurement and Reporting: A Consensus Report. Washington, DC: The National Forum for HealthCare Quality Measurement and Reporting; 2002
111. Agency for Healthcare Research and Quality. AHRQ Quality Indicators. Rockville, MD: Agency for Healthcare Research and Quality; 2003
112. Hermann RC, Palmer RH. Common ground: a framework for selecting core quality measures for mental health and substance abuse care. Psychiatr Serv.2002; 53 :281 –287[Abstract/Free Full Text]
113. Center for Quality Assessment and Improvement in Mental Health. Available at: www.cqaimh.org/quality.html. Accessed December 10, 2003
114. Bader JD, Shugars DA, White BA, Rindal DB. Development of effectiveness of care and use of services measures for dental care plans. J Public Health Dent.1999; 59 :142 –149[Web of Science][Medline]
115. Crall JJ. Oral health component of child health services research. J Dent Educ.1997; 61 :776 –780[Abstract]
116. Mouradian WE. Building and involving constituencies with the Surgeon General’s Workshop and Conference. J Dent Res.2001; 80 :1873 –1874[Free Full Text]
117. Olin SC, Hoagwood K. The Surgeon General’s National Action Agenda on Children’s Mental Health. Curr Psychiatry Rep.2002; 4 :101 –107[Medline]
118. Surgeon General of the United States. Oral health in America: a report of the Surgeon General. Available at: http://www.surgeongeneral.gov/library/oralhealth. Accessed August 30, 2002
119. Forrest CB, Glade GB, Baker AE, Bocian A, von Schrader S, Starfield B. Coordination of specialty referrals and physician satisfaction with referral care. Arch Pediatr Adolesc Med.2000; 154 :499 –506[Abstract/Free Full Text]
120. Glade GB, Forrest CB, Starfield B, Baker AE, Bocian AB, Wasserman RC. Specialty referrals made during telephone conversations with parents: a study from the pediatric research in office settings network. Ambul Pediatr.2002; 2 :93 –98[CrossRef][Web of Science][Medline]
121. Watson MR, Manski RJ, Macek MD. The impact of income on children’s and adolescent’s preventive dental visits. J Am Dent Assoc.2001; 132 :1580 –1587[Abstract/Free Full Text]
122. Agency for Healthcare Research and Quality. Fact Sheet: National Health Care Quality Report—Update. Agency for Healthcare Research and Quality; 2002
123. Patient Safety Task Force. Patient Safety Task Force fact sheet. Available at: http://www.ahrq.gov/qual/taskforce/psfactst.htm. Accessed November 11, 2002
124. Center for Health Care Strategies. Managed Care Solutions Forum. Available at: http://www.chcs.org/InformedPurchasing/mcsf.html. Accessed September 2, 2002
125. Child and Adolescent Health Measurement Initiative. CAHMI state learning networks (CSLN). Available at: http://www.facct.org/facct/site/CAHMI/CAHMI/home?action=viewToolkit&id=27. Accessed November 11, 2002
126. Blendon RJ, Young JT, McCormick MC, Kropf M, Blair J. Americans’ views on children’s health. JAMA.1998; 280 :2122 –2127[Free Full Text]
127. Kaiser Family Foundation and Agency for Healthcare Research and Quality. Americans as Health Care Consumers: The Role of Quality Information—Highlights of a National Survey. Rockville, MD: Agency for Healthcare Research and Quality; 2002
128. National Health Care Purchasing Institute. Rewarding results: aligning incentives with high quality health care. Available at: http://www.leapfroggroup.org/RewardingResults. Accessed December 10, 2003
129. Institute for Healthcare Improvement. Pursuing perfection: raising the bar for health care performance. Available at: http://www.ihi.org/pursuingperfection. Accessed September 2, 2002

---

PEDIATRICS (ISSN 1098-4275). ©2004 by the American Academy of Pediatrics

CiteULike    Connotea    Del.icio.us    Digg    Facebook    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				S. A. Spooner and D. C. Classen Data Standards and Improvement of Quality and Safety in Child Health Care Pediatrics, January 1, 2009; 123(Supplement_2): S74 - S79. [Abstract] [Full Text] [PDF]

				K. M. McDonald, S. M. Davies, C. A. Haberland, J. J. Geppert, A. Ku, and P. S. Romano Preliminary Assessment of Pediatric Health Care Quality and Patient Safety in the United States Using Readily Available Administrative Data Pediatrics, August 1, 2008; 122(2): e416 - e425. [Abstract] [Full Text] [PDF]

				A. G. Fiks, R. W. Grundmeier, L. M. Biggs, A. R. Localio, and E. A. Alessandrini Impact of Clinical Alerts Within an Electronic Health Record on Routine Childhood Immunization in an Urban Pediatric Population Pediatrics, October 1, 2007; 120(4): 707 - 714. [Abstract] [Full Text] [PDF]

				L. Simpson, D. Dougherty, D. Krause, C. M. Ku, and J. M. Perrin Measuring Children's Health Care Quality American Journal of Medical Quality, March 1, 2007; 22(2): 80 - 84. [PDF]

This Article Abstract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services E-mail this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My File Cabinet Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via CrossRef Citing Articles via Google Scholar Google Scholar Articles by Dougherty, D. Articles by Simpson, L. A. Search for Related Content PubMed PubMed Citation Articles by Dougherty, D. Articles by Simpson, L. A. Related Collections Miscellaneous Social Bookmarking                         What's this?