SUPPLEMENT ARTICLE |
Quality Measures for Children’s Health Care
* Center for Child and Adolescent Health Policy, Harvard Medical School, Massachusetts General Hospital, Boston, Massachusetts
Office of Priority Populations Research, Agency for Healthcare Research and Quality, Rockville, Maryland
Center for Quality of Care Research and Education, Harvard School of Public Health, Boston, Massachusetts
 |
ABSTRACT |
|---|
 TOP ABSTRACT CHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
Background. The ability to measure and improve the quality of children’s health care is of national importance. Despite the existence of numerous health care quality measures, the collective ability of measures to assess children’s health care quality is unclear. A review of existing health care quality measures for children is timely for both assessing the current state of quality measures for children and identifying areas requiring additional research and development.
Objectives. To identify and collect current health care quality measures for child health and then to systematically categorize and classify measures and identify gaps in child health care quality measures requiring additional development.
Design/Methods. We first identified child health care quality instruments with assistance from staff at the Agency for Healthcare Research and Quality, experts in the field, the Computerized Needs-oriented Quality Measurement Evaluation System, the Child and Adolescent Health Measurement Initiative, and a medical literature review. From these instruments, we then selected clinical performance measures applicable to children (aged 0–18 years). We categorized the individual measures into the Institute of Medicine’s framework for the National Health Care Quality Report. The framework includes health care quality domains (patient safety, effectiveness, patient-centeredness, and timeliness) and patient-perspective domains (staying healthy, getting better, living with illness, and end-of-life care). We then determined the balance of the measures (how well they assess care for all children versus children with special health care needs) and their comprehensiveness (how well the measures apply to the developmental range of children). Finally, we analyzed the ability of the measures to assess equity in care.
Results. We identified 19 measure sets, and 396 individual measures were used to assess children’s health care quality. The distribution of measures in the health care quality domains was: safety, 14.4%; effectiveness, 59.1%; patient-centeredness, 32.1%; and timeliness, 33.3%. The distribution of measures in the patient-perspective domains was: staying healthy, 24%; getting better, 40.2%; living with illness, 17.4%; end of life, 0%; and multidimensional, 23.5% (measures were multidimensional if they applied to >1 domain). Most of the measures were meant for use in the general pediatric population (81.1%), with a significant proportion designed for children with special health care needs (18.9%). The majority (
79%) of the measures could be applied to children across all age groups. However, there were relatively few measures designed specifically for each developmental stage. Regarding the use of measures to study equity in health care, 6 of the measure sets have been used in previous studies of equity. All the survey measure sets contain items that identify patients at risk for poor outcomes, and 4 are available in languages other than English. However, only 1 survey (Consumer Assessment of Health Plans) has undergone studies of cross-cultural validation. Among the measure sets based on administrative data, 3 included infant mortality, a well-known measure of health disparity.
Conclusions. There are several instruments designed to measure health care quality for children. Despite this, we found relatively few measures for assessing patient safety and living with illness and none for end-of-life care. Few measures are designed for specific age categories among children. Although equity is an overarching concern in health care quality, the application of current measures to assess disparities has been limited. These areas need additional research and development for a more complete assessment of health care quality for children.
Key Words: quality of care • child health • health care quality • Institute of Medicine • quality framework
Abbreviations: AHRQ, Agency for Healthcare Research and Quality • IOM, Institute of Medicine • SCHIP, State Children’s Health Insurance Program • CAHMI, Child and Adolescent Health Measurement Initiative • FACCT, Foundation for Accountability • CSHCN, children with special health care needs • PHDS, Promoting Healthy Development Survey • YAHCS, Young Adults Health Care Survey • CAHPS, Consumer Assessment of Health Plans
The increase in quality measures can partly be traced to the President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry. Another impetus was passage of Title IX of the Healthcare Research and Quality Act of 1999. This requires the Agency for Healthcare Research and Quality (AHRQ) to issue an annual public report on health care quality in the United States beginning in 2003.1 In preparation, AHRQ funded the Institute of Medicine (IOM) to report recommendations on selecting measures for this annual report. The report will include a report on child health care quality; however, AHRQ and others have long understood that measures of health care quality for children are not as well developed as those for adults.2–6
During the 1990s, observers remarked on the paucity of valid and reliable quality measures for children’s health care.7 In response, federal and private-sector funders have invested millions of dollars in the development, testing, and implementation of quality measures for children’s health care. Several measures have been or are being developed, and others are being used in practice. However, there has been no comprehensive look at the scope of measures in terms of the categories recommended by the IOM and the extent to which these measures have been or are likely to be useful to those responsible for assessing and improving the quality of health care for children.
|
CHALLENGES OF HEALTH CARE QUALITY ASSESSMENT FOR CHILDREN |
|---|
|
TOPABSTRACTCHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
The major challenges to assessing health care quality for children relate to their unique needs as a patient population, recent changes in health care systems for children, and the lack of pediatric focus in current health care quality measures.
Children differ from adults in many ways that affect our ability to evaluate the quality of their health care. Several authors have identified 4 specific areas of challenge to conducting child health services research, including development of health care quality measures: 1) development, 2) dependency, 3) different epidemiology, and 4) demographics.8–10
- Development: Childhood is marked by rapid developmental changes that affect functioning, cognition, and health care needs. As children grow and develop, their health care utilization changes as well as the preventive care they require.11 Quality measures based on child functioning are challenging to construct, because child functioning generally improves with age.
- Dependency: Children depend on their caregivers for access to the medical system and health-related care. Consequently, researchers often depend on caregivers to provide information regarding children’s health outcomes and experiences with care. Compared with studies of adult self-report, less work has been done to determine the validity of parental report of processes of care compared with other methods of data collection.12
- Different Epidemiology: The majority of children do not suffer from chronic conditions or disabilities, and their interactions with the health care system are focused on prevention and treatment for acute illnesses. Health care quality outcomes based on wellness present a methodological challenge to researchers because of the need to measure absence of disease, which is the normal state for most children. Also, the low rates of children with serious medical conditions pose significant methodological challenges for obtaining valid results.
- Demographics: Children are more likely to live in poverty than any other segment of the population. In addition to the effects on access to health and health outcomes, living in poverty increases the likelihood of using publicly funded sources of health insurance (ie, Medicaid and the State Children’s Health Insurance Program [SCHIP]). As a result, public policy changes regarding financing of health care will have a large impact on children. The increased racial/ethnic diversity observed in the United States is well represented among children. Racial disparities in health care have been well documented, and this increased diversity may affect the quality of care given to children.13
|
CHALLENGES OF QUALITY HEALTH CARE FOR CHILDREN |
|---|
|
TOPABSTRACTCHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
As stated above, children are more likely to live in poverty than other segments of the American population, resulting in a greater dependency on public sources of health insurance (ie, Medicaid and SCHIP). These programs have adopted managed care to contain costs, and children currently represent more than half of the Medicaid managed care patients.14 As these programs implement managed care policies to contain costs, there is increased concern about the effects of these changes on health care quality for children.15–18 In addition, chronic conditions, notably asthma, are increasing in prevalence, particularly among low-income children. The convergence of increased medical need with increased enrollment in publicly financed health care further raises concerns about the quality of health care for children.19,20 Other changes in health policy limiting hospital stays for newborns have also raised concerns about the quality of care and outcomes for newborns.21,22
A review of the current state of health care quality measures for children is timely for several reasons. In addition to Title IX of the Healthcare Research and Quality Act that requires AHRQ to issue an annual public report on health care quality in the United States, it is almost 5 years since the landmark conference, Improving Quality of Health Care for Children.4 A number of subsequent developments show that quality of care for Americans, including children, is of growing concern. These developments include creation of the National Quality Forum,23 requirements for quality assessments in Medicaid managed care and the SCHIP programs, formation of the Child and Adolescent Health Measurement Initiative (CAHMI), a national collaboration to develop a core set of child health care quality measures,24 the National Initiative for Children’s Healthcare Quality, a program to improve child health care quality at the practice level,25 large private purchaser initiatives in quality (the Leapfrog Group),26 and a Congressionally mandated study of federal quality assessment and improvement programs.1
The purpose of this project was to collect and review health care quality measures for children, systematically categorize and classify measures, assess the current status of health care quality measures for children, and identify areas in need of additional research, development, and funding.
|
PROJECT OVERVIEW |
|---|
|
TOPABSTRACTCHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
We used the framework in the report commissioned by AHRQ from the IOM, Envisioning the National Health Care Quality Report,1 to categorize health care quality measures. We classified measures according to the IOM quality and patient-perspective domains outlined in the IOM framework. In addition, we assessed the balance and comprehensiveness of the measures as a set. The IOM proposes equity as an overarching dimension across the framework; for the purposes of this project, we evaluated the application of the measures to analyses of equity. Last, using results from these analyses, we identified areas of health care quality measurement that are well developed for pediatric health as well as those that require additional development. A goal of this article is to provide information to identify priority areas for future research in the development of health care quality measures for children.¶
|
IOM FRAMEWORK FOR THE NATIONAL HEALTH CARE QUALITY REPORT |
|---|
|
TOPABSTRACTCHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
The IOM report Envisioning the National Health Care Quality Report1 is the most extensive description of quality-measurement criteria to date, although it does not focus on specific populations such as children. We will describe the IOM framework briefly here; for a detailed description of the framework, we recommend reading the full monograph.27
Domains in Quality Framework
The first dimension in the framework outlines areas of health care quality. The IOM committee chose to define the areas for improving quality by using 4 domains: 1) safety, 2) effectiveness, 3) patient-centeredness, and 4) timeliness.
- Safety refers to patient safety when interacting with the health care system. It specifically includes safety related to diagnosis of conditions (missed diagnoses and incorrect diagnoses), treatment errors, and safety from injury in health care settings.
- Effectiveness refers to the provision of appropriate health services based on a scientific understanding of the risk and benefits associated with care. It includes measures of overuse and underuse and can be applied across the spectrum of care, (ie, preventive, acute, and chronic care and care for specific conditions).
- Patient-centeredness refers to patients’ experiences of care and the presence of an effective partnership among patients, providers, and their families. These interactions allow for medical decisions that respect patients’ views and provide them with education to make decisions and participate in their own care.
- Timeliness refers to minimizing delays in accessing the medical system, delays within the medical system, and delays in coordination between different components of the medical system.
The second major dimension in the quality framework is the patient (or consumer) perspective of health care, drawn from a consumer quality information framework set forth by the Foundation for Accountability (FACCT).28 Patients turn to the health care system for essentially 4 reasons: 1) to stay healthy, 2) to get better when ill or injured, 3) to manage an ongoing or chronic condition, and 4) for care at the end of life.
Staying healthy refers to preventive care. It includes screening procedures, counseling to avoid health risk behaviors, and interventions to prevent illness. Getting better refers to recovering from an acute illness or injury. It includes acute care for limited illnesses and follow-up of an episode of care. Living with an illness or disability refers to management of an ongoing condition that affects health or functioning. It includes management of chronic conditions and efforts to prevent exacerbations of the condition. End-of-life care refers to management of the patients’ and families’ needs related to a terminal illness. Because children are a healthy population, we did not anticipate many measures within this domain.
The IOM framework proposes the classification of measures in a matrix with these 2 dimensions. As an example, measures could address the safety, effectiveness, patient-centeredness, or timeliness of care designed to keep patients healthy.
Balance and Comprehensiveness
The IOM proposes that the set of measures for the National Health Care Quality Report should provide balance and comprehensiveness. Balance and comprehensiveness refer to the scope of measures within the framework. Although some measures may focus on patient-centeredness, others will focus on effectiveness. Taken in total, they should assess all aspects of health care quality and provide a balanced assessment of care that includes both positive and negative aspects of care.
Balance and comprehensiveness also refer to a wide range of patients and clinical settings. They can refer to patients over the entire life cycle, those who are well, those with specific conditions, patients who are hospitalized, those in chronic care facilities, or a geographic range of patients. Some measures will apply to all children (eg, vaccination status), whereas others will apply to subgroups (eg, care for asthma or after hospitalization). Both types of measures are important for contributing to the overall assessment of health care quality.
Last, an overarching concern regarding health care quality is equity. The IOM has proposed that the National Health Care Quality Report should include analyses of the equitable quality of care, which can include comparative assessments of quality for patients by race, income, insurance type, or geographic location.
|
METHODS |
|---|
|
TOPABSTRACTCHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
We began by identifying health care quality-measure sets. We then established criteria to define each of the IOM health care quality components and criteria for evaluating measures. Each measure set then was analyzed by using the analytic plan we developed. We assessed the entire set of health care quality-measure sets for strengths and weaknesses in determining the health care quality of children.
Identification of Health Care Quality-Measure Sets#
Staff at AHRQ generated an initial list of health care quality-measure sets for inclusion in the study. Because many well-studied sets of measures have not been published yet, we then contacted national experts in the field (including measure developers and funders) to help identify additional measures that are well known, widely used, or well developed. Last, we conducted a 10-year review of the medical literature using the keywords "quality," "quality of health care," "quality indicators," and "quality assurance" limited to children aged 0 to 18 years. From the identified measure sets, we included health care quality measures that met our definition of clinical performance measures in the analyses. Many of the measure sets contained measures that applied to adult populations. We limited our analyses to measures that applied to patients aged 0 to 18 years.
Computerized Needs-Oriented Quality Measurement Evaluation System Measures
The Computerized Needs-Oriented Quality Measurement Evaluation System (CONQUEST) is a tool to evaluate health care quality measures in a systematic fashion.29 It allows for identification of measures used in specific patient populations. In this project we used age as an identifying characteristic for a patient population and extracted all measures applicable to patients <18 years old. Using this method, we identified additional measures for inclusion in this project.
CAHMI Measures
We identified measures developed by CAHMI,24 which is a national collaboration established in 1998 to identify priorities for developing new quality measures and to oversee development of new measures. We included all child health measures developed by CAHMI.
Definition of a Health Care Quality Measure
There are several types of measures promoted to assess the quality of care; however, we chose to focus our analyses on clinical performance measures. Clinical performance measures are defined in CONQUEST as:
"tools that assess the delivery of clinical services.... [They] estimate the extent to which a health care provider delivers clinical services that are appropriate for the patient’s condition; provides the clinical services safely, competently, and in the appropriate time frame; and achieves the desired outcomes in terms of those aspects of patient health and satisfaction that can be affected by clinical services."29
These measures can address processes, outcomes, utilization, access, patient satisfaction, and patient experiences with care. Valid and reliable clinical performance measures can be used for quality-improvement efforts.30 Although the Donabedian framework for evaluating quality includes assessments of structures, processes, and outcomes,31 we did not include structural measures as recommended by the IOM.1 There is less evidence of an association between structure and outcomes, patients are more interested in process and outcomes, and few structural elements are pediatric-specific.
Measures consisted of individual items that were clinical performance measures as well as composite measures made up of several items. For example, chlamydia screening is a stand-alone clinical performance measure, whereas a patient-satisfaction measure could be based on several items such as the patients’ perception of physician respect, politeness of the staff, and time spent waiting to see the physician. In the case of composite measures, we included only the single measure and did not conduct counts on the individual items used to make up the measure. Most of the composite measures came from survey instruments.
Classification of Measures Into IOM Framework
Our criteria for operationalizing the domains of health care quality framework were based on 2 IOM reports: Envisioning the National Health Care Quality Report1 and Crossing the Quality Chasm.32 These reports provided a conceptual framework for defining the health care quality and consumer-perspective domains. Because classification of existing pediatric measures has not been undertaken by using the IOM framework, we reviewed the definitions of each domain, defined the criteria for their operationalization, and tested our methods with measures selected from various quality-measurement sets. The criteria were reviewed by all members of the research team and modified based on collective feedback.
In the process of conducting the analyses, we found that some measures did not fall neatly into the classification system. As examples, being assigned a primary care provider and physician coordination of care can be applied to each consumer-perspective domain: staying healthy, getting better, living with illness, and end-of-life care. As a result, we added a nondimensional domain for measures that could be applied to all domains.
When analyzing measures of care related to chronic conditions with intermittent exacerbations (eg, asthma or depression), we categorized quality measures for the exacerbation both within the "getting-better" and "living-with-illness" patient-perspective domains.
We conducted a measure-by-measure analysis. Note that the categorizations are not mutually exclusive. Measures could be categorized into more than one domain of quality and the consumer perspective (see Fig 1). As an example, the Health Plan Employer Data and Information Set measure of timely follow-up after mental health hospitalization is a measure of both effectiveness and timeliness within both the getting-better and living-with-illness domains. We did not conduct analyses to provide weights or attribute importance to these categorizations. That would require a review of the evidence for the impact of each "cell" on outcomes such as costs, morbidity, mortality, or quality of life. Those analyses were beyond the scope of the current project.
|
Balance and Comprehensiveness
For the purposes of this project, we evaluated balance and comprehensiveness by determining whether measures assess care for all children versus care for children with special health care needs (CSHCN) and whether measures cover the developmental range of pediatric health care (see Fig 2). We used the developmental range of childhood because, clinically, the needs of children vary significantly from infancy to young adulthood. We defined the developmental stages of pediatrics as infancy to preschool (0–5 years), childhood (5–12 years), and adolescence (12–18 years).
|
We focused on the balance of measures for general care, which represents the vast majority of care given to all children, with measures of care for CSHCN. This subgroup of pediatric patients is at greatest risk for poor outcomes related to poor quality care and is the greatest utilizer of health care. In addition, their developmental and growth needs challenge caretakers to coordinate their care across settings that range from subspecialists to school-based care. Some of the measure sets were designed for use in CSHCN, and those measures were categorized as belonging to CSHCN. Other measure sets include screening questions for the identification of CSHCN. Although the measures themselves reflect general care (eg, has your doctor talked to you about how your child is feeling, growing, or behaving?), if they applied to CSHCN, they were categorized as belonging to the CSHCN.
Equity
Equity is an overarching dimension across the framework. We included an analysis of the capacity of health care quality-measure sets to compare quality between different populations. For each measure set we determined whether it had ever been used in comparative analyses to assess equity and listed all variables within the measure sets that could be used for comparative analyses of equity. This list of variables included primarily demographic measures such as gender, race, income, health status, and insurance status. We also reviewed the background materials and the medical literature to determine whether the survey was available in languages other than English and whether developers had conducted tests of validity of the measures across different populations. Such tests were only applicable to the survey measure sets. Administrative data often do not include race/ethnicity, which limits the capacity of administrative data to be used to study racial/ethnic disparities in care. However, we determined whether they included measures of infant mortality. This measure was chosen because it is one of the Healthy People 2010 priority areas for reductions in racial disparities in health care (see Fig 3).
|
The research assistants on our team (T.J. and J.K.) conducted the initial analyses. The results then were independently reviewed by 2 of the physician researchers on the team (A.C.B. and J.P.T.C.). When there was disagreement regarding the categorization of a measure, it was reviewed and discussed by the team. By using the criteria for operationalization, the measure was discussed until there was consensus regarding classification.
|
RESULTS |
|---|
|
TOPABSTRACTCHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
We identified 19 health care quality-measure sets containing 396 quality measures for children (Table 1). Among the 19 sets, 10 were developed specifically for children, and 9 contain measures for both adult and child health. Ten of the measure sets use administrative or medical record data, and 9 use survey data. We also identified several measure sets in development that we did not include in our analyses. However, we included a list of those measure sets to provide an overview of emerging child health care quality measures (Table 2).
|
|
Health Care Quality Domains
After categorizing the measures within the quality domains, we found that most (59.1%) measures fell within the effectiveness domain (see Table 3). As expected, there were very few (14.4%) measures within the safety domain. Most of the safety measures focused on inpatient care with the exception of those in the Rand QA Tool, which contained outpatient measures such as review of drug interactions. A moderate proportion of measures were classified as relating to patient-centeredness (32.1%) and timeliness (33.3%).
|
Patient-Perspective Domains
The largest proportion of measures (40.2%) fell within the getting-better domain (see Table 4). The next most common domains were staying healthy (24%) and nondimensional (23.5%). There were fewer measures in the living-with-illness category (17.4%). Some measures were designed for use in CSHCN, and they made up a large portion of the measures for this group of patients. As expected, there were no quality measures in the end-of-life domain.
|
Balance and Comprehensiveness
The majority (
79%) of the measures could be applied to children across all age groups (see Table 5). Although the measures were well represented across the age spectrum, there were fewer measures designed specifically for each age category; 10.9% for infants, none for children, and 8.8% for adolescents (see Fig 4). The measures that were specifically for the infant/toddler age group included infant mortality and measures from the Promoting Healthy Development Survey (PHDS). Similarly, measures exclusively for adolescents were seen only in the Rand QA Tool and the Young Adults Health Care Survey (YAHCS). School-aged children had no measures unique to their age group. With regard to health status, most of the measures were designed for use in the general pediatric population (81.1%), with a significant proportion (18.9%) designed for CSHCN (see Table 5).
|
|
Equity
Six of the measure sets had been used previously to compare outcomes among populations at risk (eg, minorities or Medicaid recipients). All 9 of the survey measure sets contained items to identify patients at risk for poor outcomes, and 4 were available in other languages. However, only one survey [Consumer Assessment of Health Plans (CAHPS)] had undergone studies of cross-cultural validation. Among the 10 measure sets using administrative or medical record data, 3 contained measures of infant mortality (see Fig 5).
|
|
DISCUSSION |
|---|
|
TOPABSTRACTCHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
Using the IOM national health care quality report framework as a template, we evaluated the current health care quality measures for children. Based on our review, we made the following observations regarding the set of quality measures for child health.
Categorization of the measures in the health care quality domains shows that there were relatively few items in the safety domain. Safety measures currently focus on serious errors in health care delivery with an emphasis on medical and surgical error in inpatient settings. Most measures fell within the effectiveness domain, with a moderate number categorized in the patient-centeredness and timeliness domains.
As expected, in the patient-perspective domains there were no measures in the end-of-life category. CSHCN represent those who are living with illness and make up between 10% and 20% of the pediatric population. At the same time, 
18% of the measures fell within the living-with-illness domain. Thus, the proportion of measures in that domain mirrors the proportion of children who are living with illness. A good amount of the measures were categorized in the staying-healthy and getting-better domains; however, nearly one fourth of all domains were classified as nondimensional.
The measure sets contained several measures that applied to children of all ages, but very few focused on specific age groups to provide a more detailed assessment of their care. When there were measures specifically for infants, they most commonly were related to low birth weight and infant mortality. The PHDS was the only measure set specifically designed for infants and preschoolers. The Rand QA Tool included some measures specifically for infants, preschoolers, and adolescents. The YAHCS was the only measure set designed specifically for adolescents. There were no measure sets designed specifically for use in school-aged children.
The second component of balance and comprehensiveness focused on the inclusion of measures for CSHCN. Nearly 19% of the measures were meant for use in this group. We found that the categorization in the living-with-illness domain overlapped significantly with this part of the balance-and-comprehensiveness analysis. We concluded that little additional information was gleaned from including this analysis in addition to the health care quality analyses.
Only one third of the measure sets had been previously used in studies of equity. However, all 9 of the surveys contain items that allow investigators to identify patients at risk for poor outcomes. The lack of previous work on equity relates more to the application of measures than to their capacity for use in such studies. Among the 9 surveys, only 4 were available in languages other than English, and 1 had undergone cross-cultural validation. With regard to the 10 measure sets based on administrative data, only 3 included measures of infant mortality.
Limitations
When we sought to identify quality measures, we did not review measure sets designed for use in adult populations or those that addressed women’s health or pregnancy care. As a result, we may have omitted measures that could be applied to pediatric patients, particularly adolescents. Pregnancy is a major cause of hospitalization for patients <18 years old14; however, we did not encounter measures that addressed pregnancy care for adolescents in our review. Rather than conclude that there are no measures addressing pregnancy care for adolescents, we assume that, had we reviewed measure sets focusing on women’s health, we would have encountered measures applicable to the adolescent age group.
We presented summary counts of the measures and did not include weighting in our analyses. As a result, a measure set that contains several individual measures would greatly impact our total counts. In addition to weighting by number of measures, we did not include weights by impact of the measure. It could be argued that some measures are more important because of greater scientific evidence for their use, some are applicable to a large proportion of patients, or some are associated with higher costs of care. To weigh the individual measures by these factors, we would have needed to conduct impact analyses for each measure, which was beyond the scope of the current project.
Similarly, we did not conduct comparative analyses of the relative validity or reliability of the measures. We did collect information reported by the measure developers regarding their validity and reliability studies. However, we did not develop a method for comparing results or for hierarchical reporting of which measures are more or less valid and reliable.
In concluding whether there are enough measures in a particular domain (eg, safety or measures for CSHCN), there are no standards to determine when "enough is enough." It could be argued that CSHCN are at great risk for poor outcomes, hence they should have measures that address all aspects of their care, which currently is not the case. It could also be argued that the proportion of measures for CSHCN should be similar to their proportion in the pediatric population, which is currently the case. Without already established criteria for determining when there are enough measures, we are limited in our ability to conclude whether we have enough measures in some domains and whether we need more measures in others.
|
CONCLUSIONS |
|---|
|
TOPABSTRACTCHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
Measures of health care quality for children are not as well developed as those for adults. However, several factors are converging to bring issues of child health care quality to the forefront. This project is the first attempt to apply the IOM framework to health care quality measures for children.
In the quality analyses, we found very few safety measures for children’s health. Medical errors is a small but growing field in health care quality for children33 but needs to grow to include both inpatient and outpatient measures of safety. Less has been done to outline and test errors related to surgical procedures for children. Safety measures for children should include a focus on surgical procedures that are more common in children such as circumcision, tonsillectomy, and appendectomy.
In the patient-perspective analyses, it was interesting to find that a large number of measures fell in the nondimensional category. Measure developers usually did not use the IOM framework as a basis for their work yet developed measures that are quite useful. Most of the nondimensional measures address important overarching issues such as patient-physician relationship. Although those measures did not fall neatly within the patient-perspective domains, they were readily classified within the health care quality domains. As more work is done to apply the IOM framework to current measures, those interested in health care quality should remain flexible in its application and recognize that some parts of the framework are applied more readily than others.
We noted a lack of health care quality measures that are specific to each age group, and found none applied to school-aged children. Although this is a developmental period that is less challenging than early childhood and adolescence, it is a period that sets the stage for adolescent risk, and this is the age when behavioral, developmental, and learning issues affect school performance. As more measures are developed for children, there will be an ongoing need to balance general measures applicable to all children with measures that are specific to the various developmental stages.
The IOM has recommended that the equitable distribution of health care and services be an overarching concern in assessing health care in the United States. Although recent work on disparities has focused on racial/ethnic disparities, issues of equity can apply to differences in care noted by gender, education, income, health status, insurance type, and medical practice setting as well as race and ethnicity.34–37 For the purposes of this project, when determining whether the measure sets have been used in studies of equity, we included any studies that looked at vulnerable populations (eg, minorities, CSHCN, or Medicaid recipients). And when determining whether the measure sets could be used for comparative analyses, we looked for several sociodemographic items that could potentially be used for comparison.
Research regarding racial/ethnic disparities in health care is fairly new. As more work is conducted to determine the root causes of observed disparities, the content of the health care quality-measure sets will need to be adjusted to accurately measure the real causes of disparities. As an example, there is evidence that not having English as a primary language is a major cause of difficulties in the health system.38 If health care quality-measure sets measure Latino ethnicity but not primary language spoken, they will not be able to determine which patients are at greatest risk for poor outcomes.
For further discussion of disparities in health care quality, we recommend the work of Fiscella et al39 They have outlined several considerations for addressing disparities and our ability to monitor disparities within health care quality: 1) disparities must be recognized as a major problem in health care quality; 2) data must be accurate and readily available to monitor disparities, which is particularly important for federal Health Insurance Portability and Accountability Act of 1996 initiatives for data collection in health care;40 3) performance measures should be stratified to highlight disparities; and 4) reimbursement of health care services should consider the race and socioeconomic status of the patients served to improve reimbursement to those who care for high risk patients.
Most of the surveys have the ability to be used for studies of equity because they contain variables that can be used for comparative analyses. Many of the measures include CAHPS-like questions of patient experiences of care. Morales et al41 studied these measures using methods of item response theory. Briefly, item response theory proposes that subjects can vary in response to an item based on an underlying or latent trait.42,43 As a result, differences observed in scales may reflect differences in the applicability of the test items rather than true differences in the state of respondents. Their studies using CAHPS found no differences in functioning by race or ethnicity. Additional work is needed in all the survey measures to assess the cross-cultural validity of their application.
Last, many of the measures rely on patient self-report to evaluate the patient-centeredness component of care. These measures need to be made available in several languages and undergo validity testing in those languages as well to insure their cross-cultural validity for measuring care.
Our final consideration was whether the current set of health care quality measures addresses common causes of illness. As more health care quality measures for children are developed, researchers need to map newer measures to common causes of health care use for inpatients and outpatients. As an example, the most common cause for hospitalization is newborn care, yet we found no measures of hepatitis B vaccine, phenylketnonuria screening at 24 hours of life, screening for jaundice, or appropriate follow-up of newborns after hospital discharge, which are recommended practices for newborn care. Some of these measures could be obtained from administrative data, whereas others would require more costly medical record reviews. However, they are all measures that assess aspects of care given to each of the 4 million children born in the United States each year.
|
ACKNOWLEDGMENTS |
|---|
This work was funded by the Commonwealth Fund and the Agency for Healthcare Research and Quality.
|
FOOTNOTES |
|---|
Received for publication May 13, 2003; Accepted Oct 2, 2003.
Address correspondence to Anne C. Beal, MD, MPH, Quality of Care for Underserved Populations, The Commonwealth Fund, 1 E 75th St, New York, NY 10021. E-mail: acb{at}cmwf.org
¶ Although mental and dental health are important components of health care for children, inclusion of those areas was beyond the scope of this study. 
# The IOM proposes selection of measures based on their importance, scientific soundness, and feasibility of use. We reviewed the scientific soundness and feasibility of each measure, and the results are available on request.
|
REFERENCES |
|---|
|
TOPABSTRACTCHALLENGES OF HEALTH CARE...CHALLENGES OF QUALITY HEALTH...PROJECT OVERVIEWIOM FRAMEWORK FOR THE...METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
- Institute of Medicine. Envisioning the National Health Care Quality Report. Washington, DC: National Academy Press; 2001
- McGlynn EA, Damberg CL, Kerr EA, Schuster MA, Eds. Quality of Care for Children and Adolescents: A Review of Selected Clinical Conditions and Quality Indicators. Santa Monica, CA: Rand; 2000
- Schor EL, Szilagyi PG. The health of children. Health Serv Res.1998; 33 :1001 –1039[Web of Science][Medline]
- Halfon N, Schuster M, Valentine W, McGlynn E. Improving the quality of healthcare for children: implementing the results of the AHSR research agenda conference. Health Serv Res.1998; 33 :955 –976
- Homer CJ, Kleinman LC, Goldman DA. Improving the quality of care for children in health systems. Health Serv Res.1998; 33 :1091 –1109
- Palmer HR, Miller, Marlene R. Methodological challenges in developing and implementing measures of quality for child health care. Ambul Pediatr.2001; 1 :39 –52[CrossRef][Web of Science][Medline]
- Mangione-Smith R, McGlynn EA. Assessing the quality of healthcare provided to children. Health Serv Res.1998; 33 :1059 –1090[Web of Science][Medline]
- Forrest CB, Simpson L, Clancy C. Child health services research. Challenges and opportunities.
JAMA.1997; 277
:1787
–1793
[Abstract/Free Full Text] - Seid M, Varni JW, Kurtin PS. Measuring quality of care for vulnerable children: challenges and conceptualization of a pediatric outcome measure of quality.
Am J Med Qual.2000; 15
:182
–188
[Abstract/Free Full Text] - Shaul JA, Fowler FJ Jr, Zaslavsky AM, Homer CJ, Gallagher PM, Cleary PD. The impact of having parents report about both their own and their children’s experiences with health insurance plans. Med Care.1999; 37(3 suppl) :MS59 –MS68[CrossRef]
- Bright Futures. Available at: http://www.brightfutures.org
- Hermida J, Nicholas DD, Blumenfeld SN. Comparative validity of three methods for assessment of the quality of primary health care.
Int J Qual Health Care.1999; 11
:429
–433
[Abstract/Free Full Text] - Joseph-Di Caprio J, Garwick AW, Kohrman C, Blum RW. Culture and the care of children with chronic conditions: their physicians’ views.
Arch Pediatr Adolesc Med.1999; 153
:1030
–1035
[Abstract/Free Full Text] - McCormick MC, Kass B, Elixhauser A, Thompson J, Simpson L. Annual report on access to and utilization of health care for children and youth in the United States–1999.
Pediatrics.2000; 105
:219
–230
[Free Full Text] - Kempe A, Beaty B, Englund BP, Roark RJ, Hester N, Steiner JF. Quality of care and use of the medical home in a state-funded capitated primary care plan for low-income children.
Pediatrics.2000; 105
:1020
–1028
[Abstract/Free Full Text] - Halfon N, Inkelas M, Newacheck PW. Enrollment in the State Child Health Insurance Program: a conceptual framework for evaluation and continuous quality improvement. Milbank Q.1999; 77 :181 –204, 173[CrossRef][Web of Science][Medline]
- Simpson L, Fraser I. Children and managed care: what research can, can’t, and should tell us about impact. Med Care Res Rev.1999; 56(suppl 2) :13 –36
- Felt-Lisk S. Monitoring quality in Medicaid managed care: accomplishments and challenges at the year 2000. J Urban Health.2000; 77 :536 –559[CrossRef][Web of Science][Medline]
- Bergman DA, Homer CJ. Managed care and the quality of children’s health services. Future Child.1998; 8 :60 –75[CrossRef][Web of Science][Medline]
- Newacheck PW, Stein RE, Walker DK, Gortmaker SL, Kuhlthau K, Perrin JM. Monitoring and evaluating managed care for children with chronic illnesses and disabilities.
Pediatrics.1996; 98
:952
–958
[Abstract/Free Full Text] - Jacobson BB, Brock KA, Keppler AB. The Post Birth Partnership: Washington State’s comprehensive approach to improve follow-up care. J Perinat Neonatal Nurs.1999; 13 :43 –52[Web of Science][Medline]
- Escobar GJ, Braveman PA, Ackerson L, et al. A randomized comparison of home visits and hospital-based group follow-up visits after early postpartum discharge.
Pediatrics.2001; 108
:719
–727
[Abstract/Free Full Text] - National Quality Forum. Available at: http://www.qualityforum.org
- Foundation for Accountability. CAHMI. Available at: http://www.facct.org/cahmi.html
- National Initiative for Children’s Healthcare Quality. Available at: http://www.nichq.org
- Leapfrog Group. Available at: http://leapfroggroup.org
- Institute of Medicine. Envisioning the National Health Care Quality Report (2001). Available at: http://www.nap.edu/catalog/10073.html. Accessed December 8, 2003
- Bethell C, Lansky D, Read D. Reporting Quality Information to Consumers. A Report to the Health Care Financing Administration. Portland, OR: FACCT-The Foundation for Accountability; 1997
- Palmer HR. Computerized Needs-Oriented Quality Measurement Evaluation System (CONQUEST). Available at: http://www.qualitymeasures.ahrq.gov. Accessed December 8, 2003
- Ferris TG, Dougherty D, Blumenthal D, Perrin JM. A report card on quality improvement for children’s health care.
Pediatrics.2001; 107
:143
–155
[Abstract/Free Full Text] - Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q.1966; 44(3 suppl) :166 –206
- Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001
- American Academy of Pediatrics, Committee on Drugs and Committee on Hospital Care. Prevention of medication errors in the pediatric inpatient setting.
Pediatrics.1998; 102
:428
–430
[Abstract/Free Full Text] - Wood DL, Corey C, Freeman HE, Shapiro MF. Are poor families satisfied with the medical care their children receive?
Pediatrics.1992; 90
:66
–70
[Abstract/Free Full Text] - van Berkestijn LG, Kastein MR, Lodder A, de Melker RA, Bartelink ML. How do we compare with our colleagues? Quality of general practitioner performance in consultations for non-acute abdominal complaints.
Int J Qual Health Care.1999; 11
:475
–486
[Abstract/Free Full Text] - Alessandrini EA, Shaw KN, Bilker WB, Schwarz DF, Bell LM. Effects of Medicaid managed care on quality: childhood immunizations.
Pediatrics.2001; 107
:1335
–1342
[Abstract/Free Full Text] - Merrick NJ, Houchens R, Tillisch S, Berlow B, Landon C, Group M. Quality of hospital care of children with asthma: Medicaid versus privately insured patients. J Health Care Poor Underserved.2001; 12 :192 –207[Web of Science][Medline]
- Flores G, Abreu M, Schwartz I, Hill M. The importance of language and culture in pediatric care: case studies from the Latino community. J Pediatr.2000; 137 :842 –848[CrossRef][Web of Science][Medline]
- Fiscella K, Franks P, Gold MR, Clancy CM. Inequality in quality: addressing socioeconomic, racial, and ethnic disparities in health care.
JAMA.2000; 283
:2579
–2584
[Abstract/Free Full Text] - Perot RT, Youdelman, Mara. Racial, Ethnic, and Primary Language Data Collection in the Health Care System: An Assessment of Federal Policies and Practices. New York, NY: The Commonwealth Fund; 2001
- Morales LS, Elliott MN, Weech-Maldonado R, Spritzer KL, Hays RD. Differences in CAHPS adult survey reports and ratings by race and ethnicity: an analysis of the national CAHPS benchmarking data 1.0. Health Serv Res.2001; 36 :595 –617[Web of Science][Medline]
- Hays RD, Morales LS, Reise SP. Item response theory and health outcomes measurement in the 21st century. Med Care.38 :II28 –II42
- Hayes RP, Baker DW. Methodological problems in comparing English-speaking and Spanish-speaking patients’ satisfaction with interpersonal aspects of care. Med Care.1998; 36 :230 –236[CrossRef][Web of Science][Medline]
PEDIATRICS (ISSN 1098-4275). ©2004 by the American Academy of Pediatrics
CiteULike 
Connotea 
Del.icio.us 
Digg 
Facebook 
Reddit 
Technorati 
Twitter What's this?
This article has been cited by other articles:
|
|
 |
|
  S. A. Spooner and D. C. Classen Data Standards and Improvement of Quality and Safety in Child Health Care Pediatrics, January 1, 2009; 123(Supplement_2): S74 - S79. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
F. L. Nkoy, B. A. Fassl, T. D. Simon, B. L. Stone, R. Srivastava, P. H. Gesteland, G. M. Fletcher, and C. G. Maloney Quality of Care for Children Hospitalized With Asthma Pediatrics, November 1, 2008; 122(5): 1055 - 1063. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. M. McDonald, S. M. Davies, C. A. Haberland, J. J. Geppert, A. Ku, and P. S. Romano Preliminary Assessment of Pediatric Health Care Quality and Patient Safety in the United States Using Readily Available Administrative Data Pediatrics, August 1, 2008; 122(2): e416 - e425. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. W J Schulpen and K. M J Lombarts Quality improvement of paediatric care in the Netherlands Arch. Dis. Child., July 1, 2007; 92(7): 633 - 636. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. Simpson, D. Dougherty, D. Krause, C. M. Ku, and J. M. Perrin Measuring Children's Health Care Quality American Journal of Medical Quality, March 1, 2007; 22(2): 80 - 84. [PDF]
|
|
|
|
|
|
P. S. Lye, D. A. Rauch, M. C. Ottolini, C. P. Landrigan, V. W. Chiang, R. Srivastava, S. Muret-Wagstaff, and S. Ludwig Pediatric Hospitalists: Report of a Leadership Conference Pediatrics, April 1, 2006; 117(4): 1122 - 1130. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. Tai, P. Dick, T. To, and J. G. Wright Development of Pediatric Comorbidity Prediction Model Arch Pediatr Adolesc Med, March 1, 2006; 160(3): 293 - 299. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. C. Beal Policies To Reduce Racial And Ethnic Disparities In Child Health And Health Care Health Aff., September 1, 2004; 23(5): 171 - 179. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Feudtner Perspectives on Quality at the End of Life Arch Pediatr Adolesc Med, May 1, 2004; 158(5): 415 - 418. [Full Text] [PDF]
|
|
|
|
|
|
D. Dougherty and L. A. Simpson Measuring the Quality of Children's Health Care: A Prerequisite to Action Pediatrics, January 1, 2004; 113(1/S1): 185 - 198. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Kuhlthau, T. G. G. Ferris, and L. I. Iezzoni Risk Adjustment for Pediatric Quality Indicators Pediatrics, January 1, 2004; 113(1/S1): 210 - 216. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Shaller Implementing and Using Quality Measures for Children's Health Care: Perspectives on the State of the Practice Pediatrics, January 1, 2004; 113(1/S1): 217 - 227. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
