Objective. Improving the quality of health care is a national priority. Nonetheless, no systematic effort has assessed the status of quality improvement (QI) initiatives for children or reviewed past research in child health care QI. This assessment is necessary to establish priorities for QI programs and research.
Methods. To assess the status of QI initiatives and research, we reviewed the literature and interviewed experts experienced in QI for child health services. We defined QI as activities intended to close the gap between desired processes and outcomes of care and what is actually delivered. We classified reports published between 1985 and 1997 by publication characteristics, study design, clinical problem addressed, site of intervention, the QI method(s) used, and explicit association with a continuous quality improvement program.
Results. We reviewed 68 reports meeting our definition of QI. More than half (48) were published after 1994. The reviewed reports included controlled evaluations in 36% of all identified interventions, and 3% of the reports were associated with continuous quality improvement. QI methods demonstrating some effectiveness included reminder systems for office-based preventive services and inpatient pathways for complex care. Reportedly successful QI initiatives more commonly described improvement in administrative measures such as rate of hospitalization or length of stay rather than functional status or quality of life. Interviews found that barriers to QI for children were similar to those for adults, but were compounded by difficulties in measuring child health outcomes, limited resources among public organizations and small provider groups, and relative lack of competition for pediatric tertiary care providers. Research and dissemination of QI for children were seen as less well developed than for adults.
Conclusions. Attempts to improve the quality of child health services have been increasing, and the evidence we reviewed suggests that it is possible to improve the quality of care for children. Nonetheless, numerous gaps remain in the understanding of QI for children, and widespread improvement in the quality of health services for children faces significant barriers.
Recent reports highlight the urgent need to improve the quality of health care.1,2 There is sufficient evidence of deficiencies in the quality of child health services to include children in the movement to improve the quality of health care.3,4 However, the unique circumstances of children suggest the need to evaluate efforts to improve the quality of child health services separately from adults.3,5 Specifically, children have distinct provider systems and different epidemiology of illnesses. In addition, children are generally dependent on caregivers for access to health care, and children have a range of developmental issues not relevant to health care for adults. The time is ripe, therefore, for an evaluation of what is known about quality improvement (QI) in the arena of child health and the implications of that information for future efforts to improve the quality of care that children receive.
Past efforts to review strategies to improve care do not focus on children or the delivery of care to this population. Recent reviews of the literature on effective strategies to change practice find evidence that some strategies work better than others.6–9 For example, physician behavior change seems to depend more on real-time reminders and effective leaders and less on passive education. The extent to which these findings apply to child health services is unknown. The review of QI for children by Homer and colleagues10 provides a useful conceptual framework for addressing QI for children and identifies key areas of concern. Nonetheless, this review does not consider the available evidence on past or ongoing initiatives to improve the care of children.
In this study we attempt to address gaps in the current understanding of the status and prospects of QI in children through a review of the literature on this topic and through interviews with expert observers of QI in child health care. For these activities, we define QI activities as “interventions that are designed to close the gap between desired processes and outcomes of care and what is actually delivered … QI interventions may be directed at any group of participants in the delivery system: administrators, physicians, laboratory technicians, parents, children, employers.”4One way of operationalizing this definition is to identify QI as interventions intended to decrease the underuse, overuse, and misuse of an intervention.1 Such interventions take at least 3 forms. The interventions may be directed at improving the care provided for a specific clinical problem or patient population or procedure; may test a generic tool or method for QI that can be applied to a range of clinical problems, populations, or procedures; or may be more holistic: aimed at enhancing performance of an entire organization or health care system. In this study, we review evidence on the status of all 3 types of QI activities.
Identifying QI Literature
Using Medline, Healthstar, and the Cochrane Library, we searched the terms frequently associated with QI efforts and combined each with “child” and “adolescent.” The search terms included: quality improvement, evaluation, satisfaction, case manager/management, disease management, benchmarking, opinion leader, guideline, critical path/pathway, reminder, continuous quality improvement, and total quality management. Because we hypothesized that a significant proportion of the QI literature may be published in journals related to health care management and health care quality, we reviewed the article titles for the following journals: American Journal of Medical Quality, Joint Commission Journal on Quality Improvement, Hospital and Health Services Administration, Quality Assurance in Health Care, Quality Management in Health Care, Journal of Clinical Outcomes Management, Quality Assurance and Utilization Review, Quality in Health Care, Quality Assurance, Journal of Quality in Clinical Practice, and the Journal of Nursing Care Quality. Finally, we reviewed the references of selected articles and solicited suggestions for articles from child health services researchers and directors of QI programs.
Our electronic search strategy resulted in the identification of over 2000 reports. Two people reviewed all titles from electronic and manual searches and selected 238 reports for review of abstracts. Based on abstract reviews, 113 reports were selected for coding.
Articles were selected for review if they met the following criteria: 1) the paper was published in English between 1985 and 1998, 2) at least 1 stated goal of the intervention was to improve the quality of care for children or adolescents <18 years old, and 3) the report included a quantitative assessment of the primary outcome. For our purposes, QI was defined as any intervention designed to decrease underuse, overuse, or misuse of an effective intervention. This operational definition left several studies equivocal with respect to their inclusion. Difficult studies to classify included public health or community-based interventions designed to decrease high-risk behaviors.11,12 Although significant improvements in child health may come from such interventions,13 we considered them to be tests of new interventions rather than attempts to improve the delivery of existing health care services and therefore excluded them from our review.
Classification and Analysis
We recorded the study characteristics included in Table 1 for each of the reports we included. We used 3 categories of study design: 1) randomized controlled, 2) non-randomized controlled, and 3) before–after. We used the designation of before–after to categorize studies that evaluated a change in a process or outcome over time where no prospective control was used as a comparison (eg, pre–post, control chart, time series). We identified the common methods used in efforts to improve quality of care and categorized each report by the methods used. QI programs frequently used multiple tools and may cross site or organizational boundaries, thus complicating attempts to classify programs. In addition, we searched each included report for a reference to an institutional commitment to improve the quality of care at the site where the QI intervention was conducted. Reports with such a reference were categorized as continuous quality improvement/total quality management (CQI/TQM).
Data were reported in an evidence table divided into 9 sections corresponding to the QI methods we use to classify the interventions. Classifying the QI methods was problematic because of lack of established criteria and definitions. Previous studies lumped interventions irrespective of method7 or reviewed the data on one particular method.14 Interventions frequently used >1 method and we considered this in our review, but to make the evidence table more efficient each study was listed only once. We then reviewed the QI literature in terms of 1) publication characteristics, study design, and site of care, 2) QI methods, 3) the clinical problem addressed, and 4) a reported association with CQI. The evidence table was analyzed qualitatively to identify broad conclusions across the included studies, concentrating on the studies with control groups.15
We report also the results of 9 semistructured telephone or face-to-face interviews with individuals involved with QI, conducted to assess past and current child QI initiatives. Interviewees represented a distribution of academic, public health, and private provider organizations, as well as different geographic locations. The interviews included questions that addressed 1) the scope of child QI activities (nationally and at the interviewee's institution), 2) the success of child QI activities, and 3) barriers to child QI. Copies of the instrument used to guide the interviews are available from the corresponding author. All calls were taped with permission from interviewee.
We present results of our literature review from 4 perspectives: 1) overview of literature based on publication characteristics, design characteristics, and site of care, 2) the QI methods used, 3) the clinical condition addressed, and 4) the use of CQI.
1. Overview of Literature
Our review of the literature identified 68 publications that met our inclusion criteria. Table 2presents 40 controlled trials of child QI published between 1980 and 1998. There was a dramatic increase in the number of studies published in the past few years, with 50% of reports published since 1994. Controlled studies were more likely to have been published in 1 of 3 journals (Pediatrics (7), Archives of Pediatrics and Adolescent Medicine (4), Medical Care (4)), whereas uncontrolled studies were more typically found in 1 of the quality of care journals (see methods). Controlled studies were more likely to report external funding (86% vs 13%) and more likely to report negative results among the primary endpoints (28% vs 18%). A majority of the US studies were located in a relatively small number of cities.
Only 22% of the reports used a randomized evaluation of the intervention. Most others used a before–after design to examine the influence of the intervention on the outcomes measured.
A majority of the interventions (68%) were directed at younger children (age <5), primarily because of the preponderance of studies on newborn care, hospitalized children, and immunizations. Half (48%) of the controlled trials were conducted in office settings.
2. The Methods of QI
Guidelines were defined as a set of rules intended to define appropriate care and guide practice. A pathway (or carepath or critical path) was defined as a method of documenting the processes of care and identifying reasons for variance from previously defined objectives. Because most efforts to improve care require some decisions about what is the best possible care, it may be argued that all QI requires guidelines of one sort or another. For example, all efforts to improve immunization practices rely on guidelines for correct practice. Our attempt to identify reports of guidelines and pathways in pediatrics was restricted to those reports that use language describing an organized rule-based approach to improving care.
We identified 7 controlled trials of guidelines or pathways for child health16–22 and 15 uncontrolled evaluations.23–37 Most of these reports found improvements in the outcomes measured, including length of hospitalization, test ordering, admission rates, or appropriate medication use. Measures related to functional status, quality of life, or even missed school days were rare. The studies of guideline implementation reveal that although the majority were applied in the hospital setting, they were increasingly applied and evaluated in ambulatory settings.
Studies of particular note included Evans' report17 of an educational intervention to improve the care of children with asthma by raising physician awareness of the National Asthma Education Promotion Program guidelines38 for asthma care. This intervention, in an inner-city environment where the social barriers have been depicted as limiting attempts from providers to improve care, increased the use of antiinflammatory medications from 2% to 25% and dramatically increased the percentage of parents who reported receiving education about asthma. Other statistically significant effects include more scheduled visits per patient per year (1.85 vs 0.88) and more patients prescribed a β-agonist metered-dose inhaler (MDI) (52% vs 15%).
The combination of a guideline with a particular method of delivering the guideline to patients and/or physicians was a recurring theme in the studies of guidelines reporting a successful intervention. Palmer and colleagues' reported a randomized controlled trial of the influence of guidelines and performance feedback on primary care physicians (including pediatricians).39 These investigators found no benefit to guidelines alone, but guidelines combined with performance feedback led to significant improvement in well-child care, gastroenteritis management, and otitis media management.
Other effects from the controlled trials of guidelines with feedback included: improved preventive counseling delivered in a primary care practice,21 reduced length of stay on a general pediatrics ward,18 fewer preoperative lab studies in outpatient child surgery,22 improved outpatient follow-up after an emergency department (ED) visit (47% vs 24%), and a reduction in unjustified hospital days.20 One controlled trial reported an unsuccessful intervention. Rodewald and colleagues16attempted to improve immunization rates during ED visits, but reported a high parental refusal rate and inability to ascertain accurate immunization status as barriers to the intervention.
A report detailing the improvements associated with using a clinical pathway for hospitalized patients with asthma illustrated the difficulty of assessing QI based on a literature search (www.ihi.org/resources/eyeoi/casestudy.asp). This report, found only on the web, described (uncontrolled) improvements in several outcomes, such as length of stay (3.1 days to 1.75 days) and readmission (25% to 5%), patient satisfaction with care and asthma knowledge, and reduced unnecessary testing (75% reduction). Kwan-Gett and colleagues had much less success with their inpatient asthma pathway,36 and it was difficult to ascertain possible reasons for these disparate findings based on the information provided.
Reminder systems were frequently implemented as part of a guideline or pathway, but they are also used in isolation to solve a particular problem. Reminders at the time that care is provided showed relatively strong positive results. For example, 2 trials demonstrated dramatic increases in preventive services delivered when encounter forms prompted clinicians at the time of a patient visit.21,40
Other demonstrated effects from controlled trials of reminder systems included increased attendance at primary care office visits for children41,42 and adolescents43; as well as increased attendance for child psychiatry visits44; improved immunization rates45–49; improved interaction between patient and physician50; decreased hospital length of stay18; and improved follow-up after ED visit19,51,52; TB testing53,54; and acute illness visits.55
There has been sufficient agreement about the ability of reminder systems to improve vaccination rates that the Centers for Disease Control and Prevention (Advisory Committee for Immunization Practices), the American Academy of Pediatrics, and the American Academy of Family Practice recommended that all providers, public and private, use reminder systems to increase vaccination rates.56 When this recommendation was announced, only 25% to 35% of pediatricians had such systems in place.56
Hunt reviewed the literature of computer-based decision support systems (CDSS) and identified 68 controlled trials.57 CDSS were frequently used in conjunction with reminder systems, although they are also used as diagnostic aids and references. The review concluded that CDSS enhance clinical performance for drug dosing and preventive care. Only 1 of the trials they identified was performed exclusively with children and child providers, and only 5 of the trials included children at all.
Disease Management/Case Management
Disease management has been referred to as “a systematic, population based approach to identify persons at risk, intervene with specific programs of care, and measure clinical and other outcomes.”58 Usually disease management involved identifying a cohort of patients with a similar disease in a population and applying strategies such as education or medication adherence reminders to the entire group. This approach may have limited applicability in children with the exception of asthma, where disease management has become almost commonplace.59 Nonetheless, we found no reports meeting our criteria describing the effectiveness of this type of intervention for asthma.
There were reports of disease management techniques used for other pediatric diseases such as cystic fibrosis (CF). Finkelstein and colleagues demonstrated the feasibility of employing a home monitoring system for children with CF similar to those used for adult patients with congestive heart failure,60 and Kretz reported both clinical improvement as well as a 33% savings with a disease management model applied to children with CF.61
Case managers have provided a variety of functions in inpatient and outpatient settings as they shepherd individual patients through the medical system. Case managers may link public and private health systems in the care of high-risk patients.62 This study was remarkable for the rare glimpse of what has been possible when unlinked systems become connected in the care of mothers and infants, as well as the study's randomized design. The intervention increased prenatal visits and use of primary care physicians as a regular source of sick care decreased waiting time for appointments, and increased patient education. Children with newly-diagnosed diabetes spent fewer days in the hospital when they had a specially trained nurse case manager assigned to their care,63 and children with asthma had fewer repeat admissions when the family received inpatient nursing instruction in self-management.64
Physician Leaders and Academic Detailing
Physician leaders or local opinion leaders were individuals within a group who have been identified by their colleagues as influential. Only 1 study of the role of physician leaders in QI for children was found.65 Palmer's analysis provided empirical evidence of the impression frequently found in the QI literature that a committed physician leader makes a difference in attempts to improve the quality of care.
Physician Financial Incentives
Despite evidence that financial incentives for child health care providers are related to quality of care,66,67 we did not find literature describing a program that used this technique as a means to improve the quality of child health care. This lack of evidence was striking considering the increasingly frequent use of financial incentives by insurance companies to influence physician practices.68
Benchmarking and Provider Performance Assessment
Benchmarking and provider performance assessment (sometimes referred to as profiling or audit and feedback) referred to the reporting of clinical and/or economic measures to a provider (or group of providers) with comparisons to colleagues (or other groups). This method has often been used in conjunction with guidelines. We identified only 1 controlled study of the effectiveness of this QI method in the care of children.39
A meta-analysis of profiling studies69 found a statistically significant but minimal effect on physician practice (odds ratio 1.091, 95% confidence interval, 1.045 to 1.136). None of the studies Balas et al identified address physicians caring for children.
Efforts to produce a common set of quality measures that can be used for benchmarking and profiling are underway. The National Committee on Quality Assurance's (NCQA) health plan assessment mechanism (HEDIS) reveals major variations in quality of care by health plans, using several measures of child health quality such as immunization status. Limits to the HEDIS approach to measuring quality for children's health care partly reflect the small number of child measures currently available in the HEDIS measurement set.70 A child version of the Consumer Assessment of Health Plans Survey has been added to the HEDIS measures.71 NCQA and the Foundation for Accountability have been developing more child and adolescent measures.72
Continuing Medical Education (CME)
We defined CME narrowly to include only educational interventions where the physician was a passive recipient of knowledge (for example grand rounds or a CME course). We found several controlled studies of the influence of CME on physician performance and patient outcomes in the care of children.17,73,74 These carefully conducted studies all described a significant change in provider behavior that improved both physician practices as well as patient outcomes in areas such as medication compliance and patient satisfaction. Although these studies were encouraging, a recent systematic review of CME by Davis and colleagues concluded that CME alone has little influence on provider behavior or patient outcomes.7 The success of the interventions noted above may reflect the careful development of the educational process or physician awareness of being observed.
Educating patients and their caregivers to improve their understanding of their health, illnesses, and appropriate use of health services has the potential to improve the quality of their care. The literature included numerous reports of educational interventions for both children and parents, but many of these addressed populations outside the health care system (for example, smoking cessation for adolescents in school75), and these studies were outside the scope of this review (see “Methods”). Nonetheless, we found several studies of educational interventions in patient care settings.54,76–80 Controlled studies have shown decreased exposure to environmental smoking,81 decreased home hazards after safety education,82 and increased compliance with ED follow-up.51,52
Controlled studies of educational interventions for patients with asthma64,77,83–86 and diabetes87,88 showed increased knowledge and/or self-care, but little or no effect on morbidity. Exceptions included the study by Hackett and colleagues that found a small but significant decrease in HbA1c in children over 11 years old,87 and the study by Madge and colleagues that reported decreased admissions for asthma.64 A recent metaanalysis of randomized controlled trials of patient education for asthma confirms the generally poor performance of patient education interventions.89
Several studies focused on the influence of education on appropriate use of acute care services.19,52,90 These studies generally reported positive results on patient satisfaction and decreased utilization where each is reported. One of the rare randomized trials of patient education found no influence of physician counseling during a child ambulatory visit on the purchasing of bicycle helmets.91
The studies we reviewed demonstrated a relatively strong influence on measures of patient satisfaction and some influence on knowledge, but little influence on patient behaviors or health outcomes.
3. Child QI and Clinical Areas
The clinical categories of prevention, acute care, and chronic care are frequently used for discussing QI. Nonetheless, this classification has some limitations. Importantly, QI programs are typically initiated in a specific site with a specific group of people, whereas the preventive care, acute care, or chronic care a child receives may come from a variety of sites or even unconnected organizations.
Among the most common QI interventions in the literature were attempts to improve prevention practices. Reminder systems to improve physician immunization, screening, and counseling practices (see above) showed the greatest improvements in process measures of any of the types of interventions we reviewed. Prevention occurred not only in physicians' offices but also through home visiting62,92and school-based programs.93 Although a majority of the studies we identified for this review report a benefit from the intervention, the studies that addressed child risk factors outside the physicians' office had the poorest outcomes as a group, possibly because these trials have attempted to change very difficult behavior problems.
Reports of improvements in acute illness care originated primarily from EDs and intensive care units. Improvements in patient satisfaction with ED care94,95 as well as the clinical care of patients with trauma96 were reported. The implementation of guidelines and pathways for outpatient surgery, complex tertiary care such as cancer care and surgery for congenital heart disease, and care for acute asthma demonstrated some improvements in process measures as noted above. The primary outcome used in most of the inpatient studies is length of stay, and length of stay does not have a direct relationship to quality.
Ninety percent of the studies (and 6 of 11 controlled trials) concerning the care of children with chronic illnesses involve asthma. As the most common chronic condition of children, asthma has been the principal vehicle for the study of different QI methods. Nearly all the uncontrolled reports presented beneficial effects on hospital readmission, length of stay, and appropriate medication use. Children entered into these programs usually had a dramatic (up to 60%) decrease in utilization regardless of any intervention. The controlled trials also showed significant improvements, but with more modest magnitude than in the uncontrolled literature.
Other chronic conditions where QI has been applied include cystic fibrosis,60,61 leukemia,97 congenital heart disease,35 renal failure,98 and diabetes.63
Nine reports of interventions described a CQI process. These reports focused on complex hospital-based patient care issues such as invasive cardiology,99 cardiac surgery,35,100leukemia,97 and pediatric peritoneal dialysis.98 Three reports described the use of CQI methods to implement guidelines in primary care practice settings. Carlin reported increasing immunization rates from 53% to 86% in 19 clinics in Minnesota.24 Solberg used CQI to assist with the implementation of guidelines for preventive services,101and Gibson reported success using CQI to implement asthma care guidelines.102
Two reviews examined the successes of CQI/TQM in health care institutions.103,104 These studies did not consider children's hospitals or child providers separately and therefore do not help to assess the special circumstances of child health care services, which may affect their success or failure. Shortell, Bennett, and Byck's review of the evidence for CQI104 included only 2 reports of CQI for children out of the 55 studies they identified,24,105 although several other studies they cite included children as part of hospital wide CQI programs.
FINDINGS FROM INTERVIEWS
Interviews of QI experts provided information on the scope of child QI activities, how successful they have been, determinants of success or failure, barriers to improving the quality of health care for children, and important unanswered questions for the future of child QI.
The Scope of Child QI
Interviews indicated the widespread use of at least a few QI programs for child health care services in hospital settings, both children's hospitals and large general hospitals, as well as many managed care organizations. The extent of an institution's commitment to child QI seemed approximately proportional to that institution's commitment to providing child health care services. Large children's hospitals may have extensive activities, whereas smaller organizations, general hospitals, and health maintenance organizations were more likely to limit QI to an asthma pathway and/or monitoring immunization rates. This view was supported in part by a study of child QI in emergency rooms by Gausche and colleagues.106 Respondents agreed that inpatient QI programs were significantly more developed than outpatient programs. Respondents also agreed that institutions primarily seek cost reductions through QI and do not generally support QI initiatives without potential cost savings for the institution (except for strategies to improve patient satisfaction).
Large provider organizations seemed to adopt improvement activities to 1) improve patient satisfaction and 2) decrease unnecessary variations in the provision of care. Programs to improve patient satisfaction included patient satisfaction surveys followed by interventions to address identified deficiencies. The most frequently cited deficiencies in patient satisfaction for child health care services included discharge planning, waiting times for appointments, waiting times in the ED, and clear explanations of tests and procedures.
The second common QI activity involved attempts to decrease unwanted variation in the care of patients through the implementation of guidelines and critical pathways. Many children's hospitals had some guidelines/pathways in place for the most expensive and complex areas of care such as bone marrow transplantation, cardiac surgery, and neonatal intensive care unit care. Asthma care has often been one of the first and most extensive pathways in hospitals and managed care organizations because of the high frequency of admissions (and therefore high cost) and the availability of interventions that were known to decrease admissions. Asthma was the only child condition where the methods of disease management were commonly applied. Some regional referral centers have developed guidelines and pathways for uncommon conditions because of their relatively high volume of these conditions, reflecting the large population base of their referral areas.
Some large holes in the spread of QI were apparent from our interviews. QI in small group practice settings and community hospitals was rarely more than the occasional chart audit necessary for licensing. Little has been done to improve the coordination of care of children with chronic illnesses, in part attributable to the poorly defined responsibilities of specialists and generalists in the care of these patients. Despite the high prevalence of mental health problems among adolescents, our respondents were generally unaware of programs addressing the quality of care this population receives. Hospital-based error reduction programs (including especially medication errors) has received almost no attention in pediatrics. Finally, few programs assessed effectiveness with measures of child health or quality of life.
Our respondents described numerous examples where a gap between knowledge and performance was closed. Frequently, however, these improvements were in administrative areas such as hospital length of stay.
What Are the Barriers to QI in Child Health Care Services?
Several common elements have conspired against improving the quality of child health care services. Some barriers noted by our respondents have been described in previous more general assessments of QI in medicine.107 These barriers included a lack of essential resources, conflicting incentives among clinicians and administrators, administrative instability in a time of rapidly changing organizational structures, and lack of cooperative skills among physicians.
Certain minimum infrastructure requirements for successful QI included support from a behavioral scientist, knowledge of billing systems, dedicated assistance in the collection of data, and access to a data analyst. Respondents observed several additional requirements for success. Senior managers had to support QI, and they must insist that everyone in their organization at least be available to QI. The importance of good ideas for how to make an improvement was often overlooked. Physician champions were necessary to support QI among clinicians who are often skeptical or disinterested. Making the recording of quality information a routine part of everyone's job also facilitated QI.
Certain aspects of the care of children may present greater barriers to QI in children than in adults. The low volume and rare negative outcomes of most childhood diseases may not allow for the data necessary for rapid cycle change. In addition, child health reflects services from institutions outside the personal health care system, such as public health services, school health programs, and child protection services. Although investigations into linking public and private delivery services have been reported,62 QI methods have not been well developed to address this situation.104A large proportion of child health services occurred in office settings, raising 3 issues that limit opportunities to improve the quality of child health. First, electronic outpatient information systems are poorly developed at this time and are still at least several years away from being widely disseminated. Second, small groups and solo practitioners did not have the resources to invest in QI. Third, the delivery of primary care to children was divided between family practitioners and pediatricians. Several respondents describe difficulties getting these 2 groups to cooperate even when they co-exist within the same organization.
Clinicians caring for children often do not perceive a problem with the quality of care they provide. Although providers of adult services may also fail to appreciate a problem with quality, this situation may be more problematic for child providers. The more limited data on the quality of child health services may make convincing pediatricians of the need to change more challenging. Respondents also reported that child health professionals often followed an advocacy model rather than an evidence-based model of child health resource allocation, so QI efforts that attempted to alter services based on data sometimes met hostility.
Competitive factors may have affected QI for adult and child services differently. The importance of competition for motivating QI has been stressed in the literature of QI.104,108 Although competition may have affected primary care providers' attention to QI, children's hospitals and other tertiary care child health units were often the sole providers of certain child health care services within a region. This monopoly may have limited the extent to which the leaders of those organizations perceive the need to improve quality.
The lack of involvement of minority QI researchers was seen as a barrier to developing effective interventions to improve the care of the growing child and adolescent minority populations. There were numerous quality issues in minority populations, and nonminority researchers and QI program directors may have had difficulty with access to these populations.
Finally, respondents agreed that there has been a significant deficit of research in the area of QI in child health. This deficiency makes their work significantly more difficult and frequently requires pursuing an inefficient trial and error approach to QI. There is much still to be learned about how best to go about the business of improving quality of care for children.
This review of QI for child health care assesses QI efforts to date and provides information to guide future initiatives. Our review of the literature and interviews provide evidence that it is possible to improve the quality of health care for children. We find the recent growth in child QI encouraging. Nonetheless, the limited distribution of QI and the limited resources applied to it also limit impact on the quality of care received by children in the United States. Numerous obstacles hinder widespread improvement in the quality of child health services.
The Methods of QI
Evidence for the effectiveness of guidelines and pathways in clinical practice has grown substantially since Merritt and colleagues reviewed this topic.14 The recent increase in reports of improvements in processes of care using guidelines and pathways in tertiary settings for complex care supports their more widespread use. Nonetheless, our interviews caution against the potential for considerable wasted effort when guidelines lack proper development and implementation. Active interventions to guide providers at the point of care delivery seem to be an essential component of this QI method.
The evidence concerning disease management, case management, physician leaders, physician financial incentives, and benchmarking/profiling is insufficient to formulate a conclusion about their usefulness. Although the literature and interviews suggest that these techniques have potential in certain circumstances (eg, disease management for asthma), the extent to which they should be part of a comprehensive QI program is unclear. The lack of controlled studies is striking given the apparent increasing use of these techniques by managed care organizations.
Physician education(CME) and patient education seem to be weak methods for changing behavior, but they have an apparent role in patient and provider satisfaction. Efforts to improve the effectiveness of educational interventions may be served by incorporating additional QI methods.9,109
The clinical areas that have benefited from effectiveness research and QI include the delivery of office-based preventive services and to a lesser extent complex inpatient care. Strong evidence supports the use of preformatted encounter forms and automated reminder systems to improve the quality of preventive practices. The literature and interviews indicate the potential to improve neonatal intensive care unit and ED care. In addition, our interviews suggest that regional and national networks of providers may be important sources for information and dissemination of QI, facilitating more widespread adoption of QIs in child health services. There is significant room for research into improving the quality of acute care services for children.
Several trials of QI in the care of children with chronic conditions indicate some promising results. In particular, evidence suggests the care of children with asthma could be dramatically improved with widespread adoption of QI techniques. Widespread adoption would be expected to eventually result in improvements in community based measures of asthma morbidity and hospital admissions. Opportunities for future efforts might well focus on higher prevalence conditions, especially adolescent mental health.
We find insufficient assessments of CQI efforts. Most reports describe successes, but with limited evaluations. Our interviews indicate cautious enthusiasm for the CQI initiatives to improve the quality of child health care.
Comparing Child and Adult QI
An underlying theme to our review is whether or not QI for children is different from QI for adults. There is no reason to think that systems like preprinted or computerized medication order forms would be less effective at reducing medication errors in children than they have been for adults. However, it remains to be established whether some QI methods suit the circumstances of child health care providers better than others do.
In many ways, QI for children seems from this review to be similar to QI for adults. The barriers to QI we found from our interviews have several similarities with those found by Blumenthal and Kilo107 and Shortell.104 The difficulties associated with ambulatory settings, cooperating across organizational boundaries, poor information systems, rare outcomes, professional resistance, lack of competition, and difficulties with measurement have all been identified previously as barriers to QI. Why then do our respondents find QI for children more difficult than QI for adults? One possibility is that the same barriers pose relatively greater challenges for child QI than adult QI. Child QI may therefore require proportionally more resources than adult QI to be equally effective. QI for children may be doomed to limp along given the tendency to focus QI on high cost areas.
There are several important limitations to this assessment of QI for child health services. The literature rarely describes failed interventions, despite the finding in our interviews that such failures are relatively common. In addition, we limited our interviews to experts with extensive knowledge of QI for children. We did not systematically survey child health institutions to determine the true scope of QI in child health services.
Difficulties associated with reviewing research on QI include unclear definitions of what constitutes QI activities, the lack of a system for classifying QI, and the low likelihood of publication of much QI work. In addition, many of the publications we cite are not among those initially identified in our search, suggesting that there may be a significant number of additional publications not included in our review.
Our analysis uses a novel method for the classification of reports of QI. Although the scheme we use has potential as a means to organize QI literature, it is limited by the lack of established criteria for many of the QI methods used such as “reminder systems” and “disease management.” Nonetheless, this classification scheme should allow for the application of metaanalytic techniques to arrive at quantitative estimates of the success of QI interventions from the 3 perspectives of QI methods, clinical areas, and association with CQI. A library of clinical effectiveness and QI literature that identifies literature based on the categories we use may be a valuable resource for researchers and managers interested in QI.
There are some reasons to be encouraged about the state of QI work in child health services. Published reports indicate that substantial improvements in the care of children can be made through QI. QI activities seem to be widespread at large provider institutions, and the publication of QI research is accelerating. Several exemplary collaborations are modeling the kind of cross-institutional work that is probably going to be necessary for widespread QI for children. There is sufficient evidence about the effectiveness of several specific interventions such as recall/reminder systems for improving the delivery of preventive services that wide dissemination is virtually guaranteed to improve the quality of care for children. The widespread use of the child supplement to the Consumer Assessment of Health Plans Survey as well as the current Foundation for Accountability/NCQA collaboration on the development of child quality measures should assist in making performance comparisons between large provider organizations. These comparisons should help stimulate QI efforts.
But there are also challenges ahead if efforts to improve the quality of health care for children are to have a significant impact on child health. Our literature review highlights the limited set of information on effectiveness of QI that clinicians and administrators interested in QI can turn to. There is insufficient evidence in most of the methodological categories we assess, and several important research questions await investigation. Child QI will need to move beyond administrative process measures and take on child health outcomes like quality of life and optimal development. Our interviews highlighted the very real danger of QI becoming an additional administrative task with little impact on quality of care.
It is disconcerting that with what we know about deficiencies in the quality of care for children and the extent of some local attempts to improve care, that widespread QI for child health care services remains elusive. There is ample opportunity to do better.
This work was supported by the Agency for Health Care Research and Quality.
Dr Ferris is a Fellow of the Pediatric Scientist Development Program (AAP/NICHD: K12-HD00850).
We thank the following persons for their contribution(s) to the interview portion of this research: Charles Homer, MD, Chistina Bethell, PhD, John Pestian, PhD, Donald Berwick, MD, Nira Bonner, MD, Mark Kirschbaum, PhD, Paul Kurtin, MD, Jill Joseph, MD, and Heather Palmer, MB, BCh. We also thank John Ausiello, who assisted with the literature search and data abstraction.
- Received April 14, 2000.
- Accepted July 19, 2000.
Reprint requests to (T.G.F.) MGH/Partners Institute for Health Policy, 50 Staniford St, 9th Floor, Boston, MA 02114. E-mail:
- QI =
- quality improvement •
- CQI =
- continuous quality improvement •
- TQM =
- total quality management •
- ED =
- emergency department •
- CDSS =
- computer-based decision support system •
- CF =
- cystic fibrosis •
- NCQA =
- National Committee on Quality Assurance •
- HEDIS =
- Health Plan Employer Data Set •
- CME =
- continuing medical education
- The President's Advisory Commission on Consumer Protection and Quality in the Health Care Industry. Quality First: Better Health Care for All Americans: Final Report to the President of the United States. Washington, DC: 1997
- Worrall G,
- Chaulk P,
- Freake D
- Merritt TA,
- Palmer D,
- Bergman DA,
- Shiono PH
- Evans D,
- Mellins R,
- Lobach K,
- et al.
- Summer T,
- Taylor DK,
- McDonald M,
- et al.
- Vardi A,
- Modan B,
- Blumstein Z,
- Lusky A,
- Schiff E,
- Barzilay Z
- Duggan AK,
- Starfield B,
- DeAngelis C
- Stewart MG,
- Harrill WC,
- Ohlms LA
- Banaszak P
- Hawkins JA, Minich LL, Tani LY, Sturtevant JE, Orsmond GS, McGough EC. Cost and efficacy of surgical ligation versus transcatheter coil occlusion of patent ductus arteriosus. J Thorac Cardiovasc Surg. 1996;112:1634–1638; discussion 1638–1639
- Turley K, Tyndall M, Turley KM, Roge C, Cooper M, Tarnoff H. Cardiovascular-radical outcome method is effective in complex congenital cardiac lesions. Ann Thorac Surg. 1996;62:386–391; discussion 391–392
- National Asthma Education and Prevention Program. Expert Panel Report 2: Guidelines for Diagnosis and Management of Asthma.Bethesda, MD: National Institutes of Health; 1997. Publ. No. 97-4051
- Palmer RH, Louis TA, Peterson HF, Rothrock JK, Strain R, Wright EA. What makes quality assurance effective? Results from a randomized, controlled trial in 16 primary care group practices. Med Care. 1996;34:SS29–SS39
- Quattlebaum TG,
- Darden PM,
- Sperry JB
- O'Brien G, Lazebnik R. Telephone call reminders and attendance in an adolescent clinic. Pediatrics. 1998;101(6). URL:http://www.pediatrics.org/cgi/content/full/101/6/e6
- Alemi F, Alemagno SA, Goldhagen J, et al. Computer reminders improve on-time immunization rates. Med Care. 1996;34:OS45–OS51
- Yarnell K,
- Rimer BK,
- Hynes D,
- et al.
- Szilagyi PG,
- Rodewald LE,
- Savageau J,
- Yoos L,
- Doane C
- Cheng TL,
- Ottolini MC,
- Baumhaft K,
- Brasseux C,
- Wolf MD,
- Scheidt PC
- Casey R,
- Rosen B,
- Glowasky A,
- Ludwig S
- Cowan FJ,
- Warner JT,
- Lowes LM,
- Riberio JP,
- Gregory JW
- Madge P,
- McColl J,
- Paton J
- Palmer RH, Hargraves JL, Orav EJ, Wright EA, Louis TA. Leadership for quality improvement in group practices. Med Care.1996;34:SS40–SS51
- Hickson GB,
- Altemeier WA,
- Perrin JM
- Leibowitz A,
- Manning WG Jr.,
- ., Keeler EB, Duan N, Lohr KN
- Kuhlthau K,
- Walker DK,
- Perrin JM,
- et al.
- Bethell C, Peck C, Schor E. Assessing health system provision of well child care: the Promoting Healthy Development Survey.Pediatrics. 2001. In press
- Clark NM,
- Gong M,
- Schork MA,
- et al.
- Bertakis KD
- Kelly B,
- Sein C,
- McCarthy PL
- Rubin DH,
- Leventhal JM,
- Sadock RT,
- et al.
- Hackett AF,
- Court S,
- Matthews JN,
- McCowen C,
- Parkin JM
- Moller JE,
- Simonsen EH,
- Moller M
- Turley K, Tyndall M, Roge C, et al. Critical pathway methodology: effectiveness in congenital heart surgery. Ann Thorac Surg.1994;58:57–63; discussion 63–65
- Shortell SM,
- Bennett CL,
- Byck GR
- Blumenthal D, Kilo CM. A report card on continuous quality improvement.Milbank Q. 1998;76:625–48:511
- Sisk JE
- Davis DA,
- Taylor-Vaisey A
- Copyright © 2001 American Academy of Pediatrics