 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
PEDIATRICS Vol. 107 No. 1 January 2001, pp. 143-155
,
From the * Institute for Health Policy and the Center for
Child and Adolescent Health Policy of Massachusetts General Medical
School, Boston, Massachusetts; and the § Agency for Health Care
Research and Quality, Silver Spring, Maryland.
 |
ABSTRACT |
|---|
Top
Abstract
Methods
Results
Discussion
Conclusion
References
|
|---|
Objective. Improving the quality of health care is a national priority. Nonetheless, no systematic effort has assessed the status of quality improvement (QI) initiatives for children or reviewed past research in child health care QI. This assessment is necessary to establish priorities for QI programs and research.
Methods. To assess the status of QI initiatives and research, we reviewed the literature and interviewed experts experienced in QI for child health services. We defined QI as activities intended to close the gap between desired processes and outcomes of care and what is actually delivered. We classified reports published between 1985 and 1997 by publication characteristics, study design, clinical problem addressed, site of intervention, the QI method(s) used, and explicit association with a continuous quality improvement program.
Results. We reviewed 68 reports meeting our definition of QI. More than half (48) were published after 1994. The reviewed reports included controlled evaluations in 36% of all identified interventions, and 3% of the reports were associated with continuous quality improvement. QI methods demonstrating some effectiveness included reminder systems for office-based preventive services and inpatient pathways for complex care. Reportedly successful QI initiatives more commonly described improvement in administrative measures such as rate of hospitalization or length of stay rather than functional status or quality of life. Interviews found that barriers to QI for children were similar to those for adults, but were compounded by difficulties in measuring child health outcomes, limited resources among public organizations and small provider groups, and relative lack of competition for pediatric tertiary care providers. Research and dissemination of QI for children were seen as less well developed than for adults.
Conclusions. Attempts to improve the quality of child health services have been increasing, and the evidence we reviewed suggests that it is possible to improve the quality of care for children. Nonetheless, numerous gaps remain in the understanding of QI for children, and widespread improvement in the quality of health services for children faces significant barriers. Key words: child, adolescent, quality, quality improvement, CQI, TQM, guidelines, benchmark, review.
Recent reports highlight the urgent need to improve the
quality of health care.1,2 There is sufficient evidence of
deficiencies in the quality of child health services to include
children in the movement to improve the quality of health
care.3,4 However, the unique circumstances of children
suggest the need to evaluate efforts to improve the quality of child
health services separately from adults.3,5 Specifically,
children have distinct provider systems and different epidemiology of
illnesses. In addition, children are generally dependent on caregivers
for access to health care, and children have a range of developmental
issues not relevant to health care for adults. The time is ripe,
therefore, for an evaluation of what is known about quality improvement
(QI) in the arena of child health and the implications of that
information for future efforts to improve the quality of care that
children receive.
Past efforts to review strategies to improve care do not focus on
children or the delivery of care to this population. Recent reviews of
the literature on effective strategies to change practice find evidence
that some strategies work better than others.6-9 For
example, physician behavior change seems to depend more on real-time
reminders and effective leaders and less on passive education. The
extent to which these findings apply to child health services is
unknown. The review of QI for children by Homer and colleagues10 provides a useful conceptual framework for
addressing QI for children and identifies key areas of concern.
Nonetheless, this review does not consider the available evidence on
past or ongoing initiatives to improve the care of children.
In this study we attempt to address gaps in the current understanding
of the status and prospects of QI in children through a review of the
literature on this topic and through interviews with expert observers
of QI in child health care. For these activities, we define QI
activities as "interventions that are designed to close the gap
between desired processes and outcomes of care and what is actually
delivered ... QI interventions may be directed at any group of
participants in the delivery system: administrators, physicians,
laboratory technicians, parents, children, employers."4
One way of operationalizing this definition is to identify QI as
interventions intended to decrease the underuse, overuse, and misuse of
an intervention.1 Such interventions take at least 3 forms. The interventions may be directed at improving the care provided
for a specific clinical problem or patient population or procedure; may
test a generic tool or method for QI that can be applied to a range of
clinical problems, populations, or procedures; or may be more holistic: aimed at enhancing performance of an entire organization or health care
system. In this study, we review evidence on the status of all 3 types
of QI activities.
Literature Review
Identifying QI Literature
Using Medline, Healthstar, and the Cochrane Library, we searched
the terms frequently associated with QI efforts and combined each with
"child" and "adolescent." The search terms included: quality
improvement, evaluation, satisfaction, case manager/management, disease
management, benchmarking, opinion leader, guideline, critical path/pathway, reminder, continuous quality improvement, and total quality management. Because we hypothesized that a significant proportion of the QI literature may be published in journals related to
health care management and health care quality, we reviewed the article
titles for the following journals: American Journal of Medical
Quality, Joint Commission Journal on Quality Improvement, Hospital and
Health Services Administration, Quality Assurance in Health Care,
Quality Management in Health Care, Journal of Clinical Outcomes
Management, Quality Assurance and Utilization Review, Quality in Health
Care, Quality Assurance, Journal of Quality in Clinical Practice, and
the Journal of Nursing Care Quality. Finally, we reviewed the
references of selected articles and solicited suggestions for articles
from child health services researchers and directors of QI programs.
METHODS
Top
Abstract
Methods
Results
Discussion
Conclusion
References
Inclusion Criteria Articles were selected for review if they met the following criteria: 1) the paper was published in English between 1985 and 1998, 2) at least 1 stated goal of the intervention was to improve the quality of care for children or adolescents <18 years old, and 3) the report included a quantitative assessment of the primary outcome. For our purposes, QI was defined as any intervention designed to decrease underuse, overuse, or misuse of an effective intervention. This operational definition left several studies equivocal with respect to their inclusion. Difficult studies to classify included public health or community-based interventions designed to decrease high-risk behaviors.11,12 Although significant improvements in child health may come from such interventions,13 we considered them to be tests of new interventions rather than attempts to improve the delivery of existing health care services and therefore excluded them from our review.
Classification and Analysis We recorded the study characteristics included in Table 1 for each of the reports we included. We used 3 categories of study design: 1) randomized controlled, 2) non-randomized controlled, and 3) before-after. We used the designation of before-after to categorize studies that evaluated a change in a process or outcome over time where no prospective control was used as a comparison (eg, pre-post, control chart, time series). We identified the common methods used in efforts to improve quality of care and categorized each report by the methods used. QI programs frequently used multiple tools and may cross site or organizational boundaries, thus complicating attempts to classify programs. In addition, we searched each included report for a reference to an institutional commitment to improve the quality of care at the site where the QI intervention was conducted. Reports with such a reference were categorized as continuous quality improvement/total quality management (CQI/TQM).
|
Interviews
We report also the results of 9 semistructured telephone or face-to-face interviews with individuals involved with QI, conducted to assess past and current child QI initiatives. Interviewees represented a distribution of academic, public health, and private provider organizations, as well as different geographic locations. The interviews included questions that addressed 1) the scope of child QI activities (nationally and at the interviewee's institution), 2) the success of child QI activities, and 3) barriers to child QI. Copies of the instrument used to guide the interviews are available from the corresponding author. All calls were taped with permission from interviewee.
| |
RESULTS |
|---|
|
Top
Abstract
Methods
Results
Discussion
Conclusion
References
|
|---|
Literature Review
We present results of our literature review from 4 perspectives: 1) overview of literature based on publication characteristics, design characteristics, and site of care, 2) the QI methods used, 3) the clinical condition addressed, and 4) the use of CQI.
1. Overview of Literature
Our review of the literature identified 68 publications that met our inclusion criteria. Table 2 presents 40 controlled trials of child QI published between 1980 and 1998. There was a dramatic increase in the number of studies published in the past few years, with 50% of reports published since 1994. Controlled studies were more likely to have been published in 1 of 3 journals (Pediatrics (7), Archives of Pediatrics and Adolescent Medicine (4), Medical Care (4)), whereas uncontrolled studies were more typically found in 1 of the quality of care journals (see methods). Controlled studies were more likely to report external funding (86% vs 13%) and more likely to report negative results among the primary endpoints (28% vs 18%). A majority of the US studies were located in a relatively small number of cities.
|
Only 22% of the reports used a randomized evaluation of the intervention. Most others used a before-after design to examine the influence of the intervention on the outcomes measured.
A majority of the interventions (68%) were directed at younger children (age <5), primarily because of the preponderance of studies on newborn care, hospitalized children, and immunizations. Half (48%) of the controlled trials were conducted in office settings.
2. The Methods of QI
Guidelines/Pathways Guidelines were defined as a set of rules intended to define appropriate care and guide practice. A pathway (or carepath or critical path) was defined as a method of documenting the processes of care and identifying reasons for variance from previously defined objectives. Because most efforts to improve care require some decisions about what is the best possible care, it may be argued that all QI requires guidelines of one sort or another. For example, all efforts to improve immunization practices rely on guidelines for correct practice. Our attempt to identify reports of guidelines and pathways in pediatrics was restricted to those reports that use language describing an organized rule-based approach to improving care.
We identified 7 controlled trials of guidelines or pathways for child health16-22 and 15 uncontrolled evaluations.23-37 Most of these reports found improvements in the outcomes measured, including length of hospitalization, test ordering, admission rates, or appropriate medication use. Measures related to functional status, quality of life, or even missed school days were rare. The studies of guideline implementation reveal that although the majority were applied in the hospital setting, they were increasingly applied and evaluated in ambulatory settings. Studies of particular note included Evans' report17 of an educational intervention to improve the care of children with asthma by raising physician awareness of the National Asthma Education Promotion Program guidelines38 for asthma care. This intervention, in an inner-city environment where the social barriers have been depicted as limiting attempts from providers to improve care, increased the use of antiinflammatory medications from 2% to 25% and dramatically increased the percentage of parents who reported receiving education about asthma. Other statistically significant effects include more scheduled visits per patient per year (1.85 vs 0.88) and more patients prescribed a
-agonist metered-dose inhaler (MDI) (52% vs
15%).
The combination of a guideline with a particular method of delivering
the guideline to patients and/or physicians was a recurring theme in
the studies of guidelines reporting a successful intervention. Palmer
and colleagues' reported a randomized controlled trial of the
influence of guidelines and performance feedback on primary care
physicians (including pediatricians).39 These
investigators found no benefit to guidelines alone, but guidelines
combined with performance feedback led to significant improvement in
well-child care, gastroenteritis management, and otitis media
management.
Other effects from the controlled trials of guidelines with feedback
included: improved preventive counseling delivered in a primary care
practice,21 reduced length of stay on a general pediatrics
ward,18 fewer preoperative lab studies in outpatient child
surgery,22 improved outpatient follow-up after an
emergency department (ED) visit (47% vs 24%), and a reduction in
unjustified hospital days.20 One controlled trial reported
an unsuccessful intervention. Rodewald and colleagues16
attempted to improve immunization rates during ED visits, but reported
a high parental refusal rate and inability to ascertain accurate
immunization status as barriers to the intervention.
A report detailing the improvements associated with using a clinical
pathway for hospitalized patients with asthma illustrated the
difficulty of assessing QI based on a literature search
(www.ihi.org/resources/eyeoi/casestudy.asp). This
report, found only on the web, described (uncontrolled) improvements in
several outcomes, such as length of stay (3.1 days to 1.75 days) and
readmission (25% to 5%), patient satisfaction with care and asthma
knowledge, and reduced unnecessary testing (75% reduction). Kwan-Gett
and colleagues had much less success with their inpatient asthma
pathway,36 and it was difficult to ascertain possible
reasons for these disparate findings based on the information provided.
Reminder Systems Reminder systems were frequently implemented as part of a guideline or pathway, but they are also used in isolation to solve a particular problem. Reminders at the time that care is provided showed relatively strong positive results. For example, 2 trials demonstrated dramatic increases in preventive services delivered when encounter forms prompted clinicians at the time of a patient visit.21,40
Other demonstrated effects from controlled trials of reminder systems included increased attendance at primary care office visits for children41,42 and adolescents43; as well as increased attendance for child psychiatry visits44; improved immunization rates45-49; improved interaction between patient and physician50; decreased hospital length of stay18; and improved follow-up after ED visit19,51,52; TB testing53,54; and acute illness visits.55 There has been sufficient agreement about the ability of reminder systems to improve vaccination rates that the Centers for Disease Control and Prevention (Advisory Committee for Immunization Practices), the American Academy of Pediatrics, and the American Academy of Family Practice recommended that all providers, public and private, use reminder systems to increase vaccination rates.56 When this recommendation was announced, only 25% to 35% of pediatricians had such systems in place.56 Hunt reviewed the literature of computer-based decision support systems (CDSS) and identified 68 controlled trials.57 CDSS were frequently used in conjunction with reminder systems, although they are also used as diagnostic aids and references. The review concluded that CDSS enhance clinical performance for drug dosing and preventive care. Only 1 of the trials they identified was performed exclusively with children and child providers, and only 5 of the trials included children at all.Disease Management/Case Management Disease management has been referred to as "a systematic, population based approach to identify persons at risk, intervene with specific programs of care, and measure clinical and other outcomes."58 Usually disease management involved identifying a cohort of patients with a similar disease in a population and applying strategies such as education or medication adherence reminders to the entire group. This approach may have limited applicability in children with the exception of asthma, where disease management has become almost commonplace.59 Nonetheless, we found no reports meeting our criteria describing the effectiveness of this type of intervention for asthma.
There were reports of disease management techniques used for other pediatric diseases such as cystic fibrosis (CF). Finkelstein and colleagues demonstrated the feasibility of employing a home monitoring system for children with CF similar to those used for adult patients with congestive heart failure,60 and Kretz reported both clinical improvement as well as a 33% savings with a disease management model applied to children with CF.61 Case managers have provided a variety of functions in inpatient and outpatient settings as they shepherd individual patients through the medical system. Case managers may link public and private health systems in the care of high-risk patients.62 This study was remarkable for the rare glimpse of what has been possible when unlinked systems become connected in the care of mothers and infants, as well as the study's randomized design. The intervention increased prenatal visits and use of primary care physicians as a regular source of sick care decreased waiting time for appointments, and increased patient education. Children with newly-diagnosed diabetes spent fewer days in the hospital when they had a specially trained nurse case manager assigned to their care,63 and children with asthma had fewer repeat admissions when the family received inpatient nursing instruction in self-management.64Physician Leaders and Academic Detailing Physician leaders or local opinion leaders were individuals within a group who have been identified by their colleagues as influential. Only 1 study of the role of physician leaders in QI for children was found.65 Palmer's analysis provided empirical evidence of the impression frequently found in the QI literature that a committed physician leader makes a difference in attempts to improve the quality of care.
Physician Financial Incentives Despite evidence that financial incentives for child health care providers are related to quality of care,66,67 we did not find literature describing a program that used this technique as a means to improve the quality of child health care. This lack of evidence was striking considering the increasingly frequent use of financial incentives by insurance companies to influence physician practices.68
Benchmarking and Provider Performance Assessment Benchmarking and provider performance assessment (sometimes referred to as profiling or audit and feedback) referred to the reporting of clinical and/or economic measures to a provider (or group of providers) with comparisons to colleagues (or other groups). This method has often been used in conjunction with guidelines. We identified only 1 controlled study of the effectiveness of this QI method in the care of children.39
A meta-analysis of profiling studies69 found a statistically significant but minimal effect on physician practice (odds ratio 1.091, 95% confidence interval, 1.045 to 1.136). None of the studies Balas et al identified address physicians caring for children. Efforts to produce a common set of quality measures that can be used for benchmarking and profiling are underway. The National Committee on Quality Assurance's (NCQA) health plan assessment mechanism (HEDIS) reveals major variations in quality of care by health plans, using several measures of child health quality such as immunization status. Limits to the HEDIS approach to measuring quality for children's health care partly reflect the small number of child measures currently available in the HEDIS measurement set.70 A child version of the Consumer Assessment of Health Plans Survey has been added to the HEDIS measures.71 NCQA and the Foundation for Accountability have been developing more child and adolescent measures.72Continuing Medical Education (CME) We defined CME narrowly to include only educational interventions where the physician was a passive recipient of knowledge (for example grand rounds or a CME course). We found several controlled studies of the influence of CME on physician performance and patient outcomes in the care of children.17,73,74 These carefully conducted studies all described a significant change in provider behavior that improved both physician practices as well as patient outcomes in areas such as medication compliance and patient satisfaction. Although these studies were encouraging, a recent systematic review of CME by Davis and colleagues concluded that CME alone has little influence on provider behavior or patient outcomes.7 The success of the interventions noted above may reflect the careful development of the educational process or physician awareness of being observed.
Patient Education Educating patients and their caregivers to improve their understanding of their health, illnesses, and appropriate use of health services has the potential to improve the quality of their care. The literature included numerous reports of educational interventions for both children and parents, but many of these addressed populations outside the health care system (for example, smoking cessation for adolescents in school75), and these studies were outside the scope of this review (see "Methods"). Nonetheless, we found several studies of educational interventions in patient care settings.54,76-80 Controlled studies have shown decreased exposure to environmental smoking,81 decreased home hazards after safety education,82 and increased compliance with ED follow-up.51,52
Controlled studies of educational interventions for patients with asthma64,77,83-86 and diabetes87,88 showed increased knowledge and/or self-care, but little or no effect on morbidity. Exceptions included the study by Hackett and colleagues that found a small but significant decrease in HbA1c in children over 11 years old,87 and the study by Madge and colleagues that reported decreased admissions for asthma.64 A recent metaanalysis of randomized controlled trials of patient education for asthma confirms the generally poor performance of patient education interventions.89 Several studies focused on the influence of education on appropriate use of acute care services.19,52,90 These studies generally reported positive results on patient satisfaction and decreased utilization where each is reported. One of the rare randomized trials of patient education found no influence of physician counseling during a child ambulatory visit on the purchasing of bicycle helmets.91 The studies we reviewed demonstrated a relatively strong influence on measures of patient satisfaction and some influence on knowledge, but little influence on patient behaviors or health outcomes.3. Child QI and Clinical Areas
The clinical categories of prevention, acute care, and chronic care are frequently used for discussing QI. Nonetheless, this classification has some limitations. Importantly, QI programs are typically initiated in a specific site with a specific group of people, whereas the preventive care, acute care, or chronic care a child receives may come from a variety of sites or even unconnected organizations.
Prevention Among the most common QI interventions in the literature were attempts to improve prevention practices. Reminder systems to improve physician immunization, screening, and counseling practices (see above) showed the greatest improvements in process measures of any of the types of interventions we reviewed. Prevention occurred not only in physicians' offices but also through home visiting62,92 and school-based programs.93 Although a majority of the studies we identified for this review report a benefit from the intervention, the studies that addressed child risk factors outside the physicians' office had the poorest outcomes as a group, possibly because these trials have attempted to change very difficult behavior problems.
Acute Illness Reports of improvements in acute illness care originated primarily from EDs and intensive care units. Improvements in patient satisfaction with ED care94,95 as well as the clinical care of patients with trauma96 were reported. The implementation of guidelines and pathways for outpatient surgery, complex tertiary care such as cancer care and surgery for congenital heart disease, and care for acute asthma demonstrated some improvements in process measures as noted above. The primary outcome used in most of the inpatient studies is length of stay, and length of stay does not have a direct relationship to quality.
Chronic Illness Ninety percent of the studies (and 6 of 11 controlled trials) concerning the care of children with chronic illnesses involve asthma. As the most common chronic condition of children, asthma has been the principal vehicle for the study of different QI methods. Nearly all the uncontrolled reports presented beneficial effects on hospital readmission, length of stay, and appropriate medication use. Children entered into these programs usually had a dramatic (up to 60%) decrease in utilization regardless of any intervention. The controlled trials also showed significant improvements, but with more modest magnitude than in the uncontrolled literature.
Other chronic conditions where QI has been applied include cystic fibrosis,60,61 leukemia,97 congenital heart disease,35 renal failure,98 and diabetes.634. CQI/TQM
Nine reports of interventions described a CQI process. These reports focused on complex hospital-based patient care issues such as invasive cardiology,99 cardiac surgery,35,100 leukemia,97 and pediatric peritoneal dialysis.98 Three reports described the use of CQI methods to implement guidelines in primary care practice settings. Carlin reported increasing immunization rates from 53% to 86% in 19 clinics in Minnesota.24 Solberg used CQI to assist with the implementation of guidelines for preventive services,101 and Gibson reported success using CQI to implement asthma care guidelines.102
Two reviews examined the successes of CQI/TQM in health care institutions.103,104 These studies did not consider children's hospitals or child providers separately and therefore do not help to assess the special circumstances of child health care services, which may affect their success or failure. Shortell, Bennett, and Byck's review of the evidence for CQI104 included only 2 reports of CQI for children out of the 55 studies they identified,24,105 although several other studies they cite included children as part of hospital wide CQI programs.
| |
FINDINGS FROM INTERVIEWS |
|---|
Interviews of QI experts provided information on the scope of child QI activities, how successful they have been, determinants of success or failure, barriers to improving the quality of health care for children, and important unanswered questions for the future of child QI.
The Scope of Child QI Interviews indicated the widespread use of at least a few QI programs for child health care services in hospital settings, both children's hospitals and large general hospitals, as well as many managed care organizations. The extent of an institution's commitment to child QI seemed approximately proportional to that institution's commitment to providing child health care services. Large children's hospitals may have extensive activities, whereas smaller organizations, general hospitals, and health maintenance organizations were more likely to limit QI to an asthma pathway and/or monitoring immunization rates. This view was supported in part by a study of child QI in emergency rooms by Gausche and colleagues.106 Respondents agreed that inpatient QI programs were significantly more developed than outpatient programs. Respondents also agreed that institutions primarily seek cost reductions through QI and do not generally support QI initiatives without potential cost savings for the institution (except for strategies to improve patient satisfaction).
Large provider organizations seemed to adopt improvement activities to 1) improve patient satisfaction and 2) decrease unnecessary variations in the provision of care. Programs to improve patient satisfaction included patient satisfaction surveys followed by interventions to address identified deficiencies. The most frequently cited deficiencies in patient satisfaction for child health care services included discharge planning, waiting times for appointments, waiting times in the ED, and clear explanations of tests and procedures. The second common QI activity involved attempts to decrease unwanted variation in the care of patients through the implementation of guidelines and critical pathways. Many children's hospitals had some guidelines/pathways in place for the most expensive and complex areas of care such as bone marrow transplantation, cardiac surgery, and neonatal intensive care unit care. Asthma care has often been one of the first and most extensive pathways in hospitals and managed care organizations because of the high frequency of admissions (and therefore high cost) and the availability of interventions that were known to decrease admissions. Asthma was the only child condition where the methods of disease management were commonly applied. Some regional referral centers have developed guidelines and pathways for uncommon conditions because of their relatively high volume of these conditions, reflecting the large population base of their referral areas. Some large holes in the spread of QI were apparent from our interviews. QI in small group practice settings and community hospitals was rarely more than the occasional chart audit necessary for licensing. Little has been done to improve the coordination of care of children with chronic illnesses, in part attributable to the poorly defined responsibilities of specialists and generalists in the care of these patients. Despite the high prevalence of mental health problems among adolescents, our respondents were generally unaware of programs addressing the quality of care this population receives. Hospital-based error reduction programs (including especially medication errors) has received almost no attention in pediatrics. Finally, few programs assessed effectiveness with measures of child health or quality of life. Our respondents described numerous examples where a gap between knowledge and performance was closed. Frequently, however, these improvements were in administrative areas such as hospital length of stay.What Are the Barriers to QI in Child Health Care Services? Several common elements have conspired against improving the quality of child health care services. Some barriers noted by our respondents have been described in previous more general assessments of QI in medicine.107 These barriers included a lack of essential resources, conflicting incentives among clinicians and administrators, administrative instability in a time of rapidly changing organizational structures, and lack of cooperative skills among physicians.
Certain minimum infrastructure requirements for successful QI included support from a behavioral scientist, knowledge of billing systems, dedicated assistance in the collection of data, and access to a data analyst. Respondents observed several additional requirements for success. Senior managers had to support QI, and they must insist that everyone in their organization at least be available to QI. The importance of good ideas for how to make an improvement was often overlooked. Physician champions were necessary to support QI among clinicians who are often skeptical or disinterested. Making the recording of quality information a routine part of everyone's job also facilitated QI. Certain aspects of the care of children may present greater barriers to QI in children than in adults. The low volume and rare negative outcomes of most childhood diseases may not allow for the data necessary for rapid cycle change. In addition, child health reflects services from institutions outside the personal health care system, such as public health services, school health programs, and child protection services. Although investigations into linking public and private delivery services have been reported,62 QI methods have not been well developed to address this situation.104 A large proportion of child health services occurred in office settings, raising 3 issues that limit opportunities to improve the quality of child health. First, electronic outpatient information systems are poorly developed at this time and are still at least several years away from being widely disseminated. Second, small groups and solo practitioners did not have the resources to invest in QI. Third, the delivery of primary care to children was divided between family practitioners and pediatricians. Several respondents describe difficulties getting these 2 groups to cooperate even when they co-exist within the same organization. Clinicians caring for children often do not perceive a problem with the quality of care they provide. Although providers of adult services may also fail to appreciate a problem with quality, this situation may be more problematic for child providers. The more limited data on the quality of child health services may make convincing pediatricians of the need to change more challenging. Respondents also reported that child health professionals often followed an advocacy model rather than an evidence-based model of child health resource allocation, so QI efforts that attempted to alter services based on data sometimes met hostility. Competitive factors may have affected QI for adult and child services differently. The importance of competition for motivating QI has been stressed in the literature of QI.104,108 Although competition may have affected primary care providers' attention to QI, children's hospitals and other tertiary care child health units were often the sole providers of certain child health care services within a region. This monopoly may have limited the extent to which the leaders of those organizations perceive the need to improve quality. The lack of involvement of minority QI researchers was seen as a barrier to developing effective interventions to improve the care of the growing child and adolescent minority populations. There were numerous quality issues in minority populations, and nonminority researchers and QI program directors may have had difficulty with access to these populations. Finally, respondents agreed that there has been a significant deficit of research in the area of QI in child health. This deficiency makes their work significantly more difficult and frequently requires pursuing an inefficient trial and error approach to QI. There is much still to be learned about how best to go about the business of improving quality of care for children.| |
DISCUSSION |
|---|
|
Top
Abstract
Methods
Results
Discussion
Conclusion
References
|
|---|
This review of QI for child health care assesses QI efforts to date and provides information to guide future initiatives. Our review of the literature and interviews provide evidence that it is possible to improve the quality of health care for children. We find the recent growth in child QI encouraging. Nonetheless, the limited distribution of QI and the limited resources applied to it also limit impact on the quality of care received by children in the United States. Numerous obstacles hinder widespread improvement in the quality of child health services.
The Methods of QI
Evidence for the effectiveness of guidelines and pathways in clinical practice has grown substantially since Merritt and colleagues reviewed this topic.14 The recent increase in reports of improvements in processes of care using guidelines and pathways in tertiary settings for complex care supports their more widespread use. Nonetheless, our interviews caution against the potential for considerable wasted effort when guidelines lack proper development and implementation. Active interventions to guide providers at the point of care delivery seem to be an essential component of this QI method.
The evidence concerning disease management, case management, physician leaders, physician financial incentives, and benchmarking/profiling is insufficient to formulate a conclusion about their usefulness. Although the literature and interviews suggest that these techniques have potential in certain circumstances (eg, disease management for asthma), the extent to which they should be part of a comprehensive QI program is unclear. The lack of controlled studies is striking given the apparent increasing use of these techniques by managed care organizations.
Physician education(CME) and patient education seem to be weak methods for changing behavior, but they have an apparent role in patient and provider satisfaction. Efforts to improve the effectiveness of educational interventions may be served by incorporating additional QI methods.9,109
Clinical Areas
The clinical areas that have benefited from effectiveness research and QI include the delivery of office-based preventive services and to a lesser extent complex inpatient care. Strong evidence supports the use of preformatted encounter forms and automated reminder systems to improve the quality of preventive practices. The literature and interviews indicate the potential to improve neonatal intensive care unit and ED care. In addition, our interviews suggest that regional and national networks of providers may be important sources for information and dissemination of QI, facilitating more widespread adoption of QIs in child health services. There is significant room for research into improving the quality of acute care services for children.
Several trials of QI in the care of children with chronic conditions indicate some promising results. In particular, evidence suggests the care of children with asthma could be dramatically improved with widespread adoption of QI techniques. Widespread adoption would be expected to eventually result in improvements in community based measures of asthma morbidity and hospital admissions. Opportunities for future efforts might well focus on higher prevalence conditions, especially adolescent mental health.
CQI/TQM
We find insufficient assessments of CQI efforts. Most reports describe successes, but with limited evaluations. Our interviews indicate cautious enthusiasm for the CQI initiatives to improve the quality of child health care.
Comparing Child and Adult QI
An underlying theme to our review is whether or not QI for children is different from QI for adults. There is no reason to think that systems like preprinted or computerized medication order forms would be less effective at reducing medication errors in children than they have been for adults. However, it remains to be established whether some QI methods suit the circumstances of child health care providers better than others do.
In many ways, QI for children seems from this review to be similar to QI for adults. The barriers to QI we found from our interviews have several similarities with those found by Blumenthal and Kilo107 and Shortell.104 The difficulties associated with ambulatory settings, cooperating across organizational boundaries, poor information systems, rare outcomes, professional resistance, lack of competition, and difficulties with measurement have all been identified previously as barriers to QI. Why then do our respondents find QI for children more difficult than QI for adults? One possibility is that the same barriers pose relatively greater challenges for child QI than adult QI. Child QI may therefore require proportionally more resources than adult QI to be equally effective. QI for children may be doomed to limp along given the tendency to focus QI on high cost areas.
Limitations
There are several important limitations to this assessment of QI for child health services. The literature rarely describes failed interventions, despite the finding in our interviews that such failures are relatively common. In addition, we limited our interviews to experts with extensive knowledge of QI for children. We did not systematically survey child health institutions to determine the true scope of QI in child health services.
Difficulties associated with reviewing research on QI include unclear definitions of what constitutes QI activities, the lack of a system for classifying QI, and the low likelihood of publication of much QI work. In addition, many of the publications we cite are not among those initially identified in our search, suggesting that there may be a significant number of additional publications not included in our review.
Our analysis uses a novel method for the classification of reports of QI. Although the scheme we use has potential as a means to organize QI literature, it is limited by the lack of established criteria for many of the QI methods used such as "reminder systems" and "disease management." Nonetheless, this classification scheme should allow for the application of metaanalytic techniques to arrive at quantitative estimates of the success of QI interventions from the 3 perspectives of QI methods, clinical areas, and association with CQI. A library of clinical effectiveness and QI literature that identifies literature based on the categories we use may be a valuable resource for researchers and managers interested in QI.
| |
CONCLUSION |
|---|
|
Top
Abstract
Methods
Results
Discussion
Conclusion
References
|
|---|
There are some reasons to be encouraged about the state of QI work in child health services. Published reports indicate that substantial improvements in the care of children can be made through QI. QI activities seem to be widespread at large provider institutions, and the publication of QI research is accelerating. Several exemplary collaborations are modeling the kind of cross-institutional work that is probably going to be necessary for widespread QI for children. There is sufficient evidence about the effectiveness of several specific interventions such as recall/reminder systems for improving the delivery of preventive services that wide dissemination is virtually guaranteed to improve the quality of care for children. The widespread use of the child supplement to the Consumer Assessment of Health Plans Survey as well as the current Foundation for Accountability/NCQA collaboration on the development of child quality measures should assist in making performance comparisons between large provider organizations. These comparisons should help stimulate QI efforts.
But there are also challenges ahead if efforts to improve the quality of health care for children are to have a significant impact on child health. Our literature review highlights the limited set of information on effectiveness of QI that clinicians and administrators interested in QI can turn to. There is insufficient evidence in most of the methodological categories we assess, and several important research questions await investigation. Child QI will need to move beyond administrative process measures and take on child health outcomes like quality of life and optimal development. Our interviews highlighted the very real danger of QI becoming an additional administrative task with little impact on quality of care.
It is disconcerting that with what we know about deficiencies in the quality of care for children and the extent of some local attempts to improve care, that widespread QI for child health care services remains elusive. There is ample opportunity to do better.
| |
ACKNOWLEDGMENTS |
|---|
This work was supported by the Agency for Health Care Research and Quality.
Dr Ferris is a Fellow of the Pediatric Scientist Development Program (AAP/NICHD: K12-HD00850).
We thank the following persons for their contribution(s) to the interview portion of this research: Charles Homer, MD, Chistina Bethell, PhD, John Pestian, PhD, Donald Berwick, MD, Nira Bonner, MD, Mark Kirschbaum, PhD, Paul Kurtin, MD, Jill Joseph, MD, and Heather Palmer, MB, BCh. We also thank John Ausiello, who assisted with the literature search and data abstraction.
| |
FOOTNOTES |
|---|
Received for publication Apr 14, 2000; accepted Jul 19, 2000.
Reprint requests to (T.G.F.) MGH/Partners Institute for Health Policy, 50 Staniford St, 9th Floor, Boston, MA 02114. E-mail: tferrris{at}partners.org
| |
ABBREVIATIONS |
|---|
QI, quality improvement; CQI, continuous quality improvement; TQM, total quality management; ED, emergency department; CDSS, computer-based decision support system; CF, cystic fibrosis; NCQA, National Committee on Quality Assurance; HEDIS, Health Plan Employer Data Set; CME, continuing medical education.
| |
REFERENCES |
|---|
|
Top
Abstract
Methods
Results
Discussion
Conclusion
References
|
|---|
