PEDIATRICS Vol. 103 No. 1 Supplement January 1999, pp. 225-232

SECTION 1: EVIDENCE-BASED QUALITY IMPROVEMENT, PRINCIPLES, AND PERSPECTIVES:
Evidence-Based Guidelines and Critical Pathways for Quality Improvement

David A. Bergman

From the Division of General Pediatrics, Department of Pediatrics, Stanford University School of Medicine, Palo Alto, California.

	ABSTRACT

Top Abstract Conclusion References

Clinical practice guidelines have a long and distinguished tradition in pediatrics. Currently, the American Academy of Pediatrics has developed more than 15 practice guidelines and more than 250 clinical policy statements. In the past, practice guidelines have been used to improve care through the dissemination of evidence-based, clinically effective practices to pediatric practitioners. In the current environment this purpose has been broadened to include cost reduction, standardization of practice, and reduction of medical liability. This has led to both confusion and distrust on the part of the pediatrician. Practice guidelines are best understood as a tool to insure that children receive evidence-based care. They are best used in association with a set of outcome and performance measures that provide feedback to clinicians and allow for modification of the guidelines to meet the needs of the local patient population. The quality of practice guidelines is directly dependent on the quality of the medical evidence supporting the recommendation. Unfortunately only a small percentage of the evidence supporting practice guidelines comes from randomized clinical trials with the majority of the evidence coming from expert clinical panels. The success of practice guidelines in improving care for children has yet to be convincingly demonstrated. Currently, there is a dearth of well designed studies that document the effectiveness of practice guidelines. Their ultimate effectiveness will depend on both an improved evidence base and effective strategies for rapid dissemination of the recommendations. The development of evidence-based practice guidelines does not insure that it will have a major impact on physician practice. In the past, effective dissemination of new knowledge has been a long process, often taking years. This cycle time can be dramatically shortened through the development of networks of practice sites that share knowledge and experience in the implementation of practice guidelines and the use of strategies that take advantage of key groups in the dissemination process. When used appropriately, practice guidelines can provide an important adjunct to clinical research by facilitating the dissemination of new clinical findings and can provide an important platform for encouraging innovations in patient care.

Clinical practice guidelines have a long and distinguished tradition in pediatrics and perinatology. The American Academy of Pediatrics' (AAP's) guidelines for immunization are regarded by many as one of the first practice guidelines in medicine and one of the most widely recognized and used. Today the AAP has developed more than 15 practice guidelines and more than 250 policy statements which recommend clinical practices.¹ Many of these guidelines are related to the care of pregnant mothers and newborn infants.² Examples include the treatment of jaundice in the newborn, screening guidelines for hypothyroidism,³ breastfeeding, HIV⁴ and other diseases, and the use of antenatal steroids to prevent respiratory disease in the newborn infant. Traditionally, these guidelines have been used as a means to disseminate new knowledge to clinicians. This is best exemplified by the AAP Red Book Committee on Infectious Diseases.⁵ Every 3 years this Committee produces a set of new recommendations for the prevention, diagnosis, and treatment of infectious diseases. These recommendations are widely disseminated to child health experts and have become an important source of new knowledge that can be used to improve practice.

The last 2 decades have brought considerable change to organized medicine. With the advent of managed care and managed care organizations, we have seen draconian efforts to reduce the cost of care, programs to reduce the variability of clinical practice, increased malpractice litigation, and greater involvement of patients in clinical decision-making and treatment.⁶ As a result of these changes, the traditional role of practice guidelines as a means to disseminate new knowledge has been broadened to include such goals as decreasing clinical variability and increasing the standardization of care, cost reduction, patient education, and protection against medical litigation. The role of practice guidelines has become so complex that some goals such as the reduction of costs and the improvement of quality may run at cross purposes. All this has led to considerable confusion and distrust about practice guidelines on the part of the patient and practitioner.

Adding to this problem, is the variability in the quality of the scientific evidence that supports the recommendations in practice guidelines. At one end of the spectrum are practice guidelines such as those developed by the AAP where the recommendations are derived from a careful analysis of the scientific evidence gleaned from a systematic review and synthesis of the world's medical literature. At the other end of the spectrum are proprietary practice guidelines developed by consulting firms that have no discernible evidence base other than the expert opinion of paid consultants. Currently, thoughtful pediatricians are confronted with a Tower of Babel of practice guidelines which are dissimilar in intent, developed with different methodologies, have varying quality in their evidence base, and often have questionable relevance to their patients.

	PRACTICE GUIDELINES AND THE IMPROVEMENT OF CARE

I will attempt to cut through this confusion by clearly stating that the primary goal of practice guidelines in pediatrics is to improve the health of infants and children. This is a premise that is readily acceptable to most pediatricians and clearly puts aside the notion that practice guidelines should be primarily used to reduce cost or protect against litigation. This is not to say that cost reduction and reduced malpractice litigation are not laudable goalsthey are. There are, however, more effective ways of reducing costs in health care through programs designed to reduce waste and improve coordination of care in the treatment of illness.⁷ Similarly there are more effective ways to reduce litigation through such means as improved patient communication, better documentation, and root-cause analysis of adverse clinical events.⁸

If the primary purpose of practice guidelines is to improve health care, what role do they play in this process? First and foremost, we have to recognize that practice guidelines do not provide a complete model for improving health care outcomes. Numerous studies have shown that the development and dissemination of a practice guideline does not guarantee any improvement in the outcome of care.^9-11 We propose a simple model for improving health care: there exists for many health care conditions in this country a gap between what medical science has shown to be effective practice and what is actually done. Health care outcomes will improve for infants and children when current practice incorporates these more effective practices. The role of practice guidelines in this process is to provide a tool for reviewing, synthesizing, and communicating evidence-based practices to the pediatrician. Practice guidelines can then be defined as evidence-based recommendations for care. Practice guidelines should not be confused with critical pathway or care maps. These are implementation tools for the evidence-based recommendations found in practice guidelines. They provide a means of translating the recommended changes in care into a more detailed set of recommendations of who provides what care and when, as well as what resources should be used.

	THE MEASUREMENT OF OUTCOMES AND PERFORMANCE

Because practice guidelines are used to improve outcomes of care, they are often associated with a set of measures that document this improvement. For example, the AAP hyperbilirubinemia guideline¹² is designed to reduce the complications of jaundice in the newborn, decrease rehospitalizations for hyperbilirubinemia, and to decrease the hospital length of stay and the cost of care. Each of these outcomes can be associated with one or more outcome measures that together define an outcome measurement set. The outcome measurement set comprises one of the most important features of a practice guideline.

Frequently a practice guideline will be found to vary in effectiveness between settings and groups of patients. One reason for this variability is that the guideline has not been modified to meet patient and site-specific needs. By collecting outcome data in their own setting, physicians are able to customize the guideline through the collection and feedback of this data on their own patients. For example, a practice guideline for the treatment of hyperbilirubinemia may need to have the thresholds for further assessment and treatment modified to address the ethnic makeup of the patient population and the resources available to the family. Customization and documentation of the guideline that can meet the needs of the targeted patient population are a critical step in successful implementation.

Our model for improvement presupposes that better clinical outcomes will occur when the recommended clinical practices are adopted. The development of performance measures in association with the practice guidelines provides a means to document and inform clinicians that these changes in practice have indeed occurred. A performance measure can be defined as an adherence rate to one or more key recommendations in the practice guideline. For example, a practice guideline that recommends antenatal corticosteroids for prevention of respiratory distress syndrome in infants may have as a performance measure the percentage of eligible mothers that receive corticosteroids. In summary, practice guidelines can be best understood in the context of an improvement model that ensures infants and children receive evidence-based care. They are an important and necessary tool for improvement. They are not, however, sufficient to achieve improvement. They are best used in association with a process that collects outcome information and a set of performance measures that provides feedback to clinicians about their adherence to the recommendations for care. This feedback then allows the clinician to modify the guidelines to more effectively meet the needs of the local patient population.

	THE QUALITY OF EVIDENCE IN EVIDENCE-BASED GUIDELINES

Webster's Dictionary defines evidence as something that furnishes proof. This definition is subjective in interpretation and leaves a wide latitude as to what constitutes sufficient proof. This subjectivity is unfortunately reflected in the type and quality of evidence that supports practice guidelines. This level of proof ranges from evidence obtained from properly randomized controlled studies to opinions from respected authorities. In their review of the evidence for appropriateness of care guidelines, health services researchers at the RAND Corporation (Personal communication, Elizabeth M. Glynn, PhD) found that only 11% of the evidence from their reviews for pediatric conditions came from randomized clinical trials. Seventy-two percent of the evidence came from expert opinion, 10% from clinical panels, and 11% from nonrandomized clinical trials.

Even when high quality studies such as randomized controlled trials are available, there is still the subjective translation of the evidence into recommendations. Recommendations from these studies can vary in strength from good evidence to support or reject the recommendation to insufficient evidence to either support or reject the recommendation. There are also vexing issues involved in interpreting the evidence from the perspective of diverse patient groups and from the perspective patient and family. For example, studies done on an all white male population may not be valid for a group of patients that is more diverse in gender and ethnicity. Similarly, when a recommendation presents options for care the preferred option may depend on patient preference. For example, a practice guideline may conclude that there is not sufficient evidence to choose between a medical and surgical intervention. The AAP practice parameter for the diagnosis and treatment of otitis media with effusion offers the option of antibiotic treatment or surgical intervention after a defined period of watchful waiting.¹³ In this case, the best choice may depend on the preference of the family for one alternative. Perhaps the best effort to date to classify the quality of medical evidence and to rank order the strength of the recommendation has been done by the United States Preventative Services Task Force in their Guide to Clinical Preventive Services.¹⁴ This classification schema is presented in Table 1. Here the quality of evidence is ranked according to the study design with randomized controlled trials being the gold standard and expert opinion occupying the other end of the spectrum. The evidence is also classified as to whether or not the results are sufficiently compelling to lead to a recommendation to adopt or reject the intervention, or if the evidence cannot discriminate between options for care. Even this effort involves a considerable degree of subjective assessment. Despite these shortcomings, practice guideline developers must include statements about the quality of evidence and the strength of the recommendations. These qualifiers enable the individual clinician to decide where a recommendation requires 100% adherence or where a weaker recommendation can allow the choice of several options. Where options are available, decisions can be made in concert with patient preferences with attention to issues of diversity.

	DO PRACTICE GUIDELINES IMPROVE HEALTH CARE OUTCOMES?

The evidence supporting the notion that practice guidelines improve health care outcomes has been mixed. Several investigators have shown that adherence to evidence-based guidelines leads to improvement in the quality of care provided.^15-18 Studies of the impact of critical pathway implementation in neonatology have shown mixed results. Wirtschafter et al¹⁶ was able to demonstrate that written clinical guidelines when used in conjunction with an educational program were able to improve clinical practice patterns and reduce variability in outcomes between institutions within a large health maintenance organization.¹⁰ Thompson and Maringer¹⁹ reported that the use of guidelines that impacted on nursing interventions were able to reduce the length of stay in the neonatal intensive care unit (NICU) by 8% and charges by 14% for selected infants. Several studies, however, have not been able to show a significant impact on health care outcomes. In one of the few controlled trials involving practice guideline intervention, Bergman showed that the implementation of a critical pathway for very low birth weight infants had no impact on clinical outcomes, practice patterns, cost, or utilization of services. Jones²⁰ studied the implementation of NeoMAP critical pathways in extreme premature infants and premature infants with major problems and was not able to demonstrate a statistically significant reduction in length of stay after the implementation. In summary, there is a dearth of well-designed studies in this area. Those studies that do exist are conflicting in their results, and frequently use a predesign-postdesign without a control group. Very few studies have measured the impact of the intervention on clinical outcomes, choosing instead to study adherence rates to the recommendation or the impact on cost and utilization.

	EXAMPLES OF PRACTICE GUIDELINES IN PERINATOLOGY

Practice guidelines currently in use by perinatologists and neonatologists vary in scope, the quality of evidence, and the methodology used in their development. Some of the guidelines with the best evidence base involve recommendations for neonatal screening. Those conditions recommended by the United States Preventive Services Task Force are listed in Table 2.¹⁴ In general, these guidelines are substantiated by a strong evidence base. The baseline prevalence of the condition and the expected utility of screen-no screen strategies have been used to develop a cost-effectiveness analysis to substantiate the recommendation. Of interest is that some of the screening guidelines for which there is a strong evidence base have not been universally mandated (eg, hypothyroidism), whereas for others there is considerable controversy concerning the results of a cost-effectiveness analysis (eg, cystic fibrosis) that have been mandated by some states.²¹ For some conditions (eg, HIV) screening has been mandated but incompletely implemented.

Some of the practice guidelines for antenatal and/or neonatal therapy that are currently in use are listed in Table 3. In general, these guidelines have a strong evidence base but vary considerably in the degree that they have been implemented. For some conditions such as antenatal steroid treatment, the work of Horbar¹¹ and others have demonstrated that a significant proportion of mothers and fetuses do not receive this effective treatment. For other conditions such as maternal and neonatal group B streptococcus infection detection, there is considerable variability in the construction and interpretation of the cost-effectiveness analyses and consequential variation in the type and strength of the recommendations.^22,23 There have also been guidelines that are concerned with appropriate usage of resources. These guidelines have addressed maternal and infant stay after childbirth, and length of stay and usage of resources for specific pediatric disease-related groups such as extreme prematurity with and without complications. There have been numerous anecdotal reports from concerned families and physicians about the safety of early discharge of infants and mothers, but little evidence from controlled trials to support the notion that early discharge is an unsafe practice. The evidence supporting the effectiveness of critical pathways that address utilization has been conflicting at best.^24-26 Although it has been demonstrated that for some of the NICUs studied these interventions can reduce utilization,¹² there is little evidence, either for or against, that these pathways significantly impact clinical outcomes or the family experience of care. When considered in the aggregate, there are probably more clinical guidelines in active use in perinatology and neonatology than any other area in pediatrics. Yet this group of guidelines varies considerably in the quality of the scientific evidence supporting the recommendations. This is somewhat surprising because the evidence base for perinatology and neonatology is probably greater than in any other area of pediatrics. Moreover, this evidence has been better reviewed and summarized than most other areas.²⁷ With this comprehensive and easily accessible evidence base, clear opportunities exist to define and disseminate new practice guidelines for the care of mothers and newborns.

	MAKING PRACTICE GUIDELINES WORK

As we noted earlier, practice guidelines in and of themselves do not provide a means to improve health care outcomes. To achieve a favorable impact on health outcomes, guidelines need to be disseminated and accepted by clinicians and patients alike. For this to occur, data need to be collected on performance and outcomes and used to modify the guideline to meet the specific needs of different patient populations. Dissemination of new knowledge to the level of the individual practitioner often takes years. The seminal work of Everett Rogers²⁸ done more than 50 years ago showed that the time it took for new technology to be disseminated to the everyday user was 6 to 10 years. Working with the dissemination of new agricultural technologies, he was able to show that this process followed an S-shaped curve with adoption taking place slowly during the first and third periods of the process and rapid dissemination occurring during the middle third period. The duration and shape of the process have been remarkably consistent between different groups and different technologies. Everett's work also provide insights into possible strategies to shorten this time interval. Through his research, he has defined the key groups that play an important role in the innovation process. The initiating groups in the dissemination process are the innovators. These are the individuals who create the technology and do the early testing. The next group is the early adopters. These individuals look outside their group or institution to innovators for new and innovative practices. They tend to be risk takers who have the opportunity to be exposed to new ideas from outside the group or organization. The early adopters are responsible for convincing the early majority within their group or institution that this new practice or technology is a good idea worth trying. The early majority are also risk takers willing to try new ideas, but they differ from early adopters in that they look within rather than outside their organizations for these ideas. More widespread adoption by the early majority convinces the next group, the late majority, to try the new practice. The late majority serves the important function of resisting too rapid adoption of new practices before they have been thoroughly tested and evaluated. The final group is the nonresponders. These individuals resist change and will only adopt new ideas when they are mandated. Considerable resources can be wasted trying to convince these individuals to change. The steep part of the S-curve, or the period of rapid change, occurs once the key leaders in the early majority try the new practice. When the early majority has adopted the practice, the dissemination process becomes self-sustaining.

This research has many important implications for the implementation of practice guidelines. These results tell us that the presentation of practice guidelines to a group of physicians without first working with the early adopters and the early majority is much more likely to fail. This is true even if the guideline's recommendations are evidence based. Some of the early majority and most of the late majority will not dispute the validity of the evidence, but will believe that the recommendations do not directly apply to their patients or will not work effectively in their setting. These concerns travel quickly through an organization and can potentially poison the well for future change. Successful guideline implementation is best accomplished by first identifying and working with the early adopters to pilot the changes and using the preliminary data to modify and refine the intervention. The second step is to insure these changes are implemented in a manner that is very visible to the rest of the organization. This allows the early majority to evaluate the impact of the changes and incorporate them into their practice. This can be facilitated by having key physicians in the early majority act as observers for the pilot physicians. Once the changes have begun in the early majority, the dissemination process is on the steep part of the curve and is usually self-sustaining at this point.

	IMPLEMENTING PRACTICE GUIDELINES IN A NICU OR WELL-BABY NURSERY

As we saw from the slow adoption of antenatal steroids to prevent respiratory distress syndrome, dissemination and adoption of new knowledge in neonatology can often take years. This long delay occurs despite strong scientific support behind the recommendations. This delayed implementation takes place at two levels: first, between NICUs with respect to how rapidly they adopt new guidelines; and second, between neonatologists within a given NICU. One answer to the first challenge of how to more rapidly implement practice guidelines between NICUs has been the development of networks or collaboratives. The Vermont Oxford Network (VON) is an example of a group of neonatologists and NICUs that came together to develop common processes for the measurement and improvement of health care outcomes.²⁹ VON engaged 10 NICUs to participate in a quality improvement effort designed to improve practices and outcomes in the rate of nosocomial infections in the NICU and the incidence of chronic lung disease in premature infants. These units developed a common set of measures to document improvement in outcomes and used a combination of evidence-based practices from the medical literature and their own benchmarking process to change practices. They were ultimately able to show significant improvement for both of their target conditions. In the context of dissemination theory, these 10 practices could be considered early adopters or part of the early majority. The challenge for VON will be to see if they can now rapidly disseminate these changes through the rest of the network and ultimately through NICUs nationwide. What the VON experience has demonstrated is that collaboratives or networks can provide an important setting for early adoption and refinement of new interventions and their eventual rapid dissemination.

The adoption and dissemination of a practice guideline within a NICU presents a similar yet distinctive challenge. Even within a small group of neonatologists, there are likely to be early adopters and representatives of the early and late majority. Adding to this complexity are groups of nurses, respiratory care practitioners, and other ancillary personnel who cannot be expected to follow in lock-step the directions of the physicians, and who represent the same spectrum from early adopters to late majority. The first step to effective implementation in such a setting is the formation of a multidisciplinary team that involves the key stakeholder in the clinical process. For example, a team targeting the implementation of the guidelines for the use of antenatal steroids must include neonatologists, perinatologists, neonatal and obstetric nurses, as well as other involved personnel. Depending on the guideline, it may be useful to involve the patient or family at some point in the process. A focus group of new parents may highlight the barrier to implementing guidelines in your institution. For example, parents may have difficulty in making an appointment with the primary care physician after early discharge from the nursery because no appointments are available. Once the team is formed, the next step is to agree on a common set of aims or outcomes for the improvement effort and a set of measures that will let you know that you have achieved the aim. Each of these outcomes can be associated with one or more outcome measures that together define an outcome measurement set. A portion of a clinical pathway for the care of extremely low birth weight infants is presented in Fig 1. The outcome measures associated with this project included: discharge disposition; achievement of certain clinical milestones such as time to reaching a stable state on feeds and getting off inotropes; complication rates for sepsis, pneumothorax, and pulmonary interstitial emphysema; mortality rates; and resource use as measured by length of stay and hospital charges. The outcome and performance measurement set comprise one of the most important features of a practice guideline.

View larger version (54K): [in this window] [in a new window]   Fig. 1.   Extreme prematurity clinical pathway.

The goal here is to be very specific in your aims, and realize that you may need to divide your initial aim into smaller and more specific aims as the project progresses. For example, your initial aim may be to have 100% of eligible mothers receive antenatal steroids. However, you may realize before you can reach this aim you need to solve the problem of how to get 100% of eligible mothers identified. A useful tool in this process is the critical pathway. A critical pathway is a matrix with time or phase of care along one axis and the type of care and/or accountable group of individuals along the other axis. Each cell is filled with the recommended practice. An example of a portion of a critical pathway for the care of extremely low birth weight infants is shown in the Figure. Critical pathways allow for the definition of all the components of care and the accountable groups or individuals. More importantly it can breakdown a large aim into a set of smaller aims directed at what needs to be achieved before the larger aim can be realized.

The next step is the definition of a set of measures that enables you to know that you have achieved your aims. If we look again at the clinical path presented in the Figure, the performance measures are the adherence rates to key recommendations found in the cells of the matrix such as the percentage of infants having their blood urea nitrogen monitored every week during the acute unstable phase or the percentage of infants receiving an eye examination during the maintenance phase. The key to this step is to keep your measurement set as parsimonious as possible and easy to collect. The propensity of many physicians is to look on measurement as a one time opportunity to gather as much data as possible, create large computerized data bases, and analyze them using sophisticated statistical packages. Increasing the size and complexity of the data collection and analysis process can create a significant impediment to rapid, iterative data collection and process revision cycles. If possible keep the measurement set small and collectible with paper and pencil technology. For most improvement projects, entering the data on a standard spreadsheet program will provide sufficient analytic power.

The next step for the improvement team is the assessment and possible revision of the recommended change. Even if the recommendation is supported by data from properly controlled randomized trials, there may be real concerns in translating these changes in practice to your patient population and to your practice setting. In most cases the practice guideline will need to be customized to meet these site-specific needs. This is best accomplished through a series of small, iterative pilot studies of the intervention. This allows for problems to be identified and solved and the guidelines revised before the next pilot. This rapid cycle quality improvement model has been described by Langley et al³⁰ and has been used by the Institute for Health Care Improvement^31,32 to bring about improvement in asthma care and reductions in cesarean section rates. A discussion of the specific aims, measures, and improvement cycles of a project to reduce medication errors in a NICU can be found in their breakthrough series report on reducing medication errors.³³ An added benefit of using rapid cycle quality improvement is that you can use your early adopters and early majority physicians to conduct the pilots. If these pilot studies are widely visible and the data shared with members of the health care team, it will increase the sense of ownership of the interventions and will decrease the time needed to effectively implement the guideline.

Once a guideline has been successfully implemented in a NICU there is still the concern that these recommended practices will become codified and as intractable to change as the previous practices. It is important that the quality improvement team stay together to continue to monitor practices and outcomes. It is also important that new innovations, no matter how small, be encouraged and tested. Some medical centers have developed an internal granting program for innovations in patient care that have spawned the development of new, cost-effective patient care practices.³⁴ One of the greatest resources that a tertiary care nursery possesses is the collective knowledge and creativity of the staff. Yet this is a resource that is often squandered. Creativity and innovation do not magically occur. They are processes like other processes that need to be supported and facilitated. Current work by Plsek³⁵ and others offer an excellent framework for facilitating creativity in your organization.

	CONCLUSIONS

Top Abstract Conclusion References

Practice guidelines and critical pathways have become the leading edge of a movement in medicine to make our practices more evidence based. They are important tools in an improvement process designed to shorten the time it takes to disseminate cost-effective interventions into clinical practice. The use of practice guidelines as a tool to lower costs or decrease resource usage has distracted from their primary purpose and has created confusion and distrust among physicians. If used correctly however, they provide an analytic framework for the development of outcome and performance measures that can be used in a process to improve care. In and of themselves, they do not provide a model for improvement. When used in conjunction with models such as rapid cycle improvement, they can become important tools for changing clinical practice and improving the outcomes of care. Practice guidelines can also become an important platform for encouraging and facilitating innovations in the care we provide. They offer an important adjunct to efficacy and effectiveness research by providing a means to insure the rapid dissemination of new knowledge into clinical practice. This is particularly true in the fields of perinatology and neonatology. Here the strong research tradition and the extensive evidence base offer considerable opportunities to improve the health of the pregnant mother and the infants under our care.

Address correspondence to David A. Bergman, MD, Lucile Salter Packard Children's Hospital at Stanford, Dept of Quality Improvement, 725 Welch Rd, Palo Alto, CA 94304.

	ABBREVIATIONS

AAP, American Academy of Pediatrics; NICU, neonatal intensive care unit; VON, Vermont Oxford Network.

	REFERENCES

Top Abstract Conclusion References

1. American Academy of Pediatrics. Policy Reference Guide of the American Academy of Pediatrics. 10th edition. Elk Grove Village, IL: American Academy of Pediatrics; 1997
2. American Academy of Pediatrics Provisional Committee on Quality Improvement Practice parameter: the management of hyperbilirubinemia in the healthy term newborn. Pediatrics 1994; 94:558-565 [Medline]
3. American Academy of Pediatrics Committee on Genetics and the American Thyroid Association Committee on Public Health Newborn screening for congenital hypothyroidism: recommended guidelines. Pediatrics 1993; 91:1203-1209 [Medline]
4. American Academy of Pediatrics Committee on Pediatric AIDS Human milk, breastfeeding, and transmission of HIV in the US. Pediatrics 1995; 96:977-979 [Medline]
5. American Academy of Pediatrics Committee on Infectious Diseases Recommended childhood immunization schedule. Pediatrics. 1996; 97:143 [Medline]
6. Bergman DA, Homer C. The impact of managed care on the cost and quality of care. In: Bergman DA, Luft H, eds. Managed Care and Children. The Future of Children: Children and Managed Care. 1998;8(Summer/ Fall)
7. Nolan TW Understanding medical systems. Ann Intern Med. 1998; 128:293-298 [Medline]
8. Avorn J Putting adverse drug events into perspective. JAMA 1997; 277:341-342 [Medline]
9. Grimshaw J, Russell I Effects of clinical guidelines on medical practice: a systematic review of rigorous evaluation. Lancet 1993; 342:1317-1322 [Medline]
10. Kanouse D When does information change practitioner's behavior? Int J Technol Assess Health Care 1988; 4:37-35
11. Horbar JD Increasing use of antenatal corticosteroid therapy between 1990 and 1993 in Vermont Oxford Network. J Perinatol. 1997; 17:309-313 [Medline]
12. American Academy of Pediatrics Provisional Committee on Quality Improvement and Subcommittee on Hyperbilirubinemia Practice parameter. Management of hyperbilirubinemia in the healthy term newborn. Pediatrics. 1994; 94:558-565 [Medline]
13. Agency for Health Care Policy and Research. Guideline Overview: Otitis Media with Effusion in Young Children. US Department of Health and Human Services. Washington, DC: Public Health Service, Agency for Health Care Policy and Research. AHCPR Publication No. 94-0620. July 1994
14. United State Preventive Services Task Force. Guide to Clinical Preventive Services. Report of the US Preventive Services Task Force. 2nd ed. US Department of Health and Human Services. Office of Public Health and Science. Office of Disease Prevention and Health Promotion. 1996;861-862
15. Turley K, Tyndall M, Roge C, Critical pathway methodology. Effectiveness in congenital heart surgery. Ann Thorac Surg. 1994; 58:57-65 [Abstract]
16. Wirtshafter D, Summers J, Jones I, Thomas J Using health care outcomes to improve patient care in the NICU. J Quality Improvement. 1994; 20:57-65
17. Cockington RA, Drew JH, Eberhard A Outcomes following the use of rational guidelines in the management of jaundiced newborn infants. Aust Pediatr J 1989; 25:346-350 [Medline]
18. McMulloch KM, Vidyasagar D Assessing adherence to standards of neonatal resuscitation taught throughout the neonatal referral area a quality assurance activity. Pediatr Clin North Am 1993; 40:431-438 [Medline]
19. Thompson D, Maringer M Using case management to improve care delivery in the NICU. Am J Matern Child Nurs. 1995; 20:257-260 [Medline]
20. Jones MLH. Outcome Measures of Managed Care in an Intensive Care Nursery. Gainesville, FL: University of Florida; 1994. Thesis
21. Merritt TA, Palmer D, Bergman DA, Shiono PH Clinical practice guidelines in pediatric and newborn medicine: implications for their use in practice. Pediatrics. 1997; 99:100-114 [Abstract/Full Text]
22. American College of Obstetricians and Gynecologists (ACOG), Committee on Obstetric Practice. ACOG Committee Opinion No. 174. Washington, DC: American College of Obstetricians and Gynecologists; 1994
23. Pylipow M, Gaddis M, Kinney JS Selective intrapartum prophylaxis for group B streptococcus colonization: management and outcome of newborns. Pediatrics. 1994; 93:631-635 [Abstract]
24. Braveman P, Egerter S, Pearl M, Marchi K, Miller C Problems associated with early discharge of newborn infants. Early discharge of newborns and mothers: a critical review of the literature. Pediatrics 1995; 96:716-726 [Abstract]
25. Beebe SA, Britton JR, Fan F, Jepson B Neonatal mortality and length of newborn stay. Pediatrics 1996; 98:231-235 [Abstract]
26. Britton JR, Britton HL, Beebe SA Early discharge of the newborn infant: a continued dilemma. Pediatrics 1994; 94:291-295 [Abstract]
27. Sinclair JC, Bracken MB. Effective Care of the Newborn Infant. Oxford, England: Oxford University Press; 1992
28. Rogers E. The Diffusion of Innovation. 4th Ed. New York, NY: The Free Press; 1995
29. Vermont-Oxford Trials Network Database Project Investigators The Vermont-Oxford Trials Network: very low birth weight outcomes for 1990. Pediatrics. 1993; 91:540-545 [Abstract]
30. Langley GJ, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Improving Organizational Performance. San Francisco, CA: Jossey-Bass Publishers; 1996
31. Flamm B, Kabcenell A, Berwick DM, Roessner J. Reducing Cesarean Section Rates While Maintaining or Improving Maternal and Fetal Outcomes. Boston, MA: Institute for Health Care Improvement; 1998
32. Weiss KB, Mendoza G, Schall MW, Berwick DM, Roessner J. Improving Asthma Care in Children and Adults. Boston, MA: Institute for Health Care Improvement; 1997
33. Leape LL, Kabcenell A, Berwick DM, Roessner J. Institute for Health Care Improvement Breakthrough Series: Reducing Adverse Drug Events. Boston, MA: Institute for Health Care Improvement; 1997:22-28
34. Franklin PD, Panzer RJ, Brideau LP, Griner PF Innovations in clinical practice through hospital-funded grants. Acad Med. 1990; 65:355-360 [Abstract]
35. Plsek P. Creativity, Innovation and Quality. Milwaukee, WI: ASQC Quality Press; 1997