PEDIATRICS Vol. 106 No. 5 Supplement November 2000, pp. 1175-1198

Final Report of the FOPE II Education of the Pediatrician Workgroup

, Robert L. Johnson, MD^*, Evan Charney, MD, Tina L. Cheng, MD, MPH^§, Diane Kittredge, MD, Lawrence F. Nazarian, MD^¶, , Russell W. Chesney, MD, Holly J. Mulvey, MA, Jimmy L. Simon, MD, and Errol R. Alden, MD

From the ^* New Jersey Medical School, Newark, New Jersey; the  Robert Wood Johnson Foundation, University of Massachusetts, Shrewsbury, Massachusetts; the ^§ Children's National Medical Center, Washington, DC; the  Dartmouth Hitchcock Medical Center, Lebanon, New Hampshire; and the ^¶ University of Rochester School of Medicine and Dentistry, Penfield, New York.

	ABSTRACT

Top Abstract Introduction References

This report from the FOPE II Education of the Pediatrician Workgroup assesses the current status and future trends of pediatric education. The attributes of each level of the education process (undergraduate, residency, fellowship, continuing medical education [CME]) are considered within the framework of lifelong learning. The pediatric education of nonpediatrician providers is carefully considered. The Workgroup proposes and describes a new model for pediatric education that encompasses educational needs assessment, curriculum development and outcomes evaluation. Particular attention is paid to CME, with a review of the strengths and problems of the current system. The proposal for improving CME in the 21st century highlights the need for each pediatrician to have a "CME home," and several models and scenarios are explored. Appendices summarize the results of several surveys conducted on behalf of the Workgroup, and list societal trends and advances in pediatric health care that will influence pediatric education in the future. Pediatrics 2000;106(suppl):1175-1198; pediatric education, educational needs assessment, curriculum development, outcomes evaluation.

Although these curricular elements fulfills several components of the medical education model we have proposed above, there is currently no continuous process of update based on national and local evaluations. Nor is there a method to ensure that specific pediatric clerkship programs will use the curriculum or engage in a process of local evaluation and program modification. Unlike pediatric residency programs, the pediatric clerkship is not regulated by any national body that regularly evaluates and accredits curricula and educational programs.

	INTRODUCTION

Top Abstract Introduction References

Review of the Pediatric Medical Education Section of the 1978 Report

The 1978 report focused primarily of the content of pediatric medical education at the undergraduate and graduate level. It emphasized an appreciation for the unique nature of infants, children, adolescents, and young adults, and called for a broadening of pediatric education to give the practitioner a facility with the diagnosis and treatment of a variety of behavioral and social issues that affect the normative growth and development of the child. Additionally, special emphasis was placed on the importance of educational approaches that allowed the pediatric resident to develop an appreciation for the continuity of the developmental process of each child. The report addressed continuing medical education (CME) briefly, recommending that pediatric departments assume increasing responsibility for the continuing education of local clinicians. During the last 2 decades these content recommendations have been key factors in the broadening of the scope of pediatrics.

The 1978 report contained little discussion and guidance about the process that should be used to implement its recommendations and achieve the specific educational goals at each level. In this report our focus will principally be on the process that faculties and organizations should undertake to achieve these objectives, and the structures that will enable them to do so. Although we will comment on educational content, that will change over time, the mechanisms by which educational change can be developed, implemented, and evaluated will have more lasting influence.

Summary of the Workgroup Directives

The focus of the Education of the Pediatrician Workgroup is the broad spectrum of education for the pediatrician. In addition to considering the training provided during medical school, this Workgroup has reviewed issues related to both the feasibility and the implementation of guidelines for pediatric residency training. Also, emphasis has been placed on CME. The Workgroup has also focused on the process by which educational change can be effected at the undergraduate, residency, and continuing education levels. The Workgroup has also considered the education of nonpediatric child health care providers.

	CURRENT STATUS/FUTURE TRENDS

Current Status and Problems

Medical education has advanced considerably in the last 20 years; there is a clearer understanding of learning processes and components of effective educational systems. Specifically medical education is viewed as a multidimensional process that includes:

 Needs assessment of learners, faculty, and program

 Competencies expected of the learner, defined for learners and faculty

 Curricula designed to achieve these competencies through a variety of learning opportunities (formal and informal teaching sessions, clinical experiences, self-directed learning)

 Faculty development to enhance educational skills

 Outcome measures for learners, curricular elements, and overall program

 Evaluation and feedback for learners, faculty, and overall program

 Continuous improvement and adaptation based on needs, outcomes, and timely feedback

Pediatric medical education has advanced with the development of new curricula and new evaluation methods. Currently, a broad array of organizations play important roles in curriculum design and program and trainee evaluation.

At the undergraduate level, the Committee of Medical School Educators in Pediatrics (COMSEP) has become an effective coordinating group for pediatric educators in medical schools, serving as a forum for presenting educational innovations and developing model curricula. COMSEP is financed by and reports to the American Medical School Pediatric Department Chairmen (AMSPDC).

At the residency level, the Association of Pediatric Program Directors (APPD) now serves a valuable coordinating role for residency directors in all accredited university and community-sponsored programs. It has evolved into an important forum for sharing ideas about educational curricula, program structure, and the role of the program director. The Ambulatory Pediatric Association (APA) and the American Academy of Pediatrics Resident Section (AAP-RS) have also been influential in residency curriculum development. All 3 organizations provided valuable input to the Residency Review Committee (RRC) for pediatrics in its 1996 revision of the program requirements.

At the fellowship level many new subspecialty boards have been established, with new curricula and board examinations, each under the auspices of its own subspecialty board. A standard duration of 3 years has been established for most programs, to emphasize and encourage research and scholarly activity within the discipline.

At the CME level, the AAP has developed extensive and sophisticated education programs both regionally and nationally. Through publications like PREP it has utilized the core content material developed by the American Board of Pediatrics (ABP) to guide practitioners in their own continuing education. Academic medical centers are increasingly active in providing continuing education as well. Faculty development has become recognized as an important component of CME for academic and community pediatricians, but as yet has not been a coordinated process under the auspices of either the AAP or the academic community. Of particular importance, the 7-year recertification requirement established by the ABP provides the structure to ensure that pediatricians are cognizant of their need for continuing education.

Unfortunately, despite progress in many areas, pediatric educational programs continue to suffer from several important shortcomings:

 Many programs lack to resources to incorporate state-of-the-art medical education concepts and technologies.

 There have been few evaluations of the effectiveness of many innovations.

 There is no organizational structure charged with the responsibility to define the core competencies required of pediatricians at all educational levels.

 There is no coordinated process to ensure that:

curricula are designed and adopted to achieve these competencies,

faculty are trained to teach these competencies, and

program accreditation and certification is closely linked to acquisition of these competencies.

An Educational Framework for Pediatric Education: Proposal for a New Model

Medical education is properly viewed as a continuum, from medical school through residency, fellowship, and continuing education, with variations both in the content and the level at which that content is taught. Therefore, a coordinated and continuous oversight process needs to be established that will involve educators at all levels. The development of core competencies should logically derive from an assessment of the current health status and needs of children within the context of their families and communities. This process should take into account advances in the biomedical and psychosocial sciences and the structure of the medical system in which health care is provided. In turn, these competencies should inform and guide the development of curricula for medical students, pediatric residents, fellows, and the continuing education of generalists and specialists in academics and community practice.

The framework we propose is outlined in Fig 1. Model of Educational Needs Assessment, Curriculum Development and Outcomes Evaluation. The model describes a competency-based educational system that is derived from the health care needs of the child, family, and community. In turn, those needs shape educational programs at medical school, residency, fellowship and CME levels. The structure of the health care system and advances in the biomedical and psychosocial sciences and technology will influence how those child health needs are translated into roles for pediatricians and other health care professionals.

View larger version (22K): [in this window] [in a new window]   Fig. 1.   Model of educational needs assessment, curriculum development, and outcomes evaluation.

The roles of health professionals at each level are defined by the set of core competencies required to address child health needs. These competencies should enable the professional to accomplish the specific tasks and functions required for that role. Because medical education is a complex, multistep process, competencies for each level of education are needed (medical school, residency, fellowship, and continuing education). Level-specific competencies help to define the outcomes expected of the learner at each stage of learning. Competencies also serve as performance standards for programsa successful program is designed to foster the acquisition of competencies by its learners.

Educational programs should be designed to teach these competencies. Curricula should derive from them, goals and objectives should be congruent with them, clinical and nonclinical learning experiences should encourage their attainment, and faculty should be adequately trained and supported to teach those competencies. They should also be used at the continuing education level to establish standards for credentialing pediatricians and nonpediatric providers of care to children.

At the national level, model curricula and faculty development programs should be developed to assist local educational planning. At the local level, curricula should be adopted to meet the specific strengths and resources of the individual program and its learners.

Finally, there must be program and learner evaluation. These evaluations should assess the program's ability to implement curricula effectively and the ability of trainees to achieve the required core competencies. The results of the evaluation process are important in redefining educational standards and, in turn, influencing educational needs assessments. It can be expected that each of those elements will change over time, and therefore the process needs to be an iterative and ongoing one, incorporating new information and refashioning educational programs so that they reflect current child health needs, societal and scientific information, and advances in medical education techniques.

In order that the structure and process have the imprimatur of the entire pediatric community, we recommend that the Federation of Pediatric Organizations (FOPO) initiate and provide oversight to its implementation. We further recommend that FOPO designate the Council on Pediatric Education (COPE) of the AAP as the operational body responsible to create the administrative structure and identify organizations that will participate. The lead organizations responsible for defining core competencies, developing curricula, and providing evaluation at each educational level are summarized in Table 1. These organizations would initially work together to define roles for pediatricians and then chair the process at their individual level, with active involvement of regulatory bodies, learners affected by these curricula, and other health professionals who provide care to children.

	MEDICAL STUDENT EDUCATION

Review of the Current Status of Medical Student Education in Pediatrics

At the beginning of this century American medical education had made a transition from the apprentice model common in the early 19th century to a didactic model that involved little patient contact.¹ After an extensive review of the outcomes of this method Abraham Flexner proposed a model of medical education that combined and systematized the basic science and clinical education aspects of medical education. This model has changed little since the beginning of this century. The curriculum in the first 2 years of medical school usually consists of a mixture of basic and clinical sciences that are prerequisite to the study of clinical medicine. This curriculum includes: anatomy, biochemistry, physiology, microbiology, pharmacology, behavioral science, pathology, and introduction to clinical medicine and clinical science topics such as, genetics, immunology, molecular biology, cell biology, and neuroscience. In most schools there are also courses and seminars in public health/preventive medicine, epidemiology, geriatrics, and biomedical ethics. Lectures are the primary method of instruction in the first 2 years.

In the third and fourth years lectures are replaced by supervised clinical clerkships in teaching hospital inpatient services and affiliated ambulatory clinics. In these settings medical students observe, discuss, and participate in diagnostic and therapeutic activities under the supervision of residents and faculty preceptors. Traditionally the patient's availability throughout the period of hospitalization has facilitated the efficiency and effectiveness of this method of clinical education.

According to the Association of American Medical Colleges (AAMC) all schools require clinical clerkships in internal medicine, surgery obstetrics/gynecology, pediatrics, and psychiatry, and more than half require a clerkship in family practice or some other primary care experience. Students generally have the option to elect additional training opportunities.

Ambulatory training is part of the clinical education program in all American medical schools. Increasingly this exposure to patients is a component of a primary care experience that begins in the first year of medical school. These preclinical ambulatory primary care experiences may include pediatric practices however the curricular supervision of these experiences is not always controlled within the academic pediatric department.

Specific pediatric medical education is restricted to a 6- to 8-week rotation during the third and fourth year of medical school. Invariably these clinical clerkships are under the academic supervision of the pediatric department. During these clerkships clinical exposure is generally divided between inpatient and outpatient experiences which are enhanced by a specific didactic sessions. Subspecialty rotations in areas such as neonatology and adolescent medicine may be included as aspects of the experience. In keeping with trends noted in the other primary medical specialties increasing amounts of time may be devoted to ambulatory experiences.

During the pediatric clerkship students are expected to gain an appreciation for the processes of human growth and development and their influence on health and disease. Students should also gain an understanding of the uniqueness of the health problems of infants, children, adolescents and young adults. Additionally they should acquire experiences in the management of health problems throughout the pediatric age group.

In most medical schools the content of the pediatric clerkships places special emphasis on the following pediatric skills and knowledge domains which were suggested in the 1978 Report on the Future of Pediatric Education and developed by a joint project of the APA and the Council on Medical Student Education in Pediatrics:

Skills Taking a detailed history from pediatric patients (infants, children, adolescents and young adults) and members of their families Clinical problem solving Physical examination of infants, children, adolescents and young adults Clinical diagnostic procedures Communication Knowledge Health supervision Behavior Growth and development Nutrition Prevention and treatment of illness and injury Issues unique to the adolescent Medical genetics and congenital malformations Issues unique to the newborn Common pediatric illnesses Therapeutics Fluid and electrolyte management Poisoning/prevention and treatment Pediatric emergencies Child abuse Normal growth and development from conception to adulthood Child advocacy

Future Trends in Medical Student Education in Pediatrics

As discussed in Chapter II, several trends are forcing medical schools to review and revise their curricula. Today only the most severely ill children who require the highest levels of care tend to be admitted to hospitals and those who are admitted rarely remain in the hospital through the whole course of their illness. This same phenomenon, which has been attributed to the effects of managed care, changing patterns of disease, and advances in medical technologies and therapeutics has also been experienced in the other primary care specialties. As a result medical students who are trained primarily in teaching hospitals learn a great deal about a limited range of the most complex pathologies. Undoubtedly they are also apt to be exposed to a broad array of complex procedures and medical technologies, which are more common in hospital settings.²

Medical schools are responding to these forces with revisions in the curriculum content and enhancements of education methods³:

Learning Objectives and Educational Goals In 1932 the AAMC through the report of the Rappeleye CommissionFinal Report of the Commission on Medical Education called for all medical schools to develop learning objectives for their curricula.⁴ During the last 6 decades several reports have supported this recommendation and the LCME has added an accreditation requirement that specifically requires medical schools to "define its objectives and make them known to faculty and students." However, a recent AAMC review of the status of medical school specific learning objectives and educational goals (Assessing Change in Medical Education: The Road to Implementation) found that few medical schools had "delineated a coherent and comprehensive set of learning objectives for the medical student education program."

Lectures Lectures continue to be the primary educational method used in the first 2 years of medical school. Increasingly, however, medical schools are decreasing lecture time and replacing it with small group discussions that connect the basic science lectures to clinical applications. Often these clinical applications are related to child health care. The participation of academic pediatric departments in the design of this curriculum component is variable.

Computer-Assisted Instructional Programs Increasingly computers are used to supplement teaching of basic and clinical science topics. This method has become so popular that the AAMC has produced a catalog that lists and critiques the various software available for teaching in medicine. One of the most exciting aspects of this technology is the ability to simulate clinical encounters with patients and teach diagnostic skills and medical decision-making with a focus on treatment costs and outcomes.

Problem-Based Learning Problem-based education is a unique approach in which basic and clinical science topics are introduced within the context of patient problems. Students are divided into small groups led by a faculty preceptor. Discussions center around of illustrative cases that are supplemented by independent research, reading materials, and occasional lectures and demonstrations. Problem-based learning has proven to be an effective method of teaching skills such as hypothesis development, deductive reasoning, and intraprofessional group communication skills. According to the AAMC, approximately 10% of medical schools currently use problem-based learning as an organizing principle for their medical education programs, many others are using it to supplement and enhance traditional modes of instruction.

Clinical Education in Community-Based Settings Clinical training in community settings is an exciting addition to clinical medical education in primary care specialties such as pediatrics. Traditionally, inpatient services hospitals have been the primary sites for the clinical education of medical students. Teaching at the bedside provided an excellent opportunity for clinical observation and discussion. In addition, student had ample time to learn diagnostic and therapeutic techniques under the supervision of residents and faculty.

	PEDIATRIC GRADUATE MEDICAL EDUCATION (GME)

Review of the Current Status of the Pediatric GME

The 1978 Task Force on the Future of Pediatric Education emphasized the goal of residency education to train general pediatricians. This goal has been reemphasized in the Accreditation Council for Graduate Medical Education (ACGME) Pediatrics RRC requirements⁵ and the APA's Educational Guidelines for Residency Training in General Pediatrics.⁶ The RRC program requirements state that "The goal of residency training in pediatrics is to provide educational experiences that prepare residents to be competent general pediatricians able to provide comprehensive, coordinated care to a broad range of pediatric patients." Residency education should lay the foundation for general pediatric practice and for subspecialty practice.

Although the main goal of residency education has not changed, the content, methodologies, and technologies in training have evolved over time. The ACGME Pediatrics RRC requirements were revised in 1996 (implementation in 1997) with the next revision scheduled for 2002. Curriculum content and clinical skills necessary for residency training are outlined in detail for general pediatrics and subspecialty pediatrics. Through a graduated 3-year experience it is expected that residents gain "a broad exposure to the health care of children and substantial experience in the management of diverse pathologic conditions."

While the RRC evaluates programs, the ABP evaluates and certifies individuals. The ABP was established in 1933 and is one of the 24 certifying boards of the American Board of Medical Specialties (ABMS). The ABP awards certificates in general pediatrics and in pediatric subspecialty areas. Program directors must certify that each applicant who has received training under his/her supervision is prepared for independent responsibility as a general pediatrician. In addition, program directors must evaluate their residents in the areas of interpersonal relationships with patients, families, and members of the health care team, work habits and personal qualities, and ethical and moral behavior. The ABP has developed core content statements that detail specific knowledge on which individuals will be tested.

The APA's Educational Guidelines for Residency Training in General Pediatrics⁶ was completed in 1996 with the aim of helping programs improve the way they prepare all residents for the practice of general pediatrics. The Guidelines outline educational goals and sample objectives for pediatric residency programs and offers suggestions about how to use these goals and objectives in planning and conducting educational programs.

Since the 1978 Task Force report 2 important educational organizations have been formed, the APPD and the AAP-Resident Section (RS). The APPD was formed in 1986 with the purpose to advance medical education by benefiting and aiding the medical education programs. The mission statement lists 4 objectives: 1) to provide a national forum for dialogue among residency program directors concerning issues, controversies, and curriculum; 2) to enhance the continued growth and development of pediatric postgraduate education; 3) to support pediatric program directors; and 4) to provide a forum for education interchange with med/pediatrics programs, medicine and family practice programs, and representation of pediatric residency program directors at the national level.

The AAP-RS was organized in 1989 with 3 objectives: 1) to provide an educational forum for the discussion of issues and problems relating to pediatric residency and fellowship training; 2) to disseminate information about training issues; and 3) to stimulate interest and involvement in child advocacy issues. The AAP-RS has representation on many pediatric organizations including APPD and other AAP committees and has become recognized as a voice for pediatricians in training.

Future Trends in the Pediatric GME

Educational Needs Assessment and Continuous Improvement The RRC requirements and the APA's Educational Guidelines for Residency Training in General Pediatrics are the primary guides used by pediatric residency program directors for the development of residency curricula. New methodologies and technologies for residency education are constantly being developed and evaluated. As inevitable changes occur in childhood diseases, sociodemographics, health care delivery, and technology, the competencies, curricula, methodologies, and technologies of residency education must be reevaluated and revised.

Length of Residency and Tracking The RRC and the ABP currently require 33 months of pediatric residency training usually completed in 3 years. With the advances in medical and psychosocial science and technology, some have suggested increasing the length of GME training for pediatricians. Those against increasing the length of training greater than 3 years raise concerns about financing additional training time and possible decreases in numbers of medical students entering the field because other primary care disciplines require 3 years of training. Currently, there is little evidence on which to suggest changing the length of pediatric training.

Education Sites Consistent with the needs of children and families and changes in the health care system, health care is increasingly delivered in the ambulatory and community settings. Early in this century the Flexner report emphasized that medical education is a process that should be centered in universities and hospitals. Today, fewer children are hospitalized and many are being seen in ambulatory and community settings such as outpatient clinics, primary care practices, schools, and community health agencies. The report entitled Pediatric Education in Community-Based Settings: A Manual¹⁰ outlines the philosophy and goals of community-based education. Transferring the achievement of educational goals to ambulatory and community setting requires a shift in current curriculum and objectives and the recruitment, selection, and development of ambulatory/community sites and faculty.

	PEDIATRIC SUBSPECIALTY EDUCATION

Future trends for pediatric subspecialist are covered in detail in the Subspecialist Workgroup Report. Broad educational trends that will impact all education, including fellowship training, are discussed in this report, under the headings "Educational Model" and "Educational Methods." To summarize, pediatric subspecialty training occurs in academic centers, under the general supervision of department chairs, and the direct supervision of subspecialist in the same field. Some fellowship programs share curricula and resources with other subspecialties, especially in the area of research training. Because of the heterogeneous organization of subspecialties, pediatric training programs occur in departments of pediatrics, surgery, radiology, psychiatry, and some are jointly sponsored (eg, allergy, genetics, infectious disease). The majority of subspecialties are board-certified (by discipline-specific boards within the ACGME) or have a certificate of added qualifications, but some specialties do not yet offer boards or certification.

In addition, training occurs in academic medical centers for pediatricians who will become academic educators and researchers but not subspecialists. Examples of this type of postgraduate training include general academic fellowships (not currently boarded), master degree programs and PhD programs (eg, public health, epidemiology, education).

These postresidency training programs are heterogeneous, with varying pediatric input, yet all produce future academicians who will provide care for children in referral centers, play a major role in pediatric research and direct a significant percent of the teaching for pediatric learners (students, residents, fellows, and practitioners).

Because the heterogeneity of fellowship training programs is likely to endure, pediatricians will not have direct influence over all of the fellowship programs training these physicians. Because fellowships will continue to exist without board certification, there will continue to be programs that are not involved with the formal RRC accreditation process (to ensure compliance with certain educational standards) and there will be no ABP certification process (to measure competency of the learner at completion of training).

Despite these organizational limitations, it is important to attain a certain level of competency for all pediatric subspecialty clinicians and researchers, and to ensure that subspecialists involved in our educational processes are skilled educators, able to use modern technologies and methods, and cognizant of the needs of our learners. Strategies to ensure this goal will vary, depending on the nature of the subspecialty and fellowship program. The Educational Model proposed in this Workgroup report applies most directly to fellowship training under the auspices of departments of pediatrics, accredited by RRC for subspecialties of pediatrics, whose graduates are subject to ABP certification processes. For other fellowships, pediatric educational oversight must be advisory, rather than directive.

	CME

Because the body of medical knowledge is constantly expanding, clinicians must accept the challenge of lifelong learning; consequently, teaching mechanisms to achieve this goal of CME must be in place and readily available. Since the first FOPE Report the human learning process has become an intense area of research, and we have broadened our understanding of how adults in general and physicians in particular learn. The intricate system for educating postresidency physicians that exists today has many strengths but requires improvement and innovation to meet the challenges of 21st century pediatric care.

Ways of Delivering CME

Teaching Options: General Principles Traditional tools for teaching physicians are lectures, seminars, workshops, and similar didactic modalities, as well as medical journals and other written materials. These methods still represent the major components of CME. Alternative systems have been devised, some relying on newer technology, that go far beyond the classic lecture or seminar format.

Teaching Options: Specific Methods Currently there are a wide variety of CME methods:

Teaching Options: Effectiveness

Degrees of Impact The effectiveness of CME teaching methods should be evaluated using tools that critically analyze the outcomes achieved by the individual practitioner. Dixon¹⁵ has suggested that the CME experience should be evaluated on several levels:

Measuring Effectiveness Currently, organizations wishing to present CME that offer learning credits to participants require accreditation from an appropriate agency. The ACCME, the AMA, national specialty societies, such as the American Academy of Family Practitioners (AAFP) and AAP, and state medical societies will accredit providers of CME that offer teaching on a state, regional, or national level. All of these accrediting bodies use similar criteria in granting approval to the groups that will provide CME.

Current Knowledge of Effective Methods Careful analysis of the many teaching modalities available today has yielded information that should guide both teachers and learners. In general, many of the more recent teaching innovations appear to be significantly more effective than the traditional methods. Reminders, academic detailing, and patient-mediated strategies have shown particular promise. Combinations of modalities may provide more benefit than the individual components. For example, a clinical protocol, based on a national practice guideline, that is incorporated into the office routine is more effective in influencing physicians to follow the advice stressed in the guideline than presentation of the original document in printed form.

Teaching Options: Preferences Although clinicians may not always be complete in their knowledge of what they should be learning, they are in a good position to judge which methods appeal to them and which methods are best suited to their individual life-styles and learning preferences. Recent surveys of pediatricians in practice and in academic settings yield a wealth of information about the kinds of CME these respondents use and prefer. The reader is encouraged to examine the specific findings that are found in the appendices, but some general observations are worth citing.

Responsibility for Providing CME

The Individual Clinician One critical principle that has emerged from analysis of CME is that self-directed learning is more effective than that imposed by outside forces. One goal of the providers of CME should be to stimulate individuals to participate in the learning experience. If self-directed CME is made relevant and exciting and shown to be effective, the clinician will respond. If barriers of time, distance, and money are reduced, the response will be even better.

National Professional Organizations On a national level, professional organizations naturally assume responsibility for providing CME as a function of their primary goals that are directed to the welfare of the patients served by their members. For pediatricians, the recertification process of the ABP and the educational programs with which that process is linked should be recognized as potential foundations of an individual's personal education plan. Not only must these organizations continue their critical role in providing CME, but they must also be aware of the best ways to teach and must improve their programs.

The Academic Department of Pediatrics The 1978 FOPE concluded that "Pediatric departments must assume increasing responsibility for the continuing education of pediatricians in their geographic areas." Although there is variability in the way this recommendation has been conducted, in a survey of pediatric departmental chairpersons that was done for this project all 94 respondents answered "yes" to the question, "Do you feel that part of the responsibility of your department is to educate pediatric clinicians who practice in your geographic area?" The extent of the perceived geographic area of responsibility varied from "our referral area" to regions encompassing 5 states. This group also made clear that "informal" education, such as telephone consultation between community and departmental physicians, is a significant component of their department's contribution to pediatric education.

Managed Care and CME

Managed care organizations must be concerned about the lifelong learning of their clinicians. These powerful agencies must encourage or even require their members to participate in CME. The surveys mentioned above asked about the ease or difficulty involved in participating in CME courses, based on the type of care setting in which the respondent was working, ranging from solo and group practice to staff and independent practice association model health maintenance organizations to faculty practices. Practitioners in 1- or 2-physician practices found it difficult to participate in CME activities, whereas in all other groups, most respondents reported easy access to CME

Financing of CME

Individual clinicians expect to pay for their ongoing, lifelong educational experiences. But the system cannot be financed solely by the contributions of the students. Just as the individual must view educational expenses as a necessary cost of doing business, so must society, through the health care industry, realize that educating those who provide the care is critical if quality is to be maintained. The financing of CME by the health care system is analogous to the support of research and development by industry. The research and development budget in many sectors of American enterprise is as high as 15% of total expenditures and is seen as an absolutely necessary investment in the future viability of that industry.

If organizations that provide health careand all sectors must be involved, not just those that fall under the definition of managed carewill contribute to the maintenance of a CME system of high standards, the benefits to society in terms of the ultimate health of its citizens will be enhanced immeasurably. A number of scenarios can be envisioned. In the following examples, an increasingly more active role is played by the organization, building on a base of the most fundamental support and moving into a proactive position:

Scenario Number 1: Supporting the Individual Dr Jones is a pediatrician who works for an organization called Health Care Givers (HCG). HCG allows Dr Jones 1 week of paid time off each year to pursue CME activities. They allow him to submit expenses of up to $1000 per year for CME costs. The choice of activity is his, and no feedback is required.

Scenario Number 2: Guiding the Individual HCG establishes a program in which each pediatrician is required to undergo a needs assessment. This process is to take place through a local department of pediatrics, or will be conducted in-house through the pediatrician who heads HCG's group of 10 pediatricians. The results of Dr Jones' needs assessment will guide the kind of CME he will seek. A list of local and national resources is available to help Dr Jones plan his education over the next year.

Scenario Number 3: Assessing the Organization's Needs HCG sets up an in-house program to monitor the collective needs of its pediatricians. It is determined that further education in the management of attention deficit disorders, especially the use of medication, is a widespread need among HCG's pediatricians. Several programs are identified that will allow the physicians to obtain more education on this topic, including a 1-day course that will be held locally, a workshop to be given at a national meeting, and an interactive computer program that can be done at home. HCG encourages each of its pediatricians to pick one of the alternatives and work it into his or her CME plans for the next year.

Scenario Number 4: Linking Support to Quality Improvement The head of the pediatric unit of HCG conducts an evening session in which the pediatricians share the experiences they have obtained from the different educational programs on attention deficit disorder. Two months later, an analysis of medical records compares the practices of the pediatricians to what was being done when the original analysis was performed. A lunchtime sharing session is held to discuss the ways in which changes in the management of attention disorders have evolved. Plans are made to choose another area in which improvement might be achieved.

Scenario Number 5: Working With Medical Educators The head of the pediatric unit at HCG consults with faculty members of the department of pediatrics at the local medical center. More formal studies are planned that will facilitate assessment of educational needs and measure outcomes in a scientific manner. Collaboration is begun on work that will enhance the quality of care given by the pediatricians and HCG and may lead to informative papers that will be published. HCG pledges a contribution to support the study, considering this expenditure to be an investment in the improvement of the care it gives and the satisfaction of its physicians.

Educating the Practitioner To Be a Teacher

No longer can we educate practicing pediatricians only in the realm of clinical medicine. The need for practitioners to be teachers of other clinicians is growing rapidly and stands out as a major aspect of practice in the 21st century. As the share of pediatric care given in the hospital declines and more and more conditions are addressed on an outpatient basis, there is a growing need to transfer the education of medical students and residents into ambulatory settings, including private offices. This trend brings exciting opportunities and significant challenges. How do we make sure practitioners are competent teachers? How can teaching be incorporated into practice without interrupting the flow of patient care? How will the practitioner be compensated for this extra effort and for the loss of time and income?

Considerable attention has been paid to these issues and the body of literature is growing.^16-18 It will be imperative that departments of pediatrics work closely with practitioners in fashioning partnerships that will foster good teaching in the ambulatory setting. The implications of such alliances go far beyond teaching, for they can create an atmosphere that will enhance the level of care in the entire community and open the door to other opportunities, such as clinical research and community advocacy.

An Overview of CME: Strengths and Problems

Strengths At the present time, CME is provided by a large number of local and national organizations in literally thousands of locations. Issues of quality, although far from being resolved, are addressed by existing agencies that require adherence to carefully constructed standards for programs that grant CME credits. A wide variety of formats already exist, ranging from home-study programs to courses that combine learning with recreation, and from paper and pencil exercises to interactive electronic programs. Efforts continue to make CME more effective and to link teaching methodology to clinical outcomes. The University of Toronto maintains a Research and Development Resource Base in Continuing Medical Education (RDRB/CME) that contains 7000 references and is constantly growing.

Problems What is lacking in the present system of providing CME is oversight and coordination of the myriad initiatives and methods. From an individual clinician's perspective, the present situation presents too many options, while perhaps not addressing that particular individual's most important needs. From a wider viewpoint, duplication of efforts in one sector undoubtedly coexists with inadequate coverage in another.

A Proposal for Improving CME in the 21st Century

Principles This proposal for a new way of looking at the overall CME system is predicated on the following principles:

The CME Home Proposal Just as the needs or children are met best when they have an established medical home that can treat them as individuals and provide coordinated, continuous care, so will the educational needs of individual practitioners be met best when they have an established CME home. The CME home will supply the following services:

CME Home Model Number 1:

The CME home based in a regional department of pediatrics in partnership with AAP chapters

The survey done by this Task Force established that 100% of pediatric chairs responding agreed that they have a responsibility to provide CME to clinicians within their geographic areas. In this model, a relationship would be established between the department and area practitioners as follows:

 Each pediatric clinician would establish a liaison with an accredited university or community-based academic department of pediatrics to establish a CME home. These relationships already exist for most clinicians, but in varying degrees of strength. In most cases, the clinician would ally with a department that is geographically close, to whom he or she may refer patients.

 Once the relationship is established, the department would decide how to help the practitioners allied with it. One approach would be to designate a faculty member who would serve as partner to a particular clinician. These individuals would meet on a periodic basis to discuss the ongoing education of the clinician and devise an appropriate educational strategy. Another option would be to have group sessions, in which clinicians would be taught principles of needs assessment and made aware of local and national resources. Sharing of ideas would be encouraged at these gatherings. In some parts of the country, these tasks could be accomplished by teleconferencing or computer technology.

The relationship of these partners in education would be symbiotic. The department would help the clinician determine his or her educational needs and develop an educational plan. Periodic reevaluation would take place. The clinician, in turn, would provide the department with grassroots feedback on CME efforts and suggestions for the future. This relationship will strengthen the interaction between departments and community practitioners. One benefit would be the facilitation of the recruiting and training of practitioners to become teachers of medical students and residents in their offices and clinics. In addition, office-based research would become much easier to accomplish, as would collaboration in community service projects.

In translating this model into reality, several scenarios are possible:

Scenario Number 1: Active cooperation and mutual benefit A department of pediatrics already has good communication with the clinicians in its area. Geographic barriers are not significant, and patterns of cooperation have been established in the past. Faculty are interested in interacting with practitioners and this responsibility does not create a major demand of on their time. Good CME programs get better with the cooperation of community clinicians and teaching of students and residents is enhanced, as are research efforts, referral patterns, and community projects.

Scenario Number 2: The spirit is willing but the flesh is weak A department desires better relations with its local practitioners but the available time for such interactions is limited. The ratio of faculty to community physicians is low. Budgetary constraints limit even modest programs aimed at town-gown cooperation. An overwhelmed department feels it cannot take on one more responsibility. An analogous situation exists with a physician who lives hundreds of miles from the nearest department of pediatrics that could supply her needs. She wants to establish the relationship, but cannot take the time away from practice to travel and is frustrated by her inability to establish what would be a desirable liaison.

Scenario Number 3: A one-sided love affair The pediatric faculty may be willing and able to establish a medical education home for the practitioners in their area, but most community physicians have stronger ties with other hospitals and are not willing to exert the effort needed to relate to the academic department. Over time, some have developed a competitive or adversarial relationship with the department of pediatrics. Or the practitioners like the idea but the department does not feel such an obligation should be assumed, even though it could be accomplished. In the worst case, both practitioners and faculty have little enthusiasm for such cooperation.

Scenario Number 1: Active cooperation and mutual benefit The AAP CME coordinators have met with the department head, have been apprised of the department's plan for working with its practitioners, and have supplied some suggestions and ideas derived from the resources of the national AAP. A meeting will be held in 6 months involving representatives of all of the academic pediatric departments in the chapter to focus on ways in which they can share experiences and improve CME throughout the region by planning together. The coordinators will keep in touch with departmental personnel, but major involvement will not be necessary.

Scenario Number 2: The spirit is willing but the flesh is weak The AAP chapter representatives have met several times with members of the department and have delineated the specific barriers to implementing the CME home initiative in that location. After consultation with the AAP on a national level, suggestions are generated that will allow the program to be started in a phased-in manner. Departmental representatives are put in touch with individuals in other departments who have had successful experiences to share ideas. Ongoing monitoring by chapter representatives, who continue to consult with the national AAP, allows continued support as the program grows. The individual who lives a long distance from a medical center now has electronic links to a department of pediatrics that has become her medical education home. She communicates with a faculty mentor by e-mail and has learned of a great many interactive CME programs that will allow her to meet her own needs through dynamic home-study programs.

Scenario Number 3: A one-sided love affair Chapter CME coordinators meet with department members and with practitioners to determine why there is inadequate support for the medical education home initiative. Consultation with members of the national AAP yields suggestions for selling the idea to reluctant participants. Individuals in other locations who have successfully implemented the idea are asked to contact those who are not convinced. Tangible support is offered.

CME Home Model Number 2:

The CME home based in a department of pediatrics without regard to location

Just as medical centers are embracing the concept of becoming "centers of excellence" in specific fields, so may individual departments choose to concentrate their efforts in well-defined areas. For example, at present there are academic departments that specialize in epidemiologic methodology and will contract with organizations outside of the university to help with the epidemiologic aspects of their research. The benefits to the department are both professional and financial.

A department of pediatrics might choose to equip itself to become a CME home for pediatricians. Individuals would contract with the department for a specified period of time and would undergo needs assessment, after which a professional educational plan would be created and access to the appropriate CME resources would be facilitated. This model might involve linking the clinician to a specific mentor, with whom a personal relationship would be established. The clinician would not have to visit the department physically. Mail, telephone, and electronic communications could accomplish the goals.

This model is similar to model number 1, but is designed to work at a distance and does not involve the depth of partnership found in a relationship with a regional department of pediatrics.

CME Home Model Number 3:

The national professional organization as CME home

The AAP has a long and successful track record in the area of education. Publications, some of which are now available on the Internet, CME courses in a variety of formats, and self-assessment curricula such as PREP all bear testimony to the viability of the AAP as a teaching agency that can reach pediatricians all over the country and the world. It is realistic to imagine the AAP serving as a CME home for individuals and performing the functions of needs assessment, exposure to CME resources, and creation of a professional educational plan. The increasing availability of electronic communications to pediatricians everywhere will enhance significantly the ability of the AAP to serve in such a coordinating capacity.

Other national organizations that are able to achieve such a capability could serve as CME homes. For clinicians in unique sectors of pediatricsand many specialists will fall into this categorya specialty society might be more appropriate as the CME home. For instance, the APA serves as a major resource for many individuals who function as academic general pediatricians and could be suited best to creating CME homes for them.

CME Home Model Number 4:

The commercial educational facility as CME home

There are many current examples of good teaching in a variety of formats provided by commercial enterprises, who hire medical experts to produce the content but supply their own production, business, and marketing support. The publication of medical textbooks is the most obvious example, but the list of resources is long. A commercial company could assume the role of CME home to clinicians just as a national professional organization might.

Implementing and Evaluating the CME Home Concept Recognizing that the concept of a specific CME Home for each individual is a new idea that is still in its formative stages, the Task Force recommends that these models be created in a test situation. A study in which the functioning of the CME home in its different manifestations could be evaluated, adjusted, and reevaluated over a specific period of time, such as 5 years, would lead ultimately to a real-world understanding of the concept and how effective it can be. Perhaps all models will work; perhaps only some. Most likely, the degree of success of each model will vary with location and other circumstances. In the study situation, new models might evolve. If the concept of the CME home proves as valuable as is anticipated, the study will lead to widespread adoption of whichever models are successful, to the ultimate betterment of clinicians and the children they serve.

Financing the CME Home Just as it is important to avoid creating a new bureaucracy when planning an educational initiative, it is critical to avoid schemes that will incur significant costs. From the practitioner's vantage point, CME is a necessary expense that is already part of the professional budget. It is appropriate for practitioners to pay for the kind of guidance that would be provided by a CME home. The system is likely to make each clinician's learning plan more efficient and more effective, in the long run saving money that might be spent on educational efforts that are less well-coordinated. At the same time, the cost to the practitioner must be reasonable and congruent with costs at the current level or the scheme will fail.

Model Number 1, Regional departments in cooperation with the local and national AAP Establishing this CME home should not be costly. From the departmental perspective, faculty time will be required. Using such mechanisms as group sessions can minimize this expenditure of time. In addition, there will be tangible benefits in that a successful liaison with practitioners may bring in more clinical referrals, will create more teaching facilities outside of the medical center, and may involve practitioners in activities at the medical center, such as rounding or precepting, that will free up faculty time. In addition, it would be appropriate to charge practitioners a reasonable fee for dispensing the guidance they receive. One plan would be to allow the practitioner to compensate the department for its assistance either monetarily or through direct service rendered in teaching, patient care, or administration.

Model Number 2: Pediatric departments serving as CME homes without regard to location One of the goals of this model would be to generate income for the department. Initial startup costs would be incurred as the services are being developed. Marketing will be necessary. As this model achieves success, additional faculty and staff will have to be added. Ultimately, these additions will strengthen the department. Because participants will be paying for the service, it is anticipated that the system will generate income once it is operational and will turn a profit once initial costs are reimbursed.

Model Number 3: National professional organizations providing CME homes This model builds on preexisting organizations that have education as one of their major functions. Some of these groups, such as the AAP, already have a broad spectrum of in-house CME resources that will bring in revenue on their own if clinicians choose to use them as part of their professional educational plans. Reasonable charges for needs assessment and the other functions of the CME home would be expected by clinicians. Once the system is set up, it should function as an increasingly productive source of revenue for the organization.

Model Number 4: The commercial educational facility Because commercial companies are by definition interested in profit, they would not undertake such a project unless it looked as if it could make money. As indicated in the discussion of the other models, the CME home is likely to be a profitable undertaking once initial costs are reimbursed. Obviously, if competition were to occur on a major level, all groups would not do as well. But the basic nature of the CME home as an ongoing service that picks up new subscribers every year carries with it the potential for ongoing revenue and an increasing profit margin as the service became better established and more efficient.

The CME Home and the Medical Education Model The concept of a medical education home fits well into the medical education model described in Section III. After the practitioner has established a CME home, the first task is to assess the needs of that individual. Through dialogue and using needs assessment instruments, the clinician defines the core competencies he or she needs to practice in his or her unique situation. The practitioner's needs are matched with curricula available nationally and locally. As the educational process evolves, adjustment of the curriculum is accomplished. At the same time, feedback from the clinician-student allows refinement of CME efforts, especially on a local level.

	THE PEDIATRIC EDUCATION OF NONPEDIATRICIAN PROVIDERS

In addition to pediatricians there are numerous professionals who provide health care to children. Other medical disciplines include family medicine, emergency medicine, obstetrics/gynecology, internal medicine, child psychiatry, public health/preventive medicine, and osteopathy. In fact, taken together these groups account for more patient visits for persons under 21 years of age than do pediatricians. In addition, there are nonphysician health care providers who are involved in child health care, including advanced practice nurses, physician assistants, and a variety of mental health professionals. It is also evident, to an extent not well-documented, that "alternative" or "complementary" practitioners provide child health services, as they do for the adult population.

Many of the traditional professional groups have expressed an interest in collaborating with pediatricians in educational endeavors for their own trainees, and in some cases have done so in the past. However, all of them are very protective of their autonomy and do not necessarily acknowledge the pediatrician to be the "chairman of the board" who sets educational or practice standards. In fact, pediatricians do not have the authority to impose standards for the care of children on these groups for the most part (physician assistants being an exception). Although pediatricians feel particularly knowledgeable and qualified to define educational standards for those who will care for children, we need to recognize that a collaborative approach will be much more likely to succeed, both in influencing the educational standards of others and obtaining their input into the education of present and future pediatricians.

Moreover, we believe that a collaborative approach is more likely to achieve the goal of establishing competency standards that will be recognized by certifying, credentialing, and practicing organizations (eg, hospitals and managed care practices), so that those who provide health care services to children are qualified to do so. We believe that it will be beneficial to child health to forge stronger ties to these groups and therefore recommend strategies by which they may be involved, rather than the content of what is to be learned, consistent with this Workgroup's overall strategy.

We have discussed this collaborative strategy with representatives of several medical and allied health professional organizations.¹⁹ All were enthusiastic about the concept of participating in the definition of core competencies for practitioners at varying levels who provide health services to children. They expressed a willingness to take a formal proposal from the Future of Pediatric Education (FOPE II) Project to participate in such a process to their governing bodies for discussion and approval.

	APPENDIX A: SUMMARY OF SURVEY RESULTS

To inform the deliberations of the members of the Future of Pediatric Education II (FOPE II) Project, a number of surveys have been conducted. Many of these surveys focused all or in part on issues pertaining to GME and CME. These surveys went to the providers of GME and CME (eg, pediatric residency program directors) as well as to the recipients of this education (ie, pediatricians).

	AAP Periodic Survey of Fellows Number 40: CME and GME Experiences of Pediatric Generalists

This survey obtained input from current pediatric residents and pediatric generalists. This survey was sent to 1602 active US FAAPS, and had a 69.1% response rate. Respondents were asked to rate their GME in the following areas:

 Stabilize ill infants and children

 Determine when to refer patients

 Teach other health care trainees

 Practice pediatrics in your current setting

 Keep updated on developments in clinical care

 Work in interdisciplinary teams

 Provide preventive care counseling

 Choose the right career path

 Provide cost-effective medical care

 Work within a managed care system

 Manage business/administrative aspects

 Care for patients from different socioeconomic backgrounds than that of your practice

 Care for patients from different cultures

 Coordinate patient care with community (cultural competence) services and resources

Pediatric generalists who have completed their residency training within the past 15 years and current pediatric residents rate their training, in most areas, as "very good" or "adequate." However, a majority of these general pediatricians think they are poorly prepared to manage the business and administrative aspects of their practice (81%) and more than half do not think they have been adequately prepared to work within a managed care system (53%). Nearly one-fourth of pediatricians (23%) think they are inadequately prepared to provide cost-effective medical care or coordinate patient care with community services and resources, and 16% say they are poorly prepared to care for patients from different cultures.

By further subdividing this group of generalists who have completed their residencies within the past 15 years, and just looking at recent/current residents (1994-1998), these respondents rate their residency education as "very good" in preparing them to:

 Determine when to refer patients

 Work in interdisciplinary teams with other professionals

 Care for patients from different socioeconomic backgrounds

 Care for patients from different cultures

Generalists in practice since 1983 and current residents highly rate their training in various aspects of general pediatrics such as: immunizations, normal newborn care, physician growth, and so forth. There is considerable variation in ratings given for training in subspecialty areas and large proportions indicated that they felt poorly trained in many clinically allied areas (see discussion below). A majority of generalists said too little time was spent during residency training in managed care settings, community physician offices, and public health clinics and too much time was spent in neonatal intensive care units (NICUs).

	Survey of Multidisciplinary Sections

The 1978 "Report of the Task Force on the Future of Pediatric Education" prominently discussed clinically allied components of pediatrics, such as behavioral and developmental pediatrics. Because of the importance of these topics, the Project initiated the FOPE II Survey of AAP Multidisciplinary Sections and obtained specific feedback (a 56% response rate) on the following topics:

 Child abuse and neglect

 Community pediatrics

 Clinical pharmacology and therapeutics

 Administration and practice management

 Computers and other technologies

 Public health epidemiology

 Home health care

 Injury and poison prevention

 Bioethics

 Sports medicine and fitness

 School health

 Clinical epidemiology

 Environmental health

 International child health

Most of these multidisciplinary topic areas were seen as important for GME by more than half of all respondents, with child abuse and neglect (84%), injury and poison prevention (80%), and community pediatrics (77%) ranked as the top three. Administration and practice management was ranked sixth (62%). However, respondents reported relatively low rates of inclusion and effectiveness of instruction in multidisciplinary topics during their own residency. Only child abuse and neglect, clinical pharmacology and therapeutics, and injury and poison prevention were reported as at least somewhat covered by more than 70% of the respondents.

Respondents were asked to rank suggested strategies for incorporating training in multidisciplinary topics into GME:

 Structured classroom training

 Diverse placements during training

 Audio cassettes, videotapes, or other media

 One-hour seminars

 Mentoring

 Other suggestions

Diverse placements during residency was by far the preferred strategy, ranked first by 45% of the respondents.

	Survey of Pediatric Residency Program Directors

To inform the deliberations of the FOPE II Education of the Pediatrician Workgroup, the Project surveyed pediatric residency program directors. In August and September of 1998, a questionnaire was sent to 201 members of the APPD. A response rate of 70.15% was achieved.

The survey was fielded to solicit focused input on 4 topics: 1) time requirements for training in clinical settings; 2) acquisition of other competencies mandated by the pediatric RRC; 3) practice settings; and 4) career development. The first 2 topics responded to specific changes in RRC requirements for residency training in pediatrics that became effective on February 1, 1997. The remaining 2 topics addressed more general issues regarding the future of pediatric education. A number of open-ended questions were added to the survey to encourage respondents to use their evaluations of current minimum requirements as a basis for recommending optimal requirements and competencies for pediatric residency training programs in the 21st century.

Time Requirements for Training in Clinical Settings:  Effective February 1, 1997, pediatric residency programs are mandated to implement a number of changes pertaining to the length of training in various settings. Survey respondents were asked if their program had experienced difficulties implementing these 7 requirements, and if so, to identify the barriers to implementation:

 Maximum of 50% training in ambulatory settings

 Maximum of 6 months' intensive care experience

 Limit on amount of time in the NICU

 Minimum of 5 months in general inpatient pediatrics

 Minimum of 4 months in emergency and acute illness care

 Minimum of 1 month block rotation in behavioral/developmental pediatrics

 Minimum of 1 month block rotation in adolescent medicine

Respondents were also asked to provide their opinion as to how well these minimum standards will prepare the pediatrician of the future to provide optimal care to children. The results revealed that very few programs reported difficulties in instituting the new RRC requirements; the two exceptions being the following items: maximum of 6 months intensive care experience and limit on amount of time in the NICU.

The common barriers identified were insufficient institutional support and service demands for other aspects of the program. The majority of the program directors believe that these minimum standards prepare the pediatrician of the future very well or sufficiently to provide optimal care. Small percentages think that the current requirements are insufficient for behavioral/developmental pediatrics (16%) and adolescent medicine (13%).

Acquisition of Other Competencies Mandated by the Pediatric RRC:  The new RRC guidelines of February 1, 1997 give only general indications regarding the proficiencies and skills that comprise these new competencies residents must acquire:

 Multicultural dimensions of health care

 Community-based experience

 Medical ethics

 Professional behavior

 Health care organization

 Financing

 Practice management

 Quality assessment and improvement

 Risk management

 Cost-effectiveness

 Child advocacy

 Medical information science emphasizing skills for self-learning

Program directors were asked if their residency program had identified the specific proficiencies and skills that residents should acquire as a result of their education in these competencies. They were also asked if a standardized curriculum for these general competencies had been developed. Of the 12 competencies listed, only 2 had not been defined in terms of proficiencies and skills by more than 50% of the respondents: the multicultural dimensions of health care (36%) and financing (41%).

In terms of curricula, only community-based experience (78%), medical ethics (62%), and child advocacy (61%) yielded numbers above 50%. Although it may be argued that some of the remaining competencies are taught and can continue to be taught without formal curricula, this assertion does not satisfy the need to verify adherence to a common set of minimum standards for skills and proficiencies in these general competencies. When asked how the program will measure/evaluate the acquisition of these competencies, the predominant responses included observation of the resident, faculty comments, and resident comments. Many program directors also linked the evaluation of competency acquisition to completion of the hours required by the program.

Practice Sites:  Most program directors regarded the exposure provided in the 6 settings to be excellent or satisfactory:

 Private office-based practice for continuity clinic

 Private office-based practice for outpatient clinic

 Predominately managed care practice

 Community clinics for outpatient rotation

 Community clinics for continuity clinic

 Hospital-based practice for continuity clinic

The predominantly managed care practice was cited as the setting for which the quality of exposure was in greatest need of improvement. About one-third of respondents to the question indicated that there was no current exposure in either the private office-based practice for continuity clinic or the community clinics for continuity clinic. To understand the import of this data, it would be necessary to determine if the lack of exposure is attributable to a lack of access to such sites, as in rural and underserved areas, or whether it is attributable to organizational, financing, or other issues.

Career Development:  Program directors were asked to comment on these types of career development:

 Counsel resident in subspecialty education options

 Provide information about the job market and practice options

 Assess residents' competencies, motivations, and experiences

 Counsel residents in lifestyle/family considerations

They were also queried as to whether or not the program tracks the career development of its graduates.

Program directors communicated no serious concerns regarding the provision of career development assistance in their residency training programs. Almost all residency training programs track board certification and participation of their residents in fellowship training programs. Half of the respondents stated that their programs tracked subspecialty certification, while a third followed their graduates' participation in research. The publication and teaching careers of residents were not tracked as frequently, according to the data, but it is possible that such tracking may be conducted on an informal and individual basis, which would explain the lower numbers in these categories.

Pediatricians in the 21^st Century:  A number of open-ended questions provided an opportunity for the respondents to comment on the effect of the RRC guidelines on residency programs. Many objected to the rigidity of the RRC guidelines. In their view, greater flexibility is necessary so that programs can prepare pediatricians to pursue diverse career paths in a variety of practice settings.

When asked about the primary challenge for pediatricians in the next century, an overwhelming majority of the respondents focused on concerns regarding the future role of pediatricians as the primary caregivers to children. Central to these concerns were the impact of managed care and allied health providers, such as pediatric nurse practitioners, physician assistants and family practitioners, on health care delivery and workforce. Respondents communicated a shared conviction that pediatricians are and will continue to be the most competent and appropriate providers of care to children.

Many program directors indicated, however, that pressures from managed care organizations and other sources to reduce the costs of health care have endangered the availability of such optimal care for the future. Excessive reliance on the services of allied health providers, respondents have asserted, will deny pediatricians the experience necessary to maintain their unique skills and to keep pace with developments in medicine and health care delivery. The general pattern of response thus indicates the need to conduct outcomes studies to determine appropriate roles for pediatricians, both general and subspecialist, and for other providers of care to children. Such studies are essential in managing pediatric health care in the 21st century in terms of cost, quality of outcomes, and workforce.

	Survey of Third-Year Residents, 1997/1998

This survey was designed to explore the employment and fellowship choices of pediatric residents completing their third year of residency in 1998. The survey was sent to a sample of 502 of the 2494 US third-year residents in categorical pediatrics residency programs in the spring of 1998, and achieved a response rate of 83.7%.

Residents were asked to evaluate how well their residency program prepared them for primary care and subspecialty practice, as well as for some of the socioeconomic considerations of modern health care:

 Primary care pediatric practice

 Practice in a managed care setting

 Child advocacy in the political arena

 Pediatric fellowship training

 Assessing the needs of your community

 Ethical decision-making

Preparation for primary care pediatric practice or pediatric fellowship training, and ethical decision-making were given an average rating above "good." Preparation for the practice in a managed care setting received an average rating between "fair" and "good," as did assessing the needs of your community and child advocacy in the political arena. Residents going into subspecialty/residency training rated their preparation for pediatric fellowship training significantly higher than those residents without a position.

	General Observations: GME

Given the variety of surveys and samples, one-to-one comparisons of responses are not possible. However, some general trends are readily observable, particularly in regard to the topics of managed care and practice management.

The area of managed care/health care organization is addressed in 3 of the surveys. Of the pediatric generalists who have completed their residency training within the past 15 years and current residents who participated in Periodic Survey Number 40, more than half (53%) do not think they were adequately prepared to work within a managed care system. The pediatric residents who participated in the Survey of Third-Year Residents, 1997/1998 used a scale of 1 ("poor") to 5 ("excellent) to rate their residency education on managed care topics. This topic received an average rating of 2.8, which is between "fair" and "good."

Looking at the "provider" end of the spectrum, respondents to the Survey of Pediatric Program Directors revealed that slightly more than half (52%) of the pediatric residency programs have identified specific proficiencies and skills for this topic. A significantly smaller number of programs (37%) teach this topic based on a standardized curriculum. The predominantly managed care practice was cited by residency program directors as the setting for which the quality of exposure was in greatest need of improvement. On this same issue, pediatric program directors identified concerns about managed care, as well as allied health providers, on health care delivery and workforce. Many believe that pressures from managed are organizations and other sources to reduce the costs of health care have endangered the future of pediatric health care.

Another issue that received attention in each of the surveys pertains to the topic of administration and practice management. Of the pediatric generalists who have completed their residency training within the past 15 years and current residents who participated in Periodic Survey Number 40, 81% think they are poorly prepared to manage the business and administrative aspects of their practice. Sixty-two percent of the respondents to the Survey of Multidisciplinary Sections felt "strongly" or "very strongly" that this topic is important for GME. Half (50%) of the pediatric program directors stated that they had identified the specific proficiencies and skills that residents should acquire as a result of their training in the area of practice management. Less than half (44%), however, had developed a standardized curriculum for this competency.

	AAP Periodic Survey of Fellows Number 40: CME and GME Experiences of Pediatric Generalists

The 3 sources of CME most frequently named by pediatric generalists as influential are traditional journals (46%), self-taught CMElike PREP(39%), and review journals (30%). Other sources of CME identified as influential by about 20% of generalists are locally sponsored CME courses, brief local programs (ie, grand rounds), AAP-sponsored CME courses (not at annual/spring meetings), academic specialists, and general pediatricians in their practice. Respondents had been given 20 options from which to select, including audiocassettes, videotapes, or other media, journal clubs, subspecialty society courses, and others. More young (less than 44 years of age) than older (44 years of age or older) pediatricians named review journals as 1 of the 3 most influential forms of CME.

Pediatricians who are located more than 70 miles from the nearest medical school are significantly more likely to say they use traditional journals as a main source of CME. A majority of generalists (67%) say their present practice organization makes it fairly easy for them to participate in CME activities. However, those in group practice and hospital practice find it significantly easier to participate in CME than do generalists in solo or 2-physician practices: only 43% of generalists in solo/2-physician practices compared with 76% of those in group practice and 65% in hospital/clinic practice say it is relatively or very easy to participate.

Respondents were asked to identify the factors that influence their CME choice:

 Opportunity to acquire practical/clinical advice

 Relevance to my practice

 Desire to learn academic foundation of clinical subjects

 Need for CME credit

 Combination of program with recreation or travel

 Flexibility to participate in CME programs on your own time

 Makeup of program faculty

 Cost of the program

 Reputation of sponsoring institution

Overall, 6 out of 10 pediatric generalists say the opportunity to acquire practical/clinical advice and the relevance to their practice are among the 3 most important factors when choosing a CME activity. Nearly one-third of the respondents say the need for CME credit (32%) and the desire to learn the academic foundation of clinical subjects (31%) are among the 3 most important influences on participation in CME.

In response to an open-ended question, pediatric generalists identified the following topics in which they plan to see CME during the next 2 years. A large proportion said they would be seeking CME in areas of general pediatrics (12%); many others named subspecialty areas such as infectious disease (21%), developmental/behavioral pediatrics (9%), adolescent medicine (7%), and dermatology (7%).

	Survey of Pediatric Subspecialists

Another survey was sent to a sample of members of the American Pediatric Society, most of whom are pediatric subspecialists. The sample was somewhat self-selected in that it included those pediatricians who had earlier identified an interest and willingness to participate in the activities of the Education of the Pediatrician Workgroup or the Subspecialists of the Future Workgroup. The questionnaire was sent to 121 members and 93 responses were received (76.8%).

These pediatric subspecialists were given a stratified list of CME modalities to rank:

 Reading traditional journals

 Reading medical newsletters

 Brief local programs (eg, grand rounds)

 Teaching sessions at AAP meetings

 Subspecialty society courses

 Other national courses

 Videotapes

 Consultation with other physicians:

General pediatricians in my practice

Community general pediatricians outside of my practice

Academic general pediatricians

Community-based specialists

Academic specialists

 Reading review journals

 Journal clubs

 Local courses

 AAP courses, not part of meeting

 Sessions at other national meetings

 Audiocassettes

 PREP programs

Of methods used to keep current in clinical pediatrics, reading traditional journals is by far the most popular, followed by consultation with other physicians (primarily academic specialists), sessions at specialty meetings, and subspecialty society courses.

Respondents were asked to identify the factors that motivate them to participate in a particular CME program. They were given the many of the same choices as the generalists who participated in Periodic Survey Number 40. In this survey, however, the option, "Flexibility to participate in CME programs on your own time" was further subdivided to include "Ability to commute from home to course" and other options. The strongest factors favoring specific CME courses are:

 Relevance of the program to practice

 Makeup of the faculty

 Ability to learn the academic foundation of clinical subjects.

Respondents for all settings (practice/faculty, group/solo, managed care) found access to participation in CME programs easy, with the notable exception of solo practitioners. Specialists stay current in general pediatrics through grand rounds and morning report, journals, and teaching. Many favor more programs that contain personal interaction, especially in small groups, and there is considerable interest in CD-ROM and other interactive technology as well as Internet learning and teleconferencing.

	Survey of Multidisciplinary Sections

The members of the AAP Multidisciplinary Sections were given nine options regarding the most useful CME formats:

 Individual reading of professional literature

 Courses at national meetings

 Professional self-taught CME curriculum

 Courses in local community

 Audio cassettes, videotapes, other media

 Brief programs (seminars, grand rounds)

 Consultation with other experts

 Journal clubs

 Internet resources

The most useful formats identified by the respondents were individual reading (23%), courses at national meetings (19%), and brief programs (19%). However, responses ranged across the full set of options offered.

The respondents reported definite location preferences for CME activities. The local hospital was only slightly more preferred than national meetings, with own home a close third. Pediatricians preferred their homes (18%) to their offices (9%) as a site for CME. The option of other locations within the community (6%) was not preferred nearly as often as the local medical school (14%) or hospital (19%). There were no differences in preferred location by age, sex, community, or distance from the nearest medical school.

Respondents were asked what was available to them and what they use to keep current in general pediatrics. The options approximated those listed above. It was clear that a wide variety of strategies are easily available to a substantial majority of the respondents. Almost everyone has access to reading professional literature, and more than three-quarters are able to participate in brief local programs and self-taught CME curricula. A few strategies, such as journal clubs, section meetings, subspecialty meetings and national courses other than at national meetings, were reported as less easily available. With the exception of journal clubs, all of the strategies mentioned in the survey were used by a majority of respondents at least sometimes.

It is clear from the responses that there is wide variation in the availability of CME opportunities in the multidisciplinary topic areas. Child abuse and neglect, injury and poison prevention, community pediatrics, and clinical pharmacology and therapeutics, the 4 topics reported as receiving the most substantial coverage during GME, are also most frequently reported as easily available for CME. However, the 2 topics reported as least often included in GME, administration and practice management and computers and other technologies, were reported as much more available as CME topics. These are also the 2 multidisciplinary topics with the most interest among respondents for CME in the next 2 years.

In responding to a question about new issues being addressed in pediatric practice, substantial majorities of respondents reported having dealt with new technology in their offices, meeting patient needs for services that are not adequately reimbursed, cultural or language barriers with their patients, and meeting the needs of underserved populations.

	Survey of Pediatric Department Chairpersons

The third survey was sent to the members of the AMSPDC. From a mailing to 145 members, 94 department chairs responded. One striking finding was that all 94 respondents feel that part of the responsibility of their departments is to educate pediatric clinicians who practice in their geographic areas. The zones of responsibility ranged in scope from "referral area" to regions encompassing 5 states. All attempted to reach pediatricians, but many also included in their target audience family practitioners, nurse practitioners, nurses, and other clinicians.

Many successful programs were cited, with grand rounds mentioned most frequently, although a few departments had poor attendance at their grand rounds. "Informal" education, such as telephone consultation between community and departmental physicians, is a significant component of the contribution to pediatric education in 90% of those departments answering.

Although only the key insights from these informal surveys are cited, it is clear that the opinions of respondents from many different pediatric venues will provide valuable insights into which CME modalities appear to be effective and which are favored by both students and teachers.

	Survey of AAP Chapters

In August 1997, leaders (chapter presidents, chapter vice presidents and those chapters that have executive directors) of the 66 AAP chapters were canvassed and 42 replied, resulting in a 63.6% response rate. They were asked to complete an informal questionnaire that addressed the following areas:

 The types of CME programs available

 Examples of successful and unsuccessful programs

 Other regional organizations that provide CME

 Suggestions for innovative teaching techniques and effective evaluation

Many examples of successful program were given. Location and timing were felt to be important influences on the outcome of a program. Refresher courses and short, focused programs with breakout sessions were among the better formats, as opposed to long lectures. Programs either broke even or made money for the sponsors 89% of the time. According to the chapter leadership, institutions playing an active role in CME include the AAP and its chapters, academic institutions, local medical centers, and state medical and pediatric societies. Respondents showed interest in teleconferencing, CD-ROM technology and on-line education.

	General Observations: CME

Given the variety of surveys and samples, one-to-one comparisons of responses are not possible. However, some general trends are readily observable. These are most notable in the types or formats for CME, reasons for acquiring CME, and sources or sites of CME.

The pediatric generalists who participated in Periodic Survey Number 40 most frequently listed traditional journals (46%); self-taught curriculum, like PREP (39%); and review journals (30%) as their top choices for CME. Brief local programs (ie, grand rounds) were among the "top 3 choices" for 21% of the generalists and almost a fourth (23%) listed locally-sponsored CME courses. Respondents to the Survey of Multidisciplinary Sections identified reading traditional journals as the most popular venue for CME (23%). Brief programs were also listed by 19% of the respondents. Reading traditional journals is the most popular method from the participants in the Survey of Pediatric Subspecialists.

Not unexpectedly, brief local programs (ie, grand rounds) were the most successful programs offered by pediatric department chairs and AAP chapters. It is important that this type of CME is also on the list of popular choices by the users of CME.

Most of the generalists listed the opportunity to acquire practical/clinical advice as their main motivation to acquire CME, while most of the subspecialists identified relevance of the program to practice as their top choice. Both groups ranked third the ability to learn the academic foundation of clinical subjects.

Because the department chairs who participated in the Survey of Pediatric Department Chairmen (AMSPDC), feel that part of the responsibility of their departments is to educate pediatric clinicians who practice in their geographic areas, the responses of the pediatric generalists to Periodic Survey Number 40 is gratifying. Pediatric generalists receive their accredited CME from a variety of sources. While, overall, relatively few pediatricians receive all or most of their accredited CME from any one source, 32% named the national AAP and 23% their local or regional university department of pediatrics as the source of all or most of the CME they receive in a typical year; 48% say these sources provide them with some of their CME.

	APPENDIX B: TRENDS AND ADVANCES

Child Health Needs in the Context of Family and Community

Changing Family Structure, Poverty, and Cultural Considerations The "American Family" is becoming more and more diverse. Physicians must be aware of this diversity with regard to ethnic or racial groups, economic classes, religious communities, and defined geographic communities (ie, rural, urban, suburban). They must also be prepared to deal with a variety of family structures including differences by marital status, sexual orientation, biological relationship to the child, and parental age among others.

Chronic and Complex Conditions Vaccines, antibiotics, and improvements in public health have eradicated previously common pediatric infections. Many other children today survive illnesses to which they may have succumbed in the past. Often these diseases result in chronic illnesses that require care from primary care pediatricians as well as subspecialists. The pediatrician is required to manage a broad range of "life issues" related to chronic health conditions which includes helping families find resources in the community, dealing with family health and mental health issues, and working with other team members including home nursing, physical and occupational therapy, developmentalists, subspecialists, and other services.

New and Changing Morbidities Almost 2 decades ago, the AAP emphasized the growing role of the pediatrician in the prevention, early detection, and management of the various behavioral, developmental, and social functional problems encountered in pediatric practice.²⁴ Today this "new morbidity" continues to account for significant morbidity and mortality in childhood and adolescents and includes issues such as injury and violence, mental health problems, substance abuse, teen pregnancy, learning problems, behavioral management, and psychopharmacology. The pediatrician of today and tomorrow must be prepared to address these "new morbidities" with both individual and population-based interventions.

Cost Management and Medical Ethics The scientific and technological changes in health care that have improved the prognoses of many of the old morbidities have also contributed to the rapidly escalating cost of health care. Today physicians are faced with moral dilemmas that are created by the financial consequences of his/her diagnostic tests and treatment decisions. The teaching of medical decision-making is becoming an important part of educational curricula at all levels. The physician of the future must be able to use medical ethics, controlled studies of patient outcomes and scientific knowledge in evaluating the costs and benefits of various treatment options.

The Health Care System

The American health care system is in the midst of evolutionary changes that are having a profound effect on the types of health care environments in which pediatricians provide care for infants, children, adolescents, and young adults.

Community-Oriented Practice Understanding communities, community resources, and community-based practice has become critical to the practice of medicine as the delivery of health care moves out of the hospital and into less traditional settings.²⁵ Physicians must understand the cultural, financial, political and environmental issues in the communities in which they work and learn to assess health needs in the community. Physicians must also be aware of and understand how to work with community services when providing care to families.

New Practice Settings The emergence of managed care has had a major impact in medicine that has required a reevaluation and restructuring of the education of the future physician. Two recent surveys of health maintenance organizations have been critical of the training of residency graduates.²⁶ Training was felt to be poor in resource management and cost-effective practice. Excessive referrals, diagnostic testing, and prescription-writing were specifically cited. Other areas of weakness included ambulatory care, knowledge of and interaction with community resources, quality assurance, understanding of practice standards and guidelines, population-based medicine, and team interaction and communication.²⁷

Teamwork in Pediatrics Pediatric practice of the future will require teamwork and a coordinated system of care. Increasingly, primary care physicians have increased responsibilities for referrals and authorizations and must learn when and how to refer. They also serve as consultants to other pediatric colleagues including nonpediatrician providers and subspecialists.

Evidence-Based Medicine Evidence-based medicine is the "conscientious, explicit, and judicious use of the current best evidence to make a decision about the care of patients."²⁹ Increasing knowledge about what does and does not work, cost-control pressures, growing interest in outcome and quality of care, and increasing sophistication on the part of patients have contributed to the emphasis on evidence-based medicine. Physicians must learn how to access, critically analyze, and use existing evidence in the care of patients.

Advances in Biomedical and Psychosocial Sciences

In the last 2 decades advances in areas such as molecular biology have transformed our understanding of illness and greatly expanded our therapeutic approaches.

Genetics Advances in molecular biology have had an enormous impact in medicine. Training programs at all levels must include education on these advances in pediatrics and knowledge on how to access further up-to-date information.

Expanding Informational Sources Technological advances in communication have dramatically influenced the information available to physicians and patients and lead to the development of a new field"medical informatics." Physicians must understand the potential promise and problems inherent in technologic change. They must develop a basic understanding of common computer applications including word processing, database management, literature searching, electronic mail, and the Internet. They should also be comfortable with computer-assisted instruction modalities and computer-based testing (ie, ABP recertification).

Roles for Pediatricians

Pediatricians of the future will function in a variety of roles in delivering health care to children. Training programs at all levels must be responsive to the training needs of future care providers in these roles.

Primary Care Practitioner The Primary Care Practitioner is the traditional role for pediatricians in the United States, but it is an uncommon role in other industrialized countries where pediatricians serve as consultants. This role primarily involves outpatient care and includes coordination of care with other health personnel including subspecialist physicians, psychologists, nutritionists, physical therapists, and others. The role of the primary care practitioner emphasizes continuity of care, comprehensiveness, and coordination.

Primary Care Team Leader Another growing role is as primary care team leader with other providers of primary care including nurse practitioners and physician assistants. This role may occur in the inpatient and outpatient settings.

Consultant Pediatricians may also be consultants to other health professionals including nurse practitioners, physician assistants, subspecialists, family practitioners, psychologists, or others.

Hospitalist Increasingly, there has been a division of labor for inpatient and outpatient care for children that has led to the development of the hospitalist,³⁰ a provider who primarily cares for pediatric patients who require inpatient care.

Subspecialist Subspecialists may function in all the above roles in inpatient and outpatient settings. These different roles have always existed for pediatricians, however, changes in health care needs and the health care environment have shifted their demand. Additionally, geography and patient population characteristics have influenced the distribution and mix of practitioners in these roles. For instance, in rural areas where inpatient services may be far away, primary care physicians may not be able to care for hospitalized patients, thus necessitating a need for hospitalists separate from primary care practitioners.

	ACKNOWLEDGMENTS

The Future of Pediatric Education II (FOPE II) Education of the Pediatrician Workgroup would like to thank the many individuals who contributed their valuable time, expertise, and perspectives to the Workgroup's endeavors. The Workgroup must first express its gratitude to Errol R. Alden, MD, FAAP, FOPE II Principal Investigator, and Holly J. Mulvey, MA, Director of the FOPE II Project, for their unwavering and unqualified investment in the Workgroup's success. The Workgroup is particularly indebted to Admiral Samuel Kessel, MD, FAAP, former Director of the Maternal and Child Health Bureau, Health Resources and Services Administration, for his constructive comments and participation in the Workgroup's deliberations. The Workgroup is likewise very appreciative of the valuable input provided by Carol D. Berkowitz, MD, FAAP, Past President, Association of Pediatric Program Directors; Edward Zimmerman, MS, Co-Director, AAP Department of Practice and Research; and Ethan Alexander Jewett, MA, Consultant to the FOPE II Project. Research and administrative support was provided by Mary Ruth Back, FOPE II Health Policy Analyst; Sarah E. Brotherton, PhD, Senior Research Associate, AAP Division of Health Policy Research; and Donna Bundy, FOPE II Project Assistant. Finally, the Workgroup extends its thanks to the many physicians who informed the development of this report through their participation in the Workgroup's surveys on graduate medical education and continuing medical education issues.

	FOOTNOTES

Received for publication Mar 1, 2000; accepted Mar 1, 2000.

	ABBREVIATIONS

CME, continuing medical education; COMSEP, Committee of Medical School Educators in Pediatrics; AMSPDC, American Medical School Pediatric Department Chairmen; APPD, Association of Pediatric Program Directors; APA, Ambulatory Pediatric Association; AAP-RS, American Academy of Pediatrics Resident Section; RRC, Residency Review Committee; ABP, American Board of Pediatrics; FOPO, Federation of Pediatric Organizations; COPE, Council on Pediatric Education; GME, graduate medical education; ACGME, Accreditation Council for Graduate Medical Education; AMA, American Medical Association; ABMS, American Board of Medical Specialties; AAFP, American Academy of Family Practitioners; HCG, Health Care Givers; RDRB/CME, Research and Development Resource Base in Continuing Medical Education; PROS, Pediatric Research in Office Settings; NICU, neonatal intensive care unit.

	REFERENCES

Top Abstract Introduction References

1. Jones RF. Academic Medicine: Institutions, Programs and Issues. Washington, DC: Association of American Medical Colleges; 1997
2. Greenberg LW, Getson PR Assessing student performance on a pediatric clerkship. Arch Pediatr Adolesc Med. 1996; 150:209-1211
3. Linzer M, Slavin T, Mutha S, Admission, recruitment and retention: finding and keeping the generalist-oriented student. J Gen Intern Med. 1994; 9:S14-S23 [Medline]
4. Association of American Medical Colleges. Medical School Objectives Project, Report I: A Framework for Review and Evaluation of the Medical School Curriculum. Washington, DC: Association of American Medical Colleges; 1998
5. Accreditation Council for Graduate Medical Education. Program Requirements for Residency Education in Pediatrics. Washington, DC: Accreditation Council for Graduate Medical Education; 1996
6. Ambulatory Pediatric Association. Kittredge D, ed. Educational Guidelines for Residency Training in General Pediatrics. McLean, VA: Ambulatory Pediatric Association; 1996
7. DeAngelis C, Duggan A, Oski F Twenty-five years of residents: what and where are they now. Pediatrics. 1990; 85:10-16 [Abstract/Free Full Text]
8. Liebelt EL, Daniels SR, Farrell MK, Myers MG Evaluation of pediatric training by the alumni of a residency program. Pediatrics. 1993; 91:360-364 [Medline]
9. Taras HL, Nader PR Ten years of graduates evaluate a pediatric residency program. Am J Dis Child. 1990; 144:1102-1105 [Abstract/Free Full Text]
10. DeWitt TG, Roberts KB, eds. Pediatric Education in Community Settings: A Manual. Arlington, VA: National Center for Education in Maternal and Child Health; 1996
11. Association of American Medical Colleges. Physicians for the Twenty-First Century: The GPEP Report. Association of American Medical Colleges; 1984:16
12. Reuben D, McCue J, Gerbert B The residency-practice training mismatch. Arch Intern Med. 1988; 148:914 [Abstract/Free Full Text]
13. Roback F, Randolf L, Seidman B. Physician Characteristics and Distribution in the United States. Chicago, IL: American Medical Association; 1992
14. Davidoff F Continuing medical education resources. J Gen Intern Med. 1997; 12:S15-S19
15. Dixon J Evaluation criteria in studies of continuing education in the health professions: a critical review and a suggested strategy. Eval Health Professions. 1978; 1:47-66
16. Roberts K, DeWitt T, eds Pediatric resident education in community settings. Proceedings of a conference held on March 23 and 24, 1996 in Chicago, Illinois. Pediatrics. 1996; 98:1249-1301 [Abstract]
17. Branch WT, Kroenke K, Levinson W The clinical educatorpresent and future roles. J Gen Intern Med. 1997; 12:S1-S4
18. Osborn LM, Sostok M, Castellano PZ, Blount W, Branch WT Recruiting and retaining clinician-educators. J Gen Intern Med. 1997; 12:S83-S898
19. Norman Kahn, MD; Director, Division of Medical Education, American Academy of Family Practice; Jack Ginsburg, MD and Herbert Waxman, MD, American College of Physicians; Ardy Dunn, RN, PhD, PNP, National Association of Pediatric Nurse Associates & Practitioners; Patrick Kileen, MS, PA-C, Society for Physician Assistants in Pediatrics
20. Howard BJ Family diversity and the pediatrician. Pediatr Ann. 1998; 27:38-43 [Medline]
21. Pachter LM Culture and clinical care. JAMA. 1994; 271:690-694 [Abstract/Free Full Text]
22. Kune-Karrer BM, Taylor EH Toward multiculturality: implications for the pediatrician. Pediatr Clin North Am. 1995; 42:21-30 [Medline]
23. Bithoney WG, McCarthy P, McGravey A, Nethersole S, Palfrey JD. Training Residents to Serve the Underserved: A Resident Education Curriculum. McLean, VA: Ambulatory Pediatric Association; 1993
24. American Academy of Pediatrics, Committee on Psychosocial Aspects of Child and Family Health The pediatrician and the new morbidity. Pediatrics. 1993; 92:731-732 [Abstract/Free Full Text]
25. Kassirer JP Redesigning graduate medical educationlocation and content. N Engl J Med. 1996; 335:507-509 [Free Full Text]
26. Mathematica Policy Research Inc. HMO Primary Care Staffing Patterns and Processes: A Cross-Site Analysis of 23 HMOs. Washington, DC: Health Resources and Services Administration; 1994:74
27. Leslie LK Can pediatric training manage in managed care? Pediatrics. 1995; 96:1143-1145 [Abstract/Free Full Text]
28. US Department of Health and Human Services, Public Health Service, Health Resources and Services Administration, Bureau of Health Professions. Council on Graduate Medical Education Resource Paper: Preparing Learners for Practice in a Managed Care Environment. Washington, DC: US Department of Health and Human Services, Public Health Service, Health Resources and Services Administration, Bureau of Health Professions; 1997
29. Marwick C Proponents gather to discuss practicing evidence-based medicine. JAMA. 1997; 278:531-532 [Abstract/Free Full Text]
30. Wachter RM, Goldman L The emerging role of "hospitalists" in the American health care system. N Engl J Med. 1996; 335:514-517 [Free Full Text]