- EPA —
- entrustable professional activity
Within this compendium of perspectives and expert consensus about the education of subspecialty fellows, we are told that “Recent advances in medical education allow the question of required duration of training to be examined using the metric of the pediatrics milestones.…” The consensus also reinforces that “Training should remain at three years for now, but over time the Board could allow a shorter or longer period to demonstrate expertise in a particular subspecialty.”1 Here we have the setting for 2 fundamental, but potentially conflicting, metrics, time versus maturation, that can collide and put substantial strain on resources intended to educate and certify fellows. If time is used as the gating mechanism for the advancement of fellows, it stands to reason that there will be heterogeneity of skills, knowledge, and maturity upon completion of the requisite time. Alternatively, if maturation is used as the gating mechanism, then there will likely be heterogeneity in the time to completion. It is worth exploring some of the consequences of these options.
To the extent that the milestones (such as the Denver Developmental Screening Test) and entrustable professional activities (EPAs; such as the driver’s road test) will provide a blueprint for assessing the maturation of a fellow, the potential benefit is a rigorous, reproducible, and logical method to recognize what competencies have been achieved throughout the course of education and clinical practice. In addition, this method should provide a means to track a fellow’s progress in real time, which would permit formative feedback and potential for midcourse adjustments and, even more importantly, for self-directed learning and development. This approach requires validation of the evaluation, establishment of confidence intervals, measurement of interrater agreement, certainty that the maturational schema is properly ordered, and assessment of ease of application. It is essential to recognize that the use of milestones will also have the natural consequence that individuals will be observed to mature at different rates. Hence, there should be established thresholds indicative of acceptable maturity in a particular domain. One might envision a profile that shows the maturation of the fellow in each of the various domains. Furthermore, if we adhere to the guiding principle that we wish subspecialty fellows to reach a certain maturity or competence before they are certified, we will need to find means to provide the experience and time, and hence the resources, to modify the training required to achieve this goal. Perhaps adherence to milestones would be no different than current expectations and consequences, but it is likely that a higher level of scrutiny will reveal differences in progress of fellows that need to be reconciled and require variability in duration of training. When we could not measure the progress well, the lines of competency were blurred. Measure with more precision and we will be obligated to know what to do with that measurement!
To the extent that time in training remains the major criterion for advancement, we must also accept some significant constraints. Society sets time frames for the acquisition of knowledge throughout much of primary education and largely relies on cognitive testing to establish achievement. However, many factors make that approach both limited and noncomprehensive for judging the agreed-upon core competencies in health care. These limitations are revealed when expected maturation is not achieved or when maturation is accelerated. We certainly have not created any means to assess whether the time frame for training is adequate, too long, or too short. Indeed, we simply fill the time frame with activities and hope that this will enrich the educational experience.
It is also worth recognizing that time-based criteria for the completion of a fellowship impede creation of novel or hybrid training. Frequently, the duration of each of 2 fellowships that might be melded together is summed and some time is discounted. This approach is based on the collective wisdom of 2 subboards to accept common elements in training, but it invariably only shortens the sum by 1 year and may discourage imaginative individuals in training from pursuing such options or force decisions very early in training. I (G.L.) am very grateful to have slid in under the wire and had the opportunity to craft my own educational mixture before flexibility was restricted. We suspect that there are others who will wish to forge their own paths on the basis of personal interests and the changing medical needs of children; we hope we can continue to encourage and facilitate this creativity.
The current binary “thumbs up” or “thumbs down” decision at the end of a fellowship creates a huge barrier and reluctance by program directors to recommend remediation or failure to progress to an independent professional role. Hence, there is often reliance on the cognitive testing as the decision-making tool. Cognitive testing, as currently used, simply does not assess well how adroitly one applies knowledge, how well technical skills are mastered, and how professional interactions and individual initiative facilitate or thwart performance, all of which are of paramount importance when considering medical practice. The addition of milestones and EPAs provides an opportunity to broaden greatly the overall assessment of our trainees.
In the ideal world, we would like to acknowledge expertise only when we have the confidence that a physician has achieved a particular level of maturity in the various competencies, the capacity to function independently, and the humility to know when collegial assistance is needed. Thus, we have an essential conundrum: how to reconcile the traditional schedule that follows the calendar year with the needs of the trainees who progress at differing rates. Because a fundamental component of professional education requires assumption of responsibility for patient care, adjustment in the schedule of an individual fellow is highly constrained by the need for integration with others who provide clinical care and by a myriad of rules that limit contact time and continuity with patients and thwart flexibility.2 Practical considerations may thereby cause programs to capitulate and advance a fellow whose performance is marginal or commit a fellow to 3 years when less time might suffice: who would pay for the additional training, if needed; how predictable is the desired maturation even if time is extended; how could a fellow manage a professional commitment made to a subsequent career position if the time in training is not predictable; and how could a shorter training time with clinical responsibilities be accommodated?
There certainly are no simple means to reconcile the potential tug of war between time-based and maturation-based completion of fellowship. We may learn, as is quite likely, that the time we have somewhat arbitrarily selected for fellowship generally fits for what we wanted to achieve. Indeed, if the 3 years for most fellowships were significantly out of line, there might have already been significant pressures created by the certification of a large number of individuals too soon. Have we hit the “sweet spot” before releasing individuals for independent functioning? Of course, if we have selected well, capable fellows would have figured out how to make their education suit their needs even if there were no real wisdom in the grand design. It is also possible that the steps into more independence during the initial career role rapidly mature the reasonably adept fellow. This postulate then raises the question of whether we are actually retarding maturation by the 3 years of fellowship and would be equally well served by shorter periods of training for many fellows.
With these thoughts in mind, we raise some questions, suggest some steps to answer these, and pose an approach for consideration in judging maturation.
Could rigorous assessment of maturation of fellows afford a genuine and unbiased opportunity to determine progress in acquisition of competencies for a particular discipline? It is likely that a concerted effort among a number of programs could amass sufficient data to track current advancement of competency and determine whether evaluators have consistency in assessment. With communal evaluation training and a means to share data there would be a foundation for addressing subsequent questions about the utility and validity of competency-based assessment. This approach would permit the evaluation of the length of training needed for most fellows to achieve competencies and to assess whether it could be accelerated in appropriate circumstances.
For those fellows who have not matured at the requisite rate, would milestones offer a means to provide ongoing assessment of individuals after a 3-year fellowship? Observation could occur during clinical practice after fellowship before one is permitted to take the secure cognitive examination. Certification, on the other hand, would be granted only when specified levels of competencies were achieved. This process is analogous to the approach for surgeons who must achieve a technical proficiency before final certification. This is an approach that has been used by the American Board of Pediatrics in selected circumstances when there have been breaches of professionalism during residency and continued structured observation is needed before one becomes eligible to take the examination. The ongoing assessment and direct observation could be a means to reconcile career development that is not in synchrony with a 3-year training model and to better judge the time needed for an individual to mature.3 Responsibility is an exceptionally valuable motivator and catalyst for maturation and is the cornerstone of clinical education. Thus, many of the qualities we may wish to track with the milestones may not reach their true potential under the bright light and prescriptive nature of a training milieu. Indeed, having a means to track development during a more independent phase of a career might provide a better assessment of maturation, if it could be accomplished without creating burdensome structures to do so. This approach would potentially provide a tool for section leaders, department chairs, or clinical practice directors who have very limited opportunity for judging competence of a new colleague before hiring. They would have a vested interest in ensuring competence because of their reliance on these new colleagues. Entry to the secure examination, at least for now, could remain the signal that maturation has occurred in other domains.
Would assessment of maturation in the various domains of clinical practice also offer the opportunity to determine whether research training has a salutary or pejorative effect of the development of clinical competency? One could certainly understand how time engaged in research training might retard development because of the dedication of attention to other activities. However, it is equally plausible that it expedites development because clinical reasoning is augmented by the rigor of the research experience. Our current model dedicates extensive human and financial resources to provide research training for fellows, with limited data regarding the success of this investment toward the goal of clinical training or generation of pediatric scientists.
Could a maturation-based approach be used for residents based on what is learned from experience with fellows? There are already means to vary the time dedicated to clinical training of residents through the use of the Accelerated Research Pathway or Integrated Research Pathway. Perhaps assessment of those who exercise these options may provide insight into needed duration of residency training before entering subspecialty training.
To summarize, the use of milestones or EPAs offers the possibility of release from some of the arbitrariness imposed by a single duration of training for all fellows. We might also have a means for assessing an underlying tenet of research training. And, we would have the potential for opening the aperture to consider novel programs. Perhaps traditional time in training will fit most fellows, and we could have comfort in knowing that. However, we could also develop means to permit assessment of maturation to proceed in a variety of settings, academic or private practice, after the traditional time in fellowship training is completed; this will take a productive partnership between the fellow, the program director, and the clinical supervisor of the fellow after the fellowship. In the end, the fellow and the observers would have a compelling responsibility for this process to function in a nimble and reliable fashion that optimizes career development and results in the delivery of the highest quality care to our patient.
- Accepted January 30, 2014.
- Address correspondence to George Lister, MD, Department of Pediatrics, Yale School of Medicine, PO Box 208064, New Haven, CT 06520-8064. E-mail:
Dr Lister conceptualized and drafted the initial manuscript and organized the authors; Dr Norwood provided initial analysis and edits of the manuscript and critically reviewed the manuscript; and both authors approved the final manuscript as submitted.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: No external funding.
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
- Copyright © 2014 by the American Academy of Pediatrics