PEDIATRICS Vol. 107 No. 4 April 2001, p. e52

ELECTRONIC ARTICLE:
Pediatric Residents' Performance of Ankle and Knee Examinations After an Educational Intervention

Albert C. Hergenroeder, MD^*, Joseph N. Chorley, MD^*, Larry Laufman, PhD, EdD, and Amy C. Fetterhoff, BS^§

From the ^* Department of Pediatrics, Adolescent Medicine and Sports Medicine Section; the  Department of Medicine, Baylor College of Medicine; and the ^§ University of Texas Houston Health Science Center School of Public Health, Houston, Texas.

	ABSTRACT

Top Abstract Methods Results Discussion Conclusion References

Background.  Pediatric residents have the need for additional training in the care of common musculoskeletal injuries.

Objectives.  To implement and evaluate the effects of a teaching intervention on pediatric residents' knowledge and skills in performing the physical examination of the ankle and knee.

Study Design.  Prospective, intervention, single-sample study design.

Methods.  Pediatric residents (n = 58) on a 1-month adolescent medicine rotation received a teaching intervention after a baseline evaluation of their knowledge and skills. The teaching intervention was designed to improve their knowledge about and skills in performing physical examinations of the ankle and knee. The intervention included watching a videotape, followed by observation of the attending physician demonstrating the techniques on a standardized patient, followed by correct demonstration of the techniques by the resident. The residents' knowledge and skills were assessed at the end of the rotation and 9 months later. Knowledge was assessed using a written examination. Skills assessment was performed using a Clinical Skills Assessment Examination.

Results.  At baseline, the residents performed 37% of the ankle and 18% of the knee physical examination techniques correctly. At 1 and 9 months, the residents' knowledge of ankle and knee examinations was greater than at baseline. The residents performed 77% of the techniques correctly at 1 month and 67% at 9 months. The residents performed 55% of the knee examination techniques correctly at 1 month and 47% at 9 months. The teaching intervention was rated highly by the residents.

Conclusions.  The residents' performance of ankle and knee examinations was suboptimal at baseline and improved significantly after the teaching intervention. Observed improvements persisted for a mean of 35 weeks. The teaching intervention described in this study could meet the need for improved ankle and knee examination skills, the 2 most common sites of skeletal injury in young athletes. The teaching model is novel in that it couples videotape and skills-based teaching methods with reliable evaluation methods. This model teaching method could be adapted for use in other pediatric residency training programs and other content areas. musculoskeletal, physical examination, resident curriculum.

Current and recent residents in pediatrics, family practice, physiatry, and medicine-pediatrics have identified the need for additional training in the care of common musculoskeletal injuries.^1-4 Recent graduates of pediatric residency programs ranked sports medicine/orthopedics second in areas where they felt least comfortable.² However, there is a gap between pediatric residents' need to learn to diagnose and treat common musculoskeletal problems and knowledge of the best educational model for teaching this material. Primary care residency faculty may not be trained in the diagnosis and rehabilitation of common musculoskeletal injuries that can propagate the cycle of inadequate training.⁵ The Centers for Disease Control and Prevention recently called for the development of model curricula, instructor training manuals, and learning materials to be distributed nationally as part of a national plan for action to address injury control in the United States.⁶ The purpose of this study was to implement a novel teaching intervention and to evaluate its effects on pediatric residents' knowledge and skills in performing the physical examination of the ankle and knee, the 2 most common sites of sport injury in young athletes.

In the investigators' pediatric residency training program, the residents on a 1-month adolescent medicine rotation consistently requested more sports medicine training. Barriers to learning to perform examinations of the musculoskeletal system during the 1-month rotation included inadequate faculty time to teach physical examination techniques in the context of patient care and with the detail required for residents to gain proficiency. To address residents' desire to view the faculty performing physical examination techniques, a teaching videotape demonstrating proper physical examination techniques of the musculoskeletal system was developed by one of the investigators. This videotape was coupled with a skills-based teaching session to create the teaching intervention described here. This article reports the implementation of this teaching intervention and its effects on pediatric residents' knowledge and skills 1 and 9 months later. The method to measure the knowledge and skill outcomes is described in the accompanying article.

	METHODS

Top Abstract Methods Results Discussion Conclusion References

Study Design

The study design was a prospective, intervention, single-sample study design. Because this was an evaluation of an existing teaching intervention provided for the residents as part of their 1-month rotation, institutional review board approval was not sought.

Study Participants

There were 60 second-year pediatric and internal medicine-pediatric residents assigned to a 1-month adolescent medicine rotation between September 1998 and December 1999, under the supervision of one of the investigators who is board certified in adolescent medicine and sports medicine. The focus of the rotation is adolescent medicine. However, depending on patient volume, residents on the adolescent medicine rotation see sports medicine patients in the sports medicine clinic of the investigators, which is conducted simultaneously with the adolescent clinic. All residents were eligible to participate unless they took the sports medicine elective with the investigators before or during the study. Fifty-eight of the 60 eligible residents participated at baseline. At the end of the 1-month rotation, 48 of the 58 repeated the evaluations. All but 2 of the 10 residents not available at 1 month were on vacation at the end of the month. Follow-up evaluations at 9 months were repeated by 36 of the original 58. Five residents were not eligible for the 9-month follow-up because they completed the sports medicine elective with the investigators. Therefore, 68% of the original eligible participants completed an evaluation at 9 months. The remaining 17 did not complete the evaluation because of commitments to other rotations.

Intervention: Method Development

Videotape A 42-minute videotape entitled "Musculoskeletal Examination: Diagnosing Ankle, Knee, Shoulder, and Back Injuries in a Primary Care Setting" was produced in 1997 (Write Eye Productions, Houston, TX) using one of the investigators to demonstrate correct physical examination techniques of the ankle, knee, back, and shoulder and actors as patients. The first 18 minutes of the videotape include the physical examination of the ankle and knee. The script for the videotape was developed using physical examination checklists for the ankle and knee currently used in the investigators' sports medicine clinic. The videotape was first given to a convenience sample pilot group of 17 health care providers including pediatric residents and physicians in practice or in full-time academic positions. Each pilot group member was also asked to rate the videotape from uninformative to very informative. All reviewers indicated that there was a need for the information provided on the videotape; 15 of the 17 rated the videotape as very informative.

Baseline Evaluation The baseline evaluation took place the day of the teaching intervention, within the first 2 days of the adolescent medicine rotation. The baseline evaluation and teaching intervention were conducted between the morning and afternoon clinic sessions and required ~1 hour and 30 minutes to complete. Using a questionnaire, the residents rated how comfortable they were in making the diagnosis of ankle and knee injuries based on history and physical examination using a 5-point scale: 1 = very uncomfortable/always refer, 2 = uncomfortable, refer most, 3 = neutral, 4 = comfortable, seldom refer, and 5 = very comfortable/almost never refer. The residents' knowledge about and skills in performing the physical examination of the ankle and knee were evaluated using a written test and Clinical Skills Assessment Examination (CSAE) checklists, described in the accompanying article. The written test contained 10 knee and 10 ankle questions. The written test-retest reliability was 0.72. This -value was calculated by comparing each resident's total test score immediately after the intervention and again at the end of the month. Any change in a resident's test score was counted as discordant. The residents examined a standardized patient using a CSAE format including short patient scenarios presented to the resident before the examination. The residents were asked to demonstrate physical examination techniques of the ankle and knee. The investigators rated their performance using checklists, which are presented in the accompanying article.

Intervention

1. Each month, a new group of residents watched the teaching videotape with one of the investigators. As a secondary reinforcer to the videotape, the residents were given ankle and knee physical examination checklists used in the investigators' sports medicine clinic on which they could take notes while watching the videotape.
2. The physical examinations of the ankle and knee were demonstrated on the standardized patient by one of the investigators in a clinic examination room. The residents used the checklists as guides to follow the investigator doing the examination.
3. The residents performed the examinations correctly under the supervision of the investigators guided by checklists and their notes.

One-Month Follow-Up At the end of the 1-month rotation, each resident completed the written test and the CSAEs on the standardized patient and estimated the number of patients with ankle and knee injuries that they had seen during the month. The checklists with notes were not allowed as a guide during the CSAEs. After the CSAEs, the investigators gave feedback to the group about areas for improvement in their physical examination skills.

Nine-Month Follow-Up Six months after the adolescent medicine rotation, the residents were contacted to schedule a time to complete the written test, perform the CSAEs on the standardized patient, and complete an evaluation of the intervention program, using a 5-point Likert scale ranging from strongly disagree to strongly agree. Limited resident availability resulted in the mean time for the follow-up being 35 weeks (23-48 weeks).

Statistical Analyses

Pearson correlation coefficients were calculated between written test scores and CSAE score at baseline and at 1 and 9 months. Spearman correlation coefficients were calculated between the self-rated comfort level in diagnosing ankle and knee injuries and the CSAE scores at baseline. Paired t tests were performed for the written test and CSAE scores at baseline and at 1 and 9 months. Univariate analyses were used to detect relationships between the dependent variables (the change in written and CSAE scores) and independent variables (gender, number of ankle and knee examinations seen during the month). General linear regression analysis modeled the relationship between the dependent and independent variables while controlling for covariates. The scale for residents' ratings of the intervention was modified, so that strongly agree and agree were collapsed into one value and strongly disagree and disagree were collapsed into one value, resulting in a 3-point scale. Data were analyzed using the Statistical Package for the Social Sciences, Version 10.0 for Windows (SPSS, Chicago, IL). A P value of <.05 was established for statistical significance.

	RESULTS

Top Abstract Methods Results Discussion Conclusion References

Resident Characteristics

Sixty-nine percent of the residents who participated in this study were female. Ten were second-year combined internal medicine-pediatrics residents; the others were pediatric residents. The residents' self-rated comfort level in making diagnoses of ankle and knee injuries based on the history and physical examination at baseline were 2.6 ± 0.8 and 2.5 ± 0.8, respectively, indicating that most residents were neutral to uncomfortable in making these diagnoses. The comfort level with making a diagnosis of an ankle injury was not related to the baseline ankle CSAE score (r = 0.22; P = .1), and the comfort level with diagnosing a knee injury was not related to the baseline knee CSAE score (r = 0.26; P = .05). The baseline ankle CSAE scores were higher in the medicine pediatrics group compared with the pediatric resident group (18.5 vs 11.3, respectively; P < .001), while there were no group differences in the baseline written test scores and knee CSAE scores.

Written Test and CSAE Scores

The scores on the written test and CSAEs increased at 1 and 9 months, compared with baseline (Table 1). Ten residents had lower or unchanged scores on the ankle component of the written test at 9 months compared with baseline (data not shown). Seven residents had lower or unchanged scores on the knee component of the written test at 9 months compared with baseline (data not shown). The remaining residents had higher written test scores at 9 months compared with baseline.

The percentage of correctly performed techniques for the ankle and knee, doubled and tripled, respectively. These improvements persisted, with some decline, over 9 months. Four residents had lower ankle CSAE scores at 9 months compared with baseline (data not shown). Two residents had lower knee CSAE scores at 9 months compared with baseline (data not shown). All the other residents had higher CSAE scores at 9 months compared with baseline. At 9 months the residents were performing two thirds of the ankle examination techniques correctly, nearly one half of the knee physical examination techniques correctly, and answered 75% of the knowledge questions correctly. The residents took longer to complete the ankle and knee CSAEs at 1 month compared with baseline and 9 months (Table 2).

At baseline, the male residents scored higher on the knee component of the written test compared with the female residents (5.8 ± 2.2 vs 4.5 ± 1.5, respectively; P = .007) and higher on the knee CSAE (6.4 ± 2.7 vs 5.0 ± 2.0, respectively; P = .03). This latter difference persisted at 1 month (P = .002) but not at 9 months (P = .2). There was no difference between male and female residents in their ankle written test and CSAE scores. There were no differences in the baseline test or CSAE scores for the residents who followed up at 1 month compared with those who did not. There were no differences in the 1-month written test scores (P = .7) or CSAE scores (P = .3 and P = .14 for knee and ankle, respectively) between those who followed up at 9 months and those who did not.

The mean number of patients with ankle and knee complaints that each resident reported seeing during the month were 2.8 ± 3.2 and 2.2 ± 2.3, respectively, with no gender difference (P = .9). The change in ankle CSAE scores from baseline to 1 month was unrelated to the number of patients with ankle complaints seen by the resident during the month. The change in knee CSAE scores from baseline to 1 month was positively related to the reported number of patients with knee complaints seen by the resident during the month (P = .03) and was greater in the male residents than in the female residents (mean difference between change scores was 3.7; P = .02). The change in knee CSAE scores at 1 month was independently related to the reported number of knee injuries seen during the month (R² = 0.26; P = .001) and male gender (R² = 0.09; P = .008, adjusting for number of injuries seen). The change in knee CSAE scores at 9 months was not related to the number of knee injuries seen during the month on the adolescent medicine rotation.

The correlation between the written test and CSAE scores improved over time for the knee examination. This correlation increased for the ankle at 1 month and returned to baseline at 9 months (Table 3). The residents' rating of the teaching intervention was highly favorable (Table 4).

	DISCUSSION

Top Abstract Methods Results Discussion Conclusion References

The residents' physical examination skills at baseline were suboptimal, as was their confidence in making diagnoses of ankle and knee problems. This is consistent with other reports.^2,3 In addition the residents' scores at baseline in this study highlight the lack of training in the physical examination of the musculoskeletal system that residents receive when they are medical students. Improved training in the physical examination of the musculoskeletal system during medical school is needed and has been suggested by others.^7,8 The teaching intervention tested in this study was associated with improvement in the knowledge and skills of the residents in performing physical examinations of the ankle and knee. The teaching intervention described in this and the accompanying article may serve as a model teaching intervention that could be incorporated into routine pediatric residency training. What is novel about this teaching intervention is that there is a videotape accompanied by written descriptions of correct performance of the physical examination techniques. This teaching intervention could fill the gap between residents' need for hands-on demonstration of techniques and the lack of written descriptions coupled with practical demonstration of techniques. The CSAEs used in this study are at the level of thoroughness and complexity used by sports medicine specialists in evaluating patients. That the majority of techniques and knowledge were mastered and maintained by the residents suggests an improved ability to make correct diagnoses. This is the ultimate goal of the teaching intervention.

There are other models for teaching the musculoskeletal examination.^9-13 McGaghie et al¹¹ reported that medical students correctly performed 44% to 49% of the items on a knee examination checklist over 3 consecutive years. This is a similar rate of accuracy for performance of the knee examination in this report. McGaghie et al¹¹ described their experience of designing, pilot testing, and using checklists for evaluating the musculoskeletal physical examination skills of medical students. Rheumatologists designed the checklists for an ideal performance of the physical examination of the knee; however, the level of detail precluded their use as practical tools for primary care residents. These checklists were not published. Lawry et al¹⁰ reported methods for teaching a screening musculoskeletal examination that lacked the detail to make a diagnosis. The checklist used to evaluate the physical examination skills was not published. The Society of Teachers of Family Medicine published a curriculum guide for teaching sports medicine to family practice residents.¹² In this curriculum guide, it is suggested that evaluation of residents include direct observation complemented by checklists with component skills; however, the checklist was not provided. Stirling et al¹³ described a teaching program in which physical therapists and physicians gave instruction and feedback about the examination of the knee and upper extremity to internal medicine residents. Performance checklists and a written examination were used, yet not provided in the article. In summary, the models available in the literature are too complex to be practical, too superficial to allow a diagnosis to be made, have not published their checklists so that they could be tested in other settings, or lack long-term follow-up of the learners' skills. The model described in this article is the first to describe the implementation and evaluation of a teaching model to learn physical examination techniques in a pediatric residency program. The residents' rating of the teaching intervention indicates that it was well received and valuable. Similar support for structured training interventions for residents have been reported.¹⁴ The next step will be to test this method in other residency programs.

There was a decline in skills and knowledge between the intervention and final follow-up. This was expected and has been reported elsewhere.¹⁰ Knowledge and skills at 9 months remained above baseline levels, suggesting that the effect of the teaching intervention was not transient. There was an added effect on improving knee examination skills related to the number of patients with knee injuries seen during the month by the resident that did not occur for the ankle examinations. The knee examination requires more technical skill in performing provocative maneuvers than does the ankle examination, explaining the lower knee CSAE scores compared with the ankle CSAE scores, at baseline. In contrast, the ankle is a smaller joint, is easier to access compared with the knee, and has only 2 provocative maneuvers associated with its examination. At baseline, residents were more skilled at ankle examinations and may have learned more quickly with the teaching intervention for the ankle examination. The residents' skills in knee examinations were lower at baseline, and because there is more to learn, reinforcement, in the form of seeing more patients in clinic, seemed to be more important for the knee examination. This speaks to a limitation of the study: all improvements in knowledge and skill during the month cannot be ascribed to the 1-time teaching intervention. It may be that if this teaching model is incorporated into other teaching programs, it may not be as effective, especially for the knee examination, if there are no patients with knee injuries seen by the residents or if an attending physician who is skilled with these examinations is not present. This will be considered when this model is adapted for use in other residency programs.

The increased correlation between the written test and CSAE scores for the ankle at 1 month and the knee at 1 and 9 months suggests a better integration of knowledge and skill in performance of the examinations. The lack of a relationship between the written test and CSAE scores for the ankle and the presence of a relationship for the knee at 9 months suggest that the knowledge and skill remained better integrated for the knee, yet the ankle CSAE score remained higher at 9 months. The knee examination is simply harder to perform and the residents improved more in the knee examination than they did with the ankle examination. Measures of academic aptitude and clinical skills do not always correlate because they may assess different features of achievement.¹¹ In addition, it was not a goal of this study to improve the correlation of written and CSAE scores.

Cost of educational interventions is important in curriculum design. We estimated the annual cost of this intervention to be $4364 assuming: 1) the study coordinator/standardized patient would spend 6 hours per month on the evaluation and intervention at an hourly rate consistent with an annual salary of $38 000 plus 25% fringe benefits; 2) 1 sports medicine pediatric faculty member would spend 3 hours per month on intervention at an hourly rate consistent with an annual salary of $100 000 plus 25% fringe benefits; 3) supplies = $200; and 4) cost of the videotape = $39.

The differences between the skills of male and female residents in test scores and CSAE scores at baseline were small and disappeared by 9 months. We do not interpret this to indicate a significant gender difference in the ankle and knee examination skills of the male and female residents.

There was a concern about follow-up bias at 9 months, ie, those who had better scores would be more willing to be reevaluated at 9 months and would overestimate the effect of the teaching intervention. Sixty-eight percent of the residents that participated at baseline and who were eligible at 9 months completed an evaluation at 9 months. There were no differences between the test and CSAE scores at 1 month in the group of residents who followed up at 9 months compared with those who did not, suggesting that follow-up bias was not significant in this sample.

A final limitation was the absence of a control group that was evaluated at baseline and 9 months but did not receive any intervention. This will be addressed in subsequent work as we export this teaching intervention to other training programs.

	CONCLUSION

Top Abstract Methods Results Discussion Conclusion References

This teaching model was associated with improvement in the knowledge and skill of pediatric residents in the physical examination of the ankle and knee. This addresses an important need in pediatric residency training. Subsequent interventions are being planned for other components of the musculoskeletal system. The next steps will include exportation of this model to other residency programs to test for efficacy in improved resident knowledge and skills.

	ACKNOWLEDGMENTS

This project was supported in part by an unrestricted educational grant from Aircast, Inc.

This project could not have occurred without the support of Ayse McCracken and Kay Tittle of Texas Children's Pediatric Associates, Inc.

	FOOTNOTES

Amy C. Fetterhoff is currently at Wyle Laboratories, Houston, Texas.

Received for publication Aug 3, 2000; accepted Nov 1, 2000.

This work was presented in part at a poster session at the Society for Pediatric Research Meetings; May 16, 2000; Boston, MA.

Reprint requests to (A.C.H.) Adolescent Medicine and Sports Medicine Section, Baylor College of Medicine, Texas Children's Hospital, 6621 Fannin St, MC 3-3340, Houston, TX 77030-2399. E-mail: alberth{at}bcm.tmc.edu

	ABBREVIATIONS

CSAE, Clinical Skills Assessment Examination.

	REFERENCES

Top Abstract Methods Results Discussion Conclusion References

1. Anderson JM, Felsenthal G Residency training in physical medicine and rehabilitation: clinical and didactic experience. Arch Phys Med Rehabil 1990; 71:372-375 [Medline]
2. Taras HL, Badar PR Ten years of graduates evaluate a pediatric residency program. Am J Dis Child 1990; 144:1102-1105 [Abstract/Free Full Text]
3. Stirling JM, Landry GL Sports medicine training during pediatric residency. Arch Pediatr Adolesc Med 1996; 150:211-215 [Abstract/Free Full Text]
4. Biro FM, Gillman MW, Parker RM, Khoury PR, Seigel DM Surveying graduates of combined internal medicine-pediatric residency programs. Acad Med 1990; 65:266-271 [Medline]
5. Stewart GW, Roig RL Sports medicine in physical medicine and rehabilitation residency programs. Arch Phys Med Rehabil 1995; 76:230-233 [CrossRef][Medline]
6. Hettle M, Draddom RL Curriculum needs in physical medicine and rehabilitation for primary care physicians: results of a survey. Am J Phys Med Rehabil 1995; 74:271-275 [Medline]
7. Whitley JD, Nyberg KL Exercise medicine in medical education in the United States. Phys Sports Med 1988; 16:93-100
8. Centers for Disease Control and Prevention. Injury Control in the 1990s: A National Plan for Action. Atlanta, GA: National Center for Injury Prevention and Control; 1993
9. McGaghie WC, Kowlowitz V, Renner BR, A randomized trial of physicians and physical therapists as instructors of the musculoskeletal examination. J Rheumatol 1993; 20:1027-1032 [Medline]
10. Lawry GV, Schuldt SS, Kreiter CD, Densen P, Albanese MA Teaching a screening musculoskeletal examination: a randomized, controlled trial of different instructional methods. Acad Med 1999; 74:199-201 [Medline]
11. McGaghie WC, Renner BR, Kowlowitz V, Development and evaluation of musculoskeletal performance measures for an objective structured clinical examination. Teach Learn Med 1994; 6:59-63
12. Society of Teachers of Family Medicine. Sports Medicine: Curriculum Guide for Family Practice Residents. Kansas City, MO: The Society of Teachers and Family Medicine; 1990:21-28. ISBN No. 0-942295-1990
13. Stirling J, Wood J, Lloyd J An instructional module in musculoskeletal examination for residents incorporating physical therapists as patient-instructors and evaluators. Acad Med 1997; 72:453-455 [Medline]
14. Joorabchi B Objective structured clinical examination in a pediatric residency program. Am J Dis Child 1991; 145:757-762 [Abstract/Free Full Text]