 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
PEDIATRICS Vol. 101 No. 3 March 1998, p. e2
From the Department of General Pediatrics at Children's National Medical Center and George Washington University School of Medicine, Washington, DC.
 |
ABSTRACT |
|---|
Top
Abstract
Introduction
Methods
Results
Discussion
References
|
|---|
Objective. Under managed care, telephone management is crucial to pediatric practice, but an effective method is needed to teach residents telephone skills. Our objective was to design an interactive CD-ROM program to teach residents an organized, consistent approach to telephone complaints and to determine whether use of the program was associated with better subsequent telephone management than reading the same information.
Setting. The general pediatric ambulatory center of a tertiary care children's hospital.
Participants. A total of 24 PL-2 and PL-3 pediatric residents.
Design. A randomized, prospective, controlled comparison was conducted of resident management of two telephone calls: a 5-year-old with cough and trouble breathing, and a 7-year-old with fever. Thirteen residents were randomized to the computer group and 11 to the reading control group.
Intervention. Scripts, scoring, and feedback for 10 CD-ROM-simulated calls were developed from texts and pediatrician survey using a modified Delphi technique. Volunteers acted out the caller's role in scenario scripts and were recorded onto a CD-ROM. The computer simulated calls by recognizing questions typed in a free-form format and answering with a voice response. Feedback was provided for omissions in history-taking and errors in assessment, triage, and home management. The computer group worked through the CD-ROM calls while the control group had equal time to read the same information.
Evaluation Measures. A trained, standardized patient acted as the mother in pretest calls placed at the beginning of the month and posttest calls at the end. Calls were recorded and scored in a blinded manner using scoring templates and on interpersonal skills using the Patient Perception Questionnaire.
Results. Pretest scores for the two calls were similar in the computer versus the control group (cough, 70.33% ± 8.36 vs 68.46% ± 6.73; fever, 75.64% ± 9.82 vs 73.59% ± 9.06). Posttest scores were significantly higher in the computer group than in the control group on both calls (cough, 79.08% ± 8.17 vs 69 ± 13.3; fever: 83.33% ± 9.96 vs 70.35% ± 9.66). Interpersonal skills also were similar pretest (19 ± 3.4 vs 20 ± 2.7). There was modest improvement in both groups without a statistically significant difference in posttest scores (24.2 ± 2.9 vs 22.5 ± 3.1).
Conclusions. Use of this CD-ROM telephone management program was associated with better postintervention telephone management. The program augments faculty instruction by teaching a consistent, general approach to telephone management.
Key words: telephone management, resident education, computer-aided instruction.
Telephone management traditionally has been an integral
component of pediatric practice.1,2 Today, as primary
care physicians serve as gatekeepers for health care services under
managed care, effective telephone management is crucial. Despite the
significance of telephone encounters in pediatric practice, fewer than
half of pediatric residency programs provide telephone
training.3 The pediatric telephone call is especially
challenging because visual assessment of a child's appearance and
interaction with the environment are not available to gauge the
severity of illness. Failure to ask appropriate questions to obtain
essential information without visual cues can lead to inappropriate
assessment and management,4 with potentially
life-threatening sequelae.5
Few telephone instructional programs have reported effectiveness in
teaching residents telephone management skills. Working with an expert
in software design, we developed and evaluated a CD-ROM interactive
telephone management program to teach residents telephone skills. The
CD-ROM allows the computer to simulate calls and provide pertinent
feedback. Our hypothesis was that guided practice with feedback using
CD-ROM cases would enable residents to obtain information, assess
patients, and provide advice over the telephone significantly better
than those who learned by reading similar material.
CD-ROM Development
Scripts for CD-ROM cases representing the 10 most common
telephone complaints6 were written to simulate common
telephone conversations with parents. The cases are listed in Table
1.
TABLE 1
INTRODUCTION
Top
Abstract
Introduction
Methods
Results
Discussion
References
METHODS
Top
Abstract
Introduction
Methods
Results
Discussion
References
CD-ROM Telephone Scenarios
To determine the questions that should be asked and feedback that should be provided for the chief complaints depicted in the CD-ROM cases, the historical information essential to forming an appropriate diagnosis and management plan was extracted from articles, standard pediatrics texts, and telephone advice texts. Questions were then formulated to elicit the essential information, and management options were listed. The lists of questions and management options for each of the 10 scenarios were presented to focus groups consisting of pediatric emergency medicine and general pediatric faculty, pediatric residents, telephone advice nurses, and community pediatricians. Members of the focus groups were asked to score questions as essential, helpful, or unnecessary, and to determine criteria for appropriate triage and home management advice. Responses were analyzed using a Delphi technique.7 There was very strong agreement among members of the focus groups regarding which questions they rated as essential and which management options they felt were appropriate.
In the introductory section of the program, residents were encouraged to use a consistent, organized format for taking a history over the telephone and to learn to use the parent's description of the child to gauge the severity of the complaint. Data-gathering was divided into nine general categories: obtaining identifying information; clarifying the chief complaint with an open-ended question; determining the severity and duration of the chief complaint; ascertaining the child's overall appearance; previous home therapy; associated symptoms; illness exposure; medical history; and acknowledging the emotional tone of the caller.
Residents began by identifying themselves and obtaining essential identifying information about the caller such as parent and child's names, child's age, name of primary care provider, telephone number, and chief complaint. They were expected to follow up with an open-ended question to learn what was most worrisome about the chief complaint. An open-ended question is emphasized because Korsch concluded in her classic communication research that failure to address the parent's main worry resulted in dissatisfaction with the doctor-patient encounter.8 Residents then asked directly about the severity and duration of the chief complaint, if these were not evident in response to the open-ended question.
Residents often have difficulty assessing how sick the child is over the telephone, because they have been trained to use visual cues such as the child's environmental interaction almost intuitively to assess the overall state of health. Therefore, they were directed to ask questions specifically about the child's overall appearance and activity level. Previous home therapy was assessed to judge the level of sophistication of the caller and to gain information about previous medical advice and current management. Residents were directed to always ask about medical history, including medication use, whereas a brief review of systems or determination of illness exposure was helpful for some complaints.
Residents also were instructed to acknowledge the emotional tone of the caller to calm or reassure parents and to establish rapport. To allow residents to respond to angry, frustrated, and frightened callers, volunteers acted out the caller's role in scripts written for each of the scenarios and were recorded onto a CD-ROM. Residents and staff were asked how they would phrase individual questions so that different ways of phrasing key questions could be programmed, enabling the computer to recognize questions typed in a free-form format and to answer with a voice response.
This program was designed to encourage a consistent, organized approach to the presenting problem. Data-gathering was divided into general categories as described, followed by the assessment and triage, and concluding with call back and home management advice. Feedback was provided for omissions in history-taking, and errors in assessment, triage, and home management. The entire program can be read from the computer monitor, with the introduction and interactive data-gathering portions having a voice component as well.
CD-ROM Evaluation
To test our hypothesis that use of the program would result in better subsequent telephone management than reading the same information, we conducted a randomized, controlled comparison of the pre- and posttest scores of 24 second- and third-year pediatric residents during their month-long ambulatory rotation on two telephone scenarios placed by a trained, standardized patient. The study was conducted over a 9-month period from June 1996 until March 1997.
All second- and third-year residents were asked to participate if they did not have vacation or other scheduling conflicts during their ambulatory rotation. No residents refused. Residents consented to allow some of their calls to be tape-recorded, but were not told which calls were from the standardized patient.
Both groups of residents attended a small group session covering the expectations and important aspects of telephone management (Fig 1). Next, each resident spent a morning with the telephone advice nurse, first listening to her answer calls and then answering some calls themselves under supervision of the nurse. Practical aspects such as protocol use, documentation, and interpersonal aspects of telephone management were emphasized during this one-to-one session. The pretest calls were placed by the standardized patient during this session.
|
Eleven residents then were randomly assigned to the reading control group, and 13 to the computer group. Control residents were given copies of advice protocols and readings covering the same content as in computer cases. Both groups had two 90-minute sessions to complete the cases or the readings. To ensure that there was no contamination between groups, residents in the reading group were told not to share their readings with those in the computer group, and individual access codes necessary to use access the computer program were given only to those in the computer group.
After completing the readings or the cases, the residents spent two half-day sessions independently answering telephone calls from parents of patients followed in the General Pediatric Ambulatory Center. The two posttest telephone calls were placed at this time. All were free to refer to advice protocols with pre- and posttest calls.
Both groups received the same two pre- and posttest telephone calls: a 5-year-old with cough and trouble breathing and a 7-year-old with fever. Calls were recorded and scored using standard templates (Table 2) and on interpersonal skills using the Patient Perception Questionnaire9 (Table 3) by the standardized patient who was blinded to the group assignment.
|
|
The standard scoring templates for the test cases were developed from the consensus of a survey of 100 pediatricians in the Washington, DC area. Sixty percent of those surveyed responded. Pediatricians were presented with a chief complaint and age of the child and asked to rate a series of questions or recommendations as essential, helpful, or unnecessary.
The standardized caller had previous training as a standardized patient. She had an additional 10 hours of training for this study, memorizing the scripts, and in role-play sessions with the investigators to ensure that her answers were correct and consistent. Interrater reliability was established at .91 by one of the investigators who listened independently to tapes and scored recorded calls blinded to resident group assignment and to the standardized patient assessment.
| |
RESULTS |
|---|
|
Top
Abstract
Introduction
Methods
Results
Discussion
References
|
|---|
Statistical analysis was performed using 
2 or
the Mann-Whitney test on Minitab statistical software (Table
4).
|
The number of third year residents (PL-3) and the pretest scores was similar. Posttest scores were statistically significantly higher in the computer group than in the reading group on both calls. Some posttest scores were lower than pretest scores in both groups. All residents reviewed advice protocols with nurses before pretest calls, and residents in the reading group were provided copies of protocols as part of their readings. Few residents in the reading group and no residents in the computer group referred to protocols when taking the posttest calls.
Interpersonal skills improved modestly in both groups on the Patient Perception Questionnaire. The computer group scored higher posttest, but the difference was not statistically significant.
In an anonymous written evaluation of the telephone training, all of the residents rated the experience positively. All of the residents in the computer group rated the program as helpful or very helpful. A total of 73% of the reading group rated the program helpful or very helpful, with the remainder giving neutral ratings. Residents commented that the computer program was "fun," "good cases, pretty typical," "nonthreatening learning situation," "it was pretty close to being real," "gave extra time to think," "pointed out important questions I should have asked/advice I should have given," "easy and fun," and "helped me organize my thoughts; very good learning tool." The primary criticism was that the computer did not recognize all ways of phrasing a question and did not have answers to all questions. The primary criticism from residents in the reading group was that they wanted to do the computer cases.
| |
DISCUSSION |
|---|
|
Top
Abstract
Introduction
Methods
Results
Discussion
References
|
|---|
Residents must learn telephone skills because telephone management is an essential part of pediatric practice. Although institutions such as our own have implemented telephone advice programs staffed by nurses using protocols, physicians are ultimately responsible. Many physicians in smaller communities still answer the majority of calls, especially after-hours. Senior residents have the knowledge and skill necessary to correctly assess and manage most patients encountered in the clinic or emergency department. However, they often fail to gather enough pertinent information to assess and manage the same complaints over the telephone. Lack of a consistent, organized approach to telephone management has been a problem identified in previous studies.4,10 We emphasized teaching a consistent, general format that could be applied to a variety of complaints.
Before the development of this computer program, our residents had little formal training in telephone management. Fosarelli found that program size and manpower were predictive of whether pediatric residency programs provided telephone training.14 Large residency programs were less likely to provide telephone training, and reported instruction was usually by lecturing. Although lecturing is an ineffectual means of teaching an interactive skill, it is the method most commonly reported because it appears to be an efficient way to deliver the content. Increasing demands on faculty to see more patients and precept trainees in the clinic have limited the time available to precept telephone calls.
Although interactive programs have been developed, there is only modest evidence that they are more effective at teaching telephone management than a traditional didactic approach. An innovative program described by Kosower at UCLA called T.A.L.K. taught telephone communication skills by allowing residents to analyze recorded calls in group and individual feedback sessions.15 Evans used patient simulators to teach telephone communication skills to family practice residents.16 Although both programs were well received by residents, neither provided an objective evaluation of their effectiveness at improving skills.
Several researchers did evaluate the effectiveness of their programs using standardized patients. Curry and Schwartz studied the effectiveness of a small group discussion regarding important questions to be asked for a complaint of vomiting and diarrhea using five residents in the discussion group and five in a control group.17 There was no statistical difference in posttest scores between the groups at either 6 days or 28 weeks. Wood found that role-play sessions in which residents simulated parent callers helped to improve some aspects of telephone history-taking in a controlled trial.10 Smith studied the effect of feedback given to 12 medical students for a single call on their performance on a subsequent call using a standard scoring template. He found that their proficiency improved significantly between the first and second calls, but there was no control group.18
We hypothesized that to become proficient at telephone management, skill-modeling using an organized approach and guided practice was necessary.19 The computer program was modeled on previous studies using standardized patients as simulated callers.20 Use of standardized patients both as an evaluation and a teaching tool has been well documented.21,22 Standardized patients offer several advantages to an educational program: 1) the learner can be evaluated and/or provided feedback in a controlled situation; 2) use of standardized patients allows for more objective assessment because criteria used to judge each learner's performance can be tailored and validated against preset standards; 3) specific simulated encounters can be used that the learner may not experience by chance during training or are of such a sensitive nature that it would be unethical to allow an inexperienced learner to interact with actual patients/parents; 4) standardized patients can be instructed to provide feedback to the learner in place of the faculty; 5) students' and residents' history-taking and physical skills can be evaluated better because they are assessed directly.
Although use of standardized patients encourages active learning, it is a costly educational intervention for a large residency program. Therefore we used the computer to simulate a series of standardized patients. Computers have been used successfully to teach assessment and management skills in other areas of medical education.23 Using a CD-ROM program, the computer can simulate a telephone call, with the resident asking questions and receiving answers without visual cues and responding not only to the content of the complaint, but also to the emotional tone of the voice. The residents can participate in self-directed guided practice sessions with the computer to become proficient in the general approach to telephone management.
Use of this CD-ROM telephone management program was associated with significantly higher posttest scores on telephone calls placed by a standardized patient in a randomized, controlled trial. We feel this program is most helpful in teaching a consistent, general approach to telephone management, including history-taking, providing an assessment, triage, call back, and home-management advice. The CD-ROM program augments, but does not replace, faculty involvement. Faculty can help residents to refine interpersonal skills once residents have gained knowledge and practice covering general content areas.
The limitations of this study are the small sample size and lack of long-term follow-up. To maintain proficiency at any skill, continued practice and feedback are needed over time. We are currently working to incorporate an integrated telephone management experience that is not limited to 1 month, but is ongoing throughout the residency program. In addition, although the program was very well received, some residents in both groups scored lower on the posttest than on the pretest calls. Unanticipated achievement declines have been reported by Swanson and others on performance-based evaluations.27 Residents may have been more highly motivated to perform well initially, because pretest calls were taken while reviewing common protocols with the advice nurse present; however posttest calls were taken by residents unobserved and unaware of which calls were recorded. Second- and third-year residents in our program already were generally proficient at managing telephone calls, with pretest scores of 68% to 75% of ideal. More significant gains may have been seen if first-year residents participated.
The major costs in developing this program was for faculty time to develop, record, score, and provide feedback for the cases, and for the computer programmer to write the program. Based on the success of this study, our goal is to share this program with other institutions seeking to refine resident telephone management skills. We also plan to make use of the computer case-based, interactive format to develop self-directed learning modules covering other areas of primary care pediatrics to help our institution maintain its commitment to teaching while facing conflicting demands on faculty time attributable to changes in the health care environment.
| |
FOOTNOTES |
|---|
Received for publication Jul 28, 1997; accepted Nov 7, 1997.
Reprint requests to (M.C.O.) Department of General Pediatrics, Children's National Medical Center, 111 Michigan Ave NW, Washington, DC 20010.
| |
ACKNOWLEDGMENTS |
|---|
This work was supported by Bayer Institute for Health Care Communication grants for the development and evaluation of this program.
We thank Ren Lan Loai and Dr Pincetl from the Computer Informatics Department at George Washington University for their help with program design. We also thank the nursing staff and pediatrics faculty at Holy Cross Hospital and Children's National Medical Center who contributed to the CD-ROM development and evaluation.
| |
ABBREVIATIONS |
|---|
PL, postgraduate level.
| |
REFERENCES |
|---|
|
Top
Abstract
Introduction
Methods
Results
Discussion
References
|
|---|
This article has been cited by other articles:
 |
|
 |
|
  A. Kempe Telephone Care in Chronic Disease Management AAP Grand Rounds, October 1, 2004; 12(4): 49 - 50. [Full Text] [PDF]
|
|
|
|
 |
|
 C O'Brien, R Hunt, and H E Jeffery Measurement of movement is an objective method to assist in assessment of opiate withdrawal in newborns Arch. Dis. Child. Fetal Neonatal Ed., July 1, 2004; 89(4): F305 - F309. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. S. Rahi, I. Manaras, and K. Barr Information Sources and Their Use by Parents of Children with Ophthalmic Disorders Invest. Ophthalmol. Vis. Sci., June 1, 2003; 44(6): 2457 - 2460. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 1998 American Academy of Pediatrics. All rights reserved. |
||
|
||
