PEDIATRICS Vol. 118 No. 4 October 2006, pp. e985-e991 (doi:10.1542/peds.2006-0515)
ARTICLE |
Clinical and Education Workload Measurements Using Personal Digital Assistant–Based Software
a Departments of Pediatrics
b Physiology, Hospital for Sick Children and University of Toronto, Toronto, Ontario, Canada
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
OBJECTIVE. There are no accepted and practical measures of the relative clinical and educational activities of pediatricians who work in an academic health science center. Such measures are necessary for justification of existing and future human resource plans and evaluation of the activities and performance of physicians. The limited literature on the measurement of physician workload usually focuses on a specific subspecialty group and does not account for such issues as indirect patient care, such as telephone calls or e-mail consultations; variables that affect the delivery of clinical care, including patient acuity and complexity; and the presence of students during the patient care activities. After completing a pilot study that assessed the educational workload of faculty members, we adapted existing personal digital assistant technology and software to document clinical and educational activities.
METHODS. Twenty full-time physicians from 4 subspecialty pediatric divisions participated in a 2-week evaluation project in May through June 2005. Clinical activities, with and without trainees, and educational activities were collected with the use of personal digital assistants. Software allowed an individualized division-specific drop-down menu. Information that was collected included clinical (location of activity, diagnosis, and time requirement) and educational activities. After completion of a 2-week data collection period, each physician was asked to complete a 5-question evaluation form.
RESULTS. The project was completed successfully with capture of additional clinical and educational activities. A 5-question evaluation form was completed by 70% of the participants at the end of the 2-week data collection. Data on clinical and educational activities were analyzed qualitatively and graphed.
CONCLUSIONS. This method of workload data collection added significant information in capturing activities that are not measured in traditional workload evaluations for either clinical activities, such as e-mail, telephone, and patient information review, or educational endeavors, including mentoring and educational lectures and presentations.
Key Words: PDA • clinical care • e-mail
Abbreviations: PDA—personal digital assistant
The measurement of workload provides important information to those who are involved in the planning of resources, both human and financial. Information can be provided on the productivity of individual employees or groups and assist in the assessment of performance. There has been an increase in the use of technology to track workload in the health care industry. Currently, limited literature is available on the measurement of physician workload.
Ideally, for a physician workload measurement to be a useful tool, it should account for all time spent in clinical care, teaching, research, and administrative tasks that are involved in a physician's daily activities. Some of the tasks in the clinical component of workload that generally are not included in standard measures are those that are of short duration, such as telephone calls, e-mail, short hallway conversations, and reviewing laboratory tests and radiograph results. Although these may take up brief periods of time in isolation, they do account for a significant amount of time each day. In an academic health science center, clinical work and teaching often overlap, and it is important to measure both aspects of work and understand the time that is taken for each activity.
A previous study conducted at our institution tested a new tool for assessment of educational workload by faculty members (S.T., L. Lingard, J. Hellmann, et al, unpublished data, 2004). The tool was designed to capture all aspects of educational work to foster excellence in teaching, facilitate academic promotion and compensation, and guide distribution of educational resources. The workload tool had 4 templates: (1) clinical/bedside teaching, (2) teaching/educational activities, (3) mentorship; and (4) administrative/evaluative activities. The educational personal digital assistant (PDA) tool that was constructed was considered to be usable by faculty with minor modifications after a pilot study.
The main focus of the current study was to evaluate the ability of pediatricians to use PDAs to capture patient care data (consisting of diagnosis, location, and type of service provision) and allow for comparison with other hospital and departmental data collection methods. A second goal was to evaluate the feasibility of faculty documentation of clinical and educational workload data. Industry-generated software was adapted for use on the basis of a template that was generated by consensus for clinical work and templates that already were developed and piloted in our institution for educational workload.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
Twenty-seven full-time physicians from 4 pediatric subspecialty divisions (nephrology, neurology, respiratory medicine, and rheumatology) were eligible to participate in a 2-week evaluation project (May–June 2005). Twenty physicians participated in the project. Seven physicians were unable to participate because they were either out of town or too new in their role to take part in the study. After the 2-week data collection period, an e-mail was sent to each participant asking them to complete a 5-question nonstandardized evaluation form. The evaluation form asked for feedback on (1) the amount of time spent entering information; (2) technical, program, or training problems; (3) whether the project provided accurate capture of clinical and educational activities; (4) what participants liked best and least about the experience; and (5) any recommendations for improvement. Fourteen pediatricians responded to the e-mail and completed the questionnaire. A second request was not sent to those who did not respond.
Clinical activities with and without trainees (including location, diagnosis, and time requirement) and educational activities (lectures/seminars, clinical teaching, evaluation, and mentorship) were collected with the use of PDAs after loading of individualized division-specific drop-down menus. The drop-down menus (Fig 1) were created using templates that were developed by the project leaders. All physicians and administrative staff completed a 1-hour training session before the start of the study. This study was approved by the Research Ethics Board of our institution.
|
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
Data that were obtained from the 2-week collection period were graphed to assess average time for clinical activities (Fig 2) and number of entries recorded at each site of clinical activity (Fig 3) and then were separated into clinical activities that were conducted by the 4 pediatric subspecialty divisions (Fig 4). Data on educational activity were graphed by type of activity, including mentoring and presentations (Fig 5).
|
|
|
|
The clinical data that were obtained were separated into activities with trainees and activities without trainees. It was noted that in most cases, physicians who were involved in activities with trainees spent a greater percentage of time compared with periods with no trainee supervision responsibilities. For example, the average time that was devoted to consultations increased by 16.4% when trainees were involved. Daily care was increased by 14.3%, follow-up care by 21.4%, and rounds duration by 17.7%.
Activities such as e-mail and telephone calls accounted for 14% of the total time entered for activities that did not involve trainees. Overall, these activities, which generally are not captured by other methods of data collection, comprised 7.8% of all time spent by participating physicians during the 2-week study period.
In Fig 2, "consultations" includes clinical activities that were performed in the emergency department, in the outpatient clinics and on the inpatient wards. Results for "rounds" consisted of both direct patient care rounds and teaching rounds. The option for "other" included such activities as travel to other hospitals and participation in business meetings.
The program allowed for additional information to be entered regarding the site of clinical activity. In Fig 3, an option for "other" was available and space was provided to enter site detail. For example, some clinical activities were conducted in the Diagnostic Imaging Suite or in Pathology.
Figure 4 provides information about the teaching patterns in the 4 divisions involved in this study. Although more data would be required to analyze this information further, these patterns could provide valuable information on educational efforts in an academic health science center. The average time and the number of entries for each type of educational activity are shown in Fig 5. Table 1 summarizes the responses of the 14 physicians who completed the questionnaire.
|
Limitations in the conduct of this pilot study are recognized. Time was entered in total hours or minutes involved in an activity on the drop-down menu. It is recognized that more accurate measures would have been provided by using start-stop times; however, the investigators elected to use this method of time calculation in the pilot to ensure compliance.
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
There is little experience specifically reviewing physician workload and the tools that are used for this purpose. The majority of the available literature focuses on performance assessment, billing, exchange of patient clinical information, and access to information to assist in the process of medical decision making.
The impact of workload on workflow, patient care, and service delivery outcomes was examined by Munoz et al,1 who reported the relationship between the number of patients per diagnosis-related group who were treated by individual physicians and the hospital resource consumption. Patients who were involved in this study were categorized by admission type (eg, emergency or nonemergency), and the physicians were grouped as either high load (8 or more patients) or low load (5 or fewer patients). The length of stay and total hospital costs were examined. Data analysis indicated that low-load physicians consumed more resources than did the high-load physicians, in part because of illness severity of the patients seen. Additional analysis of the data showed a decrease in the hospital cost per nonemergency patient as the number of patients seen per physician increased. The results were intended to inform discussions on access to care, quality of care, and the costs associated with health care service delivery. It was suggested that these findings may prompt hospital administration to encourage the employment of high-load rather than low-load physicians for the treatment of nonemergency patients.
Several studies have been conducted to examine the influence of patient complexity on physician workload. The variability in physician service times related to patient service category, length of stay, and intensity of service was evaluated by Graff et al.2 Their study concluded that the case mix of patient services did in fact affect the workload of emergency physicians and should be considered in planning departmental staffing needs.
It has been suggested that the use of PDAs at point of care can generate increased revenue. In 2003, the Department of Medicine at the Brigham and Women's Hospital completed a 3-month study using PDAs that drew an increase in revenue of 15% as a result of improved charge capture at point of care. This study was well received by physicians who use PDAs for reading and analyzing patient data as well as for billing.3
The use of PDAs can be valuable in the area of performance management. Davidson4,5 reported on the measurement of physician productivity and the validity of using certain variables for this measurement (eg, patients per hour) and suggested using a variable that reflects complexity of care, as well as actual numbers of patients. This work focused on the measurement of physician activity in relation to performance management. Another study examined the practicality of using PDAs for the collection of logbook data, procedural performance data, and critical incident reports in anesthesia trainees. This study concluded that PDA technology can be used effectively to track physician performance.6
In the field of medical education, a new instrument called the Relative Value Scale in Teaching7 was developed in response to the Association of American Medical College's "Assessing Change in Medical Education—The Road to Implementation" report,8 which recommended "the delineation of better methods to measure and document faculty members' educational accomplishments" and "administrative mechanisms to document teaching accomplishments." Design and testing of a simple spreadsheet-based system was described for calculation of faculty productivity in a number of areas, including teaching.9 A computerized spreadsheet model that estimates teaching and educational administration has been used for faculty time management and career development, department planning, budget planning, clinical scheduling, and mission cost accounting.10 Moore et al11 used PDAs as part of a study on delivery of medical education. One aspect of the initiative captured the clinical experience of students, although this was the least popular use of PDAs. A workload tool that was developed at our institution was intended to track the educational activities of faculty members to provide information that may be useful for the purpose of allocating resources. Faculty members who were involved in this study found the tool to be usable but suggested modifications to the program.
The use of PDAs can provide improved access to patient and medical information, can effect better communication among health care workers, and may assist in the prevention of medical errors.12,13 There is considerable discussion of the future of handheld devices in health care, including use in patient data management, prescribing, and billing. The exchange of clinical information at the point of care can be greatly facilitated by the use of PDAs.14 Reaction to the use of PDAs was examined in a study that included physicians who were and were not familiar with the use of PDAs. The results indicated that users believe that PDAs do increase productivity and improve the care of patients.15
The complexity of the work that is conducted by physicians in a clinical setting is not captured adequately by the current methods of measuring workload. Standard administrative data warehousing does not provide sufficient detail to capture this complexity of physician workload; therefore, this type of expanded data collection allows for in-depth analysis of functions in clinical and educational activity. Many brief tasks are not accounted for, and these tasks in total do take up substantial amounts of a physician's time. The literature provides some evidence to support the use of PDAs in clinical settings. The use of PDAs is gaining acceptance in health care, although, at present, mainly for the purpose of clinical reference and prescribing medications.16 As acceptance increases, it is anticipated that there will be more use of PDAs, including use for the measurement of workload.
This study was intended to determine the feasibility of this type of data gathering tool. Our intent is to explore this method further to evaluate other issues such as patient complexity, billing, and physician performance. It is noted that the usefulness of this particular tool for billing function in isolation likely would be unsuccessful; however, in combination with the collection of other patient care and education data, there may be the opportunity for greater effectiveness in enhancing revenues.
|
CONCLUSIONS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
This workload measurement project was considered successful. Pediatricians who participated indicated that they can accept this method of data capture despite the added workload for a short period of time if the tool is easy to use and practical.
Additional significant time is expended when clinical activities involve trainees. Physicians in academic health science centers are expected to participate in educational activities in a number of settings with substantial time commitment. The overlap of clinical and teaching work necessitates more faculty time and deserves additional assessment. It would be of interest to analyze the time spent in clinical encounters with trainees at various levels (eg, medical students compared with residents and clinical fellows). It is expected that those at an earlier stage of training will require more time commitment on the part of the physician supervisor.
This method of workload data collection added significant information in capturing activities that are not measured in traditional workload evaluations for either clinical activities, such as e-mail, telephone consultation, and patient information review, or educational endeavors, including mentoring and educational lectures and presentations. The information will be of assistance in resource allocation.
The software that was used in this pilot study will be modified taking into consideration the feedback that was received from the study participants. A second, larger study will be conducted to evaluate further the effectiveness of PDAs in obtaining physician workload information with evaluation of additional factors of patient acuity and complexity. With a larger sample size, it will be more possible to generalize the results beyond the survey population.
|
ACKNOWLEDGMENTS |
|---|
This project was supported by Pediatric Consultants of the Hospital for Sick Children. Resilience Software, Inc, was under contract to the Department of Pediatrics to develop the software and provide technical support for this project.
We thank the members of the medical staff who participated in this project.
|
FOOTNOTES |
|---|
Accepted Apr 27, 2006.
Address correspondence to Daune L. MacGregor, MD, Department of Pediatrics, Hospital for Sick Children, 555 University Ave, Room 1436, Toronto, Ontario, Canada M5G 1X8. E-mail: daune.macgregor{at}sickkids.ca
The authors have indicated they have no financial relationships relevant to this article to disclose.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONCONCLUSIONSREFERENCES |
|---|
- Munoz E, Mulloy K, Goldstein J, Josephson J, Tenenbaum N, Wise L. Physicians' patient load per DRG, the consumption of hospital resources, and the incentives of the DRG prospective payment system. Acad Med. 1990;65 :533 –538[Web of Science][Medline]
- Graff LG, Wolf S, Dinwoodie R, Buono D, Mucci D. Emergency physician workload: a time study. Ann Emerg Med. 1993;22 :1156 –1163[CrossRef][Web of Science][Medline]
- PDAStreet. Brigham & Women's Hospital chooses for handhelds. August 19, 2003. Available at: www.pdastreet.com/articles/2003/8/2003-8-19-Brigham-Women-zs.html. Accessed October 31, 2005
- Davidson SJ. Measuring productivity means setting expectations, and—possibly—managing performance. Emerg Med News. July 2000:25 ,34
- Davidson SJ. The why's and how's of relative value units (RVUs). Emerg Med News. August 2000:42 –43
- Bent PD, Bolsin SN, Creati BJ, Patrick AJ, Colson ME. Professional monitoring and critical incident reporting using personal digital assistants. Med J Aust. 2002;177 :496 –499[Web of Science][Medline]
- Bardes CL, Hayes JG. Are the teachers teaching? Measuring the educational activities of clinical faculty. Acad Med. 1995;70 :111 –114[Web of Science][Medline]
- Educating medical students: assessing change in medical education—the road to implementation. Acad Med. 1993;68 :S1 –S46[Web of Science][Medline]
- Hilton C, Fisher W Jr, Lopez A, Sanders C. A relative value-based system for calculating faculty productivity in teaching, research, administration, and patient care. Acad Med. 1997;72 :787 –793[Web of Science][Medline]
- Daugird AJ, Arndt JE, Olson PR. A computerized faculty time-management system in an academic family medicine department. Acad Med. 2003;78 :129 –136[Web of Science][Medline]
- Moore L, Richardson BR, Williams RW. The USU Medical PDA initiative: the PDA as an educational tool. Proc AMIA Symp. 2002;528 –532
- Fischer S, Stewart TE, Mehta S, Wax R, Lapinsky SE. Handheld computing in medicine.
J Am Med Inform Assoc. 2003;10
:139
–149
[Abstract/Free Full Text] - Crimino JJ, Bakken S. Personal digital educators. N Engl J Med. 2004;352 :860 –862
- Wilcox RA, La Tella RR. The personal digital assistant: a new medical instrument for the exchange of clinical information at the point of care. Med J Aust. 2001;175 :659 –662[Web of Science][Medline]
- McAlearney AS, Schweikhart SB, Medow MA. Doctors' experience with handheld computers in clinical practice: qualitative study.
BMJ. 2004;328
:1162
[Abstract/Free Full Text] - Knollmann BC, Smyth BJ, Garnett CD, et al. Personal digital assistant-based drug reference software as tools to improve rational prescribing: benchmark criteria and performance. Clin Pharmacol Ther. 1995;78 :7 –18
PEDIATRICS (ISSN 1098-4275). ©2006 by the American Academy of Pediatrics
CiteULike 
Connotea 
Del.icio.us 
Digg 
Facebook 
Reddit 
Technorati 
Twitter What's this?
This article has been cited by other articles:
|
|
 |
|
  M. J. Trotter, E. T. Larsen, N. Tait, and J. R. Wright Jr Time Study of Clinical and Nonclinical Workload in Pathology and Laboratory Medicine Am J Clin Pathol, June 1, 2009; 131(6): 759 - 767. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
