An Evaluation of Information-Seeking Behaviors of General Pediatricians


* Department of Pediatrics, Childrens Hospital of Iowa, Iowa City, Iowa
Office of Consultation and Research in Medical Education and Department of Family Medicine, University of Iowa, Iowa City, Iowa
Department of Internal Medicine, University of California at Fresno, Fresno, California
| ABSTRACT |
|---|
|
|
|---|
Objective. Usage of computer resources at the point of care has a positive effect on physician decision making. Pediatricians information-seeking behaviors are not well characterized. The goal of this study was to characterize quantitatively the information-seeking behaviors of general pediatricians and specifically compare their use of computers, including digital libraries, before and after an educational intervention.
Methods. General pediatric residents and faculty at a US Midwest childrens hospital participated. A control (year 1) versus intervention group (year 2) research design was implemented. Eligible pediatrician pools overlapped, such that some participated first in the control group and later as part of the intervention. The intervention group received a 10-minute individual training session and handout on how to use a pediatric digital library to answer professional questions. A general medical digital library was also available. Pediatricians in both the control and the intervention groups were surveyed using the critical incident technique during 2 6-month time periods. Both groups were telephoned for 1- to 2-minute interviews and were asked, "What pediatric question(s) did you have that you needed additional information to answer?" The main outcome measures were the differences between the proportion of pediatricians who use computers and digital libraries and a comparison of the number of times that pediatricians use these resources before and after intervention.
Results. A total of 58 pediatricians were eligible, and 52 participated (89.6%). Participant demographics between control (N = 41; 89.1%) and intervention (N = 31; 70.4%) were not statistically different. Twenty pediatricians were in both groups. Pediatricians were slightly less likely to pursue answers after the intervention (94.7% vs 89.2%); the primary reason cited for both groups was a lack of time. The pediatricians were as successful in finding answers in each group (95.7% vs 92.7%), but the intervention group took significantly less time (8.3 minutes vs 19.6 minutes). After the intervention, pediatricians used computers and digital libraries more to answer their questions and spent less time using them.
Conclusion. This study showed higher rates of physician questions pursued and answered and higher rates of computer use at baseline and after intervention compared with previous studies. Pediatricians who seek answers at the point of care therefore should begin to shift their information-seeking behaviors toward computer resources, as they are as effective but more time-efficient.
Key Words: information seeking general pediatricians questions patient care
Abbreviations: SD, standard deviation
During physicians professional workdays, many clinically important patient care questions arise and go unanswered.13 Rates of information needs vary from 0.013 to 5.044 questions per patient encounter.4 The information needs of internists, family practitioners, and general practitioners have been the most studied.13,510 These studies have found that patient care questions can be organized into a limited number of generic question categories and often are related to pharmacy and infectious diseases.2,1114
The information needs and seeking behaviors of general pediatricians have not been as well characterized. Pediatricians have been included in some studies, but their specific needs have not been characterized as a distinct group. Much of the research describing pediatric information needs is disease based or directed toward specific populations.1517 A recent literature-based needs assessment defined 171 common problems encountered by general pediatricians in the United States, which was then used to develop a pediatric digital library on the Internet.18 Digital libraries are organized collections of digital information, on the Internet, that are built and curated so that visitors can easily answer questions.
Information resources at the point of care, including computers, have a positive impact on physician decision making19,20 at reduced cost.21 Unfortunately, numerous obstacles to using evidence-based information have been identified,22 but the lack of time is one of the most common.15,19,23 Computers, including Medline searching and using the Internet, can be potentially time-efficient, but they have been used infrequently to answer clinical questions in the past.13,57,9,10,24
The goal of this study was to characterize quantitatively the information-seeking behaviors of general pediatricians and to compare the use of computers, including a pediatric digital library and a general medical library, to seek answers to questions before and after an educational intervention designed to encourage computer use.
| METHODS |
|---|
|
|
|---|
A control group versus an intervention group research design was implemented over 2 years (Fig 1). During the 19992001 academic years, 58 general pediatric faculty and pediatric residents at The University of Iowa were eligible to participate. Eligible pediatrician pools overlapped, such that some participated first in the control group and later as part of the intervention. Although it would have been preferable to have no overlap in the groups, having 20 physicians who experienced the control condition before the intervention should have acted to decrease rather than increase any observed differences. The control condition was designed to gain a measure of any changes that might have occurred by the measurement process alone, whereas the intervention condition included the influences of the measurement process and the intervention. Hence, if there were a change created by the control condition, then one would expect that it would reduce rather than increase the observed differences.
|
The pediatricians worked on the general and subspecialty inpatient wards and outpatient clinics and also provided telephone coverage at night and on weekends. Consent to participate was implied by the pediatricians returning of a 6-question baseline demographic survey that included level of training, amount of computer use, activities for which the computer is used, and attitudes of information source quality and accessibility assessed by Likert scales. The study was approved by the Institutional Review Board.
During year 1 (control group, November 1999May 2000), enrolled pediatricians were surveyed using a modification of the critical incident technique.25 This was a 9-question, semistructured telephone survey that we developed and pilot-tested. The main question was, "Think back to last night (or clinic today). What question(s) did you have last night (or today) for which you needed additional information to answer?" Other questions asked included whether an answer to the question was pursued, why it was or was not pursued, was an answer found, what information sources were used, and the amount of time spent answering the question. They were not asked when in the process of patient care the answer was sought. A pediatricians question was broadly defined as any professional inquiry that the pediatrician wished to verbalize. The telephone surveys took 1 to 2 minutes to complete, which is similar to other studies (MH Ebell, JW Ely, personal communication, February 5, 2003). Trained research assistants telephoned 2 to 6 pediatricians each weekday using a predetermined calling list. On the basis of call schedules, the calling list was created monthly by 1 researcher (D.M.D.). Pediatricians were contacted after being on call or at spaced intervals throughout the month. A pediatrician could be contacted up to 3 times per month. The pediatricians were unaware when they would be contacted.
During year 2 (intervention group, December 2000May 2001), enrolled pediatricians received a 10-minute personalized training session on how to use a pediatric digital library (www.generalpediatrics.com) to answer professional questions that might arise during a workday. GeneralPediatrics.com uses a problem-based organization to link to authoritative professional and layperson information on the Internet that is not otherwise or easily obtainable. In the training sessions, the pediatricians were given a general overview and then guided though the contents of the digital library using sample questions from specific examples gathered during the baseline evaluation (eg, "Is this anemia caused by iron deficiency?" "Which tests do I order to find out?") They were also shown site-wide navigation such as accessing searching functions or navigating between similar information on the professionals and patients pages. The pediatricians could ask questions at anytime. All information that was included in a 1-page handout that summarized the training session was included in the training, plus any other additional information requested. A tour of the pediatric digital library has been adapted from these training sessions (www.generalpediatrics.com/Tour.html). The pediatricians were again surveyed using the same telephone survey instrument. The intervention group also had MDConsult (www.mdconsult.com), a general medical digital library, freely accessible to them through the institution. It was not available to the control group. MDConsult offers online use of several paper-based medical textbooks, including Nelsons Textbook of Pediatrics26 and the Report of the Committee on Infectious Diseases.27 MDConsults availability was publicized to the intervention group through institutional electronic and paper mailings. Personal training sessions and online help were also offered to MDConsult users by the institution.
The demographic and telephone survey forms were then coded using a predetermined coding schema into a Microsoft Excel database. Physicians questions were entered verbatim. Statistical analysis was performed using SAS statistical software (SAS Institute Inc, Cary, NC) and included frequencies,
2 and 2-tailed t tests, and Wilcoxon rank sum tests with
set at 0.05.
| RESULTS |
|---|
|
|
|---|
A total of 58 pediatricians were eligible, and 52 participated (89.6%; 41 residents and 11 faculty). The control group participation rate was 89.1% (number eligible to participate: 46), and the intervention group rate was 70.4% (number eligible to participate: 44). Eligible pediatrician pools overlapped with 20 pediatricians in both groups. The 2 groups did not differ by level of training, self-reported computer use, attitudes regarding information source quality and accessibility, or work locations (P = .7469).
A total of 808 telephone contacts were made; 607 pediatrician questions were gathered from those contacts. There were 320 questions in the control group (52.7%) and 287 questions in the intervention group (47.3%). The number of questions per telephone contact ranged from 0 to 5. The range of total questions reported by a pediatrician was 0 to 33. The questions came from inpatient wards (58.8%), outpatient clinics (28.7%), telephone calls (9.1%), and other venues (3.4%).
The pediatricians pursued answers to 559 questions (92.1%). The control group pursued answers to 94.7% questions, and the intervention group pursued answers to 89.2% questions (P = .0124). The pediatricians pursued answers for patient care (81.4%), their own learning (9.3%), curiosity (4.1%), and to answer someone elses question (3.2%). They did not pursue answers because of inadequate time (75%) and resource inconvenience (8.3%).
The pediatricians were successful in finding an answer 94.4% of the time. The control group was successful 95.7% of the time, and the intervention group was successful 92.7% of the time (P = .13). The pediatricians were asked whether finding an answer affected care of the patient whom they were seeing. Pediatricians believed that finding an answer affected that patients care 71.8% of the time, whereas 25.9% believed that it did not affect care; there was no significant differences between the groups (P = .89).
Overall, the pediatricians spent an average of 14.4 minutes per question pursuing an answer (standard deviation [SD]: 32.3). The control group spent an average of 19.6 minutes (SD: 41.6), and the intervention group spent an average of 8.3 minutes (SD: 12.8). With the use of the Wilcoxon rank sum test, the difference between the groups was statistically significant (P < .001).
Pediatricians averaged 1.2 resources used per question, or a total of 738 resources for all questions. The information resources are shown in Table 1. The number of distinct resources was not calculated as a resource could be 1 or more people and names were not asked. Informal consultations with faculty, handbooks, textbooks, and computer resources were the most common resources used by both groups. Statistical testing of differences between the control and intervention groups was conducted for the 3 major categories (people, paper, and computers). Computers were used 14.5% of the time by the control group and 21.1% of the time by the intervention group, demonstrating a statistically significant increase in computer use (P = .0446). In addition, the intervention group used paper-based resources less often (P = .001). Overall, pediatricians were as successful using a computer to find answers as using other information resources (P = .6782).
|
The mean number of minutes spent per question using each information resource is shown in Table 2. Overall, formal consultations, Medline searches, and textbooks took the most time, whereas using a handbook or MDConsult took the least time. The average time required to answer questions using both paper and computer resources was less in the intervention group (P = .04 and P = .01, respectively).
|
| DISCUSSION |
|---|
|
|
|---|
Pediatricians information needs and information-seeking behaviors are not well characterized. After a Medline search (1966 to present), this is the first study that begins to describe objectively pediatricians information-seeking behaviors. The goal of this study was to characterize quantitatively the information-seeking behaviors of general pediatricians and to compare the use of computers, including a pediatric digital library and a general medical library, to seek answers to questions before and after an educational intervention designed to encourage computer use.
Our study had a higher rate of physicians reporting pursuing and answering questions when compared with internists, family practitioners, and general practitioners than in previous studies; pediatricians pursued 92.1% of their questions and found answers to 94.4% of them. Previous studies have shown that rates of questions elicited, pursued, and answered vary by study participants, the definition of a question, and the investigational methods used.4,9,28 Practitioner motivation for pursuing or not pursuing questioning has also been explored.6 Direct observation and self-reporting studies have found that between 36% and 82% of clinical questions are pursued and 54% to 88% are answered.2,3,5,8,10 Ebell and White (unpublished data) found that compared with directly observed questions, the number of self-reported physician questions was lower, but the questions had similar success rates and content composition. The present study included resident physicians who have less clinical experience and would need to look up answers more often than practicing pediatricians. It seems likely that our sample, containing a range of pediatrician experience levels, may have displayed heterogeneous information needs. For example, faculty in this study reported having fewer total questions on average than the residents (6.3 vs 9.2; data not shown). Unfortunately, the subgroup sample size precluded testing a hypothesis related to information seeking and level of experience. More than half of the questions also came from inpatient wards, where the patient acuity and immediacy requiring answers may be higher; these reasons may account for the higher rate of pursued and answered questions.
Pediatricians believed that finding an answer affected that patients care 71.8% of the time. Those questions not considered important to a patients care may have been more general (eg, What are the components of Larsons syndrome? What is the surgical technique for X?) and therefore were sought for personal learning rather than in reference to a specific patient.
The pediatricians used a wide variety of resources to answer their questions. The pediatricians report using paper resources the most frequently in both groups. After the intervention, there is a diminution of paper resource usage and a significant increase in computer usage (from 14.5% to 21.1%). Ely et al2 reported for family practitioners that paper resources are most commonly used for answering questions (57%), and people and computers account for 36% and 2%, respectively. This is similar to other studies of internists, family practitioners, and general practitioners in which computer resources comprise only a small percentage of the resources used.13,57,9,10,24 Reasons for the increased computer use are probably multifactorial, including that resident pediatricians are younger than other populations studied and therefore are more facile with using computers; that computer and Internet resources are more ubiquitous than in the past; and that during the 2 years of the present study, the pediatricians gained experience and success in using computers and digital libraries to answer questions and therefore used them again subsequently. The pediatricians may also have gotten busier during the study period and therefore had less time to pursue answers to questions regardless of the resource used. This may explain why after the intervention the physicians pursued questions less often and paper-based efficiency also increased.
Overall, digital library use (GeneralPediatrics.com and MDConsult) went from 0.8% to 11% after the intervention, with both digital libraries having statistically significant increases. We believe that the availability of MDConsult may have decreased the observed use of GeneralPediatrics.com because it contained familiar information resources and was easier to access. MDConsult being a general medical digital library offers several well-known paper-based pediatric textbooks. Using MDConsult may indicate a trend in which pediatricians are using familiar resources in a new way via online access. GeneralPediatrics.com provides access to information that is not available or not easily obtainable in paper, and the resource offerings may be less familiar to the pediatricians despite the training. Finally, accessing MDConsult was much easier for the pediatricians as it was integrated into their electronic medical record system, which many of the pediatricians remained logged into throughout their clinical day. Accessing GeneralPediatrics.com required an additional log-in beyond logging into the electronic medical record system, because of an institutional policy of monitoring and attempting to limit individual usage of the Internet. We believe that the training that the pediatricians received showcased how a pediatrician could use a computer to answer clinically relevant questions in a time-efficient manner regardless of the resource used.
There are a number of barriers to answering clinical questions. A recent study cited 59 different obstacles, with inadequate time to search for information being one of the most important.22 Other studies have shown similar results.1,19,22 One study reported that 12 minutes was the average time spent seeking information to a single question.5 Another study that found MEDLINE searches took 27 minutes and textbooks took 5 minutes.29 Our study also found that inadequate time was the major reason that questions were not pursued. The intervention group spent less time pursuing questions, with computers having the greatest time decrease (28.8 minutes vs 6.6 minutes). Success at finding answers was the same as for other information resources, but computer resources including digital libraries were more time efficient.
Computers are not a panacea. However, they can systematically and quickly search through large volumes of information with searches easily refined and resubmitted. For pediatricians to practice optimally, they need authoritative information resources to be as convenient as coatpocket handbooks. The development of digital libraries designed for pediatricians information needs, coupled with continuous Internet access computer technologies, should allow pediatricians to have information resources integrated into their workflow at the point of care. The ultimate beneficiaries are the children and families who look to their pediatricians to provide quality health care.
Limitations of this study include the selection of pediatricians from a single academic center and the reliance on reported rather than observed behavior. Pediatricians in a community setting may display different information-seeking behaviors, and these results may not be generalizable to these settings. An ongoing study that includes community pediatricians may allow direct comparisons with the present results. (John W. Ely, personal communication, February 5, 2003). Rates of questions elicited, pursued, and answered vary by the definition of a question and the investigational methods used.9 The Hawthorne effect (a distortion of research results caused by the response of subjects to the special attention that they receive from researchers30) by participating in a research study and receiving any intervention may have caused the pediatricians either to discount questions as being unimportant or to add value to other questions. We found that the time required to answer a question using both computer and paper resources was less in the intervention group. If the intervention had an impact on efficiency, we expected it to be displayed in the computer- but not the paper-based resources, so it is uncertain why paper-based efficiency went up in the intervention group.
| CONCLUSIONS |
|---|
|
|
|---|
This study showed higher rates of physician questions pursued and answered and higher rates of computer use at baseline and after intervention compared with previous studies. Pediatricians who seek answers at the point of care therefore should begin to shift their information-seeking behaviors toward computer resources, as they are as effective but more time-efficient. As point-of-care information access is improved through deployment of wireless networks and computers and an increased number of computer information resources become available, this behavior change should become easier and more widespread for pediatricians in all venues. It is hoped that the ultimate result will be improved patient care.
| ACKNOWLEDGMENTS |
|---|
This study was supported by Robert Wood Johnson Generalist Faculty Scholars Grant (036813).
We thank Peggy Kingsley and Jill Johnson-West for help with data acquisition.
| FOOTNOTES |
|---|
Reprint requests to (D.M.D.) Department of Pediatrics, University of Iowa Hospitals and Clinics, 200 Hawkins Dr, Iowa City, IA 52242-1009. E-mail: donna-dalessandro{at}uiowa.edu
| REFERENCES |
|---|
|
|
|---|
- Covell DG, Uman GC, Manning PR. Information needs in office practice: are they being met? Ann Intern Med.1985; 103 :596 599
- Ely JW, Osheroff JA, Ebell MH, et al. Analysis of questions asked by family doctors regarding patient care.
BMJ.1999; 319
:358
361
[Abstract/Free Full Text] - Gorman P. Information needs in primary care: a survey of rural and nonrural primary care physicians. Medinfo.2001 :338 342
- Gorman PN. Information needs of physicians. J Am Soc Inform Sci.1995; 46 :729 738[CrossRef]
- Barrie AR, Ward AM. Questioning behavior in general practice: a pragmatic study.
BMJ1997; 315
:1512
1515
[Abstract/Free Full Text] - Gorman PN, Helfand M. Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered.
Med Decis Making.1995; 15
:113
119
[Abstract/Free Full Text] - Woolf SH, Benson DA. The medical information needs of internists and pediatricians at an academic medical center. Bull Med Libr Assoc.1989; 77 :372 380[Web of Science][Medline]
- Ely JW, Burch RJ, Vinson DC. The information needs of family physicians: case-specific clinical questions. J Fam Pract.1992; 35 :265 269[Web of Science][Medline]
- Fozi K, Teng CL, Krishnan R, et al. A study of clinical questions in primary care. Med J Malaysia.2000; 55 :486 492[Medline]
- Arroll B, Pandit S, Kerins D, Tracey J, Kerse N. Use of information sources among New Zealand family physicians with high access to computers. J Fam Pract.2002; 51 :706
- Cimino JJ, Aguirre A, Johnson SB, Peng P. Generic queries for meeting clinical information needs. Bull Med Libr Assoc.1993; 81 :195 206[Web of Science][Medline]
- DAlessandro DM, Qian F, DAlessandro MP, et al. Performing continuous quality improvement for a digital health sciences library through an electronic mail analysis. Bull Med Libr Assoc.1998; 86 :594 601[Web of Science][Medline]
- Strasberg HR, Hubbs PR, Rindfleisch TC, Melsom KL. Analysis of information needs of users of the Stanford health information network for education. Proc AMIA Symp.1999 :965 969
- Ely JW, Osheroff JA, Gorman PN, et al. A taxonomy of generic clinical questions: classification study.
BMJ.2000; 321
:429
432
[Abstract/Free Full Text] - Krowchuk DP, Bradham DD, Fleischer AB. Dermatologic services provided to children and adolescents by primary care and other physicians in the United States. Pediatr Dermatol.1994; 3 :199 203
- Benjamin JT. Pediatric residents telephone triage experience: relevant to general pediatric practice?
Arch Pediatr Adolesc Med.1997; 151
:1254
1257
[Abstract/Free Full Text] - Forrest CB, Glade GB, Baker AE, Bocian AB, Kang M, Starfield B. The pediatric primary-specialty care interface: how pediatricians refer children and adolescents to specialty care.
Arch Pediatr Adolesc Med.1999; 153
:705
714
[Abstract/Free Full Text] - DAlessandro DM, Kingsley P. Creating a pediatric digital library for pediatric health care providers and families: using literature and data to define common pediatric problems.
J Am Med Inform Assoc.2002; 9
:161
170
[Abstract/Free Full Text] - McGowan JJ, Richwine M. Electronic information access in support of clinical decision making: a comparative study of the impact on rural health care outcomes. Proc AMIA Symp.2000 :565 569
- Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome.
Ann Intern Med.1994; 120
:135
142
[Abstract/Free Full Text] - Klein MS, Vantoll Ross F, Adams DL, Gilbert CM. Effect of online literature searching on length of stay and patient care costs. Acad Med.1994; 69 :489 495[Web of Science][Medline]
- Ely JW, Osheroff JA, Ebell MH, et al. Obstacles to answering doctors questions about patient care with evidence: qualitative study.
BMJ2002; 324
:710
[Abstract/Free Full Text] - DAlessandro DM, DAlessandro MP, Galvin JR, Kash JB, Wakefield DS, Erkonen WE. Barriers to rural physician use of a digital health sciences library. Bull Med Libr Assoc.1998; 86 :583 593[Web of Science][Medline]
- Ely JW, Levy BT, Hartz A. What clinical information resources are available in family physicians offices? J Fam Pract.1999; 48 :135 139[Web of Science][Medline]
- Flanagan JC. The Critical Incident Technique. Psych Bull.1954; 51 :327 358[CrossRef][Web of Science][Medline]
- Behrman RE, Kleigman R, Jenson HB, eds. Nelsons Textbook of Pediatrics. 16th ed. Philadelphia, PA: WB Saunders Co; 200
- Smith R. What clinical information do doctors need?
BMJ.1996; 313
:1062
1068
[Free Full Text] - Report of the Committee on Infectious Diseases, Red Book. 25th ed. Elk Grove Village, IL: American Academy of Pediatrics; 2000
- Chambliss ML, Conley J. Answering clinical questions. J Fam Pract.1996; 43 :140 144[Web of Science][Medline]
- Principia Cybernetics Web. Hawthorne Effect. Available at: pespmc1.vub.ac.be/ASC/HAWTHO EFFEC.html. Accessed June 24, 2002
PEDIATRICS (ISSN 1098-4275). ©2004 by the American Academy of Pediatrics
This article has been cited by other articles:
![]() |
T W Prendiville, J Saunders, and J Fitzsimons The information-seeking behaviour of paediatricians accessing web-based resources Arch. Dis. Child., August 1, 2009; 94(8): 633 - 635. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. A. Sneiderman, D. Demner-Fushman, M. Fiszman, N. C. Ide, and T. C. Rindflesch Knowledge-based Methods to Help Clinicians Find Answers in MEDLINE J. Am. Med. Inform. Assoc., November 1, 2007; 14(6): 772 - 780. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. S. Ketchell, L. St. Anna, D. Kauff, B. Gaster, and D. Timberlake PrimeAnswers: A Practical Interface for Answering Primary Care Questions J. Am. Med. Inform. Assoc., September 1, 2005; 12(5): 537 - 545. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. W. Ely, J. A. Osheroff, M. L. Chambliss, M. H. Ebell, and M. E. Rosenbaum Answering Physicians' Clinical Questions: Obstacles and Potential Solutions J. Am. Med. Inform. Assoc., March 1, 2005; 12(2): 217 - 224. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. M. Onady and M. A. Raslich Evidence-based Medicine: Searching Literature and Databases for Clinical Evidence (Search Tools) Pediatr. Rev., October 1, 2004; 25(10): 358 - 363. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||








