PEDIATRICS Vol. 105 No. 6 June 2000, p. e74

ELECTRONIC ARTICLE:
A Proposed Solution for Addressing the Challenge of Patient Cries for Help Through an Analysis of Unsolicited Electronic Mail

Donna M. D'Alessandro, MD^*, Michael P. D'Alessandro, MD, and Stephana I. Colbert, JD^§

From the Departments of ^* Pediatrics and  Radiology, and the ^§ Office of the Vice President for Research, University of Iowa, Iowa City, Iowa.

	ABSTRACT

Top Abstract Methods Results Discussion Conclusion References

Background.  Unsolicited electronic mail (e-mail) is e-mail sent to a physician from a person unknown to the physician, who is seeking professional help. The purpose of this project was to analyze unsolicited e-mails sent to a digital textbook author to: 1) characterize the e-mails, 2) determine what resources would be necessary to answer the e-mails, and 3) propose a standard approach to reply to e-mails in a helpful yet medicolegally-responsive manner.

Materials and Methods.  All e-mails (315) sent to a digital textbook author from October 1995 through October 1998 were abstracted. Variables included: date and location, sender type, patient age, subject, medical content, and resources necessary to answer the question. Data frequencies were obtained.

Results.  The most common location was the .com domain (47.6%). The most common senders were laypersons (66%). Overall, 44.4% of the e-mails concerned children. Detailed, patient-specific information was sent in 63.2% of the e-mails. The most common subjects were overviews of a disease or problem (32.4%), differential diagnosis (16.8%), and therapy/treatment questions (15.9%). The medical content covered a broad range of specialties. Specialists were overwhelmingly the resource necessary to answer the e-mails (74.9%).

Conclusions.  Pediatricians with educational information on the Internet can expect an increase in the number of unsolicited e-mails as Internet usage expands. Laypersons regard even short passages to mean the author is an expert in that particular area. Pediatricians need to consider the ethical and medicolegal implications of responding to unsolicited e-mails. A nonpersonalized, standard e-mail reply is proposed that directs the sender to quality information resources that may be of further assistance. unsolicited electronic mail, e-mail, medical informatics, legal issues, ethical issues, digital libraries.

Electronic mail (e-mail) is a revolutionary new way for pediatricians to communicate with their patients about their care. E-mail offers numerous advantages, including its asynchronous nature, which allows the convenience of replying when one chooses; easy clarification of meaning or instructions; and retention of the interaction, especially information conveyed, which would need to be referenced in the future. Disadvantages of e-mail include: again, its asynchronous nature leading to a variable delay between message receipt and reply, the inability to document that the message has been received and reviewed, and concerns about privacy.¹

Recently, the American Medical Informatics Association Internet Working Group put forth guidelines to help physicians with the clinical use of e-mail with their own patients that are now being widely recommended and adopted.¹ However, these guidelines did not comment on the standards that physicians should follow when unknown patients approach them for professional help via e-mail on the Internet. These unsolicited e-mails raise numerous ethical, compensatory, legal, confidentiality, and information quality issues when they are received by physicians.

Studies of solicited and unsolicited e-mails are few. A study of unsolicited e-mails to a digital health sciences library found that most senders were patients/family members, who were seeking information on general overviews of a disease or treatment information. One third of these e-mails were pediatric-related.² Another study of solicited pediatric e-mail consultations from unknown patients found that most information requests come from patients and family members who sent detailed patient-specific information³ that was also consistent with a study of unsolicited e-mails sent to the authors of a tutorial on the temperomandibular joint.⁴ Another recent report studied how physicians and Web masters (persons who run Internet sites) handled unsolicited e-mails and found a wide variation in practice.⁵ Clearly, there now exists a need for guidelines to help pediatricians manage unsolicited e-mails.

Therefore, the purpose of this project was to analyze unsolicited e-mails sent to a pediatric radiology digital textbook author to: 1) characterize the e-mails, 2) determine what resources would be necessary to answer the e-mails, and 3) propose a standard approach to reply to the e-mails in a helpful yet medicolegally-responsive manner.

	METHODS

Top Abstract Methods Results Discussion Conclusion References

The e-mails were sent to the personal e-mail account of 1 of the investigators (M.P.D.) from October 12, 1995 to October 12, 1998. He is the author of several pediatric radiology textbooks that exist only in digital form on the Internet on the University of Iowa's Virtual Hospital digital library.⁶ They cover a significant portion of the field of pediatric radiology.⁷ These digital textbooks initially included a hypertext link to the author's e-mail account at the bottom of every page. On March 6, 1997, the style of the Virtual Hospital digital library was revised, and the hypertext links to the author's e-mail account in the digital textbooks were eliminated. All e-mail messages received during the study were archived and then printed. Data from the e-mails were abstracted onto a separate abstraction form to aid data entry. A coding categorization scheme was modified from a previous study of e-mails sent to a digital health sciences library² developed from a review of articles in the literature.^8-12 A detailed coding manual was prepared and used during the data abstraction process. The abstraction forms were pilot-tested and modified. Abstraction forms were coded by 1 investigator (D.M.D.). All data on 10% of the abstraction forms were reviewed by another investigator (M.P.D.) and showed accurate coding. The coding of the resources necessary to answer the question posed in the e-mail on 100% of the abstraction forms was also reviewed by the same investigator (M.P.D.). Variables included the date of the e-mail message; location of the sender; whether the e-mail was codable; e-mail sender (eg, parent/family member, friend, physician, etc); patient age; whether detailed, patient-specific information was sent; e-mail subject (eg, overviews of disease, pathophysiology, therapy, treatment, etc); medical content (eg, radiology, cardiology, pediatrics, etc); and resources necessary to answer the question posed in the e-mail. The necessary resources were categorized into 1 of 4 categories (general/medical librarian using general and medical information from standard references and search techniques; allied health professional; generalist physician; and specialist physician or allied health professional such as a specialty nurse or pharmacist) in a step-wise manner. The question was asked, "Could the question posed in the e-mail be answered by _____ ?" The blank was initially filled in by a general/medical librarian. If the answer was yes, the e-mail was coded as general/medical librarian. If the answer was no, then the question was again posed and filled in with allied health professional, generalist physician, or specialist in a similar manner.

Data were coded into the most specific category available. Each individual e-mail may not include all data because specific data were inappropriate, missing, or unclear. Data of this type were coded as indeterminate.

All data were entered into a spreadsheet (Excel, Microsoft Corporation, Bellevue, WA) running on an Apple Macintosh computer (Apple Computer, Cupertino, CA). Frequencies were obtained using the same software program.

	RESULTS

Top Abstract Methods Results Discussion Conclusion References

Between October 12, 1995 to October 12, 1998, 316 e-mails were sent, of which 1 e-mail sent from Honduras was not codable because it was written in the Spanish language. A total of 315 were further analyzed. Figure 1 shows the number of e-mails received per quarter. The most common domains from which e-mails originated were .com with 47.6% and .net with 19.1% of the e-mails, respectively. The most common country outside of the United States from which e-mail originated was Canada with 3.5%. Overall, 24.1% of the e-mails were international in origin.

View larger version (90K): [in this window] [in a new window]   Fig. 1.   Number of e-mails received per calendar quarter.

The most common e-mail sender was a family member constituting 43.2% (n = 136) of the e-mails. Patients themselves sent 13% (n = 41) and friends 9.8% (n = 31) of the e-mails. Health care providers accounted for 12% (n = 38) of the e-mails. Other e-mail senders (n = 18) were students, lawyers, persons in the computer field, writers/reporters, a researcher, a medical librarian, and a legislator. The e-mail sender could not be determined or was not applicable for 16.2% of the e-mails.

Table 1 shows the distribution of the ages of the patients in the e-mails. Overall, 44.4% of the e-mails concerned children and adolescents and 20% concerned adults.

Detailed, patient-specific information was sent for 63.2% of the e-mails, with 21% not sending detailed information and another 15.8% being not applicable.

The subjects of the e-mails are shown in Table 2. Requests for general overviews of a disease or problem occurred 32.4% of the time with differential diagnosis questions occurring 16.8% of the time and therapy/treatment questions occurring 15.9% of the time. Other topics included technical comments or questions, praise for the textbooks, and seeking an expert witness.

Medical content of the e-mails is shown in Table 3. A broad range of pediatric specialties was represented with the most common content areas being neurology/neurosurgery (11.8%), allergy/pulmonary (10.5%), and orthopedics (10.2%). Only 14 e-mails concerned radiology (4.4%). The most common diseases comprising 4 or more e-mails were Kartagener's syndrome, bronchopulmonary dysplasia, hyaline membrane disease, pectus excavatum/carinatum, Crohn's disease, patent ductus arteriosus, craniosynostosis, cystic adenomatoid malformation, esophageal atresia, gastroschisis, hiatal hernia, Legg-Calvé-Perthes disease, lymphangiectasis, and phrenic nerve paralysis.

Specialists were overwhelmingly the resource necessary to answer the e-mails with 74.9% coded into this category. A general/medical librarian was the second most common resource (13.3%), followed by a generalist physician (7.6%), and an allied health professional (.3%).

	DISCUSSION

Top Abstract Methods Results Discussion Conclusion References

The first goal of this project was to characterize the e-mails sent to a single author of pediatric radiology digital textbooks. The number of e-mails sent grew steadily over time, until March 6, 1997, when the style of the digital library was revised and the direct e-mail hyperlink to the author was eliminated making it more difficult for a sender to contact the author by e-mail. Since that date, and since the addition of 3 additional digital textbooks in July and August 1997, the number of e-mails sent has again continued to grow steadily (see Fig 1). Most of the unsolicited e-mails were sent from the United States, with ~24% being international in origin. Most concerned children (44.4%), but 20% concerned adults with traditionally pediatric problems (eg, adults with congenital heart disease, cystic fibrosis, etc). Most of the unsolicited e-mails were sent by laypersons (family members, patients themselves, and friends [66%]) who sent detailed, patient-specific information (63.2%). The e-mails covered a broad range of pediatric specialties with a general overview of the disease or problem being the most common question asked (32.4%). Most of the diseases or problems were uncommon. These findings are consistent with previous studies,^3,13 including 2 evaluating the overall usage of a digital health sciences library¹⁴ and the e-mail the digital library receives.² In contrast, the most common medical problems in the e-mails were quite different from the most common problems seen in a general pediatrician's office¹⁵ or in a pediatric telephone triage line,^16,17 which are primary care problems.

There were a strikingly low number of e-mails (4.4%) that concerned radiology. This number is not erroneous. For example, a characteristic e-mail said:

"My niece has a condition called Kartagener Syndrome or lifeless cilia. She therefore suffers from one infection after another and exhibits cystic fibrosis like symptoms. I feel so helpless, she is 15 and has seen so many doctors and we are striking out. Her mood is dark. Can you give me any research about this or direct me to other experts."

This is not a pediatric radiology question but a pediatric pulmonary question and, therefore, would be coded into that category. If the e-mail had noted something about radiographic evaluation, etc, then it would have been coded into the pediatric radiology category. A previous study that investigated how individuals had been referred to digital textbooks in a digital library showed that 40% of individuals arrived at a textbook chapter by an Internet search engine, with the balance arriving by following links from Internet indices or other digital libraries.¹⁴ This reliance on Internet search engines often results in the individual finding information without understanding the context in which the information is being presented. The person in the case above misidentified the author as an expert in the field of Kartagener's syndrome. This is characteristic of the e-mails reviewed in this study, as laypersons regarded even short passages in the digital textbooks to mean that the author was an expert in that particular area. Many e-mails contained statements such as, "As a leading expert in X, could you please answer my question about... ?" This is in contrast to a study of e-mails sent to the authors of a single Internet site focusing on cardiac arrhythmias, who found that 96% of their e-mails concerned cardiology issues.¹³ It is consistent with a study of unsolicited e-mails to a digital health sciences library that provides a broad range of medical information.² Consequently, it seems that when presented with a broad range of information, such as that in the pediatric radiology digital textbooks or a digital health sciences library, laypersons have difficulty in determining the appropriateness and relevancy of the information and of whom they are asking questions. This dependence on the use of Internet search engines to locate information on the Internet may also account for the finding that the most common diseases referred to in the e-mails are in fact quite uncommon. The rarer the disease, the less information there is likely to be found on it on the Internet, and, therefore, the more likely the information on the disease in the author's digital textbooks was likely to rank high on an Internet search engine's hit list.

The second goal of the project was to determine what resources were necessary to answer the e-mail. The e-mails posed very sophisticated questions with many of the patients having already been evaluated in tertiary care hospitals. Unable to have their questions answered to their satisfaction locally, they decided to seek an answer globally. Therefore, the questions posed most often could not be answered by a general nurse or a general physician. Furthermore, "Frequently Asked Questions" lists as used by some pediatric web sites (Kidshealth. Available at: http://www.kidshealth.org/ and Mayo Health Oasis. Available at: http://www.mayohealth.org/) would also not be sufficient to answer these questions. The knowledge of a specialist was required to answer ~75% of these e-mails. These findings are also consistent with previous reports.^2,18

The third goal of the project was to develop an approach to replying to the e-mail in a thoughtful yet medicolegally-responsive manner. Initially, from 1995 to 1997, the pediatric radiology digital textbook author did not reply to any of the e-mails. In January 1998, the investigators developed a short reply that was a simple disclaimer statement ("I am sorry, but due to a lack of time and numerous ethical and legal complexities, I am unable to personally answer your medical question.") but did not offer any generalized help to answer the senders' questions. Over time, the investigators noticed how consistent the e-mails were. They were from laypersons with patient-specific cries for help that the investigators found difficult to ignore. Many times there was a grave or desperate tonal quality to the e-mails, similar to the illustration above and consistent with a previous study.¹⁸ This disclaimer, therefore, proved personally unsatisfactory to the investigators. With the help of the legal counsel of the University of Iowa, the investigators began developing a standardized reply to the unsolicited e-mail that would offer generalized help to answer the senders' questions. While in the course of developing this standardized reply, it was realized such a reply had to consider and incorporate the following ethical, compensatory, legal, confidentiality, and information quality issues.

Ethical Issues

E-mail is a very powerful tool to use to help pediatricians with their own patients. But how should it be used to help strangers? An informal survey of colleagues who are printed textbook authors revealed that they never answered questions sent to them by patients or health care providers who had read their textbooks. The opposite view is maintained by the American College of Physicians who state: "By history, tradition, and professional oath, physicians have a moral obligation to provide care for ill persons... . [E]ach individual physician is obliged to do his or her fair share to ensure that all ill persons receive appropriate treatment." They also say, "[Physicians] should support public health endeavors that provide the general public with accurate information about health care and comment on medical subjects in their area of expertise to keep the public properly informed."¹⁹ Physicians clearly have a duty to serve those in need in the community. Currently, the American Academy of Pediatrics has no similar policy statement. Pediatricians clearly have a duty to serve those in need in the community. A similar sense of duty is held by health science librarians.²⁰ However, the Internet community is a world-wide community and its scale complicates the issue as the Internet gives patients easy access to physicians whom they would never have had access to before. Clearly, a case can be made that pediatricians have a duty to respond in some way to unsolicited e-mail. However, with the number of individuals on the Internet doubling every year,²¹ even the most idealistic and conscientious pediatrician with the best intentions will eventually be overwhelmed responding to unsolicited e-mails. Therefore, a standardized reply to unsolicited e-mail is the only practical approach.

Compensatory Issues

To answer the unsolicited e-mails in a personalized manner necessitates resource expenditures. How would an individual be compensated for such work? In a study of pediatric e-mail consultation, Borowitz and Wyatt³ found 82.5 hours of work resulted in 3 referrals to their practice. To answer the majority of the e-mails, specialists are required. They generally also require the most compensation for their expertise. Currently, there is no universally applied business model for recovering the costs of answering these e-mails. Some consider e-mail part of the overhead of the individual's practice, similar to the way written and telephone correspondence is an expected cost of medicine. Others view these e-mails as requesting the services of a second opinion or expert witness and believe they should have compensation associated with the level of service provided. Institutions may come to regard digital textbooks as a form of marketing and/or goodwill to patients and may consider underwriting the monetary cost of a panel of specialists to answer these e-mails, in the hope that such answers may generate referrals. Compensation may also come in the form of academic credit for serving on a panel for those specialists who work in an academic environment. Professional societies providing an e-mail consult service would promote their organization through provision of quality information to laypersons seeking help. Any financial arrangement between a pediatrician and an e-mail sender should be clearly established in advance and should reflect the services provided.¹⁹

Legal Issues

The legal issues of unsolicited e-mails are numerous and well-outlined in several articles.^22-25A pediatrician does not legally have to reply to an unsolicited e-mail either with a standardized or personalized reply, but if a reply is sent then the pediatrician may be entering into a contract depending on the specific question and the reply sent. Unsolicited e-mails are more like phone calls and letters that are unsolicited by the pediatrician. Pediatricians do not have a legal responsibility to answer them and if they do answer, may or may not be entering into a contract with the writer or caller. This is true even if only some of the e-mails, phone calls, or letters are answered and others go unanswered. If a pediatrician chooses to reply to an e-mail, there should be a disclaimer in the e-mail³ and all correspondence should be saved as that of any other medical record correspondence. All Internet information authored by a pediatrician should also have a disclaimer because there is the potential for liability.²² Unfortunately, the ultimate solutions to limiting liability are to limit access to the author's work (such as only to professionals) or to remove it from the Internet, neither of which is a satisfactory solution. The area of Internet medical practice regulation is in its infancy. Definitions of medical practice vary from state to state. Some states may even consider responding to unsolicited e-mail to be practicing medicine across state lines and a form of telemedicine; some states have regulations regarding this, whereas others do not.

Confidentiality Issues

Senders are submitting detailed e-mails concerning their problems: in essence, medical records, which are not encrypted or secure. Any reply that is sent is also not encrypted. Therefore, pediatricians need to consider this when formulating a reply. The original message and reply should be printed and treated in a manner similar to any other medical record. Similarly, the e-mail should not be released to other individuals without the consent of the sender, such as posting the message to a newsgroup.^1,23,25

Information Quality Issues

Pediatricians who provide health care information on the Internet should be keenly aware that laypersons have a difficult time discerning the quality of the information. As noted, laypersons often think that someone who has written even a short piece of information is an expert in that area, so it is easy to see how difficult it is for them to gauge the quality of any Internet site or its information. One way to potentially decrease problems is to subscribe to information quality standards that are being developed.^26-29 One example is the code of conduct of Health on the Net Foundation, which expects that an Internet site will minimally include: name of person or organization creating the information, date of creation, date of last modification, support for the web site, clear references to source data, and web site contact information.³⁰ Pediatricians can also direct laypersons to information on how to judge the quality of information on the Internet.³¹

The end result of this process in April 1998 was a standard reply (Fig 2), which is a general statement that offers pointers to other quality information resources for the sender. These resources include emphasis on the child's primary care provider, the staff of local or regional children's hospitals, and regional medical libraries. They also include information for rare disorders. They go on to emphasize how to learn to evaluate the quality of medical information on the Internet and then point the sender to comprehensive pediatric Internet indices and Internet medical search engines as well as public access Medline. As with all writings and other material once created and placed in tangible form, this standardized reply is copyrighted. However, there is no restriction on use and/or modification by readers to suit their own circumstances, and permission to do so is readily granted. The investigators cannot scientifically state how helpful the standardized reply is to the senders except for the anecdotal receipt of some additional e-mail from senders thanking the investigators for the standardized reply. This standardized reply has subsequently been adopted for use as the standardized reply to all unsolicited e-mail sent to the Virtual Hospital digital library.

View larger version (30K): [in this window] [in a new window]   Fig. 2.   Standard e-mail reply.

Limitations of this study include:

1. A sample of e-mails sent to a single author of pediatric radiology digital textbooks. Another individual author may have different content analysis results.
2. All study questions were not answered by each individual sender's e-mail because information was inappropriate, missing, or unclear. Each e-mail did not need a preset number of variables to be included in the study.
3. Medical content was also coded into 1 specialty only; therefore, an individual specialty may be overrepresented or underrepresented.
4. Content analysis provides much information but lends itself to the potential of inaccurate coding. Coding was performed by 1 person, and accuracy was checked by a second person.³²

	CONCLUSION

Top Abstract Methods Results Discussion Conclusion References

With the continued growth of the Internet, any pediatrician publishing educational information on the Internet will be confronted with an increasing number of cries for help from laypersons in the form of unsolicited e-mail. Ethically, pediatricians have a responsibility to answer these e-mails but cannot be expected to be compensated for doing so and have finite resources to devote to the process. Therefore, we have taken a first step in proposing a standardized, nonpersonalized reply to unsolicited e-mail. Such a standardized reply should have a disclaimer statement that has been reviewed by legal counsel. The e-mails should be treated as medical records and confidentiality and documentation of the interaction should occur. Finally, pediatricians can use such a standardized reply to begin to educate laypersons about medical information quality on the Internet, by directing laypersons to resources that address this issue.

	ACKNOWLEDGMENT

This work was supported in part by Grant N01-LM-4-3511 from the National Library of Medicine.

	FOOTNOTES

Dr Colbert is currently at University of Tulsa College of Law, Tulsa, Oklahoma.

Information in this manuscript was presented as an abstract at the Annual Meeting of the Pediatric Academic Societies; May 1-4, 1999; San Francisco, CA.

Received for publication Aug 23, 1999; accepted Jan 5, 2000.

Reprint requests to (D.M.D.) Department of Pediatrics, Children's Hospital of Iowa, 200 Hawkins Dr, Iowa City, IA 52242-1009. E-mail: donna-dalessandro{at}uiowa.edu

	ABBREVIATIONS

e-mail, electronic mail.

	REFERENCES

Top Abstract Methods Results Discussion Conclusion References

1. Kane B, Sands DZ Guidelines for the clinical use of electronic mail with patients. J Am Med Inform Assoc 1998; 5:104-111 [Abstract/Free Full Text]
2. D'Alessandro DM, Qian F, D'Alessandro MP, Performing continuous quality improvement for a digital health sciences library through an electronic mail analysis. Bull Med Libr Assoc 1998; 86:594-601 [Medline]
3. Borowitz SM, Wyatt JC The origin, content, and workload of e-mail consultations. JAMA 1998; 280:1321-1324 [Abstract/Free Full Text]
4. Gillepsy T III, Richardson ML Publishing radiology educational material on the Internet: analysis of e-mail responses. J Digital Imaging 1997; 13:193
5. Eysenbach G, Diepgen TL Responses to unsolicited patient e-mail requests for medical advice on the world wide web. JAMA 1998; 280:1333-1335 [Abstract/Free Full Text]
6. D'Alessandro MP, Galvin JR, Erkonen WE, The virtual hospital: experiences in creating and sustaining a digital library. Bull Med Libr Assoc 1998; 86:553-563 [Medline]
7. D'Alessandro MP. Web documents:
   Paediapaediaan imaging encyclopedia of pediatric disease. Available at: http://www.vh.org/providers/teachingfiles/pap/paphome.html. Accessed October 1995
   Back pain in children. Available at: http://www.vh.org/providers/textbooks/backpaininchildren/backpainchildren.html. Accessed March 1996
   MetaTextbook of pediatric radiology. Available at: http://www.vh.org/providers/teachingfiles/metatextbookpedrad/metatbpedrad.html. Accessed July 1997
   Correlapaediaa correlative encyclopedia of pediatric imaging, surgery, and pathology. Available at: http://www.vh.org/providers/teachingfiles/cap/caphome.html. Accessed August 1997
   Thoracopaediaan imaging encyclopedia of pediatric thoracic disease. Available at: http://www.vh.org/providers/teachingfiles/tap/thoracopedia.html. Accessed August 1997
8. Dee C, Blazek R Information needs of the rural physician: a descriptive study. Bull Med Libr Assoc 1993; 81:259-264 [Medline]
9. Osheroff JA, Forsythe DE, Buchanan BG, Physicians and other health care practitioners information needs: analysis of questions posed during clinical teaching. Ann Intern Med 1991; 114:576-581
10. Northrup DE, Moore-West M, Skipper B, Teaf SR Characteristics of clinical information-searching: investigation using critical incident technique. J Med Educ 1983; 58:873-881 [Medline]
11. Woolf SH, Benson DA The medical information needs of internists and pediatricians at an academic medical center. Bull Med Libr Assoc 1989; 77:372-380 [Medline]
12. Osheroff JA, Bankowitz RA Physicians' use of computer software in answering clinical questions. Bull Med Libr Assoc 1993; 81:11-19 [Medline]
13. Widman LE, Tong DA Requests for medical advice from patients and families to health care providers who publish on the world wide web. Arch Intern Med 1997; 157:209-212 [Abstract/Free Full Text]
14. D'Alessandro MP, D'Alessandro DM, Galvin JR, Erkonen WE Evaluating Overall usage of a digital health sciences library. Bull Med Libr Assoc 1998; 86:602-609 [Medline]
15. American Academy of Family Physician, US Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics: 1992 data (unpublished), Table 033. Available at: http://www.aafp.org. Accessed Jun 25, 1998
16. Villarreal SF, Berman S, Groothiuis JR, Strange V, Schmitt BD Telephone encounters in a university pediatric group practice. Clin Pediatr 1983; 23:456-458
17. Poole SR, Schmitt BD, Carruth T, Peterson-Smith A, Slusarski M After-hours telephone coverage: the application of an area-wide telephone triage and advice system for pediatric practices. Pediatrics 1993; 92:670-679 [Abstract/Free Full Text]
18. Eysenbach G, Diepgen TL Patients looking for information on the Internet and seeking teleadvice: motivation, expectations, and misconceptions as expressed in e-mails sent to physicians. Arch Dermatol 1999; 135:151-156 [Abstract/Free Full Text]
19. American College of Physicians Ethics manual. Ann Intern Med 1998; 128:576-594 [Abstract/Free Full Text]
20. Medical Library Association. Code of ethics for health sciences librarianship. Available at: http://www.mlanet.org/about/ethics.html. Accessed May 13, 1999
21. Bell G, Gemmell J On-ramp prospects for the information superhighway dream. Commun ACM 1996; 39:55-61
22. Samuelson P Legally speaking: liability for defective electronic information. Commun ACM 1993; 36:21-26
23. Savkar S, Waters RJ. Telemedicine implications for patient confidentiality and privacy. Available at: http://www.arentfox.com/newslett/telemed/tele1c.html. Accessed December 30, 1998
24. Young HJ, Waters RJ. Licensure barriers to the interstate use of telemedicine. Available at: http://www.arentfox.com/newslett/telemed/tele1b.html. Accessed December 30, 1998
25. Spielberg A On call and online, sociohistorical, legal, and ethical implications of e-mail for the patient-physician relationship. JAMA 1998; 280:1353-1359 [Abstract/Free Full Text]
26. Murray P Click hereand be disappointed? Evaluating web sites. Comput Nurs 1996; 14:260-261
27. Silberg W, Lundberg, GD Musacchio RA Assessing, controlling, and assuring the quality of medical information on the Internet: caveant lector et viewor. JAMA 1997; 277:1244-1245 [Abstract/Free Full Text]
28. Wyatt J Measuring quality and impact of the www . Br Med J 1997; 314:1879-1881 [Free Full Text]
29. Berry E, Parker-Jones C, Jones RG, Systematic assessment of world wide web materials for medical education. JAMIA 1998; 5:382-389 [Abstract/Free Full Text]
30. Health on the Net Foundation. Health on the Net Foundation code of conduct for medical and health web sites. Available at: http://www.hon.ch/honcode/conduct.html. Accessed March 15, 1999
31. Medical Library Association. Deciphering MedSpeak. Available at: http://www.mlanet.org/resources/medspeakemsp;intro.html. Accessed February 17, 1999
32. Weber RP. Basic Content Analysis. Beverly Hills, CA: Sage Publications; 1985