New! Video Abstracts -- brief videos summarizing key findings of new articles
Improving Usage of Pediatric Information on the Internet: The Virtual Children's Hospital
Abstract
Objective. Digital health sciences libraries (DHSLs) bring order to the chaos of the Internet by making authoritative medical information easily and conveniently available to patrons. The goal of this project was to perform a baseline usage analysis of the pediatric-related information in a general DHSL and to determine whether reorganization of the pediatric-related information into its own pediatric DHSL could increase the usage of the pediatric-related information.
Methods. From March through August 1997, a baseline analysis of a general DHSL (Virtual Hospital) was conducted using computer server log file analysis programs. The quantity of pediatric-related information in the general DHSL and its baseline usage were determined. In September 1997, the pediatric-related information was reorganized into its own pediatric DHSL (Virtual Children's Hospital), and server log file analyses were conducted of the pediatric DHSL from September 1997 to August 1998. Statistical analysis was performed by time series autoregression.
Results. During the baseline, the general DHSL and the pediatric-related information received a monthly average of 2 320 782 and 141 444 qualified hits, respectively. After the intervention, the general DHSL and the pediatric DHSL received a monthly average of 2 765 454 and 256 998 qualified hits, respectively. This is an increase of 19.2% for the general DHSL and 81.7% for the pediatric DHSL. These changes were statistically significant at theP > .0001 level. The most requested pediatric-related content in the pediatric DHSL did not change substantively from preintervention to postintervention.
Discussion. On the Internet, as in real life, children's services must have their own distinct identity and must be differentiated from adult services. Therefore, pediatric-related information will receive increased usage if it is part of a pediatric DHSL rather than part of a general DHSL. Others can use this process and the lessons learned to develop and enhance their own pediatric-related information on the Internet. Internet, pediatrics, digital health sciences libraries, digital library, medical library.
- DHSL =
- digital health sciences library •
- e-mail =
- electronic mail •
- HTML =
- hypertext markup language
Pediatric health care providers and patients need convenient access to organized, authoritative information to provide optimal quality care and to live healthier lives. Today, the Internet has been proposed as a means of providing access to such information.1 There are now a number of pediatric-related Web sites,2–5 and initial reviews of health care professionals' and laypersons' use of such resources6 ,7have been performed. Unfortunately, although the Internet provides convenient access to almost unlimited quantities of information, the information often is unorganized8 and of questionable authority.9 Studies have shown that in its current form, it is extremely difficult to use the Internet to obtain answers to questions.8
Digital libraries are organized collections of digital information that have structured approaches to the building of their collections and the use of their digital information just as regular libraries do with print information and other materials. The World Wide Web can be said to contain raw information, whereas digital libraries can be said to contain organized information or knowledge.10 Therefore, digital libraries can bring order to the chaos of the Internet, making knowledge easily and conveniently available to patrons and helping them to obtain answers to questions.
The Virtual Hospital (http://www.vh.org) is The University of Iowa's general digital health sciences library (DHSL), which has been in operation since 1992. Its goal is to lower barriers to accessing medical information by digitizing expert information and making it conveniently available to health care providers at the point of care and patients at home. Currently, it serves >10 000 patrons daily with 20% of its use being international. It has distributed >1 terabyte of information in the past 2 years alone.11 In comparison, the entire Internet is thought to contain 2 terabytes of information, and the Library of Congress is thought to contain 20 terabytes of text information.12–14
A previous study of electronic mail (e-mail) sent to the general DHSL found that 33% of the unsolicited medical e-mails from patients and health care providers were pediatric-related.15 The most common information requests in this study were about infectious diseases (childhood illnesses), gastrointestinal disorders (lower gastrointestinal problems), and pulmonary disorders (croup). Although the general DHSL contained pediatric-related information on these topics, we suspected that the usage of the pediatric-related information in the general DHSL was dramatically less than 33%. Therefore, we were concerned that the pediatric-related information needs of the patrons were not being met by the current organizational structure of the general DHSL.
The goal of this project was to perform a baseline usage analysis of the pediatric-related information in the general DHSL and to determine whether reorganization of the pediatric-related information into its own pediatric DHSL could increase the usage of the pediatric-related information and therefore better meet the pediatric-related information needs of patrons.
METHODS
The information in the general DHSL consists of hundreds of medical booklets and books, covering hundreds of topics. This information is divided between information created primarily for use by health care providers and information created primarily for use by patients. The pediatric-related health care provider information ranges from common pediatric topics such as otitis media to uncommon medical problems like congenital diaphragmatic hernia and includes a complete general medicine textbook, The University of Iowa Family Practice Handbook (Mosby, New York, NY). Similarly, the pediatric-related patient information ranges from booklets on common pediatric issues, such as childhood immunization, to specialized books on rare psychiatric disorders, such as Tourette syndrome. The booklets and books range from one to several hundred pages in length. The pediatric-related information is created by staff clinicians in The University of Iowa Health Sciences Center, professional health care societies, nonprofit health care organizations, state and government health care agencies, and professional publishers. Copyright permission is obtained from the information's authors in all cases. The information created by staff clinicians in The University of Iowa Health Sciences Center is usually peer-reviewed by departmental peer review boards. All information is marked clearly with the name, credentials, and affiliation of its author, whether it had been peer-reviewed, and its date of last modification. All information is in English. The general DHSL is updated on a daily basis with new material and revisions to existing material.
The general DHSL is implemented using the World Wide Web, and it is stored on a Sun Enterprise 450 server computer (Sun Microsystems, Inc, Palo Alto, CA) and served using the Apache World Wide Web server (Apache Group, Lincoln, NE). All information in the general DHSL is indexed using the Glimpse search engine (The University of Arizona, Tucson, AZ), making all text within it free-text searchable or searchable by individual word or phrase.
Each World Wide Web server has a server log file. Each time a hypertext markup language (HTML) or graphics file on the World Wide Web server is accessed by a patron, the information recorded into the server log file includes the name of the file being accessed, the time of day and day of week that is being accessed, and the Internet address of the workstation from which the patron is accessing the file. The identity of the patron cannot be recorded.
From March 1997 through August 1997, a baseline server log file analysis was performed on the general DHSL using the log file analysis programs Analog 1.2.3 (the University of Cambridge Statistical Laboratory, Cambridge, UK) and Wusage (Boutell.Com, Inc, Seattle, WA) to determine the overall use of the general DHSL and of the pediatric-related information within the general DHSL.
Pediatric-related information in the general DHSL was defined as information relating to patients 0 to 21 years of age. The pediatric-related information in the general DHSL was identified by the general DHSL librarian and a pediatrician, who manually browsed through the general DHSL catalog and identified all the pediatric-related books and booklets. Information that pertained to both pediatrics and adults (eg, Crohn's disease) was classified as pediatric-related, if it was created by a pediatric provider.
The size (in megabytes) of all information in the general DHSL was determined by adding together the size of all information-containing files in the general DHSL, calculated by the UNIX utility du (Sun Microsystems, Inc, Palo Alto, CA). The size (in megabytes) of all pediatric-related information in the general DHSL was determined in August 1997, February 1998, and August 1998 by adding together the size of all these pediatric-related books and booklets. This was completed again using the UNIX utility du.
The pediatric-related information in the general DHSL was reorganized into its own pediatric DHSL and launched on the World Wide Web on September 1, 1997. The goal of this initiative was to create a virtual children's hospital within a virtual hospital. The pediatric information was not removed from the general DHSL but was reorganized into its own distinct collection or pediatric DHSL. First, all the pediatric-related information, which previously was scattered throughout the general DHSL, was rebranded. Next, an integrated interface for the pediatric DHSL was designed to provide easy and convenient access to the pediatric-related information for pediatric health care providers, patients, and children. The result was the Virtual Children's Hospital (http://vch.vh.org) that has a unique homepage, address, navigational interface, and identity from the general DHSL (Fig 1). The general DHSL and the Virtual Children's Hospital were publicized to all of the major medical indices and Internet search engines concurrently.
The Virtual Children's Hospital (http://vch.vh.org) has a unique homepage, address, navigational interface, and identity from the general DHSL.
Monthly server log file analyses were conducted from September 1997 through August 1998 to determine whether there were changes in the use of pediatric-related information once the pediatric DHSL was created. The autoregression program available on the SAS statistical computer program (SAS Institute Inc, Cary, NC) was used to calculate the statistical significance of the time series data.
In August 1997, February 1998, and August 1998, the server log files for the pediatric DHSL also were analyzed to determine the 10 most requested pediatric-related files (ie, book chapters or booklets) for health care providers and for patients.
The area of World Wide Web server log file analysis is developing rapidly, and therefore, the definition of a few terms will aid understanding of the results.16
A hit is the most basic unit of World Wide Web traffic measurement. A hit is generated by every file request made to a World Wide Web server. A hit means that the World Wide Web server has transmitted a requested file to a patron. That requested file can be a HTML file, a graphics file, a video file, an audio file, etc. An HTML file can contain other elements (such as graphics files) embedded within it. When an HTML file is accessed, it is logged in the World Wide Web server log file as a sequence of hits: one for the HTML file itself and one for each embedded element within the HTML file.
A qualified hit is any requested file, with or without other embedded files, that successfully arrives and is displayed at the patron's workstation. This eliminates the counting of files that were requested by the patron but were never delivered completely because of server error or patron cancellation, redirected requests, and requests by computer programs such as World Wide Web search engines that index information on the Internet. Therefore, qualified hits are a subset of all hits.
RESULTS
Baseline Analysis
The baseline usage analysis (Table 1) from March 1997 through August 1997 showed that the general DHSL received a total of 13 924 693 qualified hits, with a monthly average of 2 320 782 qualified hits. The pediatric-related information in the general DHSL (after September 1, 1997, termed the pediatric DHSL) received a total of 848 664 qualified hits during the 6-month baseline analysis or a monthly average of 141 444 qualified hits. The pediatric-related information accounted for ∼6.1% of the total qualified hits received by the general DHSL, and this monthly average was relatively steady during the baseline period.
Growth in Usage of the Pediatric and General Digital Health Sciences Libraries
In August 1997, the general DHSL contained 1102 megabytes of information and of that amount 133 megabytes or 12.1% was pediatric-related.
Pediatric DHSL Usage
Table 1 also shows growth in usage of the pediatric-related information after implementation of the pediatric DHSL on September 1, 1997. During the intervention period of September 1997 through August 1998, the monthly qualified hits increased for the general and pediatric DHSLs. During the intervention period, the general DHSL received a total of 33 185 451 qualified hits or a monthly average of 2 765 454. This is a monthly average increase of 19.2% for the general DHSL over the baseline time period. During the same intervention period, the pediatric DHSL received a total of 3 083 977 qualified hits or a monthly average of 256 998. This is a monthly average increase of 81.7% over the baseline time period.
An autoregressive, interrupted, time-series analysis revealed a statistically significant (P > .0001) postintervention-related increase for both the absolute number of qualified hits and the ratio of the pediatric DHSL qualified hits to general DHSL qualified hits. The adjustedR2 for the intervention impact was ∼0.80 for both variables, indicating that ∼80% of the increase that was observed might be explained by the intervention.
In February 1998, the general DHSL contained 1349 megabytes of information, and the pediatric DHSL contained 122 megabytes of pediatric-related information (9.0%). In August 1998, the general DHSL contained 1352 megabytes of information and the pediatric DHSL contained 122 megabytes of pediatric-related information (9.0%). The overall decrease in the amount of content in the pediatric DHSL from August 1997 was attributable to the removal of outdated information and the consolidation of other information. New pediatric-related information also was added during the intervention time period.
Pediatric Content
In August 1997, February 1998, and August 1998, the 10 most requested booklets or book chapters for health care providers and patients in the pediatric DHSL were determined (Table 2). The most requested information for health care providers covered several areas, including respiratory (eg, croup, tracheo/laryngomalacia, asthma, stridor, and dyspnea) and gastrointestinal diseases (eg, vomiting/diarrhea/dehydration and constipation/encopresis). Neonatal information and patient simulations, the Virtual Pediatric Patients, were also popular (http://www.vh.org/Providers/Simulations/VirtualPedsPatients/PedsVPHome.html).
Ten Most Requested Book Chapters or Booklets for Providers and Patients
For patients, the most requested booklets or books were related primarily to infectious disease (eg, fifth disease, roseola, and streptococcal pharyngitis) and dermatological problems (eg, eczema and scalp fungal infections).
The most requested content in the pediatric DHSL did not change substantively from preintervention to postintervention. Of the 15 distinct booklets or book chapters that were listed for health care providers, 10 were noted during 2 or more of the months analyzed. Of the 15 distinct booklets or books chapters listed for patients, 9 were noted in 2 or more of the months analyzed.
DISCUSSION
The goal of this project was to perform a baseline usage analysis of the pediatric-related information in the general DHSL and to determine whether reorganization of the pediatric-related information into its own pediatric DHSL could increase the usage of the pediatric-related information and therefore better meet the pediatric-related information needs of patrons.
The creation of the pediatric DHSL (the Virtual Children's Hospital) proved to be very successful, leading to an overall increase in usage of pediatric-related information of 81.7%. This increase is not solely attributable to the increase in the number of overall Internet users during the intervention time period as the general DHSL usage increased only 19.2% during the same intervention period. The pediatric DHSL usage also grew despite an actual decrease in the amount of content available (from 133 megabytes to 122 megabytes during the intervention period). Publicity of the Virtual Children's Hospital could have influenced the usage, but the general DHSL also was publicized concurrently; therefore, any increased usage attributable to publicity is most likely similar for both digital libraries. We feel the majority of the growth is attributable to the successful reorganization of the pediatric-related information into its own distinct pediatric DHSL.
This study shows that appropriate organization of DHSL information leads to increased use of pediatric-related information. The general DHSL was shown to be used heavily in the baseline analysis, but only a small percentage of that use (6.1%) was pediatric-related. A recent evaluation of medically related e-mail sent to the general DHSL found that 33% of the unsolicited e-mail from patients and health care providers was pediatric-related.15 Therefore, the baseline analysis confirmed the suspicion that the organizational structure of the general DHSL was not meeting the pediatric-related information needs of its patrons (pediatric-related usage was only 6.1%). Once created, the pediatric DHSL delivered improved service to its patrons. This effect is analogous to what happens when a children's hospital within a hospital is created within a general academic medical center. On the Internet, as in real life, children's services also must have their own distinct identity and must be differentiated from adult services.
Information organization is only one aspect of making information conveniently and easily available to patrons. There also must be appropriate information for patrons to use. This study supports previous studies showing that there is an acute demand for authoritative pediatric information on the Internet.7 ,17However, this is the first study of a pediatric DHSL that indirectly begins to define what patron information needs are. A broad spectrum of information in the pediatric DHSL was accessed regularly by both providers and patients, ranging from common pediatric problems (eg, asthma, streptococcal pharyngitis, and constipation) to less common ones (eg, tracheo/laryngomalacia, and spica cast parent guide). Interestingly, this reorganization did not lead to substantive change in the books and booklets most commonly used by patrons. This study also implies that the pediatric DHSL still is not meeting completely the broad-based information needs of our patrons. Therefore, we will continue to expand the pediatric-related information on the pediatric DHSL and link to other pediatric-related information on the Internet when appropriate. Prioritization of information creation must occur because no one entity (person, organization, or DHSL) can create or even link to all the needed information. This study showed that of the pediatric information used, respiratory, gastrointestinal, infectious disease, and dermatology information often is sought. The previously conducted e-mail analysis showed that in pediatrics, information on infectious diseases (childhood illnesses), gastrointestinal disorders (lower gastrointestinal problems), and pulmonary disorders (croup) were requested frequently.15 This also is consistent with data on common pediatric problems.18 This information can be used as a prioritization guideline for developers of pediatric DHSLs or other pediatric medical information on the Internet.
Although World Wide Web server log file analysis, which was used in this study, can provide information on the overall use of resources in a DHSL, it has the limitations of not being able to provide information on the demographics of individual patrons, their motives, their original questions, and how successful they were in answering them. Other research methods are necessary to appropriately answer these important questions.
CONCLUSION
As medical information on the Internet grows, there will be a continuing need for authoritative medical information for health care providers and patients. This study shows that pediatric-related information needs currently are partially met. Through the process of identifying patrons' needs, instituting new measures to meet those needs and reevaluating, this project has improved the use of pediatric information on the Internet. This project shows that pediatric-related information will receive increased use, if it is part of a pediatric DHSL rather than part of a general DHSL. This process and the lessons learned can be used by others as they develop and enhance their own pediatric information on the Internet. In the future, the Virtual Children's Hospital will continue to expand its information to better meet the needs of pediatric health care providers and patients around the world.
ACKNOWLEDGMENTS
This work was supported in part by Grant N01-LM-4-3511 from the National Library of Medicine.
We thank Steve Ostrem, Brian Aker, and Teresa Choi of the Electric Differential Multimedia Laboratory at The University of Iowa College of Medicine for their expert assistance.
Footnotes
- Received December 21, 1998.
- Accepted May 11, 1999.
Reprint requests to (D.M.D.) Children's Hospital of Iowa, Department of Pediatrics, 200 Hawkins Dr, Iowa City, IA 52242. E-mail: donna-dalessandro{at}uiowa.edu
REFERENCES
- Copyright © 1999 American Academy of Pediatrics
