Published online January 2, 2007
PEDIATRICS
Vol. 119
No. 1
January 2007, pp.
e53-e60
(doi:10.1542/peds.2006-1538)
The Use of the World Wide Web by Medical Journals in 2003 and 2005: An Observational Study
David L. Schriger, MD, MPHa,b,c,
Sripha Ouk, MD, MPHa,b and
Douglas G. Altman, DScc
a Emergency Medicine Center
b School of Medicine, University of California Los Angeles, Los Angeles, California
c Cancer Research UK/National Health Service Centre for Statistics in Medicine, Oxford, United Kingdom
 |
ABSTRACT
|
|---|
OBJECTIVES. The 2- to 6-page print journal article has been the standard for 200 years, yet this format severely limits the amount of detailed information that can be conveyed. The World Wide Web provides a low-cost option for posting extended text and supplementary information. It also can enhance the experience of journal editors, reviewers, readers, and authors through added functionality (eg, online submission and peer review, postpublication critique, and e-mail notification of table of contents.) Our aim was to characterize ways that journals were using the World Wide Web in 2005 and note changes since 2003.
METHODS. We analyzed the Web sites of 138 high-impact print journals in 3 ways. First, we compared the print and Web versions of March 2003 and 2005 issues of 28 journals (20 of which were randomly selected from the 138) to determine how often articles were published Web only and how often print articles were augmented by Web-only supplements. Second, we examined what functions were offered by each journal Web site. Third, for journals that offered Web pages for reader commentary about each article, we analyzed the number of comments and characterized these comments.
RESULTS. Fifty-six articles (7%) in 5 journals were Web only. Thirteen of the 28 journals had no supplementary online content. By 2005, several journals were including Web-only supplements in >20% of their papers. Supplementary methods, tables, and figures predominated. The use of supplementary material increased by 5% from 2% to 7% in the 20-journal random sample from 2003 to 2005. Web sites had similar functionality with an emphasis on linking each article to related material and e-mailing readers about activity related to each article. There was little evidence of journals using the Web to provide readers an interactive experience with the data or with each other. Seventeen of the 138 journals offered rapid-response pages. Only 18% of eligible articles had any comments after 5 months.
CONCLUSIONS. Journal Web sites offer similar functionality. The use of online-only articles and online-only supplements is increasing.
Key Words: epidemiology • medical publication • research methodology • World Wide Web
Abbreviations: WWW—World Wide Web • CI—confidence interval • CME—continuing medical education
Journal articles typically contain 2 to 6 pages of text and a few tables or figures and are bound together in a print journal.1 Although this format for presenting clinical research has been remarkably stable over 2 centuries, its persistence has had less to do with any theoretical advantage than with constraints related to the cost of typesetting and the cost of printing. Before the development of computerized page layout in the 1980s, tables and figures were used sparingly, because tables required complex manual setup, and figures had to be etched on plates, both costly processes. The overall length of articles was and is constrained by the cost of printing and, although not as costly as it once was, color remains more expensive than monochrome.
Computerized layout and printing and the World Wide Web (WWW) have removed these constraints. Now, tables, figures, and photographs can be produced and printed for essentially the same cost as text, and supplementary material (be it text, tables, figures, audio, or video) can be provided as Web-only appendices at very low cost. Journal editors and authors can now provide readers with a far more comprehensive account of a research effort than was economically feasible 10 years ago. Such comprehensiveness is highly desirable. The integrity of the medical literature has been questioned of late,2,3 and by using supplements to provide readers with a closer view of the data, authors reduce the potential for biases that can occur when the dimensionality of the data are reduced.4–9
The advantages of the WWW are not limited to supplementary appendices. Journals can post accepted articles to their Web site well before their appearance in print, speeding the dissemination of the information. The Internet can be used to e-mail journal Table of Contents to potential readers, to notify them when articles have been published in their field of interest, and to alert them when a given article is cited by others. Web versions of articles can link to articles cited in the bibliography, to other articles by the authors, and to relevant Web sites. Journals can establish Web pages that provide a forum for the postpublication critique of scientific articles and editorials.10 Finally, journals can use the WWW to facilitate article submission and peer review, eliminating postage costs, reducing printing and photocopy costs, speeding the process, and easing the task for authors who reside outside the country of origin of the journal.
In addition to enabling variability in the way that traditional print articles are structured, the WWW offers the possibility that selected articles or entire journals can exist solely on the Web. The Web-only model of journal publication eliminates printing costs, and this savings has made open access journals (the authors pay for the peer review services, and the article is available free to all with Web access) financially possible.11 The growth of Web-only journals from Biomed Central (now >140 journals) and Public Library of Science (6 journals) is clear evidence that the WWW is changing scientific publication.
We executed this research to examine how the WWW is changing traditional print journals. We were interested in knowing how print journals are currently using the Web and how quickly this usage is changing. We set 3 specific goals: (1) to characterize how journals are using the Web to provide online-only material, comparing 2003 and 2005; (2) to characterize what functionality journals are offering readers on their Web sites; and (3) to characterize how "rapid-response" online letter-to-the-editor features are being used.
|
METHODS
|
|---|
This effort consists of 3 separate, related studies. For each study, the base sample was a pool of 138 journals. These journals were the 5 journals with the highest 2003 impact factors that published original research in 28 clinical specialty areas as calculated by the Institute for Scientific Information and the top 15 journals in the "general and internal medicine journal" category.12 Some journals are ranked in more than 1 specialty, hence our total of 138 instead of 155.
Study 1: Use of Web-Only Supplementary Material
We wanted to understand how journals use the option of Web-only supplementary material. For this purpose, we selected 8 journals that we knew offered Web-only material and 2 journals that we assumed would have online-only content. These 2 journals (Annals of Surgery and Gastroenterology) represented medical specialties not covered by the other 8. We then randomly selected 20 journals from the pool of 138 top-ranked journals for the purpose of getting an unbiased estimate of the Web-only activity of higher-impact factor journals. Two journals (Annals of Internal Medicine and Lancet) in the random sample had already been selected in the first selection process. The list of 28 journals can be found in Table 1.
In July 2005, we examined both the paper and online versions of every March 2003 and 2005 issue of these journals to determine whether any articles were published only on the WWW and to determine whether articles published in print referenced any Web-only supplementary material. We included all of the original research articles, review articles, editorials, images-in-medicine articles, and case reports. In 3 instances, the April issue was used, twice because the journal was published bimonthly on even months and once because the journal did not go online until the April 2003 issue. All of the online material was categorized according to a taxonomy that we developed for this purpose (Table 2). Two raters independently piloted the taxonomy on 3 journals and found it to be feasible and perfectly reliable. Our analysis is descriptive. Data management and the calculation of confidence intervals (CIs) for the difference in the proportion of articles containing supplementary material in 2003 vs 2005 were calculated in Stata 8.0 (Stata Corp, College Station, TX).
Study 2: Functionality of Journal Web Sites
In this study, we describe the features of journal Web sites. We examined the Web site of each of the 138 higher-impact journals between April and August of 2005 for the presence or absence of each function. We developed the list of functions by having each author write down features that journals could theoretically offer on their Web sites. We took the union of these lists and ensured that we had not overlooked any functions by examining each major journal publisher's common Web platform. Once the list of functions was stabilized, 1 author scored the remaining Web sites. We included functions related to the publication itself (eg, Web availability of an article before its print publication, availability of personal digital assistant or PowerPoint downloads, and availability of online continuing medical education [CME]), functions related to postpublication activities (eg, online letters or rapid-response features and citation manager functions), linking functions (eg, to articles cited by the article or to articles that have cited the article), and electronic notification functions (eg, e-mailed table of contents and e-mailed corrections).
Study 3: The Use of "Rapid Response"
For this study we examined the 138 journals to identify those that attempt to create a public forum for the postpublication review of their articles by offering a "rapid-response" feature that allows readers to post comments about a specific article to a Web page of comments about that article. Typically, these pages are lightly edited, profanity and defamatory material is removed, but there is no attempt to censor opinion. We examined March 2005 issues of the 17 journals that had this feature and recorded the number of rapid responses per article, the continent of origin of each response, the number of days after publication that each response was posted, and the type of response (coded as: specific comment about the article's content, anecdote directly related to article, anecdote unrelated or vaguely related to article, response to another posting, general praise for the article, and response by the authors of the article). We looked at all issues of journals that were published monthly or bimonthly and the first 3 issues of journals that were published weekly. We examined each issue 
5 months after the date of paper publication. Two authors independently scored 3 journals to confirm the feasibility and reliability of the scoring method.
|
RESULTS
|
|---|
Study 1: Use of Web-Only Supplementary Material
We found 1778 articles in 94 issues of 28 journals. Fifty-six articles (7%) in 5 journals were Web-only. In 2005, 36% of articles in Pediatrics were published solely online, far more than any other journal (Table 3). The number of Web-only articles increased between 2003 and 2005, largely because of an increased use of Web-only images-in-medicine articles by Circulation and New England Journal of Medicine. In 5 instances, Web-only images-in-medicine articles were accompanied by video.
One journal, the British Medical Journal (BMJ), published the complete official version of many articles online and an abridged version in print (Table 3). The online version of the 27 articles that they published this way contained additional background (24), methods (26), results (26), technical reports (2), and references (26), as well as 56 additional tables in 24 articles and 21 additional figures in 11 articles.
We now consider the remaining 1695 articles that were essentially the same in print and online except for any Web-only supplementary material. Some of this supplementary material expanded on the standard sections of a print article (eg, detailed methods or additional results), whereas some offered nontraditional information (eg, information about the authors of the article, the reviewers of the article, or CME examinations; Table 2). We found no articles that included the study protocol or data set as an online-only supplement.
Thirteen of the 28 journals had no supplementary online content in either 2003 or 2005 (Table 1). By 2005, several journals were including Web-only supplements in >20% of their articles. Supplementary methods, tables, and figures predominated (Table 2). Over a 2-year period, the use of supplementary material increased by 5% (95% CI: 3%–8%) from 2% to 7% in the 20-journal random sample (Table 1). If we include all of the journals, the Web-only articles, and the British Medical Journal extended versions, the number of articles with supplementary material doubled from 7% to 14% (change: 7%; 95% CI: 4%–10%; Table 4). We observed increases for most journals and most supplementary material types (Tables 1 and 2). Increases were the result of both an increase in the number of journals that offer supplements and the frequency of supplements in each journal.
Study 2: Functionality of Journal Web Sites
Journals published by the same publisher tend to be Web-published on similar platforms and, hence, have similar features. More than 80% of the 138 journals in our study were on 1 of 10 Web platforms. Journals not affiliated with a major commercial publisher often contract with Highwire Press, a company that provides Web publishing for journals. Some journals can be obtained in 2 formats, the second being a service such as Science Direct, which offers the content of the journal but no journal-specific information.
Most Web sites exploit the WWW’s hypertext capabilities by linking the article to other relevant material and its e-mail capabilities by offering various forms of notification (eg, when a new issue of a journal has been released or when an article has been cited by another; Table 5). Interestingly, aspects of the WWW that would allow readers to interact with the data (eg, allowing readers to perform sensitivity analyses on study results), the narrative (eg, interactive case reports), or one another (eg, rapid-response features) were rarely used.
Study 3: The Use of Rapid Response
A total of 12% (17 of 138) of journal Web sites offered rapid response. We examined 24 issues of these 17 journals from March 2005. The issues contained 514 articles that offered the opportunity for rapid response. Six journals (6 issues containing 169 articles) did not have any postings, and 3 journals (5 issues containing 136 articles) had postings on just 1 article. Of all articles, 82% had no postings. There were 335 postings on the 90 articles that had postings. For these articles, the median number of postings was 2 (interquartile range: 1–4), the mean was 3.8, and the maximum was 42. The British Medical Journal had postings on 80% of its articles, Canadian Medical Association Journal (CMAJ) had postings on 59%, Annals of Internal Medicine had postings on 57%, Journal of Bone and Joint Surgery had postings on 35%, Gut had postings on 33%, and British Journal of Anesthesia had postings on 13% (Fig 1). The other 11 journals had responses on <10% of articles.
View larger version (12K):
[in this window]
[in a new window]
|
FIGURE 1 The graph shows how often Web pages that allow readers to comment on published articles had 1 comment for the 17 journals that offered this feature. The rapid-response pages of most of these journals were seldom used.
|
|
Most (45%) postings raised issues related to the content of the article, whereas 26% offered anecdotes related to the article, 18% responded to another posting, 5% were vaguely related to the article, 2% offered praise, and 4% were responses from the authors of the article. A total of 59% of responses were from Europe, 29% from North America, and 8% from Asia. Twenty-five percent of postings were made before print release or in the first 2 days after print release, 53% of postings were made in the first week, 84% in the first month, and 94% in the first 2 months.
|
DISCUSSION
|
|---|
Our article provides snapshots of how medical journals were using the WWW in 2005 and 2003. The following observations summarize our findings. First, there are many Web-only journals but very few print-only journals; virtually all of the high-impact print journals now have some presence on the Web. Second, there are rather few predominant journal Web site styles. As a result, most journal Web sites offer similar functionality. Features, such as linkage to relevant articles, search functions, and e-mail notification, predominate. Third, print journals are using the Web to provide additional content: Web-only articles, Web-only unabridged versions of articles, and Web-only supplementary materials. Some form of Web-only material was present in 7% of 2005 articles in our random sample, up from 2% in 2003. Fourth and finally, we found little evidence that journals and their readers are exploiting the interactive potential of the WWW. Rapid-response features that foster postpublication review by readers were offered by a minority of journals and for most of these journals were seldom used. We did not find examples of other interactive possibilities, such as reader-controlled sensitivity analyses or decision analyses or interactive case reports, although the use of video seems to be increasing.
The primary limitation of our research is the small sample size for each journal. Journal-specific estimates may be quite imprecise. Our use of high-impact journals limits our conclusions to these journals. Because lower-impact journals are less likely to use the WWW, this study likely overestimates the use of the WWW by all journals. Other limitations include our failure to blind the data abstractors to the purpose of the study or to the identity of the journals they were abstracting. Furthermore, we did not formally assess interrater reliability. Given the straightforward nature of the items in our taxonomies, we do not believe that these methodologic shortcomings are likely to introduce bias. Finally, as evidenced by these data, journal Web sites are undergoing rapid change, and our data are unlikely to represent the current state of each journal.
Although the presentation of Web-only supplementary material is still modest, perhaps the more important statistic is that it has tripled in 2 years. Should that trend continue, it will not be long before the majority of articles offer some type of Web-only supplementation. We are enthusiastic about this change. Ziman13 has argued that an essential aspect of the scientific method is "consensibility," that is "that each message should not be so obscure or ambiguous that the recipient is unable either to give it whole-hearted assent or to offer well-founded objections." In contrast to articles reporting medical bench research where, through the citation of technical methods and presentation of representations of physical evidence (eg, photographs of electrophoresis gels), consensibility can be achieved in a standard-length scientific article, most clinical research cannot achieve consensibility in this format. The typical clinical research project data set, with multiple variables collected at multiple time points on multiple patients, has a dimensionality that cannot be adequately represented in a standard format article.8 Authors reporting data of this kind must drastically reduce dimensionality to fit results into a 6-page article with a few tables and figures. The principle of consensibility mandates that the reader be given sufficient information to permit a logical consideration of whether the authors' conclusions are warranted. When a standard-length article cannot achieve consensibility, Web-based supplements can do so without unduly burdening the casual reader or creating untoward cost for the publisher.9
An extreme solution to the consensibility issue is to post the entire data set to the WWW (thereby revealing the data in its full dimensionality).14,15 We found no evidence of this practice in our sample. Similarly, a strong case has been made for the registration of trials and the publication or archiving of protocols before the inception of data collection.16,17 Again, we found no instance of a journal Web publishing the protocol associated with a trial in our sample, although the Lancet has instituted a program in which they peer review protocols and Web publish a very brief synopsis of them.18
We have argued that recognition of the theoretical benefits of enhanced consensibility will lead to an expansion of the use of Web-only supplementary materials. Pragmatic forces will also encourage this trend, because readers' attention spans for print material have shortened, and journal printing and distribution costs have increased. Other journals (eg, American Journal of Respiratory and Critical Care Medicine and American Journal of Obstetrics and Gynecology) have followed the British Medical Journal's lead and are printing pithy abridged versions of articles while posting the longer official version to the Web19,20 despite some evidence that readers and authors do not like this split format.21 It could be argued that the only thing keeping print versions of full-length articles extant is the pharmaceutical industry's willingness to purchase print advertisements and the journals' need to put something between these ads. Taken together, it is likely that the trend that we have observed will continue, and journals will increasingly offer online-only supplementation to print articles. Journal editors will need to carefully evaluate how material is distributed between print and online vehicles and how online material is peer reviewed. A standardized approach to the citation and archiving of online-only materials will also be necessary to ensure that these materials will be identifiable and obtainable in the future.22 Mechanisms must be developed to notify Web users that they are reading material that has been altered after its initial posting and to make the original posting available for perusal.
Communication among peers in an essential element of the scientific process. Electronic submission and peer review, the posting of articles to journal Web sites before their appearance in print, and Web-based postpublication review, by speeding and facilitating the interaction of researchers, should expedite and enhance the scientific process. In 2005, 57% of journals posted articles to their Web site before their appearance in print, and 12% of journals offered readers a forum for responding to each article. We were surprised that more readers did not take advantage of the postpublication review feature; 82% of such pages had no entries. Is this because readers do not read the articles, do not have anything to say, or are not interested in participating in such a forum? In the face of such low participation rates, how do we explain that the British Medical Journal averaged 6 postings per article on the 80% of articles that had postings? Perhaps it is because all of the British Medical Journal content was free to all at the time of this study or that the British Medical Journal has had a stronger Web presence for a longer period of time than many other journals and has cultivated a group of users who are comfortable using the WWW in this way.10,23 It remains to be seen whether postpublication review becomes a standard fixture in journal Web sites or fails to catch the imagination of journal readers.
We found evidence of increased use of video on the Web from 2003 to 2005, and recent activity suggests that this trend will continue.24 Journals have yet to exploit the WWW capacity for interactive statistical modeling. Articles that use decision analysis could include a Web site that allows readers to perform their own sensitivity analyses and, as Lehmann and Goodman suggest,24 the WWW could be used to foster reader-controlled Bayesian analyses of an article's data. Although such methods seem like a better way to present research findings, they have yet to become common practice. It is unclear whether this is a failure of the authors and the editors or a reflection of the readers' lack of interest for such material.
|
CONCLUSIONS
|
|---|
We have found that the WWW is playing an increasing role in journal operations and journal publication. This trend is likely to continue with a greater presence of Web-only journals and Web-only supplements to print journal articles. It remains to be seen whether journals and their readers will harness the interactive capabilities of the WWW.
|
FOOTNOTES
|
|---|
Accepted Jul 20, 2006.
Address correspondence to David L. Schriger, MD, MPH, 924 Westwood Blvd, #300, Los Angeles, CA 90024-2924. E-mail: schriger{at}ucla.edu
The authors have indicated they have no financial relationships relevant to this article to disclose.
This work was presented at the Fifth International Congress on Peer Review and Biomedical Publication; September 16-18, 2005; Chicago, IL.
|
REFERENCES
|
|---|
- Schriger DL, Sinha R, Schroter S, Liu PY, Altman DG. From submission to publication: a retrospective review of the tables and figures in a cohort of RCTs submitted to the BMJ.
Ann Emerg Med. 2006; In press
- Horton R. The dawn of McScience.
New York Rev Books. 2004;51
:7
–9
- Smith R. Medical journals are an extension of the marketing arm of pharmaceutical companies.
PLoS Med. 2005;2
:e138[CrossRef][Medline]
- Chalmers I, Altman DG. How can medical journals help prevent poor medical research? Some opportunities presented by electronic publishing.
Lancet. 1999;353
:490
–493[CrossRef][ISI][Medline]
- Tufte ER.
The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press; 1983
- Schriger DL, Cooper RJ. Achieving graphical excellence: suggestions and methods for creating high-quality visual displays of experimental data.
Ann Emerg Med. 2001;37
:75
–87[CrossRef][ISI][Medline]
- Tobin MJ. The role of a journal in a scientific controversy.
Am J Respir Crit Care Med. 2003;168
:511
–515[Free Full Text]
- Schriger DL. Presentation of data: packaging without manipulation or deception [video]. Available at: http://clinicalcenter.nih.gov/about/news/grandrounds/gr06/apr06.shtml. Accessed April 29, 2006
- Delamothe T, Mullner M, Smith R. Pleasing both authors and readers.
BMJ. 1999;318
:888
–889[Free Full Text]
- Crossan L, Delamothe T. Letters to the editor: the new order. Please respond to articles using website, email, or disk—but not paper.
BMJ. 1998;316
:1406
–1410[Free Full Text]
- Brown PO, Eisen MB, Varmus HE. Why PloS became a publisher.
PLoS Biol. 2003;1
:e36[Medline]
- Thomson. The ISI impact factor. Available at: http://scientific.thomson.com/free/essays/journalcitationreports/impactfactor. Accessed April 29, 2006
- Ziman R.
Reliable Knowledge: An Exploration of the Grounds for Belief in Science. Cambridge, England: Cambridge University Press; 1978:6–7
- Davey Smith G. Increasing the accessibility of data.
BMJ. 1994;308
:1519
–1520[Free Full Text]
- Delamothe T. Whose data are they anyway?
BMJ. 1996;312
:1241
–1242[Free Full Text]
- Eysenbach G. Peer-review and publication of research protocols and proposals: a role for open access journals.
J Med Internet Res. 2004;6
:e37
. Available at: www.jmir.org/2004/3/e37[Medline]
- Chan A-W, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
JAMA. 2004;291
:2457
–2465[Abstract/Free Full Text]
- Roy LP. Review of research protocols.
Lancet. 1999;353
:428[CrossRef][ISI][Medline]
- Tobin MJ. The official copy of AJRCCM is posted but not printed.
Am J Respir Crit Care Med. 2002;166
:905
–906[Free Full Text]
- American Journal of Obstetrics and Gynecology. Announcement: American Journal of Obstetrics and Gynecology to launch a new print format. Available at: http://journals.elsevierhealth.com/periodicals/ymob/content/announcement. Accessed April 29, 2006
- Mullner M, Waechter F, Schroter S, Squire B. How should abridged scientific articles be presented in journals? A survey of readers and authors.
CMAJ. 2005;172
:203
–205[Abstract/Free Full Text]
- Evangelou E, Trikalinos TA, Ioannidis JP. Unavailability of online supplementary scientific information from articles published in major journals.
FASEB J. 2005;19
:1943
–1944[Abstract/Free Full Text]
- Davies S, Delamothe T. Revitalising rapid responses.
BMJ. 2005;330
:1284[Free Full Text]
- McMahon GT, Ingelfinger JR, Campion EW. Videos in clinical medicine: a new journal feature.
N Engl J Med. 2006;354
:1635[Free Full Text]
- Lehmann HP, Goodman SN. Bayesian communication: a clinically significant paradigm for electronic publication.
J Am Med Inform Assoc. 2000;7
:254
–266[Abstract/Free Full Text]
PEDIATRICS (ISSN 1098-4275). ©2007 by the American Academy of Pediatrics
TOP
ABSTRACT
METHODS
