The diffusion of information is crucial to the progress of medicine. The great profusion of conferences, courses, and specialized journals allows a reasonably fast dissemination of knowledge. Nevertheless, electronic mail offers a low-cost and faster system of information exchange that has been used extensively in the academic environment. The growth of the Internet during the last few years has enhanced this way of sharing information, becoming a popular way of communication among nonacademic professionals.

Mailing lists are discussion groups usually focused on a subject, whose members or subscribers interchange messages by electronic mail. Some mailing lists are manually maintained. Others are maintained by a number of commercial and public domain software packages including Listserv, Majordomo, Listproc, and others. Postings sent to the group are received by computers which run these softwares or list servers, and automatically send them to all the members of the mailing list. Thus, the comments may be read and discussed by the other subscribers, and an electronic dialogue of comment, reply, and further replies is established as an international conference on a topic held daily.¹ The subscription to a mailing list is free. All that is required is a standard electronic-mail account and a simple piece of communication software.

The degree of control during postings to a mailing list server varies widely. The lowest level of control is restricting postings to members of the list. This prevents nonmembers, particularly advertisers, from posting to the mailing list. The next level is to subject each posting to a process in which only messages that the manager of the mailing list approves are available by the members of the mailing list. The highest level of control includes both the posting approval and its edition as needed. The more control a moderator exercises on a list the more work is involved. Although the quality of a moderated list is generally better it is at the expense of some spontaneity.

The arrival of the Internet and its electronic forums of discussion in the field of biomedicine are generating concern as to whether the opinions expressed in this media are sufficiently reliable.² Consequently it has been suggested that certain measures of control should be introduced into these ever-changing, somewhat chaotic groups that have emerged as a result of computer networks. One possible way to evaluate opinions that are expressed on-line is to check up on the credentials as authors of the participants by looking at their inclusion in bibliographical databases. This is the approach we took to analyze both the potential and real quality of a subset of pediatric Internet discussion groups. At the same time we have also compared these groups with the more traditional forum of medical journals in which the latest medical findings are presented.

MATERIALS AND METHODS

Calculation of IFa Among Subscribers on Each Mailing List

Leaving n from this equation: in which n is assumed for infinite populations.

K was obtained from the t-Student table with n_x-1 degrees of freedom for a specific probability (in our case a probability level of 95%, with type I error equaling .05). Then, an iterative process of increasing the degrees of freedom was performed until two equal consecutive n values were found. The final representative sample size for finite populations (n) from each mailing list (N) is derived as follows:

The names of all members from these samples were searched in the Silver-Platter (Silver Platter, Int, Boston, MA), Medline bibliographic data base on the January 95 to December 95 CD, 1995 edition. The impact factor of each of their papers was obtained after the Science Citation Index (Institute for Scientific Information, Philadelphia, PA) 1994. The sum of all the individual impact factors divided by the size of each sample gave an Index of Potential Quality (IPQ) for each mailing list.

We also studied all the postings sent to each mailing list. The less active lists were observed throughout a longer period of time to avoid missing any high quality posting. Each member who posted to the list was identified and searched as an author in Medline, and the impact factor of each article was obtained as mentioned above. The sum of the impact factors of these articles gave the impact factor of each participant. From this we defined the following variables:

1. The activity of the group, or the number of electronic mailings per day.
2. The index of participation, the percentage of subscribers on each list that posted to the list during the observation period.
3. The percentage of those participants that had articles published in 1995.
4. The IFA of the published authors, that is the sum of the individual impact factors of the various articles divided by the number of authors.
5. The IFa per participant (IFp), that is the sum of the individual impact factor of the published authors posting messages divided by the total number of people posting messages whether published authors or not.
6. The IFa per posting (IFe), which was calculated by the sum of the individual impact factors of participants in the discussions multiplied by the number of postings of each individual. This sum was then divided by the total number of postings during the period of observation.
7. Finally, the Yield Index of the group which is calculated by multiplying the IFe by the number of postings generated each day during the period of observation.

We also analyzed the number of postings each group member sent. This allowed us to assess which participants carried the weight of the discussions in each list and what their reputation was in terms of published articles.

Finally, a selected subset of journals specializing in pediatrics was contrasted with the mailing lists on the Internet. The journals selected were published in the same months as those in which we monitored the mailing lists. These journals were:

1. Journal of Pediatric Surgery, April 1996.
2. Journal of Pediatrics, January, February, March 1996 (the section on Neonatal and Fetal Medicine).
3. Pediatric Nephrology, April 1996.
4. Journal of the American College of Cardiology, February, April 1996 (the section on pediatric cardiology).
5. Journal of Child Neurology, May 1996.
6. Pediatric Pulmonology, December 1995.

The original articles and short communications from each journal were analyzed. The first authors were searched in Medline and their publications in 1995 were evaluated in terms of their impact factors in the Science Citation Index as mentioned above. The individual impact factor for each first author was calculated and their sum divided by the number of articles analyzed gave a coefficient for each publication. This coefficient was contrasted with the IPQ of the lists, and also with the IFa per participant (IFp) on the lists on the Internet.

Statistical analysis was performed with the Statistica software (Statsoft Inc, Tulsa, OK, 1994).

RESULTS

Table 1. Some Pediatric Mailing Lists on the Internet* [View Table]

Table 2 shows the data concerned with the potential of the lists. The most popular lists, which had most subscribers, were NICU-NET and PICU (1889 and 1197, on June and April 1996, respectively). Those with the highest percentage of published authors for 1995 among subscribers were PEDPATH and PEDIHEART (28%). When the quality of their articles was considered, the lists with the highest potential quality based on their IPQ were also PEDPATH and PEDIHEART (2.9 and 1.4, respectively). No significant correlations among the indexes were found.

Table 2. Potential of the Lists* [View Table]

The study of postings in the mailing lists generated the data shown in Table 3. The average observation period was 43.8 days (range, 22 to 86). The most active lists were NICU-NET and PICU, with 12 and 10.75 postings per day, respectively (data not shown). The most participative ones were CHILD-NEURO, PED-LUNG, PEDPATH, and PEDIHEART (17%, 16%, 14%, and 14% of subscribers, respectively). The one which showed the highest percentage of published authors who actively participated in discussions was CHILD-NEURO (29%). PEDPATH (IFA 10.5) and PEDIHEART (9.0) members had the best average quality publications (data not shown). The highest IFa of the active participants in the mailing lists (IFp), were found in PEDPATH and CHILD-NEURO (2.4 and 2.0, respectively). PEDPATH and CHILD-NEURO rated the highest average IFe, 2.7 and 2.1, respectively (data not shown). CHILD-NEURO and PEDIHEART had the highest yield (Yield Index, 11.0 and 6.0, respectively), that is to say they had on average many postings by published authors with high impact factors.

Table 3. Study of Postings on the Lists* [View Table]

When the IPQ of the lists was contrasted with the average IFp in the discussions (IFp), one can generally note a certain similarity, but on occasion the IPQs are slightly greater (Fig 1). Exceptional cases are CHILD-NEURO and PED-LUNG, with an apparently better average quality among their participants than among their subscribers (IFp = 2.0 vs IPQ = .8, and IFp = 1.0 vs IPQ = .6, respectively). The opposite cases, that is much higher quality in their subscribers than in their participants, are PED-SURG-L (IPQ = .7 vs IFp = .1), PEDI-OTO (IPQ = .5 vs IFp = .1), PICU (IPQ = .4 vs IFp = .2) and NICU-NET (IPQ = .4 vs IFp = .2).

In addition, the subscribers of each mailing list who participated in discussions during the observation period were grouped according to the number of postings they sent to the lists. The average IFp was then calculated as a function of the postings per participants (Fig 2). In general, the higher the postings per participant the higher the IFp.

The data from the analysis of the journals of pediatric specialties are shown in Table 4. The journal with the best IFa per first author was the section of pediatric cardiology from the Journal of the American College of Cardiology (5.3). Except for Pediatric Pulmonology the rest of the publications showed a high index (range, 40 to 50%) of first authors who published in 1995. The IFa per first author of the journals was higher than the IFp of the corresponding mailing list in all cases, but CHILD-NEURO reached an impact factor similar or higher than some of the journals (Fig 3).

Table 4. Reviewed Pediatric Journals* [View Table]

DISCUSSION

Some interesting findings arise when we contrast the potential quality of the subscribers of a list (IPQ, average IF per subscriber) and the postings (IFp, average IF per participant) (Fig 1). CHILD-NEURO and PED-LUNG showed much higher IFp than their IPQ, that is real quality much higher than potential quality. This may be attributed to an underestimation of the IPQ because 23% and 27% of the subscribers, respectively, were not identified (Table 2). Similarly, this may be the reason to explain why PED-SURG-L and PEDI-OTO showed such a low IFp as regards to their IPQ (0.1 vs .7, and .1 vs .5, respectively). In fact, 23% and 29% of their subscribers, respectively, were not identified. PICU and NICU-NET showed very low IFp compared with their IPQ despite their low percentages of not available data in the study of potential. This could be interpreted as a result of the higher participation of nurses and other health professionals with low publishing capacity in these lists (28% of members in NICU-NET are not physicians),⁷ or the unequal participation of members in on-line interventions.

The number of postings that each participant sent to the discussions per IFa in each group, was the variable created to know if more active subscribers necessarily had higher impact factor as published authors. From the data we conclude that on some lists published authors with high impact factor often posted messages which had a significant effect on the global impact factor of the list (eg, PICU and PEDS-UROLOGY). However, in other lists the participation was more uniformly shared (eg, PEDPATH, CHILD-NEURO, PEDNEPH, and PEDIHEART), that is to say people with high standing as authors contributed relatively little in comparison with others of lesser recognized standing.

Clearly the introduction of the Internet into the world of science brings with it a series of advantages because it allows both an exchange of ideas and knowledge, and also a greater speed of communication and easier access worldwide. A recent case exemplifies these advantages. In Haiti, when a batch of locally produced acetaminophen was contaminated with diethylene glycol, various children were intoxicated and several of them died. This fact was made known on PEDNEPH and allowed a rapid intervention of pediatric nephrology departments of various United States centers which proved vital in the ensuing evacuation of these children with renal failure. They were provided with hemodialysis in United States hospitals apart from facilitating a rapid etiological diagnosis and the identification of the toxic agent. The first published recognition of the event appeared in the New York Times on 26th June but it had been made known on PEDNEPH by 17th June.

Moreover, this is achieved at a lower cost because it eliminates time consumed on printing, transport, and distribution. We are on the threshold of a new era of information exchange in medicine which may impact the relationship and mode of contact between physician scientists, the role of biomedical publications, and the very way in which medicine is taught.¹¹ Some authors have pointed out the necessity of rethinking the role that biomedical journals should have before such publications "die a natural death" and, bearing this in mind, many journals already offer partial or complete electronic versions,¹² or even articles not available on the written edition (eg, Pediatrics electronic pages, http://www.pediatrics.org). Finally others question whether mailing lists are, in fact, the ideal forums for the interchange of medical information or whether they are useful sources of information for those who practice medicine.²

To date we are not aware of previous studies dealing with the quality of various forums of medical discussions on the Internet. Hence, the different quality indexes used in the present study are open to certain dispute. However, they are based on the accumulation of defined impact factors generated by the published articles of the various members of the discussion groups, a way for any scientific group to gain prestige in a given field of science. For this reason we have deemed it useful to assess the quality of Internet resources in other fields of study. Finally, we accept that the comparison of the coefficients that were obtained from the analysis of certain journals may be open to different interpretations.

It is important to emphasize that by no means this study suggests that a three-month subscription to CHILD-NEURO would yield more than one issue of the Journal of Pediatric Surgery or of Pediatric Pulmonology. On the contrary our data suggest that just as the live exchanges of knowledge through direct, person-to-person contact or by attending scientific congresses serve as a recognized way of enriching our knowledge, so should this new way of exchanging scientific information from personal computer to personal computer be respected and encouraged as a powerful, innovative, and successful complementary method.