Does Presentation Format at the Pediatric Academic Societies’ Annual Meeting Predict Subsequent Publication?
,||
* Robert Wood Johnson Clinical Scholars Program, University of Washington, Seattle, Washington
Department of Pediatrics, Indiana University, Indianapolis, Indiana
Department of Pediatrics, University of Washington, Seattle, Washington
|| Child Health Institute, University of Washington, Seattle, Washington
 |
ABSTRACT |
|---|
 TOP ABSTRACT METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Objective. The validity of research presented at scientific meetings continues to be a concern. Presentations are chosen on the basis of submitted abstracts, which may not contain sufficient information to assess the validity of the research. The objective of this study was to determine 1) the proportion of abstracts presented at the annual Pediatric Academic Society (PAS) meeting that were ultimately published in peer reviewed journals; 2) whether the presentation format of abstracts at the meeting predicts subsequent full publication; and whether the presentation format was related to 3) the time to full publication or 4) the impact factor of the journal in which research is subsequently published.
Methods. We assembled a list of all abstracts submitted to the PAS meetings in general pediatrics categories in 1998 and 1999, using both CD-ROM and journal publications. In each year, we chose up to 80 abstracts from each presentation format ("publish only," "poster," "poster symposium," "platform presentation"). We chose either 1) all abstracts in each format or 2) when there were >80 abstracts, a random selection of 80 of them. We assessed each selected abstract for subsequent full publication by searching Medline in March 2003; if published, then we recorded the journal, month, and year of publication. We used logistic and linear regression to determine whether publication, time to publication, and the journal’s impact factor were associated with the abstract’s presentation format.
Results. Overall, 44.6% of abstracts presented at the PAS meeting achieved subsequent full publication within 4 to 5 years. There were significant differences between the rates of subsequent full publication of abstracts submitted but not chosen for presentation at the meeting (22.2%) and those that were chosen for presentation in poster sessions (40.0%), poster symposia (44.1%), and platform presentations (53.8%). There were no meaningful differences between the presentation formats in their mean time to publication and their mean journal impact factor.
Conclusions. PAS meeting attendees and the press should be cautious when interpreting the presentation format of an abstract as a predictor of either its subsequent publication in a peer-reviewed journal or the impact factor of the journal in which it will appear.
Key Words: pediatrician • abstract • publication • meeting
Abbreviations: PAS, Pediatric Academic Society
One of the primary purposes of presenting research at scientific meetings is to disseminate important research findings as soon as possible. However, the validity of research presented at scientific meetings has been a concern.1,2 Practicing clinicians who attend meetings may choose to alter their clinical practice based on presented findings. In addition, the popular press and the Internet can disseminate findings presented at conferences widely and quickly, which has caused continued debate on the "Ingelfinger rule." This rule states that research that has received extensive prepublication coverage in the press cannot be published in a peer-reviewed journal.3–5 Although coverage by the popular press may not preclude consideration by many academic journals, releasing information to the press at the time of scientific meetings may be premature. Presentations are chosen on the basis of submitted abstracts, which may not contain enough information to assess correctly the quality of the research.1,2,6 Results presented in an abstract may be different from those that later appear in a peer-reviewed publication.7
Those who have studied the fate of abstracts from scientific meetings have used the study’s subsequent publication in a peer-reviewed journal as a measure of quality.7–11 Rates of publication have ranged from 34% to 77%, differing both by meeting and specialty. Pediatric meetings have varied in their selection of subsequently published abstracts as well. In the United Kingdom in 1996, 52% of abstracts presented in a plenary setting at the Pediatric Research Society meeting and 77% of those at the British Pediatric Association meeting were subsequently published.9 More than 20 years ago, 47% of general pediatric topic abstracts presented at the Ambulatory Pediatric Association and 54% of abstracts at the American Pediatric Society–Society for Pediatric Research meetings were subsequently published; the mean time to publication was 20 months after presentation.8
Since the mid-1980s, the number of abstracts submitted each year in these general pediatric categories has increased 3-fold, presumably making the selection process more difficult. Although the number of plenary presentations has increased somewhat, the number of abstracts chosen for presentation as posters has increased enormously. Furthermore, to our knowledge, no one has examined whether the abstract presentation format at a pediatric meeting is related to subsequent full publication as a manuscript in a peer-reviewed journal.
We sought to determine 1) the proportion of abstracts on general pediatrics topics submitted to the annual Pediatric Academic Society (PAS) meeting that were ultimately published in peer-reviewed journals; 2) whether the presentation format of abstracts at the PAS meeting predicts subsequent full publication; and whether the presentation format is related to 3) the mean time to full publication or 4) the impact factor of the journal in which research is subsequently published.
|
METHODS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Selection of Abstracts
We assembled a list of all abstracts submitted to the PAS meetings in 1998 and 1999 using both CD-ROM and journal publications, from which we collected the abstracts’ titles, authors, bodies, and presentation formats. Presentation formats included "publish only" (rejection), "poster," "poster symposium," or "platform presentation." From all abstracts, we chose those that were grouped in session categories that we believed concerned general pediatric topics. These included adolescent medicine, behavioral pediatrics, clinical bioethics, emergency medicine, epidemiology, general pediatrics, pediatric education, and preventive pediatrics. We then grouped these abstracts by year and by presentation type, from which we selected our final pool of abstracts to be examined. In each year, we chose up to 80 abstracts from each presentation format: either all abstracts in each format or, when there were >80 abstracts, a random selection of 80 of them. We used a random number generator function in Microsoft Excel (Microsoft, Redmond, WA) to make these selections.
Assessment of Subsequent Publication
Two authors (B.A.T. and S.R.) independently assessed each selected PAS abstract for subsequent full publication. These authors were provided the abstract authors’ names only, without titles or institutions. They searched Medline for articles published between January 1, 1998, and February 1, 2003. A published manuscript was considered to be a full publication of a selected PAS abstract when it satisfied the following criteria: 1) the first author of the PAS abstract was an author of the full publication 2) at least 1 outcome from the PAS abstract was an outcome of the manuscript. An outcome was defined as a finding stated in the Results section of the PAS abstract that was also stated in the Results section of the manuscript’s abstract. When a full publication was confirmed, we recorded the journal, month, and year of publication. When a journal was published every 2 months, we defined the time of publication as occurring halfway between the 2 months. When a journal claimed publication in Spring or Fall, we defined the month of publication as March or October, respectively.
The abstracted data were independently compared between the 2 data abstractors. In measuring subsequent publication, the data collectors had excellent concordance, with a 91.0% agreement and a 
of 0.81. All discrepancies were resolved by a third author (A.E.C.).
Data Analysis
For each presentation format, we calculated the proportion subsequently published in full, as well as the number of months between the meeting and publication. We used logistic regression to test for significant differences in the publication rates between the presentation formats. We also performed linear regression to determine whether the time to publication or the journal’s impact factor was related to an abstract’s presentation format. The year of the meeting was included as a covariate in all regressions to control for potential differences in the selection process between years. Differences between presentation formats were tested using the test for the equality of coefficients.12 Impact factors were obtained from the Institute for Scientific Information Web of Knowledge Journal Citation Reports (isi8.isiknowledge.com/portal.cgi). Calculations were performed using the STATA 8.0 statistical package (STATA Corporation, College Station, TX). This study was approved by the University of Washington Institutional Review Board.
|
RESULTS |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
For the 2 meetings, 1321 abstracts were submitted in the general pediatric categories (622 in 1998 and 699 in 1999), 1156 (87%) of which were presented at the meetings. Overall, 165 abstracts were chosen as "publish only" (ie, rejected for presentation), 673 as "poster," 136 as "poster symposium," and 347 as "platform presentation." Of the 614 abstracts selected for study (46.5% of submitted), 454 of them were presentations at the meetings (160 "platform presentation," 136 "poster symposium," 160 "poster," and 158 "publish only"). Of the abstracts selected for study, 210 (44.6% when weighted for the entire meeting) were subsequently published in full. There were significant differences between the rates of subsequent full publication between abstracts that were not presented at the meeting (22.2%) and those that were presented in poster sessions (40.0%; P < .002), poster symposia (44.1%; P < .001), and platform presentations (53.8%; P < .001; Table 1). Intergroup comparisons showed that there was a significant difference between the publication rates of those in the "poster" and "platform presentation" categories (P < .02) but no significant differences between the "poster" and "poster symposium" categories (P < .48) and "poster symposium" and "platform presentation" categories (P < .10).
|
Of the abstracts later published in full, the mean times to publication between abstracts not selected for presentation at the meeting ("publish only") and those that were selected were not significantly different (Fig 1). "Poster" presentations, however, took significantly longer (mean: 26.5 months) to achieve publication than the "poster symposium" (mean: 20.3; P < .008) and "platform presentation" (mean: 21.4; P < .02) formats. No significant differences existed between the mean impact factors of the journals in which abstracts were subsequently published, regardless of presentation format (Table 2).
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
Overall, 44.6% of abstracts presented at the PAS meeting in 1998 and 1999 were subsequently published in peer-reviewed journals available in Medline. The rate of full publication of abstracts presented at the PAS meeting was significantly higher than that of abstracts that were not selected for presentation. There were some statistically significant differences in the further publication rates between abstracts in "poster" and "platform presentation" formats but no differences between other presentation formats. When an abstract was later published, its selection and presentation format had no meaningful impact on its time to publication or on the impact factor of the journal in which it appeared. Our findings suggest that PAS meeting attendees and the press should be careful in interpreting a PAS abstract’s presentation or presentation format as a predictor of its subsequent publication or the impact factor of the journal in which it will appear.
Our study has some limitations that warrant consideration. We sampled a fraction of abstracts and therefore can generalize only conservatively to all abstracts presented at the PAS meeting before or after the years of study. We also limited our study to abstracts in general pediatrics categories, as this was our area of interest. It is possible that abstracts in subspecialty fields would have different results than those presented here. It is possible that some abstracts were published in journals that did not appear in the Medline database. We have no reason to believe, however, that any one presentation format would differ in its likelihood of publication in such a journal. We were able to look only for publication that occurred up to 46 months after the 1999 meeting and 56 months after the 1998 meeting. It is possible that some abstracts could still be published. However, research has shown that 90% or more of studies are published within 4 years of the meeting.9–11 We have no reason to believe that any one presentation category would be more or less likely to publish >4 years after the meeting. Some may argue that journal impact factor is an unproved measurement of journal quality. Recent work has shown, however, that both practitioners and researchers believe that it is a valid indicator of quality.13 Some may also question the validity of peer review as the gold standard of research quality.14 However, scientific peer review remains the standard by which the quality of science is assessed.
That less than half of the research presented at such a scientific meeting is ever subsequently published should cause practitioners, conference organizers, and the press to pause for consideration. Just how much credence should be given to early research findings is unclear. It is important to remember that only 12.5% of abstracts submitted to this meeting were not presented in some format at the meeting. The peer-review process that determines abstract acceptance for scientific meetings may not be as thorough as peer review that determines later publication. Because the latter is the standard used today for most changes in our fund of knowledge, the former must be viewed with some skepticism.
There is also the danger that the abstract peer-review process is actually leading to the underreporting of valid research. A study of abstracts not published at a scientific meeting found that researchers whose abstracts were rejected by a scientific meeting ("publish only") were significantly more likely to believe that a journal would reject their full manuscript.15 Other research has found that the major reason for nonpublication of research is nonsubmission.10 Therefore, the abstract review process may not only lend credence to research that later fails to meet the standards of the journal peer-review process but also may hinder valid research from being further published.
The subsequent publication rates of the PAS meeting are within the range of other meetings.7,8,10,11 Because >87% of abstracts submitted to the meeting are presented in some format and fewer than half of these are subsequently published, many of the presented results may be preliminary. As such, those who interpret information presented at meetings should exercise caution in doing so.
|
ACKNOWLEDGMENTS |
|---|
Support for Drs Carroll and Sox was provided by the Robert Wood Johnson Clinical Scholars Program. Support for Dr Christakis was provided by the Robert Wood Johnson Generalist Physician Faculty Scholars Program. We thank the Society for Pediatric Research for help in obtaining abstract publications and CD-ROMs.
|
FOOTNOTES |
|---|
Received for publication Apr 17, 2003; Accepted Aug 22, 2003.
Reprint requests to (A.E.C.) Department of Pediatrics, Indiana University School of Medicine, Riley Research, Rm 330, 699 West Dr, Indianapolis, IN 46202. E-mail: aaecarro{at}iupui.edu
The views expressed within this article are those of the authors and do not necessarily represent the views of the Robert Wood Johnson Foundation, the University of Washington, or Indiana University.
|
REFERENCES |
|---|
|
TOPABSTRACTMETHODSRESULTSDISCUSSIONREFERENCES |
|---|
1. Soffer A. Beware the 200-word abstract! Arch Intern Med.1976; 136 :1232 –1233
2. Relman AS. News reports of medical meetings: how reliable are abstracts? N Engl J Med.1980; 303 :277 –278[Web of Science][Medline]
3. Definition of "sole contribution." N Engl J Med.1969; 281 :676 –677
4. Altman LK. The Ingelfinger rule, embargoes, and journal peer review: part 2. Lancet.1996; 347 :1459 –1463[CrossRef][Web of Science][Medline]
5. Altman LK. The Ingelfinger rule, embargoes, and journal peer review: part 1. Lancet.1996; 347 :1382 –1386[CrossRef][Web of Science][Medline]
6. Froom P, Froom J. Deficiencies in structured medical abstracts. J Clin Epidemiol.1993; 46 :591 –594[CrossRef][Web of Science][Medline]
7. Bhandari M, Devereaux PJ, Guyatt GH, et al. An observational study of orthopaedic abstracts and subsequent full-text publications.
J Bone Joint Surg Am.2002; 84-A
:615
–621
8. McCormick MC, Holmes JH. Publication of research presented at the pediatric meetings. Change in selection.
Am J Dis Child.1985; 139
:122
–126
9. Riordan FA. Do presenters to paediatric meetings get their work published?
Arch Dis Child.2000; 83
:524
–526
10. Roy D, Sankar V, Hughes JP, Jones A, Fenton JE. Publication rates of scientific papers presented at the Otorhinolarygological Research Society meetings. Clin Otolaryngol.2001; 26 :253 –256[CrossRef][Web of Science][Medline]
11. Scherer RW, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts. A meta-analysis.
JAMA.1994; 272
:158
–162
12. Stata Reference Manual, Release 7. Version 7.0 ed. College Station, TX: Stata Press; 2001
13. Saha S, Saint S, Christakis DA. Impact factor: a valid measure of journal quality? J Med Libr Assoc.2003; 91 :42 –46[Web of Science][Medline]
14. Relman AS. Peer review in scientific journals—what good is it? West J Med.1990; 153 :520 –522[Web of Science][Medline]
15. Weber EJ, Callaham ML, Wears RL, Barton C, Young G. Unpublished research from a medical specialty meeting: why investigators fail to publish.
JAMA.1998; 280
:257
–259
PEDIATRICS (ISSN 1098-4275). ©2003 by the American Academy of Pediatrics
CiteULike 
Connotea 
Del.icio.us 
Digg 
Facebook 
Reddit 
Technorati 
Twitter What's this?
This article has been cited by other articles:
|
|
 |
|
  D. Greenberg, O. Wacht, and J. S. Pliskin Peer Review in Publication: Factors Associated with the Full-Length Publication of Studies Presented in Abstract Form at the Annual Meeting of the Society for Medical Decision Making Med Decis Making, November 1, 2008; 28(6): 938 - 942. [Abstract] [PDF]
|
|
|
|
 |
|
 S. G. Gandhi and W. M. Gilbert Society of Gynecologic Investigation: What Gets Published? Reproductive Sciences, December 1, 2004; 11(8): 562 - 565. [Abstract] [PDF]
|
|
|
|
 |
|
 I B Pless and F P Rivara Do world conferences live up to their promise? Inj. Prev., June 1, 2004; 10(3): 129 - 130. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
