Clinical Research Involving Children: Registration, Completeness, and Publication
Abstract
BACKGROUND AND OBJECTIVE: Effective health care for children must be based on thorough analyses of the best research evidence. The objective of this study was to examine registration, completeness, and publication of studies involving children.
METHODS: We searched the ClinicalTrials.gov registry to identify all closed studies involving children and examined them for completeness and availability of results. We examined publication in peer-reviewed journals for 160 randomly selected National Institutes of Health (NIH)–funded studies from 2000 through 2010 and for 758 randomly selected completed studies.
RESULTS: Of 3428 closed studies involving children identified in ClinicalTrials.gov, 2385 (70%) were completed, 28 (0.8%) suspended, 152 (4.4%) terminated, and 38 (1.1%) withdrawn. The proportion of non-completed studies (terminated and suspended) increased linearly by 186% between 2001 and 2009, from 1.9% to 8.4%. Of the 152 terminated studies, 48 did not report reasons for termination, 21 cited safety concerns, and 83 cited poor recruitment or other administrative reasons. Only 29% of completed studies were published. Publication that did occur was an average of 2 years after study completion. Completed interventional studies were published more often than observational studies. Completed industry-funded studies were published less often than studies funded by the NIH. Registered NIH-funded trials were published more often than unregistered.
CONCLUSIONS: Results are unavailable for more than half of the studies involving children, revealing a substantial publication bias. Registration and posting of results on ClinicalTrials.gov should be mandatory for all studies involving children.
- clinical research involving children
- databases as topic
- human experimentation/standards
- publication bias
- child advocacy
- bioethics
- CI —
- confidence interval
- FDA —
- Food and Drug Administration
- FDAAA —
- Food and Drug Administration Amendments Act of 2007
- NCT —
- National Clinical Trial number
- NIH —
- National Institutes of Health
- NLM —
- National Library of Medicine
- OR —
- odds ratio
- RePORT —
- Research Portfolio Online Reporting Tools
What’s Known on This Subject:
Existing clinical research policy does not guarantee availability of results. Registration on the Web site ClinicalTrials.gov and the Food and Drug Administration Amendments Act improved transparency in pediatric clinical research. Registration and publication remain voluntary for many trials involving children.
What This Study Adds:
Only 29% of completed registered studies and 53% of National Institutes of Health–funded trials involving children were published. Numbers of studies are increasing. Registration and posting of results on ClinicalTrials.gov should be mandatory for all studies involving children.
Effective health care for children requires thorough analysis of the best research evidence.1 Evidence-based health care decisions rely on investigators reporting the results of all studies, including those not completed or completed but not published.2 Lack of comprehensive information threatens the transparency and integrity of research.3
Publication bias (ie, when publication decisions are based on the direction and significance of the results) is a serious concern.4 Studies showing a positive significant benefit from examined treatments are more likely to be published. Publication bias can distort the basis of treatment decisions.5–7
Several regulations have helped to improve transparency and public oversight of clinical studies involving children. In 2000, the National Institutes of Health (NIH) requested (but did not mandate) that clinical trials assessing pharmacological treatments for serious or life-threatening diseases be registered in the online database ClinicalTrials.gov, a Web site established by the National Library of Medicine.8 Since 2000, 26 870 studies involving children have been registered on ClinicalTrials.gov, providing detailed information about interventions, outcomes, and sponsors.9–11
In 2005, the International Committee of Medical Journal Editors required registration of all clinical studies as a condition of publication.12 However, registration and publication of studies remain voluntary. Although policies vary among pediatric journals that publish trials, Pediatrics does require that trials be registered as a condition of publication.13
The Food and Drug Administration (FDA) Amendments Act (FDAAA) of 2007 requires many studies to provide ClinicalTrials.gov with the study flow, baseline subject characteristics, and outcomes after active and control interventions within 1 year of study completion (phase II–IV interventional studies; studies involving FDA-regulated drugs; biological products or medical devices; studies having at least 1 site in the United States or conducted under an investigational new drug application or investigational device exemption; and studies initiated or ongoing as of September 27, 2007, or later).14–19 Published analyses of compliance with registration and reporting have demonstrated substantial bias in selecting and publishing outcomes.20,21 Results of only 7.5% of phase II through IV clinical trials involving children were posted on ClinicalTrials.gov.22 Registration, completeness, publication, and posting of results from studies involving children have not been previously examined. We investigated registration rates for NIH-funded studies involving children; rates of completion, termination, or suspension of registered studies involving children; and how often investigators whose research involved children published their results in peer-reviewed journals or posted results on ClinicaTrials.gov.
Methods
We searched the Research Portfolio Online Reporting Tools (RePORT) grant database (available at http://projectreporter.nih.gov/reporter.cfm) on August 16, 2010, to find NIH-funded studies involving children. We used key words “child” AND “trial” or “randomized controlled clinical trial” in the years 2000 through 2010 to find all closed studies involving children sponsored by the NIH during this time period. This database does not provide search options using medical subject headings nor does it identify registration or recruitment status. To explore potential bias, we examined registration rate differences by funding institutes within the NIH.
We randomly selected 160 of all funded trials to ascertain whether they were (1) registered on ClinicalTrials.gov and (2) published in peer-reviewed journals indexed in Medline. Random selection was conducted by using the unique grant number. We estimated a need for 80 studies to detect statistically significant differences at a 95% confidence interval (CI) with statistical power of 80%, assuming a 25% vs 10% rate difference in registration of studies funded by different institutions. We excluded instances of continued funding of the same project titles.
To identify registration status of the NIH-funded studies, we used the grant number of each study to search ClinicalTrials.gov. To identify peer-reviewed publication of NIH-funded trials, we used the grant number of each study to search PubMed. We separately determined the proportion of published studies to test the hypothesis that registered studies were more likely to be published. We also analyzed time intervals between each study’s end date and first publication date.
To evaluate the completeness of studies registered in ClinicalTrials.gov, we searched that database for all closed studies (exact string is “Closed Studies | Child”) involving children exclusively. We used the exact data provided by sponsors, relying on quality assurance conducted by registry personnel (Table 1).19 We used theClinicalTrials.gov classification scheme for interventions to look for differences in reporting rates.
Definitions of Data Elements Available for Downloading From ClinicalTrials.gov
To evaluate peer-reviewed publication of registered studies involving children, we randomly selected and examined 30% of the completed studies. We conducted random selection by using the unique National Clinical Trial (NCT) number (SAS codes are available from the authors by request). We reviewed the publication status of all terminated and suspended studies from ClinicalTrials.gov, assessed the reasons for suspension or termination provided on ClinicalTrials.gov, and categorized them according to whether they were “safety related” (defined as significant harms or lack of efficacy)23 or “other” (eg, poor recruitment, economic, or organizational reasons).
We tested the association among availability of results from registered studies with participant characteristics, study methodology, and funding source. Results of registered studies were available for the public when sponsors posted outcomes on ClinicalTrials.gov or published them in peer-reviewed journals. We compared proportions of (1) completed and noncompleted studies, (2) studies with and without results posted on ClinicalTrials.gov, and (3) studies published or not in peer-reviewed journals. We compared the proportions by using χ2 tests and Fisher exact tests in cases of small numbers.
We examined time intervals between study completion or termination and posting of results on ClinicalTrials.gov, and between study completion and publication in peer-reviewed journals.
We used logistic regression and calculated odds ratios (ORs) with 95% CIs for all comparisons. All calculations were performed with the frequency procedure by using SAS 9.1 software (SAS Institute Inc, Cary, NC) at 95% CIs with two-sided P values.24
Results
Registration of NIH-Sponsored Studies Involving Children
We retrieved 1571 studies from the RePORT grant database. We randomly selected 160 unique pediatric studies after deleting duplicate records of continuous grants (Supplemental Fig 2). Of the 160 studies in our sample, only 52 (33%) were registered on ClinicalTrials.gov. The RePORT database provides insufficient detail about study designs and protocols to determine whether a study was subject to the FDAAA requirements for registration and posting of results. However, we searched specifically for trials described as randomized controlled clinical trials, for which registration should have been a publication requirement.
Odds of registration varied across sponsoring agencies (Supplemental Table 4). Studies sponsored by the National Heart, Lung and Blood Institute were registered more often than studies funded by other NIH institutes.
Registered Studies Involving Children
We retrieved 57 299 records of closed studies from ClinicalTrials.gov, eliminating 66 because of misplaced fields. From the remaining 57 233, we identified 3428 (6%) studies involving exclusively children for the analysis.
Completeness of the Registered Studies Involving Children
The majority of the studies were completed (Table 2). Suspended, terminated, or withdrawn studies (28, 152, and 38, respectively) constituted 6.4% of all closed studies involving children. The proportion of noncompleted studies (terminated and suspended) increased linearly by 186% between 2001 and 2009, from 1.9% to 8.4% (R2 for linear trend 84%).
Studies Involving Children by Recruitment Status and Posting of the Results on ClinicalTrials.gov
Terminated Studies
Odds of termination varied across study categories (Supplemental Table 5). Studies that examined medical devices were terminated more often than those that examined biological agents (OR of termination, 4.5; 95% CI: 1.1–19.3). Studies that examined drugs were terminated more often than those that examined disease management (OR: 6.6; 95% CI: 2.1–21.0) or biological agents (OR: 3.9 95% CI: 2.0–7.6). Studies that examined procedures were terminated more often than those that examined biological agents (OR: 3.4; 95% CI: 1.4–8.7). Odds of termination did not differ between industry- and NIH-funded studies (OR: 0.9; 95% CI: 0.5–1.7).
Of 152 terminated studies, 48 (32%) did not report reasons for termination, 21 (14%) cited safety-related reasons, and 83 (55%) cited poor recruitment or other administrative reasons. Odds of termination due to safety did not differ between industry- and NIH-funded studies. Neither odds of termination due to safety nor odds of failing to report reasons for termination differed by funding source or intervention type. Safety-related reasons for termination varied across the studies (Supplemental Table 6). In many cases, study termination was initiated by institutional review boards or by data safety monitoring boards. Terminated studies lasted 2 years on average. Results of only 7 terminated studies were posted, and 3 of those cited safety as the reason for termination.
Suspended Studies
Of the 28 suspended studies, 7 (25%) did not report reasons for suspension, 7 (25%) reported safety reasons, 11 (39%) reported other reasons, and 3 (11%) changed their status to active. Safety-related reasons for suspension varied across the studies (Supplemental Table 4). For instance, studies could be suspended for lack of approval from institutional review boards (NCT00176956; NCT00577226), an FDA hold (NCT00589953), or decisions from other regulatory agencies (NCT00657748). Suspended studies lasted on average 2.8 years, and none of their results were posted on ClinicalTrials.gov.
Posting of the Results From the Registered Studies Involving Children
The results from the majority of studies involving children were not posted on ClinicalTrials.gov (3249 or 95%; Table 2). The results of industry-funded studies were posted more often than NIH-funded studies (OR: 13.9; 95% CI: 5.1–37.6; Supplemental Table 7). The results of interventional studies were posted more often than observational studies (OR: of posting 3.6; 95% CI: 1.8–7.4). The results of phase III clinical trials were posted more often than phase II (OR: 2.6; 95% CI: 1.6–4.1) or phase IV clinical trials (OR: 1.6; 95% CI: 1–2.4). The results of studies that examined biological agents were posted more often than studies of medical devices (OR: 7.6; 95% CI: 1.8–31.6), drugs (OR: 2.0; 95% CI: 1.4–2.8), or all other interventions combined (OR: 8.1; 95% CI: 4.4–14.8; Table 3). The results of drug studies were posted more often than studies of all other interventions combined (OR: 4; 95% CI: 2.2–7.3).
Publication of the Completed Studies Involving Children
Time intervals between completion or termination dates and posting of results averaged 1.7 years (median, 1.1 years; SD, 1.6 years). Time intervals varied across sponsors; 7 of 41 sponsors posted results at 2.0 to 2.9 years, 4 sponsors at 3.0 to 3.9 years, and 1 sponsor at 4.3 years after study completion or termination. Intervals between completion and posting were longer for studies of biological agents than for drug studies (mean difference in time interval, 0.6; 95% CI: 0.2–1.1 years, P = .02). Time intervals from completion to posting averaged 1.8 years for studies closed ≥1 year at the time of our examination (SD, 1.6 years). We separately analyzed the time intervals from closing to posting of results for studies that closed >1 year before our examination to account for any studies not yet required to have posted results under the FDAAA of 2007.
Publication of Results From Studies Involving Children
Publication of NIH-Funded Studies
It proved difficult to identify publications. Most published articles included several grant numbers, and each grant project included links to several articles. Published article titles and abstracts differed from descriptions of the grants. We used the names of the principal investigators to determine which grant numbers most closely reflected the results in specific published articles. Funded studies may last several years or be renewed; thus publication time varied depending on the area of research.
In our search in Medline using grant numbers, only 85 (53%) of the randomly selected NIH-sponsored studies were published. Registered trials were published more often than unregistered trials (Fig 1). Odds of publication did not differ significantly by grant type or funding agency (Supplemental Table 8).
ORs of publication in peer-reviewed journals by study characteristics.
Time intervals between project end dates and first publication averaged 1.6 years (SD 1.7 years, median 1.6 years). Time intervals did not differ by registration status of the grants or by sponsoring agencies.
Publication of Completed Registered Studies
We examined the peer-reviewed publication of 758 (32%) studies randomly selected from 2385 completed studies involving children. Only 29% of completed registered studies were published in peer-reviewed journals indexed in Medline (Supplemental Table 9). The odds of study publication significantly differed by sponsor (industry- versus NIH-funded studies), by study type, and by intervention type (Fig 1). The results of industry-sponsored studies were published less often than those funded by the NIH. Results of interventional studies were published more often than those from observational studies. Results from randomized controlled clinical trials were published more often than results from nonrandomized studies. Results from studies examining drugs were published less often than those of medical devices.
Publication rates differed most among individual sponsors (Table 3). Of 117 sponsors that funded >1 study, 65 did not publish results. Of 17 sponsors that funded >5 studies, 3 did not publish results. Only 5 sponsors published results of more than half of their studies.
Publication of Incomplete Registered Studies
The results from terminated and suspended studies on ClinicalTrials.gov were rarely published. Results from only 6% of terminated (9/152) and none of the suspended studies were published. Time intervals between termination dates and posting of results averaged 1.6 years (median, 1.2 years; SD, 0.9 years). Time intervals did not differ by examined study characteristics or reasons for termination. However, time intervals significantly differed across sponsors.
Of 9 terminated studies for which results were published, 4 mentioned that the study was terminated,25–28 and 5 omitted this information.29–33 The 2 published studies terminated for safety reasons both provided detailed information about exact harms. The ClinicalTrials.gov listings for those studies did not provide actionable information about exact rates of harms.
Discussion
Out findings demonstrate that results from a significant number of completed and noncompleted studies are not available for comprehensive evidence analysis. Complete public availability of information about trials could be ensured by consistent registration and posting of results in ClinicalTrials.gov of all clinical trials involving children regardless of funding, completion status, or previous FDA approval.3
All federally funded trials should be registered in ClinicalTrials.gov. Although mandatory registration of applicable clinical trials involving children would improve transparency in research, less than half of federally funded studies involving children were registered on ClinicalTrials.gov.
The RePORT grant database allows for analysis of registration and publication status of all federally funded studies, published or not, thereby facilitating assessment of publication bias. Unfortunately, the RePORT database does not include a specific variable for registration status of funded grants, necessitating time-consuming manual extraction of this information. In addition, the current practice of posting all publications that mention a grant obfuscates attempts to determine a study’s publication status. Instead, registered studies should be published with trial identification numbers and links to detailed protocol descriptions and precise information about study results. NIH-funded studies involving children should be routinely monitored for registration and posting of results on the ClinicalTrials.gov. Industry-funded clinical trials should be required to register before seeking consent for or enrollment of children in the trial.
Reporting of noncompleted studies is especially important if they were terminated because of detected harms from the treatments. Although the literature includes extensive discussions of ethics and regulations related to research involving children,34–55 completeness and reporting of results have been neglected. Studies involving children should always post reasons for termination or suspension on ClinicalTrials.gov. Many of the terminated and completed studies we examined lasted more than 2 years, during which time children experienced important outcomes, including harms. Those outcomes should be available for analysis.
Poor recruitment often led to termination of studies involving children. Possible reasons for poor recruitment, and strategies to enhance children’s participation in clinical research, have been discussed in the literature.56–64 Public confidence in clinical research is integral to improving recruitment and participation in research involving children.65 Better regulation and oversight of research transparency, including consistent posting of results from all clinical research, may help improve public trust.
Publication bias presents a serious problem in children’s clinical research. Our findings are comparable to previously published analyses of completed trials involving children, which show a 42% publication rate.21 Such a substantial publication bias may threaten the validity of research summaries. Publication bias is endemic, and, along with lack of access to research results, it is evident across studies of all age groups.5–7
Posting study results in ClinicalTrials.gov provides access to research findings not otherwise available to the public,19 but the credibility of the posted protocols and findings depends on the commitment of investigators to submit complete and accurate data.19 The results of only 9% of all completed studies and 7.5% of phase II through IV clinical trials involving children were posted on ClinicalTrials.gov.22 Only 24% of the studies that posted results were also published. Previous analyses of all research across age categories have estimated a 30% compliance rate with the FDAA requirements for posting results and a 25% to 52% publication rate for studies that post results.19 Thus, publications alone provided findings from only 16% (0.3 * 0.52 = 0.156) of the studies for which the FDAA requirements apply. Policy should obligate principal investigators of all clinical trials involving children to post results on ClinicalTrials.gov.
Our study has several limitations. We did not analyze the number of suspended or terminated unregistered studies involving children. The RePORT grant database includes no variables indicating recruitment status of the studies. We did not examine studies registered in the World Health Organization clinical trials database; we assumed all studies funded by the US government should be registered on ClinicalTrials.gov. Deviations from protocols and selective reporting of outcomes in published studies were beyond our scope. We sampled completed studies involving children and did not analyze publication of all completed studies. Potential implications of our limitations include lack of causal association between study results and publication, and investigators’ true motivations to publish the results. Nevertheless, our preliminary analysis is the first to reveal low registration and publication rates and a growing number of noncompleted studies involving children.
We conclude that results are not available from the majority of studies involving children, revealing a substantial publication bias. Registration and posting of the results on ClinicalTrials.gov should be mandatory for all studies involving children.
Footnotes
- Accepted December 14, 2011.
- Address correspondence to Tatyana Shamliyan, MD, Division of Health Policy and Management, School of Public Health, University of Minnesota, MMC 729, 420 Delaware St SE, Minneapolis MN 55455. E-mail: shaml005{at}umn.edu
Dr Shamliyan conceptualized and designed the project, retrieved and analyzed the data, wrote the report and the manuscript, and she is responsible for integrity of the data; Dr Kane conceptualized the project, revised the manuscript, and approved the submission.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
FUNDING: The study was supported by University of Minnesota Evidence-based Practice Center.
COMPANION PAPERS: Companions to this article can be found on pages e1301 and e1320, online at www.pediatrics.org/cgi/doi/10.1542/peds.2011-3285 and www.pediatrics.org/cgi/doi/10.1542/peds.2012-0621.
References
- Copyright © 2012 by the American Academy of Pediatrics
