Tools and Links

Pediatrics
June 2012, VOLUME 129 / ISSUE Supplement 3
SUPPLEMENT ARTICLE

Standard 2: Containing Risk of Bias

Lisa Hartling, Michele Hamm, Terry Klassen, An-Wen Chan, Martin Meremikwu, Virginia Moyer, Shannon Scott, David Moher, Martin Offringa
Download PDF
  • Abbreviations:
    CONSORT —
    Consolidated Standards of Reporting Trials
    ITT —
    intention to treat
    SPIRIT —
    Standard Protocol Items: Recommendations for Interventional Trials
    StaR Child Health —
    Standards for Research in Child Health

    • Dilemma

    There is a crisis of credibility facing the child health research community because of the paucity of reliable estimates of the effects of interventions in children. Associations between risk of bias assessments and treatment effect estimates have important implications, for the clinician, and the families face important challenges as decision-makers stemming from results that exaggerate treatment effectiveness or safety. Consequently, interventions that are not efficacious and potentially harmful may be prescribed, whereas interventions that truly are efficacious may be withheld.1–5

    Positive trends in pediatric research have been observed since the first trial was published in 1948. Specifically, there has been a substantial increase in the number of trials published over time, the proportion of randomized to nonrandomized controlled trials, and the proportion of child to adult trials.6 Reporting of methods has also improved; however, methodological quality remains modest.6

    Three studies have specifically examined risk of bias in pediatric trials by using the Cochrane Risk of Bias tool.7–9 The results are summarized in Table 1 by risk of bias domain. In 2 reviews, the overall risk of bias was unclear or high for the vast majority of trials.7,8 Both of these articles also revealed that trials at high or unclear risk of bias had exaggerated treatment effects compared with those at low risk of bias. Sequence generation and allocation concealment appear to be the domains that are consistently problematic. Importantly, several variables have been found to be associated with risk of bias including source of funding (industry-sponsored research revealing higher risk of bias), nature of the interventions (behavioral/educational interventions having higher risk of bias), and number of authors (higher risk of bias with fewer authors).9

    View this table:
    TABLE 1

    Summary of Reviews Documenting Risk of Bias by Domain Using Cochrane Collaboration Risk of Bias Tool

    These analyses demonstrate that there is substantial room for improvement in the methodological and reporting quality of pediatric trials. The association between risk of bias assessments and treatment effect estimates has important implications for decision-making both in terms of false-positive and false-negative results. In practice, this may result in unrealistic expectations of treatment benefit and safety. Inadequate reporting can also have an impact on systematic reviews, which are being used increasingly as a basis for informed decision-making. Although systematic reviews aim to be comprehensive and include all relevant studies, the number of studies contributing to various analyses, be they qualitative or quantitative, is often considerably smaller due to inadequate reporting.

    Ensuring methodological rigor and complete reporting is essential for informed clinical decision-making. It is critical that issues of bias are recognized and addressed by every person who conducts trials, reviews trials, funds trials, or uses trials to guide practice.

    This standards article was motivated by 2 key factors: (1) emerging evidence demonstrating methodological flaws and weaknesses in child health research and (2) a growing base of empirical evidence quantifying the association between design features in randomized trials and estimates of treatment effects. The following sections address the domains of bias including (1) sequence generation and allocation concealment, (2) blinding, (3) missing outcome data, (4) selective outcome reporting, and (5) other sources of bias.

    Distorted Interpretation of Treatment Effects

    The types of bias that may occur in randomized controlled trials can generally be classified as selection, performance, detection, attrition, and reporting bias.10Figure 1 illustrates the progression of a trial and where bias may occur. The biases in a given trial can have varying impact on the magnitude and direction of the treatment effect estimates. A growing body of literature provides empirical evidence quantifying the association between specific methodological features of randomized trials and treatment effect estimates. This evidence forms the basis of the Cochrane Risk of Bias Tool, which assesses potential risk of bias in randomized trials based on 6 domains. Table 2 shows the relationship between these domains and the different types of bias. The Cochrane Handbook provides an exhaustive review of the empirical evidence.10 The following is a summary of this evidence.

    FIGURE 1
    FIGURE 1

    Flow diagram illustrating where bias can occur during the progression of a trial.

    View this table:
    TABLE 2

    A Classification Scheme for Bias (Based on Table 8.4.1 in Cochrane Handbook for Systematic Reviews of Interventions10)

    1. Sequence Generation and Allocation Concealment

    Appropriate methods for generating the randomization sequence and concealing the allocation sequence are essential to minimize selection bias. The goal of randomization is to create study groups that are balanced with respect to both known and unknown confounders, whereas allocation concealment ensures that the randomization sequence is unknown to the participants and the person enrolling participants into a trial until allocation to a study group has occurred. On average, inadequate sequence generation results in overestimation of treatment effects by 12%,11–15 whereas inadequate allocation concealment can result in an overestimate of treatment effects by 18%.11–14,16–18 The degree of bias may vary based on the nature of the outcome (eg, less effect for all-cause mortality).19,20

    2. Blinding

    Blinding has long been considered a methodological characteristic of importance.21 Blinding of key individuals in a trial (ie, study participants, study personnel, and outcome assessors) can minimize performance and detection bias. Studies not described as double-blind reveal a 9% overestimate in treatment effect.11–14,16–18 Other studies, however, have revealed no significant association between blinding and effect estimates.16,20 One reason for the discrepancy may be variations in the definition of blinding used in different studies. Experts now maintain that it is important to consider who is blinded in a trial22 and the consequences of inadequate blinding.23

    3. Missing Outcome Data

    The effects of missing outcome data and how missing data are managed have been investigated in a number of studies. Some suggest more favorable treatment estimates from per protocol analyses*,24–26 or “modified” intention-to-treat† (ITT)27 analyses compared with true ITT analyses. However, other studies have provided no evidence of an association between missing outcome data and effect estimates.11,13,14,18 A meta-epidemiologic study revealed no significant difference in effect sizes overall for studies with adequate (ie, ITT) versus inadequate or unclear approaches to analysis, but results varied across meta-analyses according to the degree of between-trial heterogeneity.28

    4. Selective Outcome Reporting

    Selective outcome reporting occurs within published studies and is defined as “the selection of a subset of the original variables recorded for inclusion in publication of trials.”29 The most apparent source of bias is when outcomes measured in a trial are deliberately not reported based on their statistical significance; however, other sources of selective outcome reporting exist, such as how the outcome is analyzed, how and when the outcome is measured, and reporting of different subsets of data or subgroups.28–30 A recent systematic review summarized 5 studies that followed inception cohorts from protocol to full publication to examine selective reporting of outcomes.30–35 Four studies “that examined the association between outcome reporting bias and statistical significance found that statistically significant outcomes were more likely to be completely reported than non-significant outcomes.”30 The studies also revealed discrepancies in the primary outcomes proposed and those reported. Other studies have revealed discrepancies in statistical methods reported in protocols and subsequent publications (eg, planned versus reported subgroup analyses).36,37 A study examining the impact of selective outcome reporting on the results of meta-analysis revealed that approximately half of the trials identified as relevant to a systematic review did not contribute to the meta-analysis of patient-important outcomes.38 Further, the effect estimates decreased as the proportion of relevant studies contributing to the meta-analysis increased. Other research has investigated discrepancies due to unpublished versus published scales39 and handling of baseline and end-point data.40

    5. Other Sources of Bias

    The final domain within the Cochrane tool includes an assortment of study characteristics that may lead to biased results, including factors associated with specific designs (eg, cross-over trials, cluster trials), blocked randomization in unblinded trials, and baseline imbalances.41,42 Sample size is not included in the tool; however, some evidence suggests “small study bias,” whereby trials with few participants may be associated with exaggerated effect estimates.13,43 This variable may be relevant within child health research given the preponderance of trials with small samples.44,45

    6. Source of Funding

    An extensive body of evidence reveals that published research that is industry-sponsored is more likely to have results or conclusions favoring the sponsor.46–48

    Guidance

    The following is a brief overview of methodological features that pediatric trialists should consider when designing, conducting, and reporting their research (Table 3). A mnemonic such as presented in Table 4 may also be helpful when designing and reporting a trial.

    View this table:
    TABLE 3

    Recommendations for Practice

    View this table:
    TABLE 4

    Mind the Gap: A Mnemonic for Designing and Reporting a Randomized Trial

    • Appropriate randomization ensures that each study participant has the same probability of being assigned to the respective study groups. The most commonly used tools to generate a randomization sequence are readily available computer programs (eg, Microsoft Excel) or random numbers tables. Any method that does not ensure the same probability is not random, such as assigning patients by odd or even numbers, timing of presentation (eg, day of the week), or based on clinician judgment, patient preference, test results, or treatment availability. For stratified or blocked randomization, trialists should consult specialized sources or statisticians.49

    • Allocation concealment ensures that the randomization sequence is not known to the participants or those enrolling participants until assignment to study groups has occurred. Allocation concealment is possible in all trials, even those that are not blinded. Moreover, allocation concealment is often mistaken for blinding in trials. A key distinction is that allocation concealment occurs until the point of assignment to study groups, whereas blinding occurs following the point of assignment and for the duration of the trial. The most commonly used methods for allocation concealment are sequentially numbered, opaque, sealed envelopes; sequentially numbered drug containers that are identical in appearance; or central allocation, wherein the individual enrolling the participant consults a central source (eg, pharmacy) for treatment assignment once the patient has consented to participate. A key point is that allocation concealment is not possible if inappropriate methods, such as alternation, have been used to generate the randomization sequence.

    • Blinding of key study personnel, patients, and outcome assessors should be considered. One common fallacy is that blinding is not applicable when the nature of the intervention precludes blinding of study personnel or patients (eg, educational intervention). In such cases, trialists should have independent, blind outcome assessors and use objective outcomes as well as reliable, valid measurement tools.

    • The outcome analysis plan should be prespecified, including specification of the following: the primary outcome and how it will be assessed, including the statistical tests to be used; subgroup or adjusted analyses, including variables of importance and how and when subgroup and adjusted analyses will be performed; interim analyses, including when and how they will be performed, what decisions will be made based on these, and how these will be made. Further, before beginning a study, trialists should conduct sample size calculations based on their primary outcome of interest with justification for the parameters used.

    • Numbers and reasons for withdrawal or loss to follow-up by study group should be tracked. Trialists should specify a priori how they will handle missing data in the analysis. Current evidence supports ITT analyses as less prone to bias than per protocol analyses. Sensitivity analyses using best case and worst case scenarios can add to the interpretation and robustness of the study findings.

    • All outcomes should be detailed (including benefits and harms), including how and when they will be assessed (eg, which tools, measurement time points) at the outset of the trial. In addition, primary versus secondary outcomes should be specified a priori. All prespecified outcomes should be reported on in the final report. Justification should be provided for changes in outcomes between protocol and publication.

    • Trialists should declare any financial support received to conduct the trial as well as the role of the sponsor in the design, conduct, analysis, or reporting. Agreements should be formalized at the outset to ensure independence of the researchers to analyze and report the study findings.

    • Specialized resources should be consulted for issues related to specific trial designs, such as cross-over trials and cluster randomized trials.

    • Treatment of baseline imbalances should be prespecified. Imbalances can occur despite appropriate randomization. Analyses accounting for baseline imbalances should be reported.

    There are several other items that trialists should consider when planning and reporting a trial.

    • Trial registration. The World Health Organization and medical journal editors worldwide have endorsed prospective registration of trials, and many journals will not publish trials if they have not been publicly registered before recruitment.50 Trial registration enhances the transparency of research. Specifically it ensures accountability in terms of what is planned and done and what is reported. Trial registration can also provide a source for identifying all research in a given area (particularly unpublished and ongoing research), raise awareness of ongoing initiatives, avoid duplication, foster collaboration (eg, to increase potential recruitment sites), and identify gaps in research. The International Committee of Medical Journal Editors has a list of approved registries.51

    • Trial protocols. The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) initiative provides guidance regarding items to consider and report when developing a trial protocol.52 The SPIRIT initiative stemmed from empirical evidence demonstrating major discrepancies between trial protocols and publications, including “data suppression, misrepresentation, and manipulation.”53 SPIRIT offers a checklist to improve the quality of protocols and interpretation of study results.54 Closely tied to this initiative is growing interest and a forum for the publication of trial protocols with the intent of providing a permanent, public record of a trial and reduce the potential impact of publication bias.55,56 The journal Trials has a specific remit to publish trial protocols.

    • Reporting guidelines. The Consolidated Standards of Reporting Trials (CONSORT) Statement provides guidance on reporting a randomized trial.57 It is recommended that trialists review the reporting guidelines even at the protocol development stage to identify important elements: once the trial is complete, clear reporting cannot eliminate bias due to deficient study design. For designs other than the 2-group parallel design, trialists should consult the CONSORT Web site.58 There is increasing evidence that use of reporting guidelines is associated with improved quality.59 Moreover, transparent and accurate reporting should be considered a moral and ethical responsibility.

    Research Agenda

    Three key areas for additional research include (1) quantitative research to improve the evidence related specifically to pediatric research methodology, (2) qualitative research to understand barriers to high methodological quality in pediatric trials, and (3) knowledge translation efforts to reduce the knowledge to practice gap in pediatric trial methodology.

    First, the majority of research quantifying the association between methodological characteristics and treatment effect estimates has been based on adult trials. There is growing evidence that interventions may work differently for children than adults due to physiologic and developmental differences and different disease pathophysiology.60–62 Design features may lead to differences in estimates of effect, such as greater response among children to placebo.63,64 The finding of differences between children and adults in their response to treatment may extend to the influence of bias on estimates of treatment effectiveness. Further, there are areas where the empirical evidence is weak or inconsistent, such as the impact of missing outcome data.

    Second, there is a need to identify reasons for the suboptimal methodological quality of pediatric trials and to understand the challenges faced by pediatric trialists when conducting research. We have recently completed a survey of a sample of international trialists to determine the following: (1) researcher knowledge and awareness of bias; (2) perceived barriers and facilitators in conducting clinical trials; (3) awareness of existing methodological initiatives, and (4) the perceived utility of potential strategies to use in knowledge translation interventions. We will also assess how researchers’ beliefs and values related to working with children and their caregivers intersect with issues of study design. The findings of this work will help clarify and contextualize the discrepancy between methodological rigor and trial conduct.

    Third, there is a need to develop knowledge translation mechanisms to bridge the gap between what is known and how trials are conducted. Accordingly, StaR Child Health is committed to bridging this gap by not only raising awareness of appropriate trial design, conduct, and reporting but also committed to assisting with implementation of evidence-based standards and becoming a “global centre for resources and training relating to the design, conduct, and reporting of clinical research with children.”65

    Conclusions

    Studies have revealed the methodological shortcomings of pediatric trials. Meta-epidemiologic research provides evidence of the association between methodological features and exaggerated treatment effect estimates. Several tools exist to guide the conduct, design, and reporting of pediatric trials including the Cochrane Risk of Bias tool, the SPIRIT initiative, and the CONSORT Statement. Trialists need to adhere to sound methodological principles in designing and conducting their trials including appropriate sequence generation, adequate allocation concealment, blinding of key study personnel particularly outcome assessors, adequate follow-up and handling of missing outcome data, and reporting of all prespecified outcomes. Further research is needed to quantify the association between methodological characteristics and treatment effect estimates, identify barriers and facilitators to the implementation of sound methodological principles, and develop knowledge translation tools to ensure the effective dissemination and uptake of these principles in child health research.

    Footnotes

    • Address correspondence to Martin Offringa, MD, PhD, Senior Scientist and Program Head, Child Health Evaluative Sciences, Research Institute, The Hospital for Sick Children, 555 University Ave, Toronto, Ontario, Canada M5G 1X8. E-mail: martin.offringa{at}sickkids.ca

    • Dr Hartling, Ms Hamm, and Dr Klassen wrote the first draft of the article; Dr Chan, Dr Meremikwu, Dr Moyer, Dr Scott, Dr Moher, and Dr Offringa contributed to the writing of the article; Dr Hartling, Ms Hamm, Dr Klassen, and Dr Scott participated in regular conference calls, identified the issues, and drafted the article; Dr Hartling, Ms Hamm, Dr Klassen, Dr Moher, and Dr Offringa participated in identifying the evidence base for Standards for Research in (StaR) Child Health standards; and all authors agree with the final version.

    • Members of the Star Child Health Risk of Bias Standard Development Group include the above authors as well as Drs Jamie Brehaut, Jeremy Grimshaw, and Prathap Tharyan.

    • This is the second in a series of standard articles resulting from an ongoing process in which a group of invited experts called a standard development group from StaR Child Health assembles and exchanges information about methods for pediatric trial design, conduct, and reporting. More detailed information about this topic can be found in the introductory article of this supplement or at the StaR Child Health Web site (www.starchildhealth.org).

    • FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

    • ↵* Per protocol analysis refers to “an analysis of the subset of participants from a randomised controlled trial who complied with the protocol sufficiently to ensure that their data would be likely to exhibit the effect of treatment.” (The Cochrane Collaboration Glossary)

    • ↵† In an ITT analysis “All participants are included in the arm to which they were allocated, whether or not they received (or completed) the intervention given to that arm.” (The Cochrane Collaboration Glossary)

    References

    1. ↵
      1. Knapp JF,
      2. Simon SD,
      3. Sharma V
      . Quality of care for common pediatric respiratory illnesses in United States emergency departments: analysis of 2005 National Hospital Ambulatory Medical Care Survey Data. Pediatrics. 2008;122(6):1165–1170pmid:19047229
      OpenUrlAbstract/FREE Full Text
      1. Hampers LC
      . Practice variation with febrile infants: delight in disorder? Pediatrics. 2009;124(2):783–785pmid:19651591
      OpenUrlFREE Full Text
      1. Johnson DW,
      2. Craig W,
      3. Brant R,
      4. Mitton C,
      5. Svenson L,
      6. Klassen TP
      . A cluster randomized controlled trial comparing three methods of disseminating practice guidelines for children with croup [ISRCTN73394937]. [ISRCTN73394937] Implement Sci. 2006;1:10pmid:16722541
      OpenUrlCrossRefPubMed
      1. Freedman SB,
      2. Gouin S,
      3. Bhatt M,
      4. et al.,
      5. Pediatric Emergency Research Canada
      . Prospective assessment of practice pattern variations in the treatment of pediatric gastroenteritis. Pediatrics. 2011;127(2). Available at: www.pediatrics.org/cgi/content/full/127/2/e287pmid:21262881
      OpenUrlAbstract/FREE Full Text
    5. ↵
      1. Grol R,
      2. Grimshaw J
      . From best evidence to best practice: effective implementation of change in patients’ care. Lancet. 2003;362(9391):1225–1230pmid:14568747
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      1. Thomson D,
      2. Hartling L,
      3. Cohen E,
      4. Vandermeer B,
      5. Tjosvold L,
      6. Klassen TP
      . Controlled trials in children: quantity, methodological quality and descriptive characteristics of pediatric controlled trials published 1948-2006. PLoS ONE. 2010;5(9)pmid:20927344
      OpenUrlCrossRefPubMed
    7. ↵
      1. Hartling L,
      2. Ospina M,
      3. Liang Y,
      4. et al
      . Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ. 2009;339:b4012pmid:19841007
      OpenUrlPubMed
    8. ↵
      1. Hamm MP,
      2. Hartling L,
      3. Milne A,
      4. et al
      . A descriptive analysis of a representative sample of pediatric randomized controlled trials published in 2007. BMC Pediatr. 2010;10:96pmid:21176224
      OpenUrlCrossRefPubMed
    9. ↵
      1. Crocetti MT,
      2. Amin DD,
      3. Scherer R
      . Assessment of risk of bias among pediatric randomized controlled trials. Pediatrics. 2010;126(2):298–305pmid:20624806
      OpenUrlAbstract/FREE Full Text
    10. ↵
      Higgins JPT, Altman DG and Sterne JAC. Chapter 8: Assessing risk of bias in included studies. In Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available at: www.cochrane-handbook.org. Accessed April 28, 2012
    11. ↵
      1. Schulz KF,
      2. Chalmers I,
      3. Hayes RJ,
      4. Altman DG
      . Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–412pmid:7823387
      OpenUrlCrossRefPubMedWeb of Science
      1. Moher D,
      2. Pham B,
      3. Jones A,
      4. et al
      . Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet. 1998;352(9128):609–613pmid:9746022
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Kjaergard LL,
      2. Villumsen J,
      3. Gluud C
      . Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135(11):982–989pmid:11730399
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Siersma V,
      2. Als-Nielsen B,
      3. Chen W,
      4. Hilden J,
      5. Gluud LL,
      6. Gluud C
      . Multivariable modelling for meta-epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials. Stat Med. 2007;26(14):2745–2758pmid:17117373
      OpenUrlCrossRefPubMedWeb of Science
    15. ↵
      Als-Nielsen B, Gluud LL, Gluud C. Methodological quality and treatment effects in randomised trials: a review of six empirical studies. In: 12th Cochrane Colloquium; October 2–6, 2004; Ottawa, Ontario, Canada
    16. ↵
      1. Balk EM,
      2. Bonis PA,
      3. Moskowitz H,
      4. et al
      . Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA. 2002;287(22):2973–2982pmid:12052127
      OpenUrlCrossRefPubMedWeb of Science
      1. Egger M,
      2. Juni P,
      3. Bartlett C,
      4. Holenstein F,
      5. Sterne J
      . How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7(1):1–76pmid:12583822
      OpenUrlPubMed
    18. ↵
      1. Pildal J,
      2. Hróbjartsson A,
      3. Jørgensen KJ,
      4. Hilden J,
      5. Altman DG,
      6. Gøtzsche PC
      . Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007;36(4):847–857pmid:17517809
      OpenUrlAbstract/FREE Full Text
    19. ↵
      1. Wood L,
      2. Egger M,
      3. Gluud LL,
      4. et al
      . Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336(7644):601–605pmid:18316340
      OpenUrlAbstract/FREE Full Text
    20. ↵
      1. Savovic J
      . The association of three bias domains with treatment effect estimates in randomised control trials: combined analysis of meta-epidemiological studies. Z Evid Fortbild Qual Gesundhwes. 2008;102(suppl VI):29–30
      OpenUrl
    21. ↵
      1. Schulz KF,
      2. Chalmers I,
      3. Altman DG
      . The landscape and lexicon of blinding in randomized trials. Ann Intern Med. 2002;136(3):254–259pmid:11827510
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      1. Montori VM,
      2. Bhandari M,
      3. Devereaux PJ,
      4. Manns BJ,
      5. Ghali WA,
      6. Guyatt GH
      . In the dark: the reporting of blinding status in randomized controlled trials. J Clin Epidemiol. 2002;55(8):787–790pmid:12384193
      OpenUrlCrossRefPubMedWeb of Science
    23. ↵
      1. Sackett DL
      . Commentary: Measuring the success of blinding in RCTs: don’t, must, can’t or needn’t? Int J Epidemiol. 2007;36(3):664–665pmid:17675306
      OpenUrlFREE Full Text
    24. ↵
      1. Tierney JF,
      2. Stewart LA
      . Investigating patient exclusion bias in meta-analysis. Int J Epidemiol. 2005;34(1):79–87pmid:15561753
      OpenUrlAbstract/FREE Full Text
      1. Melander H,
      2. Ahlqvist-Rastad J,
      3. Meijer G,
      4. Beermann B
      . Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ. 2003;326(7400):1171–1173pmid:12775615
      OpenUrlAbstract/FREE Full Text
    26. ↵
      1. Porta N,
      2. Bonet C,
      3. Cobo E
      . Discordance between reported intention-to-treat and per protocol analyses. J Clin Epidemiol. 2007;60(7):663–669pmid:17573981
      OpenUrlCrossRefPubMedWeb of Science
    27. ↵
      1. Abraha I,
      2. Duca P,
      3. Montedori A.
      Empirical evidence of bias: modified intention to treat analysis of randomised trials affects estimates of intervention efficacy. Z Evid Fortbild Qual Gesundhwes. 2008;102(suppl VI):9
      OpenUrl
    28. ↵
      1. Hutton JL,
      2. Williamson PR
      . Bias in meta-analysis due to outcome variable selection within studies. Appl Stat. 2000;49:359–370
      OpenUrlCrossRef
    29. ↵
      1. Nuesch E,
      2. Trelle S,
      3. Reichenbach S,
      4. Rutjes A,
      5. Scherrer M,
      6. Burgi E.
      Empirical evidence of attrition bias in meta-analyses of randomized controlled trials. Z Evid Fortbild Qual Gesundhwes. 2000;102(suppl VI):9
      OpenUrl
    30. ↵
      1. Dwan K,
      2. Altman DG,
      3. Arnaiz JA,
      4. et al
      . Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE. 2008;3(8):e3081pmid:18769481
      OpenUrlCrossRefPubMed
      1. Hahn S,
      2. Williamson PR,
      3. Hutton JL
      . Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. J Eval Clin Pract. 2002;8(3):353–359pmid:12164983
      OpenUrlCrossRefPubMedWeb of Science
      1. Chan AW,
      2. Krleza-Jerić K,
      3. Schmid I,
      4. Altman DG
      . Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171(7):735–740pmid:15451835
      OpenUrlAbstract/FREE Full Text
      1. Chan AW,
      2. Hróbjartsson A,
      3. Haahr MT,
      4. Gøtzsche PC,
      5. Altman DG
      . Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–2465pmid:15161896
      OpenUrlCrossRefPubMedWeb of Science
      1. Chan AW,
      2. Altman DG
      . Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330(7494):753pmid:15681569
      OpenUrlAbstract/FREE Full Text
    35. ↵
      Von Elm E, Rollin A, Blumle A, Senessie C, Low N, Egger M. Selective reporting of outcomes of drug trials? Comparison of study protocols and published articles. In: XIV Cochrane Colloquium; October 23–26, 2006; Dublin, Ireland
    36. ↵
      1. Chan AW,
      2. Hróbjartsson A,
      3. Jørgensen KJ,
      4. Gøtzsche PC,
      5. Altman DG
      . Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. BMJ. 2008;337:a2299pmid:19056791
      OpenUrlPubMed
    37. ↵
      1. Al-Marzouki S,
      2. Roberts I,
      3. Evans S,
      4. Marshall T
      . Selective reporting in clinical trials: analysis of trial protocols accepted by The Lancet. Lancet. 2008;372(9634):201pmid:18640445
      OpenUrlPubMedWeb of Science
    38. ↵
      1. Furukawa TA,
      2. Watanabe N,
      3. Omori IM,
      4. Montori VM,
      5. Guyatt GH
      . Association between unreported outcomes and effect size estimates in Cochrane meta-analyses. JAMA. 2007;297(5):468–470pmid:17284696
      OpenUrlCrossRefPubMed
    39. ↵
      1. Marshall M,
      2. Lockwood A,
      3. Bradley C,
      4. Adams C,
      5. Joy C,
      6. Fenton M
      . Unpublished rating scales: a major source of bias in randomised controlled trials of treatments for schizophrenia. Br J Psychiatry. 2000;176:249–252pmid:10755072
      OpenUrlAbstract/FREE Full Text
    40. ↵
      1. Vickers AJ
      . The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: a simulation study. BMC Med Res Methodol. 2001;1(6):6pmid:11459516
      OpenUrlCrossRefPubMed
    41. ↵
      1. Schulz KF
      . Subverting randomization in controlled trials. JAMA. 1995;274(18):1456–1458pmid:7474192
      OpenUrlCrossRefPubMedWeb of Science
    42. ↵
      1. Schulz KF,
      2. Grimes DA
      . Generation of allocation sequences in randomised trials: chance, not choice. Lancet. 2002;359(9305):515–519pmid:11853818
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      1. Juni P,
      2. Nuesch E,
      3. Reichenbach S,
      4. Rutjes A,
      5. Scherrer M,
      6. Burgi E.
      Overestimation of treatment effects associated with small sample size in osteoarthritis research. Z Evid Fortbild Qual Gesundhwes. 2008;102(suppl VI):62
      OpenUrl
    44. ↵
      1. Caldwell PH,
      2. Murphy SB,
      3. Butow PN,
      4. Craig JC
      . Clinical trials in children. Lancet. 2004;364(9436):803–811pmid:15337409
      OpenUrlCrossRefPubMedWeb of Science
    45. ↵
      1. Campbell H,
      2. Surry SA,
      3. Royle EM
      . A review of randomised controlled trials published in Archives of Disease in Childhood from 1982-96. Arch Dis Child. 1998;79(2):192–197pmid:9797608
      OpenUrlFREE Full Text
    46. ↵
      1. Lexchin J,
      2. Bero LA,
      3. Djulbegovic B,
      4. Clark O
      . Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ. 2003;326(7400):1167–1170pmid:12775614
      OpenUrlAbstract/FREE Full Text
      1. Bekelman JE,
      2. Li Y,
      3. Gross CP
      . Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA. 2003;289(4):454–465pmid:12533125
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      1. Sismondo S
      . Pharmaceutical company funding and its consequences: a qualitative systematic review. Contemp Clin Trials. 2008;29(2):109–113pmid:17919992
      OpenUrlCrossRefPubMedWeb of Science
    49. ↵
      Friedman L, Furberg CD, DeMets D. Fundamentals of Clinical Trials. New York, New York: Springer Science+Business Media, LLC; 1998
    50. ↵
      1. DeAngelis CD,
      2. Drazen JM,
      3. Frizelle FA,
      4. et al.,
      5. International Committee of Medical Journal Editors
      . Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA. 2004;292(11):1363–1364pmid:15355936
      OpenUrlCrossRefPubMedWeb of Science
    51. ↵
      International Committee of Medical Journal Editors. Frequently asked questions about Clinical Trials Registration. Available at: www.icmje.org/faq_clinical.html. Accessed September 6, 2011
    52. ↵
      Equator Network. Reporting guidelines under development. Available at: www.equatro-network.org/resource-centre/library-of-health-research-reporting/reporting-guidelines-under-development/. Accessed September 6, 2011
    53. ↵
      1. Chan AW
      . Bias, spin, and misreporting: time for full access to trial protocols and results. PLoS Med. 2008;5(11):e230pmid:19067481
      OpenUrlCrossRefPubMed
    54. ↵
      SPIRIT Group. The SPIRIT Initiative: defining standard protocol items for randomized trials. Executive Summary (August 2010). Available at: www.equator-network.org/resource-centre/library-of-health-research-reporting/reporting-guidelines-under-development/. Accessed September 6, 2011
    55. ↵
      1. Altman DG,
      2. Furberg CD,
      3. Grimshaw JM,
      4. Rothwell PM
      . Lead editorial: trials - using the opportunities of electronic publishing to improve the reporting of randomised trials. Trials. 2006;7:6pmid:16556322
      OpenUrlCrossRefPubMed
    56. ↵
      BioMed Central Blog. SPIRIT: New guidance or protocol authors. Available at: http://blogs.openaccesscentral.com/blogs/bmcblog/entry/spirit_new_guidance_for_protocol. Published 2010. Accessed September 6, 2011
    57. ↵
      1. Moher D,
      2. Hopewell S,
      3. Schulz KF,
      4. et al.,
      5. Consolidated Standards of Reporting Trials Group
      . CONSORT 2010 Explanation and Elaboration: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1–e37pmid:20346624
      OpenUrlCrossRefPubMed
    58. ↵
      CONSORT. Transparent reporting of trials. Available at: www.consort-statement.org. Accessed September 6, 2011
    59. ↵
      Turner L. The influence of CONSORT on the quality of RCTS: an updated review. In: Canadian Cochrane Symposium; February 16–17, 2011; Vancouver, British Columbia, Canada
    60. ↵
      1. Klassen TP,
      2. Hartling L,
      3. Craig JC,
      4. Offringa M
      . Children are not just small adults: the urgent need for high-quality trial evidence in children. PLoS Med. 2008;5(8):e172
      OpenUrlCrossRefPubMed
      1. Cramer K,
      2. Wiebe N,
      3. Moyer V,
      4. et al
      . Children in reviews: methodological issues in child-relevant evidence syntheses. BMC Pediatr. 2005;5:38pmid:16176579
      OpenUrlCrossRefPubMed
    62. ↵
      1. Contopoulos-Ioannidis DG,
      2. Baltogianni MS,
      3. Ioannidis JP
      . Comparative effectiveness of medical interventions in adults versus children. J Pediatr. 2010;157(2):322–330
      OpenUrlCrossRefPubMedWeb of Science
    63. ↵
      1. Rheims S,
      2. Cucherat M,
      3. Arzimanoglou A,
      4. Ryvlin P
      . Greater response to placebo in children than in adults: a systematic review and meta-analysis in drug-resistant partial epilepsy. PLoS Med. 2008;5(8):e166pmid:18700812
      OpenUrlCrossRefPubMed
    64. ↵
      1. Fernandes R,
      2. Ferreira JJ,
      3. Sampaio C
      . The placebo response in studies of acute migraine. J Pediatr. 2008;152(4):527–533
      OpenUrlCrossRefPubMedWeb of Science
    65. ↵
      Standards for Research in Child Health. Available at: www.ifsrc.org/index.php?option=com_content&view=article&id=30&Itemid=2. Accessed April 28, 2012
    View Abstract

     

    View this article with LENS
    Email

    Alerts
    Citation Tools
    Standard 2: Containing Risk of Bias
    Lisa Hartling, Michele Hamm, Terry Klassen, An-Wen Chan, Martin Meremikwu, Virginia Moyer, Shannon Scott, David Moher, Martin Offringa
    Pediatrics Jun 2012, 129 (Supplement 3) S124-S131; DOI: 10.1542/peds.2012-0055E
    del.icio.us logo Digg logo Reddit logo Technorati logo Twitter logo CiteULike logo Connotea logo Facebook logo Google logo Mendeley logo
    Print
    PDF
    Insight Alerts

    Subjects

    Back to top