pediatrics
June 2012, VOLUME129 /ISSUE Supplement 3

Standard 3: Data Monitoring Committees

  1. Susan Ellenberg, PhDa,
  2. Ricardo M. Fernandes, MDb,
  3. Haroon Saloojee, MD, MBBCh, FCPaedc,
  4. Dirk Bassler, MD, MSCd,
  5. Lisa Askie, PhD, MPH, BN, NICC, RM, RNe,
  6. Ben Vandermeer, MScf,
  7. Martin Offringa, MD, PhDg,
  8. Ingeborg Van der Tweel, PhDh,
  9. Douglas G. Altman, DSci,
  10. Johanna H. van der Lee, MD, PhDj, and
  11. for the StaR Child Health Group
  1. aDepartment of Biostatistics and Epidemiology, University of Pennsylvania School of Medicine, Philadelphia, Pennsylvania;
  2. bDepartment of Pediatrics, Santa Maria Hospital, Laboratory of Clinical Pharmacology and Therapeutics, Faculty of Medicine, Instituto de Medicina Molecular, Lisboa, Portugal;
  3. cDepartment of Paediatrics and Child Health, University of the Witwatersrand, Johannesburg, South Africa;
  4. dCenter for Pediatric Clinical Studies and Department of Neonatology, University Children's Hospital, Tuebingen, Germany;
  5. eNHMRC Clinical Trials Centre, Sydney, Australia;
  6. fDepartment of Pediatrics, Alberta Research Centre for Health Evidence, University of Alberta, Edmonton, Alberta, Canada;
  7. gChild Health Evaluative Sciences, Research Institute, The Hospital for Sick Children, University of Toronto, Toronto, Ontario, Canada;
  8. hJulius Center for Health Sciences and Primary Care, University Medical Center, Utrecht, Netherlands;
  9. iCentre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom; and
  10. jDepartment of Pediatric Clinical Epidemiology, Emma Children’s Hospital, Academic Medical Centre, Amsterdam, Netherlands
  • data monitoring committee
  • DMC charter
  • DMC reporting
  • pediatric
  • StaR Child Health
  • Abbreviations:
    DMC
    data monitoring committee
  • Dilemma

    Children are generally regarded as “vulnerable individuals.” There has long been reluctance to perform research in children except in limited settings such as pediatric cancer and vaccine development. However, avoidance of trials in children has resulted in uncertainty regarding efficacy, safety, appropriate formulations, and dosages of treatments in pediatric populations, a situation now recognized as problematic.1 As more pediatric clinical trials are undertaken, it is timely to focus on special issues in their design and conduct. One such issue is the monitoring of accumulating data from ongoing clinical trials.

    Monitoring of safety outcomes during any trial is always required. In some trials, monitoring for efficacy is also an essential component of safety assurance, such as when efficacy is measured based on an unfavorable outcome or if a treatment is quickly discovered to be a major advance in terms of saving lives or preventing another serious outcome. In many cases, monitoring can be performed adequately by investigators and sponsors. However, for some trials, a group of independent experts, typically called a data monitoring committee (DMC) or data and safety monitoring board, takes responsibility for this function.

    DMCs have been a component of certain clinical trials beginning in the 1960s. Initially used in trials funded by government agencies in which the trial objective was to decrease mortality and/or reduce major morbidity, the DMC’s tasks were to carefully monitor the accruing results to ensure that continued enrollment remained appropriate, to consider whether modifications of trial conduct were needed, and to make recommendations to the researchers regarding continuation of the trial with or without changes.2 It was recognized that there were problems with having the investigators themselves perform this interim monitoring because their approach to recruitment and their adherence to the trial protocol could be influenced by their knowledge of interim data.3,4

    Clearly, individuals or organizations with financial interests that could be affected by the trial results would also not be optimal overseers of the emerging data. Thus, both investigators and individuals or organizations with financial interests would have a “conflict of interest” when monitoring a trial. The current understanding is that a DMC is a committee of experts who are independent of the trial and who can be expected to make recommendations regarding the conduct of the trial on the basis of emerging data, while minimizing unwanted influences on their judgment.5 The role of the DMC is complementary to trial sponsors (ie, the company or institution that initiates the trial and is ultimately responsible for it), institutional review boards, and trial steering committees.

    Despite substantial attention to the operational issues for DMCs over the last 20 years5,6 and general consensus on the basic principles of monitoring, there remains variation in how these committees operate,7 and there are special issues in pediatric trials that require particular consideration. The following sections of this standards article address principles and practices of DMCs with special attention to their application in pediatric confirmatory (Phase III) trials. Recommendations for practice are summarized in Table 1.

    TABLE 1

    Recommendations for Practice

    Guidance

    What Types of Trials Require DMCs?

    Not all randomized controlled trials require a DMC. As long as safety of participants and trial integrity can be adequately assured by existing mechanisms (eg, sponsor, investigators, data centers, institutional review boards, regulatory agencies), a DMC is not essential. The installation of an independent DMC adds a level of complexity to trials and imparts a burden that may compromise trial efficiency. Nevertheless, there are categories of trials for which a DMC is essential to ensuring the safety of participants and the integrity of the trial. Both clinical and methodologic criteria can be used to determine whether a DMC is needed for a particular trial (Table 2). Because children are a vulnerable population, it is appropriate to lean toward establishing the additional protection of a DMC in marginal cases.

    TABLE 2

    Trial Characteristics That Generally Necessitate a DMC in Pediatric Populations

    Clinical Criteria

    Pediatric randomized controlled trials addressing major morbidity or mortality end points should be monitored for these outcomes at regular intervals during the trial so that the potential benefits of the experimental intervention can be carefully balanced against potential risks. The installation of a DMC may be particularly important when the intervention is novel and previous safety data are limited, such as in the early trials on therapeutic cooling for encephalopathy.8,9 Even for an intervention not likely to influence mortality or serious morbidity, a trial conducted in a population at high risk of severe outcomes during the course of the trial may benefit from a DMC. Regular review of comparative data may be necessary to ensure that the intervention is not causing harm (eg, a trial of antiemetic therapy in children with cancer undergoing emetogenic chemotherapy). Even for pediatric trials, however, a large proportion may be monitored acceptably by investigators and sponsors. For example, trials not warranting DMCs include single-arm studies in which the emerging data will be known to all, so that confidentiality of interim data is not an issue, or trials of short-term treatment of a symptomatic condition in a population of essentially healthy children (even if randomized and controlled) so that serious adverse outcomes are not expected (trials of a topical treatment of rash or an antihistamine formulated for pediatric use). Trials that do not meet the criteria shown in Table 2 generally do not require an independent DMC.

    Methodologic Criteria

    Every study design that includes planned interim analyses of the accumulating efficacy data in the separate treatment groups should have a DMC. The role of a DMC would be to review the data and make recommendations about the continuation of the trial. Such trials typically meet the clinical criteria cited here. Other study factors that may indicate the need for a DMC are the trial size and the number of participating sites. In large trials, there might be more instances of rare but serious adverse events such that a DMC would need to compare rates in treatment arms to confirm that the benefit-to-risk ratio of the trial remains acceptable. Furthermore, in multicenter trials, it may be necessary to monitor differences in conduct and outcomes to ensure that all sites are conducting study procedures in a similar manner.

    Who Should Serve on a DMC?

    A DMC typically comprises 1 or more clinicians knowledgeable in the field of the investigation (eg, condition under study, expected adverse effects of the intervention) and a statistician or clinical trial methodologist. The clinicians could be a physician, nurse, or other allied health specialist. Other specialists that may be valuable for some trials include clinical pharmacologists, bioethicists, and public health practitioners. A consumer/community advocate (often a parent of a child with the disease or condition under study) may also provide a helpful perspective.10 For international trials, the DMC should ideally include members from all participating regions.

    An essential requirement for all DMC members is the absence of major conflict of interest to ensure the independence of the committee. Financial conflicts (eg, employment by a for-profit sponsor, substantial stock holdings in or yearly income from such a sponsor) have been given most attention, but other types of conflicts (eg, personal relationships, strong intellectual investment) can also be important. For example, an individual responsible for the initial concept being tested in the trial, or for early work in establishing the rationale for the treatment being studied, might be more reluctant to consider stopping a trial for futility (or even safety) than someone without such a history because that person’s career could be substantially enhanced if the original research concept was validated in a clinical study.

    A general principle is that a DMC should be as small as possible while encompassing all relevant expertise. Some advocate that DMCs should include an odd number of participants to avoid tie votes11; however, recommendations should optimally be developed by consensus rather than by voting.5 The DMC chairperson should have substantial experience in clinical trials and at least some experience on previous DMCs. Interpersonal and organizational capabilities are important issues to consider when selecting the chairperson.

    A special challenge in pediatric trials is balancing expertise against conflict of interest concerns for DMC members. The pediatric research community is relatively small, especially when the focus is on less common diseases. The pool of potential DMC members with the requisite expertise and lacking conflicts of interest may thus be limited, particularly when many knowledgeable investigators are themselves involved in the trial (as would be the case for a multicenter trial). In some cases, nonpediatric clinicians, or pediatricians from other countries, might be included. For trials in resource-poor countries, however, it is important that DMC members have sufficient knowledge of the local issues at the trial sites. Although conflict of interest concerns are important, expertise in the subject matter and experience in clinical trials are paramount.

    Scope of Responsibilities

    The overarching purpose of a DMC is to protect the safety of study participants and the ability of the trial to yield reliable results. Thus, the 1 responsibility common to all DMCs is regular review of study data and developing recommendations regarding the continuation of the study and/or any modifications that might be needed. A DMC may, for example, recommend that a dose level should be reduced, that a subgroup of participants seeming to be at unacceptably high risk of adverse outcomes be excluded from further enrollment, or if interim data clearly demonstrate that 1 treatment produces superior outcomes, that the study be terminated before reaching its original recruitment goal. They may also consider the likelihood of the trial eventually producing results that would lead to useful information for clinical practice or as a basis for further research, and should consider any newly available external evidence when making recommendations regarding study continuation.

    Many DMCs are charged with broader responsibilities. It is common for DMCs to be asked to review and approve the study protocol before initiation. This practice is desirable because it is essential for DMC members to be in agreement with the study sponsors and investigators about the acceptability of the study design, particularly the statistical monitoring plan and any criteria for early termination.5 Selecting criteria for early stopping involves a difficult balance between protecting trial participants and assuring the value and credibility of the trial results; the acceptable balance is likely to vary among trials. For example, in a trial evaluating a new childhood vaccine, there will be great emphasis on collecting enough data to ensure that the vaccine can be given safely to millions of healthy children, and therefore only extreme efficacy results might allow consideration of early termination. In a trial evaluating a potentially life-saving treatment of seriously ill children, a somewhat less restrictive criterion might be set to allow a superior treatment to be made available more rapidly. Because is not unusual for experts to have varying views on the optimal criteria for early termination,12,13 DMC members must make certain they are comfortable with the monitoring plan proposed by the sponsor.

    Another common charge to DMCs is to review and make recommendations regarding the quality of study conduct. When site data are not reported in a timely manner to the study data center, or the data reported include many inaccuracies, the DMC’s ability to make useful interim recommendations is diminished, and the discovery of important safety issues could be delayed. If recruitment lags such that the feasibility of successful completion of the trial is in question, participants may be put at risk without expectation of any useful result. DMC recommendations may ensure that such deviances be addressed rapidly.14

    DMCs may on occasion be asked by sponsors or investigators to release certain interim data. Reasons for such requests could include the need for this information in planning future studies, evaluating safety concerns arising in other studies of the investigational treatment, or requests by regulatory authorities. Because the DMC protects the confidentiality of the interim data, it will determine whether such requests can be granted without undermining the integrity of the trial on a case-by-case basis.

    In some trials, DMCs are asked to review and approve manuscripts and presentations reporting trial results. In general, it is good practice to offer a DMC the opportunity to review and comment on such data presentations because their particular insights are likely to be valuable, especially when the DMC has had an important role in the conduct of the trial.

    Under the rubric of these general responsibilities, there are some specific issues that DMCs for trials in children should routinely consider. First, because the data for planning pediatric trials are frequently limited, a DMC should assess the validity of the design assumptions regarding the expected event rates or other data on which the sample size and trial duration specifications were based. This should be done in the early phase (before interim analyses) because the DMC may need to recommend changes in the projected trial size and duration if the original assumptions seem substantially inappropriate. Second, it may be important to give particular attention to results in different age groups, as small age differences in children may have a much stronger effect on response to treatment, either positive or adverse, than in adults. Finally, when there is the potential for delayed treatment effects (eg, effects on growth, cognitive function, fertility) the DMC may need to ensure that a plan is in place for long-term monitoring of outcomes in study participants even beyond the formal study completion date.

    Operation of DMCs

    All DMCs should be governed by a DMC charter, which is typically prepared by the sponsor and approved by the DMC before their review of any interim data.5,15 This document outlines roles and responsibilities of the DMC and includes details of membership, meeting formats, reporting procedures, conflict of interest criteria, taking and archiving of meeting minutes, and statistical monitoring plans. It is increasingly recognized that sponsors should provide DMC members with protection from liability16 and such arrangements should be documented in the charter.

    The conduct and frequency of DMC meetings will depend on the size of the study, the expected rates of accrual and occurrence of study outcomes, and the perceived risks of the study interventions. Most DMCs meet 1 to 3 times a year. In some cases, when there is particular concern about safety, the DMC or a subset of members may receive reports of adverse events more frequently and meet as necessary. Although in-person meetings allow for optimal interaction, telephone or Internet-based discussions can be effectively undertaken and are more conserving of study resources and time of DMC members. Such meetings, however, require special care to protect the confidentiality of DMC proceedings.

    The conduct of DMC meetings usually involves “open” and “closed” sessions. Participants in open sessions may include study investigators, monitors, sponsors, and regulators. Open sessions allow the DMC to engage with the study team and to raise issues related to the trial’s conduct. They also afford an opportunity for sponsors and others to inform the DMC of emerging data external to the trial and to raise any concerns they want the DMC to give special attention to. Only data on baseline characteristics, and in some cases aggregate outcome data, are discussed at this session. The closed session typically includes only DMC members and the statistician reporting to the DMC (although in cases of government-sponsored trials, a representative of the sponsoring program also attends closed sessions). It is here that the confidential study efficacy and safety data, split according to treatment arm, are fully interrogated and recommendations drafted.

    Open and closed data reports are ideally prepared by a statistician other than the primary study statistician to preserve the latter’s ability to collaborate with other investigators on study conduct issues without being influenced by knowledge of interim results.17,18 The open report is available to all meeting attendees; closed reports are provided to the DMC only. Minutes are maintained for open and closed sessions. Open session meeting minutes, including DMC recommendations, are shared with the study team, sponsors, and regulatory authorities if warranted and may then be shared with institutional review boards of participating sites. Closed session minutes capture the DMC deliberations and are kept confidential until study termination, when they can then be shared. The DMC is responsible for making recommendations about trial conduct and continuation but in most cases does not make binding decisions. These remain the responsibility of the study leadership, usually the study sponsor (a funding agency or product manufacturer). In some cases, authority for such decisions is invested in a trial steering committee, which includes representatives of trial investigators and the study funder(s).

    An area of continuing controversy is whether DMC members should review interim data on efficacy in a fully unblinded way or according to coded arm only. Some DMC members prefer to view coded data initially but reserve the right to unblind themselves if felt to be in the participants’ best interest. Many have argued that it is scientifically and ethically problematic to withhold fully unblinded data from the DMC.19,20

    DMC members should declare any conflict(s) of interest at the beginning of the study. Potential conflicts newly arising during the study should be considered by the sponsor as well as by the full DMC. A determination of unacceptable conflict may result in withdrawal of the member from the DMC. Finally, because of the smaller pool of potential DMC members in pediatrics, especially in rare disease areas, the training of individuals for future DMCs is a challenge. It is important to have a sufficient number of trained potential DMC members in the pediatric research community.

    Reporting on DMC Involvement and Activities

    Adequate reporting of DMC activities allows readers to evaluate the committee’s impact on the validity of trial results. Reporting of DMC roles, interim analysis results, and early termination has been incomplete and heterogeneous in published pediatric trials.21 The 2010 Consolidated Standards of Reporting Trials Statement recommends that authors should describe the conduct of interim analyses (eg, how many were performed, whether they were planned or ad hoc, their timing, what triggered them, which statistical methods were used) and any stopping guidelines or rules defined a priori, and should state whether the trial stopped early, for what reason, and whether the decision was based on a recommendation from a DMC.22 Names of DMC members with their affiliations and areas of expertise should be listed along with other key trial committee memberships. However, co-authorship for DMC members on articles reporting trial results is inappropriate because it would create a conflict of interest (ie, DMC members would then also be investigators). Other aspects of the DMC’s operational procedures and actions (eg, coded versus fully unblinded review) and any interim recommendations made by the DMC (eg, change in recruitment protocol or safety monitoring plan) should also be reported,21,22 either in the article or in on-line supplements. The DMC might have a role in ensuring that adequate information on its activities is easily accessible to the reader of the final trial report by requesting that this be addressed in their charter.

    Some DMCs have prepared case study reports detailing decision-making processes in certain trials.2327 These can be informative as to the dilemmas involved in balancing harms and benefits when difficult or unexpected situations emerge as trial data accumulate. We strongly encourage members of DMCs for pediatric trials to publish their experiences so that they may provide valuable lessons for the design and monitoring of future trials in children.

    Research Agenda

    Five key areas for future research have been identified to date. These include the following: (1) understanding the benefits and harms of having a DMC, particularly in cases in which it is not clear whether a DMC is needed; (2) determining the criteria for “few safety data,” “major morbidity,” and “high-risk populations” with respect to research in pediatrics; (3) evaluating the effects of the use of coded or unblinded information by DMCs on their recommendations; (4) defining the DMC role in the oversight of adaptive trials; and (5) developing effective and appropriate training programs for novice as well as experienced DMC members. Developing our understanding and evidence in these areas will not only better define the role and the impact of DMCs but will also enhance the capacity of individuals serving on DMCs to improve the quality of pediatric research while ensuring the safety of patients participating in these trials.

    Conclusions

    Numerous issues surrounding the principles and practices of DMCs, with special attention to their application in pediatric confirmatory (Phase III) trials, are currently under discussion. This standards article defines a set of minimum requirements to which DMCs should adhere to best serve pediatric researchers as well as trial participants. Both clinical and methodologic criteria can be used to determine whether a DMC is required for a particular study. Because pediatric research is an area in which participants are often considered vulnerable, it may be appropriate to include a DMC in some cases for which the criteria described earlier are not fully met. DMC membership should be broad enough to include individuals with clinical and methodologic expertise and knowledge of local context. The pediatric clinical research community is relatively small, however, and conflicts of interest must be thoroughly assessed and avoided. Due to the large scope of DMCs and potential impact on the validity of trial results, the operations of these committees should be guided by a detailed charter. Including a brief description of the DMC and its operations in manuscripts clarifies the study monitoring processes and will inform the reader in interpreting trial results. Finally, by systematically addressing the research agenda outlined in this article, unnecessary variation in DMC function and practice can be reduced and optimal practices can be advanced.

    Footnotes

      • Accepted March 23, 2012.
    • Address correspondence to Martin Offringa, MD, PhD, Senior Scientist and Program Head, Child Health Evaluative Sciences, Research Institute, The Hospital for Sick Children, 555 University Ave, Toronto, Ontario, Canada M5G 1X8. E-mail: martin.offringa{at}sickkids.ca
    • Drs Ellenberg, Van Der Tweel, and van der Lee wrote the first draft of the article; Drs Fernandes, Saloojee, Bassler, Askie, Vandermeer, Offringa, and Altman contributed to the writing of the article; Drs Ellenberg, Fernandes, Saloojee, Bassler, Askie, Vandermeer, Van der Tweel, and van der Lee participated in regular meetings and conference calls, identified the issues, and drafted the manuscript; Drs Ellenberg, Fernandes, Saloojee, Bassler, Askie, Vandermeer, Van der Tweel, Altman, and van der Lee participated in identifying the evidence base for StaR Child Health standards; and Drs Ellenberg, Fernandes, Saloojee, Bassler, Askie, Vandermeer, Offringa, Van der Tweel, Altman, and van der Lee agree with the final version.

    • This is the third in a series of standard articles resulting from an ongoing process in which a group of invited experts called a Standard Development Group from StaR Child Health assembles and exchanges information about methods for pediatric trial design, conduct, and reporting. More detailed information about this topic can be found in the introductory article of this supplement or at the StaR Child Health Web site (www.starchildhealth.org).

    • FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

    • FUNDING: Dr Altman is supported by Cancer Research UK programme grant C5529.

    References