- CONSORT —
- Consolidated Standards of Reporting Trials
- OMERACT —
- Outcome Measures in Rheumatology
- PROM —
- patient-reported outcome measure
Dilemma
People making choices in health care should use the findings of clinical trials (ideally, incorporated in systematic reviews) to inform their decisions. If these findings are to be useful and reliable, researchers must select appropriate outcomes, measure them in a scientifically robust manner, and report results thoroughly. There are difficulties, however, relating to the selection and measurement of outcomes in clinical trials, and special considerations are needed when these studies are conducted in children.
This article provides guidance for researchers working on clinical trials in children. Although this article is focused on trials of effectiveness, which are similar to therapeutic confirmatory trials, certain sections may also relate to efficacy trials, pharmacokinetic trials, therapeutic exploratory trials, and trials conducted earlier in drug development.
Selection of Appropriate Outcomes
To be useful, clinical trials that evaluate potential benefits and harms of health care interventions must measure outcomes of relevance to practitioners and patients who make shared decisions about treatment options; regulatory authorities who consider applications for marketing authorizations for medicines; organizations who decide whether to provide funding for an intervention (eg, health care commissioners, insurance companies); and policy makers interested in the impact of an intervention.
It can be difficult, however, to decide which outcomes to measure in clinical trials. One reason is that the impact of illnesses is very variable, and some, but not all, may be improved by an intervention. Various frameworks for considering the effects of illnesses have been suggested,1–4 and others are in development. It may not be immediately obvious which of these effects are of particular importance, and the key aspects of the effects of health care interventions might vary between trials. For example, domains such as short-term measures of disease activity, prevention of symptoms, functional status, longer-term consequences of the illness, overall well-being, and utilization of health care resources should all be considered, but their relative importance will depend on the research question. Researchers must also select from a potentially long list of outcomes that could represent each of these aspects of disease or well-being. The decision to measure a given outcome is complex and depends on various factors, such as its relevance in clinical practice, importance to patients, responsiveness to interventions, and feasibility of measurement. These factors must be balanced against costs and acceptability of measuring the outcome.
The selection of outcomes for clinical trials in children requires special attention. It is often difficult to recruit children into research studies because of a smaller pool of patients,5 so researchers may be tempted to select a primary outcome based on feasibility, rather than importance to the aforementioned groups. Such problems have been observed, for example, in clinical trials in neonatology.6 Another approach to the relatively small numbers of children participating in clinical research has been to include them in studies conducted primarily in adults, with the possibility of analyzing them as an age-related subgroup. This happens frequently, for example, in clinical trials of therapies for asthma.7 This approach, however, may lead to the measurement of outcomes that are not relevant or suitable for children or to the neglect of outcomes that are particularly important to children, such as school attendance and performance, growth to an optimal adult height, or physical, psychosocial, and pubertal development.
In some conditions, important outcomes have been overlooked in clinical trials with children. One study found that outcomes in clinical trials of children with asthma tend to focus on how interventions affect short-term symptoms, measures of lung function, and acute exacerbations of illness, with longer-term outcomes, quality of life, and functional status being measured much less frequently.7 In clinical trials in neonatology, a reliance on short-term outcomes, rather than longer-term benefits and harms of interventions, has also been shown.6 In juvenile idiopathic arthritis, recognition that traditional outcomes (eg, reduction in the number of painful joints) do not comprehensively represent the overall well-being of patients has led to the use of measures of quality of life and long-term outcomes reflecting permanent damage to joints.8,9 One example of how outcomes widely measured in adults may not be appropriate in children is in clinical trials of antihypertensive agents. Reduction in systolic blood pressure, which is traditionally used as the most important end point in adults, has also been measured as the primary outcome in studies in hypertensive children. It has been suggested, however, that diastolic blood pressure is a more appropriate outcome in children, and that failure to recognize differences between adult and pediatric populations has led to difficulties in formulating conclusions from clinical trials.10 One reason for measuring systolic blood pressure as an end point in clinical trials in adults is that it is a major risk factor for stroke,11 but this event is rare in children, in whom prevention of other long-term consequences of hypertension are more relevant and may be more closely related to high diastolic blood pressure.12 When measured in children, diastolic blood pressure is also more closely related to the underlying pathogenesis of hypertension in this age group.10
Measurement of Outcomes in a Scientifically Rigorous Manner
Researchers may need to choose from a variety of tools, methods, and instruments for measuring outcomes.13 These methods should be sufficiently valid, responsive, and sensitive to reliably detect any differences that would be important. One potential problem in children relates to patient-reported outcome measures (PROMs). These measures are used to collect information, directly from the patient, about how they feel or function. By using PROMs that are appropriate for adults, without validation in the relevant pediatric age groups, might produce unreliable estimates of the effects of an intervention in children. In addition, PROMs administered to children may need to be completed by parents or medical caregivers, without sufficient validation of their use as a proxy instrument. Such problems have been highlighted in clinical trials of interventions for children with sepsis,14 pain,15 inflammatory bowel disease,16 and other chronic conditions.17
In certain conditions affecting children, researchers have questioned whether instruments used for measuring outcomes have been sufficiently validated. One review of clinical trials of pediatric acute diarrhea found that several outcome measures were used, but researchers did not provide adequate information on whether these had been validated.18 In a variety of fields, such as rheumatology,8,19 cystic fibrosis,20,21 neonatal pain,22 and pediatric oncology,23 researchers have attempted to address similar gaps by identifying validated and appropriate outcome measures for children.
Core Outcome Sets
Inconsistent selection, measurement, and reporting of outcomes in clinical trials causes 3 main problems. First, important outcomes could be overlooked, especially if the outcomes of most relevance in a given condition are not known. Second, if researchers measure outcomes in a nonuniform manner, comparisons between trials asking a similar question will not be possible and nor will aggregation of the results in meta-analyses. This has been highlighted by authors of systematic reviews of clinical trials of interventions for children, with particular problems (due to heterogeneity in the original studies) of outcomes selected, their definitions,24 the tools used to measure them,24,25 and time points at which they were measured.26,27 Third, in the absence of an agreed on set of outcomes to be measured and reported in all clinical trials of interventions for a given condition, trialists may omit certain results from their final report. The selective reporting of a subset of the original recorded outcomes, based on the results of these outcomes, is referred to as outcome reporting bias.28 This action can substantially affect the interpretation of an individual study and compromise the validity of the results of meta-analyses that include it.29 It is also common for researchers to change their choice of primary outcome as the trial progresses.28
One solution to all these problems is to develop, and adopt, core outcome sets.13 These are minimum sets of outcomes that should be measured in all clinical trials in a given condition, in a standardized manner, and reported in the final publication. These sets do not need to be extensive but rather might comprise a few particularly important and agreed on outcomes. Researchers are free to measure other outcomes in addition to those specified in the core outcome set.
One key initiative relating to core outcome sets is the Outcome Measures in Rheumatology (OMERACT) collaboration. OMERACT advocates the use of core outcome sets, agreed on using structured consensus techniques, which are then measured and reported in clinical trials in rheumatology.30 One of the strengths of the OMERACT approach is that there has been broad engagement of the clinical, research, and patient communities in the development of recommended outcomes, which means that they are generally accepted as the right outcomes to measure in clinical trials. Although this approach has been adopted in other pediatric rheumatologic conditions19,31,32 and pediatric pain,15 such initiatives are uncommon elsewhere in health care research in children.1 Furthermore, the quality of work regarding development of core outcome sets for pediatric trials is variable, and children and parents are seldom asked which outcomes they regard as important.1,33
Core Outcomes in Effectiveness Trials is an international initiative that aims to collate and maintain a searchable database of existing core outcome sets, offer guidance on how other sets can be developed, and suggest issues to consider when appraising the quality of a core outcome set (http://www.comet-initiative.org.uk).
Guidance
A summary of the following recommendations for practice is provided in Table 1.
Recommendations for Practice
Adopt Core Outcome Sets
If a core outcome set suitable for a trial has been developed by using an appropriate method and has broad acceptance, researchers should measure and report the outcomes included within it. If any or all of the outcomes included within a core outcome set are not measured in a trial, the reasons for this decision should be outlined when the findings of the study are reported.
Researchers may wish to consider certain factors before deciding whether to adopt an existing core outcome set. For example, to determine whether a core outcome set is relevant to their trial, investigators should examine whether it is applicable to the age groups to be studied, pertinent to the condition of interest in the context of the trial setting, and compatible with the scope of the trial. They may also wish to appraise the method used to develop the core outcome set; for example, by considering whether it was agreed on by using structured consensus techniques and whether patients, families, and clinicians were involved in its development. Some core outcome sets have been developed by using the Delphi technique, and guidance for appraising the quality of conduct and reporting of such studies is available.33 Researchers involved with Core Outcomes in Effectiveness Trials are formulating further guidance on assessing the quality and relevance of core outcome sets.
Clarification around whether an outcome was measured in the first place reduces concerns about whether the results, for that outcome, were selectively reported. Therefore, if researchers choose not to measure 1 or more outcomes included in a core outcome set, we recommend that they should explain this decision in the final study report and strongly consider doing so in the study protocol.
Measure Important Outcomes, if an Appropriate Core Outcome Set Has Not Been Developed
When designing a clinical trial examining effectiveness of therapies, researchers should consider all aspects of the effects of interventions to identify primary and secondary outcomes that are important when making decisions about health care practice.
To evaluate the effects of interventions comprehensively, researchers should not just measure short-term outcomes reflecting disease process but should also consider other factors relevant to decision makers, including functional status, quality of life, the effects of the illness on the family, and longer-term effects of interventions. We recommend that researchers consider conducting work, before their trial, to identify those outcomes that are used to influence decisions regarding health care interventions. This action would make the eventual results of the trial more useful.
In its guidance on good practice in all clinical trials, the International Conference on Harmonisation34 states that the primary outcome should measure “a clinically relevant and important treatment benefit” and should be clearly “specified in the protocol, along with the rationale for its selection.” It also states that the “validity of the associated measurement procedures will generally need to be addressed and justified in the protocol.” Given the importance of the primary outcome, we also suggest that it would be useful if the rationale behind its selection were explained in the final study report.
Measure Outcomes in a Rigorous Manner and Report the Methods Clearly
Where possible, researchers should measure outcomes using tools, instruments, or methods that have been shown to be valid for children in the age group included in the trial, with the illness of interest, and in the setting in which the trial is conducted. If tools, instruments, or methods have been developed but not robustly validated, researchers should strongly consider performing validation studies before they embark on their trial. Where possible, researchers should use methods to enhance the quality of measurements (eg, multiple observations, training of assessors, masking of assessors). The following aspects about how and when outcomes are measured should be clearly described when the study is reported: (1) the person(s) reporting or measuring the outcome, and whether they were aware of the intervention to which the patient had been allocated; (2) tools, instruments, or methods used to measure the outcome; (3) the time points at which the outcome was measured and analyzed; (4) definitions of event end points (eg, what constitutes an “exacerbation of asthma”, when is “treatment failure” said to occur, or how it will be determined when a clinical problem is “resolved”); and (5) references to previous work, either by the researchers conducting the trial, or by others, to validate a tool, instrument, or method.
A searchable database containing outcome measures used in clinical trials in children, and information on whether these measures have been validated, is under construction (S. Vohra, MD, MSc, personal communication, 2012). If no tools, instruments, or methods for measuring an outcome have been developed, or if existing ones have not been validated, we encourage researchers to undertake this work before their trial. This will help ensure that their eventual findings are regarded as sufficiently robust to inform decisions regarding health care interventions.
Two existing tools may be particularly useful for researchers who are considering whether instruments are appropriate for measuring outcomes in their trial. One is the OMERACT filter of “truth, discrimination, and feasibility.”35 Another is the checklist developed by the Consensus-based Standards for the Selection of Health Measurement Instruments initiative to assess the quality of studies on measurement properties of health status measures (ie, health-related PROMs). This recommends that health-related PROMs should be evaluated in terms of internal consistency, reliability, measurement error, content validity (including face validity), construct validity (including structural validity, hypotheses testing, and cross-cultural validity), criterion validity, responsiveness, and interpretability.36
Individuals who use the reports of clinical trials, either as stand-alone evidence or within systematic reviews, need to be aware of how and when all outcomes reflecting benefits and harms were measured. In its guidance on reporting of clinical trials, the Consolidated Standards of Reporting Trials (CONSORT) statement recommends that researchers explicitly define “pre-specified primary and secondary outcome measures, including how and when they were assessed.”37 The specific points relating to this, highlighted in our guidance, are included either in the main CONSORT statement37 or in the addendum of special considerations for children.38 Including these descriptions in trial reports makes research methods transparent, enables the reader to appraise the quality of outcome measurement, and allows systematic reviewers to assess heterogeneity between studies.
Report Results Thoroughly
All prespecified outcomes that are measured and analyzed should be reported at the conclusion of the trial, regardless of the results for these outcomes. Ideally, this information would be available in the final study report or associated supplementary information.
Changes to Outcomes Should be Clearly Documented
In some circumstances, researchers will need to change outcomes as their trial progresses. Any such changes should be documented and explained in the final study report. This is particularly important if: (1) results for outcomes that were prespecified when designing the trial are not described in the study report; (2) an outcome prespecified as either primary or secondary in the protocol is not given the same designation in the final study report; and (3) results are described in the study report for outcomes that were not prespecified when designing the trial.
Because changes to the outcomes, especially the primary outcome, represent a major deviation from the initial research question, these should be explained when the trial is reported, as recommended in the CONSORT statement.37 In addition, it is expected that these are documented formally in a protocol amendment (if the trial has a protocol), and reported to relevant ethics committees, institutional review boards, and data monitoring committees.
Research Agenda
Ongoing research, some of which is mentioned in this article, will help ameliorate some of the problems with the measurement and reporting of outcomes in clinical trials in children. One important area of research will be to identify the most appropriate ways to develop core outcome sets and to involve children and families in this process. Future discussions among this Standard Development Group will also focus on the appropriate use of biomarkers and surrogate outcomes, composite outcomes, and measurement of short- and long-term harms and safety outcomes of interventions in clinical trials in children.
In many conditions, such as sepsis,39 survival of children has improved to the extent that short-term mortality is no longer the most relevant outcome, and surrogate outcomes are needed to predict long-term mortality and morbidity to plan feasible trials. Improved methods of early diagnosis in some diseases, such as cystic fibrosis, have driven the search for surrogate outcomes that can be used in clinical trials in infants to evaluate whether early intervention prevents later morbidity.20 There are concerns, however, about the use and interpretation of surrogate outcomes in clinical research,40–42 and in pediatric trials these may be particularly relevant because illness and interventions during childhood could have effects during adult years. One example of a misleading surrogate outcome in children was the incidence of neonatal intraventricular hemorrhage, detected by cranial ultrasound, in the ORACLE trial of antibiotics for women in preterm labor. Initial analysis of the data suggested that antibiotics were safe in this situation,43 partly because there was no increased risk of intraventricular hemorrhage in infants whose mothers were randomized to receive antibiotics rather than placebo. Follow-up 7 years later, however, showed that infants whose mothers received antibiotics had a higher risk of neurologic impairment such as cerebral palsy.44 The use of surrogate outcomes will be discussed in future StaR Child Health standards articles.
Conclusions
Researchers should measure important outcomes in clinical trials in children in a scientifically rigorous manner. Where available, core outcome sets, developed using an appropriate method, should be used to standardize the selection, measurement, and reporting of key outcomes across trials. Methods for measuring outcomes must be described explicitly, alongside the results for all prespecified outcomes. By these means, the reliability of evidence derived from clinical trials in children will be enhanced and will lead to better decision-making regarding health care interventions for children.
Footnotes
- Accepted March 23, 2012.
- Address correspondence to Rosalind L. Smyth, Department of Women’s and Children’s Health, Institute of Translational Medicine, University of Liverpool, Liverpool, United Kingdom. E-mail: r.l.smyth{at}liv.ac.uk
Drs Williamson and Smyth share the designation of last author.
Drs Sinha, Williamson, and Smyth wrote the first draft of the article; Drs Sinha, Altman, Beresford, Boers, Clarke, Craig, Alberighi, Fernandes, Hartling, Johnston, Lux, Plint, Tugwell, Turner, van der Lee, Offringa, and Williamson contributed to the writing of the article; Drs Sinha, Altman, Beresford, Boers, Clarke, Craig, Alberighi, Fernandes, Hartling, Johnston, Lux, Plint, Tugwell, Turner, van der Lee, and Williamson participated in regular conference calls, completed questionnaires, identified the issues, and drafted the manuscript; and Drs Sinha, Altman, Beresford, Boers, Clarke, Craig, Alberighi, Fernandes, Hartling, Johnston, Lux, Plint, Tugwell, Turner, van der Lee, Offringa, and Williamson agree with the final version. Drs Williamson and Smyth are joint guarantors of the article.
This is the fifth in a series of standard articles resulting from an ongoing process in which a group of invited experts called a Standard Development Group from StaR Child Health assembles and exchanges information about methods for pediatric trial design, conduct, and reporting. More detailed information about this topic can be found in the introductory article of this supplement or at the StaR Child Health Web site (www.starchildhealth.org).
FINANCIAL DISCLOSURE: Dr Turner is supported by Global Research in Paediatrics (GRIP), a Network of Excellence funded by the European Commission under Framework Programme, Project Number 261060; Dr Altman is supported by Cancer Research UK programme grant C5529; Dr Williamson is supported by a Medical Research Council for the North West Hub for Trials Methodology Research; Dr Sinha is supported by an NIHR Clinical Lectureship Award; the other authors have indicated they have no financial relationships relevant to this article to disclose.
References
- Copyright © 2012 by the American Academy of Pediatrics