PEDIATRICS Vol. 117 No. 6 June 2006, pp. 2222-2230 (doi:10.1542/10.1542/peds.2005-1630)
SPECIAL ARTICLE |
Quality of Reporting of Randomized, Controlled Trials in Cerebral Palsy
a Finnish Office for Health Technology Assessment, National Research and Development Centre for Welfare and Health, Helsinki, Finland
b Institute for Clinical Epidemiology, University Hospital Basel, Basel, Switzerland
c Hospital for Children and Adolescents, University of Helsinki, Helsinki, Finland
| ABSTRACT |
|---|
|
|
|---|
OBJECTIVES. In conducting reviews on the effectiveness of physiotherapy interventions on children with cerebral palsy, the assessment of trials can be hampered by problems in reporting. Therefore, we set out to evaluate trial reporting by using the Consolidated Standards of Reporting Trials (CONSORT) statement recommendations.
METHODS. Randomized, controlled trials published in 1990 or later were identified in literature searches for reviews on the effectiveness of physiotherapy interventions on children with cerebral palsy. Two independent reviewers evaluated the trial reporting by using a modified 33-item Consolidated Standards of Reporting Trials checklist.
RESULTS. We identified 15 randomized, controlled trials. Almost half (48%) of the applicable items were reported adequately. Inadequate reporting was found in the following items: outcome measures, sample-size determination, details of the sequence generation, allocation concealment and implementation of the randomization, success of assessor blinding, recruitment and follow-up dates, intention-to-treat analysis, precision of the effect size, co-interventions, and adverse events.
CONCLUSIONS. Only a small number of sufficiently reported trials were found. Because nearly all items had been described in at least 1 article, high-quality reporting seems feasible. Assessment of trials depends on appropriate reporting, and poor reporting jeopardizes judgments on the clinical implications. Authors of randomized, controlled trials are encouraged to follow the Consolidated Standards of Reporting Trials criteria. There is a clear need to improve the quality of reporting of trials in this field.
Key Words: CONSORT trial report physiotherapy randomized controlled trial cerebral palsy
Abbreviations: RCTrandomized, controlled trial CONSORTConsolidated Standards of Reporting Trials CPcerebral palsy PTphysical therapy OToccupational therapy NDTneurodevelopmental therapy
Randomized controlled trials (RCTs) are widely accepted as the "gold standard" for providing evidence on the effectiveness of interventions.1 Adequate reporting is essential so that the reader can evaluate how the trial was conducted and judge its validity and the applicability of the results.
The Consolidated Standards of Reporting Trials (CONSORT) statement2,3 offers a standard for reporting randomized trials. It comprises a flowchart and a checklist of 22 items. Checklists based on the CONSORT criteria have been used previously to assess the reporting quality of RCTs in dermatology,4 ophthalmology,5 antidepressants and antipsychotics,6 pediatric complementary and alternative medicine,7 and, most recently, in palliative care.8 Furthermore, the CONSORT checklist has been applied to evaluate journal policies to promote adherence to the CONSORT criteria.9,10
This study is based on a project that evaluates the effectiveness of rehabilitation in children with cerebral palsy (CP) at the Finnish Office for Health Technology Assessment. CP is an umbrella term that covers a group of nonprogressive but often changing motor-impairment syndromes secondary to lesions or abnormalities of the brain that arise in the early stages of development.11 An increasing spectrum of interventions is used to modify the natural course of the disorder. For this project, we focused on producing systematic reviews of the effectiveness of physical therapy (PT) and the impact of intense versus less-intense PT. There are intricate problems related to complex interventions, the heterogeneity of the patients, and the multitude of outcomes in CP trials. Therefore, a separate analysis of reporting quality is justified. Using the CONSORT recommendations, we set out to identify the problems of the trials highlighted in the process of carrying out 2 separate reviews. In this article we use a systematic approach to evaluate the reporting quality of RCTs on PT and occupational therapy (OT) (PT/OT) published in 1990 or later with a modified CONSORT checklist.
| METHODS |
|---|
|
|
|---|
Sampling Frame and Literature Searches
We used searches of 2 Finnish Office for Health Technology Assessment reviews: 1 was an overview of systematic reviews on PT for CP, and the other was a systematic review of controlled trials that compared 2 different intensities (hours/week) of the same type of PT. For both reviews we searched PT interventions in children or adolescents with diagnosed CP who were aged 3 months to 20 years. The search criteria initially comprised all types of PT interventions and were later classified further. Key search terms were "cerebral palsy"; various synonyms for PT such as "physical therapy techniques"; "rehabilitation"; "neurodevelopment"; "exercise therapy"; "controlled clinical trial"; "randomized, controlled trials"; "double-blind"; "random allocation"; "meta-analysis"; and "systematic review." The searches were conducted in April to June 2003 in Medline, the Cochrane library, American College of Physicians Journal Club, CINAHL (a database for allied health and nursing), the Physiotherapy Evidence Database (PEDro; www.pedro.fhs.usyd.edu.au/index.html), and the Health Technology Assessment database (www.york.ac.uk/inst/crd/htahp.htm). The reference lists of the selected review articles were checked. Only full-text articles written in English, German, Danish, Finnish, Norwegian, or Swedish were considered. Only RCTs published since 1990 were included.
The 2 searches yielded 767 reviews in the overview and 349 original trials in the systematic review. Two reviewers (H.A. and R.K./H.A. and I.A.-R., respectively) independently selected studies based on abstracts. We screened the full texts of 104 articles contained within an earlier review process and found 17 systematic reviews that fulfilled our inclusion criteria. We further identified 20 RCTs published between 1973 and 1998 from the reference lists of these reviews. In the systematic review we screened the full text of 21 trials on the intensity of therapy and identified 9 RCTs. Excluding trials published before 1990, 15 RCTs were finally included in this study.
Checklist
We (H.A. and A.M.) operationalized the 22 items of the CONSORT statement checklist3 to 34 questions that can be scored as "yes," "partly," "no," "unclear," or "not applicable." After piloting the checklist, we made corrections to the wording of 3 items and excluded all 3 items of the conclusion section because the operationalization of them proved to be difficult. In addition, we added 2 items: validation of the outcome measures and co-interventions. Thus, the modified CONSORT-based checklist comprised 33 items, as shown in Appendix 1.
|
Data Extraction
Two researchers (R.K. and H.A.) independently evaluated the quality of reporting in the identified trials by using this modified checklist. The evaluators disagreed in 23% of the evaluations. After consensus discussions, the remaining disagreements (9%) were resolved by a third researcher (A.M. or I.A.-R.). One researcher (H.A.) detected the flow of the participants throughout each trial according to the stages of the CONSORT flowcharts. The outcome measures used in the trials were extracted, and 1 researcher (R.K.) searched the bibliographies of all studies for references, confirming that the instrument had been validated; if no such reference was provided, the researcher checked the Internet for this information.
Data Analysis
We analyzed the number and overall proportion of adequately reported items (scored as "yes") and insufficiently/inadequately (scored as "no," "partly," or "unclear") for each trial. Items that were not applicable in some of the trials were excluded from the analysis. Trials published in 19901997 were compared with trials published in 19982002 to examine whether there was any improvement in the quality of reporting after the publication of the CONSORT statement.
| RESULTS |
|---|
|
|
|---|
The 15 identified trials1226 that studied PT or OT intervention in children with CP are presented in Table 1. One trial was reported in 2 stages.20,26 All trials were small, and the studied populations, interventions, and outcomes differed between the trials. Nine trials had a neurodevelopmental therapy (NDT) treatment in 1 arm, compared with less-intensive NDT, casting, infant stimulation, functional PT, or conductive education. The other interventions comprised various PT/OT methods.
|
Nine percent of the evaluations (particularly CONSORT descriptor items 10, 18, 20, 22, and 33) were not applicable in some of the trials and, thus, are not counted in the calculations. Of all evaluations of the applicable 31 items based on the CONSORT checklist, nearly half (48%) were reported adequately (Table 2). The remaining items were not reported at all (37%), were reported partially (13%), or not reported clearly (2%). Only 7 trials used validated outcome measures, and 5 trials reported on co-interventions. Adding these 2 items did not change the overall results.
|
More than half of the items were reported adequately in 7 trials, whereas 4 trials reached an adequate score in <10 items (30%). When we compared trials published between 1990 and 1997 (n = 9) and 1998 and 2002 (n = 6), no clear difference emerged in the quality of reporting; the percentages of adequately reported items were 45.5% (SD: 30.6; range: 088.9) and 40.9% (SD: 29.8; range: 083.3), respectively. Assessment scores for each CONSORT item in the 15 trials are shown in Appendix 2.
|
Items Usually Reported Adequately
Most trials reported adequately issues that are usually defined in the introduction section of a trial report. These are problem definitions and descriptions of research subjects, objectives, settings, and trial locations. Of the study methods, precise details of the actual administration of the interventions were also available, defined as intensity and short descriptions of the type of the therapy. Half of the trials reported methods (eg, training of assessors) to enhance the quality of measurements. In addition, the statistical procedures used to analyze the results were fully reported in most articles. Half of the trials had used subgroup or adjusted analyses. Despite the fact that the authors had clearly specified the choice of the variables adjusted, it was seldom clear whether these analyses were planned in the protocol or whether they were data driven.
Most trials reported adequately the number of participants of each group in each analysis and a summary of the results and effect sizes for each group and for each of the outcomes. The precision of the effect sizes expressed as confidence intervals were available in only 4 trials. From half of the trials we found adequate tables reporting baseline demographic characteristics such as type of CP, age, and mean values of the main outcome measure. Detailed charts of participant flow were available in none of the trials. However, we could construct the participant flow starting from the randomization process in 11 trials (Table 3). In nearly half of the trials it remained unclear to us how many children were assessed and excluded before the randomization.
|
Items Seldom Reported Adequately
We identified major shortcomings in the descriptions of the trial methods, particularly in reporting the randomization process. Concealment of the allocation was described in none of the trials. Only 2 articles detailed how the randomization was implemented. The method to generate the random-allocation sequence was clarified in only one third of the trials.
Primary outcome was defined in one third of the trials. Multiple outcomes were evaluated in the remaining trials without specifying a primary one. Fifty-one different outcome measures were used in the trials, of which only 4 were used in >1 study (Gross Motor Function Scale in 7 trials, Peabody Fine Motor Scale in 3, and modified Ashworth scale and Quality of Upper Extremity Movement in 2; all validated). Less than half of the trials applied only validated outcome measures.
A full rationale for sample-size calculation was provided in only a few trials; the other trials did not base the calculations on a specified primary outcome or show the sample size needed for each treatment arm. Only 1 trial16 was able to blind the participants. In the other trials, masking of the participants or the therapist was not applicable because of the nature of the intervention and the setting. Most authors reported on blinding the assessors, but only 2 articles gave additional details on how its success was evaluated. On the basis of the reported numbers in the whole participant flow, we inferred that an intention-to-treat analysis was present in one third of the trials. One third of the trials reported on co-interventions, and a few trials addressed the point of adverse events adequately. The other trials did not consider these topics.
| DISCUSSION |
|---|
|
|
|---|
We analyzed the reporting quality of 14 PT trials and 1 OT trial on children with CP. Of the applicable items for each trial (range: 2832), only half were reported adequately. All items, except group assignment and masking, had been described in at least 1 article. Seven trials demonstrated a sufficient description of at least half of the evaluated items, whereas 2 trials gave adequate descriptions in only 5 items. On the whole, several issues relating to the validity or the applicability of trial results were reported poorly or not reported at all. Even the most adequately reported trials failed to report important details such as sample-size calculation, randomization, concealment of group allocation, co-interventions, and how the assessor blinding was evaluated.
The CONSORT statement, first published in 1996, is a comprehensive quality-assessment checklist for trial reports and provided a logical and systematic view on the reporting. It is endorsed by several leading medical journals.10 Moher et al27 found that the CONSORT statement is associated with improvements in the quality of reports of RCTs. However, publication of the CONSORT statement has had little apparent effect on the quality of reporting in this sample of trials. Only 1 (Pediatrics) of the 9 different journals reviewed in this study provide online instructions for the authors and reviewers on how to follow the CONSORT criteria. The other journals do not mention the CONSORT statement on their Web pages (as of September 2005). A common journal policy that requires the authors and reviewers to use the CONSORT checklist in writing and evaluating trial reports is required to improve reporting.
The poor quality of some of the literature in this study13,19,24 has been noted before in a systematic review on OT in children with CP.28 Hill et al29 studied the association between a published report and actual conduct of a trial in rheumatology. The authors claim that characterizing RCTs as good or poor quality on the basis of the published report is likely to be inappropriate. Quite similarly, Huwiler-Müntener et al30 state that a clear distinction should be made between the reporting quality and the methodologic quality of the trials, because well-conducted trials may be reported badly. Pildal et al31 have shown that allocation concealment is often not clearly described both in study protocols and resulting publications, and discrepancies may occur between the protocol and the publication.
According to our results, many crucial aspects of the study design and analysis of the results were not reported at all, a fact that hampers the quality assessment of these trials. In a comprehensive analysis of the methodologic quality of all PT trials in the PEDro database, Moseley et al32 found that only a few PT trials reported concealment of allocation, blinding, and adequate follow-up. These issues, and all the others suggested by the CONSORT statement, could be described in the trial report. Furthermore, institutional review boards might require researchers to complete methodologic checklists as part of the trial proposal. As long as inadequate information is given, no proper judgments on methodologic quality and applicability of the trial results can be made.
Some of the original CONSORT items were not applicable in all trial contexts: participant and therapist blinding, success of assessor blinding (lack of assessor blinding), interim analyses and stopping rules, and 2 items on ancillary analysis methods (these were not used in the trials). We added 2 items to the CONSORT list that ask whether the outcome measures had been validated and if there had been co-interventions. Both are essential to know in intervention studies. Adding these items did not change the overall results. Half of the trials applied validated outcome measures. Particularly when several instruments are being used, it is crucial to report their validity. There is a clear need for standardization so that certain outcomes are required for all RCTs in a field. Otherwise, systematic reviews are severely hampered and there is scope for biased reporting.
Half of the trials succeeded in adequately reporting at least 16 particular items. However, they were not reported in a rational order as suggested by the CONSORT statement. It required time-consuming effort to locate the items. The following items were most often described inadequately: outcome measures, sample-size determination, details of the sequence generation, allocation concealment and implementation of the randomization, success of assessor blinding, recruitment and follow-up dates, intention-to-treat analysis, precision of the effect size, co-interventions, and adverse events. We think that all these items can easily be reported adequately by being taken into consideration during planning, execution, and reporting of the trial.
This study deserves a few comments. The intention was specifically to find effectiveness studies on present PT interventions that are currently in use. We therefore restricted our analysis to more recent trials, thus excluding 11 trials performed before 1990. Furthermore, in earlier systematic evaluations, it was found that despite weaknesses in the methodologic quality of CP research,33 the quality improved in the 1990s.34 We did not perform separate searches for this study but applied search results from 2 ongoing reviews that encompass trials from 17 reviews complemented with a separate review search that compared 2 different intensities of the same intervention. The sample did not include those PT trials that compared different types of interventions and were published beyond the chosen search periods. We believe that our sample of trials from 19902002 represents the variability in reporting and the key challenges in trials of PT/OT among children with CP. It seems obvious from these 15 most recent RCTs included in this review that there is a clear need to promote better reporting of clinical trials in this field.
| CONCLUSIONS |
|---|
|
|
|---|
Only a small number of sufficiently reported trials were found. Nearly all items had been described in at least 1 article, which indicates that high-quality reporting is feasible. Several items relating to the validity and applicability of PT/OT trials in children with CP are poorly reported. Poor reporting of trials jeopardizes judgments on clinical implications. Authors should be further encouraged to follow the CONSORT criteria when reporting the results of a trial. There is a clear need to improve the quality of reporting of trials in this field.
| FOOTNOTES |
|---|
Accepted Dec 21, 2005.
Address correspondence to Heidi Anttila, MSc, National Development and Research Centre for Welfare and Health, Finnish Office for Health Technology Assessment, PO Box 220, Lintulahdenkuja 4, Helsinki 00531, Finland. E-mail: heidi.anttila{at}stakes.fi
The authors have indicated they have no financial relationships relevant to this article to disclose.
| REFERENCES |
|---|
|
|
|---|
- Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-Based Medicine: How to Practice and Teach EBM. Edinburgh, Scotland: Churchill Livingstone; 2000
- Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized, controlled trials: the CONSORT statement.
JAMA. 1996;276
:637
639
[Abstract/Free Full Text] - Altman DG, Schulz KF, Moher D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration.
Ann Intern Med. 2001;134
:663
694
[Abstract/Free Full Text] - Adetugbo K, Williams H. How well are randomized controlled trials reported in the dermatology literature?
Arch Dermatol. 2000;136
:381
385
[Abstract/Free Full Text] - Sanchez-Thorin J, Cortes M, Montenegro M, Villate N. The quality of reporting of randomized clinical trials published in Ophthalmology. Ophthalmology. 2001;108 :410 415[CrossRef][Web of Science][Medline]
- Khan A, Khan S, Leventhal R, Krishnan K, Gorman J. An application of the revised CONSORT standards to FDA summary reports of recently approved antidepressants and antipsychotics. Biol Psychiatry. 2002;52 :62 67[CrossRef][Web of Science][Medline]
- Moher D, Soeken K, Sampson M, Ben-Porat L, Berman B. Assessing the quality of reports of randomized trials in pediatric complementary and alternative medicine. BMC Pediatr. 2002;2 :2[CrossRef][Medline]
- Piggott M, McGee H, Feuer D. Has CONSORT improved the reporting of randomized controlled trials in the palliative care literature? A systematic review.
Palliat Med. 2004;18
:32
38
[Abstract/Free Full Text] - Devereaux P, Manns B, Ghali W, Quan H, Guyatt G. The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist. Control Clin Trials. 2002;23 :380 388[CrossRef][Web of Science][Medline]
- Altman DG. Endorsement of the CONSORT statement by high impact medical journals: survey of instructions for authors.
BMJ. 2005;330
:1056
1057
[Free Full Text] - Mutch LW, Alberman E, Hagberg B, Kodama K, Velickovic MW. Cerebral palsy epidemiology. Dev Med Child Neurol. 1992;17 :18 25
- Palmer FB, Shapiro BK, Allen MC, et al. Infant stimulation curriculum for infants with cerebral palsy: effects on infant temperament, parent-infant interaction, and home environment. Pediatrics. 1990;85(3 pt 2) :411 415
- Law M, Cadman D, Rosenbaum P, Walter S, Russell D, DeMatteo C. Neurodevelopmental therapy and upper-extremity inhibitive casting for children with cerebral palsy. Dev Med Child Neurol. 1991;33 :379 387[Web of Science][Medline]
- Mayo NE. The effect of physical therapy for children with motor delay and cerebral palsy: a randomized clinical trial. Am J Phys Med Rehabil. 1991;70 :258 267[Web of Science][Medline]
- Girolami GL, Campbell SK. Efficacy of a neuro-developmental treatment program to improve motor control in infants born prematurely. Pediatr Phys Ther. 1994;6 :175 184
- ODwyer N, Neilson P, Nash J. Reduction of spasticity in cerebral palsy using feedback of the tonic stretch reflex: a controlled study. Dev Med Child Neurol. 1994;36 :770 786[Web of Science][Medline]
- MacKinnon J, Noh S, Lariviere J, MacPhail A, Allan DE, Laliberte D. A study of therapeutic effects of horseback riding for children with cerebral palsy. Phys Occup Ther Pediatr. 1995;15 :17 34[CrossRef]
- Bower E, McLellan DL, Arney J, Campbell MJ. A randomised controlled trial of different intensities of physiotherapy and different goal-setting procedures in 44 children with cerebral palsy. Dev Med Child Neurol. 1996;38 :226 237[Web of Science][Medline]
- Law M, Russell D, Pollock N, Rosenbaum P, Walter S, King G. A comparison of intensive neurodevelopmental therapy plus casting and a regular occupational therapy program for children with cerebral palsy. Dev Med Child Neurol. 1997;39 :664 670[Web of Science][Medline]
- Steinbok P, Reiner AM, Beauchamp R, Armstrong RW, Cochrane DD. A randomized clinical trial to compare selective posterior rhizotomy plus physiotherapy with physiotherapy alone in children with spastic diplegic cerebral palsy. Dev Med Child Neurol. 1997;39 :178 184[Web of Science][Medline]
- Reddihough DS, King J, Coleman G, Catanese T. Efficacy of programmes based on conductive education for young children with cerebral palsy. Dev Med Child Neurol. 1998;40 :763 770[Web of Science][Medline]
- Chad KE, Bailey DA, McKay HA, Zello GA, Snyder RE. The effect of a weight-bearing physical activity program on bone mineral content and estimated volumetric density in children with spastic cerebral palsy. J Pediatr. 1999;135 :115 117[CrossRef][Web of Science][Medline]
- Bower E, Michell D, Burnett M, Campbell MJ, McLellan DL. Randomized controlled trial of physiotherapy in 56 children with cerebral palsy followed for 18 months. Dev Med Child Neurol. 2001;43 :4 15[CrossRef][Web of Science][Medline]
- Bumin G, Kayihan H. Effectiveness of two different sensory-integration programmes for children with spastic diplegic cerebral palsy. Disabil Rehabil. 2001;23 :394 399[CrossRef][Web of Science][Medline]
- Ketelaar M, Vermeer A, Hart H, van Petegem-van Beek E, Helders PJ. Effects of a functional therapy program on motor abilities of children with cerebral palsy.
Phys Ther. 2001;81
:1534
1545
[Abstract/Free Full Text] - Steinbok P, McLeod K. Comparison of motor outcomes after selective dorsal rhizotomy with and without preoperative intensified physiotherapy in children with spastic diplegic cerebral palsy. Pediatr Neurosurg. 2002;36 :142 147[CrossRef][Web of Science][Medline]
- Moher D, Jones A, Lepage L; CONSORT Group (Consolidated Standards for Reporting of Trials). Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation.
JAMA. 2001;285
:1992
1995
[Abstract/Free Full Text] - Steultjens EMJ, Dekker J, Bouter LM, van de Nes JCM, Lambregts BLM, van den Ende CHM. Occupational therapy for children with cerebral palsy: a systematic review.
Clin Rehabil. 2004;18
:1
14
[Abstract/Free Full Text] - Hill C, LaValley M, Felson D. Discrepancy between published report and actual conduct of randomized clinical trials. J Clin Epidemiol. 2002;55 :783 786[CrossRef][Web of Science][Medline]
- Huwiler-Müntener K, Jüni P, Junker C, Egger M. Quality of reporting of randomized trials as a measure of methodologic quality.
JAMA. 2002;287
:2801
2804
[Abstract/Free Full Text] - Pildal J, Chan AW, Hrobjartsson A, Forfang E, Altman DG, Gotzsche PC. Comparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study.
BMJ. 2005;330
:1049
1052
[Abstract/Free Full Text] - Moseley AM, Herbert RD, Sherrington C, Maher CG. Evidence for physiotherapy practice: a survey of the Physiotherapy Evidence Database (PEDro). Aust J Physiother. 2002;48 :43 49[Web of Science][Medline]
- Vermeer A, Bakx V. Evaluating intervention research with cerebral palsied children: a literature review. J Rehabil Sci. 1990;3 :7 15
- Siebes RC, Wijnroks L, Vermeer A. Qualitative analysis of therapeutic motor intervention programmes for children with cerebral palsy: an update. Dev Med Child Neurol. 2002;44 :593 603[CrossRef][Web of Science][Medline]
PEDIATRICS (ISSN 1098-4275). ©2006 by the American Academy of Pediatrics
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||




