 |
|
 |
|
 |
 |
|
 |
 |
 |
 |
 |
 |
 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
PEDIATRICS Vol. 108 No. 4 October 2001, pp. 1047
To the Editor.
Garrison and Christakis1 justify their systematic
review by pointing at some presumed methodologic shortcomings of our
systematic review, published in 1998.2 In fact, the conclusion of their review differs only slightly from ours, and we
wonder if these differences can only be attributed to flaws in ours. We
therefore would like to comment on the reasons why our review
(according to the introduction of Garrison and Christakis' review) was
thought to be inappropriate.
First, one can always argue whether studies are sufficiently
homogeneous to be pooled; however, we don't agree that we actually pooled inhomogeneous studies and ask reader to compare. Second, we did
pool different outcome measures, using effect sizes. Given the lack of
uniformity in outcome measures, we considered this the most informative
approach. Although we agree that effect sizes are difficult to
translate to the clinical setting, they are often used in systematic
reviews and are also described in the handbook of the Cochrane
Collaboration, the leading organization in this field.3
Finally, after checking the studies reviewed by Garrison and
Christakis,1 we actually don't see any study that we
missed, as they suggested. From a methodologic point of view, we would
like to make a few comments on Garrison and Christakis' review. First,
they did not search the Embase database, which includes many
European-language journals not indexed by Medline.4
Second, the authors did not evaluate trial quality using an independent
assessment, nor did they use a validated scale.5 Third,
the numbers needed to treat (NNT), presented in Table 3, are provided
without confidence intervals, thus hindering objective interpretation
by the reader. In addition, presenting NNT based on different
definitions of "cure" actually is the same sort of standardization
that they reject in our use of effect sizes.
Systematic reviews are considered the best evidence available.
Therefore, we should aim at concordance between reviews. We propose to
collaborate on a joint updated review for the Cochrane Collaboration,
resolving the discordance in methods and results.6
Akkerroosstraat 18
5761 EX, Bakel, The Netherlands
REFERENCES
resolving discordant meta-analysis.
JAMA.
1996;
275:308-314 [Abstract]
In Reply.
We appreciate the comments of Lucassen and Assendelft in response to our systematic review of treatments for infant colic.
Regarding the issue of homogeneity, we respectfully disagree with the authors' contention that heterogeneous studies were not pooled in their meta-analysis.1 In fact, the meta-analysis by Lucassen et al appeared to have problems with heterogeneity on several levels. This issue, among others, was originally raised in a letter to the editor after publication of the meta-analysis.2 The meta-analysis pooled data from studies concerning heterogeneous interventions, such as hypoallergenic formulas and dietary modifications in breastfeeding mothers. Second, data concerning heterogeneous outcomes (such as duration of crying and presence of colic) were pooled using effect sizes, without detailing which outcomes were selected for analysis from each study and why. This is problematic because it further hinders interpretation of the effect sizes. Additionally, meta-analyses commonly include statistical tests to determine whether statistically significant heterogeneity is present in the data before pooling; their article did not state if such tests were performed. Although the authors used a random-effects model in their meta-analysis, there remains considerable controversy as to whether this statistical method can adequately compensate for heterogeneity among the data of pooled studies.3,4
We are pleased that the authors agree that effect sizes lack clinical applicability, despite being adequate tests of statistical significance. Perhaps authors of future studies will take this into account when performing meta-analyses. The primary goal of our review was to provide clinically relevant data for practitioners; hence, we did not consider effect sizes to be useful or informative. Although our NNT are calculated based on differing outcomes, these are clearly stated in the article, as is the fact that the NNT cannot therefore be directly compared.
The differences between the studies included in their meta-analysis and our systematic review are as follows: there were 3 articles that appeared in our analysis but not in theirs,5-7 and 8 articles that appeared in theirs but did not meet our inclusion criteria.8-15 The additional articles in our analysis were all published in 1997 or later, and were likely not included in their literature search. However, these articles do contain 2 interventions not studied in the meta-analysis: sucrose5,7 and intensive parental counseling.6 There were 2 articles8,11 that appeared in their article but not in our literature search; neither article is included in the Medline or Embase databases. Of the 6 remaining articles that appeared in their article but not in ours, 2 were in a language other than English,14,15 2 were comparative trials without placebo controls,10,12 and 2 were not adequately randomized.9,13 A posthoc search of Embase using the same search criteria discovered no additional articles that would have met our inclusion criteria.
We did, in fact, limit our search to the English language, because of resource limitations, and acknowledge this as a potential weakness of our review, although it must also be noted that colic is a culturally defined phenomenon and studies from non-English language countries may or may not have applicability elsewhere. Contrary to the authors' assertion, however, the articles included in our review had indeed passed a quality review, utilizing the criteria for randomized controlled trials presented in JAMA's "Users' guides to the medical literature."16 We did not employ quality scores in our review, as the utility of these scales is considered questionable.4,17
Child Health Institute
University of Washington
Seattle, WA 98103
REFERENCES
a
remedy for severe infantile colic.
Lakartidningen.
1983;
80:116-118 [Medline]
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
||
|
||
Home | My Pediatrics | Journal Information | Current Issue | Past Issues | Subscriptions & Services | Contact Us Click here for faster international access © 2001 American Academy of Pediatrics. All rights reserved. |
||
|
||
