Published online January 4, 2006
PEDIATRICS
Vol. 117
No. 1
January 2006, pp.
261-262
(doi:10.1542/peds.2005-2145)
Pooling of Studies in Meta-analysis of Observational Research Leads to Precise but Spurious Results
C.W. Bollen, MD
Pediatric Intensive Care Unit
University Medical Centre
3508 AB Utrecht, Netherlands
M.O. Hoekstra, MD
H.G.M. Arets, MD
Department of Pulmonology
University Medical Centre and Wilhelmina Children's Hospital
3508 AB Utrecht, Netherlands
To the Editor.—
In a meta-analysis comparing 3 different treatment modalities of empyema in children, primary nonoperative therapy (defined as antibiotics and thoracentesis/chest-tube drainage), primary operative therapy (defined as antibiotics and either video-assisted thoracoscopic surgery or thoracotomy), and primary fibrinolytic therapy (defined as fibrinolytics either at the time of or shortly after tube thoracostomy), the authors concluded that the primary operative treatment was associated with a lower mortality and less treatment failures.1 Preferably, a meta-analysis evaluating different treatments would be performed on randomized trials. However, given the number of patients eligible, a randomized trial would be difficult to achieve. Observational studies run into obvious problems of different forms of bias. Although Avansino et al supplied a clear description of literature-search and study-selection criteria, an essential component of meta-analysis, an explicit and well-defined validity assessment of included studies, was missing. In a meta-analysis including observational research, validity assessment should receive the greatest attention. However, descriptions of how studies differed with respect to study design and possible sources of bias were not offered. Therefore, as a reader it was not possible to tell from their meta-analysis how problems with the validity of the included studies could influence the results or explain heterogeneity between studies. In our opinion, this made it difficult to asses the validity of the conclusions. For example, Avansino et al concluded that operative treatment was associated with a lower failure rate. This was based on a pooled estimate of 9 studies, 3 of which contributed 79.3% of the total weights assigned to the studies. However, 2 of these 3 studies were retrospective in nature, and in 1 study the primary operative treatment was compared with historical controls receiving nonoperative treatment. These studies could be subject to serious issues of confounding by indication, misclassification, and selection biases. Therefore, we argue that pooling of these studies was not justified. Stratification of studies according to disease-severity characteristics, for example, would have provided more insight into the nature and extend of bias.
In conclusion, we think that in the meta-analysis by Avansino et al, absence of an explicit and well-defined validity assessment of included studies was missing. In particular, in meta-analysis of observational research, just pooling of studies could lead to precise but spurious results.2 In meta-analyses of observational research, statistical pooling should not be a prominent feature, but investigating possible sources of heterogeneity should receive principal attention.
REFERENCES
- Avansino JR, Goldman B, Sawin RS, Flum DR. Primary operative versus nonoperative therapy for pediatric empyema: a meta-analysis.
Pediatrics. 2005;115
:1652
–1659[Abstract/Free Full Text]
- Egger M, Davey Smith G, Altman DG.
Systematic Reviews in Health Care. 2nd ed. London, United Kingdom: BMJ Books; 2001
PEDIATRICS (ISSN 1098-4275). ©2006 by the American Academy of Pediatrics
