Post-publication Peer Review (P3R) is an online forum for ongoingreview peer review. To submit a P3R please go to the article you wish to respond to and click on the link that reads "P3Rs: Submit a Response." Submission of P3Rs are open to all health care professionals and experts in related fields.

Post-publication Peer Reviews to:

ELECTRONIC ARTICLE:
Tim Lynch, Robert Platt, Serge Gouin, Charles Larson, and Yves Patenaude
Can We Predict Which Children With Clinically Suspected Pneumonia Will Have the Presence of Focal Infiltrates on Chest Radiographs?
Pediatrics 2004; 113: e186-e189 [Abstract] [Full text] [PDF]
*P3Rs: Submit a response to this article

P3Rs published:

[Read P3R] The Challenge of Predicting Positive Chest Radiographs in Children with Suspected Pneumonia
Simon J. Hambidge, Diane Fairclough   (9 April 2004)

The Challenge of Predicting Positive Chest Radiographs in Children with Suspected Pneumonia 9 April 2004
Top
Simon J. Hambidge,
Associate Professor of Pediatrics
University of Colorado School of Medicine & Denver Health Medical Center,
Diane Fairclough

Send letter to journal:
Re: The Challenge of Predicting Positive Chest Radiographs in Children with Suspected Pneumonia

simon.hambidge{at}uchsc.edu Simon J. Hambidge, et al.

In the electronic pages of the March 2004 issue of Pediatrics, Lynch and colleagues derive a clinical prediction rule for the presence of focal infiltrates in children who present with clinically suspected pneumonia in a pediatric emergency department.(1) Despite the authors’ claim that “the prospective evaluation of this multivariate prediction rule in a clinical setting is still required,” limitations in the derivation of their rule make it unlikely that the resulting rule will be applicable in a clinical setting.

The prediction rule of Lynch et. al. has an area under the receiver- operator characteristic curve (AUROC) of 0.668. The receiver-operator characteristic curve plots sensitivity against 1 minus the specificity. Given that an AUC of 0.5 represents random chance, and that an AUC of 1.0 represents a model that will correctly predict the desired outcome 100% of the time, and AUC of 0.67 is not much better than a random coin toss. In general, one wants an AUC (or a c-index) that is “far above 0.5,”(2) in general at least 0.75 – 0.8, before one feels that a clinical prediction rule will be robust enough for external validation studies.

The high prevalence of radiographic pneumonia in this study population (36%) will make it difficult to apply in most clinical settings. For instance, in pediatric emergency departments in the United States, the prevalence of positive chest radiographs in children suspected of pneumonia has ranged between 7 and 19 percent.(3) We calculated the positive predictive value of this prediction rule, using the data presented in “model 7,” at 37%. If the prevalence of radiographically confirmed pneumonia were 10% instead of 36%, the positive predictive value would drop to from 37% to 10.5%.

A robust measure of any diagnostic test, be it a single laboratory test or a clinical prediction rule, is the likelihood ratio (LR), which is the probability of the test result in patients with disease (focal infiltrates) divided by the probability of the same finding in patients without disease. Using model 7 as presented in Lynch et. al., the LR of a positive test is 1.07, which lacks any diagnostic value. The LR for a negative test is 0.26, which implies a decrease in the probability of infiltrate on chest X-ray of between 25-30%.(4)

A number of more technical limitations also apply to this prediction rule. The final model was derived using backward stepwise logistic regression. It has been demonstrated that such a model will present an overly optimistic performance of the final rule,(5) and therefore the model needs to be internally validated before any consideration is given to testing on the rule in new populations. Techniques for this internal validation include split-sample, cross-validation, and bootstrapping methods.(6) Such testing of the reproducibility of the predictive variables has been recommended as a methodological standard for clinical prediction rules.(7)

In adults, the limitations of clinical prediction rules in the diagnosis of pneumonia has been demonstrated.(8) Taken in sum, the limitations in the pediatric prediction rule of Lynch et. al. would imply that the answer to the question in the authors’ title (“Can we predict which children with clinically suspected pneumonia will have the presence of focal infiltrates on chest radiographs?”) is “no,” at least not with the data presented.

References

1. Lynch T, Platt R, Gouin S, Larson C, Patenaude Y. Can we predict which children with clinically suspected pneumonia will have the presence of focal infiltrates on chest radiographs? Pediatrics 2004;113:e186-e189. URL: http://www.pediatrics.org/cgi/content/full/113/3/e186.

2. Braitman LE, Davidoff F. Predicting clinical states in individual patients. Ann Intern Med 1996;125:406-412.

3. Margolis P, Gadomski A. Does this infant have pneumonia? JAMA 1998;279:308-313.

4. McGee S. Simplifying likelihood ratios. J Gen Intern Med 2002;17:646-649.

5. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361-387.

6. Steyerberg EW, Harrell FE Jr, Borsboom GJJM, Eijkemans MJC, Vergouwe Y, Habbema JDF. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. Journal Clinical Epidemiology 2001;54:774-781.

7. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA 1997;277:488-494.

8. Metlay JP, Kapoor WN, Fine MJ. Does this patient have community- acquired pneumonia? Diagnosing pneumonia by history and physical examination. JAMA 1997;278:1440-1445.