eLetters to:

ARTICLE: Alejandro Hoberman, Ellen R. Wald, Robert W. Hickey, Marc Baskin, Martin Charron, Massoud Majd, Diana H. Kearney, Ellen A. Reynolds, Jerry Ruley, and Janine E. Janosky Oral Versus Initial Intravenous Therapy for Urinary Tract Infections in Young Febrile Children Pediatrics 1999; 104: 79-86 [Abstract] [Full text] [PDF]

eLetters: Submit a response to this article

eLetters published:

Statistical conundrum

Jay D Fisher   (14 July 1999)

Declaring the minimum detectable difference

Alex Okun   (11 August 1999)

Response to Dr. Fisher''s Letter

Alejandro Hoberman, Ellen Wald   (3 September 1999)

Response to Dr. Okun''s Letter

Alejandro Hoberman, Ellen Wald   (3 September 1999)

Statistical conundrum 14 July 1999
Jay D Fisher, pediatric emergency physician University of Nevada School of Medicine Send letter to journal: Re: Statistical conundrum jdfisher{at}earthlink.net Jay D Fisher	Dear sirs: Hoberman and colleagues are to be congratulated on their publication in this month's issue of Pediatrics. The research published by this group on the diagnosis and management of febrile UTI over the past decade has been consistently superb, and has 'raised the bar' for future researchers in this area. As a busy clinician, I have anxiously awaited the final results of this project, having read their intermediate results in an abstract a few years ago. I do have some concerns regarding the interpretation of one of the study's main endpoints, the incidence of renal scarring at 6 months by nuclear scan. The study revealed an unexpectedly low incidence of renal scarring in the IV therapy group of 7.2% (the authors anticipated a 30% incidence in their pre-hoc power calculation and wanted the power to detect a 15% increase). Because of this difference between anticipated and actual outcomes, the study can detect a 15% absolute increase in scarring, but clinically this difference between groups would be unacceptable (7.2% vs 22.2%). Looking at it another way, the study only has the power to detect an 85% relative increase in scarring over that of the IV group (7.2% vs 13.5%). To detect a 50% relative increase over IV therapy as the study originally intended (7.2% vs 10.8%), one would need over 300 patients per treatment group. If the risk of scarring in the oral group could be nearly twice that of the IV group, is it reasonable to embrace this practice? Sincerely, Jay D. Fisher MD FAAP Pediatric Emergency Services University of Nevada School of Medicine University Medical Center Las Vegas, NV
Declaring the minimum detectable difference 11 August 1999
Alex Okun, General pediatrician Department of Pediatrics, Montefiore Medical Center/Albert Einstein College of Medicine Send letter to journal: Re: Declaring the minimum detectable difference aokun{at}montefiore.org Alex Okun	To the Editor, Hoberman et al found no greater rates of renal scarring in young children after oral antibiotic treatment of urinary tract infection than after intravenous treatment. They did not declare the minimum difference in the rate of this outcome that they had the power to detect between treatment groups, a critical part of the reporting of any negative study. This letter expands on the comments of Dr. Fisher ("Statistical conundrum", P3R, July 14, 1999) and offers different derivations and suggestions to the investigators. In their sample size projections, the investigators assumed that the rate of renal scarring six months after intravenous treatment would be 30%. An absolute difference of 15% (a 45% rate of scarring in the group treated orally) was felt to be "worth detecting". This difference in the rate of scarring could also have been expressed as an increase of 50% over intravenous treatment, or as a rate ratio of 1.5 (45% divided by 30%). With equal group sizes, at a power of 0.80 and a one-tailed significance level of 0.05, the investigators projected the need for at least 128 subjects in each arm of the randomized trial. They recruited and reported on 153 in each group. In the data set, 140 of the 153 subjects treated intravenously underwent DMSA scans roughly six months after infection. Only eleven of the 140, or 7.8%, had scans interpreted as showing renal scarring, far less than the 30% predicted. (This figure was reported in the paper as eleven of 153, or 7.2%, as thirteen subjects who did not have DMSA scans were included in the denominator and presumed not to have renal scarring.) At the levels of power and significance employed in their sample size projections, given the observed rate of scarring for the group treated intravenously of 7.8%, methods described by J. Cohen (Chapter 6, "Differences between proportions", in Statistical Power Analysis for the Behavioral Sciences, 2nd ed., Lawrence Erlbaum Associates, Hillsdale, NJ. 1988) can be used to derive the following: A. In order to have been able to detect a rate of renal scarring for the group treated with oral antibiotics as low as 11.7% that constituted a statistically significant difference (an increase in scarring of 50% over intravenous treatment, or a rate ratio of 1.5), the investigators would have needed to study at least 596 subjects in each group. B. With the group sizes of 153 used in the study, the investigators could not have detected a rate of renal scarring for the group treated orally of under 17.0% that would have constituted a statistically significant difference (an increase in scarring of 120% over intravenous treatment, or a rate ratio of 2.2). Research reports of "no difference" between treatment options should include both a description of sample size projections and a quantitative representation of the strength of the evidence obtained, expressed in terms of a minimum detectable difference in outcomes between groups. In this way, readers can judge for themselves whether the evidence is strong enough to warrant a change in clinical practice. This important study has the potential to influence the way many young children with urinary tract infection are treated. For the sake of clarity, Hoberman et al should declare that from their data set, at the chosen levels of power and significance, "no greater than a 2.2-fold difference in renal scarring six months after infection was detectable in the group treated orally compared to the group treated intravenously". Alex Okun, MD Division of General Pediatrics, Department of Pediatrics Montefiore Medical Center/Albert Einstein College of Medicine 1621 Eastchester Road, Bronx, NY 10461 Tel. 718-405-8094, Fax 718-405-8091 e-mail aokun@montefiore.org
Response to Dr. Fisher''s Letter 3 September 1999
Alejandro Hoberman University of Pittsburgh School of Medicine, Ellen Wald Send letter to journal: Re: Response to Dr. Fisher''s Letter alejo+{at}pitt.edu Alejandro Hoberman, et al.	We thank Dr. Fisher for his thoughtful and kind comments regarding our article. He is correct that our anticipated incidence rate, which was based on previously published reports and was used in the sample size calculation, was quite different from the rate actually observed in the study. We did not anticipate the lower rate of new renal scarring; accordingly we underestimated the size of the study. Using a post-hoc power estimate however, with the observed level of renal scarring in the IV therapy group of 7.2%, a 2.2 fold level difference could have been detected if this true difference existed with 153 patients per group (nondirectional alpha of 0.05 and power of 0.8). In other words, the incidence of renal scarring in the oral group would have to be more than double that in the IV group to be considered clinically and statistically significant. However, the minimal differences in rates of scarring actually detected were biologically and clinically insignificant, and almost certainly related to the fact that 3 of 4 patients with Grade IV vesicoureteral reflux were randomly assigned to the oral group. High- grade reflux is often associated with renal scarring and equal representation of these patients in the two treatment groups was not possible because the voiding cystourethrogram was performed 1 month after the index infection.
Response to Dr. Okun''s Letter 3 September 1999
Alejandro Hoberman University of Pittsburgh School of Medicine, Ellen Wald Send letter to journal: Re: Response to Dr. Okun''s Letter alejo+{at}pitt.edu Alejandro Hoberman, et al.	Dr. Okun comments expand on those made by Dr. Fisher regarding our recently published article. The estimated 30% incidence of renal scarring following acute pyelonephritis was based on studies by Rushton et al. These constituted the best available estimates, even though these studies were conducted (1) almost a decade earlier, (2) on hospitalized patients and (3) before the era of early widespread screening for UTI in young febrile children. All of our comments in response to Dr. Fisher's letter apply to Dr. Okun's concerns as well. The "appropriate" denominator for determining the incidence of renal scarring is 153. Approximately 50% of patients who did not have a DMSA scan performed at 6 months had a normal scan at study entry; all children with normal scans at entry -- regardless of treatment assignment -- had normal scans at follow-up. As stated in our report "there was no significant difference between treatment groups in the incidence of new renal scarring, when we included all children, only those who completed the study, or only those with documented APN at entry." The figures on the table are expressed as percentages that added to 100% for the purposes of clarity.