The Review Process Fails to Require Appropriate Statistical Analysis of a Group-Randomized Trial

To the Editor.—

We are surprised that the journal’s review process of the school-randomized trial reported by MacKelvie et al¹ did not insist on an analysis appropriate to the group-randomized design or at least require stronger justification of the assumptions involved in ignoring the randomization design in the analysis.

Randomizing intact social groups is a common approach outside the clinic because it is often easier, and possibly only feasible, to intervene with a whole class, troop, church, or community rather than to work with individuals. Members within intact social groups tend to be more like each other than they are like members in other groups, making for some redundancy of information and increased variance compared with the same number of subjects individually randomized. As a general rule, group-randomized trials that are analyzed by using methods appropriate for individual-level trials will overestimate the significance of the effects.

Twenty-five years ago, Cornfield² warned clearly that “randomization by cluster accompanied by an analysis appropriate to randomization by individual is an exercise in self-deception and should be discouraged.” Methodological reviews (eg, Donner et al,^{3 Simpson et al,4 and Smith et al5}) show that Cornfield’s message is not well heeded and point to neglect in the review process for insisting on appropriate attention to the analytic issues incurred by the choice of a group-randomized trial. When randomization is by group but analyzed by individual, chance differences between the …