Publication bias was not a good reason to discourage trials with low power
Accepted 29 February 2008. published online 14 July 2008.
Abstract
Objective
The objective was to investigate whether it is justified to discourage trials with less than 80% power. Trials with low power are unlikely to produce conclusive results, but their findings can be used by pooling then in a meta-analysis. However, such an analysis may be biased, because trials with low power are likely to have a nonsignificant result and are less likely to be published than trials with a statistically significant outcome.
Study Design and Setting
We simulated several series of studies with varying degrees of publication bias and then calculated the “real” one-sided type I error and the bias of meta-analyses with a “nominal” error rate (significance level) of 2.5%.
Results
In single trials, in which heterogeneity was set at zero, low, and high, the error rates were 2.3%, 4.7%, and 16.5%, respectively. In multiple trials with 80%–90% power and a publication rate of 90% when the results were nonsignificant, the error rates could be as high as 5.1%. When the power was 50% and the publication rate of non-significant results was 60%, the error rates did not exceed 5.3%, whereas the bias was at most 15% of the difference used in the power calculation.
Conclusion
The impact of publication bias does not warrant the exclusion of trials with 50% power.
aDepartment of Epidemiology and Biostatistics, Radboud University Nijmegen Medical Centre, the Netherlands
bDepartment of Endocrinology, Radboud University Nijmegen Medical Centre, the Netherlands
Corresponding author.Radboud University Nijmegen Medical Centre, Department of Epidemiology and Biostatistics 133, P.O. Box 9101, NL-6500 HB Nijmegen, The Netherlands. Tel.: +31-243617667; fax: +31-243613505.
ABSTRACT