Attention should be given to multiplicity issues in systematic reviews
Accepted 7 March 2008.
Abstract
Objective
The objective of this paper is to describe the problem of multiple comparisons in systematic reviews and to provide some guidelines on how to deal with it in practice.
Study Design and Setting
We describe common reasons for multiplicity in systematic reviews, and present some examples. We provide guidance on how to deal with multiplicity when it is unavoidable.
Results
We identified six common reasons for multiplicity in systematic reviews: multiple outcomes, multiple groups, multiple time points, multiple effect measures, subgroup analyses, and multiple looks at accumulating data. The existing methods to deal with multiplicity in single trials can not always be applied in systematic reviews.
Conclusion
There is no simple and completely satisfactory solution to the problem of multiple comparisons in systematic reviews. More research is required to develop multiple comparison procedures for use in systematic reviews. Authors and consumers of systematic reviews should give serious attention to multiplicity in systematic reviews when presenting, interpreting and using the results of these reports.
aInstitute for Quality and Efficiency in Health Care, Dillenburger Street 27, Cologne D-51105, Germany
bFaculty of Medicine, University of Cologne, Joseph-Stelzmann-Street 20, Cologne D-50931, Germany
cResearch & Development, Moorfields Eye Hospital, City Road, London EC1V 2PD, UK
dUK Cochrane Centre, Summertown Pavilion, Middle Way, Oxford OX2 7LG, UK
eWarwick Clinical Trials Unit, Warwick Medical School, Coventry CV4 7AL, UK
fDepartment of Anesthesiology, University of Utah, 30 North 1900 East, Salt Lake City, Utah 84132, USA
gCopenhagen Trial Unit, Copenhagen University Hospital, Blegdamsvej 9, Copenhagen DK-2100, Denmark
Corresponding author. Department of Medical Biometry, Institute for Quality and Efficiency in Health Care, Dillenburger Street 27, D–51105 Cologne, Germany. Tel.: +49-221-35685-451; fax: +49-221-35685-891.
ABSTRACT