Reviews assessing the quality or the reporting of randomized controlled trials are increasing over time but raised questions about how quality is assessed

Dechartres, Agnes; Charles, Pierre; Hopewell, Sally; Ravaud, Philippe; Altman, Douglas G.

doi:10.1016/j.jclinepi.2010.04.015

« Previous Next »Journal of Clinical Epidemiology
Volume 64, Issue 2 , Pages 136-144, February 2011

Reviews assessing the quality or the reporting of randomized controlled trials are increasing over time but raised questions about how quality is assessed

Agnes Dechartres
- Centre for Statistics in Medicine, University of Oxford, Oxford, UK
- INSERM U738, Paris, France
- Université Paris 5 René Descartes, UFR de Médecine, Paris, France
- AP-HP, Hôpital Hôtel-Dieu, Centre d'Epidemiologie Clinique, Paris, France
- Corresponding author. Agnes Dechartres, Centre d'Epidemiologie Clinique, Hôpital Hôtel-Dieu, 1, place du parvis Notre-Dame, 75004 Paris, France. Tel.: +33-1-44-07-39-75; fax: +33-1-42-34-87-90.
Pierre Charles
- INSERM U738, Paris, France
Sally Hopewell
- Centre for Statistics in Medicine, University of Oxford, Oxford, UK
- UK Cochrane Centre, Summertown Pavilion, Oxford, UK
Philippe Ravaud
- INSERM U738, Paris, France
- Université Paris 5 René Descartes, UFR de Médecine, Paris, France
- AP-HP, Hôpital Hôtel-Dieu, Centre d'Epidemiologie Clinique, Paris, France
Douglas G. Altman
- Centre for Statistics in Medicine, University of Oxford, Oxford, UK

Accepted 23 April 2010. published online 13 August 2010.

Abstract

Objective

Many reviews specifically aimed to assess the quality of randomized controlled trials (RCTs). We evaluated the quality of reporting in such reviews.

Study Design and Setting

PubMed and the Cochrane library were searched for all reviews assessing the quality of RCTs between 1987 and 2007, and experts in the field were also contacted.

Results

We found 177 reviews published from 1987 to 2007, 58% of which were published after 2002. Of these, 131 (74%) focused on the quality of RCTs, 44 (25%) on quality of reporting, and 2 (1%) assessed both. The search strategy was well reported (92%). The criteria for assessment were reported in 97% of the reviews but were defined in only 38%. Seventy-four different items and 26 different scales were identified. Allocation sequence generation and concealment were reported in 41% and 40%, respectively, but their adequacy was assessed in 20% and 29%, respectively; scales were used in 40% and Consolidated Standards of Reporting Trials (CONSORT) checklist in 12%.

Conclusion

The number of methodological reviews has dramatically increased in recent years. Despite an improved reporting of the methodology, how quality is assessed still raises important issues. Heterogeneity of criteria used and lack of definition may limit the relevance of these reviews.

Keywords: Quality, Randomized controlled trials, Bias, Internal validity, Systematic review, Reporting