Journal of Clinical Epidemiology
Volume 64, Issue 2 , Pages 136-144 , February 2011

Reviews assessing the quality or the reporting of randomized controlled trials are increasing over time but raised questions about how quality is assessed

  • Agnes Dechartres

      Affiliations

    • Centre for Statistics in Medicine, University of Oxford, Oxford, UK
    • INSERM U738, Paris, France
    • Université Paris 5 René Descartes, UFR de Médecine, Paris, France
    • AP-HP, Hôpital Hôtel-Dieu, Centre d'Epidemiologie Clinique, Paris, France
    • Corresponding Author InformationCorresponding author. Agnes Dechartres, Centre d'Epidemiologie Clinique, Hôpital Hôtel-Dieu, 1, place du parvis Notre-Dame, 75004 Paris, France. Tel.: +33-1-44-07-39-75; fax: +33-1-42-34-87-90.
    • ,
  • Pierre Charles

      Affiliations

    • INSERM U738, Paris, France
    • ,
  • Sally Hopewell

      Affiliations

    • Centre for Statistics in Medicine, University of Oxford, Oxford, UK
    • UK Cochrane Centre, Summertown Pavilion, Oxford, UK
    • ,
  • Philippe Ravaud

      Affiliations

    • INSERM U738, Paris, France
    • Université Paris 5 René Descartes, UFR de Médecine, Paris, France
    • AP-HP, Hôpital Hôtel-Dieu, Centre d'Epidemiologie Clinique, Paris, France
    • ,
  • Douglas G. Altman

      Affiliations

    • Centre for Statistics in Medicine, University of Oxford, Oxford, UK

,Accepted 23 April 2010.

References 

  1. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001;134:663–694
  2. Jüni P, Altman DG, Egger M. Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ. 2001;323:42–46
  3. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135:982–989
  4. Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996;276:637–639
  5. Dunn HL. Application of statistical methods in physiology. Physiol Rev. 1929;9:275–398
  6. Bender JS, Halpern SH, Thangaroopan M, Jadad AR, Ohlsson A. Quality and retrieval of obstetrical anaesthesia randomized controlled trials. Can J Anaesth. 1997;44(1):14–18
  7. Solomon MJ, Laxamana A, Devore L, McLeod RS. Randomized controlled trials in surgery. Surgery. 1994;115:707–712
  8. Berlin JA. Does blinding of readers affect the results of meta-analyses? University of Pennsylvania Meta-analysis Blinding Study Group. Lancet. 1997;350:185–186
  9. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials. 1996;17(1):1–12
  10. Delgado-Rodriguez M, Ruiz-Canela M, De Irala-Estevez J, Llorca J, Martinez-Gonzalez A. Participation of epidemiologists and/or biostatisticians and methodological quality of published controlled clinical trials. J Epidemiol Community Health. 2001;55:569–572
  11. Verhagen AP, de Vet HC, de Bie RA, Kessels AG, Boers M, Bouter LM, et al. The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol. 1998;51:1235–1241
  12. Chalmers IHR, Haynes B. Reporting, updating, and correcting systematic reviews of the effects of health care. BMJ. 1994;302:862–865
  13. Maher CG, Sherrington C, Herbert RD, Moseley AM, Elkins M. Reliability of the PEDro scale for rating quality of randomized controlled trials. Phys Ther. 2003;83:713–721
  14. R Statistical Software. R Development Core Team R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2009;Available at http://www.R-project.orgAccessed April 2010
  15. Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet. 2005;365:1159–1162
  16. Boutron I, Tubach F, Giraudeau B, Ravaud P. Methodological differences in clinical trials evaluating nonpharmacological and pharmacological treatments of hip and knee osteoarthritis. JAMA. 2003;290:1062–1070
  17. Horton R. Surgical research or comic opera: questions, but few answers. Lancet. 1996;347:984–985
  18. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet. 1999;354:1896–1900
  19. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6(7):e1000100
  20. Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC. Meta-analyses of randomized controlled trials. N Engl J Med. 1987;316:450–455
  21. Sacks HS, Reitman D, Pagano D, Kupelnick B. Meta-analysis: an update. Mt Sinai J Med. 1996;63(3–4):216–224
  22. Oxman AD, Guyatt GH. Validation of an index of the quality of review articles. J Clin Epidemiol. 1991;44:1271–1278
  23. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10
  24. Egger M, Jüni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess. 2003;7(1):1–76
  25. Verhagen AP, de Vet HC, de Bie RA, Boers M, van den Brandt PA. The art of quality assessment of RCTs included in systematic reviews. J Clin Epidemiol. 2001;54:651–654
  26. Plint AC, Moher D, Morrison A, Schulz K, Altman DG, Hill C, et al. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? A systematic review. Med J Aust. 2006;185:263–267
  27. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995;16:62–73
  28. Olivo SA, Macedo LG, Gadotti IC, Fuentes J, Stanton T, Magee DJ. Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther. 2008;88:156–175
  29. Katrak P, Bialocerkowski AE, Massy-Westropp N, Kumar S, Grimmer KA. A systematic review of the content of critical appraisal tools. BMC Med Res Methodol. 2004;4:22
  30. Fenstein A. Clinimetrics. New Haven, CT: Yale University Press; 1987;
  31. Steiner D, Norman G. Health measurement scales: a practical guide to their development and use. Oxford, UK: Oxford University Press; 1989;
  32. Feinstein AR. Meta-analysis: statistical alchemy for the 21st century. J Clin Epidemiol. 1995;48:71–79
  33. Jüni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;282:1054–1060
  34. Pildal J, Hrobjartsson A, Jorgensen K, Hilden J, Altman D, Gotzsche P. Impact of allocation concealment on conclusions drawn from meta-analyses of randomized trials. Int J Epidemiol. 2007;
  35. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ. 2008;336:601–605
  36. Fourcade L, Boutron I, Moher D, Ronceray L, Baron G, Ravaud P. Development and evaluation of a pedagogical tool to improve understanding of a quality checklist: a randomised controlled trial. PLoS Clin Trials. 2007;2(5):e22
  37. Huwiler-Müntener K, Jüni P, Junker C, Egger M. Quality of reporting of randomized trials as a measure of methodologic quality. JAMA. 2002;287:2801–2804
  38. Higgins JPT, Altman DG, editors. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Chichester, UK: John Wiley & Sons; 2008. Available at www.cochrane-handbook.org. Accessed April 2010.

 Conflict of interest: none.

PII: S0895-4356(10)00180-0

doi: 10.1016/j.jclinepi.2010.04.015

Journal of Clinical Epidemiology
Volume 64, Issue 2 , Pages 136-144 , February 2011

Article Tools

* Image Usage & Resolution