Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time

Trikalinos, Thomas A.; Churchill, Rachel; Ferri, Marica; Leucht, Stefan; Tuunainen, Arja; Wahlbeck, Kristian; Ioannidis, John P.A.; for the EU-PSI project,

doi:10.1016/j.jclinepi.2004.02.018

« Previous Next »Journal of Clinical Epidemiology
Volume 57, Issue 11 , Pages 1124-1130, November 2004

Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time

Thomas A. Trikalinos
- Institute for Clinical Care Research and Health Policy Studies, Tufts-New England Medical Center, Tufts University School of Medicine, Boston, MA, USA
- Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece
Rachel Churchill
- Health Services Research Department, Institute of Psychiatry, London, United Kingdom
Marica Ferri
- Department of Epidemiology of Lazio Region, Rome, Italy
Stefan Leucht
- Department of Psychiatry and Psychotherapy, Technische Universität München, Munich, Germany
Arja Tuunainen
- Department of Psychiatry, University of Helsinki, Lapinlahti Hospital, Helsinki, Finland
Kristian Wahlbeck
- STAKES National Research and Development Center for Welfare and Health, Helsinki, Finland
John P.A. Ioannidis
- Institute for Clinical Care Research and Health Policy Studies, Tufts-New England Medical Center, Tufts University School of Medicine, Boston, MA, USA
- Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina 45110, Greece
- Corresponding author. Tel: +30-2651-097807; fax: +30-2651-097867.
for the EU-PSI project
- EU-PSI project membership is listed in the Acknowledgments.

Accepted 13 February 2004.

Abstract

Background and objective

Meta-analyses of randomized trials may incorporate new evidence, and estimated treatment effects change over time. We evaluated whether the certainty and estimates of efficacy and tolerability of mental health interventions change over time, as more trials appear on the same topics.

Methods

One hundred meta-analyses (1,024 trial entries; 99,303 participants) with an outcome of death, relapse, failure or dropout and with five or more trials published in three or more different years were examined with cumulative meta-analysis and recursive cumulative meta-analysis.

Results

Eight meta-analyses reached formal statistical significance (P < .05) at some point, but lost this significance eventually when more trials were published; typically large effect sizes in early trials were dissipated with further evidence. With 500 randomized subjects, 95% of the time, subsequent changes in odds ratio might be up to 1.5-fold. For death, relapse, and failure outcomes, a decrease in effect size was somewhat more common than an increase, when more data became available (157 vs. 125, P=.06). This was most clear for comparisons of pharmacotherapies versus placebo (79 vs. 51, P=.009).

Conclusions

Evidence based on a small number of randomized subjects should be interpreted cautiously. Early treatment efficacy of pharmacotherapies is occasionally overestimated.

Keywords: Meta-analysis, Recursive cumulative meta-analysis, Efficacy, Bias, Mental health, Effect size