Advertisement

Different control conditions can produce different effect estimates in psychotherapy trials for depression

  • Author Footnotes
    1 Equal contribution.
    Ioannis Michopoulos
    Correspondence
    Corresponding author. Eating Disorders Unit, 2nd Department of Psychiatry, Medical School, National and Kapodistrian University of Athens, “Attikon” University Hospital, Rimini 1, 12462, Athens, Greece. Tel: +30 2105832430; fax: +302105326453.
    Footnotes
    1 Equal contribution.
    Affiliations
    Eating Disorders Unit, 2nd Department of Psychiatry, Medical School, National and Kapodistrian University of Athens, ‘Attikon’ University Hospital, Athens, Greece
    Search for articles by this author
  • Author Footnotes
    1 Equal contribution.
    Toshi A. Furukawa
    Footnotes
    1 Equal contribution.
    Affiliations
    Departments of Health Promotion and Human Behavior and of Clinical Epidemiology, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan
    Search for articles by this author
  • Hisashi Noma
    Affiliations
    Department of Data Science, The Institute of Statistical Mathematics, Tokyo, Japan
    Search for articles by this author
  • Sanae Kishimoto
    Affiliations
    Department of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine / School of Public Health, Kyoto, Japan
    Search for articles by this author
  • Akira Onishi
    Affiliations
    Department of Rheumatology and Clinical Immunology, Kobe University Graduate School of Medicine, Hyogo, Japan
    Search for articles by this author
  • Edoardo G. Ostinelli
    Affiliations
    Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK; Department of Psychiatry, University of Oxford, Oxford, UK
    Search for articles by this author
  • Marketa Ciharova
    Affiliations
    Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
    Search for articles by this author
  • Clara Miguel
    Affiliations
    Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
    Search for articles by this author
  • Eirini Karyotaki
    Affiliations
    Department of Global Health and Social Medicine, Harvard Medical School, Boston MA, USA

    Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
    Search for articles by this author
  • Pim Cuijpers
    Affiliations
    Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
    Search for articles by this author
  • Author Footnotes
    1 Equal contribution.
Published:December 14, 2020DOI:https://doi.org/10.1016/j.jclinepi.2020.12.012

      Highlights

      • All control conditions (PillPlacebo, NoTreatment, and PsycholPlacebo) had higher odds ratios for response against Waiting List.
      • The effect sizes obtained in RCTs must be interpreted differently depending on the control condition used.
      • Weaker controls could be used only in the earlier phases of therapy evaluations, but stronger controls should be used in confirmatory studies.
      • Different control conditions produce different effect sizes in psychotherapy randomized controlled trials for depression when they are all considered simultaneously in network meta-analysis : control conditions should no longer be lumped into one comparison group.

      Abstract

      Objectives

      Control conditions’ influence on effect estimates of active psychotherapeutic interventions for depression has not been fully elucidated. We used network meta-analysis to estimate the differences between control conditions.

      Study Design and Setting

      We have conducted a comprehensive literature search of randomized trials of psychotherapies for adults with depression up to January 1, 2019 in four major databases (PubMed, PsycINFO, Embase, and Cochrane). The network meta-analysis included broadly conceived cognitive behavior therapies in comparison with the following control conditions: Waiting List (WL), No Treatment (NT), Pill Placebo (PillPlacebo), Psychological Placebo (PsycholPlacebo).

      Results

      123 studies with 12,596 participants were included. The I-squared was 55.9% (95% CI: 45.9%; to 64.0%) (moderate heterogeneity). The design-by-treatment global test of inconsistency was not significant (P = 0.44). Different control conditions led to different estimates of efficacy for the same intervention. WL appears to be the weakest control (odds ratio of response against NT = 1.93 (1.30 to 2.86), PsycholPlacebo = 2.03 (1.21 to 3.39), and PillPlacebo = 2.66 (1.45 to 4.89), respectively).

      Conclusions

      Different control conditions produce different effect estimates in psychotherapy randomized controlled trials for depression. WL was the weakest, followed by NT, PsycholPlacebo, and PillPlacebo in this order. When conducting meta-analyses of psychotherapy trials, different control conditions should not be lumped into a single group.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • World Health Organization
        Depression and other common mental disorders: Global Health Estimates 2017. World Health Organization, Geneva2017 (CC BY-NC-SA 3.0 IGO)
        • GBD 2015 DALYs and HALE Collaborators
        Global, regional, and national disability-adjusted life-years (DALYs) for 315 diseases and injuries and healthy life expectancy (HALE), 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015.
        Lancet. 2016; 388: 1603-1658
        • Ferrari A.J.
        • Charlson F.J.
        • Norman R.E.
        • Patten S.B.
        • Freedman G.
        • Murray C.J.L.
        • et al.
        Burden of depressive disorders by country, sex, age, and year: findings from the global burden of disease study 2010.
        PLoS Med. 2013; 10: e1001547
        • Lim G.Y.
        • Tam W.W.
        • Lu Y.
        • Ho C.S.
        • Zhang M.W.
        • Ho R.C.
        Prevalence of depression in the Community from 30 countries between 1994 and 2014.
        Sci Rep. 2018; 8: 2861
        • Lucht M.
        • Schaub R.T.
        • Meyer C.
        • Hapke U.
        • Rumpf H.J.
        • Bartels T.
        • et al.
        Gender differences in unipolar depression: a general population survey of adults between age 18 to 64 of German nationality.
        J Affect Disord. 2003; 77: 203-211
        • Murray C.J.L.
        • Vos T.
        • Lozano R.
        • Naghavi M.
        • Flaxman A.D.
        • Michaud C.
        • et al.
        Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010.
        Lancet. 2012; 380: 2197-2223
        • Cuijpers P.
        • Noma H.
        • Karyotaki E.
        • Vinkers C.H.
        • Cipriani A.
        • Furukawa T.A.
        A network meta-analysis of the effects of psychotherapies, pharmacotherapies and their combination in the treatment of adult depression.
        World Psychiatry. 2020; 19: 92-107
        • Chambless D.L.
        • Hollon S.D.
        Defining empirically supported therapies.
        J Consult Clin Psychol. 1998; 66: 7-18
        • Clark D.M.
        Implementing NICE guidelines for the psychological treatment of depression and anxiety disorders: the IAPT experience.
        Int Rev Psychiatry. 2011; 23: 318-327
        • Cuijpers P.
        • Karyotaki E.
        • de Wit L.
        • Ebert D.D.
        The effects of fifteen evidence-supported therapies for adult depression: a meta-analytic review.
        Psychother Res. 2020; 30: 279-293
        • Bellg A.J.
        • Borrelli B.
        • Resnick B.
        • Hecht J.
        • Minicucci D.S.
        • Ory M.
        • et al.
        Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium.
        Health Psychol. 2004; 23: 443-451
        • Mohr D.C.
        • Ho J.
        • Hart T.L.
        • Baron K.G.
        • Berendsen M.
        • Beckner V.
        • et al.
        Control condition design and implementation features in controlled trials: a meta-analysis of trials evaluating psychotherapy for depression.
        Transl Behav Med. 2014; 4: 407-423
        • Mohr D.C.
        • Spring B.
        • Freedland K.E.
        • Beckner V.
        • Arean P.
        • Hollon S.D.
        • et al.
        The selection and design of control conditions for randomized controlled trials of psychological interventions.
        Psychother Psychosom. 2009; 78: 275-284
        • Barth J.
        • Munder T.
        • Gerger H.
        • Nüesch E.
        • Trelle S.
        • Znoj H.
        • et al.
        Comparative efficacy of seven psychotherapeutic interventions for patients with depression: a network meta-analysis.
        Plos Med. 2013; 10: e1001454
        • Gold S.M.
        • Enck P.
        • Hasselmann H.
        • Friede T.
        • Hegerl U.
        • Mohr D.C.
        • et al.
        Control conditions for randomised trials of behavioural interventions in psychiatry: a decision framework.
        Lancet Psychiatry. 2017; 4: 725-732
        • Khan A.
        • Kolts R.L.
        • Rapaport M.H.
        • Krishnan K.R.R.
        • Brodhead A.E.
        • Browns W.A.
        Magnitude of placebo response and drug-placebo differences across psychiatric disorders.
        Psychol Med. 2005; 35: 743-749
        • Furukawa T.A.
        • Cipriani A.
        • Atkinson L.Z.
        • Leucht S.
        • Ogawa Y.
        • Takeshima N.
        • et al.
        Placebo response rates in antidepressant trials: a systematic review of published and unpublished double-blind randomised controlled studies.
        Lancet Psychiatry. 2016; 3: 1059-1066
        • Hegerl U.
        • Allgaier A.-K.
        • Henkel V.
        • Mergl R.
        Can effects of antidepressants in patients with mild depression be considered as clinically significant?.
        J Affect Disord. 2012; 138: 183-191
        • Furukawa T.A.
        • Noma H.
        • Caldwell D.M.
        • Honyashiki M.
        • Shinohara K.
        • Imai H.
        • et al.
        Waiting list may be a nocebo condition in psychotherapy trials: a contribution from network meta-analysis.
        Acta Psychiatr Scand. 2014; 130: 181-192
        • Wampold B.E.
        • Mondin G.W.
        • Moody M.
        • Stich F.
        • Benson K.
        • Ahn H.
        A meta-analysis of outcome studies comparing bona fide psychotherapies: Empiricially, “all must have prizes.
        Psychol Bull. 1997; 122: 203-215
        • Munder T.
        • Flückiger C.
        • Leichsenring F.
        • Abbass A.A.
        • Hilsenroth M.J.
        • Luyten P.
        • et al.
        Is psychotherapy effective? A re-analysis of treatments for depression.
        Epidemiol Psychiatr Sci. 2019; 28: 268-274
        • Cuijpers P.
        • van Straten A.
        • Bohlmeijer E.
        • Hollon S.D.
        • Andersson G.
        The effects of psychotherapy for adult depression are overestimated: a meta-analysis of study quality and effect size.
        Psychol Med. 2010; 40: 211-223
        • Cipriani A.
        • Higgins J.P.T.
        • Geddes J.R.
        • Salanti G.
        Conceptual and technical challenges in network meta-analysis.
        Ann Intern Med. 2013; 159: 130-137
        • Cuijpers P.
        • Turner E.H.
        • Mohr D.C.
        • Hofmann S.G.
        • Andersson G.
        • Berking M.
        • et al.
        Comparison of psychotherapies for adult depression to pill placebo control groups: a meta-analysis.
        Psychol Med. 2014; 44: 685-695
        • Furukawa T.A.
        • Schramm E.
        • Weitz E.S.
        • Salanti G.
        • Efthimiou O.
        • Michalak J.
        • et al.
        Cognitive-Behavioural Analysis System of Psychotherapy (CBASP), a drug, or their combination: differential therapeutics for persistent depressive disorder: a study protocol of an individual participant data network meta-analysis.
        BMJ Open. 2016; 6: e011769
        • Hutton B.
        • Salanti G.
        • Caldwell D.M.
        • Chaimani A.
        • Schmid C.H.
        • Cameron C.
        • et al.
        The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations.
        Ann Intern Med. 2015; 162: 777-784
        • Watts S.E.
        • Turnell A.
        • Kladnitski N.
        • Newby J.M.
        • Andrews G.
        Treatment-as-usual (TAU) is anything but usual: a meta-analysis of CBT versus TAU for anxiety and depression.
        J Affect Disord. 2015; 175: 152-167
        • Witt K.
        • de Moraes D.P.
        • Salisbury T.T.
        • Arensman E.
        • Gunnell D.
        • Hazell P.
        • et al.
        Treatment as usual (TAU) as a control condition in trials of cognitive behavioural-based psychotherapy for self-harm: impact of content and quality on outcomes in a systematic review.
        J Affect Disord. 2018; 235: 434-447
        • Cuijpers P.
        • Quero S.
        • Papola D.
        • Cristea I.A.
        • Karyotaki E.
        Care-as-usual control groups across different settings in randomized trials on psychotherapy for adult depression: a meta-analysis.
        Psychol Med. 2019; : 1-11
        • Wampold B.E.
        • Budge S.L.
        • Laska K.M.
        • Del Re A.C.
        • Baardseth T.P.
        • Fluckiger C.
        • et al.
        Evidence-based treatments for depression and anxiety versus treatment-as-usual: a meta-analysis of direct comparisons.
        Clin Psychol Rev. 2011; 31: 1304-1312
        • Cuijpers P.
        • van Straten A.
        • Andersson G.
        • van Oppen P.
        Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies.
        J Consult Clin Psychol. 2008; 76: 909-922
        • da Costa B.R.
        • Nüesch E.
        • Rutjes A.W.
        • Johnston B.C.
        • Reichenbach S.
        • Trelle S.
        • et al.
        Combining follow-up and change data is valid in meta-analyses of continuous outcomes: a meta-epidemiological study.
        J Clin Epidemiol. 2013; 66: 847-855
        • Bakbergenuly I.
        • Hoaglin D.C.
        • Kulinskaya E.
        Pitfalls of using the risk ratio in meta-analysis.
        Res Synth Methods. 2019; 10: 398-419
        • Hedges L.
        • Olkin I.
        Statistical methods in meta-analysis.
        Stat Med. 1985; 20https://doi.org/10.2307/1164953
        • Cuijpers P.
        • Karyotaki E.
        • Ciharova M.
        A meta-analytic database of randomised trials on psychotherapies for depression.
        2019https://doi.org/10.17605/OSF.IO/825C6
        • Higgins J.P.
        • Green S.
        Cochrane Handbook for Systematic Reviews of Interventions. vol. 5.1.0.
        (Available at)
        • Furukawa T.A.
        • Cipriani A.
        • Barbui C.
        • Brambilla P.
        • Watanabe N.
        Imputing response rates from means and standard deviations in meta-analyses.
        Int Clin Psychopharmacol. 2005; 20: 49-52
        • Chaimani A.
        • Salanti G.
        Using network meta-analysis to evaluate the existence of small-study effects in a network of interventions.
        Res Synth Methods. 2012; 3: 161-176
        • Rhodes K.M.
        • Turner R.M.
        • Higgins J.P.T.
        Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data.
        J Clin Epidemiol. 2015; 68: 52-60
        • Turner R.M.
        • Davey J.
        • Clarke M.J.
        • Thompson S.G.
        • Higgins J.P.
        Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews.
        Int J Epidemiol. 2012; 41: 818-827
        • Higgins J.P.T.
        • Jackson D.
        • Barrett J.K.
        • Lu G.
        • Ades A.E.
        • White I.R.
        Consistency and inconsistency in network meta-analysis: concepts and models for multi-arm studies.
        Res Synth Methods. 2012; 3: 98-110
        • Dias S.
        • Welton N.J.
        • Caldwell D.M.
        • Ades A.E.
        Checking consistency in mixed treatment comparison meta-analysis.
        Stat Med. 2010; 29: 932-944
        • Salanti G.
        • Marinho V.
        • Higgins J.P.T.
        A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered.
        J Clin Epidemiol. 2009; 62: 857-864
        • Furukawa T.A.
        • Salanti G.
        • Atkinson L.Z.
        • Leucht S.
        • Ruhe H.G.
        • Turner E.H.
        • et al.
        Comparative efficacy and acceptability of first-generation and second-generation antidepressants in the acute treatment of major depression: protocol for a network meta-analysis.
        BMJ Open. 2016; 6: e010919
        • Turner R.M.
        • Davey J.
        • Clarke M.J.
        • Thompson S.G.
        • Higgins J.P.
        Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews.
        Int J Epidemiol. 2012; 41: 818-827
        • Freedland K.E.
        • Mohr D.C.
        • Davidson K.W.
        • Schwartz J.E.
        Usual and unusual care: existing practice control groups in randomized controlled trials of behavioral interventions.
        Psychosom Med. 2011; 73: 323-335
        • Cuijpers P.
        • Karyotaki E.
        • Reijnders M.
        • Ebert D.D.
        Was Eysenck right after all? A reassessment of the effects of psychotherapy for adult depression.
        Epidemiol Psychiatr Sci. 2019; 28: 21-30
        • Furukawa T.A.
        • Weitz E.S.
        • Tanaka S.
        • Hollon S.D.
        • Hofmann S.G.
        • Andersson G.
        • et al.
        Initial severity of depression and efficacy of cognitive-behavioural therapy: individual-participant data meta-analysis of pill-placebo-controlled trials.
        Br J Psychiatry. 2017; 210: 190-196
        • Turner E.H.
        • Matthews A.M.
        • Linardatos E.
        • Tell R.A.
        • Rosenthal R.
        Selective publication of antidepressant trials and its influence on apparent efficacy.
        N Engl J Med. 2008; 358: 252-260