Questionable assumptions hampered interpretation of a network meta-analysis of primary care depression treatments

      Abstract

      Objectives

      We aimed to evaluate the underlying assumptions of a network meta-analysis investigating which depression treatment works best in primary care and to highlight challenges and pitfalls of interpretation under consideration of these assumptions.

      Study Design and Setting

      We reviewed 100 randomized trials investigating pharmacologic and psychological treatments for primary care patients with depression. Network meta-analysis was carried out within a frequentist framework using response to treatment as outcome measure. Transitivity was assessed by epidemiologic judgment based on theoretical and empirical investigation of the distribution of trial characteristics across comparisons. Homogeneity and consistency were investigated by decomposing the Q statistic.

      Results

      There were important clinical and statistically significant differences between “pure” drug trials comparing pharmacologic substances with each other or placebo (63 trials) and trials including a psychological treatment arm (37 trials). Overall network meta-analysis produced results well comparable with separate meta-analyses of drug trials and psychological trials. Although the homogeneity and consistency assumptions were mostly met, we considered the transitivity assumption unjustifiable.

      Conclusion

      An exchange of experience between reviewers and, if possible, some guidance on how reviewers addressing important clinical questions can proceed in situations where important assumptions for valid network meta-analysis are not met would be desirable.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Barth J.
        • Munder T.
        • Gerger H.
        • Nüesch E.
        • Trelle S.
        • Znoj H.
        • et al.
        Comparative efficacy of seven psychotherapeutic interventions for patients with depression: a network meta-analysis.
        PLoS Med. 2013; 10: e1001454
        • Cipriani A.
        • Furukawa T.A.
        • Salanti G.
        • Geddes J.R.
        • Higgins J.P.
        • Churchill R.
        • et al.
        Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis.
        Lancet. 2009; 373: 746-758
        • Gartlehner G.
        • Hansen R.A.
        • Morgan L.C.
        • Thaler K.
        • Lux L.
        • Van Noord M.
        • et al.
        Comparative benefits and harms of second-generation antidepressants for treating major depressive disorder: an updated meta-analysis.
        Ann Intern Med. 2011; 155: 772-785
        • Cuijpers P.
        • van Straten A.
        • van Schaik A.
        • Andersson G.
        Psychological treatment of depression in primary care: a meta-analysis.
        Br J Gen Pract. 2009; 59: e51-e60
        • Salanti G.
        • Higgins J.P.
        • Ades A.E.
        • Ioannidis J.P.
        Evaluation of networks of randomized trials.
        Stat Methods Med Res. 2008; 17: 279-301
        • Lee A.W.
        Review of mixed treatment comparisons in published systematic reviews shows marked increase since 2009.
        J Clin Epidemiol. 2014; 67: 138-143
        • Nikolakopoulou A.
        • Chaimani A.
        • Veroniki A.A.
        • Vasiliadis H.S.
        • Schmid C.H.
        • Salanti G.
        Characteristics of networks of interventions: a description of a database of 186 published networks.
        PLoS One. 2014; 9: e86754
        • Mills E.J.
        • Ioannidis J.P.
        • Thorlund K.
        • Schünemann H.J.
        • Puhan M.A.
        • Guyatt G.H.
        How to use an article reporting a multiple treatment comparison meta-analysis.
        JAMA. 2012; 308: 1246-1253
        • Salanti G.
        Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool.
        Res Synth Methods. 2012; 3: 80-97
        • Dias S.
        • Sutton A.J.
        • Ades A.E.
        • Welton N.J.
        Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials.
        Med Decis Making. 2013; 33: 607-617
        • Linde K.
        • Schumann I.
        • Meissner K.
        • Jamil S.
        • Kriston L.
        • Rücker G.
        • et al.
        Treatment of depressive disorders in primary care–protocol of a multiple treatment systematic review of randomized controlled trials.
        BMC Fam Pract. 2011; 12: 127
        • Linde K.
        • Kriston L.
        • Rücker G.
        • Jamil S.
        • Schumann I.
        • Meissner K.
        • et al.
        Efficacy and acceptability of pharmacological treatments for depressive disorders in primary care: systematic review and network-meta-analysis.
        Ann Fam Med. 2015; 13: 69-79
        • Linde K.
        • Sigterman K.
        • Kriston L.
        • Rücker G.
        • Jamil S.
        • Meissner K.
        • et al.
        Effectiveness of psychological treatments for depressive disorders in primary care: systematic review and meta-analysis.
        Ann Fam Med. 2015; 13: 56-68
        • Linde K.
        • Rücker G.
        • Sigterman K.
        • Jamil S.
        • Meissner K.
        • Schneider A.
        • et al.
        Comparative effectiveness of psychological treatments for depressive disorders in primary care.
        BMC Fam Pract. 2015; 16: 103
      1. Higgins JPT, Altman DG, Sterne JAC, Chapter 8: assessing risk of bias in included studies. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). 2011, The Cochrane Collaboration. Available at www.cochrane-handbook.org. Accessed November 17, 2015

        • Rücker G.
        Network meta-analysis, electrical networks and graph theory.
        Res Synth Methods. 2012; 3: 312-324
        • Higgins J.P.
        • Thompson S.G.
        Quantifying heterogeneity in a meta-analysis.
        Stat Med. 2002; 21: 1539-1558
        • Dias S.
        • Sutton A.J.
        • Welton N.J.
        • Ades A.E.
        Evidence synthesis for decision making 3: heterogeneity–subgroups, meta-regression, bias, and bias-adjustment.
        Med Decis Making. 2013; 33: 618-640
        • Turner R.M.
        • Davey J.
        • Clarke M.J.
        • Thompson S.G.
        • Higgins J.P.
        Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews.
        Int J Epidemiol. 2012; 41: 818-827
        • Krahn U.
        • Binder H.
        • König J.
        A graphical tool for locating inconsistency in network meta-analyses.
        BMC Med Res Methodol. 2013; 13: 35
        • Rücker G.
        • Schwarzer G.
        • Krahn U.
        • König J.
        netmeta: Network meta-analysis using frequentist methods. R package version 0.7-0.
        2015 (Available at) (Accessed November 17, 2015)
      2. Hegerl U, Mergl R. Remarks to “Efficacy and acceptability of pharmacological treatments for depressive disorders in primary care: systematic review and network meta-analysis” by Klaus Linde et al. and “Treatment of Depression in Primary Care” by Frank deGruy. Secondary Remarks to “Efficacy and Acceptability of Pharmacological Treatments for Depressive Disorders in Primary Care: Systematic Review and Network Meta-Analysis'' by Klaus Linde, et al. and “Treatment of Depression in Primary Care” by Frank deGruy. 2015. Available at http://www.annfammed.org/content/13/1/69.short/reply#annalsfm_el_28862. Accessed November 17, 2015.

        • Mills E.J.
        • Kanters S.
        • Thorlund K.
        • Chaimani A.
        • Veroniki A.A.
        • Ioannidis J.P.
        The effects of excluding treatments from network meta-analyses: survey.
        BMJ. 2013; 347: f5195
        • Kim D.D.
        • Tang J.Y.
        • Ioannidis J.P.A.
        Network geometry shows evidence sequestration for medical vs. surgical practices: treatments for basal cell carcinoma.
        J Clin Epidemiol. 2014; 67: 391-400
        • Kriston L.
        • von Wolff A.
        • Westphal A.
        • Hölzel L.P.
        • Härter M.
        Efficacy and acceptability of acute treatments for persistent depressive disorder: a network meta-analysis.
        Depress Anxiety. 2014; 31: 621-630
        • Haidich A.B.
        • Pilalas D.
        • Contopoulos-Ioannidis D.G.
        • Ioannidis J.P.
        Most meta-analyses of drug interventions have narrow scopes and many focus on specific agents.
        J Clin Epidemiol. 2013; 66: 371-378
        • Corbett M.S.
        • Rice S.J.
        • Madurasinghe V.
        • Slack R.
        • Fayter D.A.
        • Harden M.
        • et al.
        Acupuncture and other physical treatments for the relief of pain due to osteoarthritis of the knee: network meta-analysis.
        Osteoarthritis Cartilage. 2013; 21: 1290-1298
        • Mayo-Wilson E.
        • Dias S.
        • Mavranezouli I.
        • Kew K.
        • Clark D.M.
        • Ades A.E.
        • et al.
        Psychological and pharmacological interventions for social anxiety disorder in adults: a systematic review and network meta-analysis.
        Lancet Psychiatry. 2014; 1: 368-376
        • Nuesch E.
        • Hauser W.
        • Bernardy K.
        • Barth J.
        • Jüni P.
        Comparative efficacy of pharmacological and non-pharmacological interventions in fibromyalgia syndrome: network meta-analysis.
        Ann Rheum Dis. 2013; 72: 955-962
        • Cuijpers P.
        • van Straten A.
        • van Oppen P.
        • Andersson G.
        Are psychological and pharmacologic interventions equally effective in the treatment of adult depressive disorders? A meta-analysis of comparative studies.
        J Clin Psychiatry. 2008; 69: 1675-1685
        • Imel Z.E.
        • Malterer M.B.
        • McKay K.M.
        • Wampold B.E.
        A meta-analysis of psychotherapy and medication in unipolar depression and dysthymia.
        J Affect Disord. 2008; 110: 197-206
        • Salanti G.
        • Marinho V.
        • Higgins J.P.T.
        A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered.
        J Clin Epidemiol. 2009; 62: 857-864
        • Higgins J.P.T.
        • Thompson S.G.
        • Spiegelhalter D.J.
        A re-evaluation of random-effects meta-analysis.
        J R Stat Soc. 2009; 172: 137-159
        • Xiong T.
        • Parekh-Bhurke S.
        • Loke Y.K.
        • Abdelhamid A.
        • Sutton A.J.
        • Eastwood A.J.
        • et al.
        Overall similarity and consistency assessment scores are not sufficiently accurate for predicting discrepancy between direct and indirect comparison estimates.
        J Clin Epidemiol. 2013; 66: 184-191
        • Dias S.
        • Welton N.J.
        • Sutton A.J.
        • Ades A.E.
        Evidence synthesis for decision making 1: Introduction.
        Med Decis Making. 2013; 33: 597-606
        • Dias S.
        • Welton N.J.
        • Sutton A.J.
        • Caldwell D.M.
        • Lu G.
        • Ades A.E.
        Evidence synthesis for decision making 4: inconsistency in networks of evidence based on randomized controlled trials.
        Med Decis Making. 2013; 33: 641-656
        • Puhan M.A.
        • Schunemann H.J.
        • Murad M.H.
        • Li T.
        • Brignardello-Petersen R.
        • Singh J.A.
        • et al.
        A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis.
        BMJ. 2014; 349: g5630
        • Kriston L.
        Dealing with clinical heterogeneity in meta-analysis. Assumptions, methods, interpretation.
        Int J Methods Psychiatr Res. 2013; 22: 1-15
        • Kriston L.
        • Meister R.
        Incorporating uncertainty regarding applicability of evidence from meta-analyses into clinical decision making.
        J Clin Epidemiol. 2014; 67: 325-334
        • Ioannidis J.P.
        Indirect comparisons: the mesh and mess of clinical trials.
        Lancet. 2006; 368: 1470-1472