Advertisement

Borrowing of strength from indirect evidence in 40 network meta-analyses

      Abstract

      Objectives

      Network meta-analysis (NMA) is increasingly being used to synthesize direct and indirect evidence and help decision makers simultaneously compare multiple treatments. We empirically evaluate the incremental gain in precision achieved by incorporating indirect evidence in NMAs.

      Study Design and Setting

      We performed both network and pairwise meta-analyses on 40 published data sets of multiple-treatment comparisons. Their results were compared using the recently proposed borrowing of strength (BoS) statistic, which quantifies the percentage reduction in the uncertainty of the effect estimate when adding indirect evidence to an NMA.

      Results

      We analyzed 915 possible treatment comparisons, from which 484 (53%) had no direct evidence (BoS = 100%). In 181 comparisons with only one study contributing direct evidence, NMAs resulted in reduced precision (BoS < 0) and no appreciable improvements in precision (0 < BoS < 30%) for 104 (57.5%) and 23 (12.7%) comparisons, respectively. In 250 comparisons with at least two studies contributing direct evidence, NMAs provided increased precision with BoS ≥ 30% for 166 (66.4%) comparisons.

      Conclusion

      Although NMAs have the potential to provide more precise results than those only based on direct evidence, the incremental gain may reliably occur only when at least two head-to-head studies are available and treatments are well connected. Researchers should routinely report and compare the results from both network and pairwise meta-analyses.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Hutton B.
        • Salanti G.
        • Caldwell D.M.
        • Chaimani A.
        • Schmid C.H.
        • Cameron C.
        • et al.
        The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations.
        Ann Intern Med. 2015; 162: 777-784
        • Leucht S.
        • Chaimani A.
        • Cipriani A.S.
        • Davis J.M.
        • Furukawa T.A.
        • Salanti G.
        Network meta-analyses should be the highest level of evidence in treatment guidelines.
        Eur Arch Psychiatry Clin Neurosci. 2016; 266: 477-480
        • Mills E.J.
        • Ioannidis J.P.A.
        • Thorlund K.
        • Schünemann H.J.
        • Puhan M.A.
        • Guyatt G.H.
        How to use an article reporting a multiple treatment comparison meta-analysis.
        JAMA. 2012; 308: 1246-1253
        • Lumley T.
        Network meta-analysis for indirect treatment comparisons.
        Stat Med. 2002; 21: 2313-2324
        • Lu G.
        • Ades A.E.
        Combination of direct and indirect evidence in mixed treatment comparisons.
        Stat Med. 2004; 23: 3105-3124
        • Salanti G.
        Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool.
        Res Synth Methods. 2012; 3: 80-97
        • Efthimiou O.
        • Debray T.P.A.
        • van Valkenhoef G.
        • Trelle S.
        • Panayidou K.
        • Moons K.G.M.
        • et al.
        GetReal in network meta-analysis: a review of the methodology.
        Res Synth Methods. 2016; 7: 236-263
        • Lin L.
        • Chu H.
        • Hodges J.S.
        Sensitivity to excluding treatments in network meta-analysis.
        Epidemiology. 2016; 27: 562-569
        • Créquit P.
        • Trinquart L.
        • Yavchitz A.
        • Ravaud P.
        Wasted research when systematic reviews fail to provide a complete and up-to-date evidence synthesis: the example of lung cancer.
        BMC Med. 2016; 14: 8
        • Créquit P.
        • Trinquart L.
        • Ravaud P.
        Live cumulative network meta-analysis: protocol for second-line treatments in advanced non-small-cell lung cancer with wild-type or unknown status for epidermal growth factor receptor.
        BMJ Open. 2016; 6: e011841
        • Edwards S.J.
        • Clarke M.J.
        • Wordsworth S.
        • Borrill J.
        Indirect comparisons of treatments based on systematic reviews of randomised controlled trials.
        Int J Clin Pract. 2009; 63: 841-854
        • Bucher H.C.
        • Guyatt G.H.
        • Griffith L.E.
        • Walter S.D.
        The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials.
        J Clin Epidemiol. 1997; 50: 683-691
        • Jackson D.
        • Riley R.
        • White I.R.
        Multivariate meta-analysis: potential and promise.
        Stat Med. 2011; 30: 2481-2498
        • Cipriani A.
        • Furukawa T.A.
        • Salanti G.
        • Geddes J.R.
        • Higgins J.P.T.
        • Churchill R.
        • et al.
        Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis.
        Lancet. 2009; 373: 746-758
        • Brignardello-Petersen R.
        • Bonner A.
        • Alexander P.E.
        • Siemieniuk R.A.
        • Furukawa T.A.
        • Rochwerg B.
        • et al.
        Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis.
        J Clin Epidemiol. 2018; 93: 36-44
        • Li T.
        • Puhan M.A.
        • Vedula S.S.
        • Singh S.
        • Dickersin K.
        Network meta-analysis-highly attractive but more methodological research is needed.
        BMC Med. 2011; 9: 79
        • Bafeta A.
        • Trinquart L.
        • Seror R.
        • Ravaud P.
        Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review.
        BMJ. 2013; 347: f3675
        • Bafeta A.
        • Trinquart L.
        • Seror R.
        • Ravaud P.
        Reporting of results from network meta-analyses: methodological systematic review.
        BMJ. 2014; 348: g1741
        • Jansen J.P.
        • Naci H.
        Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers.
        BMC Med. 2013; 11: 159
        • Nikolakopoulou A.
        • Chaimani A.
        • Veroniki A.A.
        • Vasiliadis H.S.
        • Schmid C.H.
        • Salanti G.
        Characteristics of networks of interventions: a description of a database of 186 published networks.
        PLoS One. 2014; 9: e86754
        • Lin L.
        Quantifying and presenting overall evidence in network meta-analysis.
        Stat Med. 2018; 37: 4114-4125
        • Neupane B.
        • Richer D.
        • Bonner A.J.
        • Kibret T.
        • Beyene J.
        Network meta-analysis using R: a review of currently available automated packages.
        PLoS One. 2014; 9: e115065
        • Lin L.
        • Zhang J.
        • Hodges J.S.
        • Chu H.
        Performing arm-based network meta-analysis in R with the pcnetmeta package.
        J Stat Softw. 2017; 80: 1-25
        • Cipriani A.
        • Higgins J.P.T.
        • Geddes J.R.
        • Salanti G.
        Conceptual and technical challenges in network meta-analysis.
        Ann Intern Med. 2013; 159: 130-137
        • Mavridis D.
        • Welton N.J.
        • Sutton A.
        • Salanti G.
        A selection model for accounting for publication bias in a full network meta-analysis.
        Stat Med. 2014; 33: 5399-5412
        • Trinquart L.
        • Chatellier G.
        • Ravaud P.
        Adjustment for reporting bias in network meta-analysis of antidepressant trials.
        BMC Med Res Methodol. 2012; 12: 150
        • Trinquart L.
        • Ioannidis J.P.A.
        • Chatellier G.
        • Ravaud P.
        A test for reporting bias in trial networks: simulation and case studies.
        BMC Med Res Methodol. 2014; 14: 112
        • Egger M.
        • Davey Smith G.
        • Schneider M.
        • Minder C.
        Bias in meta-analysis detected by a simple, graphical test.
        BMJ. 1997; 315: 629-634
        • Hedges L.V.
        Modeling publication selection effects in meta-analysis.
        Stat Sci. 1992; 7: 246-255
        • Peters J.L.
        • Sutton A.J.
        • Jones D.R.
        • Abrams K.R.
        • Rushton L.
        Comparison of two methods to detect publication bias in meta-analysis.
        JAMA. 2006; 295: 676-680
        • Citkowicz M.
        • Vevea J.L.
        A parsimonious weight function for modeling publication bias.
        Psychol Methods. 2017; 22: 28-41
        • Duval S.
        • Tweedie R.
        A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis.
        J Am Stat Assoc. 2000; 95: 89-98
        • Lin L.
        • Chu H.
        Quantifying publication bias in meta-analysis.
        Biometrics. 2018; 74: 785-794
        • Trinquart L.
        • Abbé A.
        • Ravaud P.
        Impact of reporting bias in network meta-analysis of antidepressant placebo-controlled trials.
        PLoS One. 2012; 7: e35219
        • Jackson D.
        • White I.R.
        • Price M.
        • Copas J.
        • Riley R.D.
        Borrowing of strength and study weights in multivariate and network meta-analysis.
        Stat Methods Med Res. 2017; 26: 2853-2868
        • Riley R.D.
        • Jackson D.
        • Salanti G.
        • Burke D.L.
        • Price M.
        • Kirkham J.
        • et al.
        Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples.
        BMJ. 2017; 358: j3932
        • Copas J.B.
        • Jackson D.
        • White I.R.
        • Riley R.D.
        The role of secondary outcomes in multivariate meta-analysis.
        J R Stat Soc Ser C Appl Stat. 2018; 67: 1177-1205
        • Trinquart L.
        • Attiche N.
        • Bafeta A.
        • Porcher R.
        • Ravaud P.
        Uncertainty in treatment rankings: reanalysis of network meta-analyses of randomized trialsuncertainty in treatment rankings from network meta-analyses.
        Ann Intern Med. 2016; 164: 666-673
        • Veroniki A.A.
        • Vasiliadis H.S.
        • Higgins J.P.T.
        • Salanti G.
        Evaluation of inconsistency in networks of interventions.
        Int J Epidemiol. 2013; 42: 332-345
        • Bergman G.J.D.
        • Hochberg M.C.
        • Boers M.
        • Wintfeld N.
        • Kielhorn A.
        • Jansen J.P.
        Indirect comparison of tocilizumab and other biologic agents in patients with rheumatoid arthritis and inadequate response to disease-modifying antirheumatic drugs.
        Semin Arthritis Rheum. 2010; 39: 425-441
        • Coleman C.I.
        • Baker W.L.
        • Kluger J.
        • White C.M.
        Antihypertensive medication and their impact on cancer incidence: a mixed treatment comparison meta-analysis of randomized controlled trials.
        J Hypertens. 2008; 26: 622-629
        • Dakin H.
        • Fidler C.
        • Harper C.
        Mixed treatment comparison meta-analysis evaluating the relative efficacy of nucleos(t)ides for treatment of nucleos(t)ide-naive patients with chronic hepatitis B.
        Value Health. 2010; 13: 934-945
        • Gartlehner G.
        • Hansen R.A.
        • Morgan L.C.
        • Thaler K.
        • Lux L.
        • Van Noord M.
        • et al.
        Comparative benefits and harms of second-generation antidepressants for treating major depressive disorder: an updated meta-analysis.
        Ann Intern Med. 2011; 155: 772-785
        • Jansen J.P.
        • Bergman G.J.D.
        • Huels J.
        • Olson M.
        Prevention of vertebral fractures in osteoporosis: mixed treatment comparison of bisphosphonate therapies.
        Curr Med Res Opin. 2009; 25: 1861-1868
        • Thompson S.G.
        Why sources of heterogeneity in meta-analysis should be investigated.
        BMJ. 1994; 309: 1351-1355
        • Hardy R.J.
        • Thompson S.G.
        Detecting and describing heterogeneity in meta-analysis.
        Stat Med. 1998; 17: 841-856
        • Thompson S.G.
        • Sharp S.J.
        Explaining heterogeneity in meta-analysis: a comparison of methods.
        Stat Med. 1999; 18: 2693-2708
        • Higgins J.P.T.
        • Thompson S.G.
        • Deeks J.J.
        • Altman D.G.
        Measuring inconsistency in meta-analyses.
        BMJ. 2003; 327: 557-560
        • Higgins J.P.T.
        Commentary: heterogeneity in meta-analysis should be expected and appropriately quantified.
        Int J Epidemiol. 2008; 37: 1158-1160
        • Dias S.
        • Sutton A.J.
        • Ades A.E.
        • Welton N.J.
        Evidence synthesis for decision making 2: a generalized linear modeling framework for pairwise and network meta-analysis of randomized controlled trials.
        Med Decis Making. 2013; 33: 607-617
        • Turner R.M.
        • Davey J.
        • Clarke M.J.
        • Thompson S.G.
        • Higgins J.P.T.
        Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews.
        Int J Epidemiol. 2012; 41: 818-827
        • Dias S.
        • Sutton A.J.
        • Welton N.J.
        • Ades A.E.
        Evidence synthesis for decision making 3: heterogeneity—subgroups, meta-regression, bias, and bias-adjustment.
        Med Decis Making. 2013; 33: 618-640
        • Smith T.C.
        • Spiegelhalter D.J.
        • Thomas A.
        Bayesian approaches to random-effects meta-analysis: a comparative study.
        Stat Med. 1995; 14: 2685-2699
        • DerSimonian R.
        • Laird N.
        Meta-analysis in clinical trials.
        Control Clin Trials. 1986; 7: 177-188
      1. Plummer M. rjags: Bayesian graphical models using MCMC. R package version 4-8.

        • Roskell N.S.
        • Lip G.Y.H.
        • Noack H.
        • Clemens A.
        • Plumb J.M.
        Treatments for stroke prevention in atrial fibrillation: a network meta-analysis and indirect comparisons versus dabigatran etexilate.
        Thromb Haemost. 2010; 104: 1106-1115
        • Vieira M.C.
        • Kumar R.N.
        • Jansen J.P.
        Comparative effectiveness of efavirenz, protease inhibitors, and raltegravir-based regimens as first-line treatment for HIV-infected adults: a mixed treatment comparison.
        HIV Clin Trials. 2011; 12: 175-189
        • Hultcrantz M.
        • Rind D.
        • Akl E.A.
        • Treweek S.
        • Mustafa R.A.
        • Iorio A.
        • et al.
        The GRADE Working Group clarifies the construct of certainty of evidence.
        J Clin Epidemiol. 2017; 87: 4-13
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Brozek J.
        • Alonso-Coello P.
        • Rind D.
        • et al.
        GRADE guidelines 6. Rating the quality of evidence—imprecision.
        J Clin Epidemiol. 2011; 64: 1283-1293
        • Jansen J.P.
        • Bergman G.J.D.
        • Huels J.
        • Olson M.
        The efficacy of bisphosphonates in the prevention of vertebral, hip, and nonvertebral-nonhip fractures in osteoporosis: a network meta-analysis.
        Semin Arthritis Rheum. 2011; 40: 275-284
        • Kirkham J.J.
        • Riley R.D.
        • Williamson P.R.
        A multivariate meta-analysis approach for reducing the impact of outcome reporting bias in systematic reviews.
        Stat Med. 2012; 31: 2179-2195
        • Liu Y.
        • DeSantis S.M.
        • Chen Y.
        Bayesian mixed treatment comparisons meta-analysis for correlated outcomes subject to reporting bias.
        J R Stat Soc Ser C Appl Stat. 2018; 67: 127-144
        • Lin L.
        • Chu H.
        Bayesian multivariate meta-analysis of multiple factors.
        Res Synth Methods. 2018; 9: 261-272
        • Lu G.
        • Ades A.E.
        Modeling between-trial variance structure in mixed treatment comparisons.
        Biostatistics. 2009; 10: 792-805
        • Spiegelhalter D.J.
        • Best N.G.
        • Carlin B.P.
        • Van Der Linde A.
        Bayesian measures of model complexity and fit.
        J R Stat Soc Ser B Stat Methodol. 2002; 64: 583-639
        • Dias S.
        • Welton N.J.
        • Sutton A.J.
        • Caldwell D.M.
        • Lu G.
        • Ades A.E.
        Evidence synthesis for decision making 4: inconsistency in networks of evidence based on randomized controlled trials.
        Med Decis Making. 2013; 33: 641-656
        • Del Santo F.
        • Maratea D.
        • Fadda V.
        • Trippoli S.
        • Messori A.
        Treatments for relapsing–remitting multiple sclerosis: summarising current information by network meta-analysis.
        Eur J Clin Pharmacol. 2012; 68: 441-448
        • Gurusamy K.S.
        • Pissanou T.
        • Pikhart H.
        • Vaughan J.
        • Burroughs A.K.
        • Davidson B.R.
        Methods to decrease blood loss and transfusion requirements for liver transplantation.
        Cochrane Database Syst Rev. 2011; : CD009052
        • Vejakama P.
        • Thakkinstian A.
        • Lertrattananon D.
        • Ingsathit A.
        • Ngarmukos C.
        • Attia J.
        Reno-protective effects of renin–angiotensin system blockade in type 2 diabetic patients: a systematic review and network meta-analysis.
        Diabetologia. 2012; 55: 566-578
        • Woo G.
        • Tomlinson G.
        • Nishikawa Y.
        • Kowgier M.
        • Sherman M.
        • Wong D.K.H.
        • et al.
        Tenofovir and entecavir are the most effective antiviral agents for chronic hepatitis B: a systematic review and Bayesian meta-analyses.
        Gastroenterology. 2010; 139: 1218-1229
        • Dias S.
        • Welton N.J.
        • Sutton A.J.
        • Ades A.E.
        Evidence synthesis for decision making 5: the baseline natural history model.
        Med Decis Making. 2013; 33: 657-670