Original Article| Volume 154, P188-196, February 2023

Considering multiple outcomes with different weights informed the hierarchy of interventions in network meta-analysis

Published:December 26, 2022DOI:



      Ranking metrics in network meta-analysis (NMA) are computed separately for each outcome. Our aim is to 1) present graphical ways to group competing interventions considering multiple outcomes and 2) use conjoint analysis for placing weights on the various outcomes based on the stakeholders’ preferences.

      Study Design and Setting

      We used multidimensional scaling (MDS) and hierarchical tree clustering to visualize the extent of similarity of interventions in terms of the relative effects they produce through a random effect NMA. We reanalyzed a published network of 212 psychosis trials taking three outcomes into account as follows: reduction in symptoms of schizophrenia, all-cause treatment discontinuation, and weight gain.


      Conjoint analysis provides a mathematical method to transform judgements into weights that can be subsequently used to visually represent interventions on a two-dimensional plane or through a dendrogram. These plots provide insightful information about the clustering of interventions.


      Grouping interventions can help decision makers not only to identify the optimal ones in terms of benefit-risk balance but also choose one from the best cluster based on other grounds, such as cost, implementation etc. Placing weights on outcomes allows considering patient profile or preferences.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Seitidis G.
        • Nikolakopoulos S.
        • Hennessy E.
        • Tanner-Smith E.
        • Mavridis D.
        Network meta-analysis techniques for synthesizing prevention science evidence.
        Prev Sci 2021. 2021; 1: 1-10
        • Salanti G.
        Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool.
        Res Synth Methods. 2012; 3: 80-97
        • Salanti G.
        • Nikolakopoulou A.
        • Efthimiou O.
        • Mavridis D.
        • Egger M.
        • White I.R.
        Introducing the treatment hierarchy question in network meta-analysis.
        Am J Epidemiol. 2022; 191: 930-938
        • Trinquart L.
        • Attiche N.
        • Bafeta A.
        • Porcher R.
        • Ravaud P.
        Uncertainty in treatment rankings: reanalysis of network meta-analyses of randomized trials.
        Ann Intern Med. 2016; 164: 666-673
        • Rücker G.
        • Schwarzer G.
        Ranking treatments in frequentist network meta-analysis works without resampling methods.
        BMC Med Res Methodol. 2015; 15: 58
        • Salanti G.
        • Ades A.E.
        • Ioannidis J.P.A.
        Graphical methods and numerical summaries for presenting results from multiple-treatment meta-analysis: an overview and tutorial.
        J Clin Epidemiol. 2011; 64: 163-171
        • Hutton B.
        • Salanti G.
        • Caldwell D.M.
        • Chaimani A.
        • Schmid C.H.
        • Cameron C.
        • et al.
        The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations.
        Ann Intern Med. 2015; 162: 777-784
        • Chaimani A.
        • Higgins J.P.T.
        • Mavridis D.
        • Spyridonos P.
        • Salanti G.
        Graphical tools for network meta-analysis in STATA.
        PLoS One. 2013; 8: e76654
        • Veroniki A.A.
        • Straus S.E.
        • Fyraridis A.
        • Tricco A.C.
        The rank-heat plot is a novel way to present the results from a network meta-analysis including multiple outcomes.
        J Clin Epidemiol. 2016; 76: 193-199
        • Mavridis D.
        • Porcher R.
        • Nikolakopoulou A.
        • Salanti G.
        • Ravaud P.
        Extensions of the probabilistic ranking metrics of competing treatments in network meta-analysis to reflect clinically important relative differences on many outcomes.
        Biom J. 2019; 62: 375-385
        • Chung H.
        • Lumley T.
        Graphical exploration of network meta-analysis data: the use of multidimensional scaling.
        Clin Trials. 2008; 5: 301-307
        • Leucht S.
        • Cipriani A.
        • Spineli L.
        • Mavridis D.
        • Orey D.
        • Richter F.
        • et al.
        Comparative efficacy and tolerability of 15 antipsychotic drugs in schizophrenia: a multiple-treatments meta-analysis.
        Lancet. 2013; 382: 951-962
        • Chinn S.
        A simple method for converting an odds ratio to effect size for use in meta-analysis.
        Stat Med. 2000; 19: 3127-3131
        • Gustafsson A.
        • Herrmann A.
        • Huber F.
        Conjoint Measurement : Methods and Applications. eds(4). Springer Berlin, Heidelberg, Germany2007
        • Sculpher M.
        • Bryan S.
        • Fry P.
        • de Winter P.
        • Payne H.
        • Emberton M.
        Patients' preferences for the management of non-metastatic prostate cancer: discrete choice experiment.
        BMJ. 2004; 328: 382
        • Arroyo R.
        • Sempere A.P.
        • Ruiz-Beato E.
        • Prefasi D.
        • Carreño A.
        • Roset M.
        • et al.
        Conjoint analysis to understand preferences of patients with multiple sclerosis for disease-modifying therapy attributes in Spain: a cross-sectional observational study.
        BMJ Open. 2017; 7: e014433
        • Everitt B.
        • Rabe-Hesketh S.
        The Analysis of Proximity Data. Kendall’s advanced theory of statistics.
        Arnold, London, NY1997
        • Bartholomew D.J.
        • Steele F.
        • Galbraith J.
        • Moustaki I.
        Analysis of Multivariate Social Science Data. Second Edition. Chapman & Hall/CRC Statistics in the Social and Behavioral Sciences, Boca Raton, FL2008
        • Leeuw J.A.N.D.E.
        • Mair P.
        Multidimensional scaling using majorization: SMACOF in R.
        J Stat Softw. 2009; 31: 1-30
        • Rücker G.
        • Krahn U.
        • König J.
        • Efthimiou O.
        • Davies A.
        • Papakonstantinou T.
        • et al.
        netmeta: Network Meta-Analysis using Frequentist Methods. R package version 2.7-0.
        (Available at)
        • R Core Team
        R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.
        (Available at)
        • Murtagh F.
        • Legendre P.
        Ward’s hierarchical agglomerative clustering method: which algorithms implement ward’s criterion?.
        J Classification. 2014; 31: 274-295
        • Nijs V.
        radiant: Business Analytics using R and Shiny_. R package version 1.5.0.
        (Available at)
        • Brignardello-Petersen R.
        • Johnston B.C.
        • Jadad A.R.
        • Tomlinson G.
        Using decision thresholds for ranking treatments in network meta-analysis results in more informative rankings.
        J Clin Epidemiol. 2018; 98: 62-69
        • Nikolakopoulou A.
        • Higgins J.P.T.
        • Papakonstantinou T.
        • Chaimani A.
        • Giovane C.D.
        • Egger M.
        • et al.
        CINeMA: an approach for assessing confidence in the results of a network meta-analysis.
        PLoS Med. 2020; 17: e1003082
        • Say R.E.
        • Thomson R.
        The importance of patient preferences in treatment decisions—challenges for doctors.
        BMJ. 2003; 327: 542-545