Using GRADE evidence to decision frameworks to choose from multiple interventions

      Abstract

      Background and Objective

      Guideline development groups or other health care decision makers frequently encounter situations that require a simultaneous comparison of multiple interventions. This sometimes becomes apparent either when they identify questions of interest, before they formulate recommendations, or it may surface only when recommendations have already been formulated based on pairwise comparisons.

      Methods

      Using examples from the World Health Organization, the European Commission, and a professional society, we developed a flexible approach to developing recommendations when a multiple-intervention comparison (MC) is needed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) evidence to decision (EtD) frameworks. We iteratively refined this approach through user testing and then included a module in GRADE's official software GRADEpro to test the approach in two real and one theoretical guideline recommendations.

      Results

      We found the approach feasible and that all EtD criteria should be considered in an MC approach. We judged that guideline development groups and other decision makers will benefit from the availability of a network meta-analyses (NMA) of intervention effects to support decisions; however, NMA supports only one of many criteria, that is, the balance of health benefits and harms, and is therefore helpful, but not essential to the approach we propose. When similar but not identical comparators are used to address MC, challenges may arise with intransitivity and the relative rankings of interventions.

      Conclusion

      We successfully applied the MC approach and software module in generating recommendations across different scenarios and identified challenges. The MC approach allows guideline groups and other decision makers to transparently and critically assess multiple options for a given health question. Application of the approach by others may lead to refinement and allow for better understanding of its impact in developing recommendations and making choices.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Schünemann Holger
        • Brozek Jan
        • Guyatt G.
        • Oxman A.
        GRADE Handbook. GRADEpro SOFTWARE, Hamilton, Ontario Canada2013
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Atkins D.
        • Brozek J.
        • Vist G.
        • et al.
        GRADE guidelines: 2. Framing the question and deciding on important outcomes.
        J Clin Epidemiol. 2011; 64: 395-400
        • Alonso-Coello P.
        • Oxman A.D.
        • Moberg J.
        • Brignardello-Petersen R.
        • Akl E.A.
        • Davoli M.
        • et al.
        GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: clinical practice guidelines.
        BMJ. 2016; 353: i2089
        • Parmelli E.
        • Amato L.
        • Oxman A.D.
        • Alonso-Coello P.
        • Brunetti M.
        • Moberg J.
        • et al.
        GRADE evidence to decision (etd) framework for coverage decisions.
        Int J Technol Assess Health Care. 2017; 33: 176-182
        • Schunemann H.J.
        • Mustafa R.
        • Brozek J.
        • Santesso N.
        • Alonso-Coello P.
        • Guyatt G.
        • et al.
        GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health.
        J Clin Epidemiol. 2016; 76: 89-98
        • Alonso-Coello P.
        • Schünemann H.J.
        • Moberg J.
        • Brignardello-Petersen R.
        • Akl E.A.
        • Davoli M.
        • et al.
        GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.
        BMJ. 2016; 353: i2016
        • Puhan M.A.
        • Schünemann H.J.
        • Murad M.H.
        • Li T.
        • Brignardello-Petersen R.
        • Singh J.A.
        • et al.
        A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis.
        BMJ (Clinical research ed.). 2014; 349: g5630
        • Brignardello-Petersen R.
        • Bonner A.
        • Alexander P.E.
        • Siemieniuk R.A.
        • Furukawa T.A.
        • Rochwerg B.
        • et al.
        Advances in the GRADE approach to rate the certainty in estimates from a network meta-analysis.
        J Clin Epidemiol. 2018; 93: 36-44
        • Yepes-Nunez J.J.
        • Li S.A.
        • Guyatt G.
        • Jack S.M.
        • Brozek J.L.
        • Beyene J.
        • et al.
        Development of the summary of findings table for network meta-analysis.
        J Clin Epidemiol. 2019; 115: 1-13
        • Brignardello-Petersen R.
        • Izcovich A.
        • Rochwerg B.
        • Florez I.D.
        • Hazlewood G.
        • Alhazanni W.
        • et al.
        GRADE approach to drawing conclusions from a network meta-analysis using a partially contextualised framework.
        BMJ. 2020; https://doi.org/10.1136/bmj.m3907
        • Brignardello-Petersen Romina
        • Florez I.D.
        • Izcovich A.
        • Santesso N.
        • Hazlewood G.
        • Alhazanni W.
        • et al.
        GRADE approach to drawing conclusions from a network meta-analysis using a minimally contextualised framework.
        BMJ. 2020; https://doi.org/10.1136/bmj.m3900
        • Schunemann H.J.
        • et al.
        Methods for development of the European commission initiative on breast cancer guidelines: recommendations in the era of guideline transparency.
        Ann Intern Med. 2019;
      1. WHO Guidelines on tuberculosis infection prevention and control, 2019 update. World Health Organization, Geneva, Switzerland2019
        • Schünemann H.J.
        • et al.
        Schunemann HJ, Lerda D, Quinn C, Follmann M, Alonso-Coello P, Rossi PG, et al. Breast Cancer Screening and Diagnosis: A Synopsis of the European Breast Guidelines. Ann Intern Med. 2020;172:46-56..
        Ann Intern Med. 2019; 172: 46-56
      2. Should organised mammography screening vs. no mammography screening be used for early detection of breast cancer in women aged of 50 to 69? European Commission, Ispra, Italy2017
        • Raghu G.
        • Collard H.R.
        • Egan J.J.
        • Martinez F.J.
        • Behr J.
        • Brown K.K.
        • et al.
        An official ATS/ERS/JRS/ALAT statement: idiopathic pulmonary fibrosis: evidence-based guidelines for diagnosis and management.
        Am J Respir Crit Care Med. 2011; 183: 788-824
        • Schunemann H.J.
        • Wiercioch W.
        • Etxeandia I.
        • Falavigna M.
        • Santesso N.
        • Mustafa R.
        • et al.
        Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise.
        CMAJ. 2014; 186: E123-E142
        • Rochwerg B.
        • Neupane B.
        • Zhang Y.
        • Garcia C.C.
        • Raghu G.
        • Richeldi L.
        • et al.
        Treatment of idiopathic pulmonary fibrosis: a network meta-analysis.
        BMC Med. 2016; 14: 18