Other GRADE Papers| Volume 154, P125-135, February 2023

Improving grading of recommendations assessment, development, and evaluation evidence tables part 4: a three-arm noninferiority randomized trial demonstrates improved understanding of content in summary of findings tables with a new format

Published:December 08, 2022DOI:


      • New components may improve how summary of findings (SoF) tables convey information.
      • Noninferiority randomized controlled trial (RCT) identifies facilitators of understanding.
      • Number needed to treat (NNT) and narrative summaries improve user satisfaction.
      • Tailor included information in SoF tables to end user.



      To evaluate alternative formats of summary of findings (SoF) tables for single comparison with multiple outcomes.

      Study Design and Setting

      We conducted a three-arm randomized controlled noninferiority trial (RCT) in the following systematic review (SR) users: researchers, clinical practice guideline developers, health care providers, policymakers, and knowledge transfer organizations to measure understanding, accessibility, satisfaction, and preference across the current grading of recommendations assessment, development, and evaluation (GRADE) SoF, an alternative GRADE SoF, or an adapted evidence-based practice center (EPC) program SoF table.


      One Hundred Seventy-Nine participants were randomized, and 129 participants completed the RCT (n = 47 current GRADE, n = 41 alternative GRADE, n = 41 adapted EPC). Understanding the certainty of evidence and treatment effect was comparable across groups. The adapted EPC SoF table was inferior for quantifying risk and RD compared to the alternatives (<35% correct vs. >85% correct). Participants reported increased satisfaction when SoF tables presented number needed to treat (NNT), anticipated absolute effect differences, and narrative syntheses for evidence that could not be meta-analyzed. Participants reported accessibility to information as significantly better in both GRADE SoF tables, when compared with the adapted EPC SoF table. Participants preferred the alternative GRADE SoF table format.


      The alternative GRADE SoF table is a promising format for SR users preferring a comprehensive presentation of SR results for single comparisons.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Guyatt G.H.
        • Oxman A.D.
        • Santesso N.
        • Helfand M.
        • Vist G.
        • Kunz R.
        • et al.
        GRADE guidelines: 12. Preparing summary of findings tables—binary outcomes.
        J Clin Epidemiol. 2013; 66: 158-172
        • Schünemann H.J.
        • Oxman A.D.
        • Higgins J.P.T.
        • Vist G.E.
        • Glasziou P.
        • Guyatt G.H.
        Chapter 11: presenting results and ‘Summary of findings’ tables.
        in: Higgins J.P.T.G.S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0, 2011 (Available at)
        • Higgins J.
        • Green S.
        Cochrane handbook for systematic reviews of interventions. Version 5.1.0.
        (Available at)
        Date: 2011
        Date accessed: February 3, 2013
        • Schünemann H.J.
        • Santesso N.
        • Brozek J.L.
        Interactive Summary of Findings tables: the way to present and understand results of systematic reviews.
        JBI Database Syst Rev Implement Rep. 2019; 17: 259-260
        • Chu D.K.
        • Akl E.A.
        • Duda S.
        • Solo K.
        • Yaacoub S.
        • Schünemann H.J.
        • et al.
        Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis.
        Lancet. 2020; 395: 1973-1987
        • Schünemann H.
        • Lerda D.
        • Dimitrova N.
        European commission initiative on breast cancer contributor group. Methods for development of the European commission initiative on breast cancer guidelines: recommendations in the era of guideline transparency.
        Ann Intern Med. 2019; 171: 273-280
        • Akl E.A.
        • Schünemann H.J.
        Routine heparin for patients with cancer? One answer, more questions.
        N Engl J Med. 2012; 366: 661-662
        • Morgan R.L.
        • Baack B.
        • Smith B.D.
        • Yartel A.
        • Pitasi M.
        • Falck-Ytter Y.
        Eradication of hepatitis C virus infection and the development of hepatocellular carcinoma: a meta-analysis of observational studies.
        Ann Intern Med. 2013; 158: 329-337
        • Conway A.
        • Clarke M.J.
        • Treweek S.P.
        • Schünemann H.
        • Santesso N.
        • Morgan R.L.
        • et al.
        Summary of findings tables for communicating key findings of systematic reviews.
        Cochrane Database Syst Rev. 2017; MR000044
        • Carrasco-Labra A.
        • Brignardello-Petersen R.
        • Santesso N.
        • Neumann I.
        • Mustafa R.A.
        • Mbuagbaw L.
        • et al.
        Improving GRADE evidence tables part 1: a randomized trial shows improved understanding of content in summary of findings tables with a new format.
        J Clin Epidemiol. 2016; 74: 7-18
        • Santesso N.
        • Rader T.
        • Nilsen E.S.
        • Glenton C.
        • Rosenbaum S.
        • Ciapponi A.
        • et al.
        A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial.
        J Clin Epidemiol. 2015; 68: 182-190
        • Rosenbaum S.E.
        • Glenton C.
        • Oxman A.D.
        Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information.
        J Clin Epidemiol. 2010; 63: 620-626
        • Berkman N.D.
        • Lohr K.N.
        • Ansari M.T.
        • Balk E.M.
        • Kane R.
        • McDonagh M.
        • et al.
        Grading the strength of a body of evidence when assessing health care interventions: an EPC update.
        J Clin Epidemiol. 2015; 68: 1312-1324
        • Piaggio G.
        • Elbourne D.R.
        • Pocock S.J.
        • Evans S.J.
        • Altman D.G.
        • Group C.
        Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement.
        JAMA. 2012; 308: 2594-2604
        • Yepes-Nuñez J.J.
        • Morgan R.L.
        • Mbuagbaw L.
        • Carrasco-Labra A.
        • Chang S.
        • Hempel S.
        • et al.
        Two alternatives versus the standard Grading of Recommendations Assessment, Development and Evaluation (GRADE) summary of findings (SoF) tables to improve understanding in the presentation of systematic review results: a three-arm, randomised, controlled, non-inferiority trial.
        BMJ Open. 2018; 8: e015623
        • Feltner C.
        • Jones C.D.
        • Cené C.W.
        • Zheng Z.-J.
        • Sueta C.A.
        • Coker-Schwimmer E.J.
        • et al.
        Transitional care interventions to prevent readmissions for persons with heart failure: a systematic review and meta-analysis.
        Ann Intern Med. 2014; 160: 774-784
        • Carrasco-Labra A.
        • Brignardello-Petersen R.
        • Santesso N.
        • Neumann I.
        • Mustafa R.A.
        • Mbuagbaw L.
        • et al.
        Comparison between the standard and a new alternative format of the Summary-of-Findings tables in Cochrane review users: study protocol for a randomized controlled trial.
        Trials. 2015; 16: 164
        • Akl E.A.
        • Maroun N.
        • Guyatt G.
        • Oxman A.D.
        • Alonso-Coello P.
        • Vist G.E.
        • et al.
        Symbols were superior to numbers for presenting strength of recommendations to health care consumers: a randomized trial.
        J Clin Epidemiol. 2007; 60: 1298-1305
        • Vandvik P.O.
        • Santesso N.
        • Akl E.A.
        • You J.
        • Mulla S.
        • Spencer F.A.
        • et al.
        Formatting modifications in GRADE evidence profiles improved guideline panelists comprehension and accessibility to information. A randomized trial.
        J Clin Epidemiol. 2012; 65: 748-755
        • Kruskal W.H.
        • Wallis W.A.
        Use of ranks in one-criterion variance analysis.
        J Am Stat Assoc. 1952; 47: 583-621
        • Dunn O.J.
        Multiple comparisons among means.
        J Am Stat Assoc. 1961; 56: 52-64