Advertisement

Improving GRADE evidence tables part 3: detailed guidance for explanatory footnotes supports creating and understanding GRADE certainty in the evidence judgments

  • Nancy Santesso
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing Methods Group, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Alonso Carrasco-Labra
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing Methods Group, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, Universidad de Chile, Sergio Livingstone Pohlhammer 943, Independencia, Santiago, Chile
    Search for articles by this author
  • Miranda Langendam
    Affiliations
    Cochrane GRADEing Methods Group, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam, PO Box 22660, J1B-211, 1100 DD Amsterdam, The Netherlands
    Search for articles by this author
  • Romina Brignardello-Petersen
    Affiliations
    Evidence-Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Sergio Livingstone Pohlhammer 943, Independencia, Santiago, Chile

    Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, 4th Floor, Toronto, Ontario M5T 3M6, Canada

    Department of Medicine/Nephrology, University of Missouri-Kansas City, 2411 Holmes Street, Kansas City, MO 64108-2792, USA
    Search for articles by this author
  • Reem A. Mustafa
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing Methods Group, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine/Nephrology, University of Missouri-Kansas City, 2411 Holmes Street, Kansas City, MO 64108-2792, USA

    Department of Biomedical & Health Informatics, University of Missouri-Kansas City, 2411 Holmes Street, Kansas City, MO 64108-2792, USA
    Search for articles by this author
  • Pauline Heus
    Affiliations
    Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Huispostnummer Street 6.131, Postbus 85500, 3508 GA Utrecht, The Netherlands
    Search for articles by this author
  • Toby Lasserson
    Affiliations
    Cochrane Editorial Unit, St Albans House, 57-59 Haymarket, London SW1Y 4QX, UK
    Search for articles by this author
  • Newton Opiyo
    Affiliations
    Cochrane Editorial Unit, St Albans House, 57-59 Haymarket, London SW1Y 4QX, UK
    Search for articles by this author
  • Ilkka Kunnamo
    Affiliations
    Department of General Practice and Primary Health Care, Duodecim Medical Publications Ltd and University of Helsinki, PO Box 874, Kaivokatu 10 A, 7th Floor, FIN-00101 Helsinki, Finland
    Search for articles by this author
  • David Sinclair
    Affiliations
    Department of Clinical Sciences, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK
    Search for articles by this author
  • Paul Garner
    Affiliations
    Department of Clinical Sciences, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK

    Cochrane Infectious Diseases Group, Pembroke Place, Liverpool L3 5QA, UK
    Search for articles by this author
  • Shaun Treweek
    Affiliations
    Health Services Research Unit, University of Aberdeen, Health Sciences Building, Foresterhill, Aberdeen AB25 2ZD, UK
    Search for articles by this author
  • David Tovey
    Affiliations
    Cochrane Editorial Unit, St Albans House, 57-59 Haymarket, London SW1Y 4QX, UK
    Search for articles by this author
  • Elie A. Akl
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing Methods Group, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Internal Medicine, American University of Beirut, Riad-El-Solh, PO Box 11-0236, Beirut 1107 2020, Lebanon
    Search for articles by this author
  • Peter Tugwell
    Affiliations
    Clinical Epidemiology Unit, Ottawa Hospital Research Institute, Ottawa Hospital, Ottawa, Ontario, Canada
    Search for articles by this author
  • Jan L. Brozek
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing Methods Group, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Gordon Guyatt
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing Methods Group, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Holger J. Schünemann
    Correspondence
    Corresponding author. Tel.: 9055259140; fax: 9055229507.
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing Methods Group, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author

      Abstract

      Background

      The Grading of Recommendations Assessment, Development and Evaluation (GRADE) is widely used and reliable and accurate for assessing the certainty in the body of health evidence. The GRADE working group has provided detailed guidance for assessing the certainty in the body of evidence in systematic reviews and health technology assessments (HTAs) and how to grade the strength of health recommendations. However, there is limited advice regarding how to maximize transparency of these judgments, in particular through explanatory footnotes or explanations in Summary of Findings tables and Evidence Profiles (GRADE evidence tables).

      Methods

      We conducted this study to define the essential attributes of useful explanations and to develop specific guidance for explanations associated with GRADE evidence tables. We used a sample of explanations according to their complexity, type of judgment involved, and appropriateness from a database of published GRADE evidence tables in Cochrane reviews and World Health Organization guidelines. We used an iterative process and group consensus to determine the attributes and develop guidance.

      Results

      Explanations in GRADE evidence tables should be concise, informative, relevant, easy to understand, and accurate. We provide general and domain-specific guidance to assist authors with achieving these desirable attributes in their explanations associated with GRADE evidence tables.

      Conclusions

      Adhering to the general and GRADE domain-specific guidance should improve the quality of explanations associated with GRADE evidence tables, assist authors of systematic reviews, HTA reports, or guidelines with information that they can use in other parts of their evidence synthesis. This guidance will also support editorial evaluation of evidence syntheses using GRADE and provide a minimum quality standard of judgments across tables.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Schünemann H.J.
        • Best D.
        • Vist G.
        • Oxman A.D.
        • GRADE Working Group
        Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations.
        CMAJ. 2003; 169: 677-680
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.E.
        • Kunz R.
        • Falck-Ytter Y.
        • Alonso-Coello P.
        • et al.
        GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
        BMJ. 2008; 336: 924-926
        • Guyatt G.
        • Oxman A.D.
        • Akl E.A.
        • Kunz R.
        • Vist G.
        • Brozek J.
        • et al.
        GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables.
        J Clin Epidemiol. 2011; 64: 383-394
        • Mustafa R.A.
        • Santesso N.
        • Brozek J.
        • Akl E.A.
        • Walter S.D.
        • Norman G.
        • et al.
        The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses.
        J Clin Epidemiol. 2013; 66 (quiz 42 e1–5): 736-742
        • Guyatt G.
        • Oxman A.D.
        • Sultan S.
        • Brozek J.
        • Glasziou P.
        • Alonso-Coello P.
        • et al.
        GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes.
        J Clin Epidemiol. 2013; 66: 151-157
        • Guyatt G.H.
        • Oxman A.D.
        • Santesso N.
        • Helfand M.
        • Vist G.
        • Kunz R.
        • et al.
        GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes.
        J Clin Epidemiol. 2013; 66: 158-172
        • Guyatt G.H.
        • Thorlund K.
        • Oxman A.D.
        • Walter S.D.
        • Patrick D.
        • Furukawa T.A.
        • et al.
        GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes.
        J Clin Epidemiol. 2013; 66: 173-183
        • Schünemann H.J.
        • Oxman A.D.
        • Vist G.E.
        • Higgins J.P.T.
        • Deeks J.J.
        • Glasziou P.
        • et al.
        Chapter 12: interpreting results and drawing conclusions.
        in: Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 510. The Cochrane Collaboration, 2011 (updated March 2011). Available at (Accessed November 25, 2015)
        • Schünemann H.J.
        • Oxman A.D.
        • Higgins J.P.T.
        • Vist G.E.
        • Glasziou P.
        • Guyatt G.H.
        Chapter 11: presenting results and ‘Summary of findings’ tables.
        in: Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 510. The Cochrane Collaboration, 2011 (updated March 2011). Available at (Accessed November 25, 2015)
        • Carrasco-Labra A.
        • Brignardello-Petersen R.
        • Santesso N.
        • Neumann I.
        • Mustafa R.
        • Mbuagbaw L.
        • et al.
        Improving GRADE evidence tables: a randomized trial shows improved understanding of content in Summary-of-Findings tables with a new format.
        J Clin Epidemiol. 2016; 74: 7-18
        • Langendam M.
        • Carrasco-Labra A.
        • Santesso N.
        • Brignardello-Petersen R.
        • Ventresca M.
        • Heus P.
        • et al.
        Improving GRADE evidence tables part 2: a systematic survey of explanatory notes shows more guidance is needed.
        J Clin Epidemiol. 2016; 74: 19-27
        • Schunemann H.J.
        Methodological idiosyncracies, frameworks and challenges of non-pharmaceutical and non-technical treatment interventions.
        Z Evid Fortbild Qual Gesundhwes. 2013; 107: 214-220
        • Schünemann H.J.
        • Tugwell P.
        • Reeves B.C.
        • Akl E.A.
        • Santesso N.
        • Spencer F.A.
        • et al.
        Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions.
        Res Synth Methods. 2013; 4: 49-62
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines: 7. Rating the quality of evidence–inconsistency.
        J Clin Epidemiol. 2011; 64: 1294-1302
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Brozek J.
        • Alonso-Coello P.
        • Rind D.
        • et al.
        GRADE guidelines 6. Rating the quality of evidence–imprecision.
        J Clin Epidemiol. 2011; 64: 1283-1293
        • Balshem H.
        • Helfand M.
        • Schunemann H.J.
        • Oxman A.D.
        • Kunz R.
        • Brozek J.
        • et al.
        GRADE guidelines: 3. Rating the quality of evidence.
        J Clin Epidemiol. 2011; 64: 401-406
        • Andrews J.C.
        • Schunemann H.J.
        • Oxman A.D.
        • Pottie K.
        • Meerpohl J.J.
        • Coello P.A.
        • et al.
        GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength.
        J Clin Epidemiol. 2013; 66: 726-735
        • Akl E.A.
        • Schunemann H.J.
        Routine heparin for patients with cancer? One answer, more questions.
        New Engl J Med. 2012; 366: 661-662
      1. McIlwain C, Santesso N, Simi S, Napoli M, Lasserson T, Welsh E, et al. Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS). Methodological Expectations of Cochrane Intervention Reviews (MECIR) [Internet]. February, 2003 May, 2015. Available at http://editorial-unit.cochrane.org/sites/editorial-unit.cochrane.org/files/uploads/PLEACS_0.pdf.

      2. Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D. Methodological standards for the conduct of new Cochrane Intervention Reviews. Methodological Expectations of Cochrane Intervention Reviews (MECIR) [Internet]. December, 2013 May, 2015. Available at http://editorial-unit.cochrane.org/sites/editorial-unit.cochrane.org/files/uploads/MECIR_conduct_standards2.3 02122013.pdf.

        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • PRISMA Group
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        J Clin Epidemiol. 2009; 62: 1006-1012
        • Moher D.
        • Shamseer L.
        • Clarke M.
        • Ghersi D.
        • Liberati A.
        • Petticrew M.
        • et al.
        Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement.
        Syst Rev. 2015; 4: 1
        • Lasserson T.
        • Santesso N.
        • Cumpston M.
        • Marshall R.
        • Ní Ógáin O.
        Incorporating GRADE in Cochrane Reviews: feedback from the CEU screening programme.
        2015 (Available at: http://training.cochrane.org/sites/training.cochrane.org/files/public/uploads/resources/downloadable_resources/English/Incorporating%20GRADE%20in%20Cochrane%20Reviews%20PDF.pdf. Accessed November 15, 2015)