Advertisement

Improving GRADE evidence tables part 2: a systematic survey of explanatory notes shows more guidance is needed

  • Miranda Langendam
    Affiliations
    Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam, PO Box 22660, J1B-211, 1100 DD Amsterdam, The Netherlands
    Search for articles by this author
  • Alonso Carrasco-Labra
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, Universidad de Chile, Sergio Livingstone Pohlhammer 943, Independencia, Santiago, Chile

    Evidence-Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Sergio Livingstone Pohlhammer 943, Independencia, Santiago, Chile
    Search for articles by this author
  • Nancy Santesso
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Reem A. Mustafa
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, University of Missouri, 2411 Holmes St., Kansas City, MO 64108-2792, USA

    Department of Biomedical and Health Informatics, University of Missouri, 2411 Holmes St., Kansas City, MO 64108-2792, USA
    Search for articles by this author
  • Romina Brignardello-Petersen
    Affiliations
    Evidence-Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Sergio Livingstone Pohlhammer 943, Independencia, Santiago, Chile

    Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College St, 4th Floor, Toronto, Ontario M5T 3M6, Canada
    Search for articles by this author
  • Matthew Ventresca
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Pauline Heus
    Affiliations
    Dutch Cochrane Centre, Julius Center—UMC Utrecht, Huispostnummer Str. 6.131, Postbus 85500, 3508 GA Utrecht
    Search for articles by this author
  • Toby Lasserson
    Affiliations
    Cochrane Editorial Unit, St Albans House, 57-59 Haymarket, London SW1Y 4QX, United Kingdom
    Search for articles by this author
  • Rasmus Moustgaard
    Affiliations
    Nordic Cochrane Centre, Blegdamsvej 9, 7811, 2100 Copenhagen, Denmark
    Search for articles by this author
  • Jan Brozek
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Holger J. Schünemann
    Correspondence
    Corresponding author. Tel.: +1 905 525 9140; fax: +1 905 522 9507.
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane Applicability and Recommendations Methods Group, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author

      Abstract

      Objectives

      The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group has developed GRADE evidence profiles (EP) and summary of findings (SoF) tables to present evidence summaries in systematic reviews, clinical guidelines, and health technology assessments. Explanatory notes are used to explain choices and judgments in these summaries, for example, on rating of the quality of evidence.

      Study Design and Setting

      A systematic survey of the explanations in SoF tables in 132 randomly selected Cochrane Intervention reviews and in EPs of 10 guidelines. We analyzed the content of 1,291 explanations using a predefined list of criteria.

      Results

      Most explanations were used to describe or communicate results and to explain downgrading of the quality of evidence, in particular for risk of bias and imprecision. Addressing the source of baseline risk (observational data or control group risk) was often missing. For judgments about downgrading the quality of evidence, the percentage of informative explanations ranged between 41% (imprecision) and 79% (indirectness).

      Conclusion

      We found that by and large explanations were informative but detected several areas for improvement (e.g., source of baseline risk and judgments on imprecision). Guidance about explanatory footnotes and comments will be provided in the last article in this series.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Rosenbaum S.E.
        • Glenton C.
        • Nylund H.K.
        • Oxman A.D.
        User testing and stakeholder feedback contributed to the development of understandable and useful summary of findings tables for Cochrane reviews.
        J Clin Epidemiol. 2010; 63: 607-619
        • Rosenbaum S.E.
        • Glenton C.
        • Oxman A.D.
        Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information.
        J Clin Epidemiol. 2010; 63: 620-626
        • Langendam M.W.
        • Akl E.A.
        • Dahm P.
        • Glasziou P.
        • Guyatt G.
        • Schunemann H.J.
        Assessing and presenting summaries of evidence in Cochrane Reviews.
        Syst Rev. 2013; 2: 81
        • Guyatt G.
        • Oxman A.D.
        • Akl E.A.
        • Kunz R.
        • Vist G.
        • Brozek J.
        • et al.
        GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables.
        J Clin Epidemiol. 2011; 64: 383-394
      1. Schunemann H, Oxman A, Higgins J, Vist G, Glasziou P, Guyatt G. Chapter 11: presenting results and ‘summary of findings’ tables. In: Higgins J, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 510 (updated March 2011). The Cochrane Collaboration; 2011. Available at www.cochrane-handbook.org.

        • Langendam M.W.
        • Mustafa R.
        • Santesso N.
        • Carrasco-Labra A.
        • Moustgaard R.
        • Ventresca M.
        • et al.
        Harmonization of explanations for common judgments about the quality of evidence in summary of findings tables.
        Cochrane Database Syst Rev. 2013; : 1
        • Carrasco-Labra A.
        • Brignardello-Petersen R.
        • Santesso N.
        • Neumann I.
        • Mustafa R.
        • Mbuagbaw L.
        • et al.
        Improving GRADE evidence tables: A randomized trial shows improved understanding of content in Summary-of-Findings Tables with a new format.
        J Clin Epidemiol. 2016; 74: 7-18
        • Holbrook A.
        • Schulman S.
        • Witt D.M.
        • Vandvik P.O.
        • Fish J.
        • Kovacs M.J.
        • et al.
        Evidence-based management of anticoagulant therapy: antithrombotic therapy and prevention of thrombosis, 9th ed.: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines.
        Chest. 2012; 141 (e152S-84S)
        • Taichman D.B.
        • Ornelas J.
        • Chung L.
        • Klinger J.R.
        • Lewis S.
        • Mandel J.
        • et al.
        Pharmacologic therapy for pulmonary arterial hypertension in adults: CHEST guideline and expert panel report.
        Chest. 2014; 146: 449-475
        • Brozek J.L.
        • Bousquet J.
        • Baena-Cagnani C.E.
        • Bonini S.
        • Canonica G.W.
        • Casale T.B.
        • et al.
        Allergic Rhinitis and its Impact on Asthma (ARIA) guidelines: 2010 revision.
        J Allergy Clin Immunol. 2010; 126: 466-476
        • Raghu G.
        • Collard H.R.
        • Egan J.J.
        • Martinez F.J.
        • Behr J.
        • Brown K.K.
        • et al.
        An official ATS/ERS/JRS/ALAT statement: idiopathic pulmonary fibrosis: evidence-based guidelines for diagnosis and management.
        Am J Respir Crit Care Med. 2011; 183: 788-824
        • Fiocchi A.
        • Schunemann H.J.
        • Brozek J.
        • Restani P.
        • Beyer K.
        • Troncone R.
        • et al.
        Diagnosis and Rationale for Action against Cow's Milk allergy (DRACMA): a summary report.
        J Allergy Clin Immunol. 2010; 126 (1119–1128 e12)
        • WHO
        The Use of Bedaquiline in the Treatment of Multidrug-Resistant Tuberculosis: Interim Policy Guidance.
        World Health Organization Copyright (c) World Health Organization 2013, Geneva2013
        • WHO
        WHO Recommendations for Augmentation of Labour.
        World Health Organization Coypright (c) World Health Organization 2014, Geneva2014
        • WHO
        WHO Recommendation on Community Mobilization through Facilitated Participatory Learning and Action Cycles with Women's Groups for Maternal and Newborn Health.
        World Health Organization Copyright (c) World Health Organization 2014, Geneva2014
        • WHO
        Guidelines for the Identification and Management of Substance Use and Substance Use Disorders in Pregnancy.
        World Health Organization Copyright (c) World Health Organization 2014, Geneva2014
        • Schunemann H.J.
        • Hill S.R.
        • Kakad M.
        • Bellamy R.
        • Uyeki T.M.
        • Hayden F.G.
        • et al.
        WHO Rapid Advice Guidelines for pharmacological management of sporadic human infection with avian influenza A (H5N1) virus.
        Lancet Infect Dis. 2007; 7: 21-31
        • Mustafa R.A.
        • Santesso N.
        • Brozek J.
        • Akl E.A.
        • Walter S.D.
        • Norman G.
        • et al.
        The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses.
        J Clin Epidemiol. 2013; 66 (quiz 42 e1–5): 736-742
        • Rosner S.
        • Hackl-Herrwerth A.
        • Leucht S.
        • Lehert P.
        • Vecchi S.
        • Soyka M.
        Acamprosate for alcohol dependence.
        Cochrane Database Syst Rev. 2010; : CD004332
        • Guyatt G.H.
        • Oxman A.D.
        • Santesso N.
        • Helfand M.
        • Vist G.
        • Kunz R.
        • et al.
        GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes.
        J Clin Epidemiol. 2013; 66: 158-172
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.
        • Kunz R.
        • Brozek J.
        • Alonso-Coello P.
        • et al.
        GRADE guidelines: 4. Rating the quality of evidence–study limitations (risk of bias).
        J Clin Epidemiol. 2011; 64: 407-415
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Brozek J.
        • Alonso-Coello P.
        • Rind D.
        • et al.
        GRADE guidelines 6. Rating the quality of evidence –imprecision.
        J Clin Epidemiol. 2011; 64: 1283-1293
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines: 7. Rating the quality of evidence –inconsistency.
        J Clin Epidemiol. 2011; 64: 1294-1302
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines: 8. Rating the quality of evidence–indirectness.
        J Clin Epidemiol. 2011; 64: 1303-1310
        • Guyatt G.H.
        • Oxman A.D.
        • Montori V.
        • Vist G.
        • Kunz R.
        • Brozek J.
        • et al.
        GRADE guidelines: 5. Rating the quality of evidence–publication bias.
        J Clin Epidemiol. 2011; 64: 1277-1282
        • Guyatt G.H.
        • Oxman A.D.
        • Sultan S.
        • Glasziou P.
        • Akl E.A.
        • Alonso-Coello P.
        • et al.
        GRADE guidelines: 9. Rating up the quality of evidence.
        J Clin Epidemiol. 2011; 64: 1311-1316
        • Schünemann H.J.
        • Tugwell P.
        • Reeves B.C.
        • Akl E.A.
        • Santesso N.
        • Spencer F.A.
        • et al.
        Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions.
        Res Synth Methods. 2013; 4: 49-62