Advertisement

Improving GRADE evidence tables part 1: a randomized trial shows improved understanding of content in summary of findings tables with a new format

  • Alonso Carrasco-Labra
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, Universidad de Chile, Sergio Livingstone Pohlhammer 943, Independencia, Santiago, Chile

    Evidence-Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Sergio Livingstone Pohlhammer 943, Independencia, Santiago, Chile
    Search for articles by this author
  • Romina Brignardello-Petersen
    Affiliations
    Evidence-Based Dentistry Unit, Faculty of Dentistry, Universidad de Chile, Sergio Livingstone Pohlhammer 943, Independencia, Santiago, Chile

    Institute of Health Policy, Management and Evaluation, University of Toronto, 155 College Street, 4th Floor, Toronto, Ontario M5T 3M6, Canada
    Search for articles by this author
  • Nancy Santesso
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing (Applicability and Recommendations) Methods Group, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Ignacio Neumann
    Affiliations
    Pontificia Universidad Católica de Chile, Alameda 340, Santiago, Región Metropolitana, Chile
    Search for articles by this author
  • Reem A. Mustafa
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine/Nephrology, University of Missouri-Kansas City, 2411 Holmes Street, Kansas City, MO 64108-2792, USA

    Department of Biomedical & Health Informatics, University of Missouri-Kansas City, 2411 Holmes Street, Kansas City, MO 64108-2792, USA
    Search for articles by this author
  • Lawrence Mbuagbaw
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Biostatistics Unit, Father Sean O'Sullivan Research Centre, St Joseph's Healthcare, 50 Charlton Avenue East, 3rd Floor Martha Wing, Room H321, Hamilton, Ontario L8N 4A6, Canada
    Search for articles by this author
  • Itziar Etxeandia Ikobaltzeta
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Catherine De Stio
    Affiliations
    Hofstra North Shore LIJ School of Medicine, Department of Medicine, 300 Community Drive, Manhasset, NY 11030, USA
    Search for articles by this author
  • Lauren J. McCullagh
    Affiliations
    Hofstra North Shore LIJ School of Medicine, Department of Medicine, 300 Community Drive, Manhasset, NY 11030, USA
    Search for articles by this author
  • Pablo Alonso-Coello
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Iberoamerican Cochrane Centre, Biomedical Research Institute Sant Pau—CIBER of Epidemiology and Public Health (CIBERESP—IIB Sant Pau), C/ Sant Antoni Maria Claret 167, Pavelló 18, planta 0, 08025 Barcelona, Spain
    Search for articles by this author
  • Joerg J. Meerpohl
    Affiliations
    German Cochrane Centre, Medical Center—University of Freiburg, Berliner Allee 29, 79110 Freiburg, Germany
    Search for articles by this author
  • Per Olav Vandvik
    Affiliations
    Department of Medicine, Division Gjøvik, Innlandet Hospital Trust, Kyrre Greppsgt 11, 2819 Gjøvik, Norway

    Norwegian Knowledge Centre for the Health Services, Pilestredet Park 7, 0130 Oslo, Norway
    Search for articles by this author
  • Jan L. Brozek
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing (Applicability and Recommendations) Methods Group, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Elie A. Akl
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Internal Medicine, American University of Beirut, Riad-El-Solh Beirut 1107 2020, P.O. Box: 11-0236, Beirut, Lebanon
    Search for articles by this author
  • Patrick Bossuyt
    Affiliations
    Department of Clinical Epidemiology, Biostatistics and Bioinformatics, University of Amsterdam, Room J2-127, PO Box 22700, 1100 DE Amsterdam, The Netherlands
    Search for articles by this author
  • Rachel Churchill
    Affiliations
    Centre for Academic Mental Health, School of Social and Community Medicine, University of Bristol, Office Barley House, Room BF11, Oakfield House, Oakfield Grove, Bristol BS8 2BN, UK
    Search for articles by this author
  • Claire Glenton
    Affiliations
    Norwegian Branch of the Nordic Cochrane Centre, Postboks 7004 Street Olavs plass, 0130 Oslo, Norway

    Norwegian Knowledge Centre for the Health Services, P.O. Box 7004 Street Olavs plass, N-0130 Oslo, Norway
    Search for articles by this author
  • Sarah Rosenbaum
    Affiliations
    Norwegian Branch of the Nordic Cochrane Centre, Postboks 7004 Street Olavs plass, 0130 Oslo, Norway

    Norwegian Knowledge Centre for the Health Services, P.O. Box 7004 Street Olavs plass, N-0130 Oslo, Norway
    Search for articles by this author
  • Peter Tugwell
    Affiliations
    Department of Medicine, Faculty of Medicine, University of Ottawa, 451 Smyth Road, Ottawa, Ontario K1H 8M5, Canada
    Search for articles by this author
  • Vivian Welch
    Affiliations
    Bruyère Research Institute, University of Ottawa, 304b—85 Primrose Avenue, Ottawa, Ontario, Canada
    Search for articles by this author
  • Paul Garner
    Affiliations
    Evidence Synthesis for Global Health, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK
    Search for articles by this author
  • Gordon Guyatt
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing (Applicability and Recommendations) Methods Group, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Holger J. Schünemann
    Correspondence
    Corresponding author. Tel.: +1-905-525-9140x24931; fax: +1-905-522-9507.
    Affiliations
    Department of Clinical Epidemiology & Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Cochrane GRADEing (Applicability and Recommendations) Methods Group, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    McMaster GRADE Center, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author

      Abstract

      Objectives

      The current format of summary of findings (SoFs) tables for presenting effect estimates and associated quality of evidence improve understanding and assist users finding key information in systematic reviews. Users of SoF tables have demanded alternative formats to express findings from systematic reviews.

      Study Design and Setting

      We conducted a randomized controlled trial among systematic review users to compare the relative merits of a new format with the current formats of SoF tables regarding understanding, accessibility of information, satisfaction, and preference. Our primary goal was to show that the new format is not inferior to the current format.

      Results

      Of 390 potentially eligible subjects, 290 were randomized. Of seven items testing understanding, three showed similar results, two showed small differences favoring the new format, and two (understanding risk difference and quality of the evidence associated with a treatment effect) showed large differences favoring the new format [63% (95% confidence interval {CI}: 55, 71) and 62% (95% CI: 52, 71) more correct answers, respectively]. Respondents rated information in the alternative format as more accessible overall and preferred the new format over the current format.

      Conclusions

      While providing at least similar levels of understanding for some items and increased understanding for others, users prefer the new format of SoF tables.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Schunemann H.J.
        • Oxman A.D.
        • Brozek J.
        • Glasziou P.
        • Jaeschke R.
        • Vist G.E.
        • et al.
        Grading quality of evidence and strength of recommendations for diagnostic tests and strategies.
        BMJ. 2008; 336: 1106-1110
        • Jaeschke R.
        • Guyatt G.H.
        • Dellinger P.
        • Schunemann H.
        • Levy M.M.
        • Kunz R.
        • et al.
        Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive.
        BMJ. 2008; 337: a744
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.E.
        • Kunz R.
        • Falck-Ytter Y.
        • Alonso-Coello P.
        • et al.
        GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
        BMJ. 2008; 336: 924-926
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Vist G.E.
        • Falck-Ytter Y.
        • Schunemann H.J.
        What is “quality of evidence” and why is it important to clinicians?.
        BMJ. 2008; 336: 995-998
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Jaeschke R.
        • Helfand M.
        • Liberati A.
        • et al.
        Incorporating considerations of resources use into grading recommendations.
        BMJ. 2008; 336: 1170-1173
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Falck-Ytter Y.
        • Vist G.E.
        • Liberati A.
        • et al.
        Going from evidence to recommendations.
        BMJ. 2008; 336: 1049-1051
        • Schunemann, H.J.
        • Best D.
        • Vist G.
        • Oxman A.D.
        • GRADE Working Group
        Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations.
        CMAJ. 2003; 169: 677-680
        • Guyatt G.H.
        • Oxman A.D.
        • Santesso N.
        • Helfand M.
        • Vist G.
        • Kunz R.
        • et al.
        GRADE guidelines: 12. Preparing summary of findings tables-binary outcomes.
        J Clin Epidemiol. 2013; 66: 158-172
        • Guyatt G.H.
        • Thorlund K.
        • Oxman A.D.
        • Walter S.D.
        • Patrick D.
        • Furukawa T.A.
        • et al.
        GRADE guidelines: 13. Preparing summary of findings tables and evidence profiles-continuous outcomes.
        J Clin Epidemiol. 2013; 66: 173-183
        • Akl E.A.
        • Maroun N.
        • Guyatt G.
        • Oxman A.D.
        • Alonso-Coello P.
        • Vist G.E.
        • et al.
        Symbols were superior to numbers for presenting strength of recommendations to health care consumers: a randomized trial.
        J Clin Epidemiol. 2007; 60: 1298-1305
        • Rosenbaum S.E.
        • Glenton C.
        • Nylund H.K.
        • Oxman A.D.
        User testing and stakeholder feedback contributed to the development of understandable and useful summary of findings tables for Cochrane reviews.
        J Clin Epidemiol. 2010; 63: 607-619
        • Rosenbaum S.E.
        • Glenton C.
        • Oxman A.D.
        Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information.
        J Clin Epidemiol. 2010; 63: 620-626
        • Vandvik P.O.
        • Santesso N.
        • Akl E.A.
        • You J.
        • Mulla S.
        • Spencer F.A.
        • et al.
        Formatting modifications in GRADE evidence profiles improved guideline panelists comprehension and accessibility to information. A randomized trial.
        J Clin Epidemiol. 2012; 65: 748-755
        • Piaggio G.
        • Elbourne D.R.
        • Pocock S.J.
        • Evans S.J.
        • Altman D.G.
        Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement.
        JAMA. 2012; 308: 2594-2604
        • Carrasco-Labra A.
        • Brignardello-Petersen R.
        • Santesso N.
        • Neumann I.
        • Mustafa R.A.
        • Mbuagbaw L.
        • et al.
        Protocol: comparison between standard and new alternative formats of the summary-of-findings tables in Cochrane review users: a non-inferiority randomized controlled trial.
        Trials. 2015; 16: 164
        • Johnston B.C.
        • Goldenberg J.Z.
        • Vandvik P.O.
        • Sun X.
        • Guyatt G.H.
        Probiotics for the prevention of pediatric antibiotic-associated diarrhea.
        Cochrane Database Syst Rev. 2011; : CD004827
        • Glenton C.
        • Santesso N.
        • Rosenbaum S.
        • Nilsen E.S.
        • Rader T.
        • Ciapponi A.
        • et al.
        Presenting the results of Cochrane systematic reviews to a consumer audience: a qualitative study.
        Med Decis Making. 2010; 30: 566-577
        • Harrell F.E.
        Regression Modeling Strategies: with applications to linear, logistic regression, and survival analysis.
        Springer-Verlag, New York2001
        • Akl E.A.
        • Oxman A.D.
        • Herrin J.
        • Vist G.E.
        • Terrenato I.
        • Sperati F.
        • et al.
        Using alternative statistical formats for presenting risks and risk reductions.
        Cochrane Database Syst Rev. 2011; : CD006776
        • Woloshin S.
        • Schwartz L.M.
        Communicating data about the benefits and harms of treatment: a randomized trial.
        Ann Intern Med. 2011; 155: 87-96
        • Santesso N.
        • Rader T.
        • Nilsen E.S.
        • Glenton C.
        • Rosenbaum S.
        • Ciapponi A.
        • et al.
        A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial.
        J Clin Epidemiol. 2015; 68: 182-190