Advertisement

GRADE guidelines: 22. The GRADE approach for tests and strategies—from test accuracy to patient-important outcomes and recommendations

  • Holger J. Schünemann
    Correspondence
    Corresponding author: Department of Health, Evidence and Impact (formerly “Clinical Epidemiology and Biostatistics”), Director, McMaster GRADE Centre, Professor of Clinical Epidemiology and Medicine, McMaster University, HSC Room 2C16 1280 Main Street, West Hamilton, Ontario L8S 4K1, Canada. Tel: +1 905-525-9140x24931; Fax: +1-905-522-9507.
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    McMaster GRADE Centre, Michael DeGroote Cochrane Canada Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • Reem A. Mustafa
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, University of Kansas Medical Center, Kansas City, KS, USA
    Search for articles by this author
  • Jan Brozek
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    McMaster GRADE Centre, Michael DeGroote Cochrane Canada Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • Nancy Santesso
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    McMaster GRADE Centre, Michael DeGroote Cochrane Canada Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • Patrick M. Bossuyt
    Affiliations
    Clinical Epidemiology and Biostatistics and Bioinformatics Academic Medical Center, University of Amsterdam, Meibergdreef 9, P.O.Box 227001100 DE, Amsterdam, The Netherlands
    Search for articles by this author
  • Karen R. Steingart
    Affiliations
    Cochrane Infectious Diseases Group, Liverpool School of Tropical Medicine, Liverpool, L3 5QA UK
    Search for articles by this author
  • Mariska Leeflang
    Affiliations
    Clinical Epidemiology and Biostatistics and Bioinformatics Academic Medical Center, University of Amsterdam, Meibergdreef 9, P.O.Box 227001100 DE, Amsterdam, The Netherlands
    Search for articles by this author
  • Stefan Lange
    Affiliations
    Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen/Institute for Quality and Efficiency in Health Care (IQWiG), Im Mediapark 8, Köln, 50670 Cologne, Germany
    Search for articles by this author
  • Tommaso Trenti
    Affiliations
    Azienda Ospedaliera Universitaria e Azienda USL di Modena, Nuovo Ospedale S. Agostino Estense, Via Giardini 1355, Modena, 41126 Italy
    Search for articles by this author
  • Miranda Langendam
    Affiliations
    Clinical Epidemiology and Biostatistics and Bioinformatics Academic Medical Center, University of Amsterdam, Meibergdreef 9, P.O.Box 227001100 DE, Amsterdam, The Netherlands
    Search for articles by this author
  • Rob Scholten
    Affiliations
    Cochrane Netherlands/Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, P.O. Box 85500, Utrecht, 3508 GA The Netherlands
    Search for articles by this author
  • Lotty Hooft
    Affiliations
    Cochrane Netherlands/Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, P.O. Box 85500, Utrecht, 3508 GA The Netherlands
    Search for articles by this author
  • Mohammad Hassan Murad
    Affiliations
    Division of Preventive Medicine, Mayo Clinic, 200 1st, ST, SW, Rochester, MN 55902, USA
    Search for articles by this author
  • Roman Jaeschke
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • Anne Rutjes
    Affiliations
    Clinical Trial Unit (CTU) Bern, Institute of Primary Health Care; Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
    Search for articles by this author
  • Jasvinder Singh
    Affiliations
    Medicine Service, VA Medical Center, Birmingham, AL, USA

    Department of Medicine, University of Alabama at Birmingham, 510, 20th Street South, Birmingham, FOT805B AL, USA
    Search for articles by this author
  • Mark Helfand
    Affiliations
    Oregon Evidence-based Practice Center, Oregon Health & Science University, Portland VA Medical Center, Portland, OR, USA
    Search for articles by this author
  • Paul Glasziou
    Affiliations
    CREBP, Faculty Health Science & Medicine, Bond University, Gold Coast, Queensland 4229, Australia
    Search for articles by this author
  • Ingrid Arevalo-Rodriguez
    Affiliations
    Clinical Biostatistics Unit, Ramón y Cajal Hospital (IRYCIS), Madrid, Spain

    Division of Research, Fundación Universitaria de Ciencias de la Salud, Hospital de San José, Hospital Infantil de San José, Bogotá, Colombia
    Search for articles by this author
  • Elie A. Akl
    Affiliations
    Department of Internal Medicine, American University of Beirut, Riad-El-Solh Beirut, Beirut, 1107 2020 Lebanon
    Search for articles by this author
  • Jonathan J. Deeks
    Affiliations
    Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT UK
    Search for articles by this author
  • Gordon H. Guyatt
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • the GRADE Working Group
Published:February 07, 2019DOI:https://doi.org/10.1016/j.jclinepi.2019.02.003

      Abstract

      Objectives

      This article describes the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group's framework of moving from test accuracy to patient or population-important outcomes. We focus on the common scenario when studies directly evaluating the effect of diagnostic and other tests or strategies on health outcomes are not available or are not providing the best available evidence.

      Study Design and Setting

      Using practical examples, we explored how guideline developers and other decision makers can use information from test accuracy to develop a recommendation by linking evidence that addresses downstream consequences. Guideline panels should develop an analytic framework that summarizes the actions that follow from applying a test and the consequences.

      Results

      We describe GRADE's current thinking about the overall certainty of the evidence (also known as quality of the evidence or confidence in the estimates) arising from consideration of the often complex pathways that involve multiple tests and management options. Each link in the evidence can—and often does—lower the overall certainty of the evidence required to formulate recommendations and make decisions about tests. The frequency with which an outcome occurs and its importance will influence whether or not a particular step in the linked evidence is critical to decision-making.

      Conclusions

      Overall certainty may be expressed by the weakest critical step in the linked evidence. The linked approach to addressing optimal testing will often require the use of decision analytic approaches. We present an example that involves decision modeling in a GRADE Evidence to Decision framework for cervical cancer screening. However, because resources and time of guideline developers may be limited, we describe alternative, pragmatic strategies for developing recommendations addressing test use.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Schünemann H.J.
        • Mustafa R.
        • Brozek J.
        • Santesso N.
        • Alonso-Coello P.
        • Guyatt G.
        • et al.
        GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health.
        J Clin Epidemiol. 2016; 76: 89-98
        • Harris R.P.
        • Helfand M.
        • Woolf S.H.
        • Lohr K.N.
        • Mulrow C.D.
        • Teutsch S.M.
        • et al.
        Current methods of the US Preventive Services Task Force: a review of the process.
        Am J Prev Med. 2001; 20: 21-35
        • Fryback D.G.
        • Thornbury J.R.
        The efficacy of diagnostic imaging.
        Med Decis Making. 1991; 11: 88-94
        • Schunemann H.J.
        • Oxman A.D.
        • Brozek J.
        • Glasziou P.
        • Bossuyt P.
        • Chang S.
        • et al.
        GRADE: assessing the quality of evidence for diagnostic recommendations.
        ACP J Club. 2008; 149: 2
        • Schunemann H.J.
        • Oxman A.D.
        • Brozek J.
        • Glasziou P.
        • Jaeschke R.
        • Vist G.E.
        • et al.
        Grading quality of evidence and strength of recommendations for diagnostic tests and strategies.
        BMJ. 2008; 336: 1106-1110
        • Schunemann H.J.
        • Mustafa R.
        • Brozek J.
        [Diagnostic accuracy and linked evidence--testing the chain].
        Z Evid Fortbild Qual Gesundhwes. 2012; 106: 153-160
        • Ferrante di Ruffano L.
        • Hyde C.J.
        • McCaffery K.J.
        • Bossuyt P.M.M.
        • Deeks J.J.
        Assessing the value of diagnostic tests: a framework for designing and evaluating trials.
        BMJ. 2012; 344: e686
        • Andrews J.
        • Guyatt G.
        • Oxman A.D.
        • Alderson P.
        • Dahm P.
        • Falck-Ytter Y.
        • et al.
        GRADE guidelines: 14. Going from evidence to recommendations: the significance and presentation of recommendations.
        J Clin Epidemiol. 2013; 66: 719-725
        • Andrews J.C.
        • Schunemann H.J.
        • Oxman A.D.
        • Pottie K.
        • Meerpohl J.J.
        • Coello P.A.
        • et al.
        GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength.
        J Clin Epidemiol. 2013; 66: 726-735
        • Barton M.B.
        • Miller T.
        • Wolff T.
        • Petitti D.
        • LeFevre M.
        • Sawaya G.
        • et al.
        How to read the new recommendation statement: methods update from the U.S. Preventive Services Task Force.
        Ann Intern Med. 2007; 147: 123-127
        • Ferrante di Ruffano L.
        • Davenport C.
        • Eisinga A.
        • Hyde C.
        • Deeks J.J.
        A capture-recapture analysis demonstrated that randomized controlled trials evaluating the impact of diagnostic tests on patient outcomes are rare.
        J Clin Epidemiol. 2012; 65: 282-287
        • Santesso N.
        • Mustafa R.A.
        • Wiercioch W.
        • Kehar R.
        • Gandhi S.
        • Chen Y.
        • et al.
        Systematic reviews and meta-analyses of benefits and harms of cryotherapy, LEEP, and cold knife conization to treat cervical intraepithelial neoplasia.
        Int J Gynaecol Obstet. 2016; 132: 266-271
        • Hsu J.
        • Brozek J.L.
        • Terracciano L.
        • Kreis J.
        • Compalati E.
        • Stein A.T.
        • et al.
        Application of GRADE: making evidence-based recommendations about diagnostic tests in clinical practice guidelines.
        Implement Sci. 2011; 6: 62
        • Fiocchi A.
        • Schunemann H.J.
        • Brozek J.
        • Restani P.
        • Beyer K.
        • Troncone R.
        • et al.
        Diagnosis and Rationale for action against Cow's milk allergy (DRACMA): a summary report.
        J Allergy Clin Immunol. 2010; 126: 1119-1128.e12
        • WHO
        Commercial Serodiagnostic Tests for Diagnosis of Tuberculosis.
        World Health Organization, Geneva2011 (ISBN 978 92 4 150205 4)
        • Santesso N.
        • Mustafa R.A.
        • Schunemann H.J.
        • Arbyn M.
        • Blumenthal P.D.
        • Cain J.
        • et al.
        World Health Organization Guidelines for treatment of cervical intraepithelial neoplasia 2-3 and screen-and-treat strategies to prevent cervical cancer.
        Int J Gynaecol Obstet. 2016; 132: 252-258
        • Mustafa R.A.
        • Santesso N.
        • Khatib R.
        • Mustafa A.A.
        • Wiercioch W.
        • Kehar R.
        • et al.
        Systematic reviews and meta-analyses of the accuracy of HPV tests, visual inspection with acetic acid, cytology, and colposcopy.
        Int J Gynaecol Obstet. 2016; 132: 259-265
        • Santesso N.
        • Schunemann H.
        • Blumenthal P.
        • De Vuyst H.
        • Gage J.
        • Garcia F.
        • et al.
        World health organization guidelines: use of cryotherapy for cervical intraepithelial neoplasia.
        Int J Gynaecol Obstet. 2012; 118: 97-102
        • Organization WH
        (Available at)
        • Fiocchi A.
        • Brozek J.
        • Schunemann H.
        • Bahna S.L.
        • von Berg A.
        • Beyer K.
        • et al.
        World allergy organization (WAO) diagnosis and Rationale for action against Cow's milk allergy (DRACMA) guidelines.
        Pediatr Allergy Immunol. 2010; 21: 1-125
        • Mustafa R.A.
        • Wiercioch W.
        • Santesso N.
        • Cheung A.
        • Prediger B.
        • Baldeh T.
        • et al.
        Decision-making about healthcare related tests and diagnostic strategies: User testing of GRADE evidence tables.
        PLoS One. 2015; 10: e0134553
        • Spencer F.A.
        • Iorio A.
        • You J.
        • Murad M.H.
        • Schunemann H.J.
        • Vandvik P.O.
        • et al.
        Uncertainties in baseline risk estimates and confidence in treatment effects.
        BMJ. 2012; 345: e7401
        • Iorio A.
        • Spencer F.A.
        • Falavigna M.
        • Alba C.
        • Lang E.
        • Burnand B.
        • et al.
        Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients.
        BMJ. 2015; 350: h870
        • Breheny K.
        • Sutton A.J.
        • Deeks J.J.
        Model based economic evaluations of diagnostic point of care tests were rarely fit for purpose.
        J Clin Epidemiol. 2018; ([Epub ahead of print])https://doi.org/10.1016/j.jclinepi.2018.11.003
        • Lord S.J.
        • Irwig L.
        • Simes R.J.
        When is measuring sensitivity and specificity sufficient to evaluate a diagnostic test, and when do we need randomized trials?.
        Ann Intern Med. 2006; 144: 850-855
        • Schünemann H.J.
        • Hill S.R.
        • Kakad M.
        • Vist G.E.
        • Bellamy R.
        • Stockman L.
        • et al.
        Transparent development of the WHO rapid advice guidelines.
        PLoS Med. 2007; 4: e119
        • Treweek S.
        • Oxman A.D.
        • Alderson P.
        • Bossuyt P.M.
        • Brandt L.
        • Brozek J.
        • et al.
        Developing and evaluating Communication strategies to Support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results.
        Implement Sci. 2013; 8: 6
        • Alonso-Coello P.
        • Oxman A.D.
        • Moberg J.
        • Brignardello-Petersen R.
        • Akl E.A.
        • Davoli M.
        • et al.
        GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: clinical practice guidelines.
        BMJ. 2016; 353: i2089
        • Alonso-Coello P.
        • Schunemann H.J.
        • Moberg J.
        • Brignardello-Petersen R.
        • Akl E.A.
        • Davoli M.
        • et al.
        GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.
        BMJ. 2016; 353: i2016
        • Bates S.M.
        • Jaeschke R.
        • Stevens S.M.
        • Goodacre S.
        • Wells P.S.
        • Stevenson M.D.
        • et al.
        Diagnosis of DVT: Antithrombotic therapy and Prevention of Thrombosis, 9th ed.: American College of chest Physicians evidence-based clinical practice guidelines.
        Chest. 2012; 141: e351S-e418S