GRADE guidelines: 21 part 2. Test accuracy: inconsistency, imprecision, publication bias, and other domains for rating the certainty of evidence and presenting it in evidence profiles and summary of findings tables

  • Holger J. Schünemann
    Correspondence
    Corresponding author: Tel.: 1-905-525-9140x24931; fax: 1 905-522-9507.
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • Reem A. Mustafa
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, University of Kansas Medical Center, Kansas City, KS, USA
    Search for articles by this author
  • Jan Brozek
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • Karen R. Steingart
    Affiliations
    Department of Clinical Sciences, Liverpool School of Tropical Medicine, Pembroke Place, Liverpool L3 5QA, UK
    Search for articles by this author
  • Mariska Leeflang
    Affiliations
    Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, Room J1b-214, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
    Search for articles by this author
  • Mohammad Hassan Murad
    Affiliations
    Division of Preventive Medicine, Mayo Clinic, 200 1st ST. SW, Rochester, MN 55902, USA
    Search for articles by this author
  • Patrick Bossuyt
    Affiliations
    Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, Room J1b-214, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
    Search for articles by this author
  • Paul Glasziou
    Affiliations
    CREBP, Faculty Health Science & Medicine, Bond University, Gold Coast, Queensland 4229, Australia
    Search for articles by this author
  • Roman Jaeschke
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • Stefan Lange
    Affiliations
    Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen/Institute for Quality and Efficiency in Health Care (IQWiG), Im Mediapark 8, 50670 Köln, Germany
    Search for articles by this author
  • Joerg Meerpohl
    Affiliations
    Institute for Evidence in Medicine, Medical Center - University of Freiburg, Faculty of Medicine, University of Freiburg, Freiburg, Germany

    Cochrane Germany, Cochrane Germany Foundation, Freiburg, Germany
    Search for articles by this author
  • Miranda Langendam
    Affiliations
    Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, Room J1b-214, Meibergdreef 9, 1105 AZ Amsterdam, The Netherlands
    Search for articles by this author
  • Monica Hultcrantz
    Affiliations
    Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU), S:t Eriksgatan 117, SE-102 33 Stockholm, Sweden
    Search for articles by this author
  • Gunn E. Vist
    Affiliations
    Norwegian Knowledge Centre for the Health Services, PO Box 7004, St Olavs Plass, 0130 Oslo, Norway
    Search for articles by this author
  • Elie A. Akl
    Affiliations
    Department of Internal Medicine, American University of Beirut, Riad-El-Solh Beirut, Beirut 1107 2020, Lebanon
    Search for articles by this author
  • Mark Helfand
    Affiliations
    Oregon Evidence-Based Practice Center, Oregon Health & Science University, Portland VA Medical Center, Portland, OR, USA
    Search for articles by this author
  • Nancy Santesso
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • Lotty Hooft
    Affiliations
    Cochrane Netherlands/Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, P.O. Box 85500, 3508 GA Utrecht, The Netherlands
    Search for articles by this author
  • Rob Scholten
    Affiliations
    Cochrane Netherlands/Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, P.O. Box 85500, 3508 GA Utrecht, The Netherlands
    Search for articles by this author
  • Måns Rosen
    Affiliations
    Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU), S:t Eriksgatan 117, SE-102 33 Stockholm, Sweden
    Search for articles by this author
  • Anne Rutjes
    Affiliations
    Clinical Trial Unit (CTU) Bern, Institute of Primary Health Care, Institute of Social and Preventive Medicine, University of Bern, Switzerland
    Search for articles by this author
  • Mark Crowther
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada
    Search for articles by this author
  • Paola Muti
    Affiliations
    Department of Biomedical Science and Oral Health, University of Milan, Via Commenda 10, Milan, Italy
    Search for articles by this author
  • Heike Raatz
    Affiliations
    University of Basel, Klingelbergstrasse 61, CH-4056 Basel, Switzerland

    Kleijnen Systematic Reviews Ltd., 6 Escrick Business Park, Escrick, York YO19 6FD, UK
    Search for articles by this author
  • Mohammed T. Ansari
    Affiliations
    Faculty of Medicine, School of Epidemiology and Public Health, Ottawa, Canada
    Search for articles by this author
  • John Williams
    Affiliations
    Duke University Medical Center and Durham Veterans Affairs Center for Health Services Research in Primary Care Durham, NC 27705, USA
    Search for articles by this author
  • Regina Kunz
    Affiliations
    Basel Institute of Clinical Epidemiology, University Hospital Basel, Hebelstrasse 10, Basel 4031, Switzerland
    Search for articles by this author
  • Jeff Harris
    Affiliations
    Harris Associates, 386 Richardson Way, Mill Valley, CA 94941, USA
    Search for articles by this author
  • Ingrid Arévalo Rodriguez
    Affiliations
    Clinical Biostatistics Unit, Ramón y Cajal Hospital (IRYCIS), Madrid, Spain

    Centro de investigación en Salud Pública y Epidemiología Clínica, Facultad de Ciencias de la Salud Eugenio Espejo, Universidad UTE, Quito, Ecuador
    Search for articles by this author
  • Mikashmi Kohli
    Affiliations
    Department of Epidemiology, Biostatistics and Occupational Health, McGill University, 1650 Cedar Ave, Montreal, Quebec H3G 1A4, Canada
    Search for articles by this author
  • Gordon H. Guyatt
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster GRADE Centre, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, McMaster University, 1280 Main Street West, Hamilton, Ontario L8S4K1, Canada

    Department of Medicine, University of Kansas Medical Center, Kansas City, KS, USA
    Search for articles by this author
  • for theGRADE Working Group
Published:February 10, 2020DOI:https://doi.org/10.1016/j.jclinepi.2019.12.021

      Abstract

      Objectives

      This article provides updated GRADE guidance about how authors of systematic reviews and health technology assessments and guideline developers can rate the certainty of evidence (also known as quality of the evidence or confidence in the estimates) of a body of evidence addressing test accuracy (TA) on the domains imprecision, inconsistency, publication bias, and other domains. It also provides guidance for how to present synthesized information in evidence profiles and summary of findings tables.

      Study Design and Setting

      We present guidance for rating certainty in TA in clinical and public health and review the presentation of results of a body of evidence regarding tests.

      Results

      Supplemented by practical examples, we describe how raters of the evidence can apply the GRADE domains inconsistency, imprecision, and publication bias to a body of evidence of TA studies.

      Conclusion

      Using GRADE in Cochrane and other reviews as well as World Health Organization and other guidelines helped refining the GRADE approach for rating the certainty of a body of evidence from TA studies. Although several of the GRADE domains (e.g., imprecision and magnitude of the association) require further methodological research to help operationalize them, judgments need to be made on the basis of what is known so far.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Schunemann H.J.
        • Mustafa R.A.
        • Brozek J.
        • Steingart K.
        • Leeflang M.
        • Murad H.M.
        • et al.
        GRADE guidelines: 21 part 1. Study design, risk of bias and indirectness in rating the certainty across a body of evidence for test accuracy.
        J Clin Epidemiol. 2020; ([Epub ahead of print])https://doi.org/10.1016/j.jclinepi.2019.12.020
        • Santesso N.
        • Mustafa R.A.
        • Schunemann H.J.
        • Arbyn M.
        • Blumenthal P.D.
        • Cain J.
        • et al.
        World Health Organization Guidelines for treatment of cervical intraepithelial neoplasia 2-3 and screen-and-treat strategies to prevent cervical cancer.
        Int J Gynaecol Obstet. 2016; 132: 252-258
        • Fiocchi A.
        • Brozek J.
        • Schunemann H.
        • Bahna S.L.
        • von Berg A.
        • Beyer K.
        • et al.
        World allergy organization (WAO) diagnosis and rationale for action against cow's milk allergy (DRACMA) guidelines.
        Pediatr Allergy Immunol. 2010; 21: 1-125
        • Schunemann H.J.
        • Mustafa R.
        • Brozek J.
        • Santesso N.
        • Alonso-Coello P.
        • Guyatt G.
        • et al.
        GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health.
        J Clin Epidemiol. 2016; 76: 89-98
        • Schunemann H.J.
        • Mustafa R.A.
        • Brozek J.
        • Santesso N.
        • Bossuyt P.M.
        • Steingart K.R.
        • et al.
        GRADE guidelines: 22. The GRADE approach for tests and strategies-from test accuracy to patient-important outcomes and recommendations.
        J Clin Epidemiol. 2019; 111: 69-82
        • Schunemann H.J.
        • Oxman A.D.
        • Brozek J.
        • Glasziou P.
        • Jaeschke R.
        • Vist G.E.
        • et al.
        Grading quality of evidence and strength of recommendations for diagnostic tests and strategies.
        BMJ. 2008; 336: 1106-1110
        • Brozek J.L.
        • Akl E.A.
        • Jaeschke R.
        • Lang D.M.
        • Bossuyt P.
        • Glasziou P.
        • et al.
        Grading quality of evidence and strength of recommendations in clinical practice guidelines: Part 2 of 3. The GRADE approach to grading quality of evidence about diagnostic tests and strategies.
        Allergy. 2009; 64: 1109-1116
        • Kohli M.
        • Schiller I.
        • Dendukuri N.
        • Dheda K.
        • Denkinger C.M.
        • Schumacher S.G.
        • et al.
        Xpert((R)) MTB/RIF assay for extrapulmonary tuberculosis and rifampicin resistance.
        Cochrane Database Syst Rev. 2018; : CD012768
        • Steingart K.R.
        • Flores L.L.
        • Dendukuri N.
        • Schiller I.
        • Laal S.
        • Ramsay A.
        • et al.
        Commercial serological tests for the diagnosis of active pulmonary and extrapulmonary tuberculosis: an updated systematic review and meta-analysis.
        PLoS Med. 2011; 8: e1001062
        • Metcalfe J.Z.
        • Everett C.K.
        • Steingart K.R.
        • Cattamanchi A.
        • Huang L.
        • Hopewell P.C.
        • et al.
        Interferon-gamma release assays for active pulmonary tuberculosis diagnosis in adults in low- and middle-income countries: systematic review and meta-analysis.
        J Infect Dis. 2011; 204: S1120-S1129
        • Janda S.
        • Swiston J.
        Diagnostic accuracy of pleural fluid NT-pro-BNP for pleural effusions of cardiac origin: a systematic review and meta-analysis.
        BMC Pulm Med. 2010; 10: 58
        • Schunemann H.J.
        Interpreting GRADE's levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision?.
        J Clin Epidemiol. 2016; 75: 6-15
        • Deeks J.J.
        • Macaskill P.
        • Irwig L.
        The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed.
        J Clin Epidemiol. 2005; 58: 882-893
        • Duval S.
        • Tweedie R.
        Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis.
        Biometrics. 2000; 56: 455-463
        • Burkner P.C.
        • Doebler P.
        Testing for publication bias in diagnostic meta-analysis: a simulation study.
        Stat Med. 2014; 33: 3061-3077
        • Li C.R.
        • Liao C.T.
        • Liu J.P.
        A non-inferiority test for diagnostic accuracy based on the paired partial areas under ROC curves.
        Stat Med. 2008; 27: 1762-1776
        • Sarigianni M.
        • Liakos A.
        • Vlachaki E.
        • Paschos P.
        • Athanasiadou E.
        • Montori V.M.
        • et al.
        Accuracy of magnetic resonance imaging in diagnosis of liver iron overload: a systematic review and meta-analysis.
        Clin Gastroenterol Hepatol. 2015; 13: 55-63 e5
        • Wang Z.
        • Pianosi P.T.
        • Keogh K.A.
        • Zaiem F.
        • Alsawas M.
        • Alahdab F.
        • et al.
        The diagnostic accuracy of fractional exhaled nitric oxide testing in asthma: a systematic review and meta-analyses.
        Mayo Clin Proc. 2018; 93: 191-198
        • Hill S.A.
        • Devereaux P.J.
        • Griffith L.
        • Opie J.
        • McQueen M.J.
        • Panju A.
        • et al.
        Can troponin I measurement predict short-term serious cardiac outcomes in patients presenting to the emergency department with possible acute coronary syndrome?.
        CJEM. 2004; 6: 22-30
        • Guyatt G.H.
        • Oxman A.D.
        • Sultan S.
        • Glasziou P.
        • Akl E.A.
        • Alonso-Coello P.
        • et al.
        GRADE guidelines: 9. Rating up the quality of evidence.
        J Clin Epidemiol. 2011; 64: 1311-1316
        • Guyatt G.
        • Oxman A.D.
        • Akl E.A.
        • Kunz R.
        • Vist G.
        • Brozek J.
        • et al.
        GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables.
        J Clin Epidemiol. 2011; 64: 383-394
        • Mustafa R.A.
        • Wiercioch W.
        • Santesso N.
        • Cheung A.
        • Prediger B.
        • Baldeh T.
        • et al.
        Decision-making about healthcare related tests and diagnostic strategies: user testing of GRADE evidence tables.
        PLoS One. 2015; 10: e0134553
        • Hultcrantz M.
        • Mustafa R.A.
        • Leeflang M.M.G.
        • Lavergne V.
        • Estrada-Orozco K.
        • Ansari M.T.
        • et al.
        Defining ranges for certainty ratings of diagnostic accuracy: a GRADE concept paper.
        J Clin Epidemiol. 2020; 117: 138-148
        • Hsu J.
        • Brozek J.L.
        • Terracciano L.
        • Kreis J.
        • Compalati E.
        • Stein A.T.
        • et al.
        Application of GRADE: making evidence-based recommendations about diagnostic tests in clinical practice guidelines.
        Implement Sci. 2011; 6: 62