Original Article| Volume 121, P62-70, May 2020

Download started.


GRADE Guidelines 28: Use of GRADE for the assessment of evidence about prognostic factors: rating certainty in identification of groups of patients with different absolute risks



      The objective of this study was to provide guidance on the use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach to determine certainty in estimates of association between prognostic factors and future outcomes.

      Study Design and Setting

      We developed our guidance through an iterative process that involved review of published systematic reviews and meta-analyses of prognostic factors, consultation with members, feedback, presentation, and discussion at the GRADE Working Group meetings.


      For questions of prognosis, a body of observational evidence (potentially including patients enrolled in randomized controlled trials) begins as high certainty in the evidence. The five domains of GRADE for rating down certainty in the evidence, that is, risk of bias, imprecision, inconsistency, indirectness, and publication bias, as well as the domains for rating up, also apply to estimates of associations between prognostic factors and outcomes. One should determine if their ratings do not consider (noncontextualized) or consider (contextualized) the clinical context as this will may result in variable judgments on certainty of the evidence.


      The same principles GRADE proposed for bodies of evidence addressing treatment and overall prognosis work well in assessing individual prognostic factors, both in noncontextualized and contextualized settings.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Riley R.D.
        • van der Windt D.A.
        • Croft P.
        • Moons K.G.
        Prognosis research in health care: concepts, methods, and impact.
        Oxford University Press, Oxford, United Kingdom2019
        • Riley R.D.
        • Hayden J.A.
        • Steyerberg E.W.
        • Moons K.G.
        • Abrams K.
        • Kyzas P.A.
        • et al.
        Prognosis Research Strategy (PROGRESS) 2: prognostic factor research.
        PLoS Med. 2013; 10: e1001380
        • Hultcrantz M.
        • Rind D.
        • Akl E.A.
        • Treweek S.
        • Mustafa R.A.
        • Iorio A.
        • et al.
        The GRADE Working Group clarifies the construct of certainty of evidence.
        J Clin Epidemiol. 2017; 87: 4-13
        • Thompson M.
        • Vodicka T.A.
        • Blair P.S.
        • Buckley D.I.
        • Heneghan C.
        • Hay A.D.
        • et al.
        Duration of symptoms of respiratory tract infections in children: systematic review.
        BMJ. 2013; 347: f7027
        • Iorio A.
        • Spencer F.A.
        • Falavigna M.
        • Alba C.
        • Lang E.
        • Burnand B.
        • et al.
        Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients.
        BMJ. 2015; 350: h870
        • Hordijk-Trion M.
        • on behalf of the EHSCRI
        • Lenzen M.
        • Wijns W.
        • de Jaegere P.
        • Simoons M.L.
        • et al.
        Patients enrolled in coronary intervention trials are not representative of patients in clinical practice: results from the Euro Heart Survey on Coronary Revascularization.
        Eur Heart J. 2006; 27: 671-678
        • Hayden J.A.
        • van der Windt D.A.
        • Cartwright J.L.
        • Côté P.
        • Bombardier C.
        Assessing bias in studies of prognostic factors.
        Ann Intern Med. 2013; 158: 280-286
        • Wolff R.F.
        • Moons K.G.M.
        • Riley R.D.
        • Whiting P.F.
        • Westwood M.
        • Collins G.S.
        • et al.
        PROBAST: a tool to assess the risk of bias and applicability of prediction model studies.
        Ann Intern Med. 2019; 170: 51-58
        • Reeves B.C.
        • Deeks J.J.
        • Higgins J.P.
        • Wells G.A.
        Including non-randomized studies.
        in: Higgins J.P. Green S. Cochrane Handbook for Systematic Reviews of Interventions. The Cochrane Collaboration, Hoboken, New Jersey2010: 391-432
        • Cook D.J.
        • Fuller H.D.
        • Guyatt G.H.
        • Marshall J.C.
        • Leasa D.
        • Hall R.
        • et al.
        Risk factors for gastrointestinal bleeding in critically ill patients.
        N Engl J Med. 1994; 330: 377-381
        • Kucher N.
        • Kohler H.P.
        • Dornhofer T.
        • Wallmann D.
        • Lämmle B.
        Accuracy of D-dimer/fibrinogen ratio to predict pulmonary embolism: a prospective diagnostic study.
        J Thromb Haemost. 2003; 1: 708-713
        • Perrier A.
        • Roy P.M.
        • Sanchez O.
        • Le Gal G.
        • Meyer G.
        • Gourdier A.L.
        • et al.
        Multidetector-row computed tomography in suspected pulmonary embolism.
        N Engl J Med. 2005; 352: 1760-1768
        • Wells P.S.
        • Anderson D.R.
        • Bormanis J.
        • Guy F.
        • Mitchell M.
        • Gray L.
        • et al.
        Value of assessment of pretest probability of deep-vein thrombosis in clinical management.
        Lancet. 1997; 350: 1795-1798
        • Sanchez O.
        • Trinquart L.
        • Colombet I.
        • Durieux P.
        • Huisman M.V.
        • Chatellier G.
        • et al.
        Prognostic value of right ventricular dysfunction in patients with haemodynamically stable pulmonary embolism: a systematic review.
        Eur Heart J. 2008; 29: 1569-1577
        • Cheng Y.J.
        • Liu Z.H.
        • Yao F.J.
        • Zeng W.T.
        • Zheng D.D.
        • Dong Y.G.
        • et al.
        Current and former smoking and risk for venous thromboembolism: a systematic review and meta-analysis.
        PLoS Med. 2013; 10: e1001515
        • Witlox J.
        • Eurelings L.S.
        • de Jonghe J.F.
        • Kalisvaart K.J.
        • Eikelenboom P.
        • van Gool W.A.
        Delirium in elderly patients and the risk of postdischarge mortality, institutionalization, and dementia: a meta-analysis.
        JAMA. 2010; 304: 443-451
        • Foroutan F.
        • Friesen E.L.
        • Clark K.E.
        • Motaghi S.
        • Zyla R.
        • Lee Y.
        • et al.
        Risk factors for one-year graft loss post-kidney transplantation.
        Clin J Am Soc Nephrol. 2019; 14: 1642-1650
        • Andre F.
        • McShane L.M.
        • Michiels S.
        • Ransohoff D.F.
        • Altman D.G.
        • Reis-Filho J.S.
        • et al.
        Biomarker studies: a call for a comprehensive biomarker study registry.
        Nat Rev Clin Oncol. 2011; 8: 171
        • Kyzas P.A.
        • Denaxa-Kyza D.
        • Ioannidis J.P.A.
        Almost all articles on cancer prognostic markers report statistically significant results.
        Eur J Cancer. 2007; 43: 2559-2579
        • Martin B.
        • Paesmans M.
        • Berghmans T.
        • Branle F.
        • Ghisdal L.
        • Mascaux C.
        • et al.
        Role of Bcl-2 as a prognostic factor for survival in lung cancer: a systematic review of the literature with meta-analysis.
        Br J Cancer. 2003; 89: 55
        • McShane L.M.
        • Altman D.G.
        • Sauerbrei W.
        Identification of clinically useful cancer prognostic factors: what are we missing?.
        J Natl Cancer Inst. 2005; 97: 1023-1025
        • Begg C.B.
        • Mazumdar M.
        Operating characteristics of a rank correlation test for publication bias.
        Biometrics. 1994; 50: 1088-1101
        • Debray T.P.A.
        • Moons K.G.M.
        • Riley R.D.
        Detecting small-study effects and funnel plot asymmetry in meta-analysis of survival data: a comparison of new and existing tests.
        Res Synth Methods. 2018; 9: 41-50
        • Peters J.L.
        • Sutton A.J.
        • Jones D.R.
        • Abrams K.R.
        • Rushton L.
        Comparison of two methods to detect publication bias in meta-analysis.
        JAMA. 2006; 295: 676-680
        • Egger M.
        • Davey Smith G.
        • Schneider M.
        • Minder C.
        Bias in meta-analysis detected by a simple, graphical test.
        BMJ. 1997; 315: 629-634
        • Vasilevska M.
        • Ross S.A.
        • Gesink D.
        • Fisman D.N.
        Relative risk of cervical cancer in indigenous women in Australia, Canada, New Zealand, and the United States: a systematic review and meta-analysis.
        J Public Health Policy. 2012; 33: 148-164