Grading of Recommendations Assessment, Development, and Evaluations (GRADE) notes: extremely serious, GRADE's terminology for rating down by three levels

Published:December 19, 2019DOI:



      The Grading of Recommendations Assessment, Development, and Evaluations (GRADE) system for assessing certainty in a body of evidence currently uses two levels, serious and very serious, for downgrading on a single domain. In the context of newer risk of bias instruments, such as Risk of Bias in Non-Randomized Studies I (ROBINS-I), evidence generated by nonrandomized studies may justify rating down by more than two levels on a single domain. Given the importance users of GRADE assign to terminology, our objective was to assess what term GRADE stakeholders would prefer for rating down certainty by three levels.

      Study Design and Setting

      We conducted a purposefully sampled online survey of GRADE stakeholders to assess possible terms including “critically serious,” “extremely serious,” “most serious,” and “very, very serious” and conducted a descriptive and thematic analysis of responses. We then facilitated a GRADE working group workshop to generate consensus.


      A total of 225 respondents ranked and rated “extremely serious” highest, closely followed by “critically serious.” Respondents felt that “extremely serious” was “more understandable” and “easiest to interpret”. GRADE working group members described that the terms “extremely serious” appeared clearer and easier to translate in other languages.


      Based on this stakeholder-driven study, “extremely serious” is the preferred term to rate down certainty of evidence by three levels in the GRADE approach.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. GRADE Working Group.
        (Available at)
        Date accessed: September 24, 2018
        • Schunemann H.J.
        • Best D.
        • Vist G.
        • Oxman A.D.
        Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations.
        CMAJ. 2003; 169: 677-680
        • Atkins D.
        • Best D.
        • Briss P.A.
        • Eccles M.
        • Falck-Ytter Y.
        • Flottorp S.
        • et al.
        Grading quality of evidence and strength of recommendations.
        BMJ. 2004; 328: 1490
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.E.
        • Kunz R.
        • Falck-Ytter Y.
        • Alonso-Coello P.
        • et al.
        GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
        BMJ. 2008; 336: 924-926
        • Schunemann H.J.
        Interpreting GRADE’s levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision?.
        J Clin Epidemiol. 2016; 75: 6-15
        • Guyatt G.
        • Oxman A.D.
        • Akl E.A.
        • Kunz R.
        • Vist G.
        • Brozek J.
        • et al.
        GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables.
        J Clin Epidemiol. 2011; 64: 383-394
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.
        • Kunz R.
        • Brozek J.
        • Alonso-Coello P.
        • et al.
        GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias).
        J Clin Epidemiol. 2011; 64: 407-415
        • Sterne J.A.C.
        • Hernán M.A.
        • Reeves B.C.
        • Savović J.
        • Berkman N.D.
        • Viswanathan M.
        • et al.
        ROBINS-I : a tool for assessing risk of bias in non-randomised studies of interventions.
        Br Med J. 2016; : 4-10
        • Morgan R.
        • Thayer K.
        • Santesso N.
        • Holloway A.
        • Blain R.
        • Eftim S.
        • et al.
        Evaluation of the risk of bias in non-randomized studies of interventions (ROBINS-I) and the ‘target experiment’ concept in studies of exposures: rationale and preliminary instrument development.
        Environ Int. 2018; 120: 382-387
        • Schunemann H.
        • Cuello C.
        • Akl E.
        • Mustafa R.
        • Meerpohl J.
        • Thayer K.
        GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence.
        J Clin Epidemiol. 2019; 111: 105-114
        • SurveyMonkey Inc
        Survey Monkey.
        2018 (Available at)
        Date accessed: December 11, 2018