Abstract
Objectives
The Grading of Recommendations Assessment, Development, and Evaluations (GRADE) system
for assessing certainty in a body of evidence currently uses two levels, serious and
very serious, for downgrading on a single domain. In the context of newer risk of
bias instruments, such as Risk of Bias in Non-Randomized Studies I (ROBINS-I), evidence
generated by nonrandomized studies may justify rating down by more than two levels
on a single domain. Given the importance users of GRADE assign to terminology, our
objective was to assess what term GRADE stakeholders would prefer for rating down
certainty by three levels.
Study Design and Setting
We conducted a purposefully sampled online survey of GRADE stakeholders to assess
possible terms including “critically serious,” “extremely serious,” “most serious,”
and “very, very serious” and conducted a descriptive and thematic analysis of responses.
We then facilitated a GRADE working group workshop to generate consensus.
Results
A total of 225 respondents ranked and rated “extremely serious” highest, closely followed
by “critically serious.” Respondents felt that “extremely serious” was “more understandable”
and “easiest to interpret”. GRADE working group members described that the terms “extremely
serious” appeared clearer and easier to translate in other languages.
Conclusion
Based on this stakeholder-driven study, “extremely serious” is the preferred term
to rate down certainty of evidence by three levels in the GRADE approach.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- GRADE Working Group.(Available at)www.gradeworkingroup.orgDate accessed: September 24, 2018
- Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations.CMAJ. 2003; 169: 677-680
- Grading quality of evidence and strength of recommendations.BMJ. 2004; 328: 1490https://doi.org/10.1136/bmj.328.7454.1490
- GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.BMJ. 2008; 336: 924-926
- Interpreting GRADE’s levels of certainty or quality of the evidence: GRADE for statisticians, considering review information size or less emphasis on imprecision?.J Clin Epidemiol. 2016; 75: 6-15
- GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables.J Clin Epidemiol. 2011; 64: 383-394
- GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias).J Clin Epidemiol. 2011; 64: 407-415
- ROBINS-I : a tool for assessing risk of bias in non-randomised studies of interventions.Br Med J. 2016; : 4-10
- Evaluation of the risk of bias in non-randomized studies of interventions (ROBINS-I) and the ‘target experiment’ concept in studies of exposures: rationale and preliminary instrument development.Environ Int. 2018; 120: 382-387
- GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence.J Clin Epidemiol. 2019; 111: 105-114
- Survey Monkey.2018 (Available at)www.surveymonkey.comDate accessed: December 11, 2018
Article info
Publication history
Published online: December 19, 2019
Accepted:
November 13,
2019
Footnotes
Conflicts of interest: H.J.S. is one of two co-chairs of the GRADE working group, all other authors are members of the GRADE working group.
Identification
Copyright
© 2020 Elsevier Inc. All rights reserved.
