Abstract
Objective
To provide guidance on how systematic review authors, guideline developers, and health
technology assessment practitioners should approach the use of the risk of bias in
nonrandomized studies of interventions (ROBINS-I) tool as a part of GRADE's certainty
rating process.
Study Design and Setting
The study design and setting comprised iterative discussions, testing in systematic
reviews, and presentation at GRADE working group meetings with feedback from the GRADE
working group.
Results
We describe where to start the initial assessment of a body of evidence with the use
of ROBINS-I and where one would anticipate the final rating would end up. The GRADE
accounted for issues that mitigate concerns about confounding and selection bias by
introducing the upgrading domains: large effects, dose-effect relations, and when
plausible residual confounders or other biases increase certainty. They will need
to be considered in an assessment of a body of evidence when using ROBINS-I.
Conclusions
The use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence
from randomized controlled trials (RCTs) and nonrandomized studies (NRSs) because
they are placed on a common metric for risk of bias. Challenges remain, including
appropriate presentation of evidence from RCTs and NRSs for decision-making and how
to optimally integrate RCTs and NRSs in an evidence assessment.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables.J Clin Epidemiol. 2011; 64: 383-394
- GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology.J Clin Epidemiol. 2011; 64: 380-382
- A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis.BMJ. 2014; 349: g5630
- Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations.CMAJ. 2003; 169: 677-680
- GRADE: assessing the quality of evidence for diagnostic recommendations.ACP J Club. 2008; 149: 2
- Uncertainties in baseline risk estimates and confidence in treatment effects.BMJ. 2012; 345: e7401
- Grading quality of evidence and strength of recommendations for diagnostic tests and strategies.BMJ. 2008; 336: 1106-1110
- GRADE guidelines: 11. Making an overall rating of confidence in effect estimates for a single outcome and for all outcomes.J Clin Epidemiol. 2013; 66: 151-157
- GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: clinical practice guidelines.BMJ. 2016; 353: i2089
- GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.BMJ. 2016; 353: i2016
- GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health.J Clin Epidemiol. 2016; 76: 89-98
- Assessing evidence in public health: the added value of GRADE.J Public Health (Oxf). 2012; 34: 631-635
- Using GRADE to respond to health questions with different levels of urgency.Environ Int. 2016; 92-93: 585-589
- Transparent development of the WHO rapid advice guidelines.PLoS Med. 2007; 4: e119
- Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients.BMJ. 2015; 350: h870
- GRADE guidelines 17: assessing the risk of bias associated with missing participant outcome data in a body of evidence.J Clin Epidemiol. 2017; 87: 14-22
- GRADE: assessing the quality of evidence in environmental and occupational health.Environ Int. 2016; 92-93: 611-616
- Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography.Int J Epidemiol. 2007; 36: 666-676
- ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions.BMJ. 2016; 355: i4919
- How do systematic reviews incorporate risk of bias assessments into the synthesis of evidence? A methodological study.J Epidemiol Community Health. 2015; 69: 189-195
- Grading quality of evidence and strength of recommendations.BMJ. 2004; 328: 1490
- Natural experiments: an overview of methods, approaches, and contributions to public health intervention research.Annu Rev Public Health. 2017; 38: 39-56
- GRADE guidelines: 9. Rating up the quality of evidence.J Clin Epidemiol. 2011; 64: 1311-1316
- The GRADE Working Group clarifies the construct of certainty of evidence.J Clin Epidemiol. 2017; 87: 4-13
- A systematic review and meta-analysis of studies comparing mortality rates of private for-profit and private not-for-profit hospitals.CMAJ. 2002; 166: 1399-1406
Article info
Publication history
Published online: February 09, 2018
Accepted:
January 11,
2018
Footnotes
Conflict of interest: H.J.S. has no direct financial conflict of interest and other authors have not declared financial conflicts of interest. Part of the work has been presented at scientific conferences and at GRADE working group meetings. This article has been officially endorsed by the GRADE working group.
Identification
Copyright
© 2018 Elsevier Inc. All rights reserved.
