This paper is only available as a PDF. To read, Please Download here.
HIGHLIGHTS
- ●When assessing inconsistency, GRADE users should consider both relative and absolute effects
- ●Ratings of inconsistency may differ depending on the target of certainty rating
- ●Reviewers need to understand contexts when I2 is large but inconsistency is not important.
- ●The ICEMAN instrument for assessing the credibility of subgroup effects is fully compatible with GRADE's approach to rating inconsistency.
Abstract
Objectives
To update previous GRADE guidance addressing inconsistency and interpreting subgroup analyses.
Study Design and Setting
Using an iterative process, we consulted with members of the GRADE working group through multiple rounds of written feedback and discussions at GRADE working group meetings.
Results
The guidance complements previous guidance with clarification on two areas: (1) assessing inconsistency and (2) assessing the credibility of possible effect modifiers that might explain inconsistency. Specifically, the guidance clarifies that inconsistency refers to variability in results, not variability in study characteristics; that inconsistency assessment for binary outcomes requires consideration of both relative and absolute effects; how to decide between narrower and broader questions in systematic reviews and guidelines; that, with the same evidence, ratings of inconsistency may differ depending on the target of certainty rating; and how GRADE inconsistency ratings relate to a statistical measure of inconsistency I2 depending on the context in which one views results. The second part of the guidance illustrates, based on a worked example, the use of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN). The guidance explains the stepwise process of moving from a subgroup analysis to assessing the credibility of effect modification and, if found credible, to subgroup-specific effect estimates and GRADE certainty ratings.
Conclusion
This updated guidance addresses specific conceptual and practical issues that systematic review authors frequently face when considering the degree of inconsistency in estimates of treatment effects across studies.
Keywords
Article info
Publication history
Accepted:
March 1,
2023
Received in revised form:
January 30,
2023
Received:
December 8,
2022
Publication stage
In Press Journal Pre-ProofIdentification
Copyright
© 2023 The Author(s). Published by Elsevier Inc.
User license
Creative Commons Attribution (CC BY 4.0) | How you can reuse 
Elsevier's open access license policy
Creative Commons Attribution (CC BY 4.0)
Permitted
- Read, print & download
- Redistribute or republish the final article
- Text & data mine
- Translate the article
- Reuse portions or extracts from the article in other works
- Sell or re-use for commercial purposes
Elsevier's open access license policy
