Abstract
Objectives
The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working
group defines patient values and preferences as the relative importance patients place
on the main health outcomes. We provide GRADE guidance for assessing the risk of bias
and indirectness domains for certainty of evidence about the relative importance of
outcomes.
Study Design and Setting
We applied the GRADE domains to rate the certainty of evidence in the importance of
outcomes to several systematic reviews, iteratively reviewed draft guidance and consulted
GRADE members and other stakeholders for feedback.
Results
This is the first of two articles. A body of evidence addressing the importance of
outcomes starts at “high certainty”; concerns with risk of bias, indirectness, inconsistency,
imprecision, and publication bias lead to downgrading to moderate, low, or very low
certainty. We propose subdomains of risk of bias as selection of the study population,
missing data, the type of measurement instrument, and confounding; we have developed
items for each subdomain. The population, intervention, comparison, and outcome elements
associated with the evidence determine the degree of indirectness.
Conclusion
This article provides guidance and examples for rating the risk of bias and indirectness
for a body of evidence summarizing the importance of outcomes.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Improving the use of research evidence in guideline development: 10. Integrating values and consumer involvement.Health Res Policy Syst. 2006; 4: 22
- Incorporating patient preferences in evidence-based medicine.JAMA. 2008; 300 (author reply-4): 2483
- Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise.CMAJ. 2014; 186: E123-E142
- Patient values and preferences in decision making for antithrombotic therapy: a systematic review: antithrombotic therapy and prevention of thrombosis, 9th ed.: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines.Chest. 2012; 141: e1S-e23S
- GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation's direction and strength.J Clin Epidemiol. 2013; 66: 726-735
- The next step in guideline development: incorporating patient preferences.JAMA. 2008; 300: 436-438
- How to integrate individual patient values and preferences in clinical practice guidelines? A research protocol.Implement Sci. 2010; 5: 10
- The standard gamble method: what is being measured and how it is interpreted.Health Serv Res. 1994; 29: 207-224
- Measurement of health state utilities for economic appraisal.J Health Econ. 1986; 5: 1-30
- Utility measurement in healthcare: the things I never got to.PharmacoEconomics. 2006; 24: 1069-1078
- Measurement of quality of life in end-stage renal disease: the time trade-off approach.Clin Invest Med. 1987; 10: 14-20
- The time trade-off method: results from a general population study.Health Econ. 1996; 5: 141-154
- Visual analog scales: do they have a role in the measurement of preferences for health states?.Med Decis Making. 2001; 21: 329-334
- Utilities measured by rating scale, time trade-off, and standard gamble: review and reference for health care professionals.J Epidemiol. 2002; 12: 160-178
- Using discrete choice experiments to value health care programmes: current practice and future research reflections.Appl Health Econ Health Policy. 2003; 2: 55-64
- Discrete choice experiments in health care.BMJ. 2004; 328: 360-361
- Comparison of contingent valuation and conjoint analysis in ecosystem management.Ecol Econ. 2000; 32: 63-74
- Values and preferences for oral antithrombotic therapy in patients with atrial fibrillation: physician and patient perspectives.Health Expect. 2014; 18: 2318-2327
- Differences between perspectives of physicians and patients on anticoagulation in patients with atrial fibrillation: observational study.BMJ. 2001; 323: 1218-1222
- Modeling ranking, time trade-off, and visual analog scale values for EQ-5D health states: a review and comparison of methods.Med Care. 2009; 47: 634-641
- EQ-5D: a measure of health status from the EuroQol Group.Ann Med. 2001; 33: 337-343
- How to define and measure concordance between patients' preferences and medical treatments: a systematic review of approaches and recommendations for standardization.Patient Educ Couns. 2010; 78: 12-23
- Conceptual framework and systematic review of the effects of participants' and professionals' preferences in randomised controlled trials.Health Technol Assess. 2005; 9: 1-186
- A qualitative study of trends in patient preferences for the management of the partially dentate state.Gerodontology. 2009; 26: 137-142
- Patient experiences of depression and anxiety with chronic disease: a systematic review and qualitative meta-synthesis.Ont Health Technol Assess Ser. 2013; 13: 1-33
- Grading quality of evidence and strength of recommendations.BMJ. 2004; 328
- GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.BMJ. 2008; 336: 924-926
- Grading quality of evidence and strength of recommendations for diagnostic tests and strategies.BMJ. 2008; 336: 1106-1110
- GRADE guidelines: 10. Considering resource use and rating the quality of economic evidence.J Clin Epidemiol. 2013; 66: 140-150
- Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients.BMJ. 2015; 350: h870
- Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).PLoS Med. 2015; 12
- A decision aid for COPD patients considering inhaled steroid therapy: development and before and after pilot testing.BMC Med Inform Decis Mak. 2007; 7: 12
- Going from evidence to recommendations.BMJ. 2008; 336: 1049-1051
- GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.BMJ. 2016; 353: i2016
- GRADE Guidelines: 16. GRADE evidence to decision frameworks for tests in clinical practice and public health.J Clin Epidemiol. 2016; 76: 89-98
- Transparent development of the WHO rapid advice guidelines.PLoS Med. 2007; 4: e119
- Integrating values and consumer involvement in guidelines with the patient at the center: article 8 in Integrating and coordinating efforts in COPD guideline development. An official ATS/ERS workshop report.Proc Am Thorac Soc. 2012; 9: 262-268
- GRADE guidelines: 3. Rating the quality of evidence.J Clin Epidemiol. 2011; 64: 401-406
- Forty-two systematic reviews generated 23 items for assessing the risk of bias in values and preferences' studies.J Clin Epidemiol. 2017; 85: 21-31
- GRADE guidelines: 9. Rating up the quality of evidence.J Clin Epidemiol. 2011; 64: 1311-1316
- GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables.J Clin Epidemiol. 2011; 64: 383-394
- A new “mechanistic-practical” framework for designing and interpreting randomized trials.J Clin Epidemiol. 2009; 62: 479-484
- Study design III: cross-sectional studies.Evid Based Dent. 2006; 7: 24-25
- Response rates and responsiveness for surveys, standards, and the Journal.Am J Pharm Educ. 2008; 72: 43
- GRADE guidelines: 2. Framing the question and deciding on important outcomes.J Clin Epidemiol. 2011; 64: 395-400
- GRADE guidelines: 4. Rating the quality of evidence–study limitations (risk of bias).J Clin Epidemiol. 2011; 64: 407-415
- Patient preferences for the treatment of type 2 diabetes: a scoping review.PharmacoEconomics. 2013; 31: 877-892
- GRADE guidelines: 8. Rating the quality of evidence–indirectness.J Clin Epidemiol. 2011; 64: 1303-1310
- Influence of decision aids on patient preferences for anticoagulant therapy: a randomized trial.CMAJ. 2007; 176: 1583-1587
- Medical treatment of benign prostatic hyperplasia: physician and patient preferences and satisfaction.Int J Clin Pract. 2010; 64: 1425-1435
- A review of studies mapping (or cross walking) non-preference based measures of health to generic preference-based measures.Eur J Health Econ. 2010; 11: 215-225
Article info
Publication history
Published online: February 13, 2018
Accepted:
January 11,
2018
Footnotes
Ethics approval and consent to participate: Not required. This study does not involve de novo patient data collection. No patient informed consent and Institutional Review Board approval have been sought.
Availability of data and materials: The data sets supporting the conclusions of this article are included within the article and its additional file.
Conflicts of interest: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/conflicts-of-interest/).
Funding: This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. It was funded through internal research funds at McMaster University available to H.J.S. G.H. is supported by a CIHR New Investigator Salary Award and a The Arthritis Society Young Investigator Salary Award, neither of which is directly related to this research project.
Identification
Copyright
© 2018 Elsevier Inc. All rights reserved.
