GRADE guidelines: 10. Considering resource use and rating the quality of economic evidence



      In this article, we describe how to include considerations about resource utilization when making recommendations according to the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach.

      Study Design and Settings

      We focus on challenges with rating the confidence in effect estimates (quality of evidence) and incorporating resource use into evidence profiles and Summary of Findings (SoF) tables.


      GRADE recommends that important differences in resource use between alternative management strategies should be included along with other important outcomes in the evidence profile and SoF table. Key steps in considering resources in making recommendations with GRADE are the identification of items of resource use that may differ between alternative management strategies and that are potentially important to decision makers, finding evidence for the differences in resource use, making judgments regarding confidence in effect estimates using the same criteria used for health outcomes, and valuing the resource use in terms of costs for the specific setting for which recommendations are being made.


      With our framework, decision makers will have access to concise summaries of recommendations, including ratings of the quality of economic evidence, and better understand the implications for clinical decision making.


      To read this article in full you will need to make a payment


        • Drummond M.
        • Sculpher M.
        • Torrance G.
        • O'Brien B.
        • Stoddart G.
        Methods for the economic evaluation of health care programmes.
        3rd ed. Oxford University Press, Oxford, UK2005
        • Sculpher M.
        • Pang F.
        • Manca A.
        • et al.
        Generalisability in economic evaluation studies in healthcare: a review and case studies.
        Health Technol Assess. 2004; 8: 1-192
        • Buxton M.J.
        • Drummond M.F.
        • van Hout B.A.
        • Prince R.L.
        • Sheldon T.A.
        • Szucs T.
        • et al.
        Modelling in economic evaluation: an unavoidable fact of life.
        Health Econ. 1997; 6: 217-227
        • Hjelmgren J.
        • Berggren F.
        • Andersson F.
        Health economic guidelines—similarities, differences and some implications.
        Value Health. 2001; 4: 225-250
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Jaeschke R.
        • Helfand M.
        • Liberati A.
        • et al.
        Grading recommendations: incorporating considerations of resources use.
        BMJ. 2008; 336: 1170-1173
        • Impact-RSV Study Group
        Palivizumab, a humanized respiratory syncitial virus monoclonal antibody, reduces hospitalization from respiratory syncytial virus infection in high risk infants.
        Pediatrics. 1998; 102: 531-537
        • Drummond M.F.
        • Jefferson T.O.
        Guidelines for authors and peer reviewers of economic submissions to the BMJ.
        BMJ. 1996; 313: 275-283
        • Shemilt I.
        • Thomas J.
        • Morciano M.
        A web-based tool for adjusting costs to a specific target currency and price year.
        Evid Policy. 2010; 6: 51-59
        • Mattick R.P.
        • Kimber J.
        • Breen C.
        • Davoli M.
        Buprenorphine maintenance versus placebo or methadone maintenance for opioid dependence.
        Cochrane Database Syst Rev. 2008; : CD002207
        • Harris A.
        • Gospodarevskaya E.
        • Ritter A.
        A randomised trial of the cost effectiveness of buprenorphine as an alternative to methadone maintenance treatment for heroin dependence in a primary care setting.
        Pharmacoeconomics. 2005; 23: 77-91
        • Doran C.M.
        • Shanahan M.
        • Mattick R.P.
        • Ali R.
        • White J.
        • Bell J.
        Buprenorphine versus methadone maintenance: a cost-effectiveness analysis.
        Drug Alcohol Depend. 2003; 71: 295-302
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • et al.
        GRADE guidelines: 7. rating the quality of evidence—inconsistency.
        J Clin Epidemiol. 2011; 64: 1294-1302
        • Shemilt I.
        • Mugford M.
        • Byford S.
        • Drummond M.
        • Eisenstein E.
        • Knapp M.
        • et al.
        Chapter 15: incorporating economics evidence.
        in: Higgins J.P.T. Green S. Cochrane handbook for systematic reviews of interventions. John Wiley & Sons, Chichester, UK2008: 449-479
        • Anderson R.
        Systematic reviews of economic evaluations: utility or futility?.
        Health Econ. 2010; 19: 350-364
        • Drummond M.
        Chapter 15: evidence-based decisions and economics: an agenda for research.
        in: Shemilt I. Mugford M. Vale L. Marsh K. Donaldson C. Evidence-based decisions and economics: health care, social welfare, education and criminal justice. Wiley-Blackwell, Oxford, UK2010: 179-185
        • Luce B.R.
        • Manning W.G.
        • Siegel J.E.
        • Lipscomb J.
        Estimating costs in cost-effectiveness analysis.
        in: Gold M.R. Siegel J.E. Russell L.B. Weinstein M.C. Cost-effectiveness in health and medicine. Oxford University Press, New York, NY1996: 176-185
        • Evers S.
        • Goossens M.
        • de Vet H.
        • van Tulder M.
        • Ament A.
        Criteria list for assessment of methodological quality of economic evaluations: consensus on Health Economic Criteria.
        Int J Technol Assess Health Care. 2005; 21: 240-245
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • et al.
        • GRADE Working Group
        GRADE: what is “quality of evidence” and why is it important to clinicians?.
        BMJ. 2008; 336: 995-998
        • Balshem H.
        • Helfand M.
        • Schunemann H.J.
        • Oxman A.D.
        • Kunz R.
        • Brozek J.
        • et al.
        GRADE guidelines: 3 rating the quality of evidence—introduction.
        J Clin Epidemiol. 2011; 64: 401-406
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines 8: rating the quality of evidence—indirectness.
        J Clin Epidemiol. 2011; 64: 1303-1310
        • Guyatt G.H.
        • Oxman A.D.
        • Sultan S.
        • Glasziou P.
        • Akl E.A.
        • Alonso-Coello P.
        • et al.
        GRADE guidelines: 9. rating up the quality of evidence.
        J Clin Epidemiol. 2011; 64: 1311-1316
        • Coyle D.
        • Lee K.M.
        The problem of protocol driven costs in pharmacoeconomic analysis.
        Pharmacoeconomics. 1998; 14: 357-363
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.
        • Kunz R.
        • Brozek J.
        • Alonso-Coello P.
        • et al.
        GRADE guidelines: 4. rating the quality of evidence—risk of bias.
        J Clin Epidemiol. 2011; 64: 407-415
        • Coast J.
        • Richards S.H.
        • Peters T.J.
        • Gunnell D.J.
        • Darlow M.A.
        • Pounsford J.
        Hospital at home or acute hospital care? A cost minimization analysis.
        BMJ. 1998; 316: 1802-1806
        • Petrou S.
        • Murray L.
        • Cooper P.
        • Davidson L.L.
        The accuracy of self-reported healthcare resource utilization in health economic studies.
        Int J Technol Assess Health Care. 2002; 18: 705-710
        • Hughes S.L.
        • Cummings J.
        • Weaver F.
        • Manheim L.
        • Braun B.
        • Conrad K.
        A randomized trial of the cost effectiveness of VA hospital-based home care for the terminally ill.
        Health Serv Res. 1992; 26: 801-817
        • Simon J.
        • Gray A.
        • Duley L.
        • Magpie Trial Collaborative Group
        Cost-effectiveness of prophylactic 9996 women with pre-eclampsia from 33 countries: economic evaluation of the Magpie Trial.
        BJOG. 2006; 113: 144-151
        • Baltussen R.
        • Ament A.
        • Leidl R.
        Making cost assessment based on RCTs more useful to decision makers.
        Health Policy. 1996; 37: 163-183
      1. Shemilt I. Mugford M. Vale L. Marsh K. Donaldson C. Evidence-based decisions and economics. Wiley-Blackwell, Oxford, UK2010
        • Briggs A.
        Economic evaluation and clinical trials: size matters.
        BMJ. 2000; 321: 132-133
        • Doshi J.A.
        • Henry A.G.
        • Polsky D.
        Analyses of cost data in economic evaluations conducted alongside randomized controlled data.
        Value Health. 2006; 9: 334-340
        • Briggs A.
        • Gray A.M.
        Handling uncertainty when performing economic evaluation of healthcare interventions.
        Health Technol Assess. 1999; 3: 1-134
        • Guyatt G.H.
        • Oxman A.D.
        • Montori V.
        • et al.
        GRADE guidelines 5: rating the quality of evidence—publication bias.
        J Clin Epidemiol. 2011; 64: 1277-1282
        • Cooper N.
        • Coyle D.
        • Abrams K.
        • Mugford M.
        • Sutton A.
        Use of evidence in decision models: an appraisal of health technology assessments in the UK since 1997.
        J Health Serv Res Policy. 2005; 10: 245-250
        • Marson A.G.
        • Al-Kharusi A.M.
        • Alwaidh M.
        • Appleton R.
        • Baker G.A.
        • Chadwick D.W.
        • et al.
        The SANAD study of effectiveness of carbamazepine, gabapentin, lamotrigine, oxcarbazepine, or topiramate for treatment of partial epilepsy: an unblinded randomised controlled trial.
        Lancet. 2007; 369: 1000-1015
        • Glanville J.
        • Paisley S.
        Chapter 7: searching for evidence for cost-effectiveness decisions.
        in: Shemilt I. Mugford M. Vale L. Marsh K. Donaldson C. Evidence-based decisions and economics. Wiley-Blackwell, Oxford, UK2010