Guideline developers in the United States were inconsistent in applying criteria for appropriate Grading of Recommendations, Assessment, Development and Evaluation use



      We assessed whether guidelines published by organizations based in the United States comply with published criteria for the use of the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach.

      Study Design and Setting

      We performed a cross-sectional study of all clinical practice guidelines that indicated the use of the GRADE approach, were published between 2011 and 2018, and listed in the National Guidelines Clearinghouse.


      We included 67 guideline documents from 44 of 135 (32.6%) US-based organizations that indicated the use of the GRADE approach. The majority (60/67, 89.6%) of guidelines defined the certainty of evidence consistent with GRADE, but only approximately 1 in 10 (7/67, 10.4%) explicitly reported consideration of all eight criteria to assess the certainty in the evidence for rating down and up. A majority of guidelines (36/67, 53.7%) provided a summary of the evidence, described explicit consideration of all four central domains (36/67, 53.7%), and rated the strength of recommendation consistent with GRADE (36/67, 53.7%).


      Approximately one in three US-based organizations developing evidence-based guidelines report the use of GRADE, but adherence to published criteria is inconsistent. As uptake of the GRADE approach increases in the United States, continued efforts to train guideline methodologists and panel members are important.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


      1. GRADE Working Group. 2019
        Date accessed: April 27, 2020
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.E.
        • Kunz R.
        • Falck-Ytter Y.
        • Alonso-Coello P.
        • et al.
        GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
        BMJ. 2008; 336: 924-926
        • Dahm P.
        • Chapple C.R.
        • Konety B.R.
        • Joyce A.D.
        • Parsons K.
        • Wolf Jr., J.S.
        • et al.
        The future of clinical practice guidelines in urology.
        Eur Urol. 2011; 60: 72-74
        • Morgan R.L.
        • Sultan S.
        • Murad M.H.
        • Mustafa R.A.
        • Dahm P.
        • Helfand M.
        • et al.
        Advancing guideline development in the United States: a call to action by the US GRADE Network.
        Am J Med Qual. 2018; 33: 117-118
        • Institute of Medicine (U.S.)
        Committee on Standards for Developing Trustworthy Clinical Practice Guidelines., Graham R. Clinical practice guidelines we can trust.
        National Academies Press, Washington, DC2011
        • Shekelle P.G.
        Clinical practice guidelines: what's next?.
        JAMA. 2018; 320: 757-758
        • Group GW
        Criteria for applying or using GRADE.
      2. National Guideline Clearinghouse: Agency for Healthcare Research and Quality (AHRQ). 2008
        • Alonso-Coello P.
        • Oxman A.D.
        • Moberg J.
        • Brignardello-Petersen R.
        • Akl E.A.
        • Davoli M.
        • et al.
        GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 2: clinical practice guidelines.
        BMJ. 2016; 353: i2089
        • Alonso-Coello P.
        • Schunemann H.J.
        • Moberg J.
        • Brignardello-Petersen R.
        • Akl E.A.
        • Davoli M.
        • et al.
        GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction.
        BMJ. 2016; 353: i2016
        • Gupta M.
        • McCauley J.
        • Farkas A.
        • Gudeloglu A.
        • Neuberger M.M.
        • Ho Y.Y.
        • et al.
        Clinical practice guidelines on prostate cancer: a critical appraisal.
        J Urol. 2015; 193: 1153-1158
        • Chiappini E.
        • Bortone B.
        • Galli L.
        • de Martino M.
        Guidelines for the symptomatic management of fever in children: systematic review of the literature and quality appraisal with AGREE II.
        BMJ Open. 2017; 7: e015404
        • Nuckols T.K.
        • Anderson L.
        • Popescu I.
        • Diamant A.L.
        • Doyle B.
        • Di Capua P.
        • et al.
        Opioid prescribing: a systematic review and critical appraisal of guidelines for chronic pain.
        Ann Intern Med. 2014; 160: 38-47
        • Norris S.L.
        • Ford N.
        Improving the quality of WHO guidelines over the last decade: progress and challenges.
        Lancet Glob Health. 2017; 5: e855-e856
        • Kuijpers T.
        • Langendam M.
        • de Beer H.
        GRADE not uniformly applied in Dutch Guidelines.
        Guidelines International Annual Meeting, Philadelphia, PA2016
        • Schunemann H.J.
        • Woodhead M.
        • Anzueto A.
        • Buist S.
        • MacNee W.
        • Rabe K.F.
        • et al.
        A vision statement on guideline development for respiratory disease: the example of COPD.
        Lancet. 2009; 373: 774-779
        • Schunemann H.J.
        • Wiercioch W.
        • Brozek J.
        • Etxeandia-Ikobaltzeta I.
        • Mustafa R.A.
        • Manja V.
        • et al.
        GRADE Evidence to Decision Frameworks for adoption, adaptation and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT.
        J Clin Epidemiol. 2016; 81: 101-110
        • Sultan S.
        • Morgan R.L.
        • Murad M.H.
        • Falck-Ytter Y.
        • Dahm P.
        • Schunemann H.J.
        • et al.
        A theoretical framework and competency-based approach to training in guideline development.
        J Gen Intern Med. 2019; 35: 561-567
        • Munn Z.
        • Qaseem A.
        • American College of P.
        • the Joanna Briggs I.
        Disappearance of the National Guideline Clearinghouse: a huge loss for evidence-based health care.
        Ann Intern Med. 2018; 169: 648-649
        • Gerberich A.
        • Spencer S.
        • Ipema H.
        National Guideline Clearinghouse is no more: keep calm and search on.
        Ann Pharmacother. 2018; 53: 434-436
        • Kirkner R.M.
        Guideline Clearinghouse is dead. Long live the guideline clearinghouse?.
        Manag Care. 2018; 27: 9-10