Original Article| Volume 79, P150-158.e1, November 2016

The skills and experience of GRADE methodologists can be assessed with a simple tool

  • Susan L. Norris
    Corresponding author. Tel.: +41 22 7913239.
    World Health Organization, Av. Appia 20, CH-1211, Geneva 27, Switzerland
    Search for articles by this author
  • Joerg J. Meerpohl
    Cochrane Germany, Medical Center–University of Freiburg, Faculty of Medicine, University of Freiburg, Berliner Allee 29, 79110 Freiburg, Germany

    Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité–U1153, Inserm/Université Paris Descartes, Cochrane France, Hôpital Hôtel-Dieu, 1 Place du Parvis Notre Dame, 75181 Cedex 04 Paris, France
    Search for articles by this author
  • Elie A. Akl
    Department of Internal Medicine, American University of Beirut Medical Center, P.O. Box 11-0236, Riad-El-Solh Beirut, 1107 2020 Beirut, Lebanon
    Search for articles by this author
  • Holger J. Schünemann
    Department of Clinical Epidemiology & Biostatistics, McMaster GRADE Center, McMaster University, HSC Room 2C16, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada

    Department of Medicine, McMaster University, HSC Room 2C16, 1280 Main Street West, Hamilton, Ontario L8S 4K1, Canada
    Search for articles by this author
  • Gerald Gartlehner
    Department for Evidence-Based Medicine and Clinical Epidemiology, Danube University Krems, Dr. Karl Dorrek Straße 30, 3500 Krems, Austria

    RTI-International, Research Triangle Park, 3040 East Cornwallis Road, Durham, NC 27709, USA
    Search for articles by this author
  • Yaolong Chen
    Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, No. 199, Donggang West Road, Chengguan District, Lanzhou, Gansu 730000, Peoples' Republic of China
    Search for articles by this author
  • Craig Whittington
    Centre for Outcomes Research and Effectiveness, Research Department of Clinical, Educational and Health Psychology, University College London, 1-19 Torrington Place, London WC1E7HB, UK
    Search for articles by this author
  • on behalf of theGrading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group



      To suggest approaches for guideline developers on how to assess a methodologist's expertise with Grading of Recommendations Assessment, Development and Evaluation (GRADE) methods and tasks and to provide a set of minimum skills and experience required to perform specific tasks related to guideline development using GRADE.

      Study Design and Setting

      We used an iterative and consensus-based process involving individuals with in-depth experience with GRADE. We considered four main tasks: (1) development of key questions; (2) assessment of the certainty of effect estimates; (3) development of recommendations; and (4) teaching GRADE.


      There are three basic approaches to determine a methodologist's skill set. First, self-report of knowledge, skills, and experience with a standardized “GRADE curriculum vitae (CV)” focused on each of the GRADE-related tasks; second, demonstration of skills using worked examples; third, a formal evaluation using a written or oral test. We suggest that the GRADE CV is likely to be useful and feasible to implement. We also suggest minimum training including attendance at one or more full-day workshops and familiarity with the main GRADE publications and the GRADE handbook.


      The selection of a GRADE methodologist must be a thoughtful, reasoned decision, informed by the criteria suggested in this article and tailored to the specific project. Our suggested approaches need further pilot testing and validation.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.
        • Kunz R.
        • Falck-Ytter Y.
        • Alonso-Coello P.
        • et al.
        • for the GRADE Working Group
        Rating quality of evidence and strength of recommendations GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
        BMJ. 2008; 336: 924-926
        • Mustafa R.A.
        • Santesso N.
        • Brozek J.
        • Akl E.A.
        • Walter S.D.
        • Norman G.
        • et al.
        The GRADE approach is reproducible in assessing the quality of evidence of quantitative evidence syntheses.
        J Clin Epidemiol. 2013; 66: 736-742
        • Shen X.
        • Chen Y.
        • Yang K.
        The agreement for grading the quality of evidence using GRADE system.
        (20th Cochrane Colloquium) The Cochrane Collaboration, Auckland, New Zealand2012
        • Kumar A.
        • Miladinovic B.
        • Guyatt G.H.
        • Schünemann H.J.
        • Djulbegovic B.
        GRADE guidelines system is reproducible when instructions are clearly operationalized even among the guidelines panel members with limited experience with GRADE.
        J Clin Epidemiol. 2016; 75: 115-118
      1. Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, Chichester, England2011 (Available at) (Accessed July 31, 2016)