Systematic reviews of clinical practice guidelines: a methodological guide

Published:December 05, 2018DOI:



      Systematic reviews (SRs) of clinical practice guidelines (CPGs) are unique knowledge syntheses that require tailored approaches to, and greater subjectivity in, design and execution compared with other SRs in clinical epidemiology. We provide review authors structured direction on how to design and conduct methodologically rigorous SRs of CPGs.

      Study Design and Setting

      A guidance paper outlining suggested methodology for conducting all stages of an SR of CPGs. We present concrete examples of approaches used by published reviews, including a case exemplar demonstrating how this methodology was applied to our own SR of CPGs.


      Review context and the unique characteristics of CPGs as research syntheses or clinical guidance statements must be considered in all aspects of review design and conduct. Researchers should develop a “PICAR” statement to help form and focus on the research question(s) and eligibility criteria, assess CPG quality using a validated appraisal tool, and extract, analyze, and summarize data in a way that is cogent and transparent.


      SRs of CPGs can be used to systematically identify, assess, and summarize the current state of guidance on a clinical topic. These types of reviews often require methodological tailoring to ensure that their objectives and timelines are effectively and efficiently addressed; however, they should all meet the criteria for an SR, follow a rigorous methodological approach, and adhere to transparent reporting practices.


      To read this article in full you will need to make a payment


      1. Clinical practice guidelines we can trust.
        in: Graham R. Mancher M. Miller Wolman D. Greenfield S. Steinberg E. Institute of Medicine (US) Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. National Academies Press (US), Washington, DC2011
        • Crosby E.
        Review article: the role of practice guidelines and evidence-based medicine in perioperative patient safety.
        Can J Anesth. 2013; 60: 143-151
        • Rosenfeld R.M.
        • Nnacheta L.C.
        • Corrigan M.D.
        Clinical consensus statement development manual.
        Otolaryngol Head Neck Surg. 2015; 153: S1-S14
      2. Guidelines for clinical practice: from development to use.
        in: Field M.J. Lohr K.N. Institute of Medicine (US) Committee on Clinical Practice Guidelines. National Academies Press (US), Washington, DC1992
        • The Cochrane Collaboration
        About Cochrane reviews: what is a systematic review?.
        (Available at)
        • Aromataris E.
        • Fernandez R.
        • Godfrey C.M.
        • Holly C.
        • Khalil H.
        • Tungpunkom P.
        Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach.
        Int J Evid Based Healthc. 2015; 13: 132-140
      3. Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. The Cochrane Collaboration, 2011 (Available from:)
        • Gough D.
        • Oliver S.
        • Thomas J.
        An introduction to systematic reviews.
        SAGE Publications, Thousand Oaks, California2012
        • Uman L.S.
        Information management for the busy practitioner: systematic reviews and meta-analyses.
        J Am Acad Child Adolesc Psychiatry. 2011; 20: 57-59
        • Khan K.S.
        • Kunz R.
        • Kleijnen J.
        • Antes G.
        Five steps to conducting a systematic review.
        J R Soc Med. 2003; 96: 118-121
        • Godfrey C.
        • Harrison M.
        Systematic review resource package.
        Joanna Briggs Inst Reviewers’ Man. 2015; 4: 30
        • Centre for Reviews and Dissemination
        Systematic reviews: CRD's guidance for undertaking reviews in health care.
        University of York, York, UK2009
        • Dixon-Woods M.
        • Agarwal S.
        • Jones D.
        • Young B.
        • Sutton A.
        Synthesising qualitative and quantitative evidence: a review of possible methods.
        J Health Serv Res Policy. 2005; 10: 45-53
        • Grant M.J.
        • Booth A.
        A typology of reviews: an analysis of 14 review types and associated methodologies.
        Health Info Libr J. 2009; 26: 91-108
        • Korhonen A.
        • Hakulinen-Viitanen T.
        • Jylha V.
        • Holopainen A.
        Meta-synthesis and evidence-based health care--a method for systematic review.
        Scand J Caring Sci. 2013; 27: 1027-1034
        • Yao L.
        • Chen Y.
        • Wang X.
        • Shi X.
        • Wang Y.
        • Guo T.
        • et al.
        Appraising the quality of clinical practice guidelines in traditional Chinese medicine using AGREE II instrument: a systematic review.
        Int J Clin Pract. 2017; 71: 1-8
        • O'Donoghue K.J.M.
        • Reed R.D.
        • Knight S.R.
        • O'Callaghan J.M.
        • Ayaz-Shah A.A.
        • Hassan S.
        • et al.
        Systematic review of clinical practice guidelines in kidney transplantation.
        BJS Open. 2017; 1: 97-105
        • Knight S.
        • Takagi M.
        • Fisher E.
        • Anderson V.
        • Lannin N.A.
        • Tavender E.
        • et al.
        A systematic critical appraisal of evidence-based clinical practice guidelines for the rehabilitation of children with moderate or severe acquired brain injury.
        Arch Phys Med Rehabil. 2018; (pii: S0003-9993(18)30392-7)
        • Jiang M.
        • Guan W.
        • Fang Z.
        • Xie Y.
        • Xie J.
        • Chen H.
        • et al.
        A critical review of the quality of cough clinical practice guidelines.
        Chest. 2016; 150: 777-788
        • MacQueen G.
        • Santaguida P.
        • Keshavarz H.
        • Jaworska N.
        • Levine M.
        • Beyene J.
        • et al.
        Systematic review of clinical practice guidelines for failed antidepressant treatment response in major depressive disorder, dysthymia, and subthreshold depression in adults.
        Can J Psychiatry. 2017; 62: 11-23
        • Acuna S.A.
        • Lam W.
        • Daly C.
        • Kim S.J.
        • Baxter N.N.
        Cancer evaluation in the assessment of solid organ transplant candidates: a systematic review of clinical practice guidelines.
        Transplant Rev (Orlando). 2018; 32: 29-35
        • Erickson J.
        • Sadeghirad B.
        • Lytvyn L.
        • Slavin J.
        • Johnston B.C.
        The scientific basis of guideline recommendations on sugar intake: a systematic review.
        Ann Intern Med. 2017; 166: 257-267
        • Kim J.K.
        • Chua M.E.
        • Ming J.M.
        • Santos J.D.
        • Zani-Ruttenstock E.
        • Marson A.
        • et al.
        A critical review of recent clinical practice guidelines on management of cryptorchidism.
        J Pediatr Surg. 2018; 53: 2041-2047
        • Simons P.
        • Cosgrove L.
        • Shaughnessy A.F.
        • Bursztajn H.
        Antipsychotic augmentation for major depressive disorder: a review of clinical practice guidelines.
        Int J Law Psychiatry. 2017; 55: 64-71
        • Chiappini E.
        • Bortone B.
        • Galli L.
        • de Martino M.
        Guidelines for the symptomatic management of fever in children: systematic review of the literature and quality appraisal with AGREE II.
        BMJ Open. 2017; 7: e015404
        • Molenaar N.M.
        • Kamperman A.M.
        • Boyce P.
        • Bergink V.
        Guidelines on treatment of perinatal depression with antidepressants: an international review.
        Aust N Z J Psychiatry. 2018; 52: 320-327
        • Unit Ontario Drug Policy Research Network Systematic Review
        Low molecular weight heparin: a review of clinical guidelines across indications.
        2015: 5 (Toronto)
        • Bush S.H.
        • Marchington K.L.
        • Agar M.
        • Davis D.H.J.
        • Sikora L.
        • Tsang T.W.Y.
        Quality of clinical practice guidelines in delirium: a systematic appraisal.
        BMJ Open. 2017; 7: e013809
        • Jakes A.D.
        • Marec-berard P.
        • Phillips R.S.
        • Cantab M.A.
        • Oxon B.M.
        • Stark D.P.
        • et al.
        Critical review of clinical practice guidelines for fertility preservation in teenagers and young adults with cancer.
        . 2014; 3: 144-152
        • Tonelli M.
        • Conner Gorber S.
        • Moore A.
        • Thombs B.D.
        Recommendations on routine screening pelvic examination.
        Can Fam Physician. 2016; 62: 2014-2017
        • Mattivi J.T.
        • Buchberger B.
        [Rapid reviews: Sisyphos' salvation? - an inventory].
        Gesundheitswesen. 2017; 79: 438-442
        • Tricco A.C.
        • Antony J.
        • Zarin W.
        • Strifler L.
        • Ghassemi M.
        • Ivory J.
        • et al.
        A scoping review of rapid review methods.
        BMC Med. 2015; 13: 224
        • Khangura S.
        • Konnyu K.
        • Cushman R.
        • Grimshaw J.
        • Moher D.
        Evidence summaries: the evolution of a rapid review approach.
        Syst Rev. 2012; 1: 10
        • Kelly S.E.
        • Moher D.
        • Clifford T.J.
        Defining rapid reviews: a modified Delphi consensus approach.
        Int J Technol Assess Health Care. 2016; 32: 265-275
        • Jolliffe L.
        • Lannin N.
        • Hoffmann T.
        Evidence-based rehabilitation interventions for adults with acquired brain injury: a systematic review of clinical practice guidelines to identify current best practice recommendations.
        PROSPERO International prospective register of systematic reviews, University of York, York, UK2016
        • Methley A.M.
        • Campbell S.
        • Chew-Graham C.
        • McNally R.
        • Cheraghi-Sohi S.
        PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews.
        BMC Health Serv Res. 2014; 14: 579
      4. DistillerSR. Evidence Partners, Ottawa, ON2018
      5. Covidence. Cochrane, London, UK2018
        • Woolf S.H.
        • Grol R.
        • Hutchinson A.
        • Eccles M.
        • Grimshaw J.
        Potential benefits, limitations, and harms of clinical guidelines.
        BMJ. 1999; 318: 527
        • Siering U.
        • Eikermann M.
        • Hausner E.
        • Hoffmann-Eßer W.
        • Neugebauer E.A.
        Appraisal tools for clinical practice guidelines: a systematic review.
        PLoS One. 2013; 8: e82915
        • Kashyap N.
        • Dixon J.
        • Michel G.
        • Brandt C.
        • Shiffman R.
        GLIA Guideline Implementability Appraisal V. 2.0.
        Yale University: Yale Center for Medical Informatics, New Haven, CT2011
        • Brouwers M.C.
        • Kho M.E.
        • Browman G.P.
        • Burgers J.S.
        • Cluzeau F.
        • Feder G.
        • et al.
        AGREE II: advancing guideline development, reporting and evaluation in health care.
        CMAJ. 2010; 182: 1308-1311
      6. German instrument for methodological guideline appraisal in deutsches instrument zur methodischen leitlinien-bewertung (DELBI).
        in: Kopp I. Thole H. Selbmann H.K. Ollenschlaeger G. Association of the scientific medical societies in germany and agency for quality in medicine. 2008 (Available from:)
        • Hoffmann-Eßer W.
        • Siering U.
        • Neugebauer E.A.M.
        • Brockhaus A.C.
        • Lampert U.
        • Eikermann M.
        Guideline appraisal with AGREE II: systematic review of the current evidence on how users handle the 2 overall assessments.
        PLoS One. 2017; 12: 1-15
        • The A.C.
        • Collaboration A.
        Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project.
        Qual Saf Health Care. 2003; 12: 18-23
        • Brouwers M.C.
        • Kerkvliet K.
        • Spithof K.
        The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines.
        BMJ. 2016; 352: i1152
        • Hoffmann-Eßer W.
        • Siering U.
        • Neugebauer E.A.M.
        • Lampert U.
        • Eikermann M.
        Systematic review of current guideline appraisals performed with the appraisal of guidelines for research & evaluation II instrument—a third of AGREE II users apply a cut-off for guideline quality.
        J Clin Epidemiol. 2018; 95: 120-127
        • Brouwers M.
        • Florez I.D.
        • Spithoff K.
        Evaluating the clinical credibility and implementability of clinical practice guideline recommendations using the AGREE-REX tool.
        in: Global Evidence Summit. Cochrane Colloquium Abstracts, Cape Town, South Africa2017
        • Brouwers M.C.
        • Team AGREE-REX Research
        AGREE-REX: Recommendation EXcellence Application, AGREE-REX.
        2016 (Available from:) (McMaster University)
        • Johnston A.
        • Hsieh S.C.
        • Carrier M.
        • Kelly S.E.
        • Bai Z.
        • Skidmore B.
        • et al.
        A systematic review of clinical practice guidelines on the use of low molecular weight heparin and fondaparinux for the treatment and prevention of venous thromboembolism: implications for research and policy decision-making.
        PLoS One. 2018; 13: e0207410
        • Watine J.
        • Friedberg B.
        • Nagy E.
        • Onody R.
        • Oosterhuis W.
        • Bunting P.S.
        • et al.
        Conflict between guideline methodologic quality and recommendation validity: a potential problem for practitioners.
        Clin Chem. 2006; 52: 65-72
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.E.
        • Kunz R.
        • Falck Y.
        • Alonso-coello P.
        • et al.
        Rating quality of evidence and strength of recommendations: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations rating quality of evidence of recommendations grade: of evidence an emerging and consensus of on rati.
        Br Med J. 2008; 336: 924-926
        • Finfgeld-Connett D.
        A guide to qualitative meta-synthesis.
        Taylor & Francis, New York, NY2018
        • Mohammed M.A.
        • Moles R.J.
        • Chen T.F.
        Meta-synthesis of qualitative research: the challenges and opportunities.
        Int J Clin Pharm. 2016; 38: 695-704
        • Graham I.D.
        • Harrison M.B.
        Evaluation and adaptation of clinical practice guidelines.
        Evid Based Nurs. 2005; 8: 68-72
        • Sprague A.E.
        • Oppenheimer L.
        • McCabe L.
        • Brownlee J.
        • Graham I.D.
        • Davies B.
        The Ottawa hospital's clinical practice guidelinefor the second stage of labour.
        J Obstet Gynaecol Can. 2006; 28: 769-779
        • Bates S.M.
        • Greer I.A.
        • Middeldorp S.
        • Veenstra D.L.
        • Prabulos A.M.
        • Vandvik P.O.
        • et al.
        VTE, thrombophilia, antithrombotic therapy, and pregnancy: antithrombotic therapy and prevention of thrombosis, 9th ed.: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines.
        Chest. 2012; 141: e691S-e736S
        • Cardiovascular Disease Educational Research Trust, European Venous Forum, North American Thrombosis Forum, International Union of Angiology, Union Internationale du Phlebologie
        Prevention and treatment of venous thromboembolism: international consensus statement (guidelines according to scientific evidence).
        Clin Appl Thromb Hemost. 2013; 19: 116-225
        • James A.
        • Obstetrics Committee On Practice Bulletins
        Practice bulletin no. 123: thromboembolism in pregnancy.
        Obstet Gynecol. 2011; 118: 718-729
        • Guyatt A.D.
        • Oxman
        • Kunz R.
        • Vist G.E.
        • Falck-Ytter Y.
        • Schünemann H.J.
        • et al.
        What is “quality of evidence” and why is it important to clinicians?.
        BMJ. 2008; 336: 995-998
        • Schünemann H.J.
        • Fretheim A.
        • Oxman A.D.
        Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations.
        Health Res Policy Syst. 2006; 4: 1-7
        • Hoffmann T.
        • Bennett S.
        • Del Mar C.
        Chapter 13- clinical practice guidelines.
        in: Evidence-Based Practice Across the Health Professions. Elsevier, Chatswood, NSW2017
        • Team Members Abikus Project
        ABIKUS Acquired Brain Injury Knowledge Uptake Strategy ABIKUS evidence-based recommendations for rehabilitation of moderate to severe acquired brain injury (2007).
        2007 (Available from:)
        • Hsieh H.F.
        • Shannon S.E.
        Three approaches to qualitative content analysis.
        Qual Health Res. 2005; 15: 1277-1288
        • Jolliffe L.
        • Lannin N.A.
        • Cadilhac D.A.
        • Hoffmann T.
        Systematic review of clinical practice guidelines to identify recommendations for rehabilitation after stroke and other acquired brain injuries.
        BMJ Open. 2018; 8: e018791
        • Bosch M.
        • Tavender E.
        • Bragge P.
        • Gruen R.
        • Green S.
        How to define 'best practice' for use in Knowledge Translation research: a practical, stepped and interactive process.
        J Eval Clin Pract. 2013; 19: 763-768
      7. NVivo qualitative data analysis Software. QSR International Pty Ltd, Doncaster, VIC2012
        • Hess D.R.
        How to write an effective discussion.
        Respir Care. 2004; 49: 1238-1241
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • PRISMA Group
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        Ann Intern Med. 2009; 151 (w64): 264-269
        • Murad M.H.
        • Wang Z.
        Guidelines for reporting meta-epidemiological methodology research.
        Evid Based Med. 2017; 22: 139-142
      8. G. Guest, K.M. MacQueen, and E.E. Namey. Writing up thematic analyses, in Applied thematic analysis 2012, SAGE Publications, Inc.: Thousand Oaks, CA. p. 241-278.