Methods for living guidelines: early guidance based on practical experience. Paper 2: consumer engagement in living guidelines

Published:December 30, 2022DOI:


      • Consumer engagement in living guidelines builds upon best practice in engagement.
      • Larger groups of consumers can mitigate workload challenges.
      • Guideline developers must pay continual attention to consumer turnover and support.
      • Living consumer engagement means the approach can grow and improve over time.
      • Improvements should be informed by living evaluation.



      To describe and reflect on the consumer engagement approaches used in five living guidelines from the perspectives of consumers (i.e., patients, carers, the public, and their representatives) and guideline developers.

      Study Design and Setting

      In a descriptive report, we used a template to capture engagement approaches and the experiences of consumers and guideline developers in living guidelines in Australia and the United Kingdom. Responses were summarized using descriptive synthesis.


      One guideline used a Consumer Panel, three included two to three consumers in the guideline development group, and one did both. Much of our experience was common to all guidelines (e.g., consumers felt welcomed but that their role initially lacked clarity). We identified six challenges and opportunities specific to living guidelines: managing the flow of work; managing engagement in online environments; managing membership of the panel; facilitating more flexibility, variety and depth in engagement; recruiting for specific skills–although these can be built over time; developing living processes to improve; and adapting consumer engagement together.


      Consumer engagement in living guidelines should follow established principles of consumer engagement in guidelines. Conceiving the engagement as living, underpinned by a living process evaluation, allows the approach to be developed with consumers over time.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Akl E.A.
        • Meerpohl J.J.
        • Elliott J.
        • Kahale L.A.
        • Schünemann H.J.
        Living systematic reviews: 4. Living guideline recommendations.
        J Clin Epidemiol. 2017; 91: 47-53
        • Australian Living Evidence Consortium
        The Living Guidelines Handbook: Guidance for the production and publication of living clinical practice guidelines (version 1.0). Australian Living Evidence Consortium, 2022
        • Rochwerg B.
        A living WHO guideline on drugs for covid-19.
        BMJ. 2020; 370: m3379
        • National COVID19 Clinical Evidence Taskforce
        Caring for people with COVID-19.
        • The National Institute for Health and Care Excellence (NICE)
        COVID-19 rapid guideline: managing COVID-19.
        NICE, United Kingdom2021
        • Vogel J.P.
        • Dowswell T.
        • Lewin S.
        • Bonet M.
        • Hampson L.
        • Kellie F.
        • et al.
        Developing and applying a 'living guidelines' approach to WHO recommendations on maternal and perinatal health.
        BMJ Glob Health. 2019; 4: e001683
        • English C.
        • Bayley M.
        • Hill K.
        • Langhorne P.
        • Molag M.
        • Ranta A.
        • et al.
        Bringing stroke clinical guidelines to life.
        Int J Stroke. 2019; 14: 337-339
        • Stroke Foundation
        Clinical Guidelines for Stroke Management. Stroke Foundation, Melbourne, Australia2021
        • White H.
        • Tendal B.
        • Elliott J.
        • Turner T.
        • Andrikopoulos S.
        • Zoungas S.
        Breathing life into Australian diabetes clinical guidelines.
        Med J Aust. 2020; 212: 250-251.e1
      1. Living Evidence for Diabetes Consortium. Australian evidence-based clinical guidelines for diabetes v1.3.
        (Available at)
        Date: 2022
        Date accessed: January 20, 2023
      2. ANZMUSC. An Australian living guideline for the pharmacological management of inflammatory arthritis v2.6.
        (Available at)
        Date: 2023
        Date accessed: January 20, 2023
        • Institute of Medicine
        Clinical practice guidelines we can trust.
        in: Graham R. Clinical Practice Guidelines We Can Trust. National Academies Press (US), Washington (DC)2011
        • NHMRC
        Guidelines for Guidelines Handbook.
        National Health and Medical Research Council, Canberra2020
        • Légaré F.
        • Boivin A.
        • van der Weijden T.
        • Pakenham C.
        • Burgers J.
        • Légaré J.
        • et al.
        Patient and public involvement in clinical practice guidelines:A knowledge synthesis of existing programs.
        Med Decis Making. 2011; 31: E45-E74
        • Armstrong M.J.
        • Rueda J.D.
        • Gronseth G.S.
        • Mullins C.D.
        Framework for enhancing clinical practice guidelines through continuous patient engagement.
        Health Expect. 2017; 20: 3-10
        • Concannon T.W.
        • Meissner P.
        • Grunbaum J.A.
        • McElwee N.
        • Guise J.M.
        • Santa J.
        • et al.
        A new taxonomy for stakeholder engagement in patient-centered outcomes research.
        J Gen Intern Med. 2012; 27: 985-991
      3. National Institute for Health and Care Excellence. Patient and public involvement policy.
        (Available at)
        • Tendal B.
        • Vogel J.P.
        • McDonald S.
        • Norris S.
        • Cumpston M.
        • White H.
        • et al.
        Weekly updates of national living evidence-based guidelines: methods for the Australian living guidelines for care of people with COVID-19.
        J Clin Epidemiol. 2021; 131: 11-21
        • Oliver S.
        Health promotion, inequalities and young people’s health: A systematic review of research.
        EPPI-Centre, University of London, London2008
        • Staniszewska S.
        • Brett J.
        • Simera I.
        • Seers K.
        • Mockford C.
        • Goodlad S.
        • et al.
        GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research.
        BMJ. 2017; 358: j3453
        • Pollock A.
        • Campbell P.
        • Struthers C.
        • Synnot A.
        • Nunn J.
        • Hill S.
        • et al.
        Development of the ACTIVE framework to describe stakeholder involvement in systematic reviews.
        J Health Serv Res Policy. 2019; 24: 245-255
        • Wiles L.
        • Hibbert P.
        • Smith K.
        • Arnolda G.
        • Ellis L.
        • Lake R.
        • et al.
        Living Stroke Guidelines Evaluation.
        The Stroke Foundation. Australian Institute of Health Innovation., Sydney, Australia2021
        • Oxman A.D.
        • Schünemann H.J.
        • Fretheim A.
        Improving the use of research evidence in guideline development: 12. Incorporating considerations of equity.
        Health Res Policy Syst. 2006; 4: 24
        • Turner T.
        • Elliott J.
        • Tendal B.
        • Vogel J.P.
        • Norris S.
        • Tate R.
        • et al.
        The Australian living guidelines for the clinical care of people with COVID-19: what worked, what didn't and why, a mixed methods process evaluation.
        PLoS One. 2022; 17: e0261479
        • National Institute for Health and Care Excellence
        COVID-19 rapid guideline: managing the long-term effects of COVID-19.
        NICE, United Kingdom2020
      4. International Association for Public Participation Australasia. Quality assurance standard for community and stakeholder engagement.
        (Available at)
        Date: 2015
        Date accessed: November 21, 2022
      5. National Institute for Health Research, et al., National standards for public involvement in research V1. 2018, NIHR: United Kingdom.

      6. Guidelines international network. GIN public toolkit: patient and public involvement in guidelines.
        (Available at:)
        • Synnot A.
        • Hill S.
        • Jauré A.
        • Merner B.
        • Hill K.
        • Bates P.
        • et al.
        Broadening the diversity of consumers engaged in guidelines: a scoping review.
        BMJ Open. 2022; 12: e058326
        • Sarah Scott J.C.
        • Schaefer C.
        • Graham K.
        • Fielding J.
        • Howitt L.
        • Rasburn M.
        • et al.
        How to recruit and support patients and the public, and overcome barriers to their involvement in guideline development.
        in: GIN Public Toolkit: Patient and public involvement in guidelines. Guidelines International Network, Scotland2021
        • NHMRC
        Guidelines for guidelines: consumer involvement.
        2020: 1-16 (Available at)
        • Daraz L.
        • Webb S.
        • Kunkle R.
        • Murad M.H.
        • Lang E.
        Training curriculum to help patient representatives participate meaningfully in the development of clinical practice guidelines.
        BMJ Evid Based Med. 2019; 24: 227-230
        • van der Ham A.J.
        • Erp N.
        • Broerse J.E.W.
        Monitoring and evaluation of patient involvement in clinical practice guideline development: lessons from the Multidisciplinary Guideline for Employment and Severe Mental Illness, The Netherlands.
        Health Expect. 2015; 19: 471-482
        • Duff L.A.
        • Kelson M.
        • Marriott S.
        • Mcintosh A.
        • Brown S.
        • Cape J.
        • et al.
        Clinical guidelines: involving patients and users of services.
        J Clin Effectiveness. 1996; 1: 104-112
        • Brouwers M.C.
        • Vukmirovic M.
        • Spithoff K.
        • Makarski J.
        Understanding optimal approaches to patient and caregiver engagement in the development of cancer practice guidelines: a mixed methods study.
        BMC Health Serv Res. 2017; 17: 186
        • van der Ham A.J.
        • Shields L.S.
        • van der Horst R.
        • Broerse J.E.
        • van Tulder M.W.
        Facilitators and barriers to service user involvement in mental health guidelines: lessons from The Netherlands.
        Adm Policy Ment Health. 2014; 41: 712-723
        • Boivin A.
        • Currie K.
        • Fervers B.
        • Gracia J.
        • James M.
        • Marshall C.
        • et al.
        Patient and public involvement in clinical guidelines: international experiences and future perspectives.
        Qual Saf Health Care. 2010; 19: e22
        • Kopke S.
        • Giordano A.
        • Veronese S.
        • Christin Rahn A.
        • Kleiter I.
        • Basedow-Rajwich B.
        • et al.
        Patient and caregiver involvement in the formulation of guideline questions: findings from the European Academy of Neurology guideline on palliative care of people with severe multiple sclerosis.
        Eur J Neurol. 2019; 26: 41-50
        • Lindsay M.P.
        • Gierman N.
        • Harris J.E.
        • Arthur G.
        • Teed M.E.
        • Mountain A.
        • et al.
        People with lived experience at the centre of Canadian stroke best practice recommendations: a model for guideline developers.
        J Patient Experience. 2020; 7: 951-956
        • van de Bovenkamp H.M.
        • Zuiderent-Jerak T.
        An empirical study of patient participation in guideline development: exploring the potential for articulating patient knowledge in evidence-based epistemic settings.
        Health Expect. 2013; 18: 942-955
        • Armstrong M.J.
        • Bloom J.A.
        Patient involvement in guidelines is poor five years after institute of medicine standards: review of guideline methodologies.
        Res Involvement Engagement. 2017; 3: 19
        • Bennett W.
        • Robbins C.W.
        • Bayliss E.A.
        • Wilson R.
        • Tabano H.
        • Mularski R.A.
        • et al.
        Engaging stakeholders to inform clinical practice guidelines that address multiple chronic conditions.
        JGIM: J Gen Intern Med. 2017; 32: 883-890
        • Tong A.
        • Lopez-Vargas P.
        • Howell M.
        • Phoon R.
        • Johnson D.
        • Campbell D.
        • et al.
        Consumer involvement in topic and outcome selection in the development of clinical practice guidelines.
        Health Expect. 2012; 15: 410-423
        • Fraenkel L.
        • Miller A.S.
        • Clayton K.
        • Crow-Hercher R.
        • Hazel S.
        • Johnson B.
        • et al.
        When patients write the guidelines: patient panel recommendations for the treatment of rheumatoid arthritis.
        Arthritis Care Res (Hoboken). 2016; 68: 26-35
        • Armstrong C.
        • Grant S.
        • Kinnett K.
        • Denger B.
        • Martin A.
        • Coulter I.
        • et al.
        Participant experiences with a new online modified-delphi approach for engaging patients and caregivers in developing clinical guidelines.
        Eur J Pers Cent Healthc. 2019; 7: 476-489
        • Lindsay M.P.
        • Gierman N.
        • Harris J.E.
        • Arthur G.
        • Teed M.E.
        • Mountain A.
        • et al.
        People with lived experience at the centre of Canadian stroke best practice recommendations: a model for guideline developers.
        J Patient Experience. 2020; : 951-956
        • Elliott J.H.
        • Synnot A.
        • Turner T.
        • Simmonds M.
        • Akl E.A.
        • McDonald S.
        • et al.
        Living systematic review: 1. Introduction—the why, what, when, and how.
        J Clin Epidemiol. 2017; 91: 23-30
        • INVOLVE
        Guidance on co-producing a research project.
        National Institute for Health Research (NIHR), Southampton2018
        • Hamilton C.B.
        • Hoens A.M.
        • McKinnon A.M.
        • McQuitty S.
        • English K.
        • Hawke L.D.
        • et al.
        Shortening and validation of the Patient Engagement in Research Scale (PEIRS) for measuring meaningful patient and family caregiver engagement.
        Health Expect. 2021; 24: 863-879
        • Moore A.
        • Wu Y.
        • Kwakkenbos L.
        • Silveira K.
        • Straus S.
        • Brouwers M.
        • et al.
        The patient engagement evaluation tool was valid for clinical practice guideline development.
        J Clin Epidemiol. 2022; 143: 61-72
        • Karpusheff J.
        • Haynes C.
        • Glen F.
        • Leng G.
        Involving people with learning disabilities in guideline development.
        Patient. 2020; 13: 251-254
        • Raynor D.K.
        • Dickinson D.
        Key principles to guide development of consumer medicine information--content analysis of information design texts.
        Ann Pharmacother. 2009; 43: 700-706
        • Gupta S.
        • Tang R.
        • Petricca K.
        • Florez I.D.
        • Kastner M.
        The Guideline Language and Format Instrument (GLAFI): development process and international needs assessment survey.
        Implementation Sci. 2022; 17: 47