Advertisement

Cochrane Qualitative and Implementation Methods Group guidance series—paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings

Published:December 13, 2017DOI:https://doi.org/10.1016/j.jclinepi.2017.06.020

      Abstract

      The Cochrane Qualitative and Implementation Methods Group develops and publishes guidance on the synthesis of qualitative and mixed-method implementation evidence. Choice of appropriate methodologies, methods, and tools is essential when developing a rigorous protocol and conducting the synthesis. Cochrane authors who conduct qualitative evidence syntheses have thus far used a small number of relatively simple methods to address similarly written questions. Cochrane has invested in methodological work to develop new tools and to encourage the production of exemplar reviews to show the value of more innovative methods that address a wider range of questions. In this paper, in the series, we report updated guidance on the selection of tools to assess methodological limitations in qualitative studies and methods to extract and synthesize qualitative evidence. We recommend application of Grades of Recommendation, Assessment, Development, and Evaluation–Confidence in the Evidence from Qualitative Reviews to assess confidence in qualitative synthesized findings. This guidance aims to support review authors to undertake a qualitative evidence synthesis that is intended to be integrated subsequently with the findings of one or more Cochrane reviews of the effects of similar interventions. The review of intervention effects may be undertaken concurrently with or separate to the qualitative evidence synthesis. We encourage further development through reflection and formal testing.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Harris J.L.
        • Booth A.
        • Cargo M.
        • Hannes K.
        • Harden A.
        • Flemming K.
        • et al.
        Cochrane Qualitative and Implementation Methods Group guidance series—paper 2: methods for question formulation, searching and protocol development for qualitative evidence synthesis.
        J Clin Epidemiol. 2018; 97: 39-48
        • Harden A.
        • Thomas J.
        • Cargo M.
        • Harris J.
        • Pantoja T.
        • Flemming K.
        • Booth A.
        • et al.
        Cochrane Qualitative and Implementation Methods Group guidance series—paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews.
        J Clin Epidemiol. 2018; 97: 70-78
        • Flemming K.
        • Booth A.
        • Hannes K.
        • Cargo M.
        • Noyes J.
        Cochrane Qualitative and Implementation Methods Group guidance series—paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses.
        J Clin Epidemiol. 2018; 97: 79-85
        • Cargo M.
        • Harris J.
        • Pantoja T.
        • Booth A.
        • Harden A.
        • Hannes K.
        • Thomas J.
        • et al.
        Cochrane Qualitative and Implementation Methods Group guidance series—paper 4: methods for assessing evidence on intervention implementation.
        J Clin Epidemiol. 2018; 97: 59-69
        • Sparks A.C.
        Myth 94: qualitative health researchers agree about validity.
        Qual Health Res. 2001; 11: 538-552
        • Garside R.
        Should we appraise the quality of qualitative research reports for systematic reviews, and if so, how?.
        Innovat Eur J Soc Sci Res. 2014; 27: 67-79
        • Sandelowski M.
        A matter of taste: evaluating the quality of qualitative research.
        Nurs Inq. 2015; 22: 86-94
        • Carroll C.
        • Booth A.
        Quality assessment of qualitative evidence for systematic review and synthesis: is it meaningful, and if so, how should it be performed?.
        Res Syn Methods. 2015; 6: 149-154
        • Santiago-Delefosse M.
        • Gavin A.
        • Bruchez C.
        • Roux P.
        • Stephen S.L.
        Quality of qualitative research in the health sciences: analysis of the common criteria present in 58 assessment guidelines by expert users.
        Soc Sci Med. 2016; 148: 142-151
        • CASP – Critical Appraisal Skills Programme
        (2013 version) Making sense of evidence: 10 questions to help you make sense of qualitative research. Public Health Resource Unit, England.
        (Available at)
        • Hannes K.
        • Lockwood C.
        Pragmatism as the philosophical underpinning of the Joanna Briggs meta-aggregative approach to qualitative evidence synthesis.
        J Adv Nurs. 2011; 67: 1632-1642
        • Saini M.
        • Shlonsky A.
        Systematic synthesis of qualitative research.
        Oxford University Press, Oxford2012: 133-135
        • Barone T.
        • Eisner E.
        Arts based research.
        Sage, Thousand Oaks2012
        • Heyvaert M.
        • Hannes K.
        • Maes B.
        • Onghena P.
        Critical appraisal of mixed methods studies.
        J Mix Methods Res. 2013; 7: 302-327
        • Pluye P.
        • Gagnon M.P.
        • Griffiths F.
        • Johnson-Lafleur J.
        A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews.
        Int J Nurs Stud. 2009; 46: 529-546
        • Thomas J.
        • Harden A.
        • Oakley A.
        • Oliver S.
        • Sutcliffe K.
        • Rees R.
        • et al.
        Integrating Qualitative Research with trials in systematic reviews: an example review from public health shows how integration is possible and some potential benefits.
        BMJ. 2004; 328: 1010-1012
        • Lewin S.
        • Glenton C.
        • Munthe-Kaas H.
        • et al.
        Using qualitative evidence in decision making for health and social interventions: an approach to assess confidence in findings from qualitative evidence syntheses (GRADE-CERQual).
        PLoS Med. 2015; 12: e1001895
        • Glenton C.
        • Colvin C.J.
        • Carlsen B.
        • Swartz A.
        • Lewin S.
        • Noyes J.
        • et al.
        Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis.
        Cochrane Database Syst Rev. 2013; : CD010414
        • Lewin S.
        • Munabi-Babigumira S.
        • Glenton C.
        • Daniels K.
        • Bosch-Capblanch X.
        • van Wyk B.E.
        • et al.
        Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases.
        Cochrane Database Syst Rev. 2010; : CD004015
        • WHO
        Optimizing health worker roles for maternal and neonatal health.
        (Available at)
        www.optimizemnh.org
        Date accessed: March 16, 2016
        • Benoot C.
        • Hannes K.
        • Bilsen J.
        The use of purposeful sampling in a QES: a worked example on sexual adjustment to a cancer trajectory.
        BMC Med Res Methodol. 2016; 16: 1-12
        • Flemming K.
        • Booth A.
        • Garside R.
        • Tunçalp O.
        • Noyes J.
        Qualitative evidence synthesis for complex interventions and guideline development: clarification of the purpose, designs and relevant methods.
        BMJ Global Health. 2018; (In press)
        • Thomas J.
        • Harden A.
        Methods for the thematic synthesis of qualitative research in systematic reviews.
        BMC Med Res Methodol. 2008; 8: 1
        • Carroll C.
        • Booth A.
        • Cooper K.
        A worked example of “best fit” framework synthesis: a systematic review of views concerning the taking of some potential chemopreventive agents.
        BMC Med Res Methodol. 2010; 11: 29
        • Carroll C.
        • Booth A.
        • Leaviss J.
        • Rick J.
        “Best fit” framework synthesis: refining the method.
        BMC Med Res Methodol. 2013; 13: 1
        • Campbell R.
        • et al.
        Evaluating meta-ethnography: systematic analysis and synthesis of qualitative research.
        Health Technol Assess. 2011; 15: 1-164
        • Whitaker R.
        • Hendry M.
        • Booth A.
        • Carter B.
        • Charles J.
        • Craine N.
        • et al.
        Intervention Now To Eliminate Repeat Unintended Pregnancy in Teenagers (INTERUPT): a systematic review of intervention effectiveness and cost-effectiveness, qualitative and realist synthesis of implementation factors and user engagement.
        BMJ Open. 2014; 4: e004733
        • Barnett-Page E.
        • Thomas J.
        Methods for the synthesis of qualitative research: a critical review.
        BMC Med Res Methodol. 2009; 9: 1
        • Booth A.
        • Noyes J.
        • Flemming K.
        • Gerhardus A.
        • Wahlster P.
        • van der Wilt G.J.
        • et al.
        Guidance on choosing QES methods for use in health technology assessments of complex interventions.
        ([Online]. Available at) (Accessed December 1, 2017)
        • Popay J.
        • Roberts H.
        • Sowden A.
        • Pettticrew M.
        • Arai L.
        • Rodgers M.
        • et al.
        Guidance on the conduct of narrative synthesis in systematic reviews.
        (Available at) (Accessed December 1, 2017)
        • Schutz A.
        Collected papers. Vol. 1. Martinus Nijhoff, The Hague1962
        • NICE
        Methods for development of NICE public health guidance.
        NICE, UK2006 (Available at)
        http://www.nice.org.uk/phmethods
        Date accessed: March 18, 2016
        • Noyes J.
        • Hendry M.
        • Booth A.
        • Chandler J.
        • Lewin S.
        • Glenton C.
        • et al.
        Current use was established and Cochrane guidance on selection of social theories for systematic reviews of complex interventions was developed.
        J Clin Epidemiol. 2016; 75: 78-92
        • SURE Framework. The SURE (Supporting the Use of Research Evidence) Collaboration
        SURE guides for preparing and using evidence-based policy briefs: 5. Identifying and addressing barriers to implementing the policy options. Version 2.1.
        (Updated November 2011) The SURE Collaboration; 2011, 2011 (Available at)
        http://global.evipnet.org/SURE-Guides/
        Date accessed: March 13, 2016
        • Tugwell P.
        • Petticrew M.
        • Kristjansson E.
        • Welch V.
        • Ueffing E.
        • Waters E.
        • et al.
        Assessing equity in systematic reviews: realising the recommendations of the Commission on Social Determinants of Health.
        BMJ. 2010; 341: c4739
        • Anderson L.M.
        • Petticrew M.
        • Rehfuess E.
        • Armstrong R.
        • Ueffing E.
        • Baker P.
        • et al.
        Using logic models to capture complexity in systematic reviews.
        Res Synth Methods. 2011; 2: 33-42
        • Kneale D.
        • Thomas J.
        • Harris K.
        Developing and optimising the use of logic models in systematic reviews: exploring practice and good practice in the use of programme theory in reviews.
        PLoS One. 2015; 10: e0142187
        • Turley R.
        • Saith R.
        • Bhan N.
        • Doyle J.
        • Jones K.
        • Waters E.
        Slum upgrading review: methodological challenges that arise in systematic reviews of complex interventions.
        J Public Health (Oxf). 2013; 35: 171-175
        • Pawson R.
        • Tilley N.
        Realistic evaluation.
        Sage, London1997
        • Garside R.
        • Britten N.
        • Stein K.
        The experience of heavy menstrual bleeding: a systematic review and meta-ethnography of qualitative studies.
        J Adv Nurs. 2008; 63: 550-562
        • Malpass A.
        • Shaw A.
        • Sharp D.
        • Walter F.
        • Feder G.
        • Ridd M.
        • et al.
        “Medication career” or “moral career”? The two sides of managing antidepressants: a meta-ethnography of patients' experience of antidepressants.
        Soc Sci Med. 2009; 68: 154-168