Series: Emerging Knowledge Synthesis Methods for Integrating Qualitative and Quantitative Evidence Original Article| Volume 73, P29-35, May 01, 2016

Knowledge synthesis methods for integrating qualitative and quantitative data: a scoping review reveals poor operationalization of the methodological steps

  • Andrea C. Tricco
    Affiliations
    Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, Toronto, Ontario M5B 1W8, Canada

    Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, Health Sciences Building, 155 College Street, 6th Floor, Toronto, Ontario M5T 3M7, Canada
    Search for articles by this author
  • Jesmin Antony
    Affiliations
    Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, Toronto, Ontario M5B 1W8, Canada
    Search for articles by this author
  • Charlene Soobiah
    Affiliations
    Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, Toronto, Ontario M5B 1W8, Canada

    Institute of Health Policy, Management and Evaluation, University of Toronto, Canada Health Sciences Building, 155 College Street, Suite 425, Toronto, Ontario M5T 3M6, Canada
    Search for articles by this author
  • Monika Kastner
    Affiliations
    Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, Toronto, Ontario M5B 1W8, Canada

    Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, Health Sciences Building, 155 College Street, 6th Floor, Toronto, Ontario M5T 3M7, Canada
    Search for articles by this author
  • Heather MacDonald
    Affiliations
    Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, Toronto, Ontario M5B 1W8, Canada
    Search for articles by this author
  • Elise Cogo
    Affiliations
    Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, Toronto, Ontario M5B 1W8, Canada
    Search for articles by this author
  • Erin Lillie
    Affiliations
    Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, Toronto, Ontario M5B 1W8, Canada
    Search for articles by this author
  • Judy Tran
    Affiliations
    Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, Toronto, Ontario M5B 1W8, Canada
    Search for articles by this author
  • Sharon E. Straus
    Correspondence
    Corresponding author. Knowledge Translation Program, Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1W8, Canada. Tel.: 416-864-3068; fax: 416-864-5805.
    Affiliations
    Li Ka Shing Knowledge Institute of St. Michael's Hospital, 209 Victoria Street, Toronto, Ontario M5B 1W8, Canada

    Department of Geriatric Medicine, Faculty of Medicine, University of Toronto, 27 King's College Circle, Toronto, Ontario M5S 1A1, Canada
    Search for articles by this author
Published:February 15, 2016DOI:https://doi.org/10.1016/j.jclinepi.2015.12.011

      Abstract

      Objectives

      To describe and compare, through a scoping review, emerging knowledge synthesis methods for integrating qualitative and quantitative evidence in health care, in terms of expertise required, similarities, differences, strengths, limitations, and steps involved in using the methods.

      Study Design and Setting

      Electronic databases (e.g., MEDLINE) were searched, and two reviewers independently selected studies and abstracted data for qualitative analysis.

      Results

      In total, 121 articles reporting seven knowledge synthesis methods (critical interpretive synthesis, integrative review, meta-narrative review, meta-summary, mixed studies review, narrative synthesis, and realist review) were included after screening of 17,962 citations and 1,010 full-text articles. Common similarities among methods related to the entire synthesis process, while common differences related to the research question and eligibility criteria. The most common strength was a comprehensive synthesis providing rich contextual data, whereas the most common weakness was a highly subjective method that was not reproducible. For critical interpretive synthesis, meta-narrative review, meta-summary, and narrative synthesis, guidance was not provided for some steps of the review process.

      Conclusion

      Some of the knowledge synthesis methods provided guidance on all steps, whereas other methods were missing guidance on the synthesis process. Further work is needed to clarify these emerging knowledge synthesis methods.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Wilson P.M.
        • Watt I.S.
        • Hardman G.F.
        Survey of medical directors' views and use of the Cochrane Library.
        Br J Clin Governance. 2001; 6: 34-39
        • Ram F.S.
        • Wellington S.R.
        General practitioners use of the Cochrane library in London.
        Prim Care Respir J. 2002; 11: 123-124
        • Lavis J.N.
        How can we support the use of systematic reviews in policymaking?.
        PLoS Med. 2009; 6: e1000141
        • Glasziou P.
        • Meats E.
        • Heneghan C.
        • Shepperd S.
        What is missing from descriptions of treatment in trials and reviews?.
        BMJ. 2008; 336: 1472-1474
        • Anderson L.M.
        • Oliver S.R.
        • Michie S.
        • Rehfuess E.
        • Noyes J.
        • Shemilt I.
        Investigating complexity in systematic reviews of interventions by using a spectrum of methods.
        J Clin Epidemiol. 2013; 66: 1223-1229
        • Noyes J.
        • Gough D.
        • Lewin S.
        • Mayhew A.
        • Michie S.
        • Pantoja T.
        • et al.
        A research and development agenda for systematic reviews that ask complex questions about complex interventions.
        J Clin Epidemiol. 2013; 66: 1262-1270
        • Burford B.
        • Lewin S.
        • Welch V.
        • Rehfuess E.
        • Waters E.
        Assessing the applicability of findings in systematic reviews of complex interventions can enhance the utility of reviews for decision making.
        J Clin Epidemiol. 2013; 66: 1251-1261
        • Pigott T.
        • Shepperd S.
        Identifying, documenting, and examining heterogeneity in systematic reviews of complex interventions.
        J Clin Epidemiol. 2013; 66: 1244-1250
        • Petticrew M.
        • Rehfuess E.
        • Noyes J.
        • Higgins J.P.T.
        • Mayhew A.
        • Pantoja T.
        • et al.
        Synthesizing evidence on complex interventions: how meta-analytical, qualitative, and mixed-method approaches can contribute.
        J Clin Epidemiol. 2013; 66: 1230-1243
        • Squires J.E.
        • Valentine J.C.
        • Grimshaw J.M.
        Systematic reviews of complex interventions: framing the review question.
        J Clin Epidemiol. 2013; 66: 1215-1222
        • Petticrew M.
        • Anderson L.
        • Elder R.
        • Grimshaw J.
        • Hopkins D.
        • Hahn R.
        • et al.
        Complex interventions and their implications for systematic reviews: a pragmatic approach.
        J Clin Epidemiol. 2013; 66: 1209-1214
        • Anderson L.M.
        • Petticrew M.
        • Chandler J.
        • Grimshaw J.
        • Tugwell P.
        • O'Neill J.
        • et al.
        Introducing a series of methodological articles on considering complexity in systematic reviews of interventions.
        J Clin Epidemiol. 2013; 66: 1205-1208
        • Tharyan P.
        Introducing conceptual and analytical clarity on dimensions of complexity in systematic reviews of complex interventions.
        J Clin Epidemiol. 2013; 66: 1202-1204
        • Wong G.
        Is complexity just too complex?.
        J Clin Epidemiol. 2013; 66: 1199-1201
        • Straus S.E.
        • Kastner M.
        • Soobiah C.
        • Antony J.
        • Tricco A.C.
        Engaging researchers on developing, using and improving knowledge synthesis methods: introduction to a series of articles describing the results of a scoping review on emerging knowledge synthesis methods.
        J Clin Epidemiol. 2016; 73: 15-18
        • Tricco A.C.
        • Soobiah C.
        • Antony J.
        • Cogo E.
        • McDonald H.
        • Lillie E.
        • et al.
        A scoping review identifies multiple emerging knowledge synthesis methods, but few studies operationalise the method.
        J Clin Epidemiol. 2016; 73: 19-28
        • Tricco A.C.
        • Antony J.
        • Soobiah C.
        • Kastner M.
        • MacDonald H.
        • Cogo E.
        • et al.
        Knowledge synthesis methods for integrating qualitative and quantitative data: a scoping review reveals poor operationalisation of the methodological steps.
        J Clin Epidemiol. 2016; 73: 36-42
        • Kastner M.
        • Antony J.
        • Soobiah C.
        • Straus S.E.
        • Tricco AC
        Conceptual recommendations for selecting the most appropriate knowledge synthesis method to answer research questions related to complex evidence.
        J Clin Epidemiol. 2016; 73: 43-49
        • Perrier L.
        • Lightfoot D.
        • Kealey M.R.
        • Straus S.E.
        • Tricco A.C.
        Knowledge synthesis methods research: a bibliometric analysis found relevant MeSH were not applied or not available for emerging knowledge synthesis methods.
        J Clin Epidemiol. 2016; 73: 50-57
        • Kastner M.
        • Tricco A.C.
        • Soobiah C.
        • Lillie E.
        • Perrier L.
        • Horsley T.
        • et al.
        What is the most appropriate knowledge synthesis method to conduct a review? Protocol for a scoping review.
        BMC Med Res Methodol. 2012; 12: 114
        • Arksey H.
        • O'Malley L.
        Scoping studies: towards a methodological framework.
        Int J Soc Res Methodol. 2005; 8: 19-32
        • WHO
        Preamble to the Constitution of the World Health Organization as adopted by the International Health Conference, New York, 19-22 June, 1946.
        (signed on 22 July 1946 by the representatives of 61 States (Official Records of the World Health Organization, no. 2, p. 100) and entered into force on 7 April 1948) World Health Organization, New York, USA1946
        • International Q
        NVivo10 for Windows.
        QSR International, New York, USA2014
      1. Higgins J. Green S. Cochrane handbook for systematic reviews of interventions version 5.1.0. [updated March 2011]. The Cochrane Collaboration, Oxford, 2011 (Available at) (Accessed February 2016)
        • Hsieh H.F.
        • Shannon S.E.
        Three approaches to qualitative content analysis.
        Qual Health Res. 2005; 15: 1277-1288
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • PRISMA Group
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        BMJ. 2009; 339: b2535
        • Wong G.
        • Greenhalgh T.
        • Westhorp G.
        • Buckingham J.
        • Pawson R.
        RAMESES publication standards: realist syntheses.
        BMC Med. 2013; 11: 21
        • Tricco A.C.
        • Antony J.
        • Soobiah C.
        • Kastner M.
        • Cogo E.
        • MacDonald H.
        • et al.
        Knowledge synthesis methods research: a bibliometric analysis found relevant MeSH were not applied or not available for emerging knowledge synthesis methods.
        J Clin Epidemiol. 2016; 73: 36-42