Advertisement
Knowledge Translation Series - Guest Editor, Sharon Straus| Volume 64, ISSUE 1, P11-20, January 2011

Download started.

Ok

The art and science of knowledge synthesis

  • Andrea C. Tricco
    Affiliations
    Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada
    Search for articles by this author
  • Jennifer Tetzlaff
    Affiliations
    Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada

    Department of Epidemiology & Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
    Search for articles by this author
  • David Moher
    Correspondence
    Corresponding author. +Tel.: 613-738-3591; fax: 613-739-6266.
    Affiliations
    Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada

    Department of Epidemiology & Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
    Search for articles by this author

      Abstract

      Objectives

      To review methods for completing knowledge synthesis.

      Study Design and Setting

      We discuss how to complete a broad range of knowledge syntheses. Our article is intended as an introductory guide.

      Results

      Many groups worldwide conduct knowledge syntheses, and some methods are applicable to most reviews. However, variations of these methods are apparent for different types of reviews, such as realist reviews and mixed-model reviews. Review validity is dependent on the validity of the included primary studies and the review process itself. Steps should be taken to avoid bias in the conduct of knowledge synthesis. Transparency in reporting will help readers assess review validity and applicability, increasing its utility.

      Conclusion

      Given the magnitude of the literature, the increasing demands on knowledge syntheses teams, and the diversity of approaches, continuing efforts will be important to increase the efficiency, validity, and applicability of systematic reviews. Future research should focus on increasing the uptake of knowledge synthesis, how best to update reviews, the comparability between different types of reviews (eg, rapid vs. comprehensive reviews), and how to prioritize knowledge synthesis topics.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Chalmers I.
        • Hedges L.V.
        • Cooper H.
        A brief history of research synthesis.
        Eval Health Prof. 2002; 25: 12-37
        • Naylor C.D.
        Clinical decisions: from art to science and back again.
        Lancet. 2001; 358: 523-524
        • Grimshaw J.M.
        • Santesso N.
        • Cumpston M.
        • Mayhew A.
        • McGowan J.
        Knowledge for knowledge translation: the role of the Cochrane Collaboration.
        J Contin Educ Health Prof. 2006; 26: 55-62
        • Graham I.D.
        • Logan J.
        • Harrison M.B.
        • Straus S.E.
        • Tetroe J.
        • Caswell W.
        • et al.
        Lost in knowledge translation: time for a map?.
        J Contin Educ Health Prof. 2006; 26: 13-24
        • Canadian Institutes of Health Research
        Randomized Controlled Trials Registration/Application Checklist.
        (Available at) (Accessed July 30, 2008)
        • Graham I.D.
        • Tetroe J.
        Some theoretical underpinnings of knowledge translation.
        Acad Emerg Med. 2007; 14: 936-941
      1. The Cochrane Collaboration.
        (Available at) (Accessed July 30, 2008)
      2. Campbell Collaboration.
        (Available at) (Accessed July 29, 2008)
        • Atkins D.
        • Fink K.
        • Slutsky J.
        Better information for better health care: the Evidence-based Practice Center program and the Agency for Healthcare Research and Quality.
        Ann Intern Med. 2005; 142: 1035-1041
      3. Joanna Briggs Institute.
        (Available at) (Accessed July 29, 2008)
        • Dixon-Woods M.
        • Jackson C.
        • Windridge K.C.
        • Kenyon S.
        Receiving a summary of the results of a trial: qualitative study of participants' views.
        BMJ. 2006; 332: 206-210
      4. Cochrane handbook for systematic reviews of interventions.
        (Version 5.0.2. [Updated September 2009] Available at) (Accessed February 20, 2008)
        • Shekelle P.G.
        • Morton S.C.
        • Jungvig L.K.
        • Udani J.
        • Spar M.
        • Tu W.
        • et al.
        Effect of supplemental vitamin E for the prevention and treatment of cardiovascular disease.
        J Gen Intern Med. 2004; 19: 380-389
        • Lumley T.
        Network meta-analysis for indirect treatment comparisons.
        Stat Med. 2002; 21: 2313-2324
        • Elliott W.J.
        • Meyer P.M.
        Incident diabetes in clinical trials of antihypertensive drugs: a network meta-analysis.
        Lancet. 2007; 369: 201-207
        • Deeks J.
        • Glanville J.
        • Sheldon T.A.
        Undertaking systematic reviews of research on effectiveness: CRD guidelines for those carrying out or commissioning reviews.
        John Wiley & Sons, Ltd, Chichester, UK1996
        • Selman T.J.
        • Mann C.
        • Zamora J.
        • Appleyard T.L.
        • Khan K.
        Diagnostic accuracy of tests for lymph node status in primary cervical cancer: a systematic review and meta-analysis.
        CMAJ. 2008; 178: 855-862
      5. Little J, Higgins JPT (editors). The HuGENet™ HuGE Review Handbook, version 1.0. Available at http://www.hugenet.ca (accessed 23 February 2009).

        • Wang G.Y.
        • Lu C.Q.
        • Zhang R.M.
        • Hu X.H.
        • Luo Z.W.
        The E-cadherin gene polymorphism 160C->A and cancer risk: a HuGE review and meta-analysis of 26 case-control studies.
        Am J Epidemiol. 2008; 167: 7-14
        • Altman D.G.
        Systematic reviews of evaluations of prognostic variables.
        BMJ. 2001; 323: 224-228
        • Oremus M.
        • Raina P.S.
        • Santaguida P.
        • Balion C.M.
        • McQueen M.J.
        • McKelvie R.
        • et al.
        A systematic review of BNP as a predictor of prognosis in persons with coronary artery disease.
        Clin Biochem. 2008; 41: 260-265
        • Greenhalgh T.
        • Robert G.
        • Macfarlane F.
        • Bate P.
        • Kyriakidou O.
        • Peacock R.
        Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review.
        Soc Sci Med. 2005; 61: 417-430
        • Pawson R.
        • Greenhalgh T.
        • Harvey G.
        • Walshe K.
        Realist review—a new method of systematic review designed for complex policy interventions.
        J Health Serv Res Policy. 2005; 10: 21-34
        • Greenhalgh T.
        • Kristjansson E.
        • Robinson V.
        Realist review to understand the efficacy of school feeding programmes.
        BMJ. 2007; 335: 858-861
        • Atkins S.
        • Lewin S.
        • Smith H.
        • Engel M.
        • Fretheim A.
        • Volmink J.
        Conducting a meta-ethnography of qualitative literature: lessons learnt.
        BMC Med Res Methodol. 2008; 8: 21
        • Dixon-Woods M.
        • Fitzpatrick R.
        • Roberts K.
        Including qualitative research in systematic reviews: opportunities and problems.
        J Eval Clin Pract. 2001; 7: 125-133
        • Dixon-Woods M.
        • Sutton A.
        • Shaw R.
        • Miller T.
        • Smith J.
        • Young B.
        • et al.
        Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods.
        J Health Serv Res Policy. 2007; 12: 42-47
        • Dixon-Woods M.
        • Agarwal S.
        • Jones D.
        • Young B.
        • Sutton A.
        Synthesising qualitative and quantitative evidence: a review of possible methods.
        J Health Serv Res Policy. 2005; 10: 45-53
        • Harden A.
        • Thomas J.
        Methodological issues in combining diverse study types in systematic reviews.
        Int J Soc Res Methodol. 2005; 8: 257-271
        • Petticrew M.
        Systematic reviews from astronomy to zoology: myths and misconceptions.
        BMJ. 2001; 322: 98-101
        • Stone P.W.
        Popping the (PICO) question in research and evidence-based practice.
        Appl Nurs Res. 2002; 15: 197-198
        • Dixon-Woods M.
        • Bonas S.
        • Booth A.
        • Jones D.R.
        • Miller T.
        • Sutton A.J.
        • et al.
        How can systematic reviews incorporate qualitative research? A critical perspective.
        Qual Res. 2006; 6: 27-44
        • Gregoire G.
        • Derderian F.
        • Le L.J.
        Selecting the language of the publications included in a meta-analysis: is there a Tower of Babel bias?.
        J Clin Epidemiol. 1995; 48: 159-163
        • Egger M.
        • Zellweger-Zahner T.
        • Schneider M.
        • Junker C.
        • Lengeler C.
        • Antes G.
        Language bias in randomised controlled trials published in English and German.
        Lancet. 1997; 350: 326-329
        • Moher D.
        • Pham B.
        • Klassen T.P.
        • Schulz K.F.
        • Berlin J.A.
        • Jadad A.R.
        • et al.
        What contributions do languages other than English make on the results of meta-analyses?.
        J Clin Epidemiol. 2000; 53: 964-972
        • Juni P.
        • Holenstein F.
        • Sterne J.
        • Bartlett C.
        • Egger M.
        Direction and impact of language bias in meta-analyses of controlled trials: empirical study.
        Int J Epidemiol. 2002; 31: 115-123
        • Pan Z.
        • Trikalinos T.A.
        • Kavvoura F.K.
        • Lau J.
        • Ioannidis J.P.
        Local literature bias in genetic epidemiology: an empirical evaluation of the Chinese literature.
        PLoS Med. 2005; 2: e334
        • Hopewell S.
        • McDonald S.
        • Clarke M.
        • Egger M.
        Grey literature in meta-analyses of randomized trials of health care interventions.
        Cochrane Database Syst Rev. 2007; (MR000010)https://doi.org/10.1002/14651858.MR000010.pub3
        • Bhandari M.
        • Devereaux P.J.
        • Guyatt G.H.
        • Cook D.J.
        • Swiontkowski M.F.
        • Sprague S.
        • et al.
        An observational study of orthopaedic abstracts and subsequent full-text publications.
        J Bone Joint Surg Am. 2002; 84: 615-621
        • Rosmarakis E.S.
        • Soteriades E.S.
        • Vergidis P.I.
        • Kasiakou S.K.
        • Falagas M.E.
        From conference abstract to full paper: differences between data presented in conferences and journals.
        FASEB J. 2005; 19: 673-680
        • Toma M.
        • McAlister F.A.
        • Bialy L.
        • Adams D.
        • Vandermeer B.
        • Armstrong P.W.
        Transition from meeting abstract to full-length journal article for randomized controlled trials.
        JAMA. 2006; 295: 1281-1287
        • Scherer R.W.
        • Langenberg P.
        • von Elm E.
        Full publication of results initially presented in abstracts.
        Cochrane Database Syst Rev. 2007; (MR000005)https://doi.org/10.1002/14651858.MR000005.pub3
        • von Elm E.
        • Costanza M.C.
        • Walder B.
        • Tramer M.R.
        More insight into the fate of biomedical meeting abstracts: a systematic review.
        BMC Med Res Methodol. 2003; 3: 12
        • U.S. National Library of Medicine
        Medical literature analysis and retrieval system online–PubMed.
        (Available at) (Accessed March 19, 2008)
      6. Excerpta medica database.
        (Available at) (Accessed March 19, 2008)
        • Cochrane Central Register of Controlled Trials
        The Cochrane Library.
        (Available at) (Accessed March 18, 2008)
        • Tricco A.C.
        • Tetzlaff J.
        • Sampson M.
        • Fergusson D.
        • Cogo E.
        • Horsley T.
        • et al.
        Few systematic reviews exist documenting the extent of bias: a systematic review.
        J Clin Epidemiol. 2008; 61: 422-434
        • Dickersin K.
        How important is publication bias? A synthesis of available data.
        AIDS Educ Prev. 1997; 9: 15-21
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Gotzsche P.C.
        Multiple publication of reports of drug trials.
        Eur J Clin Pharmacol. 1989; 36: 429-432
        • Tramer M.R.
        • Reynolds D.J.
        • Moore R.A.
        • McQuay H.J.
        Impact of covert duplicate publication on meta-analysis: a case study.
        BMJ. 1997; 315: 635-640
        • Bailey B.J.
        Duplicate publication in the field of otolaryngology-head and neck surgery.
        Otolaryngol Head Neck Surg. 2002; 126: 211-216
        • Barden J.
        • Edwards J.E.
        • McQuay H.J.
        • Moore R.A.
        Oral valdecoxib and injected parecoxib for acute postoperative pain: a quantitative systematic review.
        BMC Anesthesiol. 2003; 3: 1
        • Moher D.
        • Soeken K.
        • Sampson M.
        • Ben-Porat L.
        • Berman B.
        Assessing the quality of reports of systematic reviews in pediatric complementary and alternative medicine.
        BMC Pediatr. 2002; 2: 3
        • Sanderson S.
        • Tatt I.D.
        • Higgins J.P.
        Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography.
        Int J Epidemiol. 2007; 36: 666-676
        • Chan A.W.
        • Hrobjartsson A.
        • Haahr M.T.
        • Gotzsche P.C.
        • Altman D.G.
        Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
        JAMA. 2004; 291: 2457-2465
        • Chan A.W.
        • Krleza-Jeric K.
        • Schmid I.
        • Altman D.G.
        Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research.
        CMAJ. 2004; 171: 735-740
        • Chan A.W.
        • Altman D.G.
        Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors.
        BMJ. 2005; 330: 753
        • Fleiss J.L.
        The statistical basis of meta-analysis.
        Stat Methods Med Res. 1993; 2: 121-145
        • Systematic Reviews in Health Care
        Meta-analysis in context.
        3rd ed. British Medical Journal Publishing Group, London, UK2008
        • Mays N.
        • Pope C.
        • Popay J.
        Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field.
        J Health Serv Res Policy. 2005; 10: 6-20
        • Sutton A.J.
        • Abrams K.R.
        Bayesian methods in meta-analysis and evidence synthesis.
        Stat Methods Med Res. 2001; 10: 277-303
        • Spiegelhalter D.J.
        • Myles J.P.
        • Jones D.R.
        • Abrams K.R.
        Bayesian methods in health technology assessment: a review.
        Health Technol Assess. 2000; 4: 1-130
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        J Clin Epidemiol. 2009; 62: 1006-1012
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.E.
        • Kunz R.
        • Falck-Ytter Y.
        • onso-Coello P.
        • et al.
        GRADE: an emerging consensus on rating quality of evidence and strength of recommendations.
        BMJ. 2008; 336: 924-926
        • Ranmal R.
        • Prictor M.
        • Scott J.T.
        Interventions for improving communication with children and adolescents about their cancer.
        Cochrane Database Syst Rev. 2008; (CD002969)
        • Kenyon S.L.
        • Taylor D.J.
        • Tarnow-Mordi W.
        Broad-spectrum antibiotics for preterm, prelabour rupture of fetal membranes: the ORACLE I randomised trial. ORACLE Collaborative Group.
        Lancet. 2001; 357: 979-988
        • Shea B.J.
        • Hamel C.
        • Wells G.A.
        • Bouter L.M.
        • Kristjansson E.
        • Grimshaw J.
        • et al.
        AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.
        J Clin Epidemiol. 2009; 62: 1013-1020
        • Clarke L.
        • Clarke M.
        • Clarke T.
        How useful are Cochrane reviews in identifying research needs?.
        J Health Serv Res Policy. 2007; 12: 101-103
        • Moher D.
        • Tetzlaff J.
        • Tricco A.C.
        • Sampson M.
        • Altman D.G.
        Epidemiology and reporting characteristics of systematic reviews.
        PLoS Med. 2007; 4: e78
        • Atallah A.N.
        • de Silva E.M.
        • Paiva E.V.
        Disseminating results of systematic reviews through a tv show in Brazil.
        6th edition. Syst Rev Evid Action Int Cochrane Colloq, Baltimore, MD1998 (6:77)
        • Oermann M.H.
        • Floyd J.A.
        • Galvin E.A.
        • Roop J.C.
        Brief reports for disseminating systematic reviews to nurses.
        Clin Nurse Spec. 2006; 20: 233-238
        • O'Connor A.M.
        Using decision aids to help patients navigate the "grey zone" of medical decision-making.
        CMAJ. 2007; 176: 1597-1598
        • Moher D.
        • Cook D.J.
        • Eastwood S.
        • Olkin I.
        • Rennie D.
        • Stroup D.F.
        Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement.
        Onkologie. 2000; 23: 597-602
        • Liberati A.
        • Altman D.G.
        • Tetzlaff J.
        • Mulrow C.
        • Gotzsche P.C.
        • Ioannidis J.P.
        • et al.
        The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration.
        BMJ. 2009; 339: b2700
        • Stroup D.F.
        • Berlin J.A.
        • Morton S.C.
        • Olkin I.
        • Williamson G.D.
        • Rennie D.
        • et al.
        Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) Group.
        JAMA. 2000; 283: 2008-2012
        • Dawes M.
        • Sampson U.
        Knowledge management in clinical practice: a systematic review of information seeking behavior in physicians.
        Int J Med Inform. 2003; 71: 9-15
      7. Glasziou P, Shepperd S. Ability to apply evidence from systematic reviews. 2007. PMID: 18583680.

        • Shojania K.G.
        • Sampson M.
        • Ansari M.T.
        • Ji J.
        • Doucette S.
        • Moher D.
        How quickly do systematic reviews go out of date? A survival analysis.
        Ann Intern Med. 2007; 147: 224-233
        • Sampson M.
        • Shojania K.G.
        • Garritty C.
        • Horsley T.
        • Ocampo M.
        • Moher D.
        Systematic reviews can be produced and published faster.
        J Clin Epidemiol. 2008; 61: 531-536
        • Moher D.
        • Tsertsvadze A.
        • Tricco A.C.
        • Eccles M.
        • Grimshaw J.
        • Sampson M.
        • et al.
        A systematic review identified few methods and strategies describing when and how to update systematic reviews.
        J Clin Epidemiol. 2007; 60: 1095-1104
        • Moher D.
        • Tsertsvadze A.
        • Tricco A.C.
        • Eccles M.
        • Grimshaw J.
        • Sampson M.
        • et al.
        When and how to update systematic reviews.
        Cochrane Database Syst Rev. 2008; (MR000023)
        • Whitlock E.P.
        • Lopez S.A.
        • Chang S.
        • Helfand M.
        • Eder M.
        • Floyd N.
        Identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the Effective Health Care Program.
        J Clin Epidemiol. 2010; 63: 491-501
        • Lomas J.
        • Fulop N.
        • Gagnon D.
        • Allen P.
        On being a good listener: setting priorities for applied health services research.
        Milbank Q. 2003; 81: 363-388
      8. Luxembourg convention definition of grey literature. Luxembourg, 1997 - Expanded in New York, 2004.