Journal of Clinical Epidemiology
Volume 64, Issue 1 , Pages 11-20 , January 2011

The art and science of knowledge synthesis

  • Andrea C. Tricco

      Affiliations

    • Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada
    • ,
  • Jennifer Tetzlaff

      Affiliations

    • Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada
    • Department of Epidemiology & Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
    • ,
  • David Moher

      Affiliations

    • Ottawa Hospital Research Institute, The Ottawa Hospital, Ottawa, Ontario, Canada
    • Department of Epidemiology & Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
    • Corresponding Author InformationCorresponding author. +Tel.: 613-738-3591; fax: 613-739-6266.

,Accepted 25 November 2009.

References 

  1. Chalmers I, Hedges LV, Cooper H. A brief history of research synthesis. Eval Health Prof. 2002;25:12–37
  2. Naylor CD. Clinical decisions: from art to science and back again. Lancet. 2001;358:523–524
  3. Grimshaw JM, Santesso N, Cumpston M, Mayhew A, McGowan J. Knowledge for knowledge translation: the role of the Cochrane Collaboration. J Contin Educ Health Prof. 2006;26(1):55–62
  4. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map?. J Contin Educ Health Prof. 2006;26(1):13–24
  5. Canadian Institutes of Health Research . Randomized Controlled Trials Registration/Application Checklist. Available at http://www.cihrirsc.gc.ca/e/documents/rct_reg_e.pdf2006;Accessed July 30, 2008
  6. Graham ID, Tetroe J. Some theoretical underpinnings of knowledge translation. Acad Emerg Med. 2007;14:936–941
  7. The Cochrane Collaboration. Available at http://www.cochrane.org/2008;Accessed July 30, 2008
  8. Campbell Collaboration. Available at www.campbellcollaboration.org2008;Accessed July 29, 2008
  9. Atkins D, Fink K, Slutsky J. Better information for better health care: the Evidence-based Practice Center program and the Agency for Healthcare Research and Quality. Ann Intern Med. 2005;142(pt 2):1035–1041
  10. Joanna Briggs Institute. Available at www.joannabriggs.edu.au/about/home.php2008;Accessed July 29, 2008
  11. Dixon-Woods M, Jackson C, Windridge KC, Kenyon S. Receiving a summary of the results of a trial: qualitative study of participants' views. BMJ. 2006;332:206–210
  12. Cochrane handbook for systematic reviews of interventions. Version 5.0.2. [Updated September 2009] Available at www.cochrane-handbook.org2009;Accessed February 20, 2008
  13. Shekelle PG, Morton SC, Jungvig LK, Udani J, Spar M, Tu W, et al. Effect of supplemental vitamin E for the prevention and treatment of cardiovascular disease. J Gen Intern Med. 2004;19:380–389
  14. Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med. 2002;21:2313–2324
  15. Elliott WJ, Meyer PM. Incident diabetes in clinical trials of antihypertensive drugs: a network meta-analysis. Lancet. 2007;369:201–207
  16. Deeks J, Glanville J, Sheldon TA. Undertaking systematic reviews of research on effectiveness: CRD guidelines for those carrying out or commissioning reviews. Chichester, UK: John Wiley & Sons, Ltd; 1996;
  17. Selman TJ, Mann C, Zamora J, Appleyard TL, Khan K. Diagnostic accuracy of tests for lymph node status in primary cervical cancer: a systematic review and meta-analysis. CMAJ. 2008;178:855–862
  18. Little J, Higgins JPT (editors). The HuGENet™ HuGE Review Handbook, version 1.0. Available at http://www.hugenet.ca (accessed 23 February 2009).
  19. Wang GY, Lu CQ, Zhang RM, Hu XH, Luo ZW. The E-cadherin gene polymorphism 160C->A and cancer risk: a HuGE review and meta-analysis of 26 case-control studies. Am J Epidemiol. 2008;167:7–14
  20. Altman DG. Systematic reviews of evaluations of prognostic variables. BMJ. 2001;323:224–228
  21. Oremus M, Raina PS, Santaguida P, Balion CM, McQueen MJ, McKelvie R, et al. A systematic review of BNP as a predictor of prognosis in persons with coronary artery disease. Clin Biochem. 2008;41:260–265
  22. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc Sci Med. 2005;61(2):417–430
  23. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10(suppl 1):21–34
  24. Greenhalgh T, Kristjansson E, Robinson V. Realist review to understand the efficacy of school feeding programmes. BMJ. 2007;335:858–861
  25. Atkins S, Lewin S, Smith H, Engel M, Fretheim A, Volmink J. Conducting a meta-ethnography of qualitative literature: lessons learnt. BMC Med Res Methodol. 2008;8:21
  26. Dixon-Woods M, Fitzpatrick R, Roberts K. Including qualitative research in systematic reviews: opportunities and problems. J Eval Clin Pract. 2001;7:125–133
  27. Dixon-Woods M, Sutton A, Shaw R, Miller T, Smith J, Young B, et al. Appraising qualitative research for inclusion in systematic reviews: a quantitative and qualitative comparison of three methods. J Health Serv Res Policy. 2007;12:42–47
  28. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy. 2005;10:45–53
  29. Harden A, Thomas J. Methodological issues in combining diverse study types in systematic reviews. Int J Soc Res Methodol. 2005;8:257–271
  30. Petticrew M. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ. 2001;322:98–101
  31. Stone PW. Popping the (PICO) question in research and evidence-based practice. Appl Nurs Res. 2002;15(3):197–198
  32. Dixon-Woods M, Bonas S, Booth A, Jones DR, Miller T, Sutton AJ, et al. How can systematic reviews incorporate qualitative research? A critical perspective. Qual Res. 2006;6(1):27–44
  33. Gregoire G, Derderian F, Le LJ. Selecting the language of the publications included in a meta-analysis: is there a Tower of Babel bias?. J Clin Epidemiol. 1995;48:159–163
  34. Egger M, Zellweger-Zahner T, Schneider M, Junker C, Lengeler C, Antes G. Language bias in randomised controlled trials published in English and German. Lancet. 1997;350:326–329
  35. Moher D, Pham B, Klassen TP, Schulz KF, Berlin JA, Jadad AR, et al. What contributions do languages other than English make on the results of meta-analyses?. J Clin Epidemiol. 2000;53:964–972
  36. Juni P, Holenstein F, Sterne J, Bartlett C, Egger M. Direction and impact of language bias in meta-analyses of controlled trials: empirical study. Int J Epidemiol. 2002;31:115–123
  37. Pan Z, Trikalinos TA, Kavvoura FK, Lau J, Ioannidis JP. Local literature bias in genetic epidemiology: an empirical evaluation of the Chinese literature. PLoS Med. 2005;2:e334
  38. Hopewell S, McDonald S, Clarke M, Egger M. Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev. 2007;(2):MR000010
  39. Bhandari M, Devereaux PJ, Guyatt GH, Cook DJ, Swiontkowski MF, Sprague S, et al. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am. 2002;84:615–621
  40. Rosmarakis ES, Soteriades ES, Vergidis PI, Kasiakou SK, Falagas ME. From conference abstract to full paper: differences between data presented in conferences and journals. FASEB J. 2005;19:673–680
  41. Toma M, McAlister FA, Bialy L, Adams D, Vandermeer B, Armstrong PW. Transition from meeting abstract to full-length journal article for randomized controlled trials. JAMA. 2006;295:1281–1287
  42. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007;(2):MR000005
  43. von Elm E, Costanza MC, Walder B, Tramer MR. More insight into the fate of biomedical meeting abstracts: a systematic review. BMC Med Res Methodol. 2003;3:12
  44. U.S. National Library of Medicine . Medical literature analysis and retrieval system online–PubMed. Available at http://www.ncbi.nlm.nih.gov/PubMed2008;Accessed March 19, 2008
  45. Excerpta medica database. Available at http://www.embase.com2008;Accessed March 19, 2008
  46. Cochrane Central Register of Controlled Trials . The Cochrane Library. Available at http://www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOME2008;Accessed March 18, 2008
  47. Tricco AC, Tetzlaff J, Sampson M, Fergusson D, Cogo E, Horsley T, et al. Few systematic reviews exist documenting the extent of bias: a systematic review. J Clin Epidemiol. 2008;61:422–434
  48. Dickersin K. How important is publication bias? A synthesis of available data. AIDS Educ Prev. 1997;9(1 suppl):15–21
  49. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174
  50. Gotzsche PC. Multiple publication of reports of drug trials. Eur J Clin Pharmacol. 1989;36:429–432
  51. Tramer MR, Reynolds DJ, Moore RA, McQuay HJ. Impact of covert duplicate publication on meta-analysis: a case study. BMJ. 1997;315:635–640
  52. Bailey BJ. Duplicate publication in the field of otolaryngology-head and neck surgery. Otolaryngol Head Neck Surg. 2002;126:211–216
  53. Barden J, Edwards JE, McQuay HJ, Moore RA. Oral valdecoxib and injected parecoxib for acute postoperative pain: a quantitative systematic review. BMC Anesthesiol. 2003;3:1
  54. Moher D, Soeken K, Sampson M, Ben-Porat L, Berman B. Assessing the quality of reports of systematic reviews in pediatric complementary and alternative medicine. BMC Pediatr. 2002;2:3
  55. Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol. 2007;36:666–676
  56. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457–2465
  57. Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ. 2004;171:735–740
  58. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330:753
  59. Fleiss JL. The statistical basis of meta-analysis. Stat Methods Med Res. 1993;2(2):121–145
  60. Systematic Reviews in Health Care . Meta-analysis in context. 3rd ed.. London, UK: British Medical Journal Publishing Group; 2008;
  61. Mays N, Pope C, Popay J. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J Health Serv Res Policy. 2005;10(suppl 1):6–20
  62. Sutton AJ, Abrams KR. Bayesian methods in meta-analysis and evidence synthesis. Stat Methods Med Res. 2001;10:277–303
  63. Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. Bayesian methods in health technology assessment: a review. Health Technol Assess. 2000;4(38):1–130
  64. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009;62:1006–1012
  65. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, onso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336:924–926
  66. Ranmal R, Prictor M, Scott JT. Interventions for improving communication with children and adolescents about their cancer. Cochrane Database Syst Rev. 2008;(4):CD002969
  67. Kenyon SL, Taylor DJ, Tarnow-Mordi W. Broad-spectrum antibiotics for preterm, prelabour rupture of fetal membranes: the ORACLE I randomised trial. ORACLE Collaborative Group. Lancet. 2001;357:979–988
  68. Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:1013–1020
  69. Clarke L, Clarke M, Clarke T. How useful are Cochrane reviews in identifying research needs?. J Health Serv Res Policy. 2007;12:101–103
  70. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG. Epidemiology and reporting characteristics of systematic reviews. PLoS Med. 2007;4(3):e78
  71. Atallah AN, de Silva EM, Paiva EV. Disseminating results of systematic reviews through a tv show in Brazil. 6th edition. Baltimore, MD: Syst Rev Evid Action Int Cochrane Colloq; 1998;6:77
  72. Oermann MH, Floyd JA, Galvin EA, Roop JC. Brief reports for disseminating systematic reviews to nurses. Clin Nurse Spec. 2006;20:233–238
  73. O'Connor AM. Using decision aids to help patients navigate the "grey zone" of medical decision-making. CMAJ. 2007;176:1597–1598
  74. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Onkologie. 2000;23:597–602
  75. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700
  76. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) Group. JAMA. 2000;283:2008–2012
  77. Dawes M, Sampson U. Knowledge management in clinical practice: a systematic review of information seeking behavior in physicians. Int J Med Inform. 2003;71(1):9–15
  78. Glasziou P, Shepperd S. Ability to apply evidence from systematic reviews. 2007. PMID: 18583680.
  79. Shojania KG, Sampson M, Ansari MT, Ji J, Doucette S, Moher D. How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med. 2007;147:224–233
  80. Sampson M, Shojania KG, Garritty C, Horsley T, Ocampo M, Moher D. Systematic reviews can be produced and published faster. J Clin Epidemiol. 2008;61:531–536
  81. Moher D, Tsertsvadze A, Tricco AC, Eccles M, Grimshaw J, Sampson M, et al. A systematic review identified few methods and strategies describing when and how to update systematic reviews. J Clin Epidemiol. 2007;60:1095–1104
  82. Moher D, Tsertsvadze A, Tricco AC, Eccles M, Grimshaw J, Sampson M, et al. When and how to update systematic reviews. Cochrane Database Syst Rev. 2008;(1):MR000023
  83. Whitlock EP, Lopez SA, Chang S, Helfand M, Eder M, Floyd N. Identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the Effective Health Care Program. J Clin Epidemiol. 2010;63:491–501
  84. Lomas J, Fulop N, Gagnon D, Allen P. On being a good listener: setting priorities for applied health services research. Milbank Q. 2003;81:363–388
  85. Luxembourg convention definition of grey literature. Luxembourg, 1997 - Expanded in New York, 2004.

 Potential competing interests: J.T., A.C.T., and D.M. are members of the Cochrane Collaboration. Parts of this article were published in the following textbook chapter: J.T., A.C.T., and D.M. Chapter 2.2 Knowledge synthesis. In: Stauss S, Tetroe J, Graham I, eds. Knowledge Translation in Health Care: Moving Evidence to Practice. NJ, Blackwell Publishing Ltd; 2009.

 Funding: At the time of study conduct, A.C.T. was funded by a Canadian Institutes of Health Research (CIHR) Frederick Banting and Charles Best Canada Graduate Scholarship Doctoral Award, a University of Ottawa Excellence Scholarship, and a CIHR Institute of Health Services and Policy Research Rising Star Award. D.M. was funded by a University of Ottawa Research Chair. The funders had no role in any of the following activities: study design; collection, analysis, and interpretation of data; writing of the report; and decision to submit the report for publication.

PII: S0895-4356(09)00361-8

doi: 10.1016/j.jclinepi.2009.11.007

Journal of Clinical Epidemiology
Volume 64, Issue 1 , Pages 11-20 , January 2011

Article Tools

* Image Usage & Resolution