AHRQ Series Part II: Methods Guide for Comparative Effectiveness - Guest Editor, Mark Helfand| Volume 64, ISSUE 11, P1208-1215, November 2011

Download started.


Updating comparative effectiveness reviews: Current efforts in AHRQ's Effective Health Care Program



      To review the current knowledge and efforts on updating systematic reviews (SRs) as applied to comparative effectiveness reviews (CERs).

      Study Design and Setting

      This article outlines considerations for updating CERs by including a definition of the updating process, describing issues around assessing whether to update, and providing general guidelines for the update process. Key points to consider include (1) identifying when to update CERs, (2) how to update CERs, and (3) how to present, report, and interpret updated results in CERs.


      Currently, there is little information about what proportion of SRs needs updating. Similarly, there is no consensus on when to initiate updating and how best to carry it out.


      CERs need to be regularly updated as new evidence is produced. Lack of attention to updating may lead to outdated and sometimes misleading conclusions that compromise health care and policy decisions. The article outlines several specific goals for future research, one of them being the development of efficient guideline for updating CERs applicable across evidence-based practice centers.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Chalmers I.
        • Enkin M.
        • Keirse M.J.
        Preparing and updating systematic reviews of randomized controlled trials of health care.
        Milbank Q. 1993; 71: 411-437
        • Chalmers I.
        • Haynes B.
        Reporting, updating, and correcting systematic reviews of the effects of health care.
        BMJ. 1994; 309: 862-865
        • Moher D.
        • Tetzlaff J.
        • Tricco A.C.
        • Sampson M.
        • Altman D.G.
        Epidemiology and reporting characteristics of systematic reviews.
        PLoS Med. 2007; 4: e78
        • Higgins J.P.T.
        • Green S.
        • Scholten R.J.P.M.
        Chapter 3. Maintaining reviews: updates, amendments and feedback.
        in: Higgins J.P.T. Green S. Cochrane Handbook for systematic reviews of intervention. Version 5.0.0 [updated February 2008]. The Cochrane Collaboration, 2008 (Available at) (Accessed April 12, 2008)
        • Guirguis-Blake J.
        • Calonge N.
        • Miller T.
        • Siu A.
        • Teutsch S.
        • Whitlock E.
        Current processes of the U.S. Preventive Services Task Force: refining evidence-based recommendation development.
        Ann Intern Med. 2007; 147: 117-122
        • Agency for Healthcare Research and Quality (AHRQ)
        Effective health care program [website].
        2010 (Available at) (Accessed February 23, 2011)
        • Moher D.
        • Tsertsvadze A.
        • Tricco A.C.
        • Eccles M.
        • Grimshaw J.
        • Sampson M.
        • et al.
        A systematic review identified few methods and strategies describing when and how to update systematic reviews.
        J Clin Epidemiol. 2007; 60: 1095-1104
      1. Merriam-Webster’s collegiate dictionary.
        10th edition. Merriam-Webster, Springfield, MA1996
        • Moher D.
        • Tsertsvadze A.
        Systematic reviews: when is an update an update?.
        Lancet. 2006; 367: 881-883
      2. Higgins J.P.T. Green S. Cochrane handbook for systematic reviews of interventions. Version 5.0.2 [updated September 2009]. The Cochrane Collaboration, 2009 (Available at) (Accessed December 21, 2009)
        • Garritty C.
        • Tsertsvadze A.
        • Tricco A.C.
        • Sampson M.
        • Moher D.
        Updating systematic reviews: an international survey.
        PLoS One. 2010; 5: e9914
        • Sampson M.
        • Shojania K.G.
        • McGowan J.
        • Daniel R.
        • Rader T.
        • Iansavichene A.E.
        • et al.
        Surveillance search techniques identified the need to update systematic reviews.
        J Clin Epidemiol. 2008; 61: 755-762
        • Greenhalgh T.
        • Peacock R.
        Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources.
        BMJ. 2005; 331: 1064-1065
        • Shekelle P.
        • Newberry S.
        • Maglione M.
        • Shanman R.
        • Johnsen B.
        • Carter J.
        • et al.
        Assessment of the need to update comparative effectiveness reviews: report of an initial rapid program assessment (2005-2009).
        Agency for Healthcare Research and Quality (AHRQ), Rockville, MD2009
      3. Relevo R, Balshem H. Finding evidence for comparing medical interventions. Agency for Healthcare Research and Quality; January 2011. Methods Guide for Comparative Effectiveness Reviews. AHRQ Publication No. 11-EHC021-EF. Available at

        • Shekelle P.
        • Eccles M.P.
        • Grimshaw J.M.
        • Woolf S.H.
        When should clinical guidelines be updated?.
        BMJ. 2001; 323: 155-157
        • Gartlehner G.
        • West S.L.
        • Lohr K.N.
        • Kahwati L.
        • Johnson J.G.
        • Harris R.P.
        • et al.
        Assessing the need to update prevention guidelines: a comparison of two methods.
        Int J Qual Health Care. 2004; 16: 399-406
        • French S.D.
        • McDonald S.
        • McKenzie J.E.
        • Green S.E.
        Investing in updating: how do conclusions change when Cochrane systematic reviews are updated?.
        BMC Med Res Methodol. 2005; 5: 33
        • Shojania K.G.
        • Sampson M.
        • Ansari M.T.
        • Ji J.
        • Doucette S.
        • Moher D.
        How quickly do systematic reviews go out of date? A survival analysis.
        Ann Intern Med. 2007; 147: 224-233
      4. Garritty C, Tricco A, Sampson M, Tsertsvadze A, Shojania K, Eccles M, et al. Updating Systematic Reviews: The Policies and Practices of Health Care Organizations Involved in Evidence Synthesis. Chapter 4: A Framework for Updating Systematic Reviews [MSc thesis]. University of Toronto: Ontario, Canada; 2009. pp. 53–9.

        • Sampson M.
        • McGowan J.
        • Cogo E.
        • Grimshaw J.
        • Moher D.
        • Lefebvre C.
        An evidence-based practice guideline for the peer review of electronic search strategies.
        J Clin Epidemiol. 2009; 62: 944-952
        • DeAngelis C.D.
        • Drazen J.M.
        • Frizelle F.A.
        • Haug C.
        • Hoey J.
        • Horton R.
        • et al.
        Clinical trial registration: a statement from the International Committee of Medical Journal Editors.
        JAMA. 2004; 292: 1363-1364
        • Manheimer E.
        • Anderson D.
        Survey of public information about ongoing clinical trials funded by industry: evaluation of completeness and accessibility.
        BMJ. 2002; 325: 528-531
        • Bennett D.A.
        • Jull A.
        FDA: untapped source of unpublished trials.
        Lancet. 2003; 361: 1402-1403
        • Moher D.
        • Pham B.
        • Klassen T.P.
        • Schulz K.F.
        • Berlin J.A.
        • Jadad A.R.
        • et al.
        What contributions do languages other than English make on the results of meta-analyses?.
        J Clin Epidemiol. 2000; 53: 964-972
        • McAuley L.
        • Pham B.
        • Tugwell P.
        • Moher D.
        Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses?.
        Lancet. 2000; 356: 1228-1231
      5. Bergerhoff K, Ebrahim S, Paletta G. Do we need to consider ‘in process citations’ for search strategies? 12th Cochrane Colloquium in Ottawa; 2004; Ontario, Canada. pp. 126.

        • Barrowman N.J.
        • Fang M.
        • Sampson M.
        • Moher D.
        Identifying null meta-analyses that are ripe for updating.
        BMC Med Res Methodol. 2003; 3: 13
        • Lau J.
        • Antman E.M.
        • Jimenez-Silva J.
        • Kupelnick B.
        • Mosteller F.
        • Chalmers T.C.
        Cumulative meta-analysis of therapeutic trials for myocardial infarction.
        N Engl J Med. 1992; 327: 248-254
        • Lau J.
        • Schmid C.H.
        • Chalmers T.C.
        Cumulative meta-analysis of clinical trials builds evidence for exemplary medical care.
        J Clin Epidemiol. 1995; 48: 45-57
        • Baum M.L.
        • Anish D.S.
        • Chalmers T.C.
        • Sacks H.S.
        • Smith Jr., H.
        • Fagerstrom R.M.
        A survey of clinical trials of antibiotic prophylaxis in colon surgery: evidence against further use of no-treatment controls.
        N Engl J Med. 1981; 305: 795-799
        • Chalmers T.
        Problems induced by meta-analyses.
        Stat Med. 1991; 10: 971-979
        • Mullen B.
        • Muerllereile P.
        • Bryant B.
        Cumulative meta-analysis: a consideration of indicators of sufficiency and stability.
        Pers Soc Psychol Bull. 2001; 27: 1450-1462
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • The PRISMA Group
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        PLoS Med. 2009; 6: e1000097
        • Rennie D.
        • Flanagin A.
        • Yank V.
        The contributions of authors.
        JAMA. 2000; 284: 89-91
        • Claxton K.
        • Ginnelly L.
        • Sculpher M.
        • Philips Z.
        • Palmer S.
        A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme.
        Health Technol Assess. 2004; 8 (iii): 1-103