Advertisement

McMaster Premium LiteratUre Service (PLUS) performed well for identifying new studies for updated Cochrane reviews

  • Brian J. Hemens
    Affiliations
    Department of Clinical Epidemiology and Biostatistics, Health Information Research Unit, CRL-133, McMaster University, 1280 Main Street West, Hamilton L8S 4K1, Ontario, Canada
    Search for articles by this author
  • R. Brian Haynes
    Correspondence
    Corresponding author. Tel.: +905-525-9140; fax: +905-526-8447.
    Affiliations
    Department of Clinical Epidemiology and Biostatistics, Health Information Research Unit, CRL-133, McMaster University, 1280 Main Street West, Hamilton L8S 4K1, Ontario, Canada

    Department of Medicine, McMaster University, Hamilton, Ontario, Canada
    Search for articles by this author

      Abstract

      Objective

      We compared the performance of McMaster Premium LiteratUre Service (PLUS) and Clinical Queries (CQs) to that of the Cochrane Controlled Trials Register, MEDLINE, and EMBASE for locating studies added during an update of reviews.

      Study Design and Setting

      A sample of new studies in updated Cochrane systematic reviews was used as a reference standard. Searches were performed for each study in each database. Where a new study was not indexed in PLUS, we examined the effect on the review of excluding the study.

      Results

      Ninety-eight updated Cochrane reviews were identified. For the 87 reviews with a usable meta-analysis, PLUS contained all new studies for 13 reviews. No statistically significant difference between PLUS and non-PLUS new studies was found when ratio of odds ratios (RORs) were pooled across 39 reviews (ROR⊕/⊖: 0.99; 95% confidence interval: 0.87–1.14). Thirty-five updated reviews had no new studies indexed in PLUS, but conclusions were seldom altered by addition of new studies.

      Conclusions

      PLUS included less than a quarter of the new studies in Cochrane updates, but most reviews appeared unaffected by the omission of these studies. Reviewers should consider adopting PLUS and CQ filters to improve the efficiency of keeping their reviews up to date.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Cook D.J.
        • Mulrow C.D.
        • Haynes R.B.
        Systematic reviews: synthesis of best evidence for clinical decisions.
        Ann Intern Med. 1997; 126: 376-380
        • Mulrow C.D.
        Rationale for systematic reviews.
        BMJ. 1994; 309: 597-599
        • Haynes R.B.
        Of studies, syntheses, synopses, summaries, and systems: the "5S" evolution of information services for evidence-based healthcare decisions.
        Evid Based Med. 2006; 11: 162-164
        • Chalmers I.
        • Enkin M.
        • Keirse M.J.
        Preparing and updating systematic reviews of randomized controlled trials of health care.
        Milbank Q. 1993; 71: 411-437
        • Moher D.
        • Tsertsvadze A.
        Systematic reviews: when is an update an update?.
        Lancet. 2006; 367: 881-883
        • Moher D.
        • Tsertsvadze A.
        • Tricco A.C.
        • Eccles M.
        • Grimshaw J.
        • Sampson M.
        • et al.
        When and how to update systematic reviews.
        Cochrane Database Syst Rev. 2008; (MR000023)
        • Egger M.
        • Juni P.
        • Bartlett C.
        • Holenstein F.
        • Sterne J.
        How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study.
        Health Technol Assess. 2003; 7: 1-76
        • Allen I.E.
        • Olkin I.
        Estimating time to conduct a meta-analysis from number of citations retrieved.
        JAMA. 1999; 282: 634-635
        • Mallett S.
        • Clarke M.
        How many Cochrane reviews are needed to cover existing evidence on the effects of healthcare interventions?.
        Evid Based Med. 2003; 8: 100-101
        • Wilczynski N.L.
        • Morgan D.
        • Haynes R.B.
        • Hedges Team
        An overview of the design and methods for retrieving high-quality studies for clinical care.
        BMC Med Inform Decis Mak. 2005; 5: 20
        • Nuesch E.
        • Trelle S.
        • Reichenbach S.
        • Rutjes A.W.
        • Burgi E.
        • Scherer M.
        • et al.
        The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study.
        BMJ. 2009; 339 (b3244)
        • Schulz K.F.
        • Chalmers I.
        • Hayes R.J.
        • Altman D.G.
        Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.
        JAMA. 1995; 273: 408-412
        • Sterne J.A.
        • Juni P.
        • Schulz K.F.
        • Altman D.G.
        • Bartlett C.
        • Egger M.
        Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research.
        Stat Med. 2002; 21: 1513-1524
        • Haynes R.B.
        • Holland J.
        • Cotoi C.
        • McKinlay R.J.
        • Wilczynski N.L.
        • Walters L.A.
        • et al.
        McMaster PLUS: a cluster randomized clinical trial of an intervention to accelerate clinical use of evidence-based information from digital libraries.
        J Am Med Inform Assoc. 2006; 13: 593-600
        • Haynes R.B.
        • Cotoi C.
        • Holland J.
        • Walters L.
        • Wilczynski N.
        • Jedraszewski D.
        • et al.
        Second-order peer review of the medical literature for clinical practitioners.
        JAMA. 2006; 295: 1801-1808
      1. Key MEDLINE(R) indicators. Available at: http://www.nlm.nih.gov/bsd/bsd_key.html (Archived by WebCite® at: http://www.webcitation.org/5ppHcbuM5). Accessed December 14, 2009.

        • Wilczynski N.L.
        • McKibbon K.A.
        • Haynes R.B.
        Enhancing retrieval of best evidence for health care from bibliographic databases: calibration of the hand search of the literature.
        Stud Health Technol Inform. 2001; 84: 390-393
        • Haynes R.B.
        • McKibbon K.A.
        • Wilczynski N.L.
        • Walter S.D.
        • Werre S.R.
        • Hedges Team
        Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey.
        BMJ. 2005; 330: 1179
        • Wilczynski N.L.
        • Haynes R.B.
        • Hedges Team
        EMBASE search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews.
        J Clin Epidemiol. 2007; 60: 29-33
        • Wong S.S.
        • Wilczynski N.L.
        • Haynes R.B.
        Developing optimal search strategies for detecting clinically sound treatment studies in EMBASE.
        J Med Libr Assoc. 2006; 94: 41-47
        • Feinstein A.R.
        Principles of medical statistics.
        Chapman & Hall/CRC Press, Boca Raton, FL/London, UK2002
        • Sampson M.
        • Barrowman N.J.
        • Moher D.
        • Klassen T.P.
        • Pham B.
        • Platt R.
        • et al.
        Should meta-analysts search Embase in addition to Medline?.
        J Clin Epidemiol. 2003; 56: 943-955
        • DerSimonian R.
        • Laird N.
        Meta-analysis in clinical trials.
        Control Clin Trials. 1986; 7: 177-188
      2. Bio-stat I. Comprehensive meta-analysis. Available at: www.meta-analysis.com. 2009.

      3. Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions. Available at: http://www.mrc-bsu.cam.ac.uk/cochrane/handbook/chapter_3/3_1_2_how_frequently_should_a_review_be_revisited.htm (Archived by WebCite® at: http://www.webcitation.org/5ppJxEdDH). Accessed December 19, 2009.

        • Shojania K.G.
        • Sampson M.
        • Ansari M.T.
        • Ji J.
        • Doucette S.
        • Moher D.
        How quickly do systematic reviews go out of date? A survival analysis.
        Ann Intern Med. 2007; 147: 224-233
        • McKibbon K.A.
        • Wilczynski N.L.
        • Haynes R.B.
        • Hedges Team
        Retrieving randomized controlled trials from medline: a comparison of 38 published search filters.
        Health Info Libr J. 2009; 26: 187-202
        • Booth A.
        The number needed to retrieve: a practically useful measure of information retrieval?.
        Health Info Libr J. 2006; 23: 229-232
        • Slobogean G.P.
        • Verma A.
        • Giustini D.
        • Slobogean B.L.
        • Mulpuri K.
        MEDLINE, EMBASE, and Cochrane index most primary studies but not abstracts included in orthopedic meta-analyses.
        J Clin Epidemiol. 2009; 62: 1261-1267
        • Suarez-Almazor M.E.
        • Belseck E.
        • Homik J.
        • Dorgan M.
        • Ramos-Remus C.
        Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough.
        Control Clin Trials. 2000; 21: 476-487
        • Sampson M.
        • Zhang L.
        • Morrison A.
        • Barrowman N.J.
        • Clifford T.J.
        • Platt R.W.
        • et al.
        An alternative to the hand searching gold standard: validating methodological search filters using relative recall.
        BMC Med Res Methodol. 2006; 6: 33