Advertisement
Original Article| Volume 79, P46-54, November 2016

A third of systematic reviews changed or did not specify the primary outcome: a PROSPERO register study

  • Andrea C. Tricco
    Correspondence
    Corresponding author. Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1W8 Canada. Tel.: 416-864-6060x77521; fax: 416-864-6057.
    Affiliations
    Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, Ontario M5B 1T8, Canada

    Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, 155 College Street, 6th Floor, Toronto, Ontario M5T 3M7, Canada
    Search for articles by this author
  • Elise Cogo
    Affiliations
    Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, Ontario M5B 1T8, Canada
    Search for articles by this author
  • Matthew J. Page
    Affiliations
    School of Public Health & Preventive Medicine, The Alfred Centre, Monash University, Level 6, 99 Commercial Road, Melbourne, Victoria 3004, Australia
    Search for articles by this author
  • Julie Polisena
    Affiliations
    Canadian Agency for Drugs and Technologies in Health (CADTH), 865 Carling Avenue Suite 600, Ottawa, Ontario K1S 5S8, Canada

    Department of Epidemiology and Community Medicine, University of Ottawa, 451 Smyth Road, Ottawa, Ontario K1H 8M5, Canada
    Search for articles by this author
  • Alison Booth
    Affiliations
    Centre for Reviews and Dissemination, University of York, Heslington, York YO10 5DD, United Kingdom
    Search for articles by this author
  • Kerry Dwan
    Affiliations
    Department of Biostatistics, Institute of Translational Medicine, University of Liverpool, Crown Street, Liverpool L69 3BX, United Kingdom
    Search for articles by this author
  • Heather MacDonald
    Affiliations
    Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, Ontario M5B 1T8, Canada
    Search for articles by this author
  • Tammy J. Clifford
    Affiliations
    Canadian Agency for Drugs and Technologies in Health (CADTH), 865 Carling Avenue Suite 600, Ottawa, Ontario K1S 5S8, Canada
    Search for articles by this author
  • Lesley A. Stewart
    Affiliations
    Centre for Reviews and Dissemination, University of York, Heslington, York YO10 5DD, United Kingdom
    Search for articles by this author
  • Sharon E. Straus
    Affiliations
    Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, Ontario M5B 1T8, Canada

    Department of Geriatric Medicine, University of Toronto, 27 King's College Circle, Toronto, Ontario M5S 1A1, Canada
    Search for articles by this author
  • David Moher
    Affiliations
    Ottawa Methods Centre, Ottawa Hospital Research Institute, The Ottawa Hospital, 501 Smyth Road, PO Box 201B, Ottawa, Ontario K1H 8L6, Canada
    Search for articles by this author

      Abstract

      Objectives

      To examine outcome reporting bias of systematic reviews registered in PROSPERO.

      Study Design and Setting

      Retrospective cohort study. The primary outcomes from systematic review publications were compared with those reported in the corresponding PROSPERO records; discrepancies in the primary outcomes were assessed as upgrades, additions, omissions, or downgrades. Relative risks (RRs) and 95% confidence intervals (CI) were calculated to determine the likelihood of having a change in primary outcome when the meta-analysis result was favorable and statistically significant.

      Results

      Ninety-six systematic reviews were published. A discrepancy in the primary outcome occurred in 32% of the included reviews and 39% of the reviews did not explicitly specify a primary outcome(s); 6% of the primary outcomes were omitted. There was no significant increased risk of adding/upgrading (RR, 2.14; 95% CI: 0.53, 8.63) or decreased risk of downgrading (RR, 0.76; 95% CI: 0.27, 2.17) an outcome when the meta-analysis result was favorable and statistically significant. As well, there was no significant increased risk of adding/upgrading (RR, 0.89; 95% CI: 0.31, 2.53) or decreased risk of downgrading (RR, 0.56; 95% CI: 0.29, 1.08) an outcome when the conclusion was positive.

      Conclusions

      We recommend review authors carefully consider primary outcome selection, and journals are encouraged to focus acceptance on registered systematic reviews.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

      1. Higgins J. Green S. Cochrane Handbook for Systematic Reviews of Interventions version 5.1.0. The Cochrane Collaboration, 2011 (Available at http://handbook.cochrane.org.)
      2. Institute of Medicine. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press, 2011
        • Liberati A.
        • Altman D.G.
        • Tetzlaff J.
        • Mulrow C.
        • Gotzsche P.C.
        • Ioannidis J.P.
        • et al.
        The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration.
        BMJ. 2009; 339: b2700
        • Dwan K.
        • Kirkham J.J.
        • Williamson P.R.
        • Gamble C.
        Selective reporting of outcomes in randomised controlled trials in systematic reviews of cystic fibrosis.
        BMJ Open. 2013; 3 (e002709)
        • Kirkham J.J.
        • Altman D.G.
        • Williamson P.R.
        Bias due to changes in specified outcomes during the systematic review process.
        PLoS One. 2010; 5: e9810
        • Silagy C.A.
        • Middleton P.
        • Hopewell S.
        Publishing protocols of systematic reviews: comparing what was done to what was planned.
        JAMA. 2002; 287: 2831-2834
        • Parmelli E.
        • Liberati A.
        • D'Amico R.
        Reporting of outcomes in systematic reviews: comparison of protocols and published systematic reviews.
        15th Cochrane Colloquium, Sao Paulo2007
        • Tricco A.C.
        • Tetzlaff J.
        • Sampson M.
        • Fergusson D.
        • Cogo E.
        • Horsley T.
        • et al.
        Few systematic reviews exist documenting the extent of bias: a systematic review.
        J Clin Epidemiol. 2008; 61: 422-434
        • Page M.J.
        • McKenzie J.E.
        • Kirkham J.
        • Dwan K.
        • Kramer S.
        • Green S.
        • et al.
        Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions.
        Cochrane Database Syst Rev. 2014; 10: MR000035
      3. PROSPERO. York, England: Centre for Reviews and Dissemination, Univeristy of York; [updated November 2015]. Available at http://www.crd.york.ac.uk/PROSPERO/.

        • Moher D.
        • Tetzlaff J.
        • Tricco A.C.
        • Sampson M.
        • Altman D.G.
        Epidemiology and reporting characteristics of systematic reviews.
        PloS Med. 2007; 4: e78
        • Moher D.
        • Dulberg C.S.
        • Wells G.A.
        Statistical power, sample size, and their reporting in randomized controlled trials.
        JAMA. 1994; 272: 122-124
        • Tricco A.C.
        • Tetzlaff J.
        • Pham B.
        • Brehaut J.
        • Moher D.
        Non-Cochrane vs. Cochrane reviews were twice as likely to have positive conclusion statements: cross-sectional study.
        J Clin Epidemiol. 2009; 62: 380-386.e1
        • Shea B.J.
        • Hamel C.
        • Wells G.A.
        • Bouter L.M.
        • Kristjansson E.
        • Grimshaw J.
        • et al.
        AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.
        J Clin Epidemiol. 2009; 62: 1013-1020
        • Mesgarpour B.
        • Griebler U.
        • Glechner A.
        • Kien C.
        • Strobelberger M.
        • Van Noord M.G.
        • et al.
        Extended-release opioids in the management of cancer pain: a systematic review of efficacy and safety.
        Eur J Pain. 2014; 18: 605-616
        • Bastian H.
        • Glasziou P.
        • Chalmers I.
        Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?.
        PloS Med. 2010; 7: e1000326