Following Cochrane review protocols to completion 10 years later: a retrospective cohort study and author survey

  • Edita Runjic
    Department of Pediatrics, General Hospital Dubrovnik, Roka Misetica 2, 20000 Dubrovnik, Croatia
    Search for articles by this author
  • Dalibora Behmen
    Cochrane Croatia, University of Split School of Medicine, Soltanska 2, Split, Croatia
    Search for articles by this author
  • Dawid Pieper
    Institute for Research in Operative Medicine (IFOM), Witten/Herdecke University, Ostmerheimer Str. 200, building 38, 51109 Cologne, Germany
    Search for articles by this author
  • Tim Mathes
    Institute for Research in Operative Medicine (IFOM), Witten/Herdecke University, Ostmerheimer Str. 200, building 38, 51109 Cologne, Germany
    Search for articles by this author
  • Andrea C. Tricco
    Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, Ontario, M5B 1T8, Canada

    Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, 6th Floor, 155 College Street, Toronto, Ontario, M5T 3M7, Canada

    Queen's Collaboration for Health Care Quality: a Joanna Briggs Institute Centre of Excellence, Queen's University, Kingston, Ontario, Canada
    Search for articles by this author
  • David Moher
    Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, 1053 Carling Ave, Ottawa, Ontario K1Y 4E9, Canada
    Search for articles by this author
  • Livia Puljak
    Corresponding author. Center for Evidence-based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000 Zagreb, Croatia.Tel.: +385-21-557-809; fax: +385-21-557-811.
    Center for Evidence-based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000 Zagreb, Croatia

    Department for Development, Research and Health Technology Assessment, Agency for Quality and Accreditation in Health Care and Social Welfare, Planinska 13, 10000 Zagreb, Croatia
    Search for articles by this author



      We analyzed patterns of publication of Cochrane review protocols (CRPs).

      Study Design and Setting

      We analyzed CRPs published in 2010, extracted their characteristics, and analyzed whether they were published by February 2018. We surveyed corresponding authors and Cochrane review groups to analyze reasons for nonpublication of protocols and analyzed factors predicting the time to publication.


      Of 576 CRPs, 446 (77.4%) were published as a full review and 130 (22.6%) were still unpublished in February 2018; among unpublished, 37 (28.5%) were withdrawn and 93 (71.5%) were still active. The most common authors' reason for abandoning a protocol was a lack of time to work on the review. The median time to publication was 2.78 years (range 0.96 to 8.05). Multivariate analysis showed that factors with the strongest association with shorter time to publication were review being an update and new authors added. Analysis only on methodological variables indicated that the strongest association for a shorter time until publication was found for including only published data.


      Almost a quarter of CRPs remains unpublished after 8 years. This figure is slightly higher than in a previous analysis 10 years ago. Strategies for enhancing completion of Cochrane reviews should be considered.


      To read this article in full you will need to make a payment


        • Smith R.
        The Cochrane Collaboration at 20 Much has been achieved, but much remains to be done.
        BMJ. 2013; 347: f7383
        • Moseley A.M.
        • Elkins M.R.
        • Herbert R.D.
        • Maher C.G.
        • Sherrington C.
        Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy.
        J Clin Epidemiol. 2009; 62: 1021-1030
        • Collier A.
        • Heilig L.
        • Schilling L.
        • Williams H.
        • Dellavalle R.P.
        Cochrane Skin Group systematic reviews are more methodologically rigorous than other systematic reviews in dermatology.
        Br J Dermatol. 2006; 155: 1230-1235
        • Boric K.
        • Dosenovic S.
        • Jelicic Kadic A.
        • Batinic M.
        • Cavar M.
        • Urlic M.
        • et al.
        Interventions for postoperative pain in children: an overview of systematic reviews.
        Paediatr Anaesth. 2017; 27: 893-904
        • Page M.J.
        • Shamseer L.
        • Altman D.G.
        • Tetzlaff J.
        • Sampson M.
        • Tricco A.C.
        • et al.
        Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study.
        PLoS Med. 2016; 13: e1002028
        • Tricco A.C.
        • Brehaut J.
        • Chen M.H.
        • Moher D.
        Following 411 Cochrane protocols to completion: a retrospective cohort study.
        PLoS One. 2008; 3: e3684
      1. The Cochrane Library. Available at Accessed April 22, 2019.

        • Green S.
        • Higgins J.P.T.
        Cochrane Handbook for Systematic Reviews of Interventions. [Chapter 2]: Preparing a Cochrane Review.
        2008 (Available at)
        Date accessed: April 22, 2019
        • Uttley L.
        • Montgomery P.
        The influence of the team in conducting a systematic review.
        Syst Rev. 2017; 6: 149
        • Wieseler B.
        • Kerekes M.F.
        • Vervoelgyi V.
        • McGauran N.
        • Kaiser T.
        Impact of document type on reporting quality of clinical drug trials: a comparison of registry reports, clinical study reports, and journal publications.
        BMJ. 2012; 344: d8141
        • Bohlius J.
        • Tonia T.
        • Nuesch E.
        • Juni P.
        • Fey M.F.
        • Egger M.
        • et al.
        Effects of erythropoiesis-stimulating agents on fatigue- and anaemia-related symptoms in cancer patients: systematic review and meta-analyses of published and unpublished data.
        Br J Cancer. 2014; 111: 33-45
        • Golder S.
        • Loke Y.K.
        • Bland M.
        Unpublished data can be of value in systematic reviews of adverse effects: methodological overview.
        J Clin Epidemiol. 2010; 63: 1071-1081
        • Hartling L.
        • Featherstone R.
        • Nuspl M.
        • Shave K.
        • Dryden D.M.
        • Vandermeer B.
        Grey literature in systematic reviews: a cross-sectional study of the contribution of non-English reports, unpublished studies and dissertations to the results of meta-analyses in child-relevant reviews.
        BMC Med Res Methodol. 2017; 17: 64
        • Rohner E.
        • Grabik M.
        • Tonia T.
        • Juni P.
        • Petavy F.
        • Pignatti F.
        • et al.
        Does access to clinical study reports from the European Medicines Agency reduce reporting biases? A systematic review and meta-analysis of randomized controlled trials on the effect of erythropoiesis-stimulating agents in cancer patients.
        PLoS One. 2017; 12: e0189309
        • Spurling G.
        • Mitchell B.
        • van Driel M.
        Unlocking the value of Cochrane reviews for general practitioners.
        Aust J Gen Pract. 2018; 47: 333-336
        • Duchen M.R.
        • Valdeolmillos M.
        • O'Neill S.C.
        • Eisner D.A.
        Effects of metabolic blockade on the regulation of intracellular calcium in dissociated mouse sensory neurones.
        J Physiol. 1990; 424: 411-426