A descriptive study found low prevalence of presumed predatory publications in a subset of Cochrane reviews

Published:September 13, 2022DOI:



      To examine the prevalence of presumed predatory publications in Cochrane reviews, which are considered the gold standard.

      Study Design and Setting

      We selected two Cochrane networks with broad scope: the Musculoskeletal, Oral, Skin and Sensory Network and the Public Health and Health Systems Network. From reviews produced by all review groups in those networks in 2018 and 2019, we extracted included study citations published after 2000. For each citation, we assessed the journal and publisher using an algorithmic process based on characteristics known to be common among predatory publishers. Knowing that predatory status can be fluid and subjective, we scored citations on a spectrum from “reputable” to “presumed predatory” based on publication characteristics available at the time of assessment.


      We assessed 6,750 citations from 300 reviews. Of these citations, 5,734 were published by entities widely accepted as reputable, leaving 1,591 for further assessment. We flagged 55 citations as concerning.


      Cochrane reviews across diverse topic areas included studies from flagged publishers, although this number is small. Because of this, there is potential for studies from predatory journals to influence the conclusions of systematic reviews. Researchers should stay aware of this potential threat to the quality of reviews.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Bunn F.
        • Trivedi D.
        • Alderson P.
        • Hamilton L.
        • Martin A.
        • Iliffe S.
        The impact of Cochrane systematic reviews: a mixed method evaluation of outputs from Cochrane review groups supported by the UK National Institute for Health Research.
        Syst Rev. 2014; 3: 125
        • Rice D.B.
        • Skidmore B.
        • Cobey K.D.
        Dealing with predatory journal articles captured in systematic reviews.
        Syst Rev. 2021; 10: 175
        • Oermann M.H.
        • Nicoll L.H.
        • Chinn P.L.
        • Ashton K.S.
        • Conklin J.L.
        • Edie A.H.
        • et al.
        Quality of articles published in predatory nursing journals.
        Nurs Outlook. 2018; 66: 4-10
        • Ross-White A.
        • Godfrey C.M.
        • Sears K.A.
        • Wilson R.
        Predatory publications in evidence syntheses.
        J Med Libr Assoc. 2019; 107: 57-61
        • Hayden J.A.
        Predatory publishing dilutes and distorts evidence in systematic reviews.
        J Clin Epidemiol. 2020; 121: 117-119
        • Shen C.
        • Björk B.C.
        ‘Predatory’ open access: a longitudinal study of article volumes and market characteristics.
        BMC Med. 2015; 13: 1-15
        • Berger M.
        • Cirasella J.
        Beyond Beall’s list: better understanding predatory publishers.
        Coll Res Libr News. 2017; 76: 132-135
        • Gonzalez J.
        • Bridgeman M.B.
        • Hermes-DeSantis E.R.
        Differentiating predatory scholarship: best practices in scholarly publication.
        Int J Pharm Pract. 2018; 26: 73-76
        • Hansoti B.
        • Langdorf M.I.
        • Murphy L.S.
        Discriminating between legitimate and predatory open access journals: report from the international federation for emergency medicine research committee.
        West J Emerg Med. 2016; 17: 497-507
        • Lopez E.
        • Gaspard C.S.
        Predatory publishing and the academic librarian: developing tools to make decisions.
        Med Ref Serv Q. 2020; 39: 1-14
        • McCann T.V.
        • Polacsek M.
        False gold: safely navigating open access publishing to avoid predatory publishers and journals.
        J Adv Nurs. 2018; 74: 809-817
        • Misra D.
        • Ravindran V.
        • Wakhlu A.
        • Sharma A.
        • Agarwal V.
        • Negi V.
        Publishing in black and white: the relevance of listing of scientific journals.
        Rheumatol Int. 2017; 37: 1773-1778
        • Somoza-Fernández M.
        • Rodríguez-Gairín J.M.
        • Urbano C.
        Presence of alleged predatory journals in bibliographic databases: analysis of Beall’s List. Social Science Research Network.
        (Available at)
        Date: 2016
        Date accessed: February 26, 2020
        • Strinzel M.
        • Severin A.
        • Milzow K.
        • Egger M.
        Blacklists and whitelists to tackle predatory publishing: a cross-sectional comparison and thematic analysis.
        mBio. 2019; 10: e00411-e00419
        • Teixeira da Silva J.A.
        • Tsigaris P.
        What value do journal whitelists and blacklists have in academia?.
        J Acad Librariansh. 2018; 44: 781-792
        • Collom C.D.
        • Oermann M.H.
        • Sabol V.K.
        • Heintz P.A.
        An assessment of predatory publication use in reviews.
        Clin Nurse Spec. 2020; 34: 152-156
        • Duc N.M.
        • Hiep D.V.
        • Thong P.M.
        • Zunic L.
        • Zildzic M.
        • Donev D.
        • et al.
        Predatory open access journals are indexed in reputable databases: a revisiting issue or an unsolved problem.
        Med Arch. 2020; 74: 318-322
        • Severin A.
        • Low N.
        Readers beware! Predatory journals are infiltrating citation databases.
        Int J Public Health. 2019; 64: 1123-1124
        • Beall J.
        Predatory publishing is just one of the consequences of gold open access.
        Learn Publ. 2013; 26: 79-84
        • Grudniewicz A.
        • Moher D.
        • Cobey K.D.
        • Bryson G.L.
        • Cukier S.
        • Allen K.
        • et al.
        Predatory journals: no definition, no defence.
        Nature. 2019; 576: 210-212
        • Cukier S.
        • Helal L.
        • Rice D.B.
        • Pupkaite J.
        • Ahmadzai N.
        • Wilson M.
        • et al.
        Checklists to detect potential predatory biomedical journals: a systematic review.
        BMC Med. 2020; 18: 104
        • Hayden J.A.
        • Ellis J.
        • Ogilvie R.
        • Boulos L.
        • Stanojevic S.
        Meta-epidemiological study of publication integrity, and quality of conduct and reporting of randomized trials included in a systematic review of low back pain.
        J Clin Epidemiol. 2021; 134: 65-78
        • Cabells Scholarly Analytics
        Cabells journalytics.
        (Available at)
        Date: 2021
        Date accessed: September 26, 2022
        • DOAJ
        Directory of open access journals.
        (Available at)
        Date: 2020
        Date accessed: September 26, 2022
        • Cabells Scholarly Analytics
        Cabells predatory Reports.
        (Available at)
        Date: 2021
        Date accessed: September 26, 2022
        • Anonymous
        Beall’s list of potential predatory journals and publishers.
        (Available at)
        Date accessed: March 3, 2021
        • Anonymous
        Stop predatory journals.
        (Available at)
        Date accessed: March 3, 2021
        • Brainard J.U.S.
        Judge rules deceptive publisher should pay $50 million in damages.
        Sci Am Assoc Adv Sci. 2019;
        • Dyer O.
        US consumer agency wins $50 m order against predatory publisher OMICS.
        BMJ. 2019; 365: l1639
        • Oransky A.I.
        Court orders publisher OMICS to pay U.S. gov’t $50 million in suit alleging “unfair and deceptive practices.” Retraction Watch.
        (Available at)
        • Krawczyk F.
        • Kulczycki E.
        How is open access accused of being predatory? The impact of Beall’s lists of predatory journals on academic publishing.
        J Acad Librariansh. 2020; 10: 102271
        • Olivarez J.D.
        • Bales S.
        • Sare L.
        Format aside: applying Beall’s criteria to assess the predatory nature of both OA and non-OA library and information science journals.
        Coll Res Libr. 2018; 79: 52-67
        • Cochrane Library
        Editorial policies.
        (Available at)
        Date: 2021
        Date accessed: November 5, 2021
        • Munn Z.
        • Barker T.
        • Stern C.
        • Pollock D.
        • Ross-White A.
        • Klugar M.
        • et al.
        Should I include studies from “predatory” journals in a systematic review? Interim guidance for systematic reviewers.
        JBI Evid Synth. 2021; 19: 1915-1923
        • Cochrane Library
        Cochrane central register of controlled trials (CENTRAL).
        (Available at)

      Linked Article

      • Peering into the dark corners of knowledge synthesis to understand the influence of predatory journals on systematic reviews
        Journal of Clinical Epidemiology
        • Preview
          Predatory journals and publishers are a well described and pervasive problem in research across the globe. Although their characteristics are often debated, an international consensus definition has described these journals and publishers as “entities that prioritize self-interest at the expense of scholarship and are characterized by false or misleading information, deviation from best editorial and publication practices, a lack of transparency, and/or the use of aggressive and indiscriminate solicitation practices [1].” These journals exist primarily to exploit the open access system of publication that has been adopted by the biomedical research community–they collect article processing charges while failing to deliver services provided by legitimate journals (e.g., editorial oversight, copyediting, arranging peer review, indexing in reputable sources).
        • Full-Text
        • PDF