Abstract
Objectives
To examine how often a significant publication bias (PB) existed when the assessment
of PB was not reported in systematic reviews.
Study Design and Setting
All systematic reviews with meta-analyses of interventions and risk/prognostic factors
published in the general medical journals with the top 10 impact factors in 2011 and
2012 were included. The results regarding PB were extracted. When the assessment of
PB was not reported, we examined the presence of PB using the Egger test and contour-enhanced
funnel plot and the impact of unreported PB by regression-based method.
Results
Among all the identified 116 reviews, the assessment of PB was not reported in 36
reviews (31.0%), particularly in reviews without a comprehensive literature search.
Of these 36 reviews, seven (19.4%) were found to have a significant PB. The original
pooled results may have been overestimated by a median of 50.9% if corrected for PB.
Among the 28 reviews with PB including both reviews that did or did not report the
assessment of PB, seven reviews (25.0%) did not report the presence of PB.
Conclusion
Significant PB was underreported in systematic reviews published in high-impact-factor
journals (eg, 19.4% of those that did not report assessment of PB had significant
PB). Readers of systematic reviews should not assume that PB does not exist when not
reported whereas researchers should report the results of assessments for PB.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Investigating and dealing with publication and other biases.in: Egger M. Smith G.D. Altman D.G. Systematic reviews in health care: meta-analysis in context. 2nd ed. BMJ Publishing Group, London, United Kingdom2001: 189-208
- Dissemination and publication of research findings: an updated review of related biases.Health Technol Assess. 2010; 14 (ix–xi, 1–193): iii
- Publication bias in reports of animal stroke studies leads to major overstatement of efficacy.PLoS Biol. 2010; 8: e1000344
- Epidemiology and reporting characteristics of systematic reviews.PLoS Med. 2007; 4: e78
- Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses.Lancet. 1999; 354: 1896-1900
- Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group.JAMA. 2000; 283: 2008-2012
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.BMJ. 2009; 339: b2535
- Association of journal quality indicators with methodological quality of clinical research articles.JAMA. 2002; 287: 2805-2808
- Systematic reviews and meta-analyses of preclinical studies: publication bias in laboratory animal experiments.Lab Anim. 2011; 45: 225-230
- Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study.BMJ. 2007; 335: 1202-1205
- Addressing reporting biases.in: Higgins J. Green S. Cochrane handbook for systematic reviews of intervention. Version 5.1.0 (updated March 2011). New York: The Cochrane Collaboration, 2011 ([Chapter 10]. Available at) (Accessed May 7, 2013)
Web of knowledge. Available at http://www.thompson.isi.com. Accessed May 7, 2013.
- Assessment of funnel plot asymmetry and publication bias in reproductive health meta-analyses: an analytic survey.Reprod Health. 2007; 4: 3
- Bias in meta-analysis detected by a simple, graphical test.BMJ. 1997; 315: 629-634
- Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry.J Clin Epidemiol. 2008; 61: 991-996
- Exploring and accounting for publication bias in mental health: a brief overview of methods.Evid Based Ment Health. 2014; 17: 11-15
- Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.BMC Med Res Methodol. 2007; 7: 10
- AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.J Clin Epidemiol. 2009; 62: 1013-1020
- Effectiveness of disease-management programs for improving diabetes care: a meta-analysis.CMAJ. 2011; 183: E115-E127
- Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials.BMJ. 2011; 343: d4002
- The prevalence and effect of publication bias in orthopaedic meta-analyses.J Orthop Sci. 2011; 16: 238-244
- Publication bias in antipsychotic trials: an analysis of efficacy comparing the published literature to the US Food and Drug Administration database.PLoS Med. 2012; 9: e1001189
- Selective publication of antidepressant trials and its influence on apparent efficacy.N Engl J Med. 2008; 358: 252-260
- Systematic review of the empirical evidence of study publication bias and outcome reporting bias.PLoS One. 2008; 3: e3081
Article info
Publication history
Published online: September 04, 2014
Accepted:
July 15,
2014
Footnotes
Funding: None.
Competing interests: T.A.F. has received honoraria for speaking at CME meetings sponsored by Asahi Kasei, Eli Lilly, GlaxoSmithKline, Mochida, MSD, Otsuka, Pfizer, Shionogi, and Tanabe-Mitsubishi. He is a diplomate of the Academy of Cognitive Therapy. He has received royalties from Igaku-Shoin, Seiwa-Shoten, and Nihon Bunka Kagaku-sha. He is on advisory board for Sekisui Chemicals and Takeda Science Foundation. The Japanese Ministry of Education, Science, and Technology; the Japanese Ministry of Health, Labor and Welfare; and the Japan Foundation for Neuroscience and Mental Health have funded his other research projects; no other relationships or activities that could appear to have influenced the submitted work.
Identification
Copyright
© 2014 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
