Abstract
Objective
To evaluate the measures taken to deal with publication bias across different categories
of systematic reviews published in 2006 and to compare these with reviews published
in 1996.
Study Design and Setting
PubMed was searched for systematic reviews published in 2006; 100 treatment effect,
50 diagnostic accuracy, 100 risk factor, and 50 gene–disease association reviews were
randomly selected.
Results
The use of MEDLINE increased from 74% to 95%; checking references increased from 42%
to 73%; use of Cochrane Library increased from 5% to 58%; and use of CINAHL increased
from 8% in 1996 to 24% in treatment reviews, 20% in diagnostic reviews, 18% in risk
factor reviews, and 0% in genetic reviews published in 2006. A 20% increase was observed
for explicit searching of non–English-language studies in all reviews published in
2006. Efforts to search for unpublished studies increased to 61% from 35% in treatment
reviews published in 1996. Twenty-six percent of the reviews used funnel plots or
related methods to test for publication bias compared with less than 6% in earlier
reviews.
Conclusion
Recent reviews show a significant improvement in the measures taken to prevent publication
bias. However, few methods exist to deal with publication bias in the nonquantitative
findings of systematic reviews.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- The existence of publication bias and risk factors for its occurrence.JAMA. 1990; 263: 1385-1389
- Dissemination and publication of research findings: an updated review of related biases.Health Technol Assess. 2010; 14: 1-219
- Publication and related biases.Health Technol Assess. 2000; 4: 1-115
- The measurement of observer agreement for categorical data.Biometrics. 1977; 33: 159-174
- (Available at) (Accessed Dec 4, 2009)
- Searching the right database. A comparison of four databases for psychiatry journals.Health Libr Rev. 1999; 16: 151-156
- Searching for studies.in: Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.2 (updated September 2009). The Cochrane Collaboration, 2009 (Available at) (Accessed Dec 4, 2009)
- WHO facilitates international collaboration in setting standards for clinical trial registration.Lancet. 2005; 365: 1829-1831
- Clinical trial registration: a statement from the International Committee of Medical Journal Editors.JAMA. 2004; 292: 1363-1364
- Compulsory registration of clinical trials.BMJ. 2004; 329: 637-638
- Clinical trial registration: looking back and moving ahead.Lancet. 2007; 369: 1909-1911
- Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data.Lancet. 2004; 363: 1341-1345
- The costs, effects and cost-effectiveness of strategies to increase coverage of routine immunizations in low- and middle-income countries: systematic review of the grey literature.Bull World Health Organ. 2004; 82: 689-696
- Selective publication of antidepressant trials and its influence on apparent efficacy.N Engl J Med. 2008; 358: 252-260
- Searching for unpublished trials in Cochrane reviews may not be worth the effort.J Clin Epidemiol. 2009; 62: 838-844.e3
- Meta-analysis: unresolved issues and future developments.BMJ. 1998; 316: 221-225
- How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study.Health Technol Assess. 2003; 7: 1-76
- The inclusion of reports of randomised trials published in languages other than English in systematic reviews.Health Technol Assess. 2003; 7: 1-90
- Language bias in neuroscience—is the Tower of Babel located in Germany?.Eur Psychiatry. 2004; 19: 230-232
- No evidence of bias in the process of publication of diagnostic accuracy studies in stroke submitted as abstracts.J Clin Epidemiol. 2009; 62: 425-430
Little J, Higgins JPT, editors. The HuGENet™ HuGE review handbook, version 1.0. Available at http://www.hugenet.ca. Accessed Dec 4, 2009.
- Local literature bias in genetic epidemiology: an empirical evaluation of the Chinese literature.PLoS Med. 2005; 2 ([see comment]): e334
- The case of the misleading funnel plot.BMJ. 2006; 333: 597-600
- The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey.CMAJ. 2007; 176: 1091-1096
- Addressing reporting biases.in: Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Intervention. Version 5.0.1 (updated September 2008). The Cochrane Collaboration, 2008 (Available at) (Accessed Dec 4, 2009)
Article info
Publication history
Published online: September 01, 2010
Accepted:
April 23,
2010
Footnotes
Competing interest: none declared.
Authors' contributions: F.S., Y.K.L., L.H., J.J.R., A.J.S., and I.H. developed the review protocol. S.P.B., J.J.R., and F.S. conducted literature search. S.P.B., F.S., L.H., Y.K.L., J.J.R., C.B.H., C.S.K., and C.P. extracted and/or checked data from included studies. F.S., A.J.S., and I.H. provided methodological support. S.P.B., F.S., C.S.K., and C.P. analyzed data and drafted the article. All authors commented on the final manuscript.
Identification
Copyright
© 2011 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
