To the Editor:
Halladay et al. [
[1]
] are to be commended for their potentially important study “Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions.” Examining 50 randomly sampled Cochrane reviews, they demonstrated that 84% of all included publications were indexed in PubMed. This confirms unpublished data that I presented at a Health Libraries Group Conference in 2012 which showed a similar figure of 85.82% for 50 reviews randomly sampled from within a purposive sample of disciplines [[2]
]. Other studies [3
, 4
] confirm that searchers are more likely to miss relevant studies from inadequate searches on an index database, in this case MEDLINE, than to miss studies from inadequate numbers of additional databases.While being understandably cautious of equating current practice with optimal practice, we note that the variety of teams involved gives some cause for confidence that these findings are generalizable. Although publication bias rightly remains an ongoing concern, that useful contributing studies might be missed, database bias favors identifying studies that are larger and published in higher-quality peer-reviewed journals.
Readers of Journal of Clinical Epidemiology will appreciate the irony of a study conducted to accompany my conference presentation. The Cochrane Collaboration (now Cochrane) has become synonymous with the need for comprehensive searching across multiple databases. The logo of Cochrane memorably depicts seven trials of corticosteroids in preterm distress that, individually, were insufficient to overturn existing practice until combined cumulatively in an iconic Forest plot.
Q. How many databases do you need to search to establish this important landmark finding?
A. Just one—all seven studies could be located on MEDLINE.
In a further irony, the logo should have included eight studies, not seven, as the Cochrane Web pages readily admit [
[5]
]. All eight studies are easily identifiable from MEDLINE and from major pediatrics and obstetrics journals.With subsequent updates, the numbers of trials and study reports increased. By 2007, when the review title changed to Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth [
[6]
] and included 21 different studies, the index article for every single trial was identifiable from MEDLINE. Additional reports are present in conference proceedings and may contribute additional data to a study. However, in terms of retrieval, the index article offers a means of backward and forward chaining (i.e., conducting searches for cited and citing articles). Identification of conference articles is unlikely to be facilitated by subject searching.There are two implications of such findings: (1) that the priority for database searching is to construct thorough searches of the index database not to pursue the progressive futility of additional databases and (2) that efforts to identify more elusive types of literature should focus on identifying a database-indexed index article and then using supplementary search techniques (citation searching, author searching, searching by trial name or registry number, etcetera) to identify additional data.
References
- Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventions.J Clin Epidemiol. 2015; 68: 1076-1084
Paper presented at Health Libraries Under the Microscope: Perfecting Your Formula, CILIP Health Libraries Group (HLG 2012). Glasgow: Glasgow Science Centre, 12–13 2012.
- Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care.Health Libr Rev. 1999; 16: 112-120
- Beyond Medline: reducing bias through extended systematic review search.Int J Technol Assess Health Care. 2003; 19: 168-178
- The story of the Cochrane logo.2013 (Available at:) (Accessed January 21, 2016)
- Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth.Cochrane Database Syst Rev. 2006; : CD004454
Article info
Publication history
Published online: April 20, 2016
Identification
Copyright
© 2016 Elsevier Inc. All rights reserved.
ScienceDirect
Access this article on ScienceDirectLinked Article
- Using data sources beyond PubMed has a modest impact on the results of systematic reviews of therapeutic interventionsJournal of Clinical EpidemiologyVol. 68Issue 9
