Abstract
Objectives
To determine whether librarian and information specialist authorship was associated
with better reported systematic review (SR) search quality.
Study Design and Setting
SRs from high-impact general internal medicine journals were reviewed for search quality
characteristics and reporting quality by independent reviewers using three instruments,
including a checklist of Institute of Medicine Recommended Standards for the Search
Process and a scored modification of the Peer Review of Electronic Search Strategies
instrument.
Results
The level of librarian and information specialist participation was significantly
associated with search reproducibility from reported search strategies (Χ2 = 23.5; P < 0.0001). Librarian co-authored SRs had significantly higher odds of meeting 8 of
13 analyzed search standards than those with no librarian participation and six more
than those with mentioned librarian participation. One-way ANOVA showed that differences
in total search quality scores between all three groups were statistically significant
(F2,267 = 10.1233; P < 0.0001).
Conclusion
Problems remain with SR search quality and reporting. SRs with librarian or information
specialist co-authors are correlated with significantly higher quality reported search
strategies. To minimize bias in SRs, authors and editors could encourage librarian
engagement in SRs including authorship as a potential way to help improve documentation
of the search strategy.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Evidence-based health care: a look into the future. EBHC Conference 2013; 30 October.(Taormina, Italy)2013 (Available at) (. Accessed December 1, 2013)
- Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?.PLoS Med. 2010; 7: e1000326
- Epidemiology and reporting characteristics of systematic reviews.PLoS Med. 2007; 4: e78
- Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.PLoS Med. 2009; 6: e1000097
- Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of reporting of meta-analyses.Lancet. 1999; 354: 1896-1900
- Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of Observational Studies in Epidemiology (MOOSE) group.JAMA. 2000; 283: 2008-2012
- Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.BMC Med Res Methodol. 2007; 7: 10
- Committee on standards for systematic reviews of comparative effectiveness research.in: Eden J. Finding what works in health care : standards for systematic reviews. National Academies Press, Washington, D.C2011 (xxii, 317 p. p)
- Cochrane handbook for systematic reviews of interventions.The Cochrane Collaboration, 2011 (Available at) ([updated March]; Accessed December 18, 2013)
- Analysis of the reporting of search strategies in Cochrane systematic reviews.J Med Libr Assoc. 2009; 97: 21-29
- No consensus exists on search reporting methods for systematic reviews.J Clin Epidemiol. 2008; 61: 748-754
- Errors in search strategies were identified by type and frequency.J Clin Epidemiol. 2006; 59: 1057-1063
- Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects.J Clin Epidemiol. 2008; 61: 440-448
- Reproducibility of literature search reporting in medical education reviews.Acad Med. 2011; 86: 1049-1054
- Search strategies in systematic reviews in periodontology and implant dentistry.J Clin Periodontol. 2013; 40: 883-888
- Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011.J Clin Epidemiol. 2013; 66: 253-260
- Systematic reviews need systematic searchers.J Med Libr Assoc. 2005; 93: 74-80
- Mounting evidence that librarians are essential for comprehensive literature searches for meta-analyses and Cochrane reports.J Med Libr Assoc. 2004; 92: 163-164
- Methodological developments in searching for studies for systematic reviews: past, present and future?.Syst Rev. 2013; 2: 78
- Evidence-based retrieval in evidence-based medicine.J Med Libr Assoc. 2004; 92: 196-199
- Reporting of the role of the expert searcher in Cochrane Reviews.Evid Based Libr Inf Pract. 2006; 1: 3-16
- Network meta-analyses could be improved by searching more sources and by involving a librarian.J Clin Epidemiol. 2014; 67: 1001-1007
- How good are clinical MEDLINE searches? A comparative study of clinical end-user and librarian searches.Comput Biomed Res. 1990; 23: 583-593
- Routine development of objectively derived search strategies.Syst Rev. 2012; 1: 19
- By the clock: how much time does an expert search take?.MLA News. 2004; 1: 12
- Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources.BMJ. 2005; 331: 1064-1065
- Journal citation reports: Thomson Reuters. 2011 ([cited 2012]. Available at) (. Accessed August 3, 2012)
- Systematic reviews: a cross-sectional study of location and citation counts.BMC Med. 2003; 1: 2
- Search strategy used to create the systematic reviews subset on PubMed. National Library of Medicine, Bethesda, MD2013 (. Available at) (. [updated 26 February]; Accessed December 13, 2013)
- Optimal search strategies for retrieving systematic reviews from Medline: analytical survey.BMJ. 2005; 330: 68
- Taking advantage of the explosion of systematic reviews: an efficient MEDLINE search strategy.Eff Clin Pract. 2001; 4: 157-162
- The analysis of means: a graphical method for comparing means, rates, and proportions.Society for Industrial and Applied Mathematics, Philadelphia, PA2005
- Answering the call for a standard reliability measure for coding data.Commun Methods Meas. 2007; 1: 77-89
- KALPHA (SAS Version).2011 ([updated 12 June; cited 2012]. Available at) (. Accessed March 23, 2013)
- An evidence-based practice guideline for the peer review of electronic search strategies.J Clin Epidemiol. 2009; 62: 944-952
- PRESS: peer review of electronic search strategies.Canadian Agency for Drugs and Technologies in Health, Ottawa, Canada2008 ([updated January] Available at) (. Accessed December 13, 2013)
- Appendix G: PRESS Checklist.Canadian Agency for Drugs and Technologies in Health, Ottawa, Canada2008 (Available at) (. Accessed December 13, 2013)
- Calculating inter-coder reliability in media content analysis using Krippendorff’s Alpha.2012 (Available at) (. [updated 2 January];Accessed December 13, 2013)
- Boolean versus ranked querying for biomedical systematic reviews.BMC Med Inform Decis Mak. 2010; 10: 58
- How PubMed works: automatic term mapping.National Library of Medicine, Bethesda, MD2013 (Available at) (. [updated 17 December]; Accessed January 17, 2014)
- PubMed now using the redesigned interface.NLM Tech Bull. 2009; : e22
- Peer review of search strategies.(Report No. 12-EHC068-EF. Methods Research Reports) Agency for Healthcare Research and Quality, Rockville, MD2012
- Recording database searches for systematic reviews—what is the value of adding a narrative to peer-review checklists? A case study of NICE interventional procedures guidance.Evid Based Libr Inf Pract. 2011; 6: 72-87
- Publishing systematic reviews in ophthalmology: new guidance for authors.Ophthalmology. 2014; 121: 438-439
- Engaging medical librarians to improve the quality of review articles.JAMA. 2014; 312: 999-1000
- Scope for improvement in the quality of reporting of systematic reviews. From the Cochrane Musculoskeletal Group.J Rheumatol. 2006; 33: 9-15
- Search and selection methodology of systematic reviews in orthodontics (2000-2004).Am J Orthod Dentofacial Orthop. 2006; 130: 214-217
- An evaluation of search and selection methods used in dental systematic reviews published in English.J Am Dent Assoc. 2006; 137: 1252-1257
- Methods guide for effectiveness and comparative effectiveness reviews. Agency for healthcare research and quality, Rockville MD2008 (Available at) (Accessed December 18, 2013)
- Methodological standards for the conduct of new Cochrane Intervention Reviews Version 2.3.The Cochrane Library, 2013 (Available at) (. [updated 2 December]; Accessed October 21, 2014)
- The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews.Syst Rev. 2013; 2: 115
- Precision of healthcare systematic review searches in a cross-sectional sample.Res Synth Methods. 2011; 2: 119-125
- EPC response to IOM standards for systematic reviews. Research white paper.(Prepared by the Tufts Evidence-based Practice Center, Tufts Medical Center under Contract No. 290-2007-10055-I). AHRQ Publication No. 13-EHC006-EF Agency for Healthcare Research and Quality, Rockville, MD2013 (Available at) (. [updated April]; Accessed December 13, 2013)
- Methodology and reporting of meta-analyses in the neurosurgical literature.J Neurosurg. 2014; 120: 796-810
- Association of study quality with completeness of reporting: have completeness of reporting and quality of systematic reviews and meta-analyses in major radiology journals changed since publication of the PRISMA statement?.Radiology. 2013; 269: 413-426
- Comparison of search strategies in systematic reviews of adverse effects to other systematic reviews.Health Inf libraries J. 2014; 31: 92-105
Article info
Publication history
Published online: February 05, 2015
Accepted:
November 19,
2014
Footnotes
Conflict of interest. None.
Funding: None.
Data sharing: Full data sets and technical appendix available from the corresponding author at [email protected]
Identification
Copyright
© 2015 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
