Abstract
Objective
This article discusses search methodology to identify evidence for comparative effectiveness
reviews (CERs) as practiced by the Effective Health Care program.
Study Design and Setting
Review.
Results
Search methods described attempt to overcome the bias inherent in the publication
and distribution of clinical evidence. Bibliographic databases and search strategies
are discussed with special emphasis on searching for observational studies and harms
data. Other techniques described include the use of key articles, citation tracking,
hand searching, and personal communications. Strategies for locating gray literature,
such as clinical trial protocols and regulatory information, are described. Search
reporting and other practical matters are also discussed.
Conclusion
Better reporting and further research on search strategies is needed to develop additional
evidence-based recommendations.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Using existing systematic reviews in complex systematic reviews.Ann Intern Med. 2008; 148: 776-782
- AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health care program.J Clin Epidemiol. 2010; 63: 491-501
- Role of expert searching in health sciences libraries: policy statement by the Medical Library Association adopted September 2003.J Med Libr Assoc. 2005; 93: 42-44
- Systematic reviews need systematic searchers.J Med Libr Assoc. 2005; 93: 74-80
- Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects.J Clin Epidemiol. 2008; 61: 440-448
- Evaluation of the methodological quality of systematic reviews of health status measurement instruments.Qual Life Res. 2009; 18: 313-333
- The use of grey literature in health sciences: a preliminary survey.Bull Med Libr Assoc. 1990; 78: 358-363
- Archiving “event knowledge”: bringing “dark data” to light.J Med Libr Assoc. 2008; 96: 189-191
Grey Literature Network Service, editor. New frontiers in grey literature. Fourth International conference on Grey Literature. Washington, DC: GL’99 proceedings; October 4–5, 1999.
- Grey literature in meta-analyses.Nurs Res. 2003; 52 ([Review]): 256-261
- Identifying relevant studies for systematic reviews.BMJ. 1994; 309 ([Review]): 1286-1291
- Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses?.Lancet. 2000; 356: 1228-1231
- Grey literature in meta-analyses of randomized trials of health care interventions.Cochrane Database Syst Rev [Serial on the Internet]. 2007; 2 (Available at) (Accessed April 28, 2011)
- Challenges in systematic reviews of therapeutic devices and procedures.Ann Intern Med. 2005; 142: 1100-1111
- Evidence-based practice: extending the search to find material for the systematic review.Bull Med Libr Assoc. 2001; 89: 346-352
- Challenges in systematic reviews of complementary and alternative medicine topics.Ann Intern Med. 2005; 142: 1042-1047
- How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study.Health Technol Asses. 2003; 7: 1-76
- Efficiency of searching the grey literature in palliative care.J Pain Symptom Manage. 2001; 22: 797-801
- What should be included in meta-analyses? An exploration of methodological issues using the ISPOT meta-analyses.Int J Technol Assess Health Care. 2000; 16: 1109-1119
- Searching for unpublished trials in Cochrane reviews may not be worth the effort.J Clin Epidemiol. 2009; 62 (e3): 838-844
- FDA: untapped source of unpublished trials.Lancet. 2003; 361: 1402-1403
- How useful are unpublished data from the Food and Drug Administration in meta-analysis?.J Clin Epidemiol. 2003; 56: 44-51
- Quinine for nocturnal leg cramps: a meta-analysis including unpublished data.J Gen Intern Med. 1998; 13: 600-606
- Selective publication of antidepressant trials and its influence on apparent efficacy.N Engl J Med. 2008; 358: 252-260
- Health Canada.(Available at) (Accessed September 21, 2010)
- European Medicines Agency.(Available at) (Accessed September 21, 2010)
- Beyond MEDLINE: reducing bias through extended systematic review search.Int J Technol Assess Health Care. 2003; 19 ([Journal Article]): 168-178
- Comparison of registered and published primary outcomes in randomized controlled trials.JAMA. 2009; 302: 977-984
U.S. National Institutes of Health. ClinicalTrials.gov. 2010; Available at http://clinicaltrials.gov. Accessed September 21, 2010.
- BioMed Central.(Available at) (Accessed September 21, 2010)
Clinical Study Results. Available at http://www.clinicalstudyresults.org/home/. Accessed September 21, 2010.
- World Health Organization.(Available at) (Accessed September 21, 2010)
- More insight into the fate of biomedical meeting abstracts: a systematic review.BMC Med Res Methodol. 2003; 3: 12
- Transition from meeting abstract to full-length journal article for randomized controlled trials.JAMA. 2006; 295: 1281-1287
- Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care.Health Libr Rev. 1999; 16: 112-120
- Comparison of bibliographic databases for information on the rehabilitation of people with severe mental illness.Bull Med Libr Assoc. 2001; 89: 353-362
- The pitfalls of a systematic MEDLINE review in palliative medicine: symptom assessment instruments.Am J Hosp Palliat Care. 2007; 24: 181-184
- How to identify randomized controlled trials in MEDLINE: ten years on.J Med Libr Assoc. 2006; 94: 130-136
- Retrieving randomized controlled trials from MEDLINE: a comparison of 38 published search filters.Health Info Libr J. 2009; 26: 187-202
- Developing optimal search strategies for detecting clinically sound studies in MEDLINE.J Am Med Inform Assoc. 1994; 1: 447-458
- Cochrane Handbook for Systematic Reviews of Interventions: 6.4.11 search filters 2008: Version 5.0.1.(Available at) (Updated September 2008. Accessed September 21, 2010)
- Search filters. Edingurgh2009.(Available at) (Updated August 3, 2009. Accessed September 21, 2010)
- Search Filter Resource 2009.(Available at) (Updated July 2, 2009. Accessed September 21, 2010)
- A pragmatic critical appraisal instrument for search filters: introducing the CADTH CAI.Health Info Libr J. 2009; 26: 211-219
- PRESS: peer review of electronic search strategies.Canadian Agency for Drugs and Technologies in Health, Ottowa, Canada2008
- Beyond MEDLINE for literature searches.J Nurs Scholarsh. 2003; 35 ([Review]): 177-182
- English-language restriction when conducting systematic review-based meta-analyses: systematic review of published studies.([Serial on the Internet]) Canadian Agency for Drugs and Technologies in Health, Ottawa, Canada2009 (Available at) (Accessed April 28, 2011)
- Choosing and using citation and bibliographic database software (BDS).Diabetes Educ. 2008; 34: 457-474
- Web-based citation management compared to end note: options for medical sciences.Med Ref Serv Q. 2008; 27: 260-271
White CM, Ip S, McPheeters M, Carey TS, Cour R, Lohr KN. Using existing systematic reviews to replace de novo processes in conducting Comparative Effectiveness Reviews. In: Agency for Healthcare Research and Quality. Methods Guide for Comparative Effectiveness Reviews [posted September 2009]. Rockville, MD. Available at http://effectivehealthcare.ahrq.gov/healthInfo.cfm?infotype=rr&ProcessID=60. Accessed September 21, 2010.
- Systematic reviews of adverse effects: framework for a structured approach.BMC Med Res Methodol. 2007; 7: 32
- Challenges in systematic reviews that assess treatment harms.Ann Intern Med. 2005; 142 ([Review]): 1090-1099
- AHRQ series paper 4: assessing harms when comparing medical interventions: AHRQ and the Effective Health-Care Program.J Clin Epidemiol. 2010; 63: 502-512
- Incomplete evidence: the inadequacy of databases in tracing published adverse drug reactions in clinical trials.BMC Med Res Methodol. 2001; 1: 7
- Developing efficient search strategies to identify reports of adverse effects in MEDLINE and EMBASE.Health Info Libr J. 2006; 23: 3-12
- Selective exposure reporting and Medline indexing limited the search sensitivity for observational studies of the adverse effects of oral contraceptives.J Clin Epidemiol. 2005; 58: 560-567
- Appendix 6b. Including adverse effects.in: Higgins J.P. Green S. Cochrane handbook for systematic reviews of interventions 425 [Updated May 2005]. Cochrane Colloboration; Cochrane Adverse Effects Subgroup, Chichester, UK2007: 194-195
- Identifying observational studies of surgical interventions in MEDLINE and EMBASE.BMC Med Res Methodol. 2006; 6: 41
- The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies.Ann Intern Med. 2007; 147: 573-577
- Room for improvement? A survey of the methods used in systematic reviews of adverse effects.BMC Med Res Methodol. 2006; 6: 3
- Limited search strategies were effective in finding relevant nonrandomized studies.J Clin Epidemiol. 2006; 59: 1303-1311
- Search filters.(Available at) (Accessed September 21, 2010)
- Strengthening the reporting of observational studies in epidemiology (STROBE): explanation and elaboration.Ann Intern Med. 2007; 147 ([Web only]): W163-W194
- Searching additional databases except PubMed are necessary for a systematic review.Stroke. 2008; 39 (author reply e40): e139
- Identifying clinical trials in the medical literature with electronic databases: MEDLINE alone is not enough.Control Clin Trials. 2000; 21: 476-487
- Effectiveness of different databases in identifying studies for systematic reviews: experience from the WHO systematic review of maternal morbidity and mortality.BMC Med Res Methodol. 2005; 5: 6
- Should meta-analysts search Embase in addition to Medline?.J Clin Epidemiol. 2003; 56: 943-955
- Developing a comprehensive search strategy for evidence based systematic reviews.Evid Based Libr Inf Pract. 2008; 3: 3-32
- Searching for observational studies: what does citation tracking add to PubMed? A case study in depression and coronary heart disease.BMC Med Res Methodol. 2006; 6: 4
Google Scholar. Available at http://scholar.google.com/. Accessed September 21, 2010.
- Elsevier.(Available at) (Accessed September 21, 2010)
PubFocus. Available at http://pubfocus.com/. Accessed September 21, 2010.
- PubReMiner.(Available at) (Accessed September 21, 2010)
- Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses.FASEB J. 2008; 22: 338-342
- Web of Science and Scopus: a comparative review of content and searching capabilities.The Charleston Advisor. 2009; 11: 5-18
- As we may search—comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases.Curr Sci. 2005; 89: 1537-1547
- Three options for citation tracking: Google Scholar, Scopus and Web of Science.Biomed Digit Libr. 2006; 3: 7
- Searching the literature. Be systematic in your searching.BMJ. 1993; 307 ([Comment]): 66
- Reference bias in reports of drug trials.Br Med J (Clin Res Ed). 1987; 295: 654-656
Armour T, Dingwall O, Sampson M. Contribution of checking reference lists to systematic reviews. Poster presentation at: XIII Cochrane Colloquium. Melbourne: Australia; 2005.
- Handsearching the EMHJ for reports of randomized controlled trials by U.K. Cochrane Centre (Bahrain).East Mediterr Health J. 2006; 12: S253-S257
- Guides for reading and interpreting systematic reviews: II. How did the authors find the studies and assess their quality?.Arch Pediatr Adolesc Med. 1998; 152: 812-817
- A comparison of handsearching versus MEDLINE searching to identify reports of randomized controlled trials.Stat Med. 2002; 21: 1625-1634
- Lessons for search strategies from a systematic review, in The Cochrane Library, of nutritional supplementation trials in patients after hip fracture.Am J Clin Nutr. 2001; 73: 505-510
- It’s in your hands: the value of handsearching in conducting systematic reviews of public health interventions.J Public Health (Oxf). 2005; 27: 388-391
- Issues in the registration of clinical trials.JAMA. 2007; 297: 2112-2120
- Impact of covert duplicate publication on meta-analysis: a case study.BMJ. 1997; 315: 635-640
- An e-mail survey identified unpublished studies for systematic reviews.J Clin Epidemiol. 2006; 59: 755-758
- Retrieval of missing data for meta-analysis: a practical example.Int J Technol Assess Health Care. 2004; 20: 296-299
- Negative pressure wound therapy: potential publication bias caused by lack of access to unpublished study results data.BMC Med Res Methodol. 2008; 8: 4
- Fair conduct and fair reporting of clinical trials.JAMA. 1999; 282: 1766-1768
- Author contacts for retrieval of data for a meta-analysis on exercise and diet restriction.Int J Technol Assess Health Care. 2006; 22: 267-270
- Analysis of the reporting of search strategies in Cochrane systematic reviews.J Med Libr Assoc. 2009; 97: 21-29
- No consensus exists on search reporting methods for systematic reviews.J Clin Epidemiol. 2008; 61: 748-754
- Value of flow diagrams in reports of randomized controlled trials.JAMA. 2001; 285: 1996-1999
- CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration.PLoS Med. 2008; 5: e20
- Use of the capture-recapture technique to evaluate the completeness of systematic literature searches.BMJ. 1996; 313: 342-343
Article info
Publication history
Published online: June 20, 2011
Accepted:
November 16,
2010
Identification
Copyright
© 2011 Elsevier Inc. Published by Elsevier Inc. All rights reserved.
