Selective exposure reporting and Medline indexing limited the search sensitivity for observational studies of the adverse effects of oral contraceptives

Abstract 

Objective

To explore development of possible approaches leading to a sensitive and precise Medline search to identify observational studies of the association between oral contraceptives and breast cancer, an adverse event.

Study Design and Setting

We compared the results of a series of Medline searches to a gold standard comprising 58 reports from a 1996 systematic review examining the relationship between oral contraceptives and the development of breast cancer. Sensitivity (the proportion of gold standard publications identified) and precision (the proportion of retrieved publications that were included in the gold standard) were calculated for each Medline search.

Results

We identified all 58 articles when the search was not limited by terms related to oral contraceptives, but precision was less than 1% (58 of 6,120). Indexing was problematic when oral contraceptives or hormones were not mentioned in the title or abstract (n=8) or full text (n=2).

Conclusion

Search strategies identifying all relevant studies were possible but arguably impractical; additional research is needed to generalize our findings. Authors and editors should ensure that all interventions and outcomes examined are reported and indexers should make sure they are indexed. Central registration of observational studies and all variables they examined should be considered to assure identification of studies examining adverse events associated with health interventions.

Keywords: Adverse events, Observational studies, Systematic reviews, Information storage and retrieval, Sensitivity, Precision, Medline searching