Unpublished data can be of value in systematic reviews of adverse effects: methodological overview

Abstract 

Objective

To assess the impact of including unpublished data on adverse effects in systematic reviews.

Study Design and Setting

We carried out a systematic review of methodological evaluations that compared the quantitative reporting of adverse effects data between published and unpublished sources, in particular, the frequency, rate, or risk of reported adverse effects. Included studies were sought from 10 databases as well as by checking references, handsearching, searching citations, and contacting experts.

Results

We identified 6,218 potential articles yielding 10 relevant methodological evaluations.

One evaluation found that adverse effects were reported more often in unpublished trials. For anecdotal case reports, two evaluations found a higher frequency of unpublished cases, whereas one study identified a greater number of published cases. Another evaluation indicated that differences in frequency of published and unpublished case reports were topic dependent.

A comparison of relative risk estimates from five studies suggested no major systematic variation in risk estimates from published and unpublished studies.

Conclusion

Inclusion of unpublished studies can provide additional adverse effects information and more precise risk estimates. However, there is insufficient evidence to indicate whether inclusion of unpublished studies has a major influence on the pooled risk estimates in meta-analyses of adverse effects.

Keywords: Systematic review, Literature review, Adverse effects, Publication bias, Meta-analysis, Unpublished data