A survey identified publication bias in the secondary literature

Abstract 

Objective

To investigate the existence of publication bias in the translation of evidence from the primary to the secondary literature, using the ACP Journal Club (ACPJC) as a representative of secondary literature and Medline as a representative of primary literature.

Methods

A cross-sectional survey of randomly selected randomized controlled trials of therapy published between 1994 and 2002 in English in Medline and all summaries of therapy trials published by ACPJC between the same dates. The main outcome measure was the rate of positive trials from among those trials aiming to find a difference between groups.

Results

831 trials from Medline and 823 summaries of trials from ACPJC met the inclusion criteria and were included in the study. Compared to trials cataloged in Medline, ACPJC preferentially summarized trials with a positive outcome (P < .001). This bias remained after controlling for other selection biases seen in the ACPJC such as preferentially summarizing multicentered trials with large sample size, no active treatment control, blinding, and in certain disease fields (adjusted odds ratio 2.8, 95% confidence interval 2.02–3.93).

Conclusion

The ACPJC preferentially summarizes trials with a positive outcome. Efforts should be made to reduce this bias.

Keywords: Publication bias, Randomized controlled trials, Therapy, Secondary literature, ACP Journal Club, Medline