Do physicians judge a study by its cover? An investigation of journal attribution bias

Christakis, Dimitri A; Saint, Sanjay; Saha, Somnath; Elmore, Joann G; Welsh, Deborah E; Baker, Paul; Koepsell, Thomas D

Next »Journal of Clinical Epidemiology
Volume 53, Issue 8 , Pages 773-778, August 2000

Do physicians judge a study by its cover?

An investigation of journal attribution bias

Dimitri A Christakis
- Division of General Pediatrics and Child Health Institute, University of Washington, Seattle, WA, USA
- Corresponding author. Child Health Institute 146 North Canal Street, Suite 300, Seattle, WA 98103-8652, USA
Sanjay Saint
- Division of General Internal Medicine, University of Michigan, Ann Arbor, MI, USA
Somnath Saha
- Division of General Internal Medicine, Portland VA Medical Center, Oregon Health Sciences University, Portland, OR, USA
Joann G Elmore
- Division of General Internal Medicine and Department of Epidemiology, University of Washington, Seattle, WA, USA
Deborah E Welsh
- Ann Arbor VA Health Services Research and Development, Ann Arbor, MI, USA
Paul Baker
- Seattle VA Medical Center, Seattle, WA, USA
Thomas D Koepsell
- Department of Epidemiology and Department of Health Services, University of Washington, Seattle, WA, USA

Received 28 May 1999; accepted 22 December 1999.

Abstract

The effect of a journal's prestige on readers' impressions of an article is unknown. Two hypotheses were tested: first, that attribution of a study to a “high” prestige journal would be associated with improved impressions and attribution to a “low” prestige journal would be associated with diminished impressions; and second, that formal training in epidemiology and biostatistics would mitigate the effects of this journal attribution bias. The study was designed as a trial among a random sample of 264 internists. Participants were asked to read an article and an abstract from either the Southern Medical Journal (SMJ) or the New England Journal of Medicine (NEJM). Questionnaires were constructed that either attributed the article or abstract to its source or presented it as unattributed. After each article or abstract, respondents were asked to rate the quality of the study, the appropriateness of the methodology employed, the significance of the findings, and its likely effects on their practice. A 20-point impression score was created based on responses to these statements. The effect of attribution to a specific journal and formal epidemiology training on impression scores were assessed using linear regression. Of the 399 eligible participants, 264 questionnaires were returned (response rate 66%). Differences in impression scores associated with attribution of an article or abstract to the NEJM were .71 [95% C.I. (−.44–1.87)] and .50 [95% C.I. (−.87–1.87)] respectively; differences in impression scores associated with attribution of an article or abstract to the SMJ were −.12 [95% C.I. (−1.53–1.30)] and −.95 [95% C.I. (−2.41–.52)]. A stratified analysis demonstrated that epidemiology training did not meaningfully alter the effect of journal attribution on participants' impression scores. If journal attribution bias exists, it is likely to exert small and clinically insignificant effects when physicians read articles carefully. Formal training in epidemiology and biostatistics does not appear to alter these results.

Keywords: Evidence based medicine, Meta-analysis, Blinding, Bias