Abstract
Objectives
We aimed to (1) identify and classify spin (i.e., a description that overstates efficacy
and/or understates harm) in systematic reviews and (2) rank spin in abstracts of systematic
reviews according to their severity (i.e., the likelihood of distorting readers' interpretation
of the results).
Study Design
First, we used a four-phase consensus process to develop a classification of different
types of spin. Second, we ranked the types of spin in abstracts according to their
severity using a Q-sort survey with members of the Cochrane Collaboration.
Results
We identified 39 types of spin, 28 from the main text and 21 from the abstract; 13
were specific to the systematic review design. Spin was classified into three categories:
(1) misleading reporting, (2) misleading interpretation, and (3) inappropriate extrapolation.
Spin ranked as the most severe by the 122 people who participated in the survey were
(1) recommendations for clinical practice not supported by findings in the conclusion,
(2) misleading title, and (3) selective reporting.
Conclusion
This study allowed for identifying spin that is likely to distort interpretation.
Our classification could help authors, editors, and reviewers avoid spin in reports
of systematic reviews.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Systematic reviews: synthesis of best evidence for clinical decisions.Ann Intern Med. 1997; 126: 376-380
- Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?.PLoS Med. 2010; 7: e1000326
- “Spin” in scientific writing: scientific mischief and legal jeopardy.Med Law. 2007; 26: 511-525
- Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes.JAMA. 2010; 303: 2058-2064
- The rhetoric of research.BMJ. 1995; 310: 985-987
- Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study.PLoS Med. 2012; 9: e1001308
- “Spin” in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes.Trials. 2013; 14: 371
- Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of Cancer: the SPIIN randomized controlled trial.J Clin Oncol. 2014; 32: 4120-4126
- Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”.Radiology. 2013; 267: 581-588
- Observational studies often make clinical practice recommendations: an empirical evaluation of authors' attitudes.J Clin Epidemiol. 2013; 66: 361-366.e4
- Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention.BMC Med Res Methodol. 2015; 15: 85
- Use of causal language in observational studies of obesity and nutrition.Obes Facts. 2010; 3: 353-356
- Belief beyond the evidence: using the proposed effect of breakfast on obesity to show 2 practices that distort scientific evidence.Am J Clin Nutr. 2013; 98: 1298-1308
- Assessment of claims of improved prediction beyond the Framingham risk score.JAMA. 2009; 302: 2345-2352
- Spin and boasting in research articles.Arch Pediatr Adolesc Med. 2012; 166: 1099-1100
- Powerful spin in the conclusion of Wampold et al.'s re-analysis of placebo versus no-treatment trials despite similar results as in original review.J Clin Psychol. 2007; 63: 373-377
- The association between exaggeration in health related science news and academic press releases: retrospective observational study.BMJ. 2014; 349: g7015
- Reviewing the reviews. How strong is the evidence? How clear are the conclusions?.Intl J Technol Assess Health Care. 2001; 17: 457-466
- The case of the misleading funnel plot.BMJ. 2006; 333: 597-600
- Colloid use for fluid resuscitation: evidence and spin.Ann Intern Med. 2001; 135: 205-208
- Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting.Lancet. 2010; 376: 20-21
- How to increase value and reduce waste when research priorities are set.Lancet. 2014; 383: 156-165
- Meta-analysis and causal inference: a case study of benzene and non-Hodgkin lymphoma.Ann Epidemiol. 2010; 20: 347-355
- Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related.Int J Epidemiol. 2000; 29: 387-390
- Attention should be given to multiplicity issues in systematic reviews.J Clin Epidemiol. 2008; 61: 857-865
- Selective publication of antidepressant trials and its influence on apparent efficacy.N Engl J Med. 2008; 358: 252-260
- Reasons or excuses for avoiding meta-analysis in forest plots.BMJ. 2008; 336: 1413-1415
- Understanding PubMed user search behavior through log analysis.Database (Oxford). 2009; 2009: bap018
- Higgins J.P.T. Green S. Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. 2011 (The Cochrane Collaboration. Available at: www.cochrane-handbook.org)
Methodological expectations of Cochrane Intervention Reviews (MECIR): Standards for the conduct and reporting of new Cochrane Intervention Reviews 2012.Version 2.0, 2013. Available at: www.editorial-unit.cochrane.org/mecir.
- The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration.BMJ. 2009; 339: b2700
- Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis.BMJ. 2001; 323: 101-105
- Bias in meta-analysis detected by a simple, graphical test. Prospectively identified trials could be used for comparison with meta-analyses.BMJ. 1998; 316: 471
- Dealing with heterogeneity of treatment effects: is the literature up to the challenge?.Trials. 2009; 10: 43
- Exploring sources of heterogeneity in systematic reviews of diagnostic tests.Stat Med. 2002; 21: 1525-1537
- Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal.Trials. 2010; 11: 85
- Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials.BMJ. 2011; 343: d4002
- Journal reading habits of internists.J Gen Intern Med. 2000; 15: 881-884
- Correlating persons instead of tests.J Pers. 1935; 4: 17-24
- Q methodology—a sneak preview.Soc Sci Med. 2005; 2: 1-30
- Q-Methodology and quality research.Qual Health Res. 1996; 6: 561-567
- Q methodology: relevance and application to nursing research.ANS Adv Nurs Sci. 1986; 8: 6-17
Hackert C, Braehler G. FlashQ software [Computer software] 2007, Available at: http://www.hackert.biz/flashq/home. Accessed February 18, 2016.
- Bispectral index for improving anaesthetic delivery and postoperative recovery.Cochrane Database Syst Rev. 2014; : CD003843
- Patient survival after human albumin administration: a meta-analysis of randomized, controlled trials.Ann Intern Med. 2001; 135: 149-164
- Efficacy of biologics in the treatment of moderate-to-severe plaque psoriasis: a systematic review and meta-analysis of randomized controlled trials with different time points.J Eur Acad Dermatol Venereol. 2013; 28: 1633-1653
- CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials.J Clin Epidemiol. 2010; 63: 834-840
- PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts.PLoS Med. 2013; 10: e1001419
- Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review.Syst Rev. 2012; 1: 60
- Interpretation of metaanalyses: pitfalls should be more widely recognized.J Rheumatol. 2012; 39: 1107-1109
- Meta-research: the art of getting it wrong.Res Synth Methods. 2010; 1: 169-184
- Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study.BMC Med. 2011; 9: 30
- Presenting the results of Cochrane Systematic Reviews to a consumer audience: a qualitative study.Med Decis Making. 2010; 30: 566-577
- Preventing scientific misconduct.Am J Public Health. 1998; 88: 125-129
- Declaration of transparency for each research article.BMJ. 2013; 347: 2-4
Article info
Publication history
Published online: February 01, 2016
Accepted:
January 19,
2016
Footnotes
Conflict of interest: None.
Funding: A.Y. was funded by a grant from the Fondation pour la Recherche Médicale (FDT20140930875). D.M. is funded through a University Research Chair.
Transparency declaration: A.Y. affirms that this article is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned have been explained [
[55]
].Identification
Copyright
© 2016 Elsevier Inc. All rights reserved.
