Advertisement

A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity

  • Amélie Yavchitz
    Correspondence
    Corresponding author. 1, Place du Parvis Notre-Dame, Paris Cedex 4 75181, France. Tel.: +1-33-0-1-42-34-78-33; fax: +1-33-0-1-42-34-87-90.
    Affiliations
    Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité (CRESS-UMR1153), Inserm/Université Paris Descartes, 1 place du Parvis Notre Dame, Paris 75004, France

    Centre d'Épidémiologie Clinique, AP-HP (Assistance Publique des Hôpitaux de Paris), Hôpital Hôtel Dieu, 1 place du Parvis Notre Dame, Paris, France

    French Cochrane Center, Paris Descartes University, Sorbonne Paris Cité, Faculté de Médecine, 1 place du Parvis Notre Dame, Paris 75004, France
    Search for articles by this author
  • Philippe Ravaud
    Affiliations
    Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité (CRESS-UMR1153), Inserm/Université Paris Descartes, 1 place du Parvis Notre Dame, Paris 75004, France

    Centre d'Épidémiologie Clinique, AP-HP (Assistance Publique des Hôpitaux de Paris), Hôpital Hôtel Dieu, 1 place du Parvis Notre Dame, Paris, France

    French Cochrane Center, Paris Descartes University, Sorbonne Paris Cité, Faculté de Médecine, 1 place du Parvis Notre Dame, Paris 75004, France

    Department of Epidemiology, Columbia University Mailman School of Public Health, New York, NY, USA
    Search for articles by this author
  • Douglas G. Altman
    Affiliations
    Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
    Search for articles by this author
  • David Moher
    Affiliations
    Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada

    School of Epidemiology, Public health and Preventive Medicine, Faculty of Medicine, University of Ottawa, Canada
    Search for articles by this author
  • Asbjørn Hrobjartsson
    Affiliations
    Nordic Cochrane Centre, Rigshospitalet Department 7811, Copenhagen, Denmark

    Research Unit for Evidence-Based Medicine, Odense University Hospital/University of Southern Denmark, Odense, Denmark
    Search for articles by this author
  • Toby Lasserson
    Affiliations
    Cochrane Editorial Unit, London, UK
    Search for articles by this author
  • Isabelle Boutron
    Affiliations
    Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité (CRESS-UMR1153), Inserm/Université Paris Descartes, 1 place du Parvis Notre Dame, Paris 75004, France

    Centre d'Épidémiologie Clinique, AP-HP (Assistance Publique des Hôpitaux de Paris), Hôpital Hôtel Dieu, 1 place du Parvis Notre Dame, Paris, France

    French Cochrane Center, Paris Descartes University, Sorbonne Paris Cité, Faculté de Médecine, 1 place du Parvis Notre Dame, Paris 75004, France
    Search for articles by this author
Published:February 01, 2016DOI:https://doi.org/10.1016/j.jclinepi.2016.01.020

      Abstract

      Objectives

      We aimed to (1) identify and classify spin (i.e., a description that overstates efficacy and/or understates harm) in systematic reviews and (2) rank spin in abstracts of systematic reviews according to their severity (i.e., the likelihood of distorting readers' interpretation of the results).

      Study Design

      First, we used a four-phase consensus process to develop a classification of different types of spin. Second, we ranked the types of spin in abstracts according to their severity using a Q-sort survey with members of the Cochrane Collaboration.

      Results

      We identified 39 types of spin, 28 from the main text and 21 from the abstract; 13 were specific to the systematic review design. Spin was classified into three categories: (1) misleading reporting, (2) misleading interpretation, and (3) inappropriate extrapolation. Spin ranked as the most severe by the 122 people who participated in the survey were (1) recommendations for clinical practice not supported by findings in the conclusion, (2) misleading title, and (3) selective reporting.

      Conclusion

      This study allowed for identifying spin that is likely to distort interpretation. Our classification could help authors, editors, and reviewers avoid spin in reports of systematic reviews.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Cook D.J.
        • Mulrow C.D.
        • Haynes R.B.
        Systematic reviews: synthesis of best evidence for clinical decisions.
        Ann Intern Med. 1997; 126: 376-380
        • Bastian H.
        • Glasziou P.
        • Chalmers I.
        Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?.
        PLoS Med. 2010; 7: e1000326
        • Fletcher R.H.
        • Black B.
        “Spin” in scientific writing: scientific mischief and legal jeopardy.
        Med Law. 2007; 26: 511-525
        • Boutron I.
        • Dutton S.
        • Ravaud P.
        • Altman D.G.
        Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes.
        JAMA. 2010; 303: 2058-2064
        • Horton R.
        The rhetoric of research.
        BMJ. 1995; 310: 985-987
        • Yavchitz A.
        • Boutron I.
        • Bafeta A.
        • Marroun I.
        • Charles P.
        • Mantz J.
        • et al.
        Misrepresentation of randomized controlled trials in press releases and news coverage: a cohort study.
        PLoS Med. 2012; 9: e1001308
        • Lockyer S.
        • Hodgson R.
        • Dumville J.C.
        • Cullum N.
        “Spin” in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes.
        Trials. 2013; 14: 371
        • Boutron I.
        • Altman D.G.
        • Hopewell S.
        • Vera-Badillo F.
        • Tannock I.
        • Ravaud P.
        Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of Cancer: the SPIIN randomized controlled trial.
        J Clin Oncol. 2014; 32: 4120-4126
        • Ochodo E.A.
        • de Haan M.C.
        • Reitsma J.B.
        • Hooft L.
        • Bossuyt P.M.
        • Leeflang M.M.G.
        Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”.
        Radiology. 2013; 267: 581-588
        • Prasad V.
        • Jorgenson J.
        • Ioannidis J.P.A.
        • Cifu A.
        Observational studies often make clinical practice recommendations: an empirical evaluation of authors' attitudes.
        J Clin Epidemiol. 2013; 66: 361-366.e4
        • Lazarus C.
        • Haneef R.
        • Ravaud P.
        • Boutron I.
        Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention.
        BMC Med Res Methodol. 2015; 15: 85
        • Cofield S.S.
        • Corona R.V.
        • Allison D.B.
        Use of causal language in observational studies of obesity and nutrition.
        Obes Facts. 2010; 3: 353-356
        • Brown A.W.
        • Brown M.M.B.
        • Allison D.B.
        Belief beyond the evidence: using the proposed effect of breakfast on obesity to show 2 practices that distort scientific evidence.
        Am J Clin Nutr. 2013; 98: 1298-1308
        • Tzoulaki I.
        • Liberopoulos G.
        • Ioannidis J.P.A.
        Assessment of claims of improved prediction beyond the Framingham risk score.
        JAMA. 2009; 302: 2345-2352
        • Cummings P.
        • Rivara F.P.
        Spin and boasting in research articles.
        Arch Pediatr Adolesc Med. 2012; 166: 1099-1100
        • Hróbjartsson A.
        • Gøtzsche P.C.
        • Hrobjartsson A.
        • Gotzsche P.C.
        Powerful spin in the conclusion of Wampold et al.'s re-analysis of placebo versus no-treatment trials despite similar results as in original review.
        J Clin Psychol. 2007; 63: 373-377
        • Sumner P.
        • Vivian-Griffiths S.
        • Boivin J.
        • Williams A.
        • Venetis C.A.
        • Davies A.
        • et al.
        The association between exaggeration in health related science news and academic press releases: retrospective observational study.
        BMJ. 2014; 349: g7015
        • Ezzo J.
        • Bausell B.
        • Moerman D.E.
        • Berman B.
        • Hadhazy V.
        Reviewing the reviews. How strong is the evidence? How clear are the conclusions?.
        Intl J Technol Assess Health Care. 2001; 17: 457-466
        • Lau J.
        • Ioannidis J.P.A.
        • Terrin N.
        • Schmid C.H.
        • Olkin I.
        The case of the misleading funnel plot.
        BMJ. 2006; 333: 597-600
        • Cook D.
        • Guyatt G.
        Colloid use for fluid resuscitation: evidence and spin.
        Ann Intern Med. 2001; 135: 205-208
        • Clarke M.
        • Hopewell S.
        • Chalmers I.
        Clinical trials should begin and end with systematic reviews of relevant evidence: 12 years and waiting.
        Lancet. 2010; 376: 20-21
        • Chalmers I.
        • Bracken M.B.
        • Djulbegovic B.
        • Garattini S.
        • Grant J.
        • Gülmezoglu A.M.
        • et al.
        How to increase value and reduce waste when research priorities are set.
        Lancet. 2014; 383: 156-165
        • Weed D.L.
        Meta-analysis and causal inference: a case study of benzene and non-Hodgkin lymphoma.
        Ann Epidemiol. 2010; 20: 347-355
        • Weed D.L.
        Interpreting epidemiological evidence: how meta-analysis and causal inference methods are related.
        Int J Epidemiol. 2000; 29: 387-390
        • Bender R.
        • Bunce C.
        • Clarke M.
        • Gates S.
        • Lange S.
        • Pace N.L.
        • et al.
        Attention should be given to multiplicity issues in systematic reviews.
        J Clin Epidemiol. 2008; 61: 857-865
        • Turner E.H.
        • Matthews A.M.
        • Linardatos E.
        • Tell R.A.
        • Rosenthal R.
        Selective publication of antidepressant trials and its influence on apparent efficacy.
        N Engl J Med. 2008; 358: 252-260
        • Ioannidis J.P.A.
        • Patsopoulos N.A.
        • Rothstein H.R.
        Reasons or excuses for avoiding meta-analysis in forest plots.
        BMJ. 2008; 336: 1413-1415
        • Islamaj Dogan R.
        • Murray G.C.
        • Névéol A.
        • Lu Z.
        Understanding PubMed user search behavior through log analysis.
        Database (Oxford). 2009; 2009: bap018
      1. Higgins J.P.T. Green S. Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. 2011 (The Cochrane Collaboration. Available at: www.cochrane-handbook.org)
      2. Methodological expectations of Cochrane Intervention Reviews (MECIR): Standards for the conduct and reporting of new Cochrane Intervention Reviews 2012.Version 2.0, 2013. Available at: www.editorial-unit.cochrane.org/mecir.

        • Liberati A.
        • Altman D.G.
        • Tetzlaff J.
        • Mulrow C.
        • Gøtzsche P.C.
        • Ioannidis J.P.A.
        • et al.
        The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration.
        BMJ. 2009; 339: b2700
        • Sterne J.A.
        • Egger M.
        • Smith G.D.
        Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis.
        BMJ. 2001; 323: 101-105
        • Langhorne P.
        Bias in meta-analysis detected by a simple, graphical test. Prospectively identified trials could be used for comparison with meta-analyses.
        BMJ. 1998; 316: 471
        • Gabler N.B.
        • Duan N.
        • Liao D.
        • Elmore J.G.
        • Ganiats T.G.
        • Kravitz R.L.
        Dealing with heterogeneity of treatment effects: is the literature up to the challenge?.
        Trials. 2009; 10: 43
        • Lijmer J.G.
        • Bossuyt P.M.M.
        • Heisterkamp S.H.
        Exploring sources of heterogeneity in systematic reviews of diagnostic tests.
        Stat Med. 2002; 21: 1525-1537
        • Kent D.M.
        • Rothwell P.M.
        • Ioannidis J.P.A.
        • Altman D.G.
        • Hayward R.A.
        Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal.
        Trials. 2010; 11: 85
        • Sterne J.A.C.
        • Sutton A.J.
        • Ioannidis J.P.A.
        • Terrin N.
        • Jones D.R.
        • Lau J.
        • et al.
        Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials.
        BMJ. 2011; 343: d4002
        • Saint S.
        • Christakis D.A.
        • Saha S.
        • Elmore J.G.
        • Welsh D.E.
        • Baker P.
        • et al.
        Journal reading habits of internists.
        J Gen Intern Med. 2000; 15: 881-884
        • Stephenson W.
        Correlating persons instead of tests.
        J Pers. 1935; 4: 17-24
        • van Exel J.
        • de Graaf G.
        Q methodology—a sneak preview.
        Soc Sci Med. 2005; 2: 1-30
        • Brown S.R.
        Q-Methodology and quality research.
        Qual Health Res. 1996; 6: 561-567
        • Dennis K.E.
        Q methodology: relevance and application to nursing research.
        ANS Adv Nurs Sci. 1986; 8: 6-17
      3. Hackert C, Braehler G. FlashQ software [Computer software] 2007, Available at: http://www.hackert.biz/flashq/home . Accessed February 18, 2016.

        • Punjasawadwong Y.
        • Phongchiewboon A.
        • Bunchungmongkol N.
        Bispectral index for improving anaesthetic delivery and postoperative recovery.
        Cochrane Database Syst Rev. 2014; : CD003843
        • Wilkes M.M.
        • Navickis R.J.
        Patient survival after human albumin administration: a meta-analysis of randomized, controlled trials.
        Ann Intern Med. 2001; 135: 149-164
        • Puig L.
        • López A.
        • Vilarrasa E.
        • García I.
        Efficacy of biologics in the treatment of moderate-to-severe plaque psoriasis: a systematic review and meta-analysis of randomized controlled trials with different time points.
        J Eur Acad Dermatol Venereol. 2013; 28: 1633-1653
        • Schulz K.F.
        • Altman D.G.
        • Moher D.
        CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials.
        J Clin Epidemiol. 2010; 63: 834-840
        • Beller E.M.
        • Glasziou P.P.
        • Altman D.G.
        • Hopewell S.
        • Bastian H.
        • Chalmers I.
        • et al.
        PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts.
        PLoS Med. 2013; 10: e1001419
        • Turner L.
        • Shamseer L.
        • Altman D.G.
        • Schulz K.F.
        • Moher D.
        Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review.
        Syst Rev. 2012; 1: 60
        • Henrotin Y.E.
        Interpretation of metaanalyses: pitfalls should be more widely recognized.
        J Rheumatol. 2012; 39: 1107-1109
        • Ioannidis J.P.
        Meta-research: the art of getting it wrong.
        Res Synth Methods. 2010; 1: 169-184
        • Lai N.M.
        • Teng C.L.
        • Lee M.L.
        Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study.
        BMC Med. 2011; 9: 30
        • Glenton C.
        • Santesso N.
        • Rosenbaum S.
        • Nilsen E.S.
        • Rader T.
        • Ciapponi A.
        • et al.
        Presenting the results of Cochrane Systematic Reviews to a consumer audience: a qualitative study.
        Med Decis Making. 2010; 30: 566-577
        • Weed D.L.
        Preventing scientific misconduct.
        Am J Public Health. 1998; 88: 125-129
        • Altman D.
        • Moher D.
        Declaration of transparency for each research article.
        BMJ. 2013; 347: 2-4