In order to make treatment decisions, clinicians often use systematic reviews of randomised controlled trials [
[1]
]. Unfortunately, many reviews present spin (distorted interpretation of results [- Bastian H
- Glasziou P
- Chalmers I.
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?.
PLoS Med. 2010; 7e1000326https://doi.org/10.1371/journal.pmed.1000326
[2]
]), which could lead consumers of research to use unhelpful or even harmful interventions [- Chiu K
- Grundy Q
- Bero L.
Spin' in published biomedical literature: A methodological systematic review.
PLoS Biol. 2017; 15e2002173https://doi.org/10.1371/journal.pbio.2002173
[3]
].We conducted an overview to investigate the presence of spin in systematic reviews of interventions from the top 5 general medical journals, as well as to investigate characteristics that were associated with spin in the abstract. The top 5 journals based on the 2016 list of Impact Factors [
[4]
] were: 1) The Lancet; 2) Journal of American Medical Association (JAMA); 3) British Medical Journal (BMJ); 4) Annals of Internal Medicine (AIM); and 5) JAMA Internal Medicine. We included systematic reviews of clinical trials from these journals published between January 2011 and November 2017, evaluating the effect of interventions in comparison with inert interventions, placebo or no treatment.Two independent authors screened articles and extracted our data. We found 295 potentially eligible articles, from which we randomly selected 196: 32 from JAMA Internal Medicine; 39 from BMJ; 40 from Lancet; 42 from JAMA; and 43 from AIM (APPENDIX 1).
We used the seven-item SPIN checklist [
The remaining percentage did not present spin (score=0).
[5]
] to assess spin in both abstract and full text. The overall classification indicated that 94% of abstracts and 67% of full texts presented at least one item of the SPIN checklist. The most prevalent item in the abstracts was ‘beneficial effect despite high risk of bias’ (78% ‘yes’ and ‘not reported’). Regarding full texts, the most prevalent spin item was ‘selective reporting of harm outcomes’ (37% ‘yes’ and ‘not reported’) (TABLE 1). All journals presented at least one item of spin in the abstract, with only two journals showing better overall classification for all reviews: abstracts in the BMJ with 85% and AIM with 93%; and full texts in the BMJ with 51% and AIM with 61% (TABLE 2). The mean SPIN-score for all abstracts and full texts was low (APPENDIX 2). Abstracts and full texts were not considered consistent in terms of spin (APPENDIX 3).- Yavchitz A
- Ravaud P
- Altman DG
- et al.
A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity.
J Clin Epidemiol. 2016; 75 ([published Online First: 2016/02/05]): 56-65https://doi.org/10.1016/j.jclinepi.2016.01.020
TABLE 1Number and percentages (%) of reviews achieving each item of spin checklist for the abstract and full text for the total sample (n=196).
| ITEM | DESCRIPTION | ABSTRACT | FULL TEXT | ||||
|---|---|---|---|---|---|---|---|
| YES | NOT REPORTED | NO | YES | NOT REPORTED | NO | ||
| 1. Recommendation | Recommendation for clinical practice not supported by the findings | 12 (6.1) | 1 (0.5) | 183 (93.4) | 16 (8.2) | 0 (0.0) | 180 (91.8) |
| 2. Title claims a beneficial effect not supported by the findings | The title claims a beneficial effect of the experimental treatment not supported by the findings | 1 (0.5) | 0 (0.0) | 195 (99.5) | 1 (0.5) | 0 (0.0) | 195 (99.5) |
| 3. Selective reporting of outcomes | Selective reporting of outcomes favouring the beneficial effect of the experimental intervention | 26 (13.3) | 0 (0.0) | 170 (86.7) | 16 (8.2) | 0 (0.0) | 180 (91.8) |
| 4. Safety based on non-statistically significant results | The conclusion claims safety based on non-statistically significant results | 3 (1.5) | 3 (1.5) | 190 (97.0) | 5 (2.5) | 6 (3.1) | 185 (94.4) |
| 5. Beneficial effect despite high risk of bias | The conclusion claims the beneficial effect of the experimental treatment despite high risk of bias in primary studies | 8 (4.1) | 145 (74.0) | 43 (21.9) | 39 (19.9) | 7 (3.6) | 150 (76.5) |
| 6. Selective reporting of harm outcomes | Selective reporting of harm outcomes of the experimental intervention | 1 (0.5) | 103 (52.6) | 92 (46.9) | 4 (2.0) | 69 (35.2) | 123 (62.8) |
| 7. Conclusion extrapolates findings to a different intervention | The conclusion extrapolates the review's findings to a different intervention | 0 (0.0) | 0 (0.0) | 196 (100.0) | 0 (0.0) | 0 (0.0) | 196 (100.0) |
TABLE 2Abstracts and full texts mean SPIN-score (SD) and number (%) of reviews with spin (levels of SPIN-score; range 0 to 7; i. e. no presence of spin to high level of spin) presented separately for each selected journal (n=196).
| Journals | Total of reviews | Abstracts mean SPIN-score (SD) | Full texts mean SPIN-score (SD) | Abstracts with spin | Full texts with spin | ||||
|---|---|---|---|---|---|---|---|---|---|
| At least one item SPIN-score 1 to 4 | SPIN-score (1 or 2) | SPIN-score (3 or 4) | At least one item SPIN-score 1 to 4 | SPIN-score (1 or 2) | SPIN-score (3 or 4) | ||||
| The Lancet | 40 | 1.6 (0.7) | 0.9 (0.7) | 39 (97.5) | 36 (90.0) | 3 (7.5) | 29 (72.5) | 29 (72.5) | 0 (0.0) |
| JAMA | 42 | 1.6 (0.7) | 0.8 (0.7) | 42 (100.0) | 40 (95.2) | 2 (4.8) | 30 (71.4) | 29 (69.0) | 1 (2.4) |
| BMJ | 39 | 1.3 (0.7) | 0.6 (0.6) | 33 (84.6) | 33 (84.6) | 0 (0.0) | 20 (51.3) | 20 (51.3) | 0 (0.0) |
| AIM | 43 | 1.5 (0.9) | 0.8 (0.8) | 40 (93.0) | 35 (81.4) | 5 (11.6) | 26 (60.5) | 24 (55.8) | 2 (4.7) |
| JAMA Internal Medicine | 32 | 1.7 (0.7) | 1.2 (0.8) | 31 (96.9) | 27 (84.4) | 4 (12.5) | 26 (81.2) | 25 (78.1) | 1 (3.1) |
Note: SD - Standard Deviation.
# None of the abstracts and full texts presented SPIN score higher than 4.
The reviews’ methodological quality was assessed by the AMSTAR-2 checklist [
[6]
]. We observed critically low overall confidence in 77% reviews, 19% with low, 3% with moderate and only 1% with high overall confidence (APPENDIX 4).We also found an association between abstracts with higher levels of spin and reviews with critically low methodological quality (ß 0.22; 95%CI: 0.14, 0.63), when compared to reviews of high methodological quality (ß -0.12; 95%CI: -2.69, 0.18) (APPENDIX 2).
A recent systematic review has proven the presence of spin in different healthcare fields [
[7]
]. While some journals in rehabilitation health have already started to modify their policies [[8]
,[9]
], awareness of spin is essentially needed among all journal editorial boards. Health professionals should ultimately know how to interpret the study results by themselves. Hopefully, this letter will help authors, journal editors and reviewers to improve the integrity and transparency of health research journals. Therefore, we strongly recommend training in adequate reporting and interpretation for all research peers.Declaration of Competing Interest
The authors declare no competing interests.
Source of Funding
All authors declare no support or no financial relationships from any organization for the submitted work with any organisations.
Prior presentation(s)
None to declare.
Appendix. Supplementary materials
References
- Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?.PLoS Med. 2010; 7e1000326https://doi.org/10.1371/journal.pmed.1000326
- Spin' in published biomedical literature: A methodological systematic review.PLoS Biol. 2017; 15e2002173https://doi.org/10.1371/journal.pbio.2002173
- Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes.Jama. 2010; 303: 2058-2064https://doi.org/10.1001/jama.2010.651
- Journal Citation Reports (Journal Impact Factor 2017).Clarivate Analytics. 2021; (Available from:)
- A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity.J Clin Epidemiol. 2016; 75 ([published Online First: 2016/02/05]): 56-65https://doi.org/10.1016/j.jclinepi.2016.01.020
- AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both.BMJ. 2017; 358: j4008https://doi.org/10.1136/bmj.j4008
- Do not make clinical decisions based on abstracts of healthcare research: A systematic review.J Clin Epidemiol. 2021; : 136-157https://doi.org/10.1016/j.jclinepi.2021.03.030
Heinemann AW, Chan L, Hoenig HM. Archives Seeks to Improve Abstract Reporting Quality. Archives of physical medicine and rehabilitation2019;100(10):1990-91. doi: 10.1016/j.apmr.2019.05.027
- Spin of results in scientific articles might kill you.Braz J Phys Ther. 2019; 23: 365-366https://doi.org/10.1016/j.bjpt.2019.07.007
Article info
Publication history
Published online: June 29, 2021
Accepted:
June 21,
2021
Received in revised form:
June 14,
2021
Received:
April 30,
2021
Identification
Copyright
© 2021 Elsevier Inc. All rights reserved.
