Abstract
Objectives
The objective of the study was to determine the change in quality of evidence in updates
of Cochrane reviews that were initially published between January 1, 2013 and June
30, 2014. We used the Grading of Recommendations, Assessment, Development and Evaluation
(GRADE) system to document evidence quality.
Study Design and Setting
We searched the Cochrane Database of Systematic Reviews on March 20, 2020 to identify
which of the reviews from the initial (2013/14) sample had been updated. Using the
same methods to determine the quality of evidence in the previous analysis, we assessed
the quality of evidence for the first-listed primary outcomes in the updated reviews.
Results
Of the 608 reviews in the original sample, 154 had been updated with and 151 contained
available data for both original and updated systematic reviews (24.8%). The updated
reviews included: 15 (9.9%) with high-quality evidence, 56 (37.1%) with moderate-quality
evidence, 47 (31.1%) with low-quality evidence, and 33 (21.9%) with very low-quality
evidence. No change in the GRADE quality of evidence was found for most (103, 68.2%)
of the updated reviews. The quality of evidence rating was downgraded in 28 reviews
(58.3%) and upgraded in 20 (41.7%), although only six reviews were promoted to high
quality.
Conclusion
Updated systematic reviews continued to suggest that only a minority of outcomes for
health care interventions are supported by high-quality evidence. The quality of the
evidence did not consistently improve or worsen in updated reviews.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Journal of Clinical EpidemiologyAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Reviewing the reviews. How strong is the evidence? How clear are the conclusions?.Int J Technol Assess Health Care. 2001; 17: 457-466
- What to do about CAM: how much of orthodox medicine is evidence based?.BMJ. 2007; 335: 951
- Mapping the Cochrane evidence for decision making in health care.J Eval Clin Pract. 2007; 13: 689-692
- GRADE guidelines: 3. Rating the quality of evidence.J Clin Epidemiol. 2011; 64: 401-406
- High quality of the evidence for medical and other health-related interventions was uncommon in Cochrane systematic reviews.J Clin Epidemiol. 2016; 78: 34-42
- Reporting, updating, and correcting systematic reviews of the effects of health care.BMJ. 1994; 309: 862-865
- Updating systematic reviews: an international survey.PLoS One. 2010; 5: e9914
- Systematic reviews: when is an update an update?.Lancet. 2006; 367: 881-883
- Investing in updating: how do conclusions change when Cochrane systematic reviews are updated?.BMC Med Res Methodol. 2005; 5: 33
- Cochrane Handbook for Systematic Reviews of Interventions version 6.0.The Cochrane Collaboration, Chichester2019
- How quickly do systematic reviews go out of date? A survival analysis.Ann Intern Med. 2007; 147: 224-233
- Buprenorphine maintenance versus placebo or methadone maintenance for opioid dependence.Cochrane Database Syst Rev. 2014; : CD002207
- The Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011].The Cochrane Collaboration, Chichester2011
- Interrater reliability of grading strength of evidence varies with the complexity of the evidence in systematic reviews.J Clin Epidemiol. 2013; 66: 1105-1107.e1
- Testing the risk of bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs.J Clin Epidemiol. 2013; 66: 973-981
- Hormone therapy for preventing cardiovascular disease in post-menopausal women.Cochrane Database Syst Rev. 2015; : CD002229
- Anticoagulation for the initial treatment of venous thromboembolism in people with cancer.Cochrane Database Syst Rev. 2018; 1: CD006649
Article info
Publication history
Published online: September 02, 2020
Accepted:
August 5,
2020
Footnotes
Support: The writing of this protocol was not independently funded.
Declaration of interest: None of the authors have any conflicts of interests related to this study.
Registration: Open Science Framework: Howick, J., Koletsi, D., Fleming, P., Schmidt, S., Loef, M., Walach, H., … Ioannidis, J. (2020, March 30). Has the Quality of Evidence for Medical Interventions Improved? Protocol for a Metaepidemiological Study. Retrieved from osf.io/bw7ky.
Identification
Copyright
© 2020 Elsevier Inc. All rights reserved.
