Empirical evaluation of the methods used in systematic reviews including observational studies and randomized trials

Published:February 20, 2023DOI:
      This paper is only available as a PDF. To read, Please Download here.



      To examine the methodological characteristics of systematic reviews and meta-analyses including observational studies (OSs) and randomized controlled trials (RCTs), in various medical disciplines.

      Study Design and Setting

      We searched Medline via PubMed to identify systematic reviews of interventions including RCTs and OSs published in 110 journals from 2015 to 2019. We extracted in duplicate general and methodological characteristics of the systematic review.


      We identified 402 systematic reviews. Only 39% (n=160) of them reported the availability of a pre-established protocol. A rationale for including observational data in the systematic review was clearly reported in 25% (n=102) of the systematic reviews. Thirty-two percent (n=130) of the reviews reported a search strategy intending to identify published and unpublished data for RCTs and OSs. The risk of bias of the individual studies was assessed in 89% (n=359) of the systematic reviews. In 74% (n=266) it was assessed for both RCTs and OSs; 180 (50%) used different tools. Information about confounding factors was reported in only 11% of systematic reviews and the type of effect estimates (crude or adjusted) used was specified in only 22% of the systematic reviews. Among the 385 systematic reviews that performed data synthesis, only 132 (33%) pooled OSs and RCTs in the same meta-analysis.


      Including OSs in systematic reviews of interventions could provide useful information but such an approach could also be misleading; thus, several methodological details are needed to ensure appropriate handling of OS and valid results. Our study revealed, though, that substantial methodological information is missing in reports published in high impact factor general and specialty journals.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect