Advertisement

Including nonrandomized studies of interventions in systematic reviews: principles and practicalities

      Health decisions should be informed by the “best available” evidence, irrespective of the study design(s) used to establish it, as highlighted in key definitions for evidence-based medicine [
      • Sackett D.L.
      • Rosenberg W.M.
      • Gray J.A.
      • Haynes R.B.
      • Richardson W.S.
      Evidence based medicine: what it is and what it isn't.
      ] and healthcare [
      • Jordan Z.
      • Lockwood C.
      • Munn Z.
      • Aromataris E.
      The updated Joanna Briggs Institute model of evidence-based healthcare.
      ]. There is now an (almost) universal acceptance within the broader scientific community that extending our healthcare decision making to evidence beyond RCTs is appropriate under the right circumstances, and gone are the days where the remit of systematic reviews was only to synthesize results from randomized controlled trials. As a result, there are now a multitude of research synthesis approaches and techniques to include the full spectrum of study designs commonly encountered across health research [
      • Aromataris E.
      • Stern C.
      • Lockwood C.
      • Barker T.H.
      • Klugar M.
      • Jadotte Y.
      • et al.
      JBI series paper 2: tailored evidence synthesis approaches are required to answer diverse questions: a pragmatic evidence synthesis toolkit from JBI.
      ,
      • Munn Z.
      • Stern C.
      • Aromataris E.
      • Lockwood C.
      • Jordan Z.
      What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences.
      ]. However, in reviews regarding the effectiveness of interventions, some uncertainties still remain regarding the inclusion of nonrandomized studies of interventions (NRSIs). The advantages of including these studies in terms of informing the benefits and harms of an intervention in the absence of RCT data (particularly for harms and in emerging topics) or as supplementary evidence to increase the directness of evidence to inform decisions has been acknowledged [
      • Cuello-Garcia C.A.
      • Santesso N.
      • Morgan R.L.
      • Verbeek J.
      • Thayer K.
      • Ansari M.T.
      • et al.
      GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines.
      ,
      • Schünemann H.J.
      • Tugwell P.
      • Reeves B.C.
      • Akl E.A.
      • Santesso N.
      • Spencer F.A.
      • et al.
      Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions.
      ,
      • Tufanaru C.
      • Munn Z.
      • Stephenson M.
      • Aromataris E.
      Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness.
      ]. Now, despite widespread agreement on the principle of including NRSIs in reviews of interventions, we are less clear on the actual practicalities of doing so, with many diverse approaches existing [
      • Cuello-Garcia C.A.
      • Morgan R.L.
      • Brozek J.
      • Santesso N.
      • Verbeek J.
      • Thayer K.
      • et al.
      A scoping review and survey provides the rationale, perceptions, and preferences for the integration of randomized and nonrandomized studies in evidence syntheses and GRADE assessments.
      ]. This is why the recent Agency for Healthcare Research and Quality guidance on inclusion of NRSIs is both timely and important [
      • Saldanha I.J.
      • Adam G.P.
      • Bañez L.L.
      • Bass E.B.
      • Berliner E.
      • Devine B.
      • et al.
      Inclusion of nonrandomized studies of interventions in systematic reviews of interventions: updated guidance from the agency for healthcare research and quality effective healthcare program.
      ].
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Sackett D.L.
        • Rosenberg W.M.
        • Gray J.A.
        • Haynes R.B.
        • Richardson W.S.
        Evidence based medicine: what it is and what it isn't.
        BMJ. 1996; 312: 71-72
        • Jordan Z.
        • Lockwood C.
        • Munn Z.
        • Aromataris E.
        The updated Joanna Briggs Institute model of evidence-based healthcare.
        Int J Evid Based Healthc. 2019; 17: 58-71
        • Aromataris E.
        • Stern C.
        • Lockwood C.
        • Barker T.H.
        • Klugar M.
        • Jadotte Y.
        • et al.
        JBI series paper 2: tailored evidence synthesis approaches are required to answer diverse questions: a pragmatic evidence synthesis toolkit from JBI.
        J Clin Epidemiol. 2022; (In press)
        • Munn Z.
        • Stern C.
        • Aromataris E.
        • Lockwood C.
        • Jordan Z.
        What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences.
        BMC Med Res Methodol. 2018; 18: 5
        • Cuello-Garcia C.A.
        • Santesso N.
        • Morgan R.L.
        • Verbeek J.
        • Thayer K.
        • Ansari M.T.
        • et al.
        GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines.
        J Clin Epidemiol. 2022; 142: 200-208
        • Schünemann H.J.
        • Tugwell P.
        • Reeves B.C.
        • Akl E.A.
        • Santesso N.
        • Spencer F.A.
        • et al.
        Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions.
        Res Synth Methods. 2013; 4: 49-62
        • Tufanaru C.
        • Munn Z.
        • Stephenson M.
        • Aromataris E.
        Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness.
        Int J Evid Based Healthc. 2015; 13: 196-207
        • Cuello-Garcia C.A.
        • Morgan R.L.
        • Brozek J.
        • Santesso N.
        • Verbeek J.
        • Thayer K.
        • et al.
        A scoping review and survey provides the rationale, perceptions, and preferences for the integration of randomized and nonrandomized studies in evidence syntheses and GRADE assessments.
        J Clin Epidemiol. 2018; 98: 33-40
        • Saldanha I.J.
        • Adam G.P.
        • Bañez L.L.
        • Bass E.B.
        • Berliner E.
        • Devine B.
        • et al.
        Inclusion of nonrandomized studies of interventions in systematic reviews of interventions: updated guidance from the agency for healthcare research and quality effective healthcare program.
        J Clin Epidemiol. 2022; (In press)

      Linked Article