Advertisement

Updated guidance from the agency for healthcare research and quality effective healthcare program on including nonrandomized studies of interventions in systematic reviews: a work in progress

  • Author Footnotes
    1 All authors are convenors of the Cochrane Non-Randomized Studies of Interventions Methods Group.
    Barnaby C. Reeves
    Correspondence
    Corresponding author. Bristol Trials Centre, University of Bristol, 1-5 Whiteladies Road, Bristol BS8 1NU, UK. Tel.: +44 117 456 0234.
    Footnotes
    1 All authors are convenors of the Cochrane Non-Randomized Studies of Interventions Methods Group.
    Affiliations
    Bristol Medical School, University of Bristol, Bristol, UK
    Search for articles by this author
  • Author Footnotes
    1 All authors are convenors of the Cochrane Non-Randomized Studies of Interventions Methods Group.
    Beverley J. Shea
    Footnotes
    1 All authors are convenors of the Cochrane Non-Randomized Studies of Interventions Methods Group.
    Affiliations
    Ottawa Hospital Research Institute, Clinical Epidemiology Program, Ottawa, Canada
    Search for articles by this author
  • Author Footnotes
    1 All authors are convenors of the Cochrane Non-Randomized Studies of Interventions Methods Group.
    George A. Wells
    Footnotes
    1 All authors are convenors of the Cochrane Non-Randomized Studies of Interventions Methods Group.
    Affiliations
    School of Epidemiology and Public Health, Faculty of Medicine and University of Ottawa Heart Institute, University of Ottawa, Ottawa, Canada
    Search for articles by this author
  • Author Footnotes
    1 All authors are convenors of the Cochrane Non-Randomized Studies of Interventions Methods Group.
    Hugh Sharma Waddington
    Footnotes
    1 All authors are convenors of the Cochrane Non-Randomized Studies of Interventions Methods Group.
    Affiliations
    London International Development Centre, London School of Hygiene and Tropical Medicine, London, UK
    Search for articles by this author
  • Author Footnotes
    1 All authors are convenors of the Cochrane Non-Randomized Studies of Interventions Methods Group.
Published:November 09, 2022DOI:https://doi.org/10.1016/j.jclinepi.2022.11.001
      The updated guidance from AHRQ is welcome and timely, given the increasing interest in using evidence from nonrandomized studies to evaluate health and social care interventions (NRSI) [
      • Saldanha I.J.
      • Adam G.P.
      • Bañez L.L.
      • et al.
      Inclusion of nonrandomized studies of interventions in systematic reviews of interventions: updated guidance from the agency for healthcare research and quality effective healthcare program.
      ]. It is particularly heartening to read endorsement of recent insights both about the opportunities afforded by such studies and the potential pitfalls [
      • Hernán M.A.
      • Robins J.M.
      Using big data to emulate a target trial when a randomized trial is not available.
      ,
      • Sterne J.A.C.
      • Hernán M.A.
      • Reeves B.C.
      • Savović J.
      • Berkman N.D.
      • Viswanathan M.
      • et al.
      ROBINS-I: a tool for assessing risk of bias in non-randomized studies of Interventions.
      ]. Of special note are recognition of the “three unique categories of bias” that arise before or at assignment to intervention or comparator status and which threaten the validity of NRSI [
      • Sterne J.A.C.
      • Hernán M.A.
      • Reeves B.C.
      • Savović J.
      • Berkman N.D.
      • Viswanathan M.
      • et al.
      ROBINS-I: a tool for assessing risk of bias in non-randomized studies of Interventions.
      ] and the principle of using “study methods rather than study design labels to differentiate among NRSI types” [
      • Reeves B.C.
      • Wells G.A.
      • Waddington H.
      Quasi-experimental study designs series – paper 5: a checklist for classifying studies evaluating effects of health - a taxonomy without labels.
      ]. These principles ensure consistency between the guidance and the Cochrane Handbook [
      • Reeves B.C.
      • Deeks J.J.
      • Higgins J.P.T.
      • Shea B.
      • Tugwell P.
      • Wells G.A.
      Chapter 24: including non-randomized studies on intervention effects.
      ].
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Saldanha I.J.
        • Adam G.P.
        • Bañez L.L.
        • et al.
        Inclusion of nonrandomized studies of interventions in systematic reviews of interventions: updated guidance from the agency for healthcare research and quality effective healthcare program.
        J Clin Epidemiol. 2022;
        • Hernán M.A.
        • Robins J.M.
        Using big data to emulate a target trial when a randomized trial is not available.
        Am J Epidemiol. 2016; 183: 758-764
        • Sterne J.A.C.
        • Hernán M.A.
        • Reeves B.C.
        • Savović J.
        • Berkman N.D.
        • Viswanathan M.
        • et al.
        ROBINS-I: a tool for assessing risk of bias in non-randomized studies of Interventions.
        BMJ. 2016; 355: i4919
        • Reeves B.C.
        • Wells G.A.
        • Waddington H.
        Quasi-experimental study designs series – paper 5: a checklist for classifying studies evaluating effects of health - a taxonomy without labels.
        J Clin Epidemiol. 2017; 89: 30-42
        • Reeves B.C.
        • Deeks J.J.
        • Higgins J.P.T.
        • Shea B.
        • Tugwell P.
        • Wells G.A.
        Chapter 24: including non-randomized studies on intervention effects.
        in: Higgins J.P.T. Thomas J. Chandler J. Cumpston M. Li T. Page M.J. Cochrane Handbook for Systematic Reviews of Interventions version 6.0. Wiley Blackwell, Hoboken, NJ2019
        • Franklin J.M.
        • Glynn R.J.
        • Martin D.
        • Schneeweiss S.
        Evaluating the use of nonrandomized real-world data analyses for regulatory decision making.
        Clin Pharmacol Ther. 2019; 105: 867-877
        • Deeks J.J.
        • Dinnes J.
        • D’Amico R.
        • Sowden A.J.
        • Sakarovitch C.
        • Song F.
        • et al.
        Evaluating non-randomised intervention studies.
        Health Technol Assess. 2003; 7 (iii-x, 1-173)
        • Higgins J.P.T.
        • Altman D.G.
        • Gøtzsche P.C.
        • et al.
        The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.
        BMJ. 2011; 343: d5928
        • Hernán M.A.
        • Alonso A.
        • Logan R.
        • et al.
        Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease.
        Epidemiology. 2008; 19: 766-779
        • GRADE Working Group
        Grading quality of evidence and strength of recommendations.
        BMJ. 2004; 328: 1490