Original Article| Volume 114, P22-29, October 2019

Download started.


Delphi consensus reached to produce a decision tool for SelecTing Approaches for Rapid Reviews (STARR)



      There are many rapid review methods; however, there is little pragmatic guidance on which methods to select. This study aimed to reach consensus among international rapid review experts outlining areas to consider when selecting approaches for rapid reviews.

      Study Design and Setting

      A two-round modified online Delphi survey was conducted between May and July 2018. Participants were asked to rank the importance of a predefined list of 19 items. A consensus definition of at least 70% agreement for each item was decided a priori.


      Thirty experts from ten countries participated in round 1 and 24 in round 2. During round 1, consensus was reached on all items. One additional item on quality assessment was suggested by respondents and comments suggested wording changes to improve clarity and understanding of the tool. Respondents in the second round indicated a high level of importance and all 20 items achieved consensus. These items addressed interaction with commissioners, scoping and searching the evidence-base, data extraction and synthesis methods, and reporting of rapid review methods.


      International consensus was reached to produce the SelecTing Approaches for Rapid Reviews (STARR) decision tool for planning rapid reviews and will lead to improved shared understanding between review teams and review commissioners.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Ganann R.
        • Ciliska D.
        • Thomas H.
        Expediting systematic reviews: methods and implications of rapid reviews.
        Implement Sci.  2010; 5: 56
        • Tsertsvadze A.
        • Chen Y.F.
        • Moher D.
        • Sutcliffe P.
        • McCarthy N.
        How to conduct systematic reviews more expeditiously?.
        Syst Rev. 2015; 4: 160
        • Hartling L.
        • Guise J.M.
        • Kato E.
        • Anderson J.
        • Belinson S.
        • Berliner E.
        • et al.
        A taxonomy of rapid reviews links report types and methods to specific decision-making contexts.
        J Clin Epidemiol. 2015; 68: 1451-14562.e3
      1. Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011 (Available at)
        Date accessed: March 4, 2019
        • Polisena J.
        • Garritty C.
        • Kamel C.
        • Stevens A.
        • Abou-Setta A.M.
        Rapid review programs to support health care and policy decision making: a descriptive analysis of processes and methods.
        Syst Rev. 2015; 4: 26
        • Tricco A.C.
        • Antony J.
        • Zarin W.
        • Strifler L.
        • Ghassemi M.
        • Ivory J.
        • et al.
        A scoping review of rapid review methods.
        BMC Med. 2015; 13: 224
        • Pluddemann A.
        • Aronson J.K.
        • Onakpoya I.
        • Heneghan C.
        • Mahtani K.R.
        Redefining rapid reviews: a flexible framework for restricted systematic reviews.
        BMJ Evid Based Med. 2018; 23: 201-203
        • Tricco A.C.
        • Zarin W.
        • Antony J.
        • Hutton B.
        • Moher D.
        • Sherifali D.
        • et al.
        An international survey and modified Delphi approach revealed numerous rapid review methods.
        J Clin Epidemiol. 2016; 70: 61-67
        • Reynen E.
        • Robson R.
        • Ivory J.
        • Hwee J.
        • Straus S.E.
        • Pham B.
        • et al.
        A retrospective comparison of systematic reviews with same-topic rapid reviews.
        J Clin Epidemiol. 2018; 96: 23-34
        • Taylor-Phillips S.
        • Geppert J.
        • Stinton C.
        • Freeman K.
        • Johnson S.
        • Fraser H.
        • et al.
        Comparison of a full systematic review versus rapid review approaches to assess a newborn screening test for tyrosinemia type 1.
        Res Synth Methods. 2017; 8: 475-484
        • Martyn-St James M.
        • Cooper K.
        • Kaltenthaler E.
        Methods for a rapid systematic review and metaanalysis in evaluating selective serotonin reuptake inhibitors for premature ejaculation.
        Evid Policy. 2017; 13: 517-538
        • Kaltenthaler E.
        • Cooper K.
        • Pandor A.
        • Martyn-St James M.
        • Chatters R.
        • Wong R.
        The use of rapid review methods in health technology assessments: 3 case studies.
        BMC Med Res Methodol. 2016; 16: 108
        • Kaltenthaler E.
        • Cooper K.
        • Martyn-St James M.
        • Pandor A.
        • Wong R.
        From evidence to action - rapid review methods for HTA (Workshop, WS23).
        (Health Technology Assessment International (HTAi) Annual Meeting. Vancouver, Canada)2018 (Available at)
      2. Negro A, Camerlingo M, Maltoni S,Trimaglio F. Challenges of rapid reviews in HTA - case study from an Italian region (Abstract PP097). Health Technology Assessment International (HTAi) Annual Meeting. Rome, Italy; 2017 International Journal of Technology Assessment in Health Care. 2017; 33 (Special issue S1): 117-118.

        • Hsu C.C.
        • Snadford B.A.
        The Delphi technique: making sense of consensus.
        Pract Assess Res Eval. 2007; 12: 1-8
        • Keeney S.
        • Hasson F.
        • McKenna H.
        Consulting the oracle: ten lessons from using the Delphi technique in nursing research.
        J Adv Nurs. 2006; 53: 205-212
        • Asch S.E.
        Studies of independence and conformity: 1. A minority of one against a unanimous majority.
        Psychol Monogr Gen Appl. 1956; 70: 1-70
        • Milgram S.
        Behavioral study of obedience.
        J Abnorm Psychol. 1963; 67: 371-378
        • Akins R.B.
        • Tolson H.
        • Cole B.R.
        Stability of response characteristics of a Delphi panel: application of bootstrap data expansion.
        BMC Med Res Methodol. 2005; 5: 37
        • Hasson F.
        • Keeney S.
        • McKenna H.
        Research guidelines for the Delphi survey technique.
        J Adv Nurs. 2000; 32: 1008-1015
        • Braun V.
        • Clarke V.
        Using thematic analysis in psychology.
        Qual Res Psychol. 2006; 3: 77-101
        • Grant S.
        • Booth M.
        • Khodyakov D.
        Lack of preregistered analysis plans allows unacceptable data mining for and selective reporting of consensus in Delphi studies.
        J Clin Epidemiol. 2018; 99: 96-105
        • Diamond I.R.
        • Grant R.C.
        • Feldman B.M.
        • Pencharz P.B.
        • Ling S.C.
        • Moore A.M.
        • et al.
        Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies.
        J Clin Epidemiol. 2014; 67: 401-409
        • Murphy M.K.
        • Black N.A.
        • Lamping D.L.
        • McKee C.M.
        • Sanderson C.F.
        • Askham J.
        • et al.
        Consensus development methods, and their use in clinical guideline development.
        Health Technol Assess. 1998; 2: 1-88
        • Okoli C.
        • Pawlowski S.D.
        The Delphi method as a research tool: an example, design considerations and applications.
        Inf Manag. 2004; 42: 15-29