A retrospective comparison of systematic reviews with same-topic rapid reviews

  • Emily Reynen
    Affiliations
    Department of Medicine, Queen's University, Etherington Hall, Rooms 3032-3043, 94 Stuart Street, Kingston, Ontario K7L 3N6, Canada
    Search for articles by this author
  • Reid Robson
    Affiliations
    Li Ka Shing Knowledge Institute of St Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1 W8, Canada
    Search for articles by this author
  • John Ivory
    Affiliations
    Li Ka Shing Knowledge Institute of St Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1 W8, Canada
    Search for articles by this author
  • Jeremiah Hwee
    Affiliations
    Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, 155 College Street, 6th floor, Toronto, Ontario M5T 3M7, Canada
    Search for articles by this author
  • Sharon E. Straus
    Affiliations
    Li Ka Shing Knowledge Institute of St Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1 W8, Canada

    Department of Geriatric Medicine, University of Toronto, 27 Kings College Circle, Toronto, Ontario M5S 1A1, Canada
    Search for articles by this author
  • Ba’ Pham
    Affiliations
    Li Ka Shing Knowledge Institute of St Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1 W8, Canada
    Search for articles by this author
  • Andrea C. Tricco
    Correspondence
    Corresponding author. Knowledge Translation Program Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, Ontario M5B 1W8, Canada. Tel.: 1-416-864-6060 ext. 77521; fax: 1-416-864-5805.
    Affiliations
    Li Ka Shing Knowledge Institute of St Michael's Hospital, 209 Victoria Street, East Building, Room 716, Toronto, Ontario M5B 1 W8, Canada

    Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, 155 College Street, 6th floor, Toronto, Ontario M5T 3M7, Canada
    Search for articles by this author
Published:December 16, 2017DOI:https://doi.org/10.1016/j.jclinepi.2017.12.001

      Abstract

      Objective

      To compare rapid reviews (RRs) to same-topic systematic reviews (SRs) for methods, studies included, and conclusions.

      Study Design and Setting

      A retrospective comparison of studies comparing RRs and SRs by searching four scoping reviews published between 2007 and 2016. Reports were included if literature searches were conducted within 24 months of each other and had common research questions. Reviews were compared for duration, studies included, population, intervention, comparisons, outcomes, study designs, quality, methods, and conclusions.

      Results

      Six studies containing 16 review pairs were included, covering nine topics. Overall, RRs used abbreviated methods more often: no search of grey literature, employing one reviewer to screen studies, engaging fewer experts, including fewer studies, and providing shorter reports, with poorer reporting quality and faster completion. Reviews reported similar conclusions, with two exceptions: one SR did not include a key study; separately, two RRs failed to highlight an association with early mortality identified by the SR. RRs tended to provide less detail and fewer considerations.

      Conclusion

      RRs used several methodological shortcuts compared with SRs on the same topic. It was challenging to discern methodological differences because of retrospective assessment and substantial nonreporting, particularly for RRs.

      Keywords

      To read this article in full you will need to make a payment

      References

        • Tricco A.C.
        • Cardoso R.
        • Thomas S.M.
        • Motiwala S.
        • Sullivan S.
        • Kealey M.R.
        • et al.
        Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review.
        Implement Sci. 2016; 11: 4
        • Tricco A.C.
        • Antony J.
        • Zarin W.
        • Strifler L.
        • Ghassemi M.
        • Ivory J.
        • et al.
        A scoping review of rapid review methods.
        BMC Med. 2015; 13: 224
        • Petticrew M.R.H.
        Systematic reviews in the social sciences: a practical guide.
        Blackwell Publishing Ltd, Oxford2006
        • Tricco A.C.
        • Zarin W.
        • Rios P.
        • Pham B.
        • Straus S.E.
        • Langlois E.V.
        Barriers, facilitators, strategies and outcomes to engaging policymakers, healthcare managers and policy analysts in knowledge synthesis: a scoping review protocol.
        BMJ Open. 2016; 6: e013929
        • Khangura S.
        • Konnyu K.
        • Cushman R.
        • Grimshaw J.
        • Moher D.
        Evidence summaries: the evolution of a rapid review approach.
        Syst Rev. 2012; 1: 10
        • Hartling L.
        • Guise J.M.
        • Kato E.
        • Anderson M.R.
        • Aronson N.
        • Belinson S.
        EPC Methods: an exploration of methods and context for the production of rapid reviews.
        Agency for Healthcare Research and Quality (US), Rockville2015 (Report No.: 15-EHC008-EF)
        • Cameron A.
        • Watt A.
        • Lathlean T.
        • Sturm T.
        Rapid versus full systematic reviews: an inventory of current methods and practice in Health Technology Assessment.
        Australian Safety and Efficacy Register of New Interventional Procedures – Surgical (ASERNIP-S), Adelaide2007 (Report No.: 60. Available at)
        • Abou-Setta A.M.
        • Jeyaraman M.M.
        • Attia A.
        • Al-Inany H.G.
        • Ferri M.
        • Ansari M.T.
        • et al.
        Methods for Developing Evidence Reviews in Short Periods of Time: A Scoping Review.
        PLoS One. 2016; 11: e0165903
        • Shea B.J.
        • Grimshaw J.M.
        • Wells G.A.
        • Boers M.
        • Andersson N.
        • Hamel C.
        • et al.
        Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews.
        BMC Med Res Methodol. 2007; 7: 10
        • Corabian P.
        • Harstall C.
        Rapid assessments provide acceptable quality advice.
        Annu Meet Int Soc Technol Assess Health Care Int Soc Technol Assess Health Care Meet. 2002; 18 (Abstract 70)
        • Best L.
        • Stevens A.
        • Colin-Jones D.
        Rapid and responsive health technology assessment: the development and evaluation process in the South and West region of England.
        J Clin Effectiveness. 1997; 2: 51-56
        • Rizzo M.
        • Llewellyn A.
        • Martin A.
        A rapid systematic review versus a Cochrane systematic review: an empirical comparison. 19th Cochrane Colloquium, Supplement. (Suppl(CD000003)).
        Spain, Madrid2011: 102
        • Cairns J.
        Providing guidance to the NHS: The Scottish Medicines Consortium and the National Institute for Clinical Excellence compared.
        Health Policy. 2006; 76: 134-143
        • Warren V.
        Health technology appraisal of interventional procedures: comparison of rapid and slow methods.
        J Health Serv Res Policy. 2007; 12: 142-146
        • Saz Parkinson Z.
        • Lopez-Cuadrado T.
        • Plama-Ruíz Bouza C.
        • Amate J.M.
        • Sarria A.
        Preliminary study of rapid vs. exhaustive HTA reports: comparison of their usefulness in the decision-making process [abstract].
        (HTAi 7th Annual Meeting: Maximising the Value of HTA; Dublin)2010: 162-163
        • Van de Velde S.
        • De Buck E.
        • Dieltjens T.
        • Aertgeerts B.
        Medicinal use of potato-derived products: conclusions of a rapid versus full systematic review.
        Phytother Res. 2011; 25: 787-788
        • Peinemann F.
        • McGauran N.
        • Sauerland S.
        • Lange S.
        Disagreement in primary study selection between systematic reviews on negative pressure wound therapy.
        BMC Med Res Methodol. 2008; 8: 41
        • Kaltenthaler E.
        • Tappenden P.
        • Booth A.
        • Akehurst R.
        Comparing methods for full versus single technology appraisal: a case study of docetaxel and paclitaxel for early breast cancer.
        Health Policy. 2008; 87: 389-400
        • Health Technology Inquiry Service (HTIS)
        Transcatheter Aortic Valve Implantation: A Critical Appraisal of a Health Technology Assessment and Comparison with a Rapid Review.
        (CADTH), Canadian Agency for Drugs and Technologies in Health, 2010
        • Lopez T.
        On Archimedes.
        Arch Dis Child. 2003; 88: 176-177
        • Vlachojannis J.E.
        • Cameron M.
        • Chrubasik S.
        Medicinal use of potato-derived products: a systematic review.
        Phytother Res. 2010; 24: 159-162
        • De Buck E.
        • Van de Velde S.
        Towards evidence based emergency medicine: best BETs from the Manchester Royal Infirmary. BET 2: potato peel dressings for burn wounds.
        Emerg Med J. 2010; 27: 55-56
        • Hill R.
        • Bagust A.
        • Bakhai A.
        • Dickson R.
        • Dundar Y.
        • Haycox A.
        • et al.
        Coronary artery stents: a rapid systematic review and economic evaluation.
        Health Technol Assess. 2004; 8: 1-242
        • Brophy J.
        An evaluation of drug eluting (coated) stents for percutaneous coronary interventions; What should their role be at the McGill University Health Centre (MUHC)?: the Technology Assessment Unit (TAU) of the McGill University Health Centre (MUHC).
        (Report No.: 10. Available at)
        https://www.mcgill.ca/tau/files/tau/coated.pdf
        Date: 2003
        Date accessed: January 8, 2018
      1. Institute for Clinical Systems Improvement. Drug-eluting stents for the prevention of restenosis in native coronary arteries. Bloomington, MN: Institute for Clinical Systems Improvement (ICSI); 2003. Available at http://www.crd.york.ac.uk/CRDWeb/ShowRecord.asp?ID=32004000013. Accessed January 8, 2018.

        • Banerjee S.
        • Babidge W.
        • Miller J.
        • Smith J.
        • Noorani H.Z.
        • Cuncins-Hearn A.
        • et al.
        Comparison of lung volume reduction surgery with medical management for emphysema [Technology report no 48].
        Canadian Coordinating Office for Health Technology Assessment, Ottawa2004 (Available at)
      2. Institute for Clinical Systems Improvement. Lung volume reduction surgery for emphysema. Bloomington MN: Institute for Clinical Systems Improvement (ICSI); 2003. Available at http://www.crd.york.ac.uk/CRDWeb/ShowRecord.asp?ID=32003001182. Accessed January 8, 2018.

        • National Institute for Clinical Excellence (NICE)
        Lung volume reduction surgery for advanced emphysema.
        NICE: Information from Interventional Procedure Guidance 114, London2005 (Available at)
        https://www.nice.org.uk/guidance/ipg114
        Date accessed: January 8, 2018
        • Vale L.
        • Wyness L.
        • McCormack K.
        • McKenzie L.
        • Brazzelli M.
        • Stearns S.C.
        A systematic review of the effectiveness and cost-effectiveness of metal-on-metal hip resurfacing arthroplasty for treatment of hip disease.
        Health Technol Assess. 2002; 15: 1-109
        • Bernath V.
        Hip resurfacing in patients with osteoarthritis.
        Centre for Clinical Effectiveness (CCE), Clayton, Victoria2002
        • Alberta Heritage Foundation for Medical Research (AHFMR)
        Metal-on-metal hip resurfacing for young, active adults with degenerative hip disease.
        Alberta Heritage Foundation for Medical Research (AHFMR), Edmonton2002 (Technote TN 33)
        • Middleton P.
        • Duffield M.
        • Lynch S.V.
        • Padbury R.T.
        • House T.
        • Stanton P.
        • et al.
        Living donor liver transplantation–adult donor outcomes: a systematic review.
        2006 (Contract No.: 1. Available at)
        • Alberta Heritage Foundation for Medical Research (AHFMR)
        Adult to adult living donor liver transplantation.
        Alberta Heritage Foundation for Medical Research (AHFMR), Edmonton2004 (Technote TN 45)
        • Costa V.
        • Brophy J.
        • McGregor M.
        Vacuum-Assisted Wound Closure Therapy (V.A.C.®). Technology Assessment Unit (TAU) of the McGill University Health Centre (MUHC).
        (Report No.: 19. Available at)
        http://www.mcgill.ca/tau/files/tau/VAC_REPORT_FINAL.pdf
        Date: 2005
        Date accessed: January 8, 2018
      3. Institute for Quality and Efficiency in Health Care (IQWiG). Negative pressure wound therapy (final report, translated 2006 German version to English). Cologne, Germany: Institute for Quality and Efficiency in Health Care (IQWiG); 2008. Available at https://www.iqwig.de/en/projects-results/projects/non-drug-interventions/n04-03-negative-pressure-wound-therapy.1198.html. Accessed January 8, 2018.

        • Ontario Health Technology Advisory Committee (OHTAC)
        Negative Pressure Wound Therapy. Ontario Health Technology Advisory Committee (OHTAC).
        (Available at)
        • Pham C.
        • Middleton P.F.
        • Maddern G.J.
        The safety and efficacy of topical negative pressure in non-healing wounds: a systematic review.
        J Wound Care. 2006; 15: 240-250
      4. Institute for Quality and Efficiency in Health Care (IQWiG). Negative pressure wound therapy (rapid report). Cologne, Germany: Institute for Quality and Efficiency in Health Care (IQWiG); 2007. Available at https://www.iqwig.de/en/projects-results/projects/non-drug-interventions/n06-02-negative-pressure-wound-therapy-rapid-report.1157.html. Accessed January 8, 2018.

        • Ward S.
        • Simpson E.
        • Davis S.
        • Hind D.
        • Rees A.
        • Wilkinson A.
        Taxanes for the adjuvant treatment of early breast cancer: systematic review and economic evaluation.
        Health Technol Assess. 2007; 11: 1-144
        • Chilcott J.
        • Lloyd Jones M.
        • Wilkinson A.
        Docetaxel for the adjuvant treatment of early node-positive breast cancer: a single technology appraisal.
        Health Technol Assess. 2009; 13: 7-13
        • Griffin S.
        • Dunn G.
        • Palmer S.
        • Macfarlane K.
        • Brent S.
        • Dyker A.
        • et al.
        The use of paclitaxel in the management of early stage breast cancer.
        Health Technol Assess. 2009; 13: 15-22
        • McGregor M.
        • Esfandiari S.
        Transcatheter aortic valve implantation (TAVI) at the MUHC: a health technology assessment.
        Technology Assessment Unit of the McGill University Health Centre, Montreal2009 (Report No.: 45. Available at)
        • Murphy G.
        • Cunningham J.
        Percutaneous heart valve replacement for valvular heart disease: a review of the clinical effectiveness, cost-effectiveness, and guidelines.
        Canadian Agency for Drugs and Technologies in Health (CADTH), Ottawa2010 (Available at)
        • Craig J.
        • Lancaster G.A.
        • Taylor S.
        • Williamson P.R.
        • Smyth R.L.
        Infrared ear thermometry compared with rectal thermometry in children: a systematic review.
        Lancet Infect Dis. 2002; 360: 603-609
        • Riddell A.
        • Eppich W.
        Should tympanic temperature measurement be trusted?.
        Arch Dis Child. 2001; 85: 433-434
        • Page M.J.
        • Shamseer L.
        • Altman D.G.
        • Tetzlaff J.
        • Sampson M.
        • Tricco A.C.
        • et al.
        Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study.
        PLoS Med. 2016; 13: e1002028
      5. Cochrane Handbook for Systematic Reviews of Interventions; 2011.
        (Available at) (Accessed January 8, 2018)
        • Whiting P.
        • Savovic J.
        • Higgins J.P.
        • Caldwell D.M.
        • Reeves B.C.
        • Shea B.
        • et al.
        ROBIS: a new tool to assess risk of bias in systematic reviews was developed.
        J Clin Epidemiol. 2016; 69: 225-234
      6. Tricco A, Clifford TJ, Straus SE. Systematic Prospective Assessment of Rapid Knowledge Synthesis - SPARKS Study. Canadian Research Information System; 2016. Available at http://webapps.cihr-irsc.gc.ca/funding/detail_e?pResearchId=7713308&p_version=CIHR&p_language=E&p_session_id=. Accessed January 8, 2018