Advertisement
Review Article| Volume 61, ISSUE 3, P241-246, March 2008

Download started.

Ok

Early stopping of randomized clinical trials for overt efficacy is problematic

  • Dirk Bassler
    Affiliations
    Department of Neonatology, University Children's Hospital, Tübingen, Germany
    Search for articles by this author
  • Victor M. Montori
    Affiliations
    Knowledge and Encounter Research Unit, Division of Endocrinology and Internal Medicine, Mayo Clinic College of Medicine, Rochester, MN, USA
    Search for articles by this author
  • Matthias Briel
    Affiliations
    CLARITY Research Group, Department of Clinical Epidemiology and Biostatistics, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada

    Basel Institute for Clinical Epidemiology, University Hospital, Basel, Switzerland
    Search for articles by this author
  • Paul Glasziou
    Affiliations
    Centre for Evidence-Based Medicine, Department of Primary Health Care, University of Oxford, Oxford, UK
    Search for articles by this author
  • Gordon Guyatt
    Correspondence
    Corresponding author. Department of Clinical Epidemiology and Biostatistics, Health Sciences Centre, Room 2C12, McMaster University, Hamilton, Ontario, Canada. Tel.: +905-525-9140 ext. 22900; fax: +905-524-3841.
    Affiliations
    CLARITY Research Group, Department of Clinical Epidemiology and Biostatistics, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
    Search for articles by this author

      Abstract

      Objective

      To illustrate controversial issues associated with stopping randomized controlled trials (RCTs) early for apparent benefit.

      Study Design and Setting

      The article presents our review of prior relevant work and our research group's reflections on early stopping.

      Results

      Compelling evidence suggests that trials stopped early for benefit systematically overestimate treatment effects, sometimes by a large amount. Unresolved controversies in trials stopped early for benefit include ethical and statistical problems in the interpretation of results.

      Conclusions

      The best strategy to minimize the problems associated with early stopping of RCTs for benefit is not to stop early. As an alternative, we suggest a threefold approach: a low P-value as the threshold for stopping at the time of interim analyses, not to look before a sufficiently large number of events has accrued and continuation of enrollment and follow-up for a further period.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Peltoniemi O.
        • Kari M.A.
        • Heinonen K.
        • Saarela T.
        • Nikolajev K.
        • Andersson S.
        • et al.
        Pretreatment cortisol values may predict responses to hydrocortisone administration for the prevention of bronchopulmonary dysplasia in high-risk infants.
        J Pediatr. 2005; 146: 632-637
        • Thistle P.
        • Spitzer R.F.
        • Glazier R.H.
        • Pilon R.
        • Arbess G.
        • Simor A.
        • et al.
        A randomized, double-blind, placebo-controlled trial of combined nevirapine and zidovudine compared with nevirapine alone in the prevention of perinatal transmission of HIV in Zimbabwe.
        Clin Infect Dis. 2007; 44 (Epub 2006 Nov 22): 111-119
        • Laupacis A.
        • Connolly S.J.
        • Gent M.
        • Roberts R.S.
        • Cairns J.
        • Joyner C.
        How should results from completed studies influence ongoing clinical trials? The CAFA study experience.
        Ann Intern Med. 1991; 115: 818-822
        • Smith M.R.
        • Manola J.
        • Kaufman D.S.
        • Oh W.K.
        • Bubley G.J.
        • Kantoff P.W.
        Celecoxib versus placebo for men with prostate cancer and a rising serum prostate-specific antigen after radical prostatectomy and/or radiation therapy.
        J Clin Oncol. 2006; 24: 2691-2693
        • Papanikolaou E.G.
        • Camus M.
        • Kolibianakis E.M.
        • Van Landuyt L.
        • Van Steirteghem A.
        • Devroey P.
        In vitro fertilization with single blastocyst-stage versus single cleavage-stage embryos.
        N Engl J Med. 2006; 354: 1190-1193
      1. Draft guidance for clinical trial sponsors on the establishment and operation of clinical trial data monitoring committees. 66 Federal Register 58151–58153 (2001).

        • Sydes M.R.
        • Altman D.G.
        • Babiker A.B.
        • Parmar M.K.
        • Spiegelhalter D.J.
        Reported use of data monitoring committees in the main published reports of randomized controlled trials: a cross-sectional study.
        Clin Trials. 2004; 1: 48-59
        • DAMOCLES Study Group
        NHS health technology assessment programme: a proposed charter for clinical trial data monitoring committees: helping them to do their job well.
        Lancet. 2005; 365: 711-722
        • Slutsky A.S.
        • Lavery J.V.
        Data safety and monitoring boards.
        N Engl J Med. 2004; 350: 1143-1147
        • Freedman B.
        Equipoise and the ethics of clinical research.
        N Engl J Med. 1987; 317: 141-145
        • Berry D.A.
        Bayesian clinical trials.
        Nat Rev Drug Discov. 2006; 5: 27-36
        • Montori V.M.
        • Devereaux P.J.
        • Adhikari N.K.
        • Burns K.E.A.
        • Eggert C.H.
        • Briel M.
        • et al.
        Randomized trials stopped early for benefit: a systematic review.
        JAMA. 2005; 294: 2203-2209
        • Moher D.
        • Schulz K.F.
        • Altman D.G.
        The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials.
        Lancet. 2001; 357: 1191-1194
        • Bassler D.
        • Ferreira-Gonzalez I.
        • Briel M.
        • Cook D.J.
        • Devereaux P.J.
        • Heels-Ansdell D.
        • et al.
        Systematic reviewers neglect bias that results from trials stopped early for benefit.
        J Clin Epidemiol. 2007; 60: 869-873
        • Moher D.
        • Cook D.J.
        • Eastwood S.
        • Olkin I.
        • Rennie D.
        • Stroup D.F.
        Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses.
        Lancet. 1999; 354: 1896-1900
        • O'Brien P.C.
        • Fleming T.R.
        A multiple testing procedure for clinical trials.
        Biometrics. 1979; 35: 549-556
        • Peto R.
        • Pike M.C.
        • Armitage P.
        • Breslow N.E.
        • Cox D.R.
        • Howard S.V.
        • et al.
        Design and analysis of randomized clinical trials requiring prolonged observations of each patient. I. Introduction and design.
        Br J Cancer. 1976; 34: 585-612
        • Haybittle J.L.
        Repeated assessment of results in clinical trials of cancer treatment.
        Br J Radiol. 1971; 44: 793-797
        • Lan K.K.G.
        • DeMets D.L.
        Discrete sequential boundaries for clinical trials.
        Biometrika. 1983; 70: 659-663
        • Schulz K.F.
        • Grimes D.A.
        Multiplicity in randomised trials. II: Subgroup and interim analyses.
        Lancet. 2005; 365: 1657-1661
        • Pocock S.
        • White I.
        Trials stopped early: too good to be true?.
        Lancet. 1999; 353: 943-944
        • Hughes M.D.
        • Pocock S.J.
        Stopping rules and estimation problems in clinical trials.
        Stat Med. 1988; 7: 1231-1242
        • Pocock S.J.
        • Hughes M.D.
        Practical problems in interim analyses, with particular regard to estimation.
        Control Clin Trials. 1989; 10: 209S-221S
        • Yusuf S.
        • Collins R.
        • Peto R.
        Why do we need some large, simple, randomized trials?.
        Stat Med. 1984; 3: 409-420
        • Ioannidis J.
        • Lau J.
        Evolution of treatment effects over time: empirical insight from recursive cumulative metaanalyses.
        Proc Natl Acad Sci USA. 2001; 98: 831-836
        • Trikalinos T.A.
        • Churchill R.
        • Ferri M.
        • Leucht S.
        • Tuunainen A.
        • Wahlbeck K.
        • et al.
        • EU-PSI project
        Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time.
        J Clin Epidemiol. 2004; 57: 1124-1130
        • Mueller P.S.
        • Montori V.M.
        • Bassler D.
        • Koenig B.A.
        • Guyatt G.H.
        Ethical issues in stopping randomized trials early because of apparent benefit.
        Ann Intern Med. 2007; 146: 878-881
        • Emanuel E.J.
        • Wendler D.
        • Grady C.
        What makes clinical research ethical?.
        JAMA. 2000; 283: 2701-2711
        • Poldermans D.
        • Boersma E.
        • Bax J.J.
        • Thomson I.R.
        • van de Ven L.L.
        • Blankensteijn J.D.
        • et al.
        The effect of bisoprolol on perioperative mortality and myocardial infarction in high-risk patients undergoing vascular surgery. Dutch Echocardiographic Cardiac Risk Evaluation Applying Stress Echocardiography Study Group.
        N Engl J Med. 1999; 341: 1789-1794
        • Devereaux P.J.
        • Yusuf S.
        • Yang H.
        • Choi P.T.
        • Guyatt G.H.
        Are the recommendations to use perioperative beta-blocker therapy in patients undergoing noncardiac surgery based on reliable evidence?.
        CMAJ. 2004; 171: 245-247
      2. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group.
        Lancet. 1988; 2: 349-360
        • Auvert B.
        • Taljaard D.
        • Lagarde E.
        • Sobngwi-Tambekou J.
        • Sitta R.
        • Puren A.
        Randomized, controlled intervention trial of male circumcision for reduction of HIV infection risk: the ANRS 1265 trial.
        PLoS Med. 2005; (Epub 2005 Oct 25): e298
        • Bailey R.C.
        • Moses S.
        • Parker C.B.
        • Agot K.
        • Maclean I.
        • Krieger J.N.
        • et al.
        Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomised controlled trial.
        Lancet. 2007; 369: 643-656
        • Gray R.H.
        • Kigozi G.
        • Serwadda D.
        • Makumbi F.
        • Watya S.
        • Nalugoda F.
        • et al.
        Male circumcision for HIV prevention in men in Rakai, Uganda: a randomised trial.
        Lancet. 2007; 369: 657-666
        • Pocock S.J.
        Current controversies in data monitoring for clinical trials.
        Clin Trials. 2006; 3: 513-521