Advertisement

Single pivotal trials with few corroborating characteristics were used for FDA approval of cancer therapies

  • Aviv Ladanie
    Affiliations
    Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland

    Swiss Tropical and Public Health Institute (Swiss TPH), Basel, Switzerland
    Search for articles by this author
  • Benjamin Speich
    Affiliations
    Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland

    Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, UK
    Search for articles by this author
  • Matthias Briel
    Affiliations
    Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland

    Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada
    Search for articles by this author
  • Francesco Sclafani
    Affiliations
    Gastrointestinal Unit, Department of Medical Oncology, Institut Jules Bordet, Université Libre de Bruxelles (ULB), Brussels, Belgium
    Search for articles by this author
  • Heiner C. Bucher
    Affiliations
    Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland
    Search for articles by this author
  • Arnav Agarwal
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Canada

    Department of Medicine, University of Toronto, Toronto, Ontario, Canada
    Search for articles by this author
  • John P.A. Ioannidis
    Affiliations
    Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA

    Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA

    Department of Health Research and Policy, Stanford University School of Medicine, Stanford, CA, USA

    Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, CA USA

    Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA, USA
    Search for articles by this author
  • Tiago V. Pereira
    Affiliations
    Health Technology Assessment Unit, Institute of Education and Health Sciences, Oswaldo Cruz German Hospital, São Paulo, Brazil
    Search for articles by this author
  • Benjamin Kasenda
    Affiliations
    Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland

    Department of Medical Oncology, University Hospital and University of Basel, Basel, Switzerland

    Department of Haematology/Oncology and Palliative Care, Klinikum Stuttgart, Stuttgart, Germany
    Search for articles by this author
  • Lars G. Hemkens
    Correspondence
    Corresponding author: Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland. Tel.: +41-(0)61-55-65100; fax: +41-(0)61-26-53109.
    Affiliations
    Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University Hospital and University of Basel, Basel, Switzerland
    Search for articles by this author

      Abstract

      Background and objective

      Novel cancer therapies are often approved with evidence from a single pivotal trial alone. There are concerns about the credibility of this evidence. Higher validity may be indicated by five methodological and statistical characteristics of pivotal trial evidence that were described by the U.S. Food and Drug Administration (FDA), which may corroborate the reliance on a single trial alone for approval decisions.

      Study design

      We did a metaepidemiologic evaluation of all single pivotal trials supporting FDA approval of novel drugs and therapeutic biologicals for cancers between 2000 and 2016. For each trial, we determined the presence of these five characteristics, which we operationalized as (1) large and multicenter trial (≥200 patients; more than one center); consistent treatment benefits across (2) multiple patient subgroups (in view of FDA reviewers), (3) multiple endpoints (including overall survival, progression-free survival, response rate, health related quality of life), and (4) multiple treatment comparisons (e.g., multi-arm studies); and (5) “statistically very persuasive” results (P-values <0.00125).

      Results

      Thirty-five of 100 approvals were based on evidence from a single pivotal trial without any further supporting evidence on beneficial effects (20 randomized controlled trials and 15 single-arm trials). The number increased substantially from one approval before 2006 to 23 after 2011. Sixty-six percent (23/35) of the trials were large multicenter trials (median 301 patients and 63 centers). Consistent effects were demonstrated across subgroups in 66% (23/35), across endpoints in 43% (15/35), and across multiple comparisons in 3% (1/35). Very low P-values for the primary endpoint were seen in 34% (12/35). At least one of the corroborating characteristics was present in 94% (33/35) of all approvals, two or more were present in 54% (19/35), and none had all characteristics.

      Conclusions

      Single pivotal trials typically have some of the corroborating characteristics, but often only one or two. These characteristics need to be better operationalized, defined, and reported and whether single trials with such characteristics provide similar evidence about benefits and harms of novel treatments as multiple trials would do needs to be shown.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Krist A.H.
        "Needs more research"-Implications of the Proteus effect for researchers and evidence adopters.
        Mayo Clin Proc. 2018; 93: 273-275
        • Naci H.
        • Wouters O.J.
        • Gupta R.
        • Ioannidis J.P.A.
        Timing and characteristics of cumulative evidence available on novel therapeutic agents receiving Food and Drug Administration accelerated approval.
        Milbank Q. 2017; 95: 261-290
        • Mullins C.D.
        • Montgomery R.
        • Tunis S.
        Uncertainty in assessing value of oncology treatments.
        Oncologist. 2010; 15: 58-64
      1. Kefauver-Harris Amendments of 1962. Amendment to section 505(d) of the FD&C Act (21 USC 355(d)).

      2. Section 115 of the FDA modernization act of 1997. Amendment to section 505(d) of the FD&C act (21 USC 355(d)).

        • US Food and Drug Administration
        Guidance for industry: providing clinical evidence of effectiveness for human drug and biological products.
        • Ladanie A.
        • Speich B.
        • Naudet F.
        • Agarwal A.
        • Pereira T.V.
        • Sclafani F.
        • et al.
        The Comparative Effectiveness of Innovative Treatments for Cancer (CEIT-Cancer) project: rationale and design of the database and the collection of evidence available at approval of novel drugs.
        Trials. 2018; 19: 505
        • Ladanie A.
        • Ewald H.
        • Kasenda B.
        • Hemkens L.G.
        How to use FDA drug approval documents for evidence syntheses.
        BMJ. 2018; 362: k2815
      3. US Food and Drug Administration. [email protected]: FDA approved drug products. https://www.fda.gov/drugsatfda.

        • US Food and Drug Administration
        New Drugs at FDA: CDER’s new molecular entities and new therapeutic biological products.
        • US Food and Drug Administration
        CDER fast track products Approved Since 1998 through June 1, 2010.
        • Downing N.S.
        • Aminawung J.A.
        • Shah N.D.
        • Krumholz H.M.
        • Ross J.S.
        Clinical trial evidence supporting FDA approval of novel therapeutic agents, 2005-2012.
        JAMA. 2014; 311: 368-377
        • Sridhara R.
        • Johnson J.R.
        • Justice R.
        • Keegan P.
        • Chakravarty A.
        • Pazdur R.
        Review of oncology and hematology drug product approvals at the US Food and Drug Administration between july 2005 and December 2007.
        J Natl Cancer Inst. 2010; 102: 230-243
        • Martell R.E.
        • Sermer D.
        • Getz K.
        • Kaitin K.I.
        Oncology drug development and approval of systemic anticancer therapy by the U.S. Food and Drug Administration.
        Oncologist. 2013; 18: 104-111
        • Morant A.V.
        • Vestergaard H.T.
        European marketing authorizations granted based on a single pivotal clinical trial: the rule or the exception?.
        Clin Pharmacol Ther. 2017; 104: 169-177
        • Gentry L.
        One and done: are single pivotal studies the new norm in cancer therapeutics?.
        • Tibau A.
        • Molto C.
        • Ocana A.
        • Templeton A.J.
        • Del Carpio L.P.
        • Del Paggio J.C.
        • et al.
        Magnitude of clinical benefit of cancer drugs approved by the US Food and Drug Administration.
        J Natl Cancer Inst. 2017; 110: 486-492
        • Ioannidis J.P.
        • Cappelleri J.C.
        • Lau J.
        Issues in comparisons between meta-analyses and large trials.
        JAMA. 1998; 279: 1089-1093
        • Cheng J.
        • Zhang H.
        • Tang S.
        • Sridhara R.
        Inference based on small randomized oncology clinical trials: is the observed treatment effect true?.
        Int J Clin Biostat Biom. 2017; 3: 010
        • Deng C.Q.
        Significant level of 0.00125.
        • Fisher L.
        One large, well-designed, multicenter study as an alternative to the usual FDA paradigm.
        Drug Inf J. 1999; 33: 265-271
        • Senn S.
        Statistical issues in drug development. Chapter 12.2.8.: The two-trials rule.
        2nd ed. John Wiley, Chichester2007
        • Shun Z.
        • Chi E.
        • Durrleman S.
        • Fisher L.
        Statistical consideration of the strategy for demonstrating clinical evidence of effectiveness-one larger vs two smaller pivotal studies.
        Stat Med. 2005; 24: 1619-1637
        • Temple R.
        How FDA currently makes decisions on clinical studies.
        Clin Trials. 2005; 2: 276-281
        • Sivakumar H.
        • Peyton P.J.
        Poor agreement in significant findings between meta-analyses and subsequent large randomized trials in perioperative medicine.
        Br J Anaesth. 2016; 117: 431-441
        • Kjaergard L.L.
        • Villumsen J.
        • Gluud C.
        Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses.
        Ann Intern Med. 2001; 135: 982-989
        • LeLorier J.
        • Gregoire G.
        • Benhaddad A.
        • Lapierre J.
        • Derderian F.
        Discrepancies between meta-analyses and subsequent large randomized, controlled trials.
        N Engl J Med. 1997; 337: 536-542
        • Jones C.W.
        • Handler L.
        • Crowell K.E.
        • Keil L.G.
        • Weaver M.A.
        • Platts-Mills T.F.
        Non-publication of large randomized clinical trials: cross sectional analysis.
        BMJ. 2013; 347: f6104
        • Myles P.S.
        Why we need large randomized studies in anaesthesia.
        Br J Anaesth. 1999; 83: 833-834
        • Dechartres A.
        • Boutron I.
        • Trinquart L.
        • Charles P.
        • Ravaud P.
        Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study.
        Ann Intern Med. 2011; 155: 39-51
        • Dechartres A.
        • Trinquart L.
        • Boutron I.
        • Ravaud P.
        Influence of trial sample size on treatment effect estimates: meta-epidemiological study.
        BMJ. 2013; 346: f2304
        • Ioannidis J.P.
        Why most discovered true associations are inflated.
        Epidemiology. 2008; 19: 640-648
        • Pereira T.V.
        • Horwitz R.I.
        • Ioannidis J.P.
        Empirical evaluation of very large treatment effects of medical interventions.
        JAMA. 2012; 308: 1676-1684
        • Sterne J.A.
        • Sutton A.J.
        • Ioannidis J.P.
        • Terrin N.
        • Jones D.R.
        • Lau J.
        • et al.
        Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials.
        BMJ. 2011; 343: d4002
        • Turner R.M.
        • Bird S.M.
        • Higgins J.P.
        The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews.
        PLoS One. 2013; 8: e59202
        • Gluud L.L.
        • Thorlund K.
        • Gluud C.
        • Woods L.
        • Harris R.
        • Sterne J.A.
        Correction: reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses.
        Ann Intern Med. 2008; 149: 219
        • Brookes S.T.
        • Whitely E.
        • Egger M.
        • Smith G.D.
        • Mulheran P.A.
        • Peters T.J.
        Subgroup analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for the interaction test.
        J Clin Epidemiol. 2004; 57: 229-236
        • Sun X.
        • Briel M.
        • Busse J.W.
        • You J.J.
        • Akl E.A.
        • Mejza F.
        • et al.
        Credibility of claims of subgroup effects in randomised controlled trials: systematic review.
        BMJ. 2012; 344: e1553
        • Wallach J.D.
        • Sullivan P.G.
        • Trepanowski J.F.
        • Steyerberg E.W.
        • Ioannidis J.P.
        Sex based subgroup differences in randomized controlled trials: empirical evidence from Cochrane meta-analyses.
        BMJ. 2016; 355: i5826
        • Alahdab F.
        • Farah W.
        • Almasri J.
        • Barrionuevo P.
        • Zaiem F.
        • Benkhadra R.
        • et al.
        Treatment effect in earlier trials of patients with chronic medical conditions: a meta-epidemiologic study.
        Mayo Clin Proc. 2018; 93: 278-283
        • Lord E.M.
        • Weir I.R.
        • Trinquart L.
        Design analysis indicates potential overestimation of treatment effects in randomized controlled trials supporting Food and Drug Administration cancer drug approvals.
        J Clin Epidemiol. 2018; 103: 1-9
        • Senn S.
        Statistical issues in drug development. Chapter 12.2.7: Should the two-sided p-value always be twice the one-sided value?.
        2nd ed. John Wiley, Chichester2007