Advertisement

Causal analyses with target trial emulation for real-world evidence removed large self-inflicted biases: Systematic bias assessment of ovarian cancer treatment effectiveness

  • Felicitas Kuehne
    Affiliations
    Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
    Search for articles by this author
  • Marjan Arvandi
    Affiliations
    Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
    Search for articles by this author
  • Lisa M. Hess
    Affiliations
    Eli Lilly and Company, Indianapolis, IN, USA
    Search for articles by this author
  • Douglas E. Faries
    Affiliations
    Eli Lilly and Company, Indianapolis, IN, USA
    Search for articles by this author
  • Raffaella Matteucci Gothe
    Affiliations
    Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
    Search for articles by this author
  • Holger Gothe
    Affiliations
    Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria

    Chair of Health Sciences / Public Health, Medical Faculty “Carl Gustav Carus”, Technical University Dresden, Dresden, Germany
    Search for articles by this author
  • Julie Beyrer
    Affiliations
    Eli Lilly and Company, Indianapolis, IN, USA
    Search for articles by this author
  • Alain Gustave Zeimet
    Affiliations
    Department of Obstetrics and Gynecology, Innsbruck Medical University, Innsbruck, Austria
    Search for articles by this author
  • Igor Stojkov
    Affiliations
    Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
    Search for articles by this author
  • Nikolai Mühlberger
    Affiliations
    Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria
    Search for articles by this author
  • Willi Oberaigner
    Affiliations
    Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria

    Institute for Clinical Epidemiology, Cancer Registry Tyrol, Tirol Kliniken, Innsbruck, Austria
    Search for articles by this author
  • Christian Marth
    Affiliations
    Department of Obstetrics and Gynecology, Innsbruck Medical University, Innsbruck, Austria
    Search for articles by this author
  • Uwe Siebert
    Correspondence
    Corresponding author: Uwe Siebert, MD, MPH, MSc; ScD Adjunct Professor of Epidemiology and Health Policy & Management , Department of Epidemiology, Harvard T.H. Chan School of Public Health, 677 Huntington Ave, Boston, MA 02115, USA. Professor of Public Health, Medical Decision Making and Health Technology Assessment, Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Eduard-Wallnoefer-Zentrum 1, A-6060 Hall i.T., Austria. Phone: +43 (0)50 8648 3930, Fax: +43 (0)50 678648
    Affiliations
    Institute of Public Health, Medical Decision Making and Health Technology Assessment, Department of Public Health, Health Services Research and Health Technology Assessment, UMIT TIROL – University for Health Sciences, Medical Informatics and Technology, Hall i.T., Austria

    Center for Health Decision Science and Departments of Epidemiology and Health Policy & Management, Harvard T.H. Chan School of Public Health, Boston, MA, USA

    Institute for Technology Assessment and Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
    Search for articles by this author
Open AccessPublished:October 14, 2022DOI:https://doi.org/10.1016/j.jclinepi.2022.10.005
      This paper is only available as a PDF. To read, Please Download here.

      Highlights

      • -
        To assess potential biases in real-world evidence (RWE), this paper compares the HR of the reference trial to the estimated hazard ratios (HRs) following different analytic approaches.
      • -
        Biases resulting from the different analytic approaches varied in size and direction, ranging from 75% underestimating the HR to 36% overestimating the HR.
      • -
        The full causal analysis (including the target trial emulation using a marginal-structural-Cox-model) yielded the smallest bias, overestimating the HR by 10%.
      • -
        In RWE, a thorough causal-inference-based design and analysis is important.

      Abstract

      Background

      Drawing causal conclusions from real-world data (RWD) poses methodological challenges and risk of bias. We aimed to systematically assess the type and impact of potential biases that may occur when analyzing RWD using the case of progressive ovarian cancer.

      Methods

      We retrospectively compared overall survival with and without second-line chemotherapy using electronic medical records. Potential biases were determined using directed acyclic graphs. We followed a stepwise analytic approach ranging from crude analysis and multivariable-adjusted Cox model up to a full causal analysis using a marginal-structural-Cox-model (MSCM) with replicates emulating a reference randomized controlled trial. To assess biases, we compared effect estimates (hazard ratios [HRs]) of each approach to the HR of the reference trial.

      Results

      The reference trial showed a HR for second-line versus delayed-therapy of 1.01 (95% confidence interval [95%CI]: 0.82-1.25). The corresponding HRs from the RWD analysis ranged from 0.51 for simple baseline adjustments to 1.41 (95%CI 1.22-1.64) accounting for immortal time bias with time-varying covariates. Causal trial emulation yielded a HR of 1.12 (95%CI: 0.96-1.28).

      Conclusions

      Our study, using ovarian cancer as an example, shows the importance of a thorough causal design and analysis if one is expecting RWD to emulate clinical trial results.

      Keywords