Advertisement

GRADE Guidelines: 29. Rating the certainty in time-to-event outcomes—Study limitations due to censoring of participants with missing data in intervention studies

  • Marius Goldkuhle
    Correspondence
    Corresponding author. Department of Internal Medicine Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf University Hospital of Cologne Kerpener Str. 62 50637 Köln, Germany. Tel.: +49 221 478-62032; fax: +49 221 478-96654.
    Affiliations
    Department of Internal Medicine, University of Cologne, Faculty of Medicine and University Hospital Cologne, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Kerpener Str. 62, 50937, Cologne, Germany
    Search for articles by this author
  • Ralf Bender
    Affiliations
    Department of Medical Biometry, Institute for Quality and Efficiency in Health Care, Im Mediapark 8, D-50670 Cologne, Germany
    Search for articles by this author
  • Elie A. Akl
    Affiliations
    Department of Internal Medicine, American University of Beirut, P.O.Box 11-0236, Lebanon, Canada

    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St. W., Hamilton, Ontario, L8S 4K1, Canada
    Search for articles by this author
  • Elvira C. van Dalen
    Affiliations
    Princess Máxima Center for Pediatric Oncology, Heidelberglaan 25, 3584CS, Utrecht, The Netherlands
    Search for articles by this author
  • Sarah Nevitt
    Affiliations
    Department of Biostatistics, University of Liverpool, Block F, Waterhouse Building, 1-5 Brownlow Street, Liverpool, L69 3GL, UK
    Search for articles by this author
  • Reem A. Mustafa
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St. W., Hamilton, Ontario, L8S 4K1, Canada

    Department of Medicine, University of Kansas Health System, 3901 Rainbow Blvd, MS3002, Kansas City, KS, 66160, USA
    Search for articles by this author
  • Gordon H. Guyatt
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St. W., Hamilton, Ontario, L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main St. W., Hamilton, Ontario, L8S 4K1, Canada

    McMaster GRADE Centre & Michael G DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada
    Search for articles by this author
  • Marialene Trivella
    Affiliations
    Centre for Statistics in Medicine, University of Oxford, Botnar Research Centre, Windmill Rd, Oxford, OX3 7LD, UK
    Search for articles by this author
  • Benjamin Djulbegovic
    Affiliations
    City of Hope, 1500 Duarte Rd, Duarte, CA, 91010, USA
    Search for articles by this author
  • Holger Schünemann
    Affiliations
    Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St. W., Hamilton, Ontario, L8S 4K1, Canada

    Department of Medicine, McMaster University, 1280 Main St. W., Hamilton, Ontario, L8S 4K1, Canada

    McMaster GRADE Centre & Michael G DeGroote Cochrane Canada Centre, McMaster University, Hamilton, Ontario, Canada
    Search for articles by this author
  • Michela Cinquini
    Affiliations
    Unit of Systematic Reviews and Guidelines Production, Mario Negri Institute for Pharmacological Research IRCCS, Via Giuseppe La Masa 19, 20156, Milan, Italy
    Search for articles by this author
  • Nina Kreuzberger
    Affiliations
    Department of Internal Medicine, University of Cologne, Faculty of Medicine and University Hospital Cologne, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Kerpener Str. 62, 50937, Cologne, Germany
    Search for articles by this author
  • Nicole Skoetz
    Affiliations
    Department of Internal Medicine, University of Cologne, Faculty of Medicine and University Hospital Cologne, Center for Integrated Oncology Aachen Bonn Cologne Duesseldorf, Kerpener Str. 62, 50937, Cologne, Germany
    Search for articles by this author
  • GRADE Working Group
Published:September 29, 2020DOI:https://doi.org/10.1016/j.jclinepi.2020.09.017

      Abstract

      Objectives

      To provide Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) guidance for the consideration of study limitations (risk of bias) due to missing participant outcome data for time-to-event outcomes in intervention studies.

      Study Design and Setting

      We developed this guidance through an iterative process that included membership consultation, feedback, presentation, and iterative discussion at meetings of the GRADE working group.

      Results

      The GRADE working group has published guidance on how to account for missing participant outcome data in binary and continuous outcomes. When analyzing time-to-event outcomes (e.g., overall survival and time-to-treatment failure) data of participants for whom the outcome of interest (e.g., death and relapse) has not been observed are dealt with through censoring. To do so, standard methods require that censored individuals are representative for those remaining in the study. Two types of censoring can be distinguished, end of study censoring and censoring because of missing data, commonly named loss to follow-up censoring. However, both types are not distinguishable with the usual information on censoring available to review authors. Dealing with individuals for whom data are missing during follow-up in the same way as individuals for whom full follow-up is available at the end of the study increases the risk of bias. Considerable differences in the treatment arms in the distribution of censoring over time (early versus late censoring), the overall degree of missing follow-up data, and the reasons why individuals were lost to follow-up may reduce the certainty in the study results. With often only very limited data available, review and guideline authors are required to make transparent and well-considered judgments when judging risk of bias of individual studies and then come to an overall grading decision for the entire body of evidence.

      Conclusion

      Concern for risk of bias resulting from censoring of participants for whom follow-up data are missing in the underlying studies of a body of evidence can be expressed in the study limitations (risk of bias) domain of the GRADE approach.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Brozek J.
        • Alonso-Coello P.
        • Rind D.
        • et al.
        GRADE guidelines 6. Rating the quality of evidence--imprecision.
        J Clin Epidemiol. 2011; 64: 1283-1293
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines: 8. Rating the quality of evidence--indirectness.
        J Clin Epidemiol. 2011; 64: 1303-1310
        • Guyatt G.H.
        • Oxman A.D.
        • Kunz R.
        • Woodcock J.
        • Brozek J.
        • Helfand M.
        • et al.
        GRADE guidelines: 7. Rating the quality of evidence--inconsistency.
        J Clin Epidemiol. 2011; 64: 1294-1302
        • Guyatt G.H.
        • Oxman A.D.
        • Montori V.
        • Vist G.
        • Kunz R.
        • Brozek J.
        • et al.
        GRADE guidelines: 5. Rating the quality of evidence--publication bias.
        J Clin Epidemiol. 2011; 64: 1277-1282
        • Guyatt G.H.
        • Oxman A.D.
        • Sultan S.
        • Glasziou P.
        • Akl E.A.
        • Alonso-Coello P.
        • et al.
        GRADE guidelines: 9. Rating up the quality of evidence.
        J Clin Epidemiol. 2011; 64: 1311-1316
        • Guyatt G.H.
        • Oxman A.D.
        • Vist G.
        • Kunz R.
        • Brozek J.
        • Alonso-Coello P.
        • et al.
        GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias).
        J Clin Epidemiol. 2011; 64: 407-415
        • Guyatt G.H.
        • Ebrahim S.
        • Alonso-Coello P.
        • Johnston B.C.
        • Mathioudakis A.G.
        • Briel M.
        • et al.
        GRADE guidelines 17: assessing the risk of bias associated with missing participant outcome data in a body of evidence.
        J Clin Epidemiol. 2017; 87: 14-22
        • Kahale L.A.
        • Diab B.
        • Brignardello-Petersen R.
        • Agarwal A.
        • Mustafa R.A.
        • Kwong J.
        • et al.
        Systematic reviews do not adequately report or address missing outcome data in their analyses: a methodological survey.
        J Clin Epidemiol. 2018; 99: 14-23
        • Kaplan E.L.
        • Meier P.
        Nonparametric estimation from incomplete observations.
        J Am Stat Assoc. 1958; 53: 457-481
        • Cox D.R.
        Regression models and life-tables.
        J R Stat Soc Ser B Methodol. 1972; 34: 187-220
        • Tierney J.F.
        • Stewart L.A.
        • Ghersi D.
        • Burdett S.
        • Sydes M.R.
        Practical methods for incorporating summary time-to-event data into meta-analysis.
        Trials. 2007; 8: 16
        • Leung K.-M.
        • Elashoff R.M.
        • Afifi A.A.
        Censoring issues IN survival analysis.
        Annu Rev Public Health. 1997; 18: 83-104
        • Kleinbaum D.G.
        • Klein M.
        Survival Analysis.
        3 ed. Springer-Verlag, New York2012
        • Putter H.
        • Fiocco M.
        • Geskus R.B.
        Tutorial in biostatistics: competing risks and multi-state models.
        Stat Med. 2007; 26: 2389-2430
        • Lagakos S.W.
        General right censoring and its impact on the analysis of survival data.
        Biometrics. 1979; 35: 139-156
        • Batson S.
        • Greenall G.
        • Hudson P.
        Review of the reporting of survival analyses within randomised controlled trials and the implications for meta-analysis.
        PLoS One. 2016; 11: e0154870
        • Abraira V.
        • Muriel A.
        • Emparanza J.I.
        • Pijoan J.I.
        • Royuela A.
        • Plana M.N.
        • et al.
        Reporting quality of survival analyses in medical journals still needs improvement. A minimal requirements proposal.
        J Clin Epidemiol. 2013; 66: 1340-1346.e5
        • Altman D.G.
        • De Stavola B.L.
        • Love S.B.
        • Stepniewska K.A.
        Review of survival analyses published in cancer journals.
        Br J Cancer. 1995; 72: 511-518
        • Mathoulin-Pelissier S.
        • Gourgou-Bourgade S.
        • Bonnetain F.
        • Kramar A.
        Survival end point reporting in randomized cancer clinical trials: a review of major journals.
        J Clin Oncol. 2008; 26: 3721-3726
        • Vervölgyi E.
        • Kromp M.
        • Skipka G.
        • Bender R.
        • Kaiser T.
        Reporting of loss to follow-up information in randomised controlled trials with time-to-event outcomes: a literature survey.
        BMC Med Res Methodol. 2011; 11: 130
        • Altman D.G.
        Practical Statistics for Medical Research.
        Chapman & Hall, London1999 (ISBN 0-412-27630-5)
        • Pocock S.J.
        • Clayton T.C.
        • Altman D.G.
        Survival plots of time-to-event outcomes in clinical trials: good practice and pitfalls.
        Lancet. 2002; 359: 1686-1689
        • Higgins J.P.T.
        • Sterne J.A.C.
        • Savović J.
        • Page M.J.
        • Hróbjartsson A.
        • Boutron I.
        • et al.
        A revised tool for assessing risk of bias in randomized trials2019.
        (Available at)
        • Sterne J.
        • Savović J.
        • Page M.
        • Elbers R.
        • Blencowe N.
        • Boutron I.
        • et al.
        RoB 2: a revised tool for assessing risk of bias in randomised trials.
        BMJ. 2019; 366: l4898
        • Cochrane Community
        Glossary: The Cochrane Colloaboration.
        (Available at)
        https://community.cochrane.org/glossary
        Date: 2019
        Date accessed: August 6, 2019
      1. Kahale LA, Guyatt GH, Agoritsas T, Briel M, Busse JW, Carrasco-Labra A, et al. A guidance was developed to identify participants with missing outcome data in randomized controlled trials. J Clin Epidemiol.

        • Montori V.M.
        • Guyatt G.H.
        Intention-to-treat principle.
        CMAJ. 2001; 165: 1339-1341
        • Brinkhof M.W.G.
        • Pujades-Rodriguez M.
        • Egger M.
        Mortality of patients lost to follow-up in antiretroviral treatment programmes in resource-limited settings: systematic review and meta-analysis.
        PLOS ONE. 2009; 4: e5790
        • Brinkhof M.W.G.
        • Spycher B.D.
        • Yiannoutsos C.
        • Weigel R.
        • Wood R.
        • Messou E.
        • et al.
        Adjusting mortality for loss to follow-up: analysis of five ART programmes in sub-Saharan Africa.
        PLoS One. 2010; 5: e14149
        • Campigotto F.
        • Weller E.
        Impact of informative censoring on the Kaplan-Meier estimate of progression-free survival in phase II clinical trials.
        J Clin Oncol. 2014; 32: 3068-3074
        • Persson I.
        • Khamis H.
        Bias of the Cox model hazard ratio.
        J Mod Appl Stat Methods. 2005; 4: 90-99
        • Denis F.
        • Lethrosne C.
        • Pourel N.
        • Molinier O.
        • Pointreau Y.
        • Domont J.
        • et al.
        Randomized trial comparing a web-mediated follow-up with routine surveillance in Lung cancer patients.
        J Natl Cancer Inst. 2017; 109
        • Guyot P.
        • Ades A.E.
        • Ouwens M.J.N.M.
        • Welton N.J.
        Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan-Meier survival curves.
        BMC Med Res Methodol. 2012; 12: 9
        • Kahale L.A.
        • Diab B.
        • Khamis A.M.
        • Chang Y.
        • Lopes L.C.
        • Agarwal A.
        • et al.
        Potentially missing data are considerably more frequent than definitely missing data: a methodological survey of 638 randomized controlled trials.
        J Clin Epidemiol. 2019; 106: 18-31
        • Parmar M.K.B.
        • Torri V.
        • Stewart L.
        Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints.
        Stat Med. 1998; 17: 2815-2834
        • Martin M.
        • Holmes F.A.
        • Ejlertsen B.
        • Delaloge S.
        • Moy B.
        • Iwata H.
        • et al.
        Neratinib after trastuzumab-based adjuvant therapy in HER2-positive breast cancer (ExteNET): 5-year analysis of a randomised, double-blind, placebo-controlled, phase 3 trial.
        Lancet Oncol. 2017; 18: 1688-1700
        • Lee S.-Y.
        • Wolfe R.A.
        A simple test for independent censoring under the proportional hazards model.
        Biometrics. 1998; 54: 1176-1182
        • Higgins J.P.T.
        • Li T.
        • Deeks J.J.
        Chapter 6: choosing effect measures and computing estimates of effect. Draft version (29 January 2019) for inclusion.
        in: Higgins J.P.T. Thomas J. Chandler J. Cumpston M. Li T. Page M.J. Cochrane Handbook for Systematic Reviews of Interventions. 2019
        • Gooley T.A.
        • Leisenring W.
        • Crowley J.
        • Storer B.E.
        Estimation of failure probabilities in the presence of competing risks: new representations of old estimators.
        Stat Med. 1999; 18: 695-706
        • Hernán M.A.
        • Hernández-Díaz S.
        • Robins J.M.
        A structural approach to selection bias.
        Epidemiology. 2004; 15: 615-625
        • Emoto S.E.
        • Matthews P.C.
        A weibull model for dependent censoring.
        Ann Stat. 1990; 18: 1556-1577
        • Jackson D.
        • White I.R.
        • Seaman S.
        • Evans H.
        • Baisley K.
        • Carpenter J.
        Relaxing the independent censoring assumption in the Cox proportional hazards model using multiple imputation.
        Stat Med. 2014; 33: 4681-4694
        • Faucett C.L.
        • Schenker N.
        • Taylor J.M.
        Survival analysis using auxiliary variables via multiple imputation, with application to AIDS clinical trial data.
        Biometrics. 2002; 58: 37-47
        • Huang X.
        • Wolfe R.A.
        A frailty model for informative censoring.
        Biometrics. 2002; 58: 510-520
        • Kaciroti N.A.
        • Raghunathan T.E.
        • Taylor J.M.
        • Julius S.
        A Bayesian model for time-to-event data with informative censoring.
        Biostatistics (Oxford, England). 2012; 13: 341-354
        • Hsu C.-H.
        • Taylor J.M.G.
        • Murray S.
        • Commenges D.
        Survival analysis using auxiliary variables via non-parametric multiple imputation.
        Stat Med. 2006; 25: 3503-3517
        • Siannis F.
        Applications of a parametric model for informative censoring.
        Biometrics. 2004; 60: 704-714
        • Siannis F.
        Sensitivity analysis for multiple right censoring processes: investigating mortality in psoriatic arthritis.
        Stat Med. 2011; 30: 356-367
        • Siannis F.
        • Copas J.
        • Lu G.
        Sensitivity analysis for informative censoring in parametric survival models.
        Biostatistics (Oxford, England). 2005; 6: 77-91
        • Robins J.M.
        • Finkelstein D.M.
        Correcting for noncompliance and dependent censoring in an AIDS clinical trial with inverse probability of censoring weighted (IPCW) log-rank tests.
        Biometrics. 2000; 56: 779-788
        • Cole S.R.
        • Hernán M.A.
        Constructing inverse probability weights for marginal structural models.
        Am J Epidemiol. 2008; 168: 656-664
        • Tsiatis A.A.
        • Robins J.M.
        Correcting for non-compliance in randomized trials using rank preserving structural failure time models.
        Commun Stat - Theor Methods. 1991; 20: 2609-2631
        • Bender R.
        • Beckmann L.
        • Lange S.
        Biometrical issues in the analysis of adverse events within the benefit assessment of drugs.
        Pharm Stat. 2016; 15: 292-296
        • Allignol A.
        • Beyersmann J.
        • Schmoor C.
        Statistical issues in the analysis of adverse events in time-to-event data.
        Pharm Stat. 2016; 15: 297-305