Advertisement

Patient reported outcome measures in clinical trials should be initially analyzed as continuous outcomes for statistical significance and responder analyses should be reserved as secondary analyses

Published:February 05, 2021DOI:https://doi.org/10.1016/j.jclinepi.2021.01.026

      Abstract

      Objective

      To evaluate the power of responder analyses in a randomized controlled trial.

      Study design and setting

      Simulations were based on the Chronic Kidney Disease Antidepressant Sertraline Trial (CAST), which compared sertraline to placebo for the treatment of depression in kidney disease. Baseline disease severity, placebo response, effect size, and the proportion of responders were varied across 72 scenarios. Power was assessed using a t-test for change scores, and the chi-square test for dichotomized outcomes of the minimal important difference (MID), improvement and remission in 10,000 datasets with a fixed sample size of 193.

      Results

      The t-test had >80% power except for scenarios with the lowest sertraline effect size. The chi-square test using the MID had <7% power in all scenarios while improvement and remission of achieved >80% power only at higher effect sizes and/or when the proportion of responders was highest at 0.5. The chi-square test for improvement had marginal power increases compared to the t-test (4/72 scenarios = 5.6%) and that for remission did not outperform the t-test in any scenario.

      Conclusions

      The t-test outperforms the chi-square test for dichotomized outcomes regardless of baseline disease severity, placebo response, effect size and the proportion of responders to the intervention.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • MacCallum RC
        • Zhang S
        • Preacher KJ
        • Rucker DD.
        On the practice of dichotomization of quantitative variables.
        Psychol Methods. 2002; 7: 19-40https://doi.org/10.1037/1082-989x.7.1.19
      1. Services USDoHaH. Guidance for industry patient-reported outcome measures: use in medical product development to support labelling claims. 2009. Available at: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/patient-reported-outcome-measures-use-medical-product-development-support-labeling-claims

        • Altman DG
        • Royston P.
        The cost of dichotomising continuous variables.
        BMJ. 2006; 332: 1080https://doi.org/10.1136/bmj.332.7549.1080
        • Senn S.
        Disappointing dichotomies.
        Pharm Stat. 2003; 2: 239-240
        • Cohen J.
        The cost of dichotimization.
        Appl Psychol Meas. 1983; 7: 249-253
        • Deyi BA
        • Kosinski AS
        • Snapinn SM.
        Power considerations when a continuous outcome variable is dichotomized.
        J Biopharm Stat. 1998; 8: 337-352https://doi.org/10.1080/10543409808835243
        • Kieser M
        • Rohmel J
        • Friede T.
        Power and sample size determination when assessing the clinical relevance of trial results by 'responder analyses.
        Stat Med. 2004; 23: 3287-3305https://doi.org/10.1002/sim.1910
        • Caille A
        • Leyrat C
        • Giraudeau B.
        Dichotomizing a continuous outcome in cluster randomized trials: impact on power.
        Stat Med. 2012; 31: 2822-2832https://doi.org/10.1002/sim.5409
        • Austin PC
        • Brunner LJ.
        Inflation of the type I error rate when a continuous confounding variable is categorized in logistic regression analyses.
        Stat Med. 2004; 23: 1159-1178https://doi.org/10.1002/sim.1687
        • Barnwell-Menard JL
        • Li Q
        • Cohen AA.
        Effects of categorization method, regression type, and variable distribution on the inflation of Type-I error rate when categorizing a confounding variable.
        Stat Med. 2015; 34: 936-949https://doi.org/10.1002/sim.6387
        • Spruijt B
        • Vergouwe Y
        • Nijman RG
        • Thompson M
        • Oostenbrink R.
        Vital signs should be maintained as continuous variables when predicting bacterial infections in febrile children.
        J Clin Epidemiol. 2013; 66: 453-457https://doi.org/10.1016/j.jclinepi.2012.09.014
        • Snapinn SM
        • Jiang Q.
        Responder analyses and the assessment of a clinically relevant treatment effect.
        Trials. 2007; 8: 31https://doi.org/10.1186/1745-6215-8-31
        • Jaeschke R
        • Singer J
        • Guyatt GH.
        Measurement of health status. Ascertaining the minimal clinically important difference.
        Control Clin Trials. 1989; 10: 407-415
        • Fedorov V
        • Mannino F
        • Zhang R.
        Consequences of dichotomization.
        Pharm Stat. 2009; 8: 50-61https://doi.org/10.1002/pst.331
        • DeCoster J
        • Iselin AM
        • Gallucci M.
        A conceptual and empirical examination of justifications for dichotomization.
        Psychol Methods. 2009; 14: 349-366https://doi.org/10.1037/a0016956
        • Burton A
        • Altman DG
        • Royston P
        • Holder RL.
        The design of simulation studies in medical statistics.
        Stat Med. 2006; 25: 4279-4292https://doi.org/10.1002/sim.2673
        • Hedayati SS
        • Gregg LP
        • Carmody T
        • Jain N.
        • Toups M.
        • Rush A.J.
        • et al.
        Effect of sertraline on depressive symptoms in patients with chronic kidney disease without dialysis dependence: the CAST randomized clinical trial.
        JAMA. 2017; 318: 1876-1890https://doi.org/10.1001/jama.2017.17131
        • Hedayati SS
        • Minhajuddin AT
        • Toto RD
        • Morris DW
        • Rush AJ.
        Validation of depression screening scales in patients with CKD.
        Am J Kidney Dis. 2009; 54 (doi[doi]): 433-439https://doi.org/10.1053/j.ajkd.2009.03.016
        • Furukawa TA
        • Cipriani A
        • Atkinson LZ
        • Leucht S.
        • Ogawa Y.
        • Takeshima N.
        • et al.
        Placebo response rates in antidepressant trials: a systematic review of published and unpublished double-blind randomised controlled studies.
        Lancet Psychiatry. 2016; 3: 1059-1066https://doi.org/10.1016/s2215-0366(16)30307-8
        • Cipriani A
        • Furukawa TA
        • Salanti G
        • Chaimani A
        • Atkinson LZ
        • Ogawa Y.
        • et al.
        Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis.
        Lancet. 2018; 391 (17)32802-7: 1357-1366https://doi.org/10.1016/s0140-6736
        • Rush AJ
        • Trivedi MH
        • Ibrahim HM
        • Carmody TJ
        • Arnow B
        • Klein DN
        • et al.
        The 16-Item Quick Inventory of Depressive Symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression.
        Biol Psychiatry. 2003; 54: 573-583https://doi.org/10.1016/s0006-3223(02)01866-8
        • Hedayati SS
        • Finkelstein FO.
        Epidemiology, diagnosis, and management of depression in patients with CKD.
        Am J Kidney Dis. 2009; 54 (doi[doi]): 741-752https://doi.org/10.1053/j.ajkd.2009.05.003
        • Murtagh FE
        • Addington-Hall J
        • Higginson IJ.
        The prevalence of symptoms in end-stage renal disease: a systematic review.
        Adv Chronic Kidney Dis. 2007; 14 (S1548-5595(06)00163-7): 82-99
        • Cukor D
        • Ver Halen N
        • Asher DR
        • Coplan JD
        • Weedon J
        • Wyka KE
        • et al.
        Psychosocial intervention improves depression, quality of life, and fluid adherence in hemodialysis.
        J Am Soc Nephrol. 2014; 25 (doi[doi]): 196-206https://doi.org/10.1681/ASN.2012111134
        • Beck AT
        • Steer RA
        • Ball R
        • Ranieri W.
        Comparison of Beck depression inventories -IA and -II in psychiatric outpatients.
        J Pers Assess. 1996; 67: 588-597https://doi.org/10.1207/s15327752jpa6703_13
        • Tavernier E
        • Giraudeau B.
        Sample size calculation: inaccurate a priori assumptions for nuisance parameters can greatly affect the power of a randomized controlled trial.
        PLoS One. 2015; 10e0132578https://doi.org/10.1371/journal.pone.0132578
        • Tomitaka S
        • Kawasaki Y
        • Ide K
        • Yamada H
        • Miyake H
        • Furukawa TA.
        Distribution of total depressive symptoms scores and each depressive symptom item in a sample of Japanese employees.
        PLoS One. 2016; 11e0147577https://doi.org/10.1371/journal.pone.0147577
        • Tomitaka S
        • Kawasaki Y
        • Ide K
        • Akutagawa M
        • Ono Y
        • Furukawa TA.
        Distribution of item responses and total item scores for the Center for Epidemiologic Studies Depression Scale (CES-D): data from the Irish Longitudinal Study on Ageing (TILDA).
        PLoS One. 2018; 13e0202607https://doi.org/10.1371/journal.pone.0202607
        • Hernandez AV
        • Steyerberg EW
        • Habbema JD.
        Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements.
        J Clin Epidemiol. 2004; 57: 454-460https://doi.org/10.1016/j.jclinepi.2003.09.014
        • Kunz M.
        On responder analyses when a continuous variable is dichotomized and measurement error is present.
        Biom J. 2011; 53: 137-155https://doi.org/10.1002/bimj.201000069
        • Boessen R
        • Groenwold RH
        • Knol MJ
        • Grobbee DE
        • Roes KC.
        Classifying responders and non-responders; does it help when there is evidence of differentially responding patient groups?.
        J Psychiatr Res. 2012; 46: 1169-1173https://doi.org/10.1016/j.jpsychires.2012.05.005
        • Guyatt GH
        • Juniper EF
        • Walter SD
        • Griffith LE
        • Goldstein RS.
        Interpreting treatment effects in randomised trials.
        BMJ. 1998; 316: 690-693https://doi.org/10.1136/bmj.316.7132.690
        • Guyatt GH
        • Thorlund K
        • Oxman AD
        • Walter SD
        • Patrick D
        • Furukawa TA
        • et al.
        GRADE guidelines: preparing summary of findings tables and evidence profiles-continuous outcomes.
        J Clin Epidemiol. 2013; 66: 173-183https://doi.org/10.1016/j.jclinepi.2012.08.001