Advertisement

Controversy and Debate: Questionable utility of the relative risk in clinical research: Paper 2: Is the Odds Ratio “portable” in meta-analysis? Time to consider bivariate generalized linear mixed model

      Abstract

      Objectives

      A recent paper by Doi et al. advocated completely replacing the relative risk (RR) with the odds ratio (OR) as the effect measure in clinical trials and meta-analyses with binary outcomes. Besides some practical advantages of RR over OR, Doi et al.’s key assumption that the OR is “portable” in the meta-analysis, that is, study-specific ORs are likely not correlated with baseline risks, was not well justified.

      Study designs and settings

      We summarized Spearman's rank correlation coefficient between study-specific ORs and baseline risks in 40,243 meta-analyses from the Cochrane Database of Systematic Reviews.

      Results

      Study-specific ORs tend to be higher in studies with lower baseline risks of disease for most meta-analyses in Cochrane Database of Systematic Reviews. Using an actual meta-analysis example, we demonstrate that there is a strong negative correlation between OR (RR or RD) with the baseline risk and the conditional effects notably vary with baseline risks.

      Conclusions

      Replacing RR or RD with OR is currently unadvisable in clinical trials and meta-analyses. It is possible that no effect measure is “portable” in a meta-analysis. In addition to the overall (or marginal) effect, we suggest presenting the conditional effect based on the baseline risk using a bivariate generalized linear mixed model.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      Reference

        • Doi SA
        • Furuya-Kanamori L
        • Xu C
        • Lin L
        • Chivese T
        • Thalib L
        Questionable utility of the relative risk in clinical research: a call for change to practice.
        J Clin Epidemiol. 2020; (in press)
        • Schwartz LM
        • Woloshin S
        • Welch HG
        Misunderstandings about the effects of race and sex on physicians' referrals for cardiac catheterization.
        N Engl J Med. 1999; 341: 279-283
        • Grant RL
        Converting an odds ratio to a range of plausible relative risks for better communication of research findings.
        Br Med J. 2014; 348: f7450
        • Sinclair JC
        • Bracken MB
        Clinically useful measures of effect in binary analyses of randomized trials.
        J Clin Epidemiol. 1994; 47: 881-889
        • Sainani KL
        • Schmajuk G
        • Liu V
        A caution on interpreting odds ratios.
        Sleep. 2009; 32: 976
        • Newcombe RG
        A deficiency of the odds ratio as a measure of effect size.
        Stat Med. 2006; 25: 4235-4240
        • Chen H
        • Cohen P
        • Chen S
        How big is a big odds ratio? interpreting the magnitudes of odds ratios in epidemiological studies.
        Commun Stat. 2010; 39: 860-864
        • McNutt LA
        Estimating the relative risk in cohort studies and clinical trials of common outcomes.
        Am J Epidemiol. 2003; 157: 940-943
        • Zhang J
        • Yu KF
        What's the relative risk?a method of correcting the odds ratio in cohort studies of common outcomes.
        JAMA. 1998; 280: 1690-1691
        • Chu H
        • Cole SR
        Estimation of risk ratios in cohort studies with common outcomes:" a bayesian approach".
        Epidemiology. 2010; : 855-862
        • Skove T
        • Deddens J
        • Petersen MR
        • Endahl L
        Prevalence proportion ratios: estimation and hypothesis testing.
        Int J Epidemiol. 1998; 27: 91-95
        • Wacholder S
        Binomial regression in glim: estimating risk ratios and risk differences.
        Am J Epidemiol. 1986; 123: 174-184
        • Deeks J
        When can odds ratios mislead?: odds ratios should be used only in case-control studies and logistic regression analyses.
        Br Med J. 1998; 317: 1155
        • Katz KA
        The (Relative) risks of using odds ratios.
        Arch Dermatol. 2006; 142: 761-764
        • Holcomb Jr, WL
        • Chaiworapongsa T
        • Luke DA
        • Burgdorf KD
        An odd measure of risk: use and misuse of the odds ratio.
        Obstetr Gynecol. 2001; 98: 685-688
        • Grimes DA
        • Schulz KF
        Making sense of odds and odds ratios.
        Obstetr Gynecol. 2008; 111: 423-426
        • Guyatt GH
        • Sackett DL
        • Sinclair JC
        • Hayward R
        • Cook DJ
        • Cook RJ
        • et al.
        Users’ guides to the medical literature: Ix. a method for grading health care recommendations.
        JAMA. 1995; 274: 1800-1804
        • Greenland S
        Interpretation and choice of effect measures in epidemiologic analyses.
        Am J Epidemiol. 1987; 125: 761-768
        • Greenland S
        • Robins JM
        • Pearl J
        Confounding and collapsibility in causal inference.
        Stat Sci. 1999; 14: 29-46
        • Greenland S
        • Morgenstern H
        Confounding in health research.
        Ann Rev Public Health. 2001; 22: 189-212
        • Samuels ML
        Matching and design efficiency in epidemiological studies.
        Biometrika. 1981; 68: 577-588
        • Pang M
        • Kaufman JS
        • Platt RW
        Studying noncollapsibility of the odds ratio with marginal structural and logistic regression models.
        Stat Methods Med Res. 2016; 25: 1925-1937
        • Hernán MA
        • Hernández-Díaz S
        • Werler MM
        • Mitchell AA
        Causal knowledge as a prerequisite for confounding evaluation: an application to birth defects epidemiology.
        Am J Epidemiol. 2002; 155: 176-184
        • Samuels ML
        Simpson's paradox and related phenomena.
        J Am Stat Assoc. 1993; 88: 81-88
        • Norton EC
        Log odds and ends.
        Nat Bureau Economic Res Working Paper Series. 2012; Working Paper 18252: 0898-2937
        • Burgess S
        Estimating and contextualizing the attenuation of odds ratios due to non collapsibility.
        Commun Stat. 2017; 46: 786-804
        • Persson E
        • Waernbaum I
        Estimating a marginal causal odds ratio in a case‐control design: analyzing the effect of low birth weight on the risk of type 1 diabetes mellitus.
        Stat Med. 2013; 32: 2500-2512
        • Wermuth N
        Parametric collapsibility and the lack of moderating effects in contingency tables with a dichotomous response variable.
        J Royal Stat Soc. 1987; 49: 353-364
        • Hernán MA
        • Clayton D
        • Keiding N
        The Simpson's paradox unraveled.
        Int J Epidemiol. 2011; 40: 780-785
        • Brand R
        • Kragt H
        Importance of trends in the interpretation of an overall odds ratio in the meta-analysis of clinical trials.
        Stat Med. 1992; 11: 2077-2082
        • Thompson SG
        Systematic review: why sources of heterogeneity in meta-analysis should be investigated.
        BMJ. 1994; 309: 1351
        • Schmid CH
        • Lau J
        • McIntosh MW
        • Cappelleri JC
        An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta‐analysis of clinical trials.
        Stat Med. 1998; 17: 1923-1942
        • Walter SD
        Variation in baseline risk as an explanation of heterogeneity in meta-analysis.
        Stat Med. 1997; 16: 2883-2900
        • Smith GD
        • Song F
        • Sheldon TA
        Cholesterol lowering and mortality: the importance of considering initial level of risk.
        Br Med J. 1993; 306: 1367-1373
        • Hulley SB
        • Newman TB
        • Grady D
        • Garber AM
        • Baron RB
        • Browner WS
        Should we be measuring blood cholesterol levels in young adults?.
        JAMA. 1993; 269: 1416-1419
        • McIntosh MW
        The population risk as an explanatory variable in research synthesis of clinical trials.
        Stat Med. 1996; 15: 1713-1728
        • Horwitz RI
        “Large-scale randomized evidence: large, simple trials and overviews of trials”: discussion. a clinician's perspective on meta-analyses.
        J Clin Epidemiol. 1995; 48: 41-44
        • Cummings P
        The relative merits of risk ratios and odds ratios.
        Arch Pediatr Adolesc Med. 2009; 163: 438-445
        • Hauck WW
        • Anderson S
        • Marcus SM
        Should we adjust for covariates in nonlinear regression analyses of randomized trials?.
        Control Clin Trials. 1998; 19: 249-256
        • Knol MJ
        • Duijnhoven RG
        • Grobbee DE
        • Moons KG
        • Groenwold RH
        Potential misinterpretation of treatment effects due to use of odds ratios and logistic regression in randomized controlled trials.
        PLoS One. 2011; 6: e21248
        • Fleiss JL
        • Levin B
        • Paik MC
        Statistical Methods for Rates and Proportions.
        John Wiley & Sons, 2013
        • Weber F
        • Knapp G
        • Ickstadt K
        • Kundt G
        • Glass Ä
        Zero‐cell corrections in random‐effects meta‐analyses.
        Res Synth Methods. 2020; 11: 913-919
        • Hinkle DE
        Applied Statistics for the Behavioral Sciences.
        2nd ed. Houghton Mifflin, Boston1988
        • Lau J
        • Ioannidis JP
        • Schmid CH
        Summing up evidence: one answer is not always enough.
        Lancet. 1998; 351: 123-127
        • Weiss CO
        • Segal JB
        • Varadhan R
        Assessing the applicability of trial evidence to a target sample in the presence of heterogeneity of treatment effect.
        Pharmacoepidemiol Drug Safety. 2012; 21: 121-129
        • Sharp SJ
        • Thompson SG
        • Altman DG
        The relation between treatment benefit and underlying risk in meta-analysis.
        BMJ. 1996; 313: 735-738
        • Sutton AJ
        • Higgins JP
        Recent developments in meta‐analysis.
        Stat Med. 2008; 27: 625-650
        • Senn S
        • Brand R
        Importance of trends in the interpretation of an overall odds ratio in the meta-analysis of clinical trials.
        Stat Med. 1994; 13: 293-296
        • Schmid CH
        • Stijnen T
        • White I
        Handbook of Meta-Analysis.
        CRC Press, 2020
        • Thompson SG
        • Smith TC
        • Sharp SJ
        Investigating underlying risk as a source of heterogeneity in meta-analysis.
        Stat Med. 1997; 16: 2741-2758
        • Chu H
        • Nie L
        • Chen Y
        • Huang Y
        • Sun W
        Bivariate random effects models for meta-analysis of comparative studies with binary outcomes: methods for the absolute risk difference and relative risk.
        Stat Methods Med Res. 2012; 21: 621-633
        • Derry CJ
        • Derry S
        • Moore RA
        Sumatriptan (Oral route of administration) for acute migraine attacks in adults.
        Cochrane Database Syst Rev. 2012;
        • Viechtbauer W
        Conducting meta-analyses in R with the metafor package.
        J Stat Software. 2010; 36: 1-48
        • Bonett DG
        • Wright TA
        Sample size requirements for estimating pearson, kendall and spearman correlations.
        Psychometrika. 2000; 65: 23-28
        • Zeger SL
        • Liang K-Y
        • Albert PS
        Models for longitudinal data: a generalized estimating equation approach.
        Biometrics. 1988; : 1049-1060
        • Ma X
        • Nie L
        • Cole SR
        • Chu H
        Statistical methods for multivariate meta-analysis of diagnostic tests: an overview and tutorial.
        Stat Methods Med Res. 2016; 25: 1596-1619