Advertisement

Controversy and Debate : Questionable utility of the relative risk in clinical research: Paper 4 :Odds Ratios are far from “portable” — A call to use realistic models for effect variation in meta-analysis

      Abstract

      Objective

      Recently Doi et al. argued that risk ratios should be replaced with odds ratios in clinical research. We disagreed, and empirically documented the lack of portability of odds ratios, while Doi et al. defended their position. In this response we highlight important errors in their position.

      Study design and setting

      We counter Doi et al.’s arguments by further examining the correlations of odds ratios, and risk ratios, with baseline risks in 20,198 meta-analyses from the Cochrane Database of Systematic Reviews.

      Results

      Doi et al.’s claim that odds ratios are portable is invalid because 1) their reasoning is circular: they assume a model under which the odds ratio is constant and show that under such a model the odds ratio is portable; 2) the method they advocate to convert odds ratios to risk ratios is biased; 3) their empirical example is readily-refuted by counter-examples of meta-analyses in which the risk ratio is portable but the odds ratio isn't; and 4) they fail to consider the causal determinants of meta-analytic inclusion criteria: Doi et al. mistakenly claim that variation in odds ratios with different baseline risks in meta-analyses is due to collider bias. Empirical comparison between the correlations of odds ratios, and risk ratios, with baseline risks show that the portability of odds ratios and risk ratios varies across settings.

      Conclusion

      The suggestion to replace risk ratios with odds ratios is based on circular reasoning and a confusion of mathematical and empirical results. It is especially misleading for meta-analyses and clinical guidance. Neither the odds ratio nor the risk ratio is universally portable. To address this lack of portability, we reinforce our suggestion to report variation in effect measures conditioning on modifying factors such as baseline risk; understanding such variation is essential to patient-centered practice.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Doi S.A.
        • Furuya-Kanamori L.
        • Xu C.
        • Lin L.
        • Chivese T.
        • Thalib L.
        Questionable utility of the relative risk in clinical research: a call for change to practice.
        J. Clin. Epidemiol. 2020; (In press)https://doi.org/10.1016/j.jclinepi.2020.08.019
        • Doi S.A.
        • Furuya-Kanamori L.
        • Xu C.
        • Chivese T.
        • Lin L.
        • Musa O.A.H.
        • et al.
        The OR is “portable” but not the RR: Time to do away with the log link in binomial regression.
        J. Clin. Epidemiol. 2021; (In press)
        • Jewell N.P.
        Estimation of Logistic Regression Model Parameters Ch. 13.
        Statistics for Epidemiology. Chapman and Hall/CRC, 2003: 223-229
        • Greenland S.
        Interactions in epidemiology: Relevance, identification, and estimation.
        Epidemiology. 2009; 20: 14-16
        • Schmidt A.F.
        • Dudbridge F.
        • Groenwold R.H.H
        Re: Is the risk difference really a more heterogeneous measure?.
        Epidemiology. 2016; 27: e12
        • Poole C.
        • Shrier I.
        • Ding P.
        • VanderWeele T.
        The authors respond.
        Epidemiology. 2016; 27: e12-e13
        • Poole C.
        • Shrier I.
        • VanDerWeele T.J.
        Is the risk difference really a more heterogeneous measure?.
        Epidemiology. 2015; 26: 714-718
        • Schmid C.H.
        • Lau J.
        • Mcintosh M.W.
        • Cappelleri J.C.
        An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials.
        Stat. Med. 1998; 17: 1923-1942
        • Xiao M.
        • Chen Y.
        • Cole S.R.
        • MacLehosed R.F.
        • Richardson D.B.
        • Chu H.
        Is OR “portable” in meta-analysis? Time to consider bivariate generalized linear mixed model.
        J. Clin. Epidemiol. 2021; (In press)
        • Chu H.
        • Cole S.R.
        Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach.
        J. Clin. Epidemiol. 2006; 59: 1331-1332
        • Rothman K.J.
        • Greenland S.
        • Walker A.M.
        Concepts of interaction.
        Am. J. Epidemiol. 1980; 112: 467-470
        • Chu H.
        • Nie L.
        • Chen Y.
        • Huang Y.
        • Sun W.
        Bivariate random effects models for meta-analysis of comparative studies with binary outcomes: Methods for the absolute risk difference and relative risk.
        Stat. Methods Med. Res. 2012; 21: 621-633
        • Rafi Z.
        • Greenland S.
        Semantic and cognitive tools to aid statistical science: Replace confidence and significance by compatibility and surprise.
        BMC Med. Res. Methodol. 2020; 20: 1-13
        • Cole S.R.
        • Edwards J.K.
        • Greenland S.
        Surprise!.
        Am. J. Epidemiol. 2021; 190: 191-193
        • Rothman K.J.
        Rothman responds to ‘surprise!’.
        Am. J. Epidemiol. 2021; 190: 194-195
        • Wasserstein R.L.
        • Schirm A.L.
        • Lazar N.A.
        Moving to a world beyond “p < 0.05”.
        Am. Stat. 2019; 73: 1-19
        • Rothman K.J.
        Disengaging from statistical significance.
        Eur. J. Epidemiol. 2016; 31: 443-444
        • Amrhein V.
        • Greenland S.
        • Mcshane B.
        Retire statistical significance.
        Nature. 2019; 567: 305-307
        • Greenland S.
        • Pearce N.
        Statistical foundations for model-based adjustments.
        Annu. Rev. Public Health. 2015; 36: 89-108
        • Greenland S.
        Tests for interaction in epidemiologic studies: A review and a study of power.
        Stat. Med. 1983; 2: 243-251
        • Greenland S.
        Basic problems in interaction assessment.
        Environ. Health Perspect. 1993; 101: 59
        • Greenland S.
        Smoothing observational data: a philosophy and implementation for the health sciences.
        Int. Stat. Rev. 2006; 74: 31-46
        • Lesko C.R.
        • Henderson N.C.
        • Varadhan R.
        Considerations when assessing heterogeneity of treatment effect in patient-centered outcomes research.
        J. Clin. Epidemiol. 2018; 100: 22-31
        • Zhang J.
        • Yu K.F.
        What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes.
        J. Am. Med. Assoc. 1998; 280: 1690-1691
        • McNutt L.A.
        • Wu C.
        • Xue X.
        • Hafner J.P.
        Estimating the relative risk in cohort studies and clinical trials of common outcomes.
        Am. J. Epidemiol. 2003; 157: 940-943
        • Karp I.
        Re: Estimating the relative risk in cohort studies and clinical trials of common outcomes.
        Am. J. Epidemiol. 2014; 179: 1034-1035
        • Cole S.R.
        • Hernán M.A.
        Constructing inverse probability weights for marginal structural models.
        Am. J. Epidemiol. 2008; 168: 656-664
        • Richardson D.B.
        • Kinlaw A.C.
        • MacLehose R.F.
        • Cole S.R.
        Standardized binomial models for risk or prevalence ratios and differences.
        Int. J. Epidemiol. 2015; 44: 1660-1672
        • Greenland S.
        Model-based estimation of relative risks and other epidemiologic measures in studies of common outcomes and in case-control studies.
        Am. J. Epidemiol. 2004; 160: 301-305
        • Localio A.R.
        • Margolis D.J.
        • Berlin J.A.
        Relative risks and confidence intervals were easily computed indirectly from multivariable logistic regression.
        J. Clin. Epidemiol. 2007; 60: 874-882
        • Muller C.J.
        • Maclehose R.F.
        Estimating predicted probabilities from logistic regression: Different methods correspond to different target populations.
        Int. J. Epidemiol. 2014; 43: 962-970
        • Ross R.K.
        • Cole S.R.
        • Richardson D.B.
        Decreased susceptibility of marginal odds ratios to finite-sample bias.
        Epidemiology. 2021; (Publish Ahead of Print)
        • Greenland S.
        • Robins J.M.
        • Pearl J.
        Confounding and collapsibility in causal inference.
        Stat. Sci. 1999; 14: 29-46
        • Huitfeldt A.
        • Stensrud M.J.
        • Suzuki E.
        On the collapsibility of measures of effect in the counterfactual causal framework.
        Emerg. Themes Epidemiol. 2019; 16: 1-5
        • Greenland S.
        • Pearl J.
        Adjustments and their consequences-collapsibility analysis using graphical models.
        Int. Stat. Rev. 2011; 79: 401-426
        • Greenland S.
        Noncollapsibility, confounding, and sparse-data bias. Part 2: What should researchers make of persistent controversies about the odds ratio?.
        J. Clin. Epidemiol. 2021; (; In press)https://doi.org/10.1016/j.jclinepi.2021.06.004
        • Pang M.
        • Kaufman J.S.
        • Platt R.W.
        Studying noncollapsibility of the odds ratio with marginal structural and logistic regression models.
        Stat. Methods Med. Res. 2016; 25: 1925-1937
        • Mood C.
        Logistic regression: Why we cannot do what We think we can do, and what we can do about it.
        Eur. Sociol. Rev. 2010; 26: 67-82
        • Pang M.
        • Kaufman J.S.
        • Platt R.W.
        Studying noncollapsibility of the odds ratio with marginal structural and logistic regression models.
        Stat. Methods Med. Res. 2016; 25: 1925-1937
        • Didelez V.
        • Stensrud M.J.
        On the logic of collapsibility for causal effect measures.
        Biometrical J. 2021; 1–8https://doi.org/10.1002/bimj.202000305
        • Smith G.D.
        • Song F.
        • Sheldon T.A.
        Cholesterol lowering and mortality: the importance of considering initial level of risk.
        BMJ. 1993; 306: 1367-1373
        • Thompson S.G.
        Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated.
        BMJ. 1994; 309: 1351-1355
        • Walter S.D.
        Variation in baseline risk as an explanation of heterogeneity in meta-analysis.
        Stat. Med. 1997; 16: 2883-2900
        • Horwitz R.I.
        Large-scale randomized evidence: Large, simple trials and overviews of trials”: Discussion. A clinician's perspective on meta-analyses.
        J. Clin. Epidemiol. 1995; 48: 41-44
        • Didelez V.
        • Stensrud M.J.
        On the logic of collapsibility for causal effect measures.
        Biometrical J. 2021; 1–8https://doi.org/10.1002/bimj.202000305
        • Kackar R.N.
        • Harville D.A.
        Unbiasedness of two-stage estimation and prediction procedures for mixed linear models.
        Commun. Stat. - Theory Methods. 1981; 10: 1249-1261
        • Viechtbauer W.
        Conducting meta-analyses in R with the metafor.
        J. Stat. Softw. 2010; https://doi.org/10.18637/jss.v036.i03
        • Weber F.
        • Knapp G.
        • Ickstadt K.
        • Kundt G.
        • Glass Ä.
        Zero-cell corrections in random-effects meta-analyses.
        Res. Synth. Methods. 2020; https://doi.org/10.1002/jrsm.1460
        • Mukaka M.M.
        Statistics corner: A guide to appropriate use of correlation coefficient in medical research.
        Malawi Med. J. 2012; 24: 69-71
        • Greenland S.
        • Mansournia M.A.
        • Altman D.G.
        Sparse data bias: A problem hiding in plain sight.
        BMJ. 2016; 353: 1-6
        • Richardson D.B.
        • Cole S.R.
        • Ross R.K.
        • Poole C.
        • Chu H.
        • Keil A.P.
        Meta-analysis and sparse-data bias.
        Am. J. Epidemiol. 2021; 190: 336-340