Covariate adjustment in RCTs results in increased power to detect conditional effects compared with the power to detect unadjusted or marginal effects
Article Outline
In reply:
We thank Dr Lingsma et al. [1], [2] for their letter, which expands on our recently published review of how baseline covariates are treated in reports of randomized controlled trials (RCTs). Their letter represents an important component of the discussion that we hoped to stimulate regarding the advantages of adjusted vs. unadjusted analyses in the reports of RCTs.
Steyerberg [3] has recently highlighted differences between adjusted and unadjusted estimates in RCTs when linear or generalized linear models are used. Compared with the unadjusted analysis, adjustment using a linear model results in unchanged effect estimates, decreased standard errors for the treatment effect (and thus greater precision), and increased power. However, when using a logistic regression or Cox proportional hazards model for adjustment, regression adjustment results in effect estimates that are further from the null value, increased standard errors for the treatment effect (and thus reduced precision), and increased power compared with unadjusted analyses. These observations initially appear paradoxical: in only one of the two cases does adjustment result in increased precision, whereas in both cases, adjustment results in increased power.
These seemingly paradoxical observations can be explained, in part, by the difference between marginal and conditional estimates of treatment effect (the marginal effect is the average effect, at the population level, of moving an entire population from untreated to treated; the conditional effect is the average effect, at the individual level, of moving an individual from untreated to treated) [4]. In our original article, we devoted a considerable amount of discussion to marginal (or population average) and conditional (or adjusted) estimates of treatment effects. As we noted, unadjusted analyses provide estimates of marginal treatment effects, whereas adjusted analyses provide estimates of conditional treatment effects. When a difference in means is used as the measure of treatment effect for continuous outcomes in an RCT, the conditional and marginal estimates will coincide in expectation. However, when the odds ratio or hazard ratio is used as the measure of effect for binary or time-to-event outcomes, respectively, in an RCT, then the marginal and conditional estimates will not, in most circumstances, coincide [5].
A difference in means is said to be collapsible because the adjusted and unadjusted difference in means will, on average, coincide in the absence of confounding. Similarly, an odds ratio or hazard ratio is said to be not collapsible because the adjusted and unadjusted estimates will not, on average, coincide, even in the absence of confounding. Although our original review discussed the effect of adjustment on precision, our discussion emphasized marginal vs. conditional effects rather than the relative precision of adjusted vs. unadjusted estimates for two reasons. First, it is our perception that many researchers and applied analysts are unaware of the interpretative differences between these different measures of effect, and our review was meant to be educational in nature. Second, there is a lack of consensus in the literature as to which estimate is of greater utility. For instance, Hauck et al. [6] argue that the conditional estimate is more meaningful from a clinical perspective, whereas Martens et al. [7] suggest that the marginal treatment effect is better defined and appears to be of greater interest.
The noncollapsibility of the odds ratio and the hazard ratio help explain the seemingly paradoxical observations noted above. When logistic regression or a Cox proportional hazards model is used for adjustment, the adjusted estimate is further from the null value than the unadjusted estimate. This reflects the noncollapsibility of the odds ratio or the hazard ratio. However, it should be noted in this setting that adjustment leads to the estimation of a different quantity—the conditional effect—that does not coincide with the marginal effect. Given a set of baseline covariates, the marginal and conditional effects have distinct interpretations. Similarly, the increase in the standard error reflects the fact that the standard error of the conditional (adjusted) estimate exceeds that of the marginal (unadjusted) estimate. Here again, the increase in power reflects the fact that the power to detect a non-null conditional effect is greater than the power to detect a non-null marginal effect. When comparing changes in precision and power induced by adjustment, one must remember that, for these nonlinear models, one is dealing with two different effects that do not coincide in their interpretation. For linear treatment effects, these differences are less germane because of the fact that the conditional and marginal effects coincide.
As Lingsma et al. correctly note, it is important to highlight that adjustment induces an increase in statistical power, both for linear models and logistic regression and Cox proportional hazards models. However, its merits emphasis that it is the statistical power to detect a conditional effect that is larger than the statistical power to detect a marginal effect. As stated in our original article, there is a need for a discussion about the relative merits of conditional vs. marginal effects from a clinical, policy, and societal perspective. We appreciate the more detailed exposition of this issue afforded by the letter of Lingsma et al.
Acknowledgments
This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results, and conclusions reported in this article are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred. Dr Austin is supported in part by a Career Investigator Award from the Heart and Stroke Foundation of Ontario.
References
- . Covariate adjustment increases statistical power in randomized controlled trials. J Clin Epidemiol. 2010;63:1387
- . A substantial and confusing variation exists in handling of baseline covariates in randomized controlled trials: a review of trials published in leading medical journals. J Clin Epidemiol. 2010;63:142–153
- . Clinical prediction models: a practical approach to development, validation, and updating. New York, NY: Springer; 2009;
- . Interpretation and choice of effect measures in epidemiologic analyses. Am J Epidemiol. 1987;125:761–768
- . Biased estimates of treatment effect in randomized experiments with nonlinear regressions and omitted covariates. Biometrika. 1984;7:431–444
- . Should we adjust for covariates in nonlinear regression analyses of randomized trials?. Control Clin Trials. 1998;19:249–256
- . Conditioning on the propensity score can result in biased estimation of common measures of treatment effect: a Monte Carlo study (p n/a) by Peter C. Austin, Paul Grootendorst, Sharon-Lise T. Normand, Geoffrey M. Anderson, Statistics in Medicine, Published Online: 16 June 2006. DOI: 10.1002/sim. 2618. Stat Med. 2007;26:3208–3210
PII: S0895-4356(10)00189-7
doi:10.1016/j.jclinepi.2010.05.004
© 2010 Elsevier Inc. All rights reserved.
