Propensity score methods gave similar results to traditional regression modeling in observational studies: a systematic review

Abstract 

Objective

To determine whether adjusting for confounder bias in observational studies using propensity scores gives different results than using traditional regression modeling.

Methods

Medline and Embase were used to identify studies that described at least one association between an exposure and an outcome using both traditional regression and propensity score methods to control for confounding. From 43 studies, 78 exposure–outcome associations were found. Measures of the quality of propensity score implementation were determined. The statistical significance of each association using both analytical methods was compared. The odds or hazard ratios derived using both methods were compared quantitatively.

Results

Statistical significance differed between regression and propensity score methods for only 8 of the associations (10%), κ=0.79 (95% CI=0.65–0.92). In all cases, the regression method gave a statistically significant association not observed with the propensity score method. The odds or hazard ratio derived using propensity scores was, on average, 6.4% closer to unity than that derived using traditional regression.

Conclusions

Observational studies had similar results whether using traditional regression or propensity scores to adjust for confounding. Propensity scores gave slightly weaker associations; however, many of the reviewed studies did not implement propensity scores well.

Keywords: Statistical methods, Observational studies, Propensity scores, Regression modeling, Systematic reviews, Confounding