Research Article| Volume 56, ISSUE 6, P553-558, June 2003

Bias in estimates of confidence intervals for health outcome report cards


      Health researchers commonly use logistic regression when profiling health providers. Data from the patients treated by the providers are used to construct models predicting the expected number of outcomes for providers and the ratio of observed to expected outcomes (O/E ratio) used as a risk-adjusted measure of provider performance. Typically, when calculating the standard deviation (SD) of O/E ratios, only O is treated as a random variable. We used the propagation of errors (Pe) to derive a SD estimate that accounted for variability in O and the estimate of E. Using data previously used to profile Canadian cardiac surgery providers, we compared Pe-SD estimates with typical SD (SDT) estimates. The SDT estimates and confidence intervals were always larger than the Pe estimates, most notably when one or more providers treated a large proportion of the patients. This was confirmed using computer simulations. SDT estimates should be abandoned in favor of more sophisticated estimates.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Clinical Epidemiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Shwartz M
        • Ash A.S
        • Iezzoni L.L
        Comparing outcomes across providers.
        in: Iezzoni L.I Risk adjustment for measuring healthcare outcomes. 2nd edition. Health Administration Press, Chicago1997: 471-516
        • Hosmer D
        • Lemeshow S
        Confidence interval estimates of an index of quality performance based on logistic regression models.
        Stat Med. 1995; 14: 2161-2172
        • Rice J.A
        Expected values: mathematical statistics and data analysis. 2nd edition. Duxbury Press, Belmont (CA)1995 (p. 111–62)
        • Ghali W.A
        • Quan H
        • Brant R
        Coronary artery bypass grafting in Canada: national and provincial mortality trends, 1992–1995.
        CMAJ. 1998; 159: 25-31
        • Ghali W.A
        • Brant R
        • Quan H
        CABG in Canada.
        CMAJ. 1999; 161: 941
        • Ghali W.A
        • Knudtson M.L
        for the APPROACH Investigators. Overview of ‘APPROACH’ the Alberta provincial project for outcome assessment in coronary heart disease.
        Can J Cardiol. 2000; 16: 1225-1230