Comment by M.E. Charlson and M. Wells

The Mehta et al.[] study conducted an empirical evaluation of different methods of weighting the 17 chronic conditions in the original 1987 Charlson [] comorbidity index. In summarizing the original study, the authors missed that the validation used 10-year mortality rates in the breast cancer population rather than one-year mortality rates in hospitalized patients. This is a critical difference, since any estimate of predictive power in one population will deteriorate in a subsequent population, particularly if it is overfit. We have always believed that the remarkable robustness of the Charlson comorbidity index (CCI) was that it was developed and validated in very different populations over different follow up intervals.

Mehta et al. did all the calculations in a single population. Claims from one year were used to calculate the comorbidity score, and claims for the second year were used to assess mortality. They did not use either split sample techniques or validation samples so it is a given that their estimate of fit overestimate the explained variance for their derived “new” indices. Also of note, the Mehta et al. population is over 75 years of age, where the drivers of mortality may differ from a general population.

A primary motivation for the Mehta et al. study was that in the calculation of the CCI score an additive approach is being applied instead of a multiplicative one, they infer that incorrect mathematics is being used. If one casts the construction of the risk measure in terms of utility theory from discrete choice economics [], it turns out that the sum of HRs is the mathematically correct method for assessing an individual's specific unobserved comorbidity. Specifically, let β = (β₁, …, β_M) be a vector of length M which represents the components of comorbidity associated with each of M diseases. Suppose β_j, the comorbidity (ie regression coefficient) contribution associated with disease j has an additional unobservable individual specific random component e_j. If these random components are independent and identically distributed across j and follow the type-I extreme value distribution, these random error account for individual specific unmeasured contributions to morbidity. Then it can be shown that the expected value of the Max {β₁+e₁, β₂+e₂, …, β_M+e_M} = log{exp(β₁) + exp(β₂) + …+ exp(β_M)}. The exp(β_j) are exactly the HRs if the β_j's are regression coefficients. Since the log is monotone, it is equivalent to use the sum of the HRs (= CCI) the risk measure. The expected value of Max {β₁+e₁, β₂+e₂, …, β_M+e_M} can thought of as an expected worse case comorbidity value for an individual (as measured by regression coefficients) that accounts for specific unobserved heterogeneity. The use of the type-I extreme value distribution is standard in discrete choice economics. Consequently, there is a well established theoretical framework that suggests that the mathematics that underlies the CCI is well-grounded.

Mehta et al. contrasts the uses of beta coefficients (original, rounded, multiplied by 10, increased by 1 for each 0.3 increased in β (Schneewiess) or divided by the smallest regression coefficient). They also contrast he use of hazard/odds ratio, using two different weighting system. The Schneewiess weighting system is based on a one-unit weight increase for a 35% increase in the HR, so it can also be considered a HR scoring system.

In Mehta et al.'s Table 4 the AIC and Adjusted R² are remarkably similar across all of the measure within the CPRD population, both likely overestimate the predictive validity in a different population due to overfitting. The existing CCS measures listed in Table 4 also have strikingly similar performance. It is hard to conclude from Table 4 that the use of the regression coefficient based scoring system performs meaningfully better than a risk ratio based scoring system in predicting mortality. The simplicity of calculating the Charlson comorbidity index and its interpretability has likely propelled its widespread use.