It is a good thing that widely used instruments, including those whose validations have been published in peer reviewed journals, are regularly evaluated, as this ensures further development and improvement. At the same time, this can be challenging both for developers and users if an instrument as designed has been working already for a long time.
In this issue, Mehta et al.[
[1]
] report on the scoring system of the Charlson Comorbidity Index (CCI). This well-known instrument was the result of a very creative and successful clinical research initiative back in the mid-eighties to objectify and measure the phenomenon of comorbidity. This was reported in a seminal paper on a topic of ongoing interest in clinical epidemiology: enabling researchers to better predict survival, resource utilization and functional decline, and to control for confounding [[2]
].Given the importance, significance and impact of the CCI, it was useful that Mehta and co-workers evaluated its scoring system. One of the points they raise is that in the calculation of the CCI score an additive approach is being applied instead of a multiplicative one. Harrell [
[3]
] and Moons et al.[[4]
] have earlier signaled that adding ratios is mathematically incorrect, and will cause protective factors to be weighted as harmful due to anti-logging a negative value which yields a positive score [[4]
]. The more unequal ratios are, the more incorrect adding anti-logs becomes. The values of the ratios in the CCI are such that adding them may often yield an outcome that order patients' prognoses in the same order as the correct score if the ratios had been multiplied. But this does not mean that the correct algebra should not be used.In response to Harrell's correspondence, Charlson [] said that, since publication, the original Charlson comorbidity index has been surprisingly useful across a wide variety of applications, and that whether or not alteration of the scaling would improve its usefulness is certainly an issue that could be empirically addressed. She added that the original index was never envisioned as the final definitive statement, but instead as an important foundation on which to build. The comments of Harrell, Moons et al., and Mehta et al. imply that the scoring system of the CCI should indeed not be considered final definitive.
Some might hold the view that an approach that has been accepted and used for decades, and included in many top tier journal papers and clinical applications, should not be altered. They might also think that, as long as even algebraically incorrect approaches are being validated using empirical data, there is no question of a problem.
The alternative, methodologically rigorous position is that irrespective of previous empirical outcomes, an algebraically correct approach should be followed, also if this implies re-validation and, where potentially relevant, re-evaluation of previous outcomes as to possible clinical impact. The fact that, in applying a very important instrument, a method has been followed for so long and still is followed by many researchers, could be seen as an extra motivation for choosing this route.
We endorse the latter approach, as using correct algebra is surely not up for debate. In taking this position, we feel a special responsibility since the original seminal paper on the CCI was published in the predecessor of the Journal of Clinical Epidemiology, the Journal of Chronic Diseases, and at the time neither reviewers nor editors seem to have raised this issue.
References
- Regression Coefficient Based Scoring System Should be Used to Assign Weights to the Risk Index.J Clin Epidemiol. 2016; 72 (….- …)
- A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.J Chronic Dis. 1987; 40: 373-383
- Regression coefficients and scoring rules.J Clin Epidemiol. 1996; 49: 819
- Should scoring rules be based on odds ratios or regression coefficients?.J Clin Epidemiol. 2002; 55: 1054-1055
- Response.J Clin Epidemiol. 1996; 49: 819
Article info
Identification
Copyright
© 2016 Elsevier Inc. All rights reserved.
ScienceDirect
Access this article on ScienceDirectLinked Article
- Regression coefficient–based scoring system should be used to assign weights to the risk indexJournal of Clinical EpidemiologyVol. 79
- PreviewSome previously developed risk scores contained a mathematical error in their construction: risk ratios were added to derive weights to construct a summary risk score. This study demonstrates the mathematical error and derived different versions of the Charlson comorbidity score (CCS) using regression coefficient–based and risk ratio–based scoring systems to further demonstrate the effects of incorrect weighting on performance in predicting mortality.
- Full-Text
- Preview
- Comment by M.E. Charlson and M. WellsJournal of Clinical EpidemiologyVol. 79
- PreviewThe Mehta et al.[1] study conducted an empirical evaluation of different methods of weighting the 17 chronic conditions in the original 1987 Charlson [2] comorbidity index. In summarizing the original study, the authors missed that the validation used 10-year mortality rates in the breast cancer population rather than one-year mortality rates in hospitalized patients. This is a critical difference, since any estimate of predictive power in one population will deteriorate in a subsequent population, particularly if it is overfit.
- Full-Text
- Preview
