Advertisement

A systematic review identifies valid comorbidity indices derived from administrative health data

  • Marko Yurkovich
    Affiliations
    Division of Rheumatology, Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada

    Milan Ilich Arthritis Research Centre, 5591 No. 3 Rd, Richmond, British Columbia, Canada V6X 2C7
    Search for articles by this author
  • J. Antonio Avina-Zubieta
    Affiliations
    Division of Rheumatology, Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada

    Milan Ilich Arthritis Research Centre, 5591 No. 3 Rd, Richmond, British Columbia, Canada V6X 2C7
    Search for articles by this author
  • Jamie Thomas
    Affiliations
    Faculty of Pharmaceutical Sciences, University of British Columbia, Vancouver, British Columbia, Canada
    Search for articles by this author
  • Mike Gorenchtein
    Affiliations
    Faculty of Medicine, University of Limerick, Castletroy, Co. Limerick, Ireland
    Search for articles by this author
  • Diane Lacaille
    Correspondence
    Corresponding author. Milan Ilich Arthritis Research Centre, 5591 No. 3 Rd, Richmond, British Columbia, Canada V6X 2C7. Tel.: 604-207-4019; fax: 604-207-4059.
    Affiliations
    Division of Rheumatology, Department of Medicine, University of British Columbia, Vancouver, British Columbia, Canada

    Milan Ilich Arthritis Research Centre, 5591 No. 3 Rd, Richmond, British Columbia, Canada V6X 2C7
    Search for articles by this author

      Abstract

      Objectives

      To conduct a systematic review of studies reporting on the development or validation of comorbidity indices using administrative health data and compare their ability to predict outcomes related to comorbidity (ie, construct validity).

      Study Design and Setting

      We conducted a comprehensive literature search of MEDLINE and EMBASE, until September 2012. After title and abstract screen, relevant articles were selected for review by two independent investigators. Predictive validity and model fit were measured using c-statistic for dichotomous outcomes and R2 for continuous outcomes.

      Results

      Our review includes 76 articles. Two categories of comorbidity indices were identified: those identifying comorbidities based on diagnoses, using International Classification of Disease codes from hospitalization or outpatient data, and based on medications, using pharmacy data. The ability of indices studied to predict morbidity-related outcomes ranged from poor (C statistic ≤0.69) to excellent (C statistic >0.80) depending on the specific index, outcome measured, and study population. Diagnosis-based measures, particularly the Elixhauser Index and the Romano adaptation of the Charlson Index, resulted in higher ability to predict mortality outcomes. Medication-based indices, such as the Chronic Disease Score, demonstrated better performance for predicting health care utilization.

      Conclusion

      A number of valid comorbidity indices derived from administrative data are available. Selection of an appropriate index should take into account the type of data available, study population, and specific outcome of interest.

      Keywords

      1. Introduction

      What is new?
      • A number of comorbidity indices are available for use in studies with administrative health data, in order to control for the overall burden of comorbidities.
      • To guide researchers and health policy makers in selecting the index most appropriate for their purpose, this systematic review describes the conceptual and methodological differences among the various indices and compares their ability to predict outcomes related to comorbidity (i.e. construct validity).
      • The review reveals that a number of comorbidity indices demonstrate validity in predicting mortality.
      • A diagnosis-based index, such as the Quan- or van Walraven- EI or Romano-CCI, is recommended in studies where the outcome of interest is mortality.
      • For studies evaluating healthcare utilization, where medication data is available, a medication-based index, such as the RxRisk-V, is recommended.
      Administrative databases are being increasingly used for research purposes. They play an important role in epidemiologic, quality of care, pharmacovigilance, and outcome studies. These databases provide complementary information to randomized controlled trials because of their real-life setting, large samples, long follow-up duration, and their ability to provide population-based samples, free of selection bias. These data, however, have some limitations including lack of clinical, lifestyle, and demographic data and because of the observational nature, which can introduce biases. These biases include selection and channeling bias, as well as confounding by indication. These limitations can be minimized by careful adjustment in statistical analyses.
      In observational studies, the outcomes of interest are often influenced by concurrent or preexisting comorbidities. Comorbidity may be defined as the total burden of illnesses unrelated to the principal diagnosis [
      • Perkins A.J.
      • Kroenke K.
      • Unutzer J.
      • Katon W.
      • Williams J.W.
      • Hope C.
      • et al.
      Common comorbidity scales were similar in their ability to predict health care costs and mortality.
      ]. It is important to adequately adjust for comorbidities in studies in which comorbidities could act as confounders. Given the large number of comorbidities that may be relevant to a given outcome, controlling for individual comorbidities may not be practical for methodological reasons, including loss of power. It may also be necessary to control for the overall burden of comorbidity, rather than the individual effect of each comorbidity.
      For that purpose, a number of comorbidity indices have been developed to measure and weigh the overall burden of comorbidities. Some of these instruments have been developed exclusively for use with administrative data, such as the Elixhauser Index (EI) [
      • Elixhauser A.
      • Steiner C.
      • Harris D.R.
      • Coffey R.M.
      Comorbidity measures for use with administrative data.
      ], whereas others have been developed in other contexts but adapted for use with administrative data, such as the Charlson Comorbidity Index (CCI) [
      • Charlson M.E.
      • Pompei P.
      • Ales K.L.
      • MacKenzie C.R.
      A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.
      ]. These comorbidity indices have been widely used in studies using administrative data to control for the overall burden of comorbidities.
      However, given the large number of indices available in the literature and the conceptual and methodological differences among them, researchers and health policy makers wishing to control for comorbidity need guidance in selecting the index most appropriate for their specific study. Although previous studies have compared the validity of comorbidity indices, they were limited by not systematically reviewing all indices available or by not explaining the conceptual and methodological differences between indices [
      • Sharabiani M.T.
      • Aylin P.
      • Bottle A.
      Systematic review of comorbidity indices for administrative data.
      ,
      • Schneeweiss S.
      • Maclure M.
      Use of comorbidity scores for control of confounding in studies using administrative databases.
      ,
      • Needham D.M.
      • Scales D.C.
      • Laupacis A.
      • Pronovost P.J.
      A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research.
      ]. Our systematic review will guide scientists' choice by reviewing all the indices available, explaining their conceptual and methodological differences, and comparing their construct validity. Because there is no “gold standard” in comorbidity measurement, indices are often validated by measuring how well they are able to predict outcomes related to comorbidity, such as mortality or health care utilization (ie, construct validity) [
      • Schneeweiss S.
      • Seeger J.D.
      • Maclure M.
      • Wang P.S.
      • Avorn J.
      • Glynn R.J.
      Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data.
      ,
      • Schneeweiss S.
      • Wang P.S.
      • Avorn J.
      • Glynn R.J.
      Improved comorbidity adjustment for predicting mortality in Medicare populations.
      ,
      • Newschaffer C.J.
      • Bush T.L.
      • Penberthy L.T.
      Comorbidity measurement in elderly female breast cancer patients with administrative and medical records data.
      ].
      Accordingly, our aim was to conduct a systematic review with the following objectives: (1) to identify the different instruments used in administrative data studies to measure comorbidity, (2) to compare the instruments at the conceptual level, that is, to describe how each index was developed and/or adapted for use with administrative data and what concept the index aimed to measure, and (3) to evaluate and compare their ability to predict comorbidity-related outcomes.

      2. Methods

      2.1 Search strategy

      A methodological literature search was conducted as of September 2012, using the Ovid platform to search MEDLINE (MEDLINE(R) In-Process & Other Non-Indexed Citations, Ovid MEDLINE(R) Daily, and Ovid OLDMEDLINE(R) from 1946) and EMBASE (from 1980). The year limits were dictated by the scope of the databases. We searched for and combined with the Boolean operator “OR” all relevant subject headings, using the “explosion” function where needed, and keywords in titles and abstracts for the two concepts: “Administrative data” and “Comorbidity index.” We combined these two concepts with the Boolean operator “AND.” We excluded articles that were solely abstracts, comments, conference proceedings, editorials, letters, or news. We included only articles published in English. The titles and abstracts of the articles identified by this search were screened by one investigator (M.Y.) and selected for full-text review if relevant to our objectives. From this initial screen, a list of comorbidity indices potentially used in administrative data was identified. To ensure that we captured all relevant indices and their corresponding validation studies, an additional literature search was performed using the same databases. This involved searching titles and abstracts for specific index names. The same screening process was applied to select articles potentially relevant to our objectives.

      2.2 Study selection

      Full-length articles of studies identified as potentially relevant to our objectives were independently reviewed by two authors (M.Y. and J.T.) to determine if they met the prespecified inclusion criteria. Disagreements were settled by consensus. For inclusion, studies had to have developed or validated a comorbidity index for use with administrative data. Of note, we only included studies that related specifically to comorbidity indices and excluded studies that focused on the development or adaptation of risk scores or other groupers for risk adjustment. Adaptation of an index initially designed for use in a different context was permitted, such as adaptations of the CCI, which was initially designed for medical chart review. Validation studies could be prospective or retrospective and could include any patient population (adult or pediatric). We defined a validation study as one that evaluated the ability of a comorbidity index to predict a specific outcome (ie, construct validity) in a given population sample. This could be achieved by reporting the C statistic (for dichotomous outcome variables) or the R2 (for linear outcome variables). Alternatively, odds ratios (ORs), relative risk, or hazard ratios (HR) (Cox) could be reported.

      2.3 Data abstraction and reporting

      Data were abstracted using a standardized data collection form. Abstracted data included the comorbidity indices evaluated, study population, type of administrative data used to calculate the comorbidity score, outcome, and statistics used to evaluate predictive ability. For index development studies, we collected information on the study population, type of data used, and information on how the index was developed or adapted.
      For validation studies, we report on construct validity by presenting results of the ability of indices to predict outcomes related to comorbidity, which we have labeled predictive ability. For dichotomous outcomes, such as mortality, the predictive ability was reported using the area under the curve in a receiver operating characteristic curve, which is equivalent to the C statistic, a measure of model fit. The C statistic ranges from no predictive ability (when equaling 0.50) to perfect prediction (when equaling 1.0). Consistent with recommended guidelines [
      • Hanley J.A.
      • McNeil B.J.
      The meaning and use of the area under a receiver operating characteristic (ROC) curve.
      ], we considered C statistics of 0.7–0.8 as acceptable and ≥0.8 as excellent. For continuous outcomes in linear regression models, the predictive ability was measured using the R2 value, which represents the improvement in explained variance obtained by adding the comorbidity score to a baseline model. R2 values range from 0 to 1, where 1 indicates that all the observed variance in the outcome is explained by the model.

      3. Results

      3.1 Study selection

      The primary literature search revealed 565 citations for title and abstract review. The second search, to identify validation studies of the indices identified in the first search, identified 390 additional articles. Thus, a total of 955 articles were selected for title and abstract review (Fig. 1). Of these, 37 were duplicates and 713 were not relevant to the study objectives, leaving 205 articles for full-text review. Of those, 18 studies did not involve the development or validation of an index, 55 involved an index not using administrative data, and 56 discussed a risk score rather than a comorbidity index. Therefore, these 112 studies were excluded, and a total of 76 articles were included in the final review.
      Figure thumbnail gr1
      Fig. 1Flow chart of studies included in the review.
      Comorbidity indices identified were categorized into two groups: (1) those based on diagnoses from administrative data, using International Classification of Disease, Ninth or Tenth revision diagnostic coding system (ICD-9 or ICD-10) and (2) those based on medications, using prescription data to identify comorbid conditions. The 76 articles included 39 studies of diagnosis-based indices, including 35 related to the CCI and its adaptations, two specifically to the EI, and two reporting study-specific diagnosis-based indices. An additional 13 studies investigated medication-based indices such as the Chronic Disease Score (CDS) and the RxRisk. The remaining 24 articles compared the main indices identified.

      3.2 Diagnosis-based indices

      3.2.1 Charlson comorbidity index

      The CCI was created by Charlson et al. [
      • Charlson M.E.
      • Pompei P.
      • Ales K.L.
      • MacKenzie C.R.
      A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.
      ] in 1987. It was developed using chart review to predict 1-year mortality in a cohort of 604 patients admitted to a medical service at New York Hospital during 1 month in 1984. The CCI was then validated in the same study using a cohort of 685 breast cancer patients admitted to a Connecticut teaching hospital from 1962 to 1969. The final index is a list of 19 conditions, with each condition assigned a weight of 1, 2, 3, or 6 based on adjusted HR for each comorbid condition derived from Cox proportional hazards regression models. A total score is calculated from the sum of the weighted scores [
      • Charlson M.E.
      • Pompei P.
      • Ales K.L.
      • MacKenzie C.R.
      A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.
      ].
      The CCI is the most widely used comorbidity index and has been validated in patient populations with various diagnoses or undergoing various surgical procedures [
      • D'Hoore W.
      • Sicotte C.
      • Tilquin C.
      Risk adjustment in outcome assessment: the Charlson comorbidity index.
      ,
      • Melfi C.
      • Holleman E.
      • Arthur D.
      • Katz B.
      Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data.
      ,
      • Poses R.M.
      • Smith W.R.
      • McClish D.K.
      • Anthony M.
      Controlling for confounding by indication for treatment. Are administrative data equivalent to clinical data?.
      ,
      • Ghali W.A.
      • Hall R.E.
      • Rosen A.K.
      • Ash A.S.
      • Moskowitz M.A.
      Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data.
      ,
      • D'Hoore W.
      • Bouckaert A.
      • Tilquin C.
      Practical considerations on the use of the Charlson comorbidity index with administrative data bases.
      ,
      • Roos L.L.
      • Stranc L.
      • James R.C.
      • Li J.
      Complications, comorbidities, and mortality: improving classification and prediction.
      ,
      • Cleves M.A.
      • Sanchez N.
      • Draheim M.
      Evaluation of two competing methods for calculating Charlson's comorbidity index when analyzing short-term mortality using administrative data.
      ,
      • Librero J.
      • Peiro S.
      • Ordinana R.
      Chronic comorbidity and outcomes of hospital care: length of stay, mortality, and readmission at 30 and 365 days.
      ,
      • Wang P.S.
      • Walker A.
      • Tsuang M.
      • Orav E.J.
      • Levin R.
      • Avorn J.
      Strategies for improving comorbidity measures based on Medicare and Medicaid claims data.
      ,
      • Klabunde C.N.
      • Potosky A.L.
      • Legler J.M.
      • Warren J.L.
      Development of a comorbidity index using physician claims data.
      ,
      • Sundararajan V.
      • Henderson T.
      • Perry C.
      • Muggivan A.
      • Quan H.
      • Ghali W.A.
      New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality.
      ,
      • Harse J.D.
      • Holman C.D.
      Charlson's index was a poor predictor of quality of life outcomes in a study of patients following joint replacement surgery.
      ,
      • Holman C.D.
      • Preen D.B.
      • Baynham N.J.
      • Finn J.C.
      • Semmens J.B.
      A multipurpose comorbidity scoring system performed better than the Charlson index.
      ,
      • Martins M.
      • Blais R.
      Evaluation of comorbidity indices for inpatient mortality prediction models.
      ,
      • Norena M.
      • Wong H.
      • Thompson W.D.
      • Keenan S.P.
      • Dodek P.M.
      Adjustment of intensive care unit outcomes for severity of illness and comorbidity scores.
      ,
      • Nuttall M.
      • van der Meulen J.
      • Emberton M.
      Charlson scores based on ICD-10 administrative data were valid in assessing comorbidity in patients undergoing urological cancer surgery.
      ,
      • Hollenbeak C.S.
      • Stack Jr., B.C.
      • Daley S.M.
      • Piccirillo J.F.
      Using comorbidity indexes to predict costs for head and neck cancer.
      ,
      • Sundararajan V.
      • Quan H.
      • Halfan P.
      • Fushimi K.
      • Luthi J.-
      • Burnand B.
      • et al.
      Cross-national comparative performance of three versions of the ICD-10 Charlson index.
      ,
      • Klabunde C.N.
      • Legler J.M.
      • Warren J.L.
      • Baldwin L.-
      • Schrag D.
      A refined comorbidity measurement algorithm for claims-based studies of breast, prostate, colorectal, and lung cancer patients.
      ,
      • Radley D.C.
      • Gottlieb D.J.
      • Fisher E.S.
      • Tosteson A.N.A.
      Comorbidity risk-adjustment strategies are comparable among persons with hip fracture.
      ,
      • Sessler D.I.
      • Sigl J.C.
      • Manberg P.J.
      • Kelley S.D.
      • Schubert A.
      • Chamoun N.G.
      Broadly applicable risk stratification system for predicting duration of hospitalization and mortality.
      ,
      • Hanley G.E.
      • Morgan S.
      • Reid R.J.
      Explaining prescription drug use and expenditures using the adjusted clinical groups case-mix system in the population of British Columbia, Canada.
      ,
      • Khan N.F.
      • Perera R.
      • Harper S.
      • Rose P.W.
      Adaptation and validation of the Charlson index for Read/OXMIS coded databases.
      ,
      • Liu J.
      • Huang Z.
      • Gilbertson D.T.
      • Foley R.N.
      • Collins A.J.
      An improved comorbidity index for outcome analyses among dialysis patients.
      ]. Numerous adaptations of the CCI have been developed for use with ICD-9 or ICD-10 codes in administrative databases [
      • D'Hoore W.
      • Sicotte C.
      • Tilquin C.
      Risk adjustment in outcome assessment: the Charlson comorbidity index.
      ,
      • Ghali W.A.
      • Hall R.E.
      • Rosen A.K.
      • Ash A.S.
      • Moskowitz M.A.
      Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data.
      ,
      • D'Hoore W.
      • Bouckaert A.
      • Tilquin C.
      Practical considerations on the use of the Charlson comorbidity index with administrative data bases.
      ,
      • Deyo R.A.
      • Cherkin D.C.
      • Ciol M.A.
      Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases.
      ,
      • Romano P.S.
      • Roos L.L.
      • Jollis J.G.
      Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives.
      ,
      • Romano P.S.
      • Roos L.L.
      • Jollis J.G.
      Further evidence concerning the use of a clinical comorbidity index with ICD-9-CM administrative data.
      ,
      • Quan H.
      • Sundararajan V.
      • Halfon P.
      • Fong A.
      • Burnand B.
      • Luthi J.C.
      • et al.
      Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
      ,
      • Quan H.
      • Li B.
      • Couris C.M.
      • Fushimi K.
      • Graham P.
      • Hider P.
      • et al.
      Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.
      ] as described in the following paragraph. For each adaptation, the study populations and primary end points used for development, along with a list of comorbid conditions included, are summarized in Table 1. The results of validation studies are summarized in Table 1/Appendix A at www.jclinepi.com.
      Table 1Description of adaptations of the Charlson Comorbidity Index (CCI)
      CCI adaptationDeyo
      • Deyo R.A.
      • Cherkin D.C.
      • Ciol M.A.
      Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases.
      Romano
      • Romano P.S.
      • Roos L.L.
      • Jollis J.G.
      Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives.
      ,
      • Romano P.S.
      • Roos L.L.
      • Jollis J.G.
      Further evidence concerning the use of a clinical comorbidity index with ICD-9-CM administrative data.
      D'Hoore
      • D'Hoore W.
      • Sicotte C.
      • Tilquin C.
      Risk adjustment in outcome assessment: the Charlson comorbidity index.
      ,
      • D'Hoore W.
      • Bouckaert A.
      • Tilquin C.
      Practical considerations on the use of the Charlson comorbidity index with administrative data bases.
      Ghali
      • Ghali W.A.
      • Hall R.E.
      • Rosen A.K.
      • Ash A.S.
      • Moskowitz M.A.
      Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data.
      Quan
      • Quan H.
      • Sundararajan V.
      • Halfon P.
      • Fong A.
      • Burnand B.
      • Luthi J.C.
      • et al.
      Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
      ,
      • Quan H.
      • Li B.
      • Couris C.M.
      • Fushimi K.
      • Graham P.
      • Hider P.
      • et al.
      Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.
      Study populationLumbar spine surgery patients, United States, 1985 (N = 27,111)
      • 1.
        CABG surgery patients, Manitoba, 1980–1992 (N = 4,121)
      • 2.
        Lumbar discectomy patients, California State Hospital, 1988–1990 (N = 55,407)
      Patients hospitalized with ischemic heart disease, CHF, stroke, or bacterial pneumonia, Quebec, 1989–1990 (N = 62,456)CABG surgery patients, Massachusetts, 1990 (N = 6,326)Hospitalized patients from the Calgary Health Region; 2001–2002 (N = 56,585) and 2002–2003 (N = 58,805)
      Indicates original Quan CCI developed in 2005 [38].
      2004 (N = 55,929)
      Indicates updated Quan CCI developed in 2011 [39].
      Data source, diagnostic codesMedicare, United States; ICD-9-CM diagnosis and procedure codes
      • 1.
        Manitoba Health Services Commission files. ICD-9-CM codes, up to 16 diagnoses
      • 2.
        California Office State-wide Health Planning and Developments Hospital discharge data. ICD-9-CM codes, up to 24 secondary diagnoses, up to 25 procedures
      Quebec MED-ECHO hospital database. First 3 digits of ICD-9 code without CM, in principal diagnosis, up to 15 secondary diagnoses and up to 5 proceduresMassachusetts Health Data Consortium discharge abstracts. ICD-9-CM codes, up to 15 diagnoses and 15 proceduresCalgary Health Region Hospitalization data. ICD-9-CM (2001–2002) and ICD-10-CA (2002–2003) codes,
      The Calgary Health Region has coded diagnostic data using ICD-10-CA since April 1, 2002.
      up to 16 diagnoses.
      Indicates original Quan CCI developed in 2005 [38].
      ICD-10-CA, up to 25 diagnoses and 20 procedures.
      Indicates updated Quan CCI developed in 2011 [39].
      Outcome predictedPostoperative mortality (in-hospital or 6 wk after discharge), postoperative complications, LOS, hospital chargesIn-hospital mortality; in-hospital complicationsIn-hospital mortalityIn-hospital mortalityIn-hospital mortality
      Description of CCI modificationIdentification of ICD-9-CM codes for all 19 CCI conditions; leukemia and lymphoma combined with other malignancies; use of original CCI weights.Use of the same comorbid conditions and weights as Deyo CCI; with broader definitions for PVD, complicated diabetes, and malignancy.Identification of ICD-9 codes without CM for all 19 conditions of original CCI; use of original CCI weights.Use of Deyo's ICD codes includes only the 5 comorbidities associated with mortality in their study population; assigned study-derived weights.Identification of ICD-10 codes and enhanced ICD-9-CM codes for Deyo CCI comorbidities; use of original CCI weights.
      Indicates original Quan CCI developed in 2005 [38].
      Assigned study-derived weights to original Quan CCI; includes only 12 comorbidities associated with mortality.
      Indicates updated Quan CCI developed in 2011 [39].
      Original CCIDeyo and Romano comorbid conditionsD'Hoore comorbid conditionsGhaliQuan
      Comorbid conditions includedWeightComorbid conditionsNew weightComorbid conditionsNew weight
      Myocardial infarct1XXX1
      Ghali divided MI into old and new MI, only new MI was found to be associated with in-hospital mortality in the study population.
      X
      Indicates original Quan CCI developed in 2005 [38].
      Indicates updated Quan CCI developed in 2011 [39].
      0
      Congestive heart failure1XXX4XX2
      Peripheral vascular disease1XXX2X0
      Cerebrovascular disease1XXX1X0
      Dementia1XX0XX2
      Chronic pulmonary disease1XX0XX1
      Connective tissue disease1XX0XX1
      Ulcer disease1XX0X0
      Mild liver disease1XX0XX2
      Diabetes1XX0X0
      Hemiplegia2XX0XX2
      Moderate or severe renal disease2XXX3XX1
      Diabetes with end-organ damage2XX0XX1
      Any tumor2XX0XX2
      Leukemia2X
      Lymphoma2X
      Moderate or severe liver disease3XX0XX4
      Metastatic solid tumor6XX0XX6
      AIDS6XX0XX4
      Abbreviations: CCI, Charlson Comorbidity Index; CABG, coronary artery bypass grafting; CHF, congestive heart failure; ICD-9-CM, International Classification of Disease, Ninth Revision, Clinical Modification; ICD-10-CA, International Classification of Disease and Related Health Problems, 10th Revision, Canada; LOS, length of stay; PVD, peripheral vascular disease; AIDS, acquired immunodeficiency syndrome; MI, myocardial infarction.
      a Indicates original Quan CCI developed in 2005
      • Quan H.
      • Sundararajan V.
      • Halfon P.
      • Fong A.
      • Burnand B.
      • Luthi J.C.
      • et al.
      Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
      .
      b Indicates updated Quan CCI developed in 2011
      • Quan H.
      • Li B.
      • Couris C.M.
      • Fushimi K.
      • Graham P.
      • Hider P.
      • et al.
      Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.
      .
      c The Calgary Health Region has coded diagnostic data using ICD-10-CA since April 1, 2002.
      d Ghali divided MI into old and new MI, only new MI was found to be associated with in-hospital mortality in the study population.

      3.2.1.1 Deyo CCI

      In 1992, Deyo et al. [
      • Deyo R.A.
      • Cherkin D.C.
      • Ciol M.A.
      Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases.
      ] adapted the CCI by identifying the ICD-9 codes corresponding to the 19 original comorbid conditions. The codes for leukemia and lymphoma were combined with the category “any malignancy” leaving the Deyo CCI as a list of 17 comorbid conditions [
      • Deyo R.A.
      • Cherkin D.C.
      • Ciol M.A.
      Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases.
      ] (Table 1).
      Eight studies have specifically evaluated the ability of the Deyo CCI to predict various outcomes [
      • Melfi C.
      • Holleman E.
      • Arthur D.
      • Katz B.
      Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data.
      ,
      • Poses R.M.
      • Smith W.R.
      • McClish D.K.
      • Anthony M.
      Controlling for confounding by indication for treatment. Are administrative data equivalent to clinical data?.
      ,
      • Wang P.S.
      • Walker A.
      • Tsuang M.
      • Orav E.J.
      • Levin R.
      • Avorn J.
      Strategies for improving comorbidity measures based on Medicare and Medicaid claims data.
      ,
      • Sundararajan V.
      • Henderson T.
      • Perry C.
      • Muggivan A.
      • Quan H.
      • Ghali W.A.
      New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality.
      ,
      • Nuttall M.
      • van der Meulen J.
      • Emberton M.
      Charlson scores based on ICD-10 administrative data were valid in assessing comorbidity in patients undergoing urological cancer surgery.
      ,
      • Hollenbeak C.S.
      • Stack Jr., B.C.
      • Daley S.M.
      • Piccirillo J.F.
      Using comorbidity indexes to predict costs for head and neck cancer.
      ,
      • Sessler D.I.
      • Sigl J.C.
      • Manberg P.J.
      • Kelley S.D.
      • Schubert A.
      • Chamoun N.G.
      Broadly applicable risk stratification system for predicting duration of hospitalization and mortality.
      ,
      • Liu J.
      • Huang Z.
      • Gilbertson D.T.
      • Foley R.N.
      • Collins A.J.
      An improved comorbidity index for outcome analyses among dialysis patients.
      ,
      • Bernabeu-Wittel M.
      • Ollero-Baturone M.
      • Moreno-Gavino L.
      • Baron-Franco B.
      • Fuertes A.
      • Murcia-Zaragoza J.
      • et al.
      Development of a new predictive model for polypathological patients. The PROFUND index.
      ,
      • Austin P.C.
      • van Walraven C.
      • Wodchis W.P.
      • Newman A.
      • Anderson G.M.
      Using the Johns Hopkins Aggregated Diagnosis Groups (ADGs) to predict mortality in a general adult population cohort in Ontario, Canada.
      ]. The Deyo CCI's ability to predict mortality ranged from poor to excellent, with C statistics ranging from 0.64 to 0.86 for in-hospital mortality and 0.59–0.85 for 1-year mortality. A number of studies demonstrated that other indices or risk scores performed better than the Deyo CCI in predicting mortality [
      • Melfi C.
      • Holleman E.
      • Arthur D.
      • Katz B.
      Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data.
      ,
      • Poses R.M.
      • Smith W.R.
      • McClish D.K.
      • Anthony M.
      Controlling for confounding by indication for treatment. Are administrative data equivalent to clinical data?.
      ,
      • Sessler D.I.
      • Sigl J.C.
      • Manberg P.J.
      • Kelley S.D.
      • Schubert A.
      • Chamoun N.G.
      Broadly applicable risk stratification system for predicting duration of hospitalization and mortality.
      ,
      • Bernabeu-Wittel M.
      • Ollero-Baturone M.
      • Moreno-Gavino L.
      • Baron-Franco B.
      • Fuertes A.
      • Murcia-Zaragoza J.
      • et al.
      Development of a new predictive model for polypathological patients. The PROFUND index.
      ,
      • Austin P.C.
      • van Walraven C.
      • Wodchis W.P.
      • Newman A.
      • Anderson G.M.
      Using the Johns Hopkins Aggregated Diagnosis Groups (ADGs) to predict mortality in a general adult population cohort in Ontario, Canada.
      ], length of stay (LOS) [
      • Melfi C.
      • Holleman E.
      • Arthur D.
      • Katz B.
      Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data.
      ,
      • Sessler D.I.
      • Sigl J.C.
      • Manberg P.J.
      • Kelley S.D.
      • Schubert A.
      • Chamoun N.G.
      Broadly applicable risk stratification system for predicting duration of hospitalization and mortality.
      ], or costs [
      • Hollenbeak C.S.
      • Stack Jr., B.C.
      • Daley S.M.
      • Piccirillo J.F.
      Using comorbidity indexes to predict costs for head and neck cancer.
      ]. The Deyo CCI demonstrated better ability to predict 1-year mortality when both prior inpatient and outpatient data were used to calculate the index [
      • Wang P.S.
      • Walker A.
      • Tsuang M.
      • Orav E.J.
      • Levin R.
      • Avorn J.
      Strategies for improving comorbidity measures based on Medicare and Medicaid claims data.
      ]. In 2004, the Deyo CCI was modified for use with ICD-10 codes, which performed similarly to the original ICD-9 version in predicting in-hospital mortality [
      • Sundararajan V.
      • Henderson T.
      • Perry C.
      • Muggivan A.
      • Quan H.
      • Ghali W.A.
      New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality.
      ].

      3.2.1.2 Romano CCI

      The Romano CCI, originally known as the Dartmouth-Manitoba CCI, adapted the CCI for use with administrative data. The identification of corresponding International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) codes was first done by Roos et al. [
      • Roos L.L.
      • Sharp S.M.
      • Cohen M.M.
      • Wajda A.
      Risk adjustment in claims-based research: the search for efficient approaches.
      ] in 1989 and was subsequently revised and modified by Romano et al. [
      • Romano P.S.
      • Roos L.L.
      • Jollis J.G.
      Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives.
      ,
      • Romano P.S.
      • Roos L.L.
      • Jollis J.G.
      Further evidence concerning the use of a clinical comorbidity index with ICD-9-CM administrative data.
      ] in 1993 to become what is known as the Romano CCI. Compared with the Deyo CCI, the Romano adaptation includes broader definitions, encompassing more ICD-9-CM codes, for peripheral vascular diseases, complicated diabetes, and malignancy. Romano et al. evaluated the Romano and Deyo versions of the CCI in patients who underwent coronary artery bypass grafting (CABG) surgery in Manitoba and lumbar discectomy in California (Table 1). Although no direct comparison of the two adaptations was tested, both demonstrated similar ability to predict outcomes related to comorbidity, when evaluated in the same population and using the same outcome. However, the risk estimate for each comorbidity varied widely across populations. Accordingly, Romano et al. [
      • Romano P.S.
      • Roos L.L.
      • Jollis J.G.
      Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives.
      ] recommended that investigators use data from their own study population to reestimate the weights assigned to each comorbidity.
      Three studies have modified and/or evaluated the Romano CCI for its ability to predict various outcomes [
      • Roos L.L.
      • Stranc L.
      • James R.C.
      • Li J.
      Complications, comorbidities, and mortality: improving classification and prediction.
      ,
      • Harse J.D.
      • Holman C.D.
      Charlson's index was a poor predictor of quality of life outcomes in a study of patients following joint replacement surgery.
      ,
      • Radley D.C.
      • Gottlieb D.J.
      • Fisher E.S.
      • Tosteson A.N.A.
      Comorbidity risk-adjustment strategies are comparable among persons with hip fracture.
      ]. Roos et al. created an augmented version of the Romano CCI, which demonstrated improved predictive ability compared with the original (Table 1/Appendix A at www.jclinepi.com). However, the authors cautioned that this augmented index may factor in complications resulting in an overestimation of comorbidity [
      • Roos L.L.
      • Stranc L.
      • James R.C.
      • Li J.
      Complications, comorbidities, and mortality: improving classification and prediction.
      ]. In regression analyses, the Romano CCI was a poor predictor of postoperative change in health-related quality of life scores [
      • Harse J.D.
      • Holman C.D.
      Charlson's index was a poor predictor of quality of life outcomes in a study of patients following joint replacement surgery.
      ]. Romano CCI performed slightly lower than the other two instruments but still demonstrated acceptable predictive ability for 1-year mortality [
      • Radley D.C.
      • Gottlieb D.J.
      • Fisher E.S.
      • Tosteson A.N.A.
      Comorbidity risk-adjustment strategies are comparable among persons with hip fracture.
      ].
      Three studies directly compared the Romano and Deyo adaptations of the CCI [
      • Ghali W.A.
      • Hall R.E.
      • Rosen A.K.
      • Ash A.S.
      • Moskowitz M.A.
      Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data.
      ,
      • Cleves M.A.
      • Sanchez N.
      • Draheim M.
      Evaluation of two competing methods for calculating Charlson's comorbidity index when analyzing short-term mortality using administrative data.
      ,
      • Nuttall M.
      • van der Meulen J.
      • Emberton M.
      Charlson scores based on ICD-10 administrative data were valid in assessing comorbidity in patients undergoing urological cancer surgery.
      ]. The scores derived from both adaptations in each study demonstrated substantial or almost perfect agreement, indicating that the two comorbidity classifications are similar [
      • Ghali W.A.
      • Hall R.E.
      • Rosen A.K.
      • Ash A.S.
      • Moskowitz M.A.
      Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data.
      ]. Both adaptations demonstrated similar ability to predict mortality (Table 1/Appendix A at www.jclinepi.com); however, the Romano method was slightly superior for predicting mortality, and models with study-derived weights outperformed Charlson weighted models [
      • Cleves M.A.
      • Sanchez N.
      • Draheim M.
      Evaluation of two competing methods for calculating Charlson's comorbidity index when analyzing short-term mortality using administrative data.
      ]. ICD-10 adaptations of both the Deyo and Romano CCIs demonstrated acceptable predictive ability for 1-year mortality, with the Romano performing slightly better [
      • Nuttall M.
      • van der Meulen J.
      • Emberton M.
      Charlson scores based on ICD-10 administrative data were valid in assessing comorbidity in patients undergoing urological cancer surgery.
      ].

      3.2.1.3 D'Hoore CCI

      D'Hoore et al. created a CCI adaptation using only the first three digits of ICD-9 coding without CM, as many institutions outside the United States use ICD-9 codes without CM (which includes procedural codes and additional morbidity details). Because coding of the tailing digits in ICD-9 codes can lead to inconsistencies, they claim to have created a simpler and more reliable adaptation [
      • D'Hoore W.
      • Sicotte C.
      • Tilquin C.
      Risk adjustment in outcome assessment: the Charlson comorbidity index.
      ]. The D'Hoore index demonstrated excellent ability to predict in-hospital mortality in populations with a principal diagnosis of myocardial infarction, ischemic heart disease, and bacterial pneumonia but demonstrated poor discrimination in stroke and congestive heart failure populations [
      • D'Hoore W.
      • Bouckaert A.
      • Tilquin C.
      Practical considerations on the use of the Charlson comorbidity index with administrative data bases.
      ]. The APACHE II score showed better ability to predict in-hospital mortality than D'Hoore CCI in a cohort of intensive care unit patients [
      • Norena M.
      • Wong H.
      • Thompson W.D.
      • Keenan S.P.
      • Dodek P.M.
      Adjustment of intensive care unit outcomes for severity of illness and comorbidity scores.
      ].

      3.2.1.4 Ghali CCI

      Using Deyo's coding scheme, Ghali et al. [
      • Ghali W.A.
      • Hall R.E.
      • Rosen A.K.
      • Ash A.S.
      • Moskowitz M.A.
      Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data.
      ] created a study-specific index with a reduced number of comorbidities, by including only comorbidities found to be associated with in-hospital mortality (OR > 1.2), and with study-specific weights for each comorbidity, derived from multiple logistic regression analyses on the study sample used for index development. The Ghali CCI includes only five comorbidities. When tested on the same sample, it performed better than the Deyo CCI in predicting in-hospital mortality. However, the performance of the Ghali CCI improved further when the coefficients from the original CCI were used instead of the study-specific weights.

      3.2.1.5 Quan CCI

      In 2005, Quan et al. [
      • Quan H.
      • Sundararajan V.
      • Halfon P.
      • Fong A.
      • Burnand B.
      • Luthi J.C.
      • et al.
      Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
      ] identified the ICD-10 codes corresponding to the Deyo CCI coding algorithm and also expanded the selection of codes for each comorbidity, using physicians to assess the face validity of the selected ICD-10 codes.
      In 2007, Sundararajan et al. [
      • Sundararajan V.
      • Henderson T.
      • Perry C.
      • Muggivan A.
      • Quan H.
      • Ghali W.A.
      New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality.
      ] compared the Quan CCI with two ICD-10 adaptations: one developed previously by Sundararajan et al. [
      • Sundararajan V.
      • Henderson T.
      • Perry C.
      • Muggivan A.
      • Quan H.
      • Ghali W.A.
      New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality.
      ] and one developed by Halfon et al. [
      • Halfon P.
      • Eggli Y.
      • van Melle G.
      • Chevalier J.
      • Wasserfallen J.B.
      • Burnand B.
      Measuring potentially avoidable hospital readmissions.
      ], which is not included in our review because the adaptation was not validated. In cohorts from four countries, the Quan CCI was a better predictor of in-hospital mortality than the Sundararajan and Halfon adaptations [
      • Sundararajan V.
      • Quan H.
      • Halfan P.
      • Fushimi K.
      • Luthi J.-
      • Burnand B.
      • et al.
      Cross-national comparative performance of three versions of the ICD-10 Charlson index.
      ]. Hanley et al. [
      • Hanley G.E.
      • Morgan S.
      • Reid R.J.
      Explaining prescription drug use and expenditures using the adjusted clinical groups case-mix system in the population of British Columbia, Canada.
      ] compared the ability of the Quan CCI and adjusted clinical groups (ACGs) to predict medication use and found that the ACGs predicted better. Of note, ACGs use ICD-9 or ICD-10 codes to develop a composite measure of patient illness burden, estimated from the mix of conditions experienced for a defined interval.
      In 2011, Quan et al. [
      • Ghali W.A.
      • Hall R.E.
      • Rosen A.K.
      • Ash A.S.
      • Moskowitz M.A.
      Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data.
      ,
      • Quan H.
      • Li B.
      • Couris C.M.
      • Fushimi K.
      • Graham P.
      • Hider P.
      • et al.
      Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.
      ] created an updated version of their index and derived study-specific weights in a similar method to Ghali's. The updated index includes 12 comorbidities with new weights assigned to each [Table 1] and demonstrated slightly better ability to predict in-hospital, 30-day, and 1-year mortality than the original Quan CCI [
      • Quan H.
      • Li B.
      • Couris C.M.
      • Fushimi K.
      • Graham P.
      • Hider P.
      • et al.
      Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.
      ] (Table 1/Appendix A at www.jclinepi.com).

      3.2.1.6 CCI adaptations for administrative data

      Six studies have developed or evaluated study-specific CCI adaptations for use with administrative data [
      • Librero J.
      • Peiro S.
      • Ordinana R.
      Chronic comorbidity and outcomes of hospital care: length of stay, mortality, and readmission at 30 and 365 days.
      ,
      • Klabunde C.N.
      • Potosky A.L.
      • Legler J.M.
      • Warren J.L.
      Development of a comorbidity index using physician claims data.
      ,
      • Holman C.D.
      • Preen D.B.
      • Baynham N.J.
      • Finn J.C.
      • Semmens J.B.
      A multipurpose comorbidity scoring system performed better than the Charlson index.
      ,
      • Martins M.
      • Blais R.
      Evaluation of comorbidity indices for inpatient mortality prediction models.
      ,
      • Klabunde C.N.
      • Legler J.M.
      • Warren J.L.
      • Baldwin L.-
      • Schrag D.
      A refined comorbidity measurement algorithm for claims-based studies of breast, prostate, colorectal, and lung cancer patients.
      ]. Two study-specific CCIs demonstrated poor predictive ability for 30-day readmission [
      • Librero J.
      • Peiro S.
      • Ordinana R.
      Chronic comorbidity and outcomes of hospital care: length of stay, mortality, and readmission at 30 and 365 days.
      ,
      • Holman C.D.
      • Preen D.B.
      • Baynham N.J.
      • Finn J.C.
      • Semmens J.B.
      A multipurpose comorbidity scoring system performed better than the Charlson index.
      ]. Others demonstrated acceptable-to-excellent ability to predict mortality [
      • Holman C.D.
      • Preen D.B.
      • Baynham N.J.
      • Finn J.C.
      • Semmens J.B.
      A multipurpose comorbidity scoring system performed better than the Charlson index.
      ,
      • Martins M.
      • Blais R.
      Evaluation of comorbidity indices for inpatient mortality prediction models.
      ,
      • Khan N.F.
      • Perera R.
      • Harper S.
      • Rose P.W.
      Adaptation and validation of the Charlson index for Read/OXMIS coded databases.
      ].
      Martins and Blais developed a study-specific index including 23 comorbidities, eight from the original Charlson index and 13 identified as frequent comorbidities in the study population. This index demonstrated superior performance for predicting in-hospital mortality compared with a CCI adaptation (specific adaptation not specified) [
      • Martins M.
      • Blais R.
      Evaluation of comorbidity indices for inpatient mortality prediction models.
      ].
      Klabunde et al. created two new indices referred to as the National Cancer Index (NCI)—one using inpatient claims and the other using outpatient claims—by assigning weights to the comorbidities from the original Charlson index based on a Cox proportional hazards model predicting 2-year noncancer mortality. It was evaluated for its ability to predict future treatment in both prostate and breast cancer populations, demonstrating acceptable and excellent predictive ability, respectively [
      • Klabunde C.N.
      • Potosky A.L.
      • Legler J.M.
      • Warren J.L.
      Development of a comorbidity index using physician claims data.
      ]. The index was revised by combining the score from inpatient and outpatient data and was further evaluated, compared with the original version, a uniform weights index and the CCI (specific adaptation not specified), for its ability to predict 2-year noncancer mortality in four cohorts with breast, prostate, colorectal, and lung cancer. The new NCI demonstrated similar predictive ability to the original NCI, and both versions performed better than the CCI or the uniform weight index in all four study populations [
      • Klabunde C.N.
      • Legler J.M.
      • Warren J.L.
      • Baldwin L.-
      • Schrag D.
      A refined comorbidity measurement algorithm for claims-based studies of breast, prostate, colorectal, and lung cancer patients.
      ] (Table 1/Appendix A at www.jclinepi.com).

      3.2.1.7 Comparison of self-report vs. administrative data-derived CCI

      Three studies compared the predictive ability of CCI adaptations derived from self-reported data with the same index derived from administrative data [
      • Zhang J.X.
      • Iwashyna T.J.
      • Christakis N.A.
      The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims.
      ,
      • Susser S.R.
      • McCusker J.
      • Belzile E.
      Comorbidity information in older patients at an emergency visit: self-report vs. administrative data had poor agreement but similar predictive validity.
      ,
      • Ronksley P.E.
      • Tsai W.H.
      • Quan H.
      • Faris P.
      • Hemmelgarn B.R.
      Data enhancement for co-morbidity measurement among patients referred for sleep diagnostic testing: an observational study.
      ]. Two studies [
      • Zhang J.X.
      • Iwashyna T.J.
      • Christakis N.A.
      The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims.
      ,
      • Susser S.R.
      • McCusker J.
      • Belzile E.
      Comorbidity information in older patients at an emergency visit: self-report vs. administrative data had poor agreement but similar predictive validity.
      ] found that self-reported data and administrative data adaptations had similar ability to predict various outcomes. Ronksley et al. [
      • Ronksley P.E.
      • Tsai W.H.
      • Quan H.
      • Faris P.
      • Hemmelgarn B.R.
      Data enhancement for co-morbidity measurement among patients referred for sleep diagnostic testing: an observational study.
      ] found that self-report of comorbid conditions had varying levels of agreement with those derived from administrative data, ranging from poor to substantial agreement depending on the comorbid condition (κ = 0.14–0.79).

      3.2.1.8 Comparison of chart review vs. administrative data-derived CCI

      In 2010, Leal and Laupland [
      • Leal J.R.
      • Laupland K.B.
      Validity of ascertainment of co-morbid illness using administrative databases: a systematic review.
      ] conducted a systematic review of studies comparing CCI adaptations derived from administrative data and chart review. They found that CCI scores calculated from administrative data were consistently lower than those derived from chart review, and agreement between the two sources was poor to fair (κ ranging from 0.30 to 0.56) [
      • Leal J.R.
      • Laupland K.B.
      Validity of ascertainment of co-morbid illness using administrative databases: a systematic review.
      ]. Two additional studies have compared chart review vs. administrative data-derived CCIs [
      • Seo H.J.
      • Yoon S.J.
      • Lee S.I.
      • Lee K.S.
      • Yun Y.H.
      • Kim E.J.
      • et al.
      A comparison of the Charlson comorbidity index derived from medical records and claims data from patients undergoing lung cancer surgery in Korea: a population-based investigation.
      ,
      • Jang S.H.
      • Chea J.W.
      • Lee K.B.
      Charlson comorbidity index using administrative database in incident PD patients.
      ]. One study, evaluating the Quan CCI, found that kappa agreement ranged greatly (from 0.02 to 0.47) according to the comorbidity identified [
      • Zhang J.X.
      • Iwashyna T.J.
      • Christakis N.A.
      The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims.
      ]. Another study [
      • Jang S.H.
      • Chea J.W.
      • Lee K.B.
      Charlson comorbidity index using administrative database in incident PD patients.
      ], evaluating a study-specific administrative data CCI using ICD-10 codes, found CCI scores derived from the two sources to be well correlated (r = 0.88, P < 0.01) [
      • Jang S.H.
      • Chea J.W.
      • Lee K.B.
      Charlson comorbidity index using administrative database in incident PD patients.
      ].

      3.2.2 Elixhauser comorbidity index

      Elixhauser et al. developed a comorbidity index comprised of a comprehensive set of 30 comorbidities defined using ICD-9-CM codes from administrative data (Table 2/Appendix B at www.jclinepi.com). The EI comorbidities were significant predictors of LOS and hospital charges. Many of the individual EI comorbidities were associated with in-hospital mortality, but as a group, the association was not significant [
      • Elixhauser A.
      • Steiner C.
      • Harris D.R.
      • Coffey R.M.
      Comorbidity measures for use with administrative data.
      ]. One disadvantage of the original EI is that it includes 30 dichotomous variables, representing each comorbidity, without a weighting system to provide a single score.
      Three studies validated the EI for its ability to predict mortality, with two providing a modification to the EI [
      • Quan H.
      • Sundararajan V.
      • Halfon P.
      • Fong A.
      • Burnand B.
      • Luthi J.C.
      • et al.
      Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
      ,
      • Zhu H.
      • Hill M.D.
      Stroke: the Elixhauser index for comorbidity adjustment of in-hospital case fatality.
      ,
      • van Walraven C.
      • Austin P.C.
      • Jennings A.
      • Quan H.
      • Forster A.J.
      A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data.
      ]. Additional studies have validated and compared the EI with other indices (Section 3.5). In predicting in-hospital mortality, all EI versions demonstrated acceptable-to-excellent predictive ability [
      • Quan H.
      • Sundararajan V.
      • Halfon P.
      • Fong A.
      • Burnand B.
      • Luthi J.C.
      • et al.
      Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
      ,
      • Zhu H.
      • Hill M.D.
      Stroke: the Elixhauser index for comorbidity adjustment of in-hospital case fatality.
      ,
      • van Walraven C.
      • Austin P.C.
      • Jennings A.
      • Quan H.
      • Forster A.J.
      A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data.
      ]. In the Quan study, the enhanced ICD-9-CM version performed the best, followed by the ICD-10 version and the original EI [
      • Quan H.
      • Sundararajan V.
      • Halfon P.
      • Fong A.
      • Burnand B.
      • Luthi J.C.
      • et al.
      Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
      ]. Van Walraven et al. developed a scoring system for the EI, using the regression coefficients for each comorbidity from a multivariate logistic regression model predicting in-hospital mortality. It demonstrated acceptable prediction of in-hospital mortality, with similar results for models using the EI including all comorbidities and the EI including only the 21 comorbidities significantly associated with mortality [
      • van Walraven C.
      • Austin P.C.
      • Jennings A.
      • Quan H.
      • Forster A.J.
      A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data.
      ].

      3.3 Other diagnosis-based indices

      Two studies developed study-specific diagnosis-based indices, which were not based on the EI or CCI [
      • Fleming S.T.
      • Pearce K.A.
      • McDavid K.
      • Pavlov D.
      The development and validation of a comorbidity index for prostate cancer among Black men.
      ,
      • Abildstrom S.Z.
      • Hvelplund A.
      • Rasmussen S.
      • Nielsen P.H.
      • Mortensen P.E.
      • Kruse M.
      Prognostic information in administrative co-morbidity data following coronary artery bypass grafting.
      ]. Using hospitalization and outpatient data from the Surveillance, Epidemiology, and End Results—Medicare linked database, Fleming et al. developed an index with 27 comorbidity categories based on the prevalence of diseases specifically in their study population, black males with prostate cancer [
      • Fleming S.T.
      • Pearce K.A.
      • McDavid K.
      • Pavlov D.
      The development and validation of a comorbidity index for prostate cancer among Black men.
      ]. Abildstrom et al. [
      • Abildstrom S.Z.
      • Hvelplund A.
      • Rasmussen S.
      • Nielsen P.H.
      • Mortensen P.E.
      • Kruse M.
      Prognostic information in administrative co-morbidity data following coronary artery bypass grafting.
      ] created a comorbidity index based on administrative data to predict 30-day mortality after CABG. Their index performed similarly to the additive EuroSCORE registered in a clinical database.

      3.4 Diagnosis-based indices comparison studies

      Fourteen studies compared the EI to various CCI adaptations [
      • Quan H.
      • Li B.
      • Couris C.M.
      • Fushimi K.
      • Graham P.
      • Hider P.
      • et al.
      Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.
      ,
      • Austin P.C.
      • van Walraven C.
      • Wodchis W.P.
      • Newman A.
      • Anderson G.M.
      Using the Johns Hopkins Aggregated Diagnosis Groups (ADGs) to predict mortality in a general adult population cohort in Ontario, Canada.
      ,
      • van Walraven C.
      • Austin P.C.
      • Jennings A.
      • Quan H.
      • Forster A.J.
      A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data.
      ,
      • Stukenborg G.J.
      • Wagner D.P.
      • Connors AF Jr.,
      Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations.
      ,
      • Southern D.A.
      • Quan H.
      • Ghali W.A.
      Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data.
      ,
      • Yan Y.
      • Birman-Deych E.
      • Radford M.J.
      • Nilasena D.S.
      • Gage B.F.
      Comorbidity indices to predict mortality from Medicare data: results from the national registry of atrial fibrillation.
      ,
      • Kurichi J.E.
      • Stineman M.G.
      • Kwong P.L.
      • Bates B.E.
      • Reker D.M.
      Assessing and using comorbidity measures in elderly veterans with lower extremity amputations.
      ,
      • Livingston E.H.
      Development of bariatric surgery-specific risk assessment tool.
      ,
      • Machnicki G.
      • Pinsky B.
      • Takemoto S.
      • Balshaw R.
      • Salvalaggio P.R.
      • Buchanan P.M.
      • et al.
      Predictive ability of pretransplant comorbidities to predict long-term graft loss and death.
      ,
      • Chu Y.T.
      • Ng Y.Y.
      • Wu S.C.
      Comparison of different comorbidity measures for use with administrative data in predicting short- and long-term mortality.
      ,
      • Martins M.
      Use of comorbidity measures to predict the risk of death in Brazilian in-patients.
      ,
      • Li P.
      • Kim M.M.
      • Doshi J.A.
      Comparison of the performance of the CMS Hierarchical Condition Category (CMS-HCC) risk adjuster with the Charlson and Elixhauser comorbidity measures in predicting mortality.
      ,
      • Lieffers J.R.
      • Baracos V.E.
      • Winget M.
      • Fassbender K.
      A comparison of Charlson and Elixhauser comorbidity measures to predict colorectal cancer survival using administrative health data.
      ,
      • Gagne J.J.
      • Glynn R.J.
      • Avorn J.
      • Levin R.
      • Schneeweiss S.
      A combined comorbidity score predicted mortality in elderly patients better than existing scores.
      ,
      • Austin P.C.
      • Shah B.R.
      • Newman A.
      • Anderson G.M.
      Using the Johns Hopkins' Aggregated Diagnosis Groups (ADGs) to predict 1-year mortality in population-based cohorts of patients with diabetes in Ontario, Canada.
      ,
      • Mnatzaganian G.
      • Ryan P.
      • Norman P.E.
      • Hiller J.E.
      Accuracy of hospital morbidity data and the performance of comorbidity scores as predictors of mortality.
      ]. A description of the cohorts and results for each study are found in Table 3/Appendix C at www.jclinepi.com. Overall, both the EI and CCI demonstrated poor-to-excellent ability to predict various outcomes. When predicting in-hospital mortality, C statistics ranged from 0.632 to 0.878 and 0.608–0.860 for the EI and CCI, respectively. For 1-year mortality, results ranged from 0.69 to 0.909 and 0.65–0.906, respectively. Nine studies [
      • Quan H.
      • Sundararajan V.
      • Halfon P.
      • Fong A.
      • Burnand B.
      • Luthi J.C.
      • et al.
      Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
      ,
      • van Walraven C.
      • Austin P.C.
      • Jennings A.
      • Quan H.
      • Forster A.J.
      A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data.
      ,
      • Stukenborg G.J.
      • Wagner D.P.
      • Connors AF Jr.,
      Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations.
      ,
      • Southern D.A.
      • Quan H.
      • Ghali W.A.
      Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data.
      ,
      • Kurichi J.E.
      • Stineman M.G.
      • Kwong P.L.
      • Bates B.E.
      • Reker D.M.
      Assessing and using comorbidity measures in elderly veterans with lower extremity amputations.
      ,
      • Livingston E.H.
      Development of bariatric surgery-specific risk assessment tool.
      ,
      • Chu Y.T.
      • Ng Y.Y.
      • Wu S.C.
      Comparison of different comorbidity measures for use with administrative data in predicting short- and long-term mortality.
      ,
      • Lieffers J.R.
      • Baracos V.E.
      • Winget M.
      • Fassbender K.
      A comparison of Charlson and Elixhauser comorbidity measures to predict colorectal cancer survival using administrative health data.
      ,
      • Mnatzaganian G.
      • Ryan P.
      • Norman P.E.
      • Hiller J.E.
      Accuracy of hospital morbidity data and the performance of comorbidity scores as predictors of mortality.
      ] demonstrated that various versions of the EI (including the original, Quan, and van Walraven adaptations) predicted mortality outcomes better than various adaptations of the CCI (Deyo, Romano, and Quan adaptations). In contrast, four studies found no difference [
      • Austin P.C.
      • van Walraven C.
      • Wodchis W.P.
      • Newman A.
      • Anderson G.M.
      Using the Johns Hopkins Aggregated Diagnosis Groups (ADGs) to predict mortality in a general adult population cohort in Ontario, Canada.
      ,
      • Machnicki G.
      • Pinsky B.
      • Takemoto S.
      • Balshaw R.
      • Salvalaggio P.R.
      • Buchanan P.M.
      • et al.
      Predictive ability of pretransplant comorbidities to predict long-term graft loss and death.
      ,
      • Martins M.
      Use of comorbidity measures to predict the risk of death in Brazilian in-patients.
      ,
      • Gagne J.J.
      • Glynn R.J.
      • Avorn J.
      • Levin R.
      • Schneeweiss S.
      A combined comorbidity score predicted mortality in elderly patients better than existing scores.
      ]. Only one study found that a CCI adaptation (Romano CCI) predicted mortality better than the EI [
      • Li P.
      • Kim M.M.
      • Doshi J.A.
      Comparison of the performance of the CMS Hierarchical Condition Category (CMS-HCC) risk adjuster with the Charlson and Elixhauser comorbidity measures in predicting mortality.
      ]. Three studies examined the effect of using inpatient and/or outpatient data on predicting mortality and found that combining data from both sources resulted in higher C statistics [
      • Kurichi J.E.
      • Stineman M.G.
      • Kwong P.L.
      • Bates B.E.
      • Reker D.M.
      Assessing and using comorbidity measures in elderly veterans with lower extremity amputations.
      ,
      • Li P.
      • Kim M.M.
      • Doshi J.A.
      Comparison of the performance of the CMS Hierarchical Condition Category (CMS-HCC) risk adjuster with the Charlson and Elixhauser comorbidity measures in predicting mortality.
      ,
      • Baldwin L.M.
      • Klabunde C.N.
      • Green P.
      • Barlow W.
      • Wright G.
      In search of the perfect comorbidity measure for use with administrative claims data: does it exist?.
      ]. Lieffers et al. augmented the EI by adding performance status and substituting clinical data instead of administrative records for body weight. This augmented version of EI predicted 2- and 3-year survival in patients with stages II–IV colorectal cancer better than the Quan EI. Another study evaluated a combination of the van Walraven EI and the Romano CCI and found the combined index predicted mortality better than each individual index [
      • Gagne J.J.
      • Glynn R.J.
      • Avorn J.
      • Levin R.
      • Schneeweiss S.
      A combined comorbidity score predicted mortality in elderly patients better than existing scores.
      ].

      3.5 Medication-based indices

      Medication-based indices use pharmacy data to identify comorbidities by linking medications to specific disease categories. Fourteen studies examined the development or validation of these indices [
      • Von Korff M.
      • Wagner E.H.
      • Saunders K.
      A chronic disease score from automated pharmacy data.
      ,
      • Johnson R.E.
      • Hornbrook M.C.
      • Nichols G.A.
      Replicating the chronic disease score (CDS) from automated pharmacy data.
      ,
      • Clark D.O.
      • Von Korff M.
      • Saunders K.
      • Baluch W.M.
      • Simon G.E.
      A chronic disease score with empirically derived weights.
      ,
      • Fishman P.A.
      • Shay D.K.
      Development and estimation of a pediatric chronic disease score using automated pharmacy data.
      ,
      • Lamers L.M.
      Pharmacy costs groups: a risk-adjuster for capitation payments based on the use of prescribed drugs.
      ,
      • Kaye K.S.
      • Sands K.
      • Donahue J.G.
      • Chan K.A.
      • Fishman P.
      • Platt R.
      Preoperative drug dispensing as predictor of surgical site infection.
      ,
      • Putnam K.G.
      • Buist D.S.
      • Fishman P.
      • Andrade S.E.
      • Boles M.
      • Chase G.A.
      • et al.
      Chronic disease score as a predictor of hospitalization.
      ,
      • Fishman P.A.
      • Goodman M.J.
      • Hornbrook M.C.
      • Meenan R.T.
      • Bachman D.J.
      • O'Keeffe Rosetti M.C.
      Risk adjustment using automated ambulatory pharmacy data: the RxRisk model.
      ,
      • Sloan K.L.
      • Sales A.E.
      • Liu C.F.
      • Fishman P.
      • Nichol P.
      • Suzuki N.T.
      • et al.
      Construction and characteristics of the RxRisk-V: a VA-adapted pharmacy-based case-mix instrument.
      ,
      • Joish V.N.
      • Malone D.C.
      • Wendel C.
      • Draugalis J.R.
      • Mohler M.J.
      Development and validation of a diabetes mellitus severity index: a risk-adjustment tool for predicting health care resource use and costs.
      ,
      • McGregor J.C.
      • Perencevich E.N.
      • Furuno J.P.
      • Langenberg P.
      • Flannery K.
      • Zhu J.
      • et al.
      Comorbidity risk-adjustment measures were developed and validated for studies of antibiotic-resistant infections.
      ,
      • George J.
      • Vuong T.
      • Bailey M.J.
      • Kong D.C.
      • Marriott J.L.
      • Stewart K.
      Development and validation of the medication-based disease burden index.
      ,
      • Vitry A.
      • Wong S.A.
      • Roughead E.E.
      • Ramsay E.
      • Barratt J.
      Validity of medication-based co-morbidity indices in the Australian elderly population.
      ], as summarized in Table 2/Appendix B at www.jclinepi.com.

      3.6 Chronic Disease Score

      In 1992, von Korff et al. [
      • Von Korff M.
      • Wagner E.H.
      • Saunders K.
      A chronic disease score from automated pharmacy data.
      ] created the CDS, using medications instead of diagnostic codes to identify comorbidities. Using a population-based pharmacy database, a panel of experts evaluated patterns of use of selected medications to create disease categories, and weights were assigned by consensus [
      • Von Korff M.
      • Wagner E.H.
      • Saunders K.
      A chronic disease score from automated pharmacy data.
      ,
      • Sloan K.L.
      • Sales A.E.
      • Liu C.F.
      • Fishman P.
      • Nichol P.
      • Suzuki N.T.
      • et al.
      Construction and characteristics of the RxRisk-V: a VA-adapted pharmacy-based case-mix instrument.
      ]. The original CDS included 17 diseases and was validated against chart review and physician rating of physical disease severity [
      • Von Korff M.
      • Wagner E.H.
      • Saunders K.
      A chronic disease score from automated pharmacy data.
      ]. Its ability to predict health outcomes was validated on two patient populations [
      • Von Korff M.
      • Wagner E.H.
      • Saunders K.
      A chronic disease score from automated pharmacy data.
      ,
      • Johnson R.E.
      • Hornbrook M.C.
      • Nichols G.A.
      Replicating the chronic disease score (CDS) from automated pharmacy data.
      ].
      Clark et al. [
      • Clark D.O.
      • Von Korff M.
      • Saunders K.
      • Baluch W.M.
      • Simon G.E.
      A chronic disease score with empirically derived weights.
      ] modified the original CDS by updating medications, expanding the disease categories to 28, and weighting the disease categories based on regression models. The CDS-2 ubiquitously replaced the original CDS, and adaptations were subsequently made for application to specific populations, including the Pediatric Chronic Disease Score [
      • Fishman P.A.
      • Shay D.K.
      Development and estimation of a pediatric chronic disease score using automated pharmacy data.
      ] and adaptations for diabetics by Joish et al. [
      • Joish V.N.
      • Malone D.C.
      • Wendel C.
      • Draugalis J.R.
      • Mohler M.J.
      Development and validation of a diabetes mellitus severity index: a risk-adjustment tool for predicting health care resource use and costs.
      ]. Other studies compared the predictive ability of the CDS-2 based on outpatient pharmacy data compared with in-hospital prescription data [
      • Kaye K.S.
      • Sands K.
      • Donahue J.G.
      • Chan K.A.
      • Fishman P.
      • Platt R.
      Preoperative drug dispensing as predictor of surgical site infection.
      ], stratified based on total prescription number [
      • Lamers L.M.
      Pharmacy costs groups: a risk-adjuster for capitation payments based on the use of prescribed drugs.
      ], and predicted surgical site or nosocomial infections in hospitalized patient populations [
      • Kaye K.S.
      • Sands K.
      • Donahue J.G.
      • Chan K.A.
      • Fishman P.
      • Platt R.
      Preoperative drug dispensing as predictor of surgical site infection.
      ,
      • McGregor J.C.
      • Perencevich E.N.
      • Furuno J.P.
      • Langenberg P.
      • Flannery K.
      • Zhu J.
      • et al.
      Comorbidity risk-adjustment measures were developed and validated for studies of antibiotic-resistant infections.
      ]. Fishman et al. [
      • Fishman P.A.
      • Goodman M.J.
      • Hornbrook M.C.
      • Meenan R.T.
      • Bachman D.J.
      • O'Keeffe Rosetti M.C.
      Risk adjustment using automated ambulatory pharmacy data: the RxRisk model.
      ] continued updating the CDS-2 to include new medication classes and expand disease categories, ultimately developing a new but related instrument, RxRisk.

      3.7 RxRisk and RxRisk-V

      The RxRisk was developed as an all-age risk assessment instrument using outpatient pharmacy data to identify chronic diseases and predict future health care costs [
      • Fishman P.A.
      • Goodman M.J.
      • Hornbrook M.C.
      • Meenan R.T.
      • Bachman D.J.
      • O'Keeffe Rosetti M.C.
      Risk adjustment using automated ambulatory pharmacy data: the RxRisk model.
      ]. The RxRisk included 57 adult and pediatric weighted disease categories and associated drug classes. Validation was based on a large general population sample using multiple measures of predictive power. The RxRisk-V was a subsequent modification adapted to the Veterans Health Administration population [
      • Sloan K.L.
      • Sales A.E.
      • Liu C.F.
      • Fishman P.
      • Nichol P.
      • Suzuki N.T.
      • et al.
      Construction and characteristics of the RxRisk-V: a VA-adapted pharmacy-based case-mix instrument.
      ].

      3.8 Medication-Based Disease Burden Index

      The final medication-based index is the Medication-Based Disease Burden Index (MDBI), developed as an alternative to the original CDS to deal with the same issues addressed by Clark's and Fishman's revisions [
      • George J.
      • Vuong T.
      • Bailey M.J.
      • Kong D.C.
      • Marriott J.L.
      • Stewart K.
      Development and validation of the medication-based disease burden index.
      ]. The MDBI showed weak correlation with the CCI and CDS, moderate ability to predict 12-week death and readmission [
      • George J.
      • Vuong T.
      • Bailey M.J.
      • Kong D.C.
      • Marriott J.L.
      • Stewart K.
      Development and validation of the medication-based disease burden index.
      ], and a poorer ability to predict 6-month mortality than the RxRisk-V [
      • Vitry A.
      • Wong S.A.
      • Roughead E.E.
      • Ramsay E.
      • Barratt J.
      Validity of medication-based co-morbidity indices in the Australian elderly population.
      ].

      3.9 Cross-index comparison

      Thirteen studies compared medication- and diagnosis-based indices [
      • Perkins A.J.
      • Kroenke K.
      • Unutzer J.
      • Katon W.
      • Williams J.W.
      • Hope C.
      • et al.
      Common comorbidity scales were similar in their ability to predict health care costs and mortality.
      ,
      • Schneeweiss S.
      • Seeger J.D.
      • Maclure M.
      • Wang P.S.
      • Avorn J.
      • Glynn R.J.
      Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data.
      ,
      • Schneeweiss S.
      • Wang P.S.
      • Avorn J.
      • Glynn R.J.
      Improved comorbidity adjustment for predicting mortality in Medicare populations.
      ,
      • Schneeweiss S.
      • Wang P.S.
      • Avorn J.
      • Maclure M.
      • Levin R.
      • Glynn R.J.
      Consistency of performance ranking of comorbidity adjustment scores in Canadian and U.S. utilization data.
      ,
      • Parker J.P.
      • McCombs J.S.
      • Graddy E.A.
      Can pharmacy data improve prediction of hospital outcomes? Comparisons with a diagnosis-based comorbidity measure.
      ,
      • McGregor J.C.
      • Kim P.W.
      • Perencevich E.N.
      • Bradham D.D.
      • Furuno J.P.
      • Kaye K.S.
      • et al.
      Utility of the chronic disease score and Charlson comorbidity index as comorbidity measures for use in epidemiologic studies of antibiotic-resistant organisms.
      ,
      • Farley J.F.
      • Harley C.R.
      • Devine J.W.
      A comparison of comorbidity measurements to predict healthcare expenditures.
      ,
      • Johnson M.L.
      • El-Serag H.B.
      • Tran T.T.
      • Hartman C.
      • Richardson P.
      • Abraham N.S.
      Adapting the Rx-Risk-V for mortality prediction in outpatient populations.
      ,
      • Fan V.S.
      • Maciejewski M.L.
      • Liu C.-F.
      • McDonell M.B.
      • Fihn S.D.
      Comparison of risk adjustment measures based on self-report, administrative data, and pharmacy records to predict clinical outcomes.
      ,
      • Dominick K.L.
      • Dudley T.K.
      • Coffman C.J.
      • Bosworth H.B.
      Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis.
      ,
      • Baser O.
      • Palmer L.
      • Stephenson J.
      The estimation power of alternative comorbidity indices.
      ,
      • Maciejewski M.L.
      • Liu C.-F.
      • Fihn S.D.
      Performance of comorbidity, risk adjustment, and functional status measures in expenditure prediction for patients with diabetes.
      ,
      • Bayliss E.A.
      • Ellis J.L.
      • Steiner J.F.
      Seniors' self-reported multimorbidity captured biopsychosocial factors not incorporated into two other data-based morbidity measures.
      ]. Schneeweiss et al. conducted three studies comparing medication- and diagnosis-based indices for their ability to predict various outcomes (Table 4/Appendix A at www.jclinepi.com). Two studies found the following performance ranking when predicting 1-year mortality, long-term care admissions, and hospitalizations: Romano CCI ≥ Deyo CCI > D'Hoore CCI > Ghali CCI > CDS-1 > CDS-2, but a different ranking when predicting physician visits or expenditures for physician visits: D'Hoore CCI > CDS-2 ≥ Romano CCI > Deyo CCI > CDS-1 > Ghali CCI [
      • Schneeweiss S.
      • Seeger J.D.
      • Maclure M.
      • Wang P.S.
      • Avorn J.
      • Glynn R.J.
      Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data.
      ,
      • Schneeweiss S.
      • Wang P.S.
      • Avorn J.
      • Maclure M.
      • Levin R.
      • Glynn R.J.
      Consistency of performance ranking of comorbidity adjustment scores in Canadian and U.S. utilization data.
      ]. Another study evaluated a study-specific adaptation of the Romano CCI, which derived its own study-specific weights and found that it predicted 1-year mortality better than the original Romano CCI and that both versions outperformed the CDS-1. However, the EI demonstrated the best predictive ability of the four indices compared [
      • Schneeweiss S.
      • Wang P.S.
      • Avorn J.
      • Glynn R.J.
      Improved comorbidity adjustment for predicting mortality in Medicare populations.
      ].
      Ten additional studies compared diagnosis- and medication-based indices [
      • Perkins A.J.
      • Kroenke K.
      • Unutzer J.
      • Katon W.
      • Williams J.W.
      • Hope C.
      • et al.
      Common comorbidity scales were similar in their ability to predict health care costs and mortality.
      ,
      • Parker J.P.
      • McCombs J.S.
      • Graddy E.A.
      Can pharmacy data improve prediction of hospital outcomes? Comparisons with a diagnosis-based comorbidity measure.
      ,
      • McGregor J.C.
      • Kim P.W.
      • Perencevich E.N.
      • Bradham D.D.
      • Furuno J.P.
      • Kaye K.S.
      • et al.
      Utility of the chronic disease score and Charlson comorbidity index as comorbidity measures for use in epidemiologic studies of antibiotic-resistant organisms.
      ,
      • Farley J.F.
      • Harley C.R.
      • Devine J.W.
      A comparison of comorbidity measurements to predict healthcare expenditures.
      ,
      • Johnson M.L.
      • El-Serag H.B.
      • Tran T.T.
      • Hartman C.
      • Richardson P.
      • Abraham N.S.
      Adapting the Rx-Risk-V for mortality prediction in outpatient populations.
      ,
      • Fan V.S.
      • Maciejewski M.L.
      • Liu C.-F.
      • McDonell M.B.
      • Fihn S.D.
      Comparison of risk adjustment measures based on self-report, administrative data, and pharmacy records to predict clinical outcomes.
      ,
      • Dominick K.L.
      • Dudley T.K.
      • Coffman C.J.
      • Bosworth H.B.
      Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis.
      ,
      • Baser O.
      • Palmer L.
      • Stephenson J.
      The estimation power of alternative comorbidity indices.
      ,
      • Maciejewski M.L.
      • Liu C.-F.
      • Fihn S.D.
      Performance of comorbidity, risk adjustment, and functional status measures in expenditure prediction for patients with diabetes.
      ,
      • Bayliss E.A.
      • Ellis J.L.
      • Steiner J.F.
      Seniors' self-reported multimorbidity captured biopsychosocial factors not incorporated into two other data-based morbidity measures.
      ] (Table 4/Appendix D at www.jclinepi.com). When predicting mortality outcomes, results were not consistent across studies. In one study [
      • Perkins A.J.
      • Kroenke K.
      • Unutzer J.
      • Katon W.
      • Williams J.W.
      • Hope C.
      • et al.
      Common comorbidity scales were similar in their ability to predict health care costs and mortality.
      ], the predictive ability of the diagnosis-based index (Deyo CCI) was better than the medication-based index (CDS-1); yet, another study demonstrated the opposite [
      • Johnson M.L.
      • El-Serag H.B.
      • Tran T.T.
      • Hartman C.
      • Richardson P.
      • Abraham N.S.
      Adapting the Rx-Risk-V for mortality prediction in outpatient populations.
      ] (RxRisk-V predicted mortality better than the Deyo CCI) and another found no difference between the same two indices [
      • Fan V.S.
      • Maciejewski M.L.
      • Liu C.-F.
      • McDonell M.B.
      • Fihn S.D.
      Comparison of risk adjustment measures based on self-report, administrative data, and pharmacy records to predict clinical outcomes.
      ]. Generally, medication-based indices demonstrated better ability to predict various health care utilization outcomes, including prescription medication use [
      • Dominick K.L.
      • Dudley T.K.
      • Coffman C.J.
      • Bosworth H.B.
      Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis.
      ], total costs [
      • Farley J.F.
      • Harley C.R.
      • Devine J.W.
      A comparison of comorbidity measurements to predict healthcare expenditures.
      ], disease burden [
      • Bayliss E.A.
      • Ellis J.L.
      • Steiner J.F.
      Seniors' self-reported multimorbidity captured biopsychosocial factors not incorporated into two other data-based morbidity measures.
      ], and hospitalizations [
      • Fan V.S.
      • Maciejewski M.L.
      • Liu C.-F.
      • McDonell M.B.
      • Fihn S.D.
      Comparison of risk adjustment measures based on self-report, administrative data, and pharmacy records to predict clinical outcomes.
      ]. However, the EI demonstrated better ability to predict physician visits than the RxRisk-V [
      • Dominick K.L.
      • Dudley T.K.
      • Coffman C.J.
      • Bosworth H.B.
      Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis.
      ]. Medication- and diagnosis-based indices demonstrated similar ability to predict hospital readmission and LOS [
      • Parker J.P.
      • McCombs J.S.
      • Graddy E.A.
      Can pharmacy data improve prediction of hospital outcomes? Comparisons with a diagnosis-based comorbidity measure.
      ], hospitalization [
      • Perkins A.J.
      • Kroenke K.
      • Unutzer J.
      • Katon W.
      • Williams J.W.
      • Hope C.
      • et al.
      Common comorbidity scales were similar in their ability to predict health care costs and mortality.
      ], spending [
      • Farley J.F.
      • Harley C.R.
      • Devine J.W.
      A comparison of comorbidity measurements to predict healthcare expenditures.
      ], and costs [
      • Maciejewski M.L.
      • Liu C.-F.
      • Fihn S.D.
      Performance of comorbidity, risk adjustment, and functional status measures in expenditure prediction for patients with diabetes.
      ].

      4. Discussion

      In this report, we summarize the published literature on the development and validity of comorbidity indices used in administrative data studies. The body of literature on this topic is broad, as we identified a total of 76 primary articles for inclusion.
      All indices identified could be grouped as either diagnosis based, using ICD coding, or medication based, using pharmacy dispensing data. The main diagnosis-based indices were the EI and the various adaptations of the CCI for use with administrative data. Medication-based indices included versions of the CDS, which later became known as the RxRisk, and its adaptation for use in the veteran population, the RxRisk-V.
      Of the diagnosis-based measures, we found that the EI consistently outperformed the CCI in predicting both short- and long-term mortality. Of the main adaptations of the CCI, the Romano CCI demonstrated equal or better performance in its ability to predict various outcomes compared with the Deyo CCI, despite the fact that the Deyo CCI is the more commonly used measure. Although both the EI and all administrative data adaptations of the CCI were developed for use with inpatient hospitalization data, several studies found that using combination of both inpatient and outpatient data consistently improved the performance of the index studied. Furthermore, a number of studies examined adaptations of the CCI and EI, which derived empirical weights based on the study-specific population and outcome measure. Assigning study-specific weights for both EI and CCI adaptations tended to improve their predictive performance. Accordingly, for diagnosis-based indices, we recommend the use of the EI or the Romano CCI, particularly when predicting mortality outcomes. When available, we recommend calculating these indices using a combination of both inpatient and outpatient data and, when possible, deriving study-specific empirically derived weights for the index selected.
      Of the medication-based indices, we found that the original version of the CDS developed by von Korff et al. [
      • Von Korff M.
      • Wagner E.H.
      • Saunders K.
      A chronic disease score from automated pharmacy data.
      ] tended to outperform the CDS-2, developed by Clark et al. [
      • Clark D.O.
      • Von Korff M.
      • Saunders K.
      • Baluch W.M.
      • Simon G.E.
      A chronic disease score with empirically derived weights.
      ]. However, the later version known as the RxRisk-V was the most commonly used medication-based measure and demonstrated the best predictive ability. Thus, in studies using pharmacy dispensing data, we recommend use of the RxRisk-V.
      In studies comparing the predictive ability of indices, we found that diagnosis-based measures were better predictors of mortality outcomes than medication-based indices. Some studies found that medication-based indices were better predictors of health care utilization and costs; however, other studies found that diagnosis-based measures were better at predicting such outcomes. Disadvantages of diagnosis-based indices include a wide variability in ICD coding practices, underreporting of chronic conditions in the secondary diagnosis fields, and difficulty in distinguishing between acute conditions present on admission from subsequent complications of care [
      • Needham D.M.
      • Scales D.C.
      • Laupacis A.
      • Pronovost P.J.
      A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research.
      ,
      • Parker J.P.
      • McCombs J.S.
      • Graddy E.A.
      Can pharmacy data improve prediction of hospital outcomes? Comparisons with a diagnosis-based comorbidity measure.
      ,
      • Jette N.
      • Quan H.
      • Hemmelgarn B.
      • Drosler S.
      • Maass C.
      • Moskal L.
      • et al.
      The development, evolution, and modifications of ICD-10: challenges to the international comparability of morbidity data.
      ]. Furthermore, certain CCI adaptations can only be used with specific ICD versions (eg, Deyo CCI with ICD-9-CM and the Quan CCI with ICD-10). Additionally, a number of country-specific ICD-10 versions exist [
      • Jette N.
      • Quan H.
      • Hemmelgarn B.
      • Drosler S.
      • Maass C.
      • Moskal L.
      • et al.
      The development, evolution, and modifications of ICD-10: challenges to the international comparability of morbidity data.
      ], which may further limit the application of the diagnosis-based indices. Hence, it is important, when choosing a CCI adaptation, to consider the ICD version used in the administrative data of the study and to select the CCI index accordingly. Pharmacy data are credited as a more timely, complete, and reliable data source than diagnosis-based data [
      • Sloan K.L.
      • Sales A.E.
      • Liu C.F.
      • Fishman P.
      • Nichol P.
      • Suzuki N.T.
      • et al.
      Construction and characteristics of the RxRisk-V: a VA-adapted pharmacy-based case-mix instrument.
      ], but it is not readily available in many jurisdictions. The major criticism of medication-based indices lies in the back coding from prescription to diagnosis [
      • Schneeweiss S.
      • Seeger J.D.
      • Maclure M.
      • Wang P.S.
      • Avorn J.
      • Glynn R.J.
      Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data.
      ]. This limits the definition of comorbidity to include only chronic diseases treated with prescription medication. Furthermore, medication-based indices require continual updates to accommodate the development and reassignment of new medications for specific indications.
      We found that the predictive ability of comorbidity indices varied widely, ranging from poor (C statistic 0.50–0.69) to good (C statistic 0.70–0.79) and excellent (C statistic >0.80), with similar variability observed when R2 values were reported. Performance varied according to the specific index, outcome measured, and study population. In 2000, Schneeweiss et al. [
      • Schneeweiss S.
      • Maclure M.
      Use of comorbidity scores for control of confounding in studies using administrative databases.
      ] conducted the first review of studies evaluating the ability of comorbidity indices to predict comorbidity-related outcomes, using administrative data. They concluded that comorbidity indices using administrative data provide only a modest improvement over adjustment for age alone [
      • Schneeweiss S.
      • Maclure M.
      Use of comorbidity scores for control of confounding in studies using administrative databases.
      ]. In 2001, they conducted an extensive validation study of four adaptations of the CCI as well as the CDS-1 and CDS-2. They concluded that comorbidity indices provide only a limited ability to control for confounding, acknowledging nonetheless their usefulness because of their ease of use and time and resources savings [
      • Schneeweiss S.
      • Seeger J.D.
      • Maclure M.
      • Wang P.S.
      • Avorn J.
      • Glynn R.J.
      Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data.
      ]. In 2005, Needham et al. conducted a review of 10 articles on CCI adaptations for administrative data with a specific emphasis on risk adjustment in critical care research. They found no difference in mortality prediction whether using CCI derived from administrative data or chart review and with the various adaptations of the CCI [
      • Needham D.M.
      • Scales D.C.
      • Laupacis A.
      • Pronovost P.J.
      A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research.
      ].
      A recent systematic review has been published comparing the predictive ability of various diagnosis-based indices using administrative data [
      • Sharabiani M.T.
      • Aylin P.
      • Bottle A.
      Systematic review of comorbidity indices for administrative data.
      ]. The authors performed a meta-analysis examining the indices' performance in predicting short- and long-term mortality. Their extensive comparative analysis resulted in findings similar to those in our study; specifically, the EI and Romano CCI demonstrated significantly superior performance in predicting mortality outcomes. Although our study and that by Sharabiani et al. focus on comorbidity indices using administrative data, they examine different aspects of the topic and use distinct approaches. Our study provides an extensive description of the various indices available, including how they were developed and results of their validation studies, and outlines the differences between them. The aforementioned review only includes diagnosis-based indices and reports only on studies comparing the predictive performance of various indices. Therefore, they did not discuss medication-based indices or include studies comparing medication- and diagnosis-based indices. Our study offers complementary information valuable to researchers trying to understand the differences between available instruments. It will assist researchers in selecting a comorbidity index that best meets the needs of their specific study, including when administrative data are available on both diagnosis and medication information.
      There are, however, limitations to our systematic review. First, we chose to focus specifically on comorbidity indices using administrative data and did not evaluate comorbidity indices using chart review or self-reported data. Predictive validity of CCI adaptations may differ with these data sources. Second, although our search strategy was comprehensive, it is possible that studies were missed. However, given the consistency of our results, it is unlikely that missed studies would significantly alter the main findings of our review. Finally, the number of published comparison studies between medication- and diagnosis-based indices was limited, and results were not entirely consistent across studies.

      5. Conclusion

      Comorbidity indices are used to control for the overall burden of comorbidities in administrative data studies and demonstrate validity in predicting mortality; however, their ability to fully adjust for confounding due to comorbidity may be limited. We recommend using a diagnosis-based index, such as the Quan EI, van Walraven EI, or Romano CCI, in studies in which the outcome of interest is mortality. One must consider the ICD version used when selecting a specific index. For studies evaluating health care utilization, in which medication data are available, we recommend using a medication-based index, such as the RxRisk-V. Overall, the appropriate selection of a comorbidity index for use with administrative data should take into account the type of data available, the study population, and the specific outcome of interest in the study.

      Supplementary data

      References

        • Perkins A.J.
        • Kroenke K.
        • Unutzer J.
        • Katon W.
        • Williams J.W.
        • Hope C.
        • et al.
        Common comorbidity scales were similar in their ability to predict health care costs and mortality.
        J Clin Epidemiol. 2004; 57: 1040-1048
        • Elixhauser A.
        • Steiner C.
        • Harris D.R.
        • Coffey R.M.
        Comorbidity measures for use with administrative data.
        Med Care. 1998; 36: 8-27
        • Charlson M.E.
        • Pompei P.
        • Ales K.L.
        • MacKenzie C.R.
        A new method of classifying prognostic comorbidity in longitudinal studies: development and validation.
        J Chronic Dis. 1987; 40: 373-383
        • Sharabiani M.T.
        • Aylin P.
        • Bottle A.
        Systematic review of comorbidity indices for administrative data.
        Med Care. 2012; 50: 1109-1118
        • Schneeweiss S.
        • Maclure M.
        Use of comorbidity scores for control of confounding in studies using administrative databases.
        Int J Epidemiol. 2000; 29: 891-898
        • Needham D.M.
        • Scales D.C.
        • Laupacis A.
        • Pronovost P.J.
        A systematic review of the Charlson comorbidity index using Canadian administrative databases: a perspective on risk adjustment in critical care research.
        J Crit Care. 2005; 20: 12-19
        • Schneeweiss S.
        • Seeger J.D.
        • Maclure M.
        • Wang P.S.
        • Avorn J.
        • Glynn R.J.
        Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data.
        Am J Epidemiol. 2001; 154: 854-864
        • Schneeweiss S.
        • Wang P.S.
        • Avorn J.
        • Glynn R.J.
        Improved comorbidity adjustment for predicting mortality in Medicare populations.
        Health Serv Res. 2003; 38: 1103-1120
        • Newschaffer C.J.
        • Bush T.L.
        • Penberthy L.T.
        Comorbidity measurement in elderly female breast cancer patients with administrative and medical records data.
        J Clin Epidemiol. 1997; 50: 725-733
        • Hanley J.A.
        • McNeil B.J.
        The meaning and use of the area under a receiver operating characteristic (ROC) curve.
        Radiology. 1982; 143: 29-36
        • D'Hoore W.
        • Sicotte C.
        • Tilquin C.
        Risk adjustment in outcome assessment: the Charlson comorbidity index.
        Methods Inf Med. 1993; 32: 382-387
        • Melfi C.
        • Holleman E.
        • Arthur D.
        • Katz B.
        Selecting a patient characteristics index for the prediction of medical outcomes using administrative claims data.
        J Clin Epidemiol. 1995; 48: 917-926
        • Poses R.M.
        • Smith W.R.
        • McClish D.K.
        • Anthony M.
        Controlling for confounding by indication for treatment. Are administrative data equivalent to clinical data?.
        Med Care. 1995; 33: AS36-46
        • Ghali W.A.
        • Hall R.E.
        • Rosen A.K.
        • Ash A.S.
        • Moskowitz M.A.
        Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data.
        J Clin Epidemiol. 1996; 49: 273-278
        • D'Hoore W.
        • Bouckaert A.
        • Tilquin C.
        Practical considerations on the use of the Charlson comorbidity index with administrative data bases.
        J Clin Epidemiol. 1996; 49: 1429-1433
        • Roos L.L.
        • Stranc L.
        • James R.C.
        • Li J.
        Complications, comorbidities, and mortality: improving classification and prediction.
        Health Serv Res. 1997; 32 (discussion 239–42): 229-238
        • Cleves M.A.
        • Sanchez N.
        • Draheim M.
        Evaluation of two competing methods for calculating Charlson's comorbidity index when analyzing short-term mortality using administrative data.
        J Clin Epidemiol. 1997; 50: 903-908
        • Librero J.
        • Peiro S.
        • Ordinana R.
        Chronic comorbidity and outcomes of hospital care: length of stay, mortality, and readmission at 30 and 365 days.
        J Clin Epidemiol. 1999; 52: 171-179
        • Wang P.S.
        • Walker A.
        • Tsuang M.
        • Orav E.J.
        • Levin R.
        • Avorn J.
        Strategies for improving comorbidity measures based on Medicare and Medicaid claims data.
        J Clin Epidemiol. 2000; 53: 571-578
        • Klabunde C.N.
        • Potosky A.L.
        • Legler J.M.
        • Warren J.L.
        Development of a comorbidity index using physician claims data.
        J Clin Epidemiol. 2000; 53: 1258-1267
        • Sundararajan V.
        • Henderson T.
        • Perry C.
        • Muggivan A.
        • Quan H.
        • Ghali W.A.
        New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality.
        J Clin Epidemiol. 2004; 57: 1288-1294
        • Harse J.D.
        • Holman C.D.
        Charlson's index was a poor predictor of quality of life outcomes in a study of patients following joint replacement surgery.
        J Clin Epidemiol. 2005; 58: 1142-1149
        • Holman C.D.
        • Preen D.B.
        • Baynham N.J.
        • Finn J.C.
        • Semmens J.B.
        A multipurpose comorbidity scoring system performed better than the Charlson index.
        J Clin Epidemiol. 2005; 58: 1006-1014
        • Martins M.
        • Blais R.
        Evaluation of comorbidity indices for inpatient mortality prediction models.
        J Clin Epidemiol. 2006; 59: 665-669
        • Norena M.
        • Wong H.
        • Thompson W.D.
        • Keenan S.P.
        • Dodek P.M.
        Adjustment of intensive care unit outcomes for severity of illness and comorbidity scores.
        J Crit Care. 2006; 21: 142-150
        • Nuttall M.
        • van der Meulen J.
        • Emberton M.
        Charlson scores based on ICD-10 administrative data were valid in assessing comorbidity in patients undergoing urological cancer surgery.
        J Clin Epidemiol. 2006; 59: 265-273
        • Hollenbeak C.S.
        • Stack Jr., B.C.
        • Daley S.M.
        • Piccirillo J.F.
        Using comorbidity indexes to predict costs for head and neck cancer.
        Arch Otolaryngol Head Neck Surg. 2007; 133: 24-27
        • Sundararajan V.
        • Quan H.
        • Halfan P.
        • Fushimi K.
        • Luthi J.-
        • Burnand B.
        • et al.
        Cross-national comparative performance of three versions of the ICD-10 Charlson index.
        Med Care. 2007; 45: 1210-1215
        • Klabunde C.N.
        • Legler J.M.
        • Warren J.L.
        • Baldwin L.-
        • Schrag D.
        A refined comorbidity measurement algorithm for claims-based studies of breast, prostate, colorectal, and lung cancer patients.
        Ann Epidemiol. 2007; 17: 584-590
        • Radley D.C.
        • Gottlieb D.J.
        • Fisher E.S.
        • Tosteson A.N.A.
        Comorbidity risk-adjustment strategies are comparable among persons with hip fracture.
        J Clin Epidemiol. 2008; 61: 580-587
        • Sessler D.I.
        • Sigl J.C.
        • Manberg P.J.
        • Kelley S.D.
        • Schubert A.
        • Chamoun N.G.
        Broadly applicable risk stratification system for predicting duration of hospitalization and mortality.
        Anesthesiology. 2010; 113: 1026-1037
        • Hanley G.E.
        • Morgan S.
        • Reid R.J.
        Explaining prescription drug use and expenditures using the adjusted clinical groups case-mix system in the population of British Columbia, Canada.
        Med Care. 2010; 48: 402-408
        • Khan N.F.
        • Perera R.
        • Harper S.
        • Rose P.W.
        Adaptation and validation of the Charlson index for Read/OXMIS coded databases.
        BMC Fam Pract. 2010; 11: 1
        • Liu J.
        • Huang Z.
        • Gilbertson D.T.
        • Foley R.N.
        • Collins A.J.
        An improved comorbidity index for outcome analyses among dialysis patients.
        Kidney Int. 2010; 77: 141-151
        • Deyo R.A.
        • Cherkin D.C.
        • Ciol M.A.
        Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases.
        J Clin Epidemiol. 1992; 45: 613-619
        • Romano P.S.
        • Roos L.L.
        • Jollis J.G.
        Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives.
        J Clin Epidemiol. 1993; 46: 1075-1090
        • Romano P.S.
        • Roos L.L.
        • Jollis J.G.
        Further evidence concerning the use of a clinical comorbidity index with ICD-9-CM administrative data.
        J Clin Epidemiol. 1993; 46: 1085-1090
        • Quan H.
        • Sundararajan V.
        • Halfon P.
        • Fong A.
        • Burnand B.
        • Luthi J.C.
        • et al.
        Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.
        Med Care. 2005; 43: 1130-1139
        • Quan H.
        • Li B.
        • Couris C.M.
        • Fushimi K.
        • Graham P.
        • Hider P.
        • et al.
        Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.
        Am J Epidemiol. 2011; 173: 676-682
        • Bernabeu-Wittel M.
        • Ollero-Baturone M.
        • Moreno-Gavino L.
        • Baron-Franco B.
        • Fuertes A.
        • Murcia-Zaragoza J.
        • et al.
        Development of a new predictive model for polypathological patients. The PROFUND index.
        Eur J Intern Med. 2011; 22: 311-317
        • Austin P.C.
        • van Walraven C.
        • Wodchis W.P.
        • Newman A.
        • Anderson G.M.
        Using the Johns Hopkins Aggregated Diagnosis Groups (ADGs) to predict mortality in a general adult population cohort in Ontario, Canada.
        Med Care. 2011; 49: 932-939
        • Roos L.L.
        • Sharp S.M.
        • Cohen M.M.
        • Wajda A.
        Risk adjustment in claims-based research: the search for efficient approaches.
        J Clin Epidemiol. 1989; 42: 1193-1206
        • Halfon P.
        • Eggli Y.
        • van Melle G.
        • Chevalier J.
        • Wasserfallen J.B.
        • Burnand B.
        Measuring potentially avoidable hospital readmissions.
        J Clin Epidemiol. 2002; 55: 573-587
        • Zhang J.X.
        • Iwashyna T.J.
        • Christakis N.A.
        The performance of different lookback periods and sources of information for Charlson comorbidity adjustment in Medicare claims.
        Med Care. 1999; 37: 1128-1139
        • Susser S.R.
        • McCusker J.
        • Belzile E.
        Comorbidity information in older patients at an emergency visit: self-report vs. administrative data had poor agreement but similar predictive validity.
        J Clin Epidemiol. 2008; 61: 511-515
        • Ronksley P.E.
        • Tsai W.H.
        • Quan H.
        • Faris P.
        • Hemmelgarn B.R.
        Data enhancement for co-morbidity measurement among patients referred for sleep diagnostic testing: an observational study.
        BMC Med Res Methodol. 2009; 9: 50
        • Leal J.R.
        • Laupland K.B.
        Validity of ascertainment of co-morbid illness using administrative databases: a systematic review.
        Clin Microbiol Infect. 2010; 16: 715-721
        • Seo H.J.
        • Yoon S.J.
        • Lee S.I.
        • Lee K.S.
        • Yun Y.H.
        • Kim E.J.
        • et al.
        A comparison of the Charlson comorbidity index derived from medical records and claims data from patients undergoing lung cancer surgery in Korea: a population-based investigation.
        BMC Health Serv Res. 2010; 10: 236
        • Jang S.H.
        • Chea J.W.
        • Lee K.B.
        Charlson comorbidity index using administrative database in incident PD patients.
        Clin Nephrol. 2010; 73: 204-209
        • Zhu H.
        • Hill M.D.
        Stroke: the Elixhauser index for comorbidity adjustment of in-hospital case fatality.
        Neurology. 2008; 71: 283-287
        • van Walraven C.
        • Austin P.C.
        • Jennings A.
        • Quan H.
        • Forster A.J.
        A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data.
        Med Care. 2009; 47: 626-633
        • Fleming S.T.
        • Pearce K.A.
        • McDavid K.
        • Pavlov D.
        The development and validation of a comorbidity index for prostate cancer among Black men.
        J Clin Epidemiol. 2003; 56: 1064-1075
        • Abildstrom S.Z.
        • Hvelplund A.
        • Rasmussen S.
        • Nielsen P.H.
        • Mortensen P.E.
        • Kruse M.
        Prognostic information in administrative co-morbidity data following coronary artery bypass grafting.
        Eur J Cardiothorac Surg. 2010; 38: 573-576
        • Stukenborg G.J.
        • Wagner D.P.
        • Connors AF Jr.,
        Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations.
        Med Care. 2001; 39: 727-739
        • Southern D.A.
        • Quan H.
        • Ghali W.A.
        Comparison of the Elixhauser and Charlson/Deyo methods of comorbidity measurement in administrative data.
        Med Care. 2004; 42: 355-360
        • Yan Y.
        • Birman-Deych E.
        • Radford M.J.
        • Nilasena D.S.
        • Gage B.F.
        Comorbidity indices to predict mortality from Medicare data: results from the national registry of atrial fibrillation.
        Med Care. 2005; 43: 1073-1077
        • Kurichi J.E.
        • Stineman M.G.
        • Kwong P.L.
        • Bates B.E.
        • Reker D.M.
        Assessing and using comorbidity measures in elderly veterans with lower extremity amputations.
        Gerontology. 2007; 53: 255-259
        • Livingston E.H.
        Development of bariatric surgery-specific risk assessment tool.
        Surg Obes Relat Dis. 2007; 3: 14-20
        • Machnicki G.
        • Pinsky B.
        • Takemoto S.
        • Balshaw R.
        • Salvalaggio P.R.
        • Buchanan P.M.
        • et al.
        Predictive ability of pretransplant comorbidities to predict long-term graft loss and death.
        Am J Transplant. 2009; 9: 494-505
        • Chu Y.T.
        • Ng Y.Y.
        • Wu S.C.
        Comparison of different comorbidity measures for use with administrative data in predicting short- and long-term mortality.
        BMC Health Serv Res. 2010; 10: 140
        • Martins M.
        Use of comorbidity measures to predict the risk of death in Brazilian in-patients.
        Rev Saude Publica. 2010; 44: 448-456
        • Li P.
        • Kim M.M.
        • Doshi J.A.
        Comparison of the performance of the CMS Hierarchical Condition Category (CMS-HCC) risk adjuster with the Charlson and Elixhauser comorbidity measures in predicting mortality.
        BMC Health Serv Res. 2010; 10: 245
        • Lieffers J.R.
        • Baracos V.E.
        • Winget M.
        • Fassbender K.
        A comparison of Charlson and Elixhauser comorbidity measures to predict colorectal cancer survival using administrative health data.
        Cancer. 2011; 117: 1957-1965
        • Gagne J.J.
        • Glynn R.J.
        • Avorn J.
        • Levin R.
        • Schneeweiss S.
        A combined comorbidity score predicted mortality in elderly patients better than existing scores.
        J Clin Epidemiol. 2011; 64: 749-759
        • Austin P.C.
        • Shah B.R.
        • Newman A.
        • Anderson G.M.
        Using the Johns Hopkins' Aggregated Diagnosis Groups (ADGs) to predict 1-year mortality in population-based cohorts of patients with diabetes in Ontario, Canada.
        Diabet Med. 2012; 29: 1134-1141
        • Mnatzaganian G.
        • Ryan P.
        • Norman P.E.
        • Hiller J.E.
        Accuracy of hospital morbidity data and the performance of comorbidity scores as predictors of mortality.
        J Clin Epidemiol. 2012; 65: 107-115
        • Baldwin L.M.
        • Klabunde C.N.
        • Green P.
        • Barlow W.
        • Wright G.
        In search of the perfect comorbidity measure for use with administrative claims data: does it exist?.
        Med Care. 2006; 44: 745-753
        • Von Korff M.
        • Wagner E.H.
        • Saunders K.
        A chronic disease score from automated pharmacy data.
        J Clin Epidemiol. 1992; 45: 197-203
        • Johnson R.E.
        • Hornbrook M.C.
        • Nichols G.A.
        Replicating the chronic disease score (CDS) from automated pharmacy data.
        J Clin Epidemiol. 1994; 47: 1191-1199
        • Clark D.O.
        • Von Korff M.
        • Saunders K.
        • Baluch W.M.
        • Simon G.E.
        A chronic disease score with empirically derived weights.
        Med Care. 1995; 33: 783-795
        • Fishman P.A.
        • Shay D.K.
        Development and estimation of a pediatric chronic disease score using automated pharmacy data.
        Med Care. 1999; 37: 874-883
        • Lamers L.M.
        Pharmacy costs groups: a risk-adjuster for capitation payments based on the use of prescribed drugs.
        Med Care. 1999; 37: 824-830
        • Kaye K.S.
        • Sands K.
        • Donahue J.G.
        • Chan K.A.
        • Fishman P.
        • Platt R.
        Preoperative drug dispensing as predictor of surgical site infection.
        Emerg Infect Dis. 2001; 7: 57-65
        • Putnam K.G.
        • Buist D.S.
        • Fishman P.
        • Andrade S.E.
        • Boles M.
        • Chase G.A.
        • et al.
        Chronic disease score as a predictor of hospitalization.
        Epidemiology. 2002; 13: 340-346
        • Fishman P.A.
        • Goodman M.J.
        • Hornbrook M.C.
        • Meenan R.T.
        • Bachman D.J.
        • O'Keeffe Rosetti M.C.
        Risk adjustment using automated ambulatory pharmacy data: the RxRisk model.
        Med Care. 2003; 41: 84-99
        • Sloan K.L.
        • Sales A.E.
        • Liu C.F.
        • Fishman P.
        • Nichol P.
        • Suzuki N.T.
        • et al.
        Construction and characteristics of the RxRisk-V: a VA-adapted pharmacy-based case-mix instrument.
        Med Care. 2003; 41: 761-774
        • Joish V.N.
        • Malone D.C.
        • Wendel C.
        • Draugalis J.R.
        • Mohler M.J.
        Development and validation of a diabetes mellitus severity index: a risk-adjustment tool for predicting health care resource use and costs.
        Pharmacotherapy. 2005; 25: 676-684
        • McGregor J.C.
        • Perencevich E.N.
        • Furuno J.P.
        • Langenberg P.
        • Flannery K.
        • Zhu J.
        • et al.
        Comorbidity risk-adjustment measures were developed and validated for studies of antibiotic-resistant infections.
        J Clin Epidemiol. 2006; 59: 1266-1273
        • George J.
        • Vuong T.
        • Bailey M.J.
        • Kong D.C.
        • Marriott J.L.
        • Stewart K.
        Development and validation of the medication-based disease burden index.
        Ann Pharmacother. 2006; 40: 645-650
        • Vitry A.
        • Wong S.A.
        • Roughead E.E.
        • Ramsay E.
        • Barratt J.
        Validity of medication-based co-morbidity indices in the Australian elderly population.
        Aust N Z J Public Health. 2009; 33: 126-130
        • Schneeweiss S.
        • Wang P.S.
        • Avorn J.
        • Maclure M.
        • Levin R.
        • Glynn R.J.
        Consistency of performance ranking of comorbidity adjustment scores in Canadian and U.S. utilization data.
        J Gen Intern Med. 2004; 19: 444-450
        • Parker J.P.
        • McCombs J.S.
        • Graddy E.A.
        Can pharmacy data improve prediction of hospital outcomes? Comparisons with a diagnosis-based comorbidity measure.
        Med Care. 2003; 41: 407-419
        • McGregor J.C.
        • Kim P.W.
        • Perencevich E.N.
        • Bradham D.D.
        • Furuno J.P.
        • Kaye K.S.
        • et al.
        Utility of the chronic disease score and Charlson comorbidity index as comorbidity measures for use in epidemiologic studies of antibiotic-resistant organisms.
        Am J Epidemiol. 2005; 161: 483-493
        • Farley J.F.
        • Harley C.R.
        • Devine J.W.
        A comparison of comorbidity measurements to predict healthcare expenditures.
        Am J Manag Care. 2006; 12: 110-119
        • Johnson M.L.
        • El-Serag H.B.
        • Tran T.T.
        • Hartman C.
        • Richardson P.
        • Abraham N.S.
        Adapting the Rx-Risk-V for mortality prediction in outpatient populations.
        Med Care. 2006; 44: 793-797
        • Fan V.S.
        • Maciejewski M.L.
        • Liu C.-F.
        • McDonell M.B.
        • Fihn S.D.
        Comparison of risk adjustment measures based on self-report, administrative data, and pharmacy records to predict clinical outcomes.
        Health Serv Outcomes Res Methodol. 2006; 6: 21-36
        • Dominick K.L.
        • Dudley T.K.
        • Coffman C.J.
        • Bosworth H.B.
        Comparison of three comorbidity measures for predicting health service use in patients with osteoarthritis.
        Arthritis Rheum. 2005 15; 53: 666-672
        • Baser O.
        • Palmer L.
        • Stephenson J.
        The estimation power of alternative comorbidity indices.
        Value Health. 2008; 11: 946-955
        • Maciejewski M.L.
        • Liu C.-F.
        • Fihn S.D.
        Performance of comorbidity, risk adjustment, and functional status measures in expenditure prediction for patients with diabetes.
        Diabetes Care. 2009; 32: 75-80
        • Bayliss E.A.
        • Ellis J.L.
        • Steiner J.F.
        Seniors' self-reported multimorbidity captured biopsychosocial factors not incorporated into two other data-based morbidity measures.
        J Clin Epidemiol. 2009; 62: 550-557.e1