| | Major adverse outcomes after percutaneous transluminal coronary angioplasty: a clinical prediction ruleReceived 29 October 2001; received in revised form 4 June 2002; accepted 5 September 2002. Abstract In this study, we developed and internally validated a clinical model for predicting major adverse outcomes in patients undergoing percutaneous transluminal coronary angioplasty (PTCA) using a multi-institutional prospective cohort study involving all adult patients who underwent PTCA at 12 participating institutions from August 1993 to October 1995. A major adverse outcome, defined as death, renal failure, myocardial infarction, cardiac arrest, stroke, or coma, occurred in 3.3 and 3.2% of patients in the derivation and validation sets, respectively. Death occurred in 1.5% in both sets. Fourteen variables were independently correlated with major adverse outcomes. The rule, which stratifies PTCA patients into six levels of risk based on the severity score, showed excellent discrimination (receiver-operating characteristic curve area 0.82) and calibration (Hosmer-Lemeshow chi-square statistic P = .90) and performed well on internal validation. This rule allows accurate preprocedure stratification of PTCA candidates according to their risk of suffering a major adverse outcome.
1. Introduction  Percutaneous transluminal coronary angioplasty (PTCA) is a common procedure that can reopen occluded coronary arteries, but as with most interventions, there is associated risk. Variation in severity of illness before PTCA presents obstacles to accurate, standardized comparison of patient outcomes. Thus, clinical prediction rules that can identify risk factors and accurately stratify patients according to risk of having a major adverse outcome after PTCA, similar to those prediction models widely used in coronary artery bypass surgery [1], may be helpful in improving patient care and outcomes. A number of published studies have attempted to identify risk factors for mortality and/or morbidity after PTCA 2, 3, 4, 5. Risk factors for death or major morbidity after PTCA that have been identified by previous studies include: female gender, advancing age, diabetes mellitus, unstable angina, multivessel disease, presence of a thrombus, lesion length, and certain lesion morphologies such as eccentric, calcified, or nondiscrete lesions 6, 7, 8, 9. However, few published studies to date present tools that can stratify patients according to overall level of risk of having an adverse outcome after PTCA [10]. In this study, we develop and internally validate a clinical model for predicting major adverse outcomes in patients undergoing PTCA, and compare its performance to another published model. 2. Methods 2.1. Patient population Data collection for the study took place from August 1993 to October 1995, at 12 medical centers. All 12 centers were large tertiary care centers and were members of the Academic Medical Center Consortium, which sponsored the study, called the Quality Measurement and Management Initiative (QMMI) Coronary Revascularization Project. Patients enrolled in the study included all patients who underwent a PTCA procedure at any of the 12 participating medical centers. The unit of analysis was the PTCA; thus, a patient could be counted more than once if another PTCA procedure was performed either during the same admission or during a subsequent admission during the QMMI study period. The cohort, consisting of 14,030 episodes (12,133 unique patients), was divided randomly into two mutually exclusive subsets of episodes: the derivation set and the validation set. The QMMI score was developed using the derivation subset and internally validated using the validation subset. The same two sets were used for all analyses. 2.2. Definitions 2.2.1. Outcome definitions The only variables eligible for entry into the predictive models in the study were ones that were available before the procedure. A major adverse outcome was defined as any of the following occurring after the PTCA procedure and before discharge from the hospital: death, renal failure (defined as new dialysis), myocardial infarction (defined as chest pain or nausea or diaphoresis or hypotension associated with the development of new Q waves), cardiac arrest (defined as use of Advanced Cardiac Life Support or cardiac or respiratory arrest), stroke (defined as a loss of neurological function of abrupt onset, caused by ischemia, persisting ⩾24 hr or leaving residual signs), or coma (defined as a state of complete mental unresponsiveness with no evidence of purposeful motor, verbal, or ocular responses to stimulation). To attain a broader definition of adverse outcome than either major morbidity or mortality, we included death as a subtype of major adverse outcome. If one or more of the subtypes of major adverse outcome was present, the summary variable “any major adverse outcome” was considered present. Each patient was counted only once in the analyses regardless of the total number of major adverse outcomes present in that patient. 2.2.2. Risk factor variable definitions Cardiogenic shock was defined as systolic blood pressure <80 requiring treatment with pressors and/or inotropes. Congestive heart failure was defined using a modified form of the New York Heart Association scoring system. Prior myocardial infarction required documentation in the medical record by electrocardiogram or cardiac enzymes. Three vessel disease was defined as simultaneous occlusion of >70% diameter reduction of the left anterior descending, left circumflex, and right coronary arteries. Stroke history required a documented history of cerebrovascular accident, stroke, or transient ischemic attack that resulted in abnormalities in vision, speech, sensation, or motor function; or a history of carotid surgery. Peripheral vascular disease history required documented history of classic claudication or peripheral vascular surgery or angioplasty. Hypercholesterolemia was defined as documented diagnosis and/or treatment by a physician of a total cholesterol >240. Chronic obstructive pulmonary disease was defined as chronic obstructive pulmonary disease or asthma requiring inhalers, aminophylline, or steroids. Renal disease history required documented serum creatinine >3.0 mg/dL or dialysis or previous kidney transplant. Liver disease history was defined as the presence of cirrhosis, chronic active hepatitis, or primary biliary cirrhosis, with any of the following sequelae: ascites, esophageal varices, portal hypertension, or active hepatic encephalopathy. 2.3. Data collection All patients that underwent a PTCA procedure at any of the 12 participating institutions during the study period were included in the study. Detailed information was obtained prospectively through patient interviews and retrospectively through medical record abstraction including: demographic information, conditions and risks prior to procedure, therapy within 48 hr prior to PTCA, last preprocedure labs, use of services and key statistics during the PTCA, and postprocedure hospital course, including any adverse events that occurred, and discharge labs. Preprocedure data and procedural variables were collected on the day of the procedure by the cardiology teams or afterwards by research nurses. Information on postprocedure hospital course was collected after patient discharge by research nurses. To assess completeness and accuracy of QMMI data collection, data elements were compared to the Uniform Hospital Discharge Data Set (UHDDS). Discrepancies found between UHDDS and QMMI data in critical fields were addressed by the individual institutions. Missing data were handled as follows: (1) for binary variables, missing data were coded as absent, (2) for categorical and continuous variables, an additional category was created to label these data as “missing,” such that for a variable with three categories, a fourth was created for missing data. 2.4. Analysis The only variables considered for entry into the predictive models were ones that were available prior to the PTCA, and all outcome variables included were events that occurred after the procedure and before hospital discharge. Factors suggested as correlates of morbidity and mortality published in previous studies as well as other variables suggested by members of the Consortium were considered in the analyses. Therapies received prior to the PTCA and intraprocedural characteristics were not considered as potential predictors of major adverse outcomes but are reported (Table 3). Relationships between potential risk factors and the occurrence of a major adverse outcome were first assessed in univariate analyses using the SAS statistical programming package, version 6.11 [11]. The chi-square statistic was used for categorical variables, t-tests for normally distributed continuous variables, and Wilcoxon rank-sum tests for nonparametric comparisons. Variables significantly correlated (P < .05) with major adverse outcomes were then entered into a stepwise logistic regression analysis [12]. In the logistic model, factors with P-values <.05 were retained. After completing the logistic regression analyses, the resulting independent correlates of any major adverse outcome were used to develop the clinical prediction rule. Construction of the model involved the following steps: (1) the smallest beta coefficient value was assigned a risk score value of one, (2) each subsequent beta coefficient was divided by the smallest beta coefficient's raw value, and (3) the quotient of each division was rounded to the nearest integer to arrive at the risk score value for that variable. After assigning a risk score value to each variable, a total risk score was calculated for each episode by summing the risk score values for each of the significant predictor variables that apply to that PTCA episode. The resulting continuous distribution of total risk scores across all episodes in the derivation set was then stratified into six categories of points that group episodes according to level of risk. Discriminatory performance of the prediction model was internally validated by comparing the receiver-operating characteristic (ROC) curve area in the derivation subset of the QMMI cohort with that in the validation subset [13]. Calibration performance of the model was also assessed by plotting observed and predicted event frequencies by deciles of risk using the Hosmer-Lemeshow goodness-of-fit statistic [14]. Additionally, to make the model easier to interpret, we derived likelihood ratios for each risk group. In addition to developing and internally validating the QMMI score, we crossvalidated a clinical prediction rule previously published by Kimmel et al. [10] that was also designed to predict adverse outcomes after PTCA and compared its performance in the validation subset of the QMMI cohort to that of the QMMI score. In doing this, we used the following five steps. First, we applied the published risk score values for the Kimmel et al. model that predicts adverse outcomes after PTCA to the validation subset of the QMMI cohort. It is important to note that in this analysis we redefined the adverse event outcome definition to adhere to that used in the original Kimmel et al. analysis [10]. Therefore, we tested the Kimmel et al. score's ability to predict its original outcome of interest, not the QMMI outcome. Second, we added up each of the Kimmel et al. risk score values present in each episode in the QMMI cohort to arrive at a total risk score for each episode. Third, we assessed the Kimmel et al. score's ability to discriminate outcomes in the QMMI cohort by plotting sensitivity against (1-specificity) across all levels of risk and calculating the area under the ROC curve. Fourth, we assessed the Kimmel et al score's calibration performance in the QMMI cohort by entering the total risk scores for all episodes into a logistic regression analysis modeled on the outcome in question, converting each episode's total risk score into an expected probability of the outcome in question with a range from 0 to 1, calculating the observed and expected outcome rates across deciles of risk, and then calculating the Hosmer-Lemeshow goodness-of-fit chi square statistic. Last, we compared the results of the crossvalidation of the Kimmel et al. score in the validation subset of the QMMI cohort to the performance of the QMMI score. All crossvalidation studies and comparisons between the QMMI and Kimmel et al. scores were performed using the validation subset of the QMMI cohort; the derivation subset was used only in the construction of the QMMI score. 3. Results 3.1. Characteristics of the derivation and validation sets The QMMI dataset included 14,030 episodes of PTCA, of which 12,133 represented unique patients. Of the 14,030 total episodes, 9,286 were randomly allocated to the derivation set and the remaining 4,744 to the validation set (Table 1). Preprocedure patient characteristics including age, gender, and disease histories were not significantly different across derivation and validation sets (Table 1). In the derivation subset, the PTCA was elective for 61.2% of cases, urgent for 28.7%, and emergent for 10.1% (Table 1). One or more major adverse outcomes were found to occur in 3.3% of patients in the derivation subset (Table 2). Altogether, 308 episodes of PTCA were associated with a major adverse outcome. Adverse event rates were also similar across derivation and validation subsets. | | |  | Characteristic | Derivation set (n = 9286) | Validation set (n = 4744) | |
|---|
| Demographic information | | | |
| Age, years (mean ± SD) | 62.1 (11.7) | 62.1 (11.7) | |
| Female gender | 2772 (29.9%) | 1403 (29.6%) | |
| Nonwhite race | 171 (1.8%) | 87 (1.8%) | |
| Preprocedure clinical characteristics | | | |
| Diabetes | 1987 (21.4%) | 990 (20.9%) | |
| Hypertension | 5281 (56.9%) | 2720 (57.3%) | |
| Myocardial infarction within 24 h of PTCA | 752 (8.1%) | 338 (7.1%) | |
| Postmyocardial angina | 1227 (13.2%) | 607 (12.8%) | |
| Angina at rest | 3977 (42.8%) | 2016 (42.5%) | |
| Congestive heart failure | 907 (9.8%) | 460 (9.7%) | |
| Renal disease | 308 (3.3%) | 137 (2.9%) | |
| Last serum creatinine prior to PTCA (mean±SD) | 1.3 (1.1) | 1.3 (1.0) | |
| Cardiogenic shock | 150 (1.6%) | 86 (1.8%) | |
| Ejection fraction | | | |
| < 50% | 836 (9.0%) | 418 (8.8%) | |
| < 30% | 179 (1.9%) | 87 (1.8%) | |
| Prior PTCA | 2096 (22.6%) | 1065 (22.5%) | |
| Prior coronary artery bypass grafting surgery | 1659 (17.8%) | 841 (17.7%) | |
| Priority of PTCA | | | |
| Elective | 5684 (61.2%) | 2964 (62.5%) | |
| Urgent | 2667 (28.7%) | 1329 (28.0%) | |
| Emergent | 935 (10.1%) | 451 (9.5%) | |
| Surgical backup present | 6912 (74.4%) | 3550 (74.8%) | |
| Therapies received prior to procedure | | | |
| Thrombolytics within 48 h prior to PTCA | 319 (3.4%) | 148 (3.1%) | |
| Intraprocedure characteristics | | | |
| Left main coronary artery attempted | 65 (0.7%) | 42 (0.9%) | |
| Thrombolytics received during procedure | 256 (2.8%) | 109 (2.3%) | |
| Number of vessels attempted | | | |
| 1 | 6204 (73.5%) | 3160 (72.9%) | |
| 2 | 1793 (21.2%) | 932 (21.5%) | |
| 3 | 374 (4.4%) | 204 (4.7%) | |
| 4 | 52 (0.6%) | 30 (0.7%) | |
| 5 or more | 18 (0.2%) | 8 (0.2%) | |
| Vessel redilation | 64 (0.7%) | 33 (0.7%) | | | | |
| | | | Event | Derivation (n = 9286) | Validation set (n = 4744) | |
|---|
| Any major adverse outcome | 306 (3.3%) | 150 (3.2%) | |
| Death | 142 (1.5%) | 71 (1.5%) | |
| Renal failure | 53 (0.6%) | 33 (0.7%) | |
| Reinfarction | 113 (1.2%) | 51 (1.1%) | |
| Cardiac arrest | 98 (1.1%) | 46 (1.0%) | |
| Stroke | 18 (0.2%) | 10 (0.2%) | |
| Coma | 19 (0.2%) | 13 (0.3%) | | | | |
3.3. Development of the QMMI clinical prediction rule To develop the clinical prediction rule, each of the 14 identified preprocedure risk factors were assigned an integer weight (Table 4) as described earlier (see Methods). The maximum possible total risk score was 38 points, and the maximum total risk score observed in a patient in the derivation set was 28 points. The mean total risk score in patients who had a major adverse outcome in the derivation group was 8.2 points, and the mean total risk score in patients without a major adverse outcome was 2.8 points. The continuous distribution of scores was stratified into six categories of risk, resulting in groups with risks of having a major adverse event ranging from 0.4 to 20.1% in the derivation set (Table 5). The rule performed well in predicting major adverse outcomes on receiver operating characteristic analysis (area under the curve 0.80 ± 0.01, Fig. 1a). Calibration of the model in predicting major adverse outcomes in the derivation set was tested using the Hosmer-Lemeshow test and was fair (P = .07, Fig. 2a). In plotting the calibration curves, some of the deciles were grouped together due to the relatively small numbers of patients in each risk group. Grouping deciles in this situation is an accepted practice when expected event rates in the groups are small [15]. For descriptive purposes, Figure 3 depicts the average total risk score, percent major adverse outcomes per group, and percent deaths per group across the six risk groups. The distribution of episodes and cases across risk groups for the entire cohort (derivation and validation sets combined) is shown in Figure 4. | | | | | Event | Distribution of total risk scores assigned to patients | |
|---|
| | | 0 | 1 to 2 | 3 to 4 | 5 to 6 | 7 to 8 | 9 or more | |
|---|
| Derivation set | No (n = 8980) | 2579 (99.6) | 2609 (98.5) | 1671 (97.7) | 1018 (95.5) | 603 (93.6) | 500 (79.9) | |
| | Yes (n = 306) | 11 (0.4) | 40 (1.5) | 40 (2.3) | 48 (4.5) | 41 (6.4) | 126 (20.1) | |
| Validation set | No (n = 4594) | 1305 (99.4) | 1377 (98.8) | 835 (98.4) | 533 (95.3) | 303 (94.4) | 241 (78.2) | |
| | Yes (n = 150) | 8 (0.6) | 17 (1.2) | 14 (1.7) | 26 (4.7) | 18 (5.6) | 67 (21.8) | |
| | Likelihood Ratioa | 0.2 (0.1,0.2) | 0.4 (0.3,0.5) | 0.6 (0.5,0.8) | 1.4 (1.1,1.8) | 1.9 (1.5,2.5) | 7.8 (6.8,8.8) | |
| | | Mortality | |
| | | Distribution of total risk scores assigned to patients | |
| | Death | 0 | 1 to 2 | 3 to 4 | 5 to 6 | 7 to 8 | 9 or more | |
| Derivation set | No (n = 9144) | 2587 (99.9) | 2641 (99.7) | 1697 (99.2) | 1050 (98.5) | 628 (97.5) | 541 (86.4) | |
| | Yes (n = 142) | 3 (0.1) | 8 (0.3) | 14 (0.8) | 16 (1.5) | 16 (2.5) | 85 (13.6) | |
| Validation set | No (n = 4673) | 1313 (100.0) | 1390 (99.7) | 841 (99.1) | 549 (98.2) | 311 (96.9) | 269 (87.3) | |
| | Yes (n = 71) | 0 (0.0) | 4 (0.3) | 8 (0.9) | 10 (1.8) | 10 (3.1) | 39 (12.7) | |
| | Likelihood Ratioa | 0.1 (0.0,0.1) | 0.2 (0.1,0.3) | 0.6 (0.4,0.8) | 1.0 (0.7,1.5) | 1.8 (1.2,2.6) | 9.9 (8.6,11.3) | | | | |
|
a
The derivation and validation sets were combined for calculation of likelihood ratios. |
In addition, we determined how well the QMMI model could predict the outcome of mortality exclusively and found that it performed well. On ROC analysis, the model achieved an area under the ROC curve in predicting mortality of 0.87 ± 0.02 (Fig. 1b). Calibration performance of the model in predicting mortality in the derivation set using the Hosmer-Lemeshow test was good (P = .70, Fig. 2b). 3.4. Validation The validation group contained 4,744 patients who underwent PTCA, of which 150 (3.2%) had a major adverse outcome. When stratified into six risk groups, the group at lowest risk in the validation cohort had an 0.6% adverse event rate while the highest risk group had a 21.8% adverse event rate. The maximum total risk score observed in the validation group was 25 points. The mean total risk score among episodes involving a major adverse outcome was 8.8 points in the validation study, while the mean total risk score among episodes without a major adverse outcome was 2.8 points. The model performed well in discriminating major adverse outcomes on ROC analysis in the validation set (area under the curve 0.82 vs. 0.80 for the derivation set, Fig. 1a). In predicting mortality, the model also performed well in the validation cohort (area under the ROC curve 0.88 vs. 0.87 for the derivation set, Fig. 1b). Calibration of the model in predicting major adverse outcomes in the validation set was tested using the Hosmer-Lemeshow test and was good (HL chi square P = .90, Fig. 2a). In predicting mortality exclusively, calibration performance in the validation set was fair (P = .13, Fig. 2b). 3.5. Crossvalidation of the Kimmel et al. score and comparison with the QMMI score On applying the prediction model developed by Kimmel et al. [10] to the validation subset of the QMMI cohort, the total possible risk score assigned to patients ranged from zero to six points. Patients were stratified into four risk groups as in the original study (Table 6). The lowest risk group had an adverse event rate of 3.5%, while in the highest-risk group 33.3% suffered an adverse outcome. In the original validation study by Kimmel et al., 1.3% fell into the lowest risk group while 16.7% fell into the highest risk group. The maximum total risk score observed was four points. The mean total risk score among episodes involving a major adverse outcome was 0.9 points, while the mean total risk score among episodes not involving a major adverse outcome was 0.5 points. The Kimmel et al. model performed similarly in discriminating adverse outcomes on crossvalidation in the QMMI validation cohort as in its own validation study (area under the ROC curve 0.63 vs. 0.65, respectively). In contrast, the Kimmel et al. model performed significantly less well than the QMMI score in discriminating adverse outcomes in the QMMI validation cohort (area under the ROC curve 0.63 for the Kimmel et al. score vs. 0.82 for the QMMI score, Fig. 1a). Calibration of the Kimmel et al. model in predicting adverse outcomes in the validation set was tested using the Hosmer-Lemeshow test and was good (P = .28, Fig. 2b). Although the Kimmel et al model was not designed to predict mortality exclusively, we tested its ability to predict this endpoint and found that it performed better than it did in predicting adverse outcomes (ROC curve area 0.75 vs. 0.88 for the QMMI score, Fig. 1b). Calibration of the Kimmel et al. score in predicting mortality was fair (P = .18, Fig. 2b). | | | | Event | Distribution of total risk scores assigned to patients | |
|---|
| | 0 | 1 | 2 | 3 | 4 or more | |
|---|
| No (n = 4594) | 2824 (96.5) | 1338 (93.7) | 300 (90.9) | 24 (51.1) | 2 (66.7) | |
| Yes (n = 150) | 102 (3.5) | 90 (6.3) | 31 (9.1) | 23 (48.9) | 1 (33.3) | |
| Likelihood ratio | 0.7 (0.6,0.8) | 1.2 (1.0,1.4) | 1.9 (1.3,2.6) | 17.4 (10.0,30.1) | 9.1 (1.2,69.0) | | | | |
4. Discussion Although several clinical prediction models are currently available for use in patients undergoing coronary artery bypass surgery, some of which have achieved wide clinical use in helping to identify patients at highest risk of dying or suffering a nonfatal major morbidity after surgery, few published studies have attempted to identify risk factors that are independently correlated with adverse outcomes after PTCA. Like coronary bypass surgery, PTCA is a commonly performed procedure that on the whole is beneficial but carries some risk. As the practice of PTCA continues to improve and the procedure is used in an increasing number of clinical settings, a well-constructed, validated clinical prediction rule for predicting adverse outcomes in this patient population is needed. In this study, we developed and internally validated a clinical severity score built from logistic regression analysis that predicts death and certain nonfatal major adverse outcomes after PTCA The QMMI rule is an additive risk score that is calculated using 14 variables, all of which are objectively reported and easily attainable prior to the procedure. Only presence of left main coronary artery disease and elevated left-ventricular end-diastolic pressure require information from a cardiac catheterization (which is necessary in any event prior to PTCA). The data used in developing the rule included all 9,286 episodes of PTCA performed at 12 large, tertiary medical centers located throughout the United States. To avoid overfitting the prediction model to the study dataset, no more than one predictor variable was retained in the final logistic model for every 10 outcome events. In developing the final model, each risk factor was assigned a risk score value based solely on its performance in the logistic model as indicated by its parameter estimate and weighted proportionately to the parameter estimate. This objective approach to developing the additive score avoids assigning risk score values to variables based on clinician judgments regarding each variable's potential impact on patient outcomes, which would have introduced bias into the weighting system. Finally, the model was internally validated on a large cohort that was collected at the same time as the initial study cohort data collection. Attention to each of these methodologic guidelines have been shown in previous studies to improve the performance and generalizability of the clinical prediction model 1, 16, 17. According to the observation by Swets et al. [18] that an area of 0.7 or more on ROC analysis is diagnostically useful, the QMMI rule performed very well in discriminating adverse outcomes (ROC curve area 0.80). Although further study is called for to determine the QMMI score's generalizability to other patient populations, the score performed very well on internal validation (ROC curve area 0.82). It is important to note that the practice of PTCA has changed and improved markedly since its introduction into clinical practice over 20 years ago, and even since the collection of the data used in this study, particularly with regard to the increasing use of coronary stenting and glycoprotein IIb/III inhibitors, improvements in the prevention of postprocedure restenosis, and decreasing lengths of stay after the procedure 19, 20, 21. Nonetheless, the QMMI clinical prediction model fills an important role in outcomes assessment and quality improvement in this patient population in that it was developed from a large, multi-institutional cohort of patients using rigorous methodologic standards, and can serve as a baseline for future analyses. The QMMI score can serve as a useful tool for both clinicians and methodologists alike. For the practicing clinician, it can be used to calculate risk of major adverse events after PTCA fairly quickly without the use of a computer or calculator, and thus may aid in optimizing patient care by risk stratifying patients prior to PTCA and allowing additional resources to be targeted to higher risk patients. For methodologists, the PTCA score can serve as an example of an objectively calculated, rigorously designed prediction model with good predictive performance, the construction techniques of which would be applied to future clinical prediction models in several areas of medicine. We believe the QMMI PTCA score breaks new methodologic ground in that the tool is an example of how multisite observational studies can be methodologically sound and display pertinent clinical results. Finally, while the overall adverse outcome rate in the practice of PTCA is likely to continue to decline as techniques improve, patient risk factors associated with postprocedure complications are likely to remain relatively stable. A carefully constructed risk stratification tool such as the QMMI PTCA score thus may have enduring usefulness in this evolving era of interventional cardiology. In addition to developing and validating the QMMI score, we assessed the ability of another published model for use in risk-stratifying patients undergoing PTCA to discriminate outcomes in the validation subset of the QMMI cohort [10]. Crossvalidation studies such as this are important in assessing a tool's ability to discriminate outcomes in a patient population other than that in which the tool was developed. In a study published in 1995, Kimmel and colleagues presented a predictive index for major complications occurring in patients undergoing PTCA [10]. The Kimmel et al. index was developed using data from 4,289 patients undergoing a first angioplasty procedure at 42 U.S. hospitals and subsequently validated on data from 5,250 patients collected from the same 42 hospitals during the following year. In discriminating outcomes on crossvalidation, the Kimmel score performed similarly in discriminating outcomes to how it performed in its own internal validation study and less well than the QMMI score (see Results). Although it stands to reason that the QMMI score will perform somewhat better than any other model compared to it in such a comparison study because the validation subset of the QMMI cohort includes data collected from patients receiving treatment at the same 12 institutions by the same personnel using the same set of parameters as the patient cohort on which the QMMI score was developed, except for the presence of aortic valve disease, the QMMI dataset contained all the variables required for calculation of the Kimmel et al. score and assessment of its dependent variable. Another potential explanation for this difference in discrimination performance between the QMMI and Kimmel scores is that although Kimmel et al collected data from 42 institutions, all were community hospitals and the total patient number in the derivation cohort was relatively small at 4,289. Models developed on small or unique patient cohorts may be less generalizable to other patient populations. Additionally, Kimmel et al. simplified their model by limiting the number of variables in the score to 7 and assigning a weight of 1 to all variables. Although such techniques may enhance ease of use of the tool, it may compromise precision. If the QMMI score is limited to the seven strongest independent predictor variables and each are assigned a score of 1, its ability to discriminate outcomes in the QMMI validation cohort diminishes (ROC area 0.73). 5. Conclusions In this study, we developed and internally validated a clinical prediction rule for use in patients undergoing PTCA. The rule is an additive severity score that can estimate a patient's risk of suffering an adverse outcome after PTCA, using data available prior to the procedure, with very good discrimination and reasonable calibration performance. Such tools may allow more accurate quality comparisons across institutions and physicians, help target specific resources to patients according to level of risk, and aid in making care decisions for individual patients. Although further research is needed to assess the QMMI score's generalizability to independent patient populations, these data suggest that it is a valid tool that can accurately stratify patients according to their risk of suffering an adverse outcome after PTCA. Our findings are consistent with those of current PTCA data collection registries 22, 23; future registries of PTCA procedures should continue to include these variables in the data collected. Acknowledgements A list of the members of the Academic Medical Center Consortium Quality Measurement and Management Initiative Working Group appears at the end of this article.
Appendix. AMCC Member Institutions and Working Group Members: (1) Brigham and Women's Hospital (Boston, MA): David Bates, MD, and Helen Burstin; (2) Dartmouth-Hitchcock Medical Center (Lebanon, NH): Eugene Nelson, ScD; (3) Duke University Medical Center (Durham, NC): James Jollis, MD; (4) Johns Hopkins Hospital (Baltimore, MD): Haya Rubin, MD; (5) Massachusetts General Hospital (Boston, MA): Elizabeth Mort, MD; (6) Mayo Foundation (Rochester, MN): Christopher Chute, MD; (7) New England Medical Center (Boston, MA): Allyson Ross Davies, PhD; (8) Alton Ochsner Medical Foundation (New Orleans, LA): Tonette Krousel-Wood, MD; (9) University of California, Los Angeles, Medical Center (Los Angeles, CA): Katherine Kahn, MD; (10) University of Iowa Hospitals and Clinics (Iowa City, IA): Robert Reiter, MD; (11) University of Pennsylvania Health System (Philadelphia, PA): David Shulkin, MD; and (12) University of Rochester Medical Center (Rochester, NY): Edgar Black, MD. Tools Committee: Academic Medical Center Consortium (AMCC): Lesley Curtis and David Witter. References 1.
1
Fortescue EB, Kahn K, Bates DW.
Prediction rules for major adverse outcomes in coronary bypass surgery (a comparison and methodological critique).
Med Care. 2000;38(8):820–835. MEDLINE |
CrossRef
2.
2
Detre KM, Holmes DR, Holubkov R, Cowley MJ, Bourassa MG, Faxon DP, et al.
Incidence and consequences of periprocedural occlusion (the 1985–1986 National Heart, Lung, and Blood Institute Percutaneous Transluminal Coronary Angioplasty Registry).
Circulation. 1990;82:739–750. MEDLINE 3.
3
Ellis SG, Roubin GS, King SB, Doublas JS, Weintraub WS, Thomas RG, et al.
Angiographic and clinical predictors of acute closure after native vessel coronary angioplasty.
Circulation. 1988;77:372–379. MEDLINE 4.
4
Ellis SG, Vandormael MG, Cowley MJ, DiSciascio G, Deligonul U, Topol EJ, et al.
Coronary morphologic and clinical determinants of procedural outcome with antioplasty for multivessel coronary disease (implications for patient selection).
Circulation. 1990;82:1193–1202. MEDLINE 5.
5
De Feyter PJ, van den Brand M, Jaarman GJ, van Domburg R, Serruys PW, Suryapranata H.
Acute coronary artery occlusion during and after percutaneous transluminal coronary angioplasty (frequency, prediction, clinical course, management, and follow-up).
Circulation. 1991;83:927–936. MEDLINE 6.
6
Kelsey SF, James M, Holubkov AL, Holubkov R, Cowley MJ, Detre KM.
Results of percutaneous transluminal coronary angioplasty in women (1985–1986 National Heart, Lung, and Blood Institute's Percutaneous Transluminal Coronary Angioplasty Registry).
Circulation. 1993;87:720–727. MEDLINE 7.
7
Bell MR, Holmes DR, Berger PB, Garratt KN, Bailey KR, Gersh BJ.
The changing in-hospital mortality of women undergoing percutaneous transluminal coronary angioplasty.
JAMA. 1993;269:2091–2095. MEDLINE 8.
8
Thompson RC, Holmes DR, Gersh BJ, Bailey KR.
Predicting early and intermediate-term outcome of coronary angioplasty in the elderly.
Circulation. 1993;88:1579–1587. MEDLINE 9.
9
Gowda MS, Vacek JL, Hallas D.
One-year outcomes of diabetic versus nondiabetic patients with non-Q-wave acute myocardial infarction treated with percutaneous transluminal coronary angioplasty.
Am J Cardiol. 1998;81(9):1067–1071. Abstract | Full Text |
Full-Text PDF (85 KB)
|
CrossRef
10.
10
Kimmel SE, Berlin JA, Strom BL, Laskey WK.
Development and validation of a simplified predictive index for major complications in contemporary percutaneous transluminal coronary angioplasty practice.
J Am Coll Cardiol. 1995;26:931–938. Abstract |
Full-Text PDF (771 KB)
|
CrossRef
11.
11
SAS Institute, Inc .
Statistical analysis system (SAS). Cary, NC: SAS Institute, Inc.;; 1986;. 12.
12
Harrell F.
The LOGIST procedure. SAS supplemental library user's guide. Cary, NC: SAS Institute, Inc.;; 1980;
. 13.
13
Metz CE.
Basic principles of ROC analysis.
Semin Nucl Med. 1978;8:283–298. MEDLINE |
CrossRef
14.
14
Lemeshow S, Hosmer DW.
A review of goodness of fit statistics for use in the development of logistic regression models.
Am J Epidemiol. 1982;115:92–106. MEDLINE 15.
15
Hosmer DW, Lemeshow S.
Applied logistic regression. New York: John Wiley & Sons, Inc.;; 1989;
. 16.
16
Laupacis A, Sekar N, Stiell IG.
Clinical prediction rules. A review and suggested modifications of methodological standards.
JAMA. 1997;277:488–494. MEDLINE 17.
17
Wasson JH, Sox HC, Neff RK, Goldman L.
Clinical prediction rules. Applications and methodological standards.
N Engl J Med. 1985;313:793–799. MEDLINE 18.
18
Swets JA.
Measuring the accuracy of diagnostic systems.
Science. 1988;240:1285–1293. MEDLINE 19.
19
Ellis SG, Guetta V, Miller D, Whitlow PL, Topol EJ.
Relation between lesion characteristics and risk with percutaneous intervention in the stent and glycoprotein IIb/IIIa era (an analysis of results from 10,907 lesions and proposal for new classification scheme).
Circulation. 1999;100(19):1971–1976. 20.
20
Mathew V, Garratt KN, Holmes DR.
Clinical outcome after multivessel coronary stent implantation.
Am Heart J. 1999;138(6 Pt 1):1105–1110. Abstract | Full Text |
Full-Text PDF (58 KB)
|
CrossRef
21.
21
Laskey WK, Kimmel S, Krone RJ.
Contemporary trends in coronary intervention (a report from the Registry of the Society for Cardiac Angiography and Interventions).
Cathet Cardiovasc Intervent. 2000;49:19–22. 22.
22
Block PC, Peterson ED, Krone R, Kesler K, Hannan E, O'Connor GT, et al.
Identification of variables needed to risk adjust outcomes of coronary interventions (evidence-based guidelines for efficient data collection).
J Am Coll Cardiol. 1998;32:275–282. Abstract | Full Text |
Full-Text PDF (87 KB)
|
CrossRef
23.
23
Harrell L, Schunkert EH, Palacios IF.
Risk predictors in patients scheduled for percutaneous coronary revascularization.
Cathet Cardiovasc Intervent. 1999;48:253–260. a Division of General Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Neville House Room 341, 75 Francis Street, Boston, MA 02115, USA b Divisions of General Internal Medicine and Health Services Research, UCLA Medical Center, B-976 Factor Building, 10833 La Comte Ave., Los Angeles, CA, 90095-173 USA  Corresponding author. Tel.: 617- 732-7063; fax: 617-732-7072.
PII: S0895-4356(02)00538-3 © 2003 Elsevier Science Inc. All rights reserved. | |
|
FULL TEXT00538-3/journalimage?img=PIIS0895435602005383.gr1.lrg.gif&fig=FIG1&kwhquery=&issn=0895-4356&ishighres=false&allhighres=false&free=yes','journalimage');)
00538-3/journalimage?img=PIIS0895435602005383.gr2.lrg.gif&fig=FIG2&kwhquery=&issn=0895-4356&ishighres=false&allhighres=false&free=yes','journalimage');)
00538-3/journalimage?img=PIIS0895435602005383.gr3.lrg.gif&fig=FIG3&kwhquery=&issn=0895-4356&ishighres=false&allhighres=false&free=yes','journalimage');)
00538-3/journalimage?img=PIIS0895435602005383.gr4.lrg.gif&fig=FIG4&kwhquery=&issn=0895-4356&ishighres=false&allhighres=false&free=yes','journalimage');)