Developing a prognostic model in the presence of missing data: an ovarian cancer case study

2.1. Study population 

We used data from the 1,189 patients diagnosed at the Western General Hospital (Edinburgh) between 01/01/1984 and 31/12/1999 with primary cases of epithelial ovarian cancer [5]. The date of diagnosis refers to the time at which tissue samples were taken. Follow-up data were available up until the end of February 2001. Patients were aged between 15 and 90 years at the time of diagnosis (median 61 years), presented at initial diagnosis with predominantly FIGO stages III and IV (64.6%), and predominantly with a serous papillary histology (51.8%). One thousand one hundred forty (95.9%) patients underwent surgery (their date of diagnosis), 894 (72.3%) patients received chemotherapy, and 860 (75.4%) had both surgery and chemotherapy. Of those patients who received chemotherapy, 654 (73.2%) patients were treated with single agent platinum regimens.

2.2. Outcomes of interest 

Overall survival, or equivalently, time to death, was the outcome of interest.

2.3. Prognostic factors 

The prognostic factors investigated were age at diagnosis, FIGO stage, grade of tumor (I = well differentiated, II = moderately differentiated, III = poorly differentiated), histology, the presence or absence of ascites, the diameter of the largest residual tumor mass after primary cytoreductive surgery (<2 cm, 2–5 cm, and >5 cm), performance status using the ZUBROD-ECOG-WHO scale (0 = Normal activity, 1 = Symptoms, but nearly ambulatory, 2 = Some bed time, but needs to be in bed less than 50% of the normal daytime, 3 = Needs to be in bed more than 50% of the normal daytime, and 4 = Unable to get out of bed), and the first CA125, alkaline phosphatase, and albumin laboratory test results that fell between diagnosis and 7 days after the first chemotherapy.

2.4. Auxiliary variables 

There were a number of additional variables that are not considered as factors in prognostic studies, but were potentially associated with missing data. These included: surgery (yes, no), clinical trial participation (yes, no), and chemotherapy regimen (none, platinum-based, nonplatinum based). These variables were free of missing data.

2.5. Missing data 

We were only interested in missing values of variables that are potentially prognostic. Of the 10 prognostic variables considered, 8 had noncomplete data with the proportion of missing data varying between 2 and 43%. We did not assume that our missing data were a random sample of the whole dataset, that is, missing completely at random (MCAR) [6]. Instead, we have assumed that the missing data is missing at random (MAR) [6]. This missing data mechanism assumes that the probability that a data value is missing depends on values of variables that were actually measured. In other words, we assumed that the missingness of a variable cannot depend on the values of variables that we did not collect data on. Conceptually, it also excludes a dependence of the occurrence of missing values on the true, but unobserved, value of the variable (called missing not at random—MNAR; [6]).

Most reasonably sophisticated statistical techniques assume the data are MAR. These methods can be broadly classified as either imputation approaches or likelihood-based approaches [7] (where the Expectation-Maximization algorithm is usually applied). The major distinction is that the imputation methods fill in missing data, while likelihood-based approaches do not require the missing data to be estimated explicitly. The latter approach requires correct model specification, software is less readily available for survival data, and it is more difficult to model a mixture of binary, polytomous, and continuous prognostic factors in this setting. Hence, we have applied an imputation procedure known as multiple imputation 6, 8.

2.6. Multiple imputation 

The simplest form of imputation or “filling in the missing data” is “single imputation,” where missing values are estimated using single “best guesses” (e.g., the mean of other values). However, an analysis using the resulting “complete” dataset, as if the imputed values were real measurements, ignores the uncertainty in the imputation process. In contrast, multiple imputation (MI) is the analysis of multiple “complete” datasets, and incorporates an adjustment of standard errors and other statistics for the imputation uncertainty.

We applied a MI framework to account for missing prognostic factor data, and used Bayesian simulation to generate the missing data. Our approach is similar to that described by Van Buuren et al. [9]. Suppose that the statistic of interest in the analysis—here the log of the hazard ratio (HR)—is represented by a quantity θ. (In reality θ is a vector of log HRs for different variables, but without loss of generality we have simplified the notation). The steps in the MI procedure we applied were:

1.Investigating the missing data

2.Quantifying the multivariate patterns of the missing data.

3.Plotting the proportion of missing data for each potential prognostic factor against diagnosis year to show time trends in measurement practice.

4.Exploring the relationship between missing data of potential prognostic factors with other prognostic variables, survival information [i.e., (log) survival time and the censoring indicator], and auxiliary variables.

5.Specifying an imputation model.

6.Using the model to generate (via a random sampling procedure) M sets of imputed values for the missing data points, thus creating M completed datasets.

7.For each completed dataset, carrying out a Cox regression (see below), obtaining estimate of interest and its estimated variance vr for i = 1… M.

8.Combining the results from the different datasets to obtain a prognostic model. The MI estimate of θ is

9.e., the mean across the models fitted to the imputed datasets) and the MI estimate of variance is

10.Constructing a final “completed data” model (later referred to as Model 2) by removing the covariate with the highest P-value and repeating steps 4 and 5 until all remaining covariates were significant at a 5% level (backward elimination).

The first term after the equals sign relates to the variance within the imputed datasets, whereas the second term captures the uncertainty due to the variability in the estimates θi from the imputed datasets. The term 1+1/M is a bias correction factor.

Simulation studies have shown that the required number of repeated imputations (M) can be as low as three for data with 20% of missing entries [9]. We had three predictors with approximately twice this percentage, and decided that 10 repeated imputations (i.e., M = 10) would be a conservative choice. Unless rates of missing information are unusually high, there tends to be little or no practical benefit to using more than 10 imputations [6]. In the following sections we describe some aspects of steps 1 to 3, where steps 2 and 3 are the Bayesian simulation. These sections are quite technical, especially the sections describing the imputation model and the sampling process.

2.7. Investigating the missing data 

As we assumed that the missing data were MAR, we looked for evidence of MAR data by assessing associations between missing data and observed variables within our dataset. The relationships between missing potential prognostic factors and survival time were explored using Kaplan Meier curves stratified by a missing value indicator for each of the potential prognostic factors of interest. The log-rank method was used to test for differences between the distribution of survival times for the missing data and nonmissing data for a particular potential prognostic factor. For each prognostic variable the relationship between missingness (yes vs. no) and other variables was assessed using univariate logistic regressions. A more correct approach would be to produce adjusted survival curves and multivariate logistic regressions, but as the adjustment requires prognostic factors, some of which are missing data, this was not possible.

2.8. Imputation model 

Step 2 is an explicit attempt to model the MAR process. Imputation models were specified for each potential prognostic factor with missing data irrespective of the quantity of missing data. The purpose of the models was to provide a set of plausible values for the missing potential prognostic factor data, and involved two modeling choices: the form of the model (linear, logistic, etc.) and the set of predictors that enter the model. The form is considered to be less important, and is largely immaterial unless the uncertainty associated with the missing entries is small [9].

For binary variables (e.g., the presence or absence of ascites) we used a logistic model, for categorical variables with three or more ordered levels (e.g., performance status) we applied a polytomous (>2 levels) logistic model, and for continuous variables (e.g., log CA125) we used normal linear regression truncated where appropriate to the credible range of values.

Composition of the sets of predictors in these models was based on guidelines in Van Buuren et al. (1999) [9]. In particular, all variables that appear in a model constructed using complete cases should be included in the imputation models. Failure to do so may bias the complete data analysis. It has also been observed that including as many predictors in the imputation model as possible tends to make the MAR assumptions more plausible. However, we recognized that there may be no substantial increase in explained variance after selecting a certain number of predictors, and identified computational problems associated with multicollinearity and inclusion of many predictors with missing data. We proposed to confine the imputation model for each prognostic factor to include all other prognostic variables from Table 1, survival variables, and auxiliary variables.

Table 1. Potential prognostic factors

		n = 1189
	Prognostic variable	N	(%)	Median (range)	P–value
	Age at diagnosis (years)	1189	(100.0)	61 (15–90)	<.001
	FIGO stage				<.001
	I	281	(23.6)
	II	119	(10.0)
	III	590	(49.6)
	IV	178	(15.0)
	Unknown	21	(1.8)
	Grade				<.001
	I	131	(11.0)
	II	278	(23.4)
	III	641	(53.9)
	Unknown	139	(11.7)
	Histology				<.001
	Serous papillary	616	(51.8)
	Endometrioid	240	(20.2)
	Mucinous	131	(11.0)
	Mesonephroid (clear cell)	101	(8.5)
	Mixed mesodermal	42	(3.5)
	Adenocarcinoma	38	(3.2)
	Undifferentiated	21	(1.8)
	Ascites				<.001
	Presence	707	(59.5)
	Absence	417	(35.1)
	Unknown	65	(5.5)
	Performance status (Zubrod)				<.001
	0	328	(27.6)
	1	228	(19.2)
	2	84	(7.1)
	3+4	38	(3.2)
	Unknown	511	(43.0)
	Residual disease				<.001
	<2 cm	641	(53.9)
	2–5 cm	165	(13.9)
	> 5 cm	302	(25.4)
	Unknown	81	(6.8)
	CA125	749	(63.0)	208 (6–22878)	—
	Log(CA125)	749	(63.0)	5.3 (1.8–10.0)	<.001
	Alkaline phosphatase (ALP)	793	(66.7)	94 (26–1810)	—
	Log(ALP)	793	(66.7)	4.5 (3.3–7.5)	<.001
	Albumin	797	(67.0)	39 (20–50)	<.001

For example, the imputation model for ascites would have a logistic form with age, FIGO stage, grade, histology, residual disease, performance status, log CA125, log alkaline phosphatase, albumin, log survival time, death, surgery, clinical trial participation, and chemotherapy regimen as predictors.

2.9. Creating 10 complete datasets 

To generate replacement data for the missing values in each of the 10 imputations, we did not assume a particular form for the multivariate distribution of our data. This contrasts with the approach of Schafer (1997), which assumes the distribution to be a multivariate Normal for continuous data, a log-linear model for categorical data, or a general location model for a mixture of continuous and categorical. We assumed only that a multivariate distribution existed, and that Gibbs sampling [10] of the conditional distributions (based on the models above) could generate samples from it. Specifically, this was an iterative process in which the missing data for a prognostic variable were estimated using its imputation model, and then these completed data were used in the estimation of the missing values for another variable. Each iteration ended when all variables had been updated. The application of Gibbs sampling ensured that the imputation process was not deterministic, there was random variation between the completed datasets, and is the main reason the approach is considered (misleadingly) to be a form of Bayesian simulation. The Gibbs sampling algorithm was run for 80 iterations updating for each of the 10 imputations. The time taken for each variable was a function of the prevalence of missing values. It has been found that in the presence of large amounts of missing data, convergence can be obtained in as few as 10 iterations [9]. By plotting the standard deviations and means of the imputations by iteration, an assessment of convergence was made. These plots allowed us to assess whether between imputation-variability had stabilized and whether estimates were free of trend.

The frequency distributions of prognostic variables in the 10 datasets with imputed missing values were compared with the frequency distribution in the original dataset using histograms and basic summary statistics (e.g., medians, ranges, and proportions). This comparison assessed the extent to which the imputed datasets were consistent among themselves and with the original data.

2.10. Statistical models 

Univariate analyses on continuous prognostic factors were performed using Cox regression [10], and categorical (including categorized continuous) prognostic factors were analyzed using the Kaplan-Meier and the log-rank methods. The linearity of continuous variables was assessed using fractional polynomials [11]. Univariate analyses were performed on complete cases in the original dataset.

The fundamental method of multivariate analysis was Cox regression [12]. The proportional hazard assumption for each predictor was tested using an approximate score statistic of linear correlation between the rank order of failure times in the sample and Schoenfeld partial residuals [13].

Like the final “completed data” model, the final “complete case” model (later referred to as Model 1) was obtained using backward elimination, but was based on the original data and not the completed data.

2.11. Measures of model performance 

We evaluated the predictive performance of models by considering measures of discrimination and calibration. Discrimination refers to the ability to distinguish between high risk and low risk patients, and was quantified using the c-index [13] (a generalization of the area under the Receiver Operating Characteristic (ROC) curve [13]) and the Nagelkerke R2 (RN2) [13]. The c-index is the probability of concordance between predicted and observed survival based on pairs of individuals, with c = 0.5 for random predictions and c = 1 for a perfectly discriminating model. Similarly, RN2 = 0 indicates no predictive ability and RN2 = 1 indicates perfect predictions. The term predictive ability is synonymous with discrimination and describes the ability of a set of prognostic factors to explain variation in outcome. In theory, high predictive ability implies accurate prediction for individual patients.

Calibration refers to whether the predicted probabilities agree with observed probabilities, or equivalently, indicates the ability of the model to make unbiased estimates of outcome. Calibration was quantified using an estimate of slope shrinkage and was based on partitioning the patients into 10 risk groups [13]. Shrinkage is the flattening of the plot of predicted (x-axis) and observed (y-axis) probabilities away from the 45 degree line, caused by overfitting. The estimate of slope was based on the mean slope from 200 bootstrap samples of the data. An adjusted slope close to one indicates little shrinkage and good calibration.

2.12. Statistical software 

All statistical analyses were carried out with S-Plus 2000 using the Hmisc, Design, and MICE libraries. MICE is available from www.multiple-imputation.com.

3.1. Long-term outcome 

Follow-up information was available on all 1,189 patients, but some were lost to follow-up before the end of study period. 842 (70.8%) patients had died at the time of censoring the data. Median follow-up in the 347 (29.2%) patients who were not known to have died was 1,665 days (range 29–5,852 days). Five-year survival in the cohort was 29.6% (95% CI: 26.8 to 32.5%) in keeping with international mortality rates.

3.2. Missing data 

The potential prognostic factors are summarized in Table 1. Performance status levels 3 and 4 were combined because of small numbers. CA125 and alkaline phosphatase laboratory results were skewed, so a summary of both variables transformed using natural logs is also presented. Data on age and histology were complete for all patients. Performance status, CA125, alkaline phosphatase, and albumin were missing for 43.0, 37.0, 33.3, and 33.0% of patients, respectively. Of the 1,1890 prognostic factor data values, 2,045 (17.2%) were missing in 831 (69.9%) patients, representing less than two missing values per individual in the entire cohort. Two hundred thirty-six (19.8%) patients had four or more missing values, but only 4 (0.4%) patients with seven or more missing prognostic factors. One thousand seven hundred thirty-nine (85.0%) of the missing cells resulted from missing information on albumin, alkaline phosphatase, CA125, and performance status. The number of patients contributing to a complete case analysis using all the prognostic factors in Table 1 would be 358 (245 deaths).

Plots of the proportion of missing data by diagnosis year show that the proportions for ascites, alkaline phosphatase, albumin, grade, and residual disease were constant. In contrast, the proportion of missing CA125 data decreased linearly in time from 85 to 21% between 1984 and 1999, and the proportion of missing performance status had an increasing trend in time with a minimum of 18% in 1986 and a maximum of 71% in 1995. These trends can be explained by the fact that CA125 was not commonly reported in the literature until the mid-1980s, and there has been decrease in prominence of performance status as a prognostic factor in the ovarian cancer literature.

3.3. Evidence of MAR data 

An analysis of the survival distributions of non-missing and missing strata within each of the factors (log) CA125, grade, FIGO stage, and performance status (Fig. 1), showed no visual or statistical evidence of significant differences (Table 2). However, there was a difference between the survival distributions of patients with and without missing data for ascites (P < .002) (Fig. 1), albumin (P < .003), alkaline phosphatase (P = .020) (Fig. 1), and residual disease (P = .020) (Fig. 1). Those patients missing albumin and alkaline phosphatase results had a better prognosis, suggesting that eliminating the patients with missing values would lead to an underestimate of the true survival of the cohort. The opposite effect was seen for ascites and residual disease. Those patients missing ascites data may be more likely to have ascites present at diagnosis. However, as the survival curves are not adjusted for other variables, these effects may not be real and, of course, in theory one might miss an effect if masked by some other association.

View Large Image Download to PowerPoint

Fig. 1. Differences in survival of patients with and without recorded performance status, ascites, residual disease, and alkaline phosphatase, tested using log rank methods.

Table 2. Associations between missingness and other potential prognostic factors, auxiliary variables and survival time

		Associated with
		Prognostic variables										Auxiliary variables
	Missingness of	Age	FIGO	Grade	Histology	Ascites	Perf. Stat.	Residual disease	Log CA125	Log ALKP	Albumin	Surgery	Clinical Trial	Chemo.	Survival Timea
	FIGO stage	—		—	X	—	—	—	—	—	—	X	—	—	—
	Grade	X	—		X	—	—	X	—	—	—	X	X	X	—
	Ascites	X	—	—	—		—	—	—	—	—	X	X	X	<<
	Performance status	X	X	X	X	X		X	—	—	X	X	X	X	—
	Residual disease	X	X	X	X	—	—		—	—	—	—	X	—	<<
	Log CA125	—	X	X	X	X	X	—		—	—	X	X	X	—
	Log ALKP	X	X	X	X	X	X	X	—		X	—	X	X	>>
	Albumin	X	X	X	X	X	X	X	—	—		X	X	X	>>

a Based on a log rank test comparing survival distributions in missing and nonmissing groups, X = an association found using univariate logistic regression (P<.05), << those with missing data had a significantly worse survival, >> those with missing data had a significantly better survival,— no association.

The univariate logistic models indicated that the missingness of potential prognostic factors were (potentially) associated with other potential prognostic factors and auxiliary variables. Histology and clinical trial participation were associated with the missingness of all but one prognostic variable (Table 2). These results must be interpreted with the same caution because they were from univariate analyses.

3.4. Imputed data 

We completed 10 data sets by imputing 2,045 values in each. As a consequence, 6,265 additional real data values were incorporated into each dataset. The summary statistics for the datasets with imputed data are shown in Table 3. The narrow ranges of imputation values for each potential prognostic variable coincides with the visual impression that the distributions for each of the potential prognostic factors in the 10 imputed datasets were similar. The prevalences (%) of categorical prognostic factors in the original data (ignoring missing data) were consistent with those from the 10 imputations. The median and range of albumin, log CA125, and alkaline phosphatase in the original data were consistent with the median of the median of the 10 imputation distributions, and the extreme values of these distributions, respectively.

Table 3. Results of the missing data imputation compared with original data

		Original		Completeda
	Prognostic Factor	# or (median)	% or (range)	Median	Range	Overall %
	FIGO stage
	I	281	24.1	286	284–289	24.1
	II	119	10.2	122	120–124	10.2
	III	590	50.5	599	599–602	50.4
	IV	178	15.2	183	179–186	15.3
	Unknown	21	0	—	—	—
	Grade
	I	131	12.5	149	144–153	12.5
	II	278	26.5	315	310–321	26.5
	III	641	61.0	724	716–732	60.9
	Unknown	139	0	—	—	—
	Ascites
	Presence	707	62.9	750	747–752	63.0
	Absence	417	37.1	440	437–442	37.0
	Unknown	65	0	—	—	—
	Perf. Stat
	0	328	48.4	581	569–590	48.9
	1	228	33.6	369	363–382	31.1
	2	84	12.4	151	141–158	12.6
	3+4	38	5.6	87	81–104	7.4
	Unknown	511	0	—	—	—
	Res. Disease
	<2cms	641	57.9	681	676–687	57.3
	2–5cms	165	14.9	181	177–186	15.2
	>5cms	302	27.3	326	323–333	27.5
	Unknown	81	0	—	—	—
	Log CA125	(5.34)	(1.79–10.04)	5.16	1.79–10.04	—
	Albumin	(39.0)	(20.0–50.0)	39.0	20.0–50.0	—
	Log Alk. Phos.	(4.54)	(3.26–7.50)	4.54	3.26–7.50	—

a Ten datasets with original data augmented by imputed missing values.

3.5. Univariate survival analysis 

Univariate analyses suggested that age, FIGO stage, the presence or absence of ascites, performance status, histology, residual disease, albumin, grade, and (log) CA125 and alkaline phosphatase were significant (P < .05) prognostic factors of overall survival (Table 1). Investigations of linearity of CA125, alkaline phosphatase, age, and albumin in the univariate models suggested that CA125 and alkaline phosphatase required a natural logarithmic transformation.

3.6. Cox models 

Table 4 shows the results from fitting the Cox models. The first column represents an analysis of complete cases in the original data (Model 1). As four factors, each with missing values, were found not to be prognostic, the analysable dataset was 518 (380 deaths), that is, larger than the 358 patients available when fitting the full model. The second column represents a pooled analysis using 10 complete datasets with imputed missing values (Model 2). Age, FIGO stage, performance status, residual disease, and (log) alkaline phosphatase were significant (P < .05) prognostic factors in both models. Specifically, there was a greater risk of mortality associated with older age, higher FIGO stage, the presence of ascites, worse performance status, and greater residual disease and alkaline phosphatase levels. In both models, the histology comparison between the “Serous” and “Mixed Mesodermal” groups was significant (P < .02). Using a likelihood ratio test the overall histology effects in each model were significant (P < .001). There were a greater number of statistically significant differences (P < .05) between “Mixed Mesodermal” and the other histologies in Model 2 than in Model 1. Grade and ascites were statistically significant in Model 2, but not in Model 1. Log CA125 (P = .95) and albumin (P = .06) were not statistically significant prognostic variables in either model, and therefore, not included in Table 4. It is worth noting that a complete case analysis based on Model 2 would include only 449 patients (319 deaths).

Table 4. Prognostic models for overall survival

		Multivariate models
		Model 1a (n = 518, # deaths=380)	Model 2b (n = 1189, # deaths = 842)
	Predictors	HR (95% CI)	HR (95% CI)
	Age (years)	1.013 (1.002,1.023)	1.022 (1.015,1.030)
	FIGO stage
	I	1.000	1.000
	II	2.227 (1.297,3.823)	1.842 (1.335,2.541)
	III	4.154 (2.580,6.689)	3.207 (2.430,4.232)
	IV	4.477 (2.607,7.687)	3.035 (2.127,4.330)
	Grade
	I	—	1.000
	II	—	1.506 (1.075,2.110)
	III	—	1.621 (1.164,2.258)
	Histology
	Serous papillary	1.000	1.000
	Endometrioid	0.698 (0.520,0.936)	0.827 (0.671,1.020)
	Mucinous	0.873 (0.554,1.375)	0.932 (0.683,1.272)
	Clear cell	1.372 (0.933,2.018)	1.357 (1.027,1.795)
	Mixed mesodermal	2.540 (1.404,4.596)	2.095 (1.446,3.036)
	Adenocarcinoma	1.361 (0.594,3.115)	1.273 (0.834,1.941)
	Undifferentiated	1.649 (0.911,2.986)	1.570 (0.968,2.547)
	Ascites
	Absence	—	1.000
	Presence	—	1.395 (1.181,1.648)
	Perf. status
	0	1.000	1.000
	1	1.183 (0.924,1.515)	1.144 (0.942,1.388)
	2	1.490 (1.044,2.127)	1.535 (1.198,1.967)
	3+4	3.309 (1.959,5.591)	3.977 (2.518,6.281)
	Residual disease
	>5 cm	1.000	1.000
	2–5 cm	0.903 (0.669,1.219)	0.900 (0.721,1.123)
	<2 cms	0.543 (0.410,0.720)	0.511 (0.417,0.627)
	Log alkaline phos.	1.732 (1.324,2.266)	1.637 (1.238,2.166)

a Model 1 was constructed using complete cases in the original dataset. b Model 2 was constructed using 10 datasets with original values augmented by imputed missing data.

The confidence limits are narrower in the augmented data, especially for those with less missing observations in the original dataset, which is a direct result of including three additional actual values for each imputed value.

There was no evidence of any violation of the proportional hazard assumptions in any model.

3.7. Measures of model performance 

Table 5 shows the measures of predictive accuracy for Model 1 and Model 2 applied to the original data and completed data (i.e., original data augmented with imputed missing values). Models applied to the completed data had slope shrinkage values close to 1, indicating less overfitting and less need for recalibration of prognostic factor effects. In addition, the models applied to completed data had substantially better measures of discrimination.

Table 5. Measures of predictive accuracy for Models 1 and 2

		Multivariate models
		Complete case analysis		Completed datab
		Model 1 (n = 518)	Model 2 (n = 454)a	Model 1 (n = 1189)	Model 2 (n = 1189)
				Median (range)c
	Slope	0.864	0.830	0.951 (0.941–0.954)	0.947 (0.939–0.958)
	c-index	0.751	0.745	0.783 (0.779–0.786)	0.785 (0.780–0.787)
	RN2	0.411	0.402	0.499 (0.486–0.512)	0.510 (0.498–0.524)

See Table 4 for composition.

a 326 deaths. b Original complete cases augmented by imputed missing data. c Median and range from the 10 analyses.

Model 1 was marginally inferior to Model 2 in overall discrimination when applied to the completed data. However, high values of the c-index (∼0.78) and Nagelkerke R2 (∼0.50) for both models applied to the completed data indicate that their respective set of prognostic factors were explaining the variation in outcome reasonably well.

If we compare the discrimination between Model 1 and Model 2 when applied to the datasets used in their construction, there is clear difference. Figure 2 (left) shows this difference. Four risk groups were created for each of Model 1 and Model 2 by partitioning linear predictions from the respective models into quartiles. The plot shows there is greater separation between risk groups using Model 2, particularly at lower survival times. The differences are a result of increased sample size and two extra prognostic factors. Figure 2 (right) shows the effect of increased sample size only, with the same risk groups from Model 2 as before, applied to all patients and only those with complete data for Model 2. This plot shows that a complete case analysis using Model 2 may overestimate the survival in both risk group 2 after 3 years and risk group 4 in the first year. In both plots, the confidence intervals around the survival curves are narrower in the augmented data.

View Large Image Download to PowerPoint

Fig. 2. Risk groups and their survival using Models 1 and 2 (see Table 4 for composition).