Internal validation of predictive models: efficiency of some procedures for logistic regression analysis

EW Steyerberg, FE Harrell Jr, GJJM Borsboom… - Journal of clinical …, 2001 - Elsevier
EW Steyerberg, FE Harrell Jr, GJJM Borsboom, MJC Eijkemans, Y Vergouwe…
Journal of clinical epidemiology, 2001Elsevier
The performance of a predictive model is overestimated when simply determined on the
sample of subjects that was used to construct the model. Several internal validation methods
are available that aim to provide a more accurate estimate of model performance in new
subjects. We evaluated several variants of split-sample, cross-validation and bootstrapping
methods with a logistic regression model that included eight predictors for 30-day mortality
after an acute myocardial infarction. Random samples with a size between n= 572 and n …
The performance of a predictive model is overestimated when simply determined on the sample of subjects that was used to construct the model. Several internal validation methods are available that aim to provide a more accurate estimate of model performance in new subjects. We evaluated several variants of split-sample, cross-validation and bootstrapping methods with a logistic regression model that included eight predictors for 30-day mortality after an acute myocardial infarction. Random samples with a size between n = 572 and n = 9165 were drawn from a large data set (GUSTO-I; n = 40,830; 2851 deaths) to reflect modeling in data sets with between 5 and 80 events per variable. Independent performance was determined on the remaining subjects. Performance measures included discriminative ability, calibration and overall accuracy. We found that split-sample analyses gave overly pessimistic estimates of performance, with large variability. Cross-validation on 10% of the sample had low bias and low variability, but was not suitable for all performance measures. Internal validity could best be estimated with bootstrapping, which provided stable estimates with low bias. We conclude that split-sample validation is inefficient, and recommend bootstrapping for estimation of internal validity of a predictive logistic regression model.
Elsevier
以上显示的是最相近的搜索结果。 查看全部搜索结果