文献速递流行病学与生物统计

预后模型的性能取决于缺失值插补算法的选择:一项模拟研究

The performance of prognostic models depended on the choice of missing value imputation algorithm: a simulation study.
PubMed 文献 · J Clin Epidemiol · 2024-09-24

该模拟研究基于真实数据集分布生成开发队列与验证队列,在36种缺失机制与不完全病例比例场景下比较mice、aregImpute和missForest三种插补算法对临床预测模型性能的影响。结果显示完整病例分析性能最差,aregImpute与mice(100次插补)预测准确性最高,模型校准受插补方法选择的影响大于区分度。

要点

  • 完整病例分析在缺失数据场景下模型性能最差。
  • aregImpute与mice(100次插补)在各场景下预测准确性最高。
  • 模型校准受插补方法选择的影响大于模型区分度。

核心结论

  • 在本模拟研究中,模型校准受插补方法选择的影响大于模型区分度。
  • 尽管不同插补方法间的模型性能差异通常较小,能处理预测变量与结局间线性或非线性关联的多重插补方法如mice和aregImpute在多数情况下是有吸引力且可靠的选择。

研究设计与样本

  • 通过模拟真实数据集中预测变量与结局变量的观测分布来生成开发队列和验证队列。
  • 在开发队列中,按缺失机制和不完全病例比例定义36种场景创建缺失预测变量值。
  • 应用R软件中的三种插补算法:mice、aregImpute和missForest。
  • 拟合逻辑回归模型,比较完整数据分析、完整病例分析和插补数据后的模型性能。
  • 在无缺失值的验证队列中评估模型性能,以完整数据分析的性能作为理想参照。

主要发现

  • 没有一种插补方法能达到无缺失情况下模型的预测准确性。
  • 完整病例分析通常性能最差,偏离理想性能的程度随缺失百分比增加和样本量减小而增大。
  • 在所有场景和性能指标中,aregImpute和mice(均为100次插补)的预测准确性最高。
  • aregImpute在所有场景和结局模型中实现的校准斜率非常接近1,优于完整数据分析。
  • mice从5次插补增加到100次插补的性能提升仅为边际性的。

关键数值与推荐

  • 预后模型性能通过缩放Brier评分、c统计量、校准截距和斜率以及在无缺失验证队列中评估的平均绝对预测误差来衡量。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: The development of clinical prediction models is often impeded by the occurrence of missing values in the predictors. Various methods for imputing missing values before modeling have been proposed. Some of them are based on variants of multiple imputations by chained equations, while others are based on single imputation. These methods may include elements of flexible modeling or machine learning algorithms, and for some of them user-friendly software packages are available. The aim of this study was to investigate by simulation if some of these methods consistently outperform others in performance measures of clinical prediction models. STUDY DESIGN AND SETTING: We simulated development and validation cohorts by mimicking observed distributions of predictors and outcome variable of a real data set. In the development cohorts, missing predictor values were created in 36 scenarios defined by the missingness mechanism and proportion of noncomplete cases. We applied three imputation algorithms that were available in R software (R Foundation for Statistical Computing, Vienna, Austria): mice, aregImpute, and missForest. These algorithms differed in their use of linear or flexible models, or random forests, the way of sampling from the predictive posterior distribution, and the generation of a single or multiple imputed data set. For multiple imputation, we also investigated the impact of the number of imputations. Logistic regression models were fitted with the simulated development cohorts before (full data analysis) and after missing value generation (complete case analysis), and with the imputed data. Prognostic model performance was measured by the scaled Brier score, c-statistic, calibration intercept and slope, and by the mean absolute prediction error evaluated in validation cohorts without missing values. Performance of full data analysis was considered as ideal. RESULTS: None of the imputation methods achieved the model's predictive accuracy that would be obtained in case of no missingness. In general, complete case analysis yielded the worst performance, and deviation from ideal performance increased with increasing percentage of missingness and decreasing sample size. Across all scenarios and performance measures, aregImpute and mice, both with 100 imputations, resulted in highest predictive accuracy. Surprisingly, aregImpute outperformed full data analysis in achieving calibration slopes very close to one across all scenarios and outcome models. The increase of mice's performance with 100 compared to five imputations was only marginal. The differences between the imputation methods decreased with increasing sample sizes and decreasing proportion of noncomplete cases. CONCLUSION: In our simulation study, model calibration was more affected by the choice of the imputation method than model discrimination. While differences in model performance after using imputation methods were generally small, multiple imputation methods as mice and aregImpute that can handle linear or nonlinear associations between predictors and outcome are an attractive and reliable choice in most situations.

来源:J Clin Epidemiol,查看原文。版权归原作者所有。

引用

  1. Deforth M, Heinze G, Held U, et al. The performance of prognostic models depended on the choice of missing value imputation algorithm: a simulation study. J Clin Epidemiol. 2024. PMID: 39326470. doi:10.1016/j.jclinepi.2024.111539

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。