文献速递流行病学与生物统计医学人工智能与信息学

多变量预测模型预测准确度指标的自助法乐观度校正置信区间

Confidence intervals of prediction accuracy measures for multivariable prediction models based on the bootstrap-based optimism correction methods.
PubMed 文献 · Stat Med · 2021-07-24

本文针对多变量预测模型预测准确度评估中乐观度校正的问题,指出目前多数论文仅校正点估计而未校正置信区间(如常用DeLong法估计C统计量),导致覆盖概率可能严重低于名义水平。作者提出两种通用自助法:位置平移自助置信区间与两阶段自助置信区间,可配合Harrell偏差校正、0.632与0.632+方法使用,并适用于ridge、lasso等收缩方法。模拟显示所提区间覆盖表现良好,而未校正乐观度的方法严重覆盖不足,建议临床实践采用校正方法。

要点

  • 预测准确度点估计经自助法校正后,其置信区间也须校正乐观度偏倚。
  • 未校正乐观度的置信区间覆盖概率可严重低于名义水平,不应在实践中使用。
  • 所提两种自助置信区间可配合Harrell、0.632与0.632+校正及ridge、lasso开发方法。

核心结论

  • 预测准确度指标的置信区间应采用乐观度校正方法,未校正乐观度的置信区间不应在实践中使用,推荐使用所提校正方法。

研究设计与样本

  • 方法学文章,提出两种通用自助置信区间方法并配合三类自助乐观度校正方法,通过模拟进行数值评估,并以GUSTO-I临床试验作应用示例。

主要发现

  • 模拟中提出的置信区间显示出良好的覆盖表现;基于未校正乐观度方法的现行标准做法表现出严重的覆盖不足。

关键数值与推荐

  • 原文以名义水平95%为例说明未校正方法的覆盖概率可严重低于名义水平,未报告具体数值。

临床怎么用

  • 评估多变量预测模型预测准确度时,点估计与置信区间均应进行乐观度校正,可采用位置平移或两阶段自助置信区间,并可使用R包predboot实现。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

In assessing prediction accuracy of multivariable prediction models, optimism corrections are essential for preventing biased results. However, in most published papers of clinical prediction models, the point estimates of the prediction accuracy measures are corrected by adequate bootstrap-based correction methods, but their confidence intervals are not corrected, for example, the DeLong's confidence interval is usually used for assessing the C-statistic. These naïve methods do not adjust for the optimism bias and do not account for statistical variability in the estimation of parameters in the prediction models. Therefore, their coverage probabilities of the true value of the prediction accuracy measure can be seriously below the nominal level (eg, 95%). In this article, we provide two generic bootstrap methods, namely, (1) location-shifted bootstrap confidence intervals and (2) two-stage bootstrap confidence intervals, that can be generally applied to the bootstrap-based optimism correction methods, that is, the Harrell's bias correction, 0.632, and 0.632+ methods. In addition, they can be widely applied to various methods for prediction model development involving modern shrinkage methods such as the ridge and lasso regressions. Through numerical evaluations by simulations, the proposed confidence intervals showed favorable coverage performances. Besides, the current standard practices based on the optimism-uncorrected methods showed serious undercoverage properties. To avoid erroneous results, the optimism-uncorrected confidence intervals should not be used in practice, and the adjusted methods are recommended instead. We also developed the R package predboot for implementing these methods ( https://github.com/nomahi/predboot). The effectiveness of the proposed methods are illustrated via applications to the GUSTO-I clinical trial.

来源:Stat Med,查看原文。版权归原作者所有。

引用

  1. Noma H, Shinozaki T, Iba K, et al. Confidence intervals of prediction accuracy measures for multivariable prediction models based on the bootstrap-based optimism correction methods. Stat Med. 2021. PMID: 34302372. doi:10.1002/sim.9148

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。