风险预测模型开发中样本信息的期望价值计算
风险预测模型的预测因有限样本训练而不确定,传统样本量计算关注统计精度和预测稳定性,但决策理论视角下有限样本导致效用损失。本文定义样本信息期望价值(EVSI)为增加额外开发样本带来的临床效用(净获益)预期增益,提出基于bootstrap的算法,并在案例研究中展示其可行性与表面效度,认为决策理论指标可补充经典推断方法。
要点
- EVSI可量化增加开发样本对模型临床效用(净获益)的预期增益。
- 提出基于bootstrap的EVSI计算算法,案例研究显示可行且表面效度良好。
- 决策理论指标可补充经典推断方法用于风险预测模型开发研究设计。
核心结论
- 决策理论指标(如EVSI)可补充经典推断方法,用于设计风险预测模型开发研究。
研究设计与样本
- 方法学论文,提出基于bootstrap的EVSI计算算法,并在案例研究中验证可行性与表面效度。
主要发现
- 有限样本导致拟合模型与正确模型间的差异,从而产生效用损失。
- 获取更多开发数据与模型使用效用的预期增益相关。
- 案例研究显示bootstrap算法计算EVSI可行且具有表面效度。
关键数值与推荐
- EVSI定义为在净获益框架下,获取给定大小额外开发样本带来的临床效用预期增益。
临床怎么用
- 设计风险预测模型开发研究时,可考虑用EVSI等决策理论指标补充传统样本量计算。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Risk prediction models are often advertised as deterministic functions that map covariates to predicted risks. However, they are typically trained using finite samples, and as such, their predictions are inherently uncertain. This uncertainty has been addressed in terms of uncertainty around metrics of model performance (e.g., confidence interval around c-statistic), as well as uncertainty or instability of predictions. Correspondingly, sample size calculations for model development studies target the precision of estimates of summary statistics and the stability of predictions. However, when evaluating the clinical utility of a model (as in net benefit [NB] calculations in decision curve analysis), statistical inference is less relevant. From a decision-theoretic perspective, the finite size of the sample results in utility loss due to the discrepancy between the fitted model and the correct model. From this perspective, procuring more development data is associated with an expected gain in the utility of using the model. In this work, we define the Expected Value of Sample Information (EVSI) as the expected gain in clinical utility, defined in NB terms, by procuring an additional development sample of a given size. We propose a bootstrap-based algorithm for EVSI computations and demonstrate its feasibility and face validity in a case study. We conclude that decision-theoretic metrics can complement classical inferential methods when designing studies aimed at developing risk prediction models.
引用
- Safari A, Gustafson P, Sadatsafavi M, et al. Expected Value of Sample Information Calculations for Risk Prediction Model Development. Stat Med. 2026. PMID: 41974569. doi:10.1002/sim.70484
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。