部署临床预测模型时应保留连续预测风险
该研究通过模拟不同模型区分度和事件患病率下按预测风险排序患者与仅使用风险分组的差异,并在新加坡某三级医院急诊科的真实数据上验证。结果显示,按预测概率排序患者较仅用风险分组能提升模型性能,且避免结局二分类的获益随模型区分度和事件患病率升高而增大,排序对模型校准不佳具有稳健性。作者认为连续概率用于风险组内患者优先级排序具有经济获益潜力。
要点
- 按预测概率排序患者较仅用风险分组可提升模型性能。
- 避免结局二分类的获益随模型区分度和事件患病率升高而增大。
- 在新加坡急诊数据中,资源约束最高时排序获益最大。
核心结论
- 使用连续概率在风险组内对患者进行优先级排序显示出经济获益潜力。
- 未来的预测模型应共享推导连续风险评分的方程。
- 已部署的模型应考虑将这些评分与临床判断一起用于患者优先级排序。
研究设计与样本
- 模拟研究:在不同模型区分度和事件患病率下,比较按预测风险排序患者与仅使用风险分组的影响。
- 在模型校准不同水平下,以阳性预测值、灵敏度和真阳性平均排名评估模型性能。
- 将发现应用于基于机器学习的序数评分系统,使用来自新加坡某大型三级急诊科的真实数据。
主要发现
- 使用预测概率对患者按预测风险排序,与仅使用风险分组相比,带来了模型性能获益。
- 避免结局二分类的获益随模型区分度和事件患病率增加而增大,且排序对模型校准不佳具有稳健性。
- 在新加坡急诊数据上重复分析显示,当资源约束最高时,排序的获益最大。
临床怎么用
- 作者认为,使用连续概率在风险组内对患者进行优先级排序显示出经济获益潜力。
- 作者认为,未来的预测模型应共享推导连续风险评分的方程。
- 作者认为,已部署的模型应考虑将这些评分与临床判断一起用于患者优先级排序。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVE: Clinical prediction models are used to obtain predicted risks of a diagnosis or future event. While models can produce continuous predicted probabilities, these are often dichotomized or categorized into risk groups using probability thresholds for the sake of operational convenience. Thresholds or risk groups may be required, but discarding the continuous probability is not, and risks throwing away information that can be useful for prioritizing patients. The economic value of continuous risk prediction estimates is not well understood. STUDY DESIGN AND SETTING: We simulated the impact of ranking patients by predicted risks compared to using risk groups alone when faced with resource constraints at varying levels of model discrimination and event prevalence. We evaluated model performance in terms of positive predictive value, sensitivity, and mean rank of true positives under different levels of model calibration. We then applied our findings to a machine learning-based ordinal scoring system using real data from a large tertiary Singaporean emergency department. RESULTS: Using predicted probabilities to rank patients by predicted risk led to model performance benefits when compared to risk groups alone. The benefits of avoiding outcome dichotomization increased as model discrimination and outcome prevalence increased, and ranking was robust to poor model calibration. Repeating this analysis on Singaporean emergency department data showed that benefits of ranking were greatest when resource constraints were highest. CONCLUSION: Using continuous probabilities to prioritize patients within risk groups shows the potential for economic benefits. Future prediction models should share equations for deriving continuous risk scores, and deployed models should consider using these scores together with clinical judgment for patient prioritization.
引用
- Blythe R, Parsons R, Ong ME, et al. Continuous predicted risks should be retained when deploying clinical prediction models. J Clin Epidemiol. 2025. PMID: 41062044. doi:10.1016/j.jclinepi.2025.112009
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。