采用逐步法同步开发与验证基于机器学习的预测模型
该文讨论机器学习预测模型在临床诊断中的应用,指出数据量大而样本量相对有限时易过拟合且需外部验证,但ML技术快速迭代可能导致原模型过时。作者展示一种逐步设计,可在同一研究中同步开发与验证ML预测模型,评估模型随样本量增加的稳定性与稳健性,以及选定截断值下敏感度/特异度的稳定性,从而缩短新检测进入医疗实践的时间。
要点
- ML预测模型在患者数据量大而样本量有限时可能优于经典回归,但过拟合风险高且外部验证必要。
- 逐步设计可在同一研究中同步开发与验证模型,评估随样本量增加的稳定性及选定截断值下敏感度/特异度的稳定性。
- 将常规临床参数与ML预测结合,可进一步增强患病与非患病个体的区分。
核心结论
- 逐步设计可在单一研究中同步开发与验证基于ML的预测模型,缩短新检测引入医疗的时间。
研究设计与样本
- 方法学文章,展示基于ML技术的预测模型同步开发与验证的逐步设计。
主要发现
- 该设计可在同一研究中评估预测模型随样本量增加的稳定性与稳健性。
- 该设计可评估选定截断值下敏感度/特异度的稳定性。
- 常规临床参数与ML预测结合可进一步增强患病与非患病个体的区分。
临床怎么用
- 在ML预测模型开发中采用逐步设计,可同步评估模型稳定性并缩短验证周期。
- 将常规临床参数与ML预测联合使用,以提升诊断区分能力。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Accurate diagnosis of a disease is essential in healthcare. Prediction models, based on classical regression techniques, are widely used in clinical practice. Machine Learning (ML) techniques might be preferred in case of a large amount of data per patient and relatively limited numbers of subjects. However, this increases the risk of overfitting, and external validation is imperative. However, in the field of ML, new and more efficient techniques are developed rapidly, and if recruiting patients for a validation study is time consuming, the ML technique used to develop the first model might have been surpassed by more efficient ML techniques, rendering this original model no longer relevant. We demonstrate a stepwise design for simultaneous development and validation of prediction models based on ML techniques. The design enables - in one study - evaluation of the stability and robustness of a prediction model over increasing sample size as well as assessment of the stability of sensitivity/specificity at a chosen cut-off. This will shorten the time to introduction of a new test in health care. We finally describe how to use regular clinical parameters in conjunction with ML based predictions, to further enhance differentiation between subjects with and without a disease.
引用
- Haalboom M, Kort S, van der Palen J, et al. Using a stepwise approach to simultaneously develop and validate machine learning based prediction models. J Clin Epidemiol. 2021. PMID: 34157373. doi:10.1016/j.jclinepi.2021.06.008
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。