系统综述:机器学习对临床预测模型的性能未优于逻辑回归
该系统综述检索了2016年1月至2017年8月的Medline文献,纳入71项比较逻辑回归与机器学习用于二分类结局预测的研究。在低偏倚风险的145项比较中,两种模型的logit(AUC)差异为0.00(95%CI -0.18至0.18);高偏倚风险的137项比较中机器学习logit(AUC)高0.34(0.20-0.47)。结论认为无证据表明机器学习优于逻辑回归。
要点
- 低偏倚风险比较中,机器学习与逻辑回归的预测区分度无显著差异。
- 高偏倚风险研究更易报告机器学习优势,提示方法学偏倚影响结论。
- 多数研究未报告校准度,且验证程序存在潜在偏倚。
核心结论
- 未发现机器学习优于逻辑回归的证据,比较建模算法的研究需改进方法与报告。
研究设计与样本
- 系统综述,检索Medline(2016年1月至2017年8月),纳入71项比较逻辑回归与机器学习用于二分类结局预测的研究。
- 中位样本量1,250(范围72-3,994,872),中位预测变量数19(范围5-563),中位每个预测变量事件数8(范围0.3-6,697)。
主要发现
- 共识别282项逻辑回归与机器学习的比较(AUC范围0.52-0.99)。
- 低偏倚风险的145项比较中,logit(AUC)差异为0.00(95%CI -0.18至0.18)。
- 高偏倚风险的137项比较中,机器学习的logit(AUC)高0.34(95%CI 0.20-0.47)。
临床怎么用
- 临床预测建模时,逻辑回归仍可作为可靠的基准模型,机器学习未必带来性能提升。
局限性
- 纳入研究普遍存在验证偏倚和校准度报告不足,可能影响比较结果的可靠性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: The objective of this study was to compare performance of logistic regression (LR) with machine learning (ML) for clinical prediction modeling in the literature. STUDY DESIGN AND SETTING: We conducted a Medline literature search (1/2016 to 8/2017) and extracted comparisons between LR and ML models for binary outcomes. RESULTS: We included 71 of 927 studies. The median sample size was 1,250 (range 72-3,994,872), with 19 predictors considered (range 5-563) and eight events per predictor (range 0.3-6,697). The most common ML methods were classification trees, random forests, artificial neural networks, and support vector machines. In 48 (68%) studies, we observed potential bias in the validation procedures. Sixty-four (90%) studies used the area under the receiver operating characteristic curve (AUC) to assess discrimination. Calibration was not addressed in 56 (79%) studies. We identified 282 comparisons between an LR and ML model (AUC range, 0.52-0.99). For 145 comparisons at low risk of bias, the difference in logit(AUC) between LR and ML was 0.00 (95% confidence interval, -0.18 to 0.18). For 137 comparisons at high risk of bias, logit(AUC) was 0.34 (0.20-0.47) higher for ML. CONCLUSION: We found no evidence of superior performance of ML over LR. Improvements in methodology and reporting are needed for studies that compare modeling algorithms.
引用
- Christodoulou E, Ma J, Collins GS, et al. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. 2019. PMID: 30763612. doi:10.1016/j.jclinepi.2019.02.004
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。