文献速递流行病学与生物统计医学人工智能与信息学

系统综述:机器学习对临床预测模型的性能未优于逻辑回归

A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models.
PubMed 文献 · J Clin Epidemiol · 2019-02-11

该系统综述检索了2016年1月至2017年8月的Medline文献,纳入71项比较逻辑回归与机器学习用于二分类结局预测的研究。在低偏倚风险的145项比较中,两种模型的logit(AUC)差异为0.00(95%CI -0.18至0.18);高偏倚风险的137项比较中机器学习logit(AUC)高0.34(0.20-0.47)。结论认为无证据表明机器学习优于逻辑回归。

要点

  • 低偏倚风险比较中,机器学习与逻辑回归的预测区分度无显著差异。
  • 高偏倚风险研究更易报告机器学习优势,提示方法学偏倚影响结论。
  • 多数研究未报告校准度,且验证程序存在潜在偏倚。

核心结论

  • 未发现机器学习优于逻辑回归的证据,比较建模算法的研究需改进方法与报告。

研究设计与样本

  • 系统综述,检索Medline(2016年1月至2017年8月),纳入71项比较逻辑回归与机器学习用于二分类结局预测的研究。
  • 中位样本量1,250(范围72-3,994,872),中位预测变量数19(范围5-563),中位每个预测变量事件数8(范围0.3-6,697)。

主要发现

  • 共识别282项逻辑回归与机器学习的比较(AUC范围0.52-0.99)。
  • 低偏倚风险的145项比较中,logit(AUC)差异为0.00(95%CI -0.18至0.18)。
  • 高偏倚风险的137项比较中,机器学习的logit(AUC)高0.34(95%CI 0.20-0.47)。

临床怎么用

  • 临床预测建模时,逻辑回归仍可作为可靠的基准模型,机器学习未必带来性能提升。

局限性

  • 纳入研究普遍存在验证偏倚和校准度报告不足,可能影响比较结果的可靠性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: The objective of this study was to compare performance of logistic regression (LR) with machine learning (ML) for clinical prediction modeling in the literature. STUDY DESIGN AND SETTING: We conducted a Medline literature search (1/2016 to 8/2017) and extracted comparisons between LR and ML models for binary outcomes. RESULTS: We included 71 of 927 studies. The median sample size was 1,250 (range 72-3,994,872), with 19 predictors considered (range 5-563) and eight events per predictor (range 0.3-6,697). The most common ML methods were classification trees, random forests, artificial neural networks, and support vector machines. In 48 (68%) studies, we observed potential bias in the validation procedures. Sixty-four (90%) studies used the area under the receiver operating characteristic curve (AUC) to assess discrimination. Calibration was not addressed in 56 (79%) studies. We identified 282 comparisons between an LR and ML model (AUC range, 0.52-0.99). For 145 comparisons at low risk of bias, the difference in logit(AUC) between LR and ML was 0.00 (95% confidence interval, -0.18 to 0.18). For 137 comparisons at high risk of bias, logit(AUC) was 0.34 (0.20-0.47) higher for ML. CONCLUSION: We found no evidence of superior performance of ML over LR. Improvements in methodology and reporting are needed for studies that compare modeling algorithms.

来源:J Clin Epidemiol,查看原文。版权归原作者所有。

引用

  1. Christodoulou E, Ma J, Collins GS, et al. A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol. 2019. PMID: 30763612. doi:10.1016/j.jclinepi.2019.02.004

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。