机器学习与逻辑回归临床预测模型比较评估中的系统性偏倚:以创伤死亡预测为实证案例的元研究分析
该系统综述与随机效应荟萃分析以创伤死亡预测为实证案例,比较机器学习(ML)与逻辑回归(LR)在同一数据集上的预测性能。17项研究(243,324例患者)汇总显示,支持ML的AUC差异为0.026(95%CI 0.009-0.043),但异质性极高(I2=97.9%),95%预测区间跨零(-0.034至0.086)。作者指出当前证据不能可靠支持ML优于LR的论断,并提出六项最低方法学标准。
要点
- ML相对LR的汇总AUC优势仅0.026,异质性极高且预测区间跨零。
- 模型选择不对称、依赖内部验证、选择性报告和仅汇总AUC可能共同夸大ML优势。
- 提出预设、公平对照、外部/时间验证、校准与决策分析报告等六项最低标准。
核心结论
- 当前证据不能可靠支持ML优于LR的论断,观察到的差异可能因现行评估惯例而被优先高估。
研究设计与样本
- 系统综述与随机效应荟萃分析,结合以创伤死亡预测为实证案例的元研究评估。
- 检索Medline、Scopus、Web of Science和Embase至2025年1月15日,纳入直接在同一数据集上比较ML与LR的研究(PROSPERO CRD42025636303)。
- 20项研究符合综述纳入标准,其中17项(243,324例患者)进入主要定量合成。
- 采用PROBAST评估偏倚风险,并预设以研究内AUC差异为估计目标。
主要发现
- 支持ML的汇总AUC差异为0.026(95%CI 0.009-0.043),共同主要分析估计为0.017(95%CI 0.005-0.029)。
- 异质性极高(I2=97.9%),95%预测区间跨零(-0.034至0.086)。
- 最佳锦标赛集成方法的优势(0.034)大于单一ML算法(0.005),但交互作用无统计学显著性。
- 17项评估研究中4项达到PROBAST低偏倚风险,汇总估计在偏倚风险分层间无差异,3项研究使用外部或时间验证。
关键数值与推荐
- 提出六项最低方法学标准:预设、公平对照设计、稳健的外部或时间验证、校准与决策分析报告、完全透明、偏倚感知的合成。
临床怎么用
- 读者应谨慎看待ML优越性声明,除非比较公平、模型在不同环境中测试且报告了排序准确性以外的指标。
局限性
- 异质性极高(I2=97.9%),95%预测区间跨零。
- 仅3项研究使用外部或时间验证,多数依赖内部验证。
- 估计目标为最佳ML模型与单一LR比较器的研究内AUC差异,本身是偏倚来源。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: Comparative evaluations of machine learning (ML) and logistic regression (LR) for clinical prediction frequently report ML as superior, but the methodological framework producing those comparisons has received limited scrutiny. We aimed to quantify the apparent discrimination advantage of ML over LR using trauma mortality prediction as an empirical case and to characterize the evaluation practices that shape it. STUDY DESIGN AND SETTING: Systematic review and random-effects meta-analysis combined with a meta-research appraisal of evaluation practices using trauma mortality prediction as an empirical case. Medline (Ovid), Scopus, Web of Science, and Embase were searched through January 15, 2025, for studies directly comparing any ML algorithm with LR on the same dataset (PROSPERO CRD42025636303); a sensitivity search using controlled vocabulary and a broader concept structure was performed during revision. The estimand was prespecified as within-study area under the receiver operating characteristic curve (AUC) differences between the best-performing ML model and a single LR comparator, itself a source of bias. Risk of bias was assessed with Prediction model Risk Of Bias ASsessment Tool (PROBAST). A co-primary analysis was restricted to studies reporting confidence intervals (CIs) for both models. RESULTS: Twenty studies met review-level eligibility, of which 17 (243,324 patients) contributed to the primary quantitative synthesis. The pooled AUC difference favoring ML was 0.026 (95% CI 0.009-0.043); the co-primary estimate was 0.017 (95% CI 0.005-0.029). Heterogeneity was extreme (I2 = 97.9%), and the 95% prediction interval crossed zero (-0.034 to 0.086). The advantage was larger for best-of-tournament ensemble methods (0.034) than for single ML algorithms (0.005), but the interaction was not statistically significant. Four of 17 assessed studies achieved low PROBAST risk of bias, pooled estimates did not differ across risk of bias strata, and three studies used external or temporal validation. Four convergent evaluation practices-model selection asymmetry, reliance on internal validation, selective reporting, and AUC-only synthesis-may jointly inflate apparent ML superiority and are not addressed by current evidence synthesis frameworks. CONCLUSION: This study contributes to the meta-science of prediction model evaluation, and it characterizes the methodological architecture that produces apparent ML superiority and translates that diagnosis into minimum standards. Current evidence does not reliably support claims that ML outperforms LR; observed differences may be preferentially overestimated under prevailing evaluation conventions. We propose six minimum methodological standards-prespecification, fair comparator design, robust external or temporal validation, calibration and decision-analytic reporting, full transparency, and bias-aware synthesis-which may also be relevant to other clinical prediction settings. PLAIN LANGUAGE SUMMARY: Computer programs that learn patterns from data (ML) are often reported to predict death after injury better than traditional statistical models. We reviewed published studies that compared both approaches in the same patients. Across 17 studies, the difference in ranking accuracy was small, varied widely between studies, and, in a new setting, could favor either approach. Part of the apparent advantage may come from how the comparisons were designed and reported rather than from the methods themselves; the best of several machine learning models is usually compared against a single, less carefully built traditional model, almost always in the same data used to develop it. Readers should be cautious about claims of superiority unless the comparison was fair, the models were tested in a different setting, and measures beyond ranking accuracy were reported.
引用
- Wang SH, Shao SC, Yang MR, et al. Systematic bias in comparative evaluations of machine learning vs logistic regression for clinical prediction models: a meta-research analysis using trauma mortality as an empirical case. J Clin Epidemiol. 2026. PMID: 42727617. doi:10.1016/j.jclinepi.2026.112508
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。