文献速递外周动脉疾病

精准性的错觉:人工智能尚不能可靠预测膝下慢性肢体威胁性缺血的个体结局

The Illusion of Precision: Artificial Intelligence Cannot yet Reliably Predict Individual Outcomes in Infrapopliteal Chronic Limb-Threatening Ischemia.
PubMed 文献 · Ann Vasc Surg · 2026-07-14

该单中心回顾性研究纳入49例因慢性肢体威胁性缺血(CLTI)接受血运重建的患者,使用两种大语言模型(GPT-5.5和DeepSeek-V4)以零样本提示策略预测6个月、1年和5年主要不良事件。结果显示模型校准度与区分度严重分离:DS朴素提示6个月Brier评分达0.068,但所有模型在6个月和1年时敏感度均为0.00,呈现「精算对冲」策略,即预测值聚集于人群均值而非识别高危个体。

要点

  • LLMs在CLTI预后预测中校准度尚可但区分度完全失败,6个月和1年敏感度均为0.00。
  • 模型通过聚集预测于人群均值来最小化数学误差,而非识别特定高危患者。
  • 当前LLMs应限于行政辅助角色,不应直接用于患者决策中的预后预测。

核心结论

  • 大语言模型虽具备医学语义能力,但尚不具备高风险预后判断所需的实用效能。
  • 观察到的「幻觉校准」和欺骗性稳定性制造了危险的可靠性错觉。
  • 在区分度失败问题解决之前,LLMs应限于行政角色而非预测性角色。

研究设计与样本

  • 回顾性单中心研究,纳入49例因CLTI接受血运重建的患者。
  • 测试GPT-5.5和DeepSeek-V4两种LLM架构。
  • 采用「朴素」(非结构化)和「结构化」(表格)零样本提示策略。
  • 预测6个月、1年和5年主要不良事件(MAEs)。
  • 采用Brier评分评估校准度、敏感度评估区分度、Wasserstein距离评估稳定性。

主要发现

  • DS朴素提示6个月Brier评分为0.068,提示较高的概率准确性。
  • 所有模型在6个月和1年时间点的敏感度均为0.00。
  • 模型采取「精算对冲」策略,将预测值聚集在人群均值附近以最小化数学误差。
  • 内部可重复性(ICC>0.500)和跨提示分布稳定性较高,但基于非区分性的安全偏倚。

临床怎么用

  • 当前不应将LLMs用于CLTI患者的个体化预后预测。
  • LLMs可限于行政辅助角色,而非直接参与患者决策。
  • 临床医生需警惕模型高校准度掩盖的区分度失败风险。

局限性

  • 单中心回顾性设计,样本量49例。
  • 仅测试两种LLM架构和零样本提示策略。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: The integration of large language models (LLMs) into vascular surgery promises scalable clinical decision support, yet their prognostic reliability remains unproven. This study evaluates the accuracy, calibration, and stability of generative artificial intelligence (AI) in predicting outcomes for chronic limb-threatening ischemia (CLTI). METHODS: In a retrospective single-center study, we evaluated 49 patients undergoing revascularization for CLTI. Two LLM architectures (GPT-5.5 and DeepSeek-V4 [DS]) were tested using "naive" (unstructured) and "structured" (tabular) zero-shot prompting strategies to predict major adverse events (MAEs) at 6-month, 1-year, and 5-year horizons. Performance was assessed using Brier scores for calibration, sensitivity for discrimination, and Wasserstein distance for stability. RESULTS: A critical dissociation between calibration and discrimination was observed. DS naive achieved a superior Brier score of 0.068 at 6 months, suggesting high probabilistic accuracy. However, this mathematical precision masked a failure in clinical utility: all models exhibited a sensitivity of 0.00 at 6-month and 1-year horizons. The models adopted an "actuarial hedging" strategy, clustering predictions around the population mean to minimize mathematical error rather than identifying specific high-risk patients. While internal reproducibility (intraclass correlation coefficient [ICCs] >0.500) and distributional stability across prompts were surprisingly high, this consistency proved deceptive, based entirely on nondiscriminatory safety biases. CONCLUSION: While LLMs demonstrate semantic competence in medical discourse, they currently lack the pragmatic utility required for high-stakes prognostication. The observed "hallucinated calibration" and deceptive stability creates a dangerous illusion of reliability. Until issues of discriminatory failure are resolved, LLMs should remain restricted to administrative rather than predictive roles in patient decision-making.

来源:Ann Vasc Surg,查看原文。版权归原作者所有。

引用

  1. Gómez Gutiérrez OA, Imbrognio F, De la Torre OA, et al. The Illusion of Precision: Artificial Intelligence Cannot yet Reliably Predict Individual Outcomes in Infrapopliteal Chronic Limb-Threatening Ischemia. Ann Vasc Surg. 2026. PMID: 42448030. doi:10.1016/j.avsg.2026.07.004

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。