文献速递主动脉疾病

大语言模型从腹部影像报告中精准提取主动脉信息

Large language models accurately extract aortic information from abdominal imaging reports in a large, real-world database.
PubMed 文献 · J Vasc Surg · 2025-11-05

本研究评估大语言模型Llama 3.3 70B从腹部影像报告中提取AAA相关数据,无需任务训练。纳入2008至2024年16331份报告,模型整体准确率0.93、敏感度0.96、阳性预测值0.96、F1分数0.96,最大AAA直径3至7 cm时F1为0.97至0.99,表明提取可靠且成本低。

要点

  • Llama 3.3 70B无需任务训练即可从腹部影像报告提取AAA数据。
  • 整体准确率0.93,敏感度0.96,阳性预测值0.96,F1分数0.96。
  • 最大AAA直径3至7 cm时,模型F1分数为0.97至0.99。

核心结论

  • 大语言模型无需额外人工训练即可从腹部影像报告中可靠提取主动脉信息。
  • 大语言模型可灵活高效数据挖掘,减少AAA监测登记管理负担且成本低。

研究设计与样本

  • 回顾性研究,随机选取2008至2024年综合健康系统AAA监测登记中腹部超声及横断面影像报告。
  • 共纳入16331份专家审核报告,来自11799例患者;超声6102份(37.4%),横断面10229份(62.6%)。
  • 横断面影像包括CT(81.9%)、MRI(12.5%)和PET-CT(5.6%)。

主要发现

  • 模型提取AAA相关数据的整体准确率0.93,敏感度(召回率)0.96,阳性预测值(精确率)0.96,F1分数0.96。
  • 8478份报告(51.9%)包含离散腹主动脉直径数值。
  • 最大AAA直径在3至7 cm时,模型F1分数为0.97至0.99。

关键数值与推荐

  • 模型从每份报告提取最大腹主动脉直径;若无直径数值,则解读描述性术语判定AAA阳性、阴性或未知。
  • 以独立专家审核结果为参考标准,计算准确率、敏感度、阳性预测值和F1分数。

临床怎么用

  • 可部署于防火墙内本地服务器的大语言模型辅助AAA监测登记数据提取,减少人工审核负担。
  • 公开可用且无处理成本的大语言模型有助于提升临床研究质量与效率。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVE: Maintaining robust surveillance programs for abdominal aortic aneurysms (AAAs) is important, but these programs are expensive and labor-intensive, typically requiring manual data review by trained health care professionals. Studies have shown that natural language processing software can assist in these functions, but each task-specific algorithm requires human-directed training before use. Our objective was to evaluate the use of a large language model (LLM) to extract AAA-related data using generalized artificial intelligence, negating the need for task-based training. METHODS: This study examined ultrasound and cross-sectional (computed tomography [CT], magnetic resonance imaging) abdominal imaging reports randomly selected for human review from a prospectively maintained AAA surveillance registry within an integrated health system from 2008 to 2024. Llama 3.3 70B (Meta) was utilized on a local Ollama server behind health system firewalls without external telemetry. The model extracted the maximal abdominal aortic diameter from each radiology report, or, if no diameter was stated, interpreted descriptive terms to determine whether a report was positive/negative for AAA, or if the aneurysmal status could not be determined (unknown). These data were compared with results extracted by independent expert reviewers for standard machine learning metrics of accuracy, sensitivity (recall), positive predictive value (precision), and F1-score (harmonic mean of precision and recall). RESULTS: There were 16,331 human expert-reviewed abdominal imaging reports for 11,799 patients included in the LLM analysis. Of these studies, 6102 were ultrasounds (37.4%), and 10,229 were cross-sectional studies (62.6%). The cross-sectional imaging studies included CT (81.9%), magnetic resonance imaging (12.5%) and positron emission tomography-CT (5.6%). The overall accuracy, sensitivity (recall), positive predictive value (precision), and F1-score for the assigned task were 0.93, 0.96, 0.96, and 0.96, respectively. A discrete abdominal aortic diameter was present in 8478 imaging reports (51.9%). At maximal AAA diameters between 3 and 7 cm, model F1 score was 0.97 to 0.99. CONCLUSIONS: LLM extraction of aortic information from abdominal imaging reports is exceptionally reliable without the need for additional human-directed training. In general, LLMs allow for flexible and efficient data mining with minimal human effort. Many LLMs are publicly available and incur no processing costs, making them easily accessible and cost-effective tools to decrease the administrative burden of running complex AAA surveillance registries, with the added opportunity to improve the quality and efficiency of clinical research in the field.

来源:J Vasc Surg,查看原文。版权归原作者所有。

引用

  1. Flanagan CP, Gerstley LD, Okuhn S, et al. Large language models accurately extract aortic information from abdominal imaging reports in a large, real-world database. J Vasc Surg. 2025. PMID: 41203087. doi:10.1016/j.jvs.2025.10.044

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。