文献速递心胸血管麻醉

大型语言模型集成自动提取术中超声心动图报告结构化数据

Automated structured data extraction from intraoperative echocardiography reports using large language models.
PubMed 文献 · Br J Anaesth · 2025-03-03

本研究采用横断面设计,从宾夕法尼亚大学医疗系统两家医院的7106例成人心脏手术患者中随机抽取600份术中经食管超声报告,比较五种开源大语言模型四种投票策略集成的数据提取性能。一致投票集成共识准确率最高(术前99.4%,术后97.9%)但提取率最低(术前81.7%,术后80.5%);相对多数投票集成原始准确率最高(术前96.1%,术后93.7%)且提取率最高(术前99.4%,术后98.9%)。基于共识的大语言模型集成能成功从术中经食管超声报告的非结构化文本中生成结构化数据。

要点

  • 一致投票集成共识准确率最高但数据提取率最低。
  • 相对多数投票集成提取率最高但错误率也最高。
  • 大语言模型集成可从术中经食管超声报告提取结构化数据。

核心结论

  • 基于共识的大语言模型集成能成功从术中经食管超声报告的非结构化文本中生成结构化数据。

研究设计与样本

  • 横断面研究,从宾夕法尼亚大学医疗系统两家医院的7106例成人心脏手术患者中随机抽取600份术中经食管超声报告,其中100份用于提示工程,500份用于测试。
  • 从报告的术前和术后部分提取左心室射血分数、右心室收缩功能和三尖瓣反流三个超声心动图参数。
  • 使用五种开源大语言模型和四种投票策略(一致、超级多数、多数、相对多数)生成大语言模型集成。

主要发现

  • 一致集成共识准确率最高(术前99.4%,术后97.9%),错误率最低(术前0.6%,术后2.1%),但数据提取率最低(术前81.7%,术后80.5%)且原始准确率最低(术前81.2%,术后78.9%)。
  • 相对多数集成原始准确率最高(术前96.1%,术后93.7%),数据提取率最高(术前99.4%,术后98.9%),但共识准确率最低(术前96.7%,术后94.7%)且错误率最高(术前3.3%,术后5.3%)。

临床怎么用

  • 临床可根据对准确性与提取率的需求选择不同投票策略的大语言模型集成,如优先准确性可选一致集成,优先提取率可选相对多数集成。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: Consensus-based large language model (LLM) ensembles might provide an automated solution for extracting structured data from unstructured text in echocardiography reports. METHODS: This cross-sectional study utilised 600 intraoperative transoesophageal reports (100 for prompt engineering; 500 for testing) randomly sampled from 7106 adult patients undergoing cardiac surgery at two hospitals within the University of Pennsylvania Healthcare System. Three echocardiographic parameters (left ventricular ejection fraction, right ventricular systolic function, and tricuspid regurgitation) were extracted from both the presurgical and postsurgical sections of the reports. LLM ensembles were generated using five open-source LLMs and four voting strategies: (1) unanimous (five out of five in agreement); (2) supermajority (four or more of five in agreement); (3) majority (three or more of five in agreement); and (4) plurality (two or more of five in agreement). Returned LLM ensemble responses were compared with the reference standard dataset to calculate raw accuracy, consensus accuracy, error rate, and yield. RESULTS: Of the four LLM ensembles, the unanimous LLM ensemble achieved the highest consensus accuracies (99.4% presurgical; 97.9% postsurgical) and the lowest error rates (0.6% presurgical; 2.1% postsurgical) but had the lowest data extraction yields (81.7% presurgical; 80.5% postsurgical) and the lowest raw accuracies (81.2% presurgical; 78.9% postsurgical). In contrast, the plurality LLM ensemble achieved the highest raw accuracies (96.1% presurgical; 93.7% postsurgical) and the highest data extraction yields (99.4% presurgical; 98.9% postsurgical) but had the lowest consensus accuracies (96.7% presurgical; 94.7% postsurgical) and highest error rates (3.3% presurgical; 5.3% postsurgical). CONCLUSIONS: A consensus-based LLM ensemble successfully generated structured data from unstructured text contained in intraoperative transoesophageal reports.

来源:Br J Anaesth,查看原文。版权归原作者所有。

引用

  1. MacKay EJ, Goldfinger S, Chan TJ, et al. Automated structured data extraction from intraoperative echocardiography reports using large language models. Br J Anaesth. 2025. PMID: 40037947. doi:10.1016/j.bja.2025.01.028

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。