基于大语言模型的心脏事件提取框架在电子健康记录中的跨机构验证
本研究开发并验证了基于大语言模型(LLM)的自动化心脏事件识别框架TRACER,使用两个机构共411例肺癌和乳腺癌患者数据。TRACER结合结构化术语匹配与LLM对非结构化笔记的分析,在开发集、内部验证集和外部验证集上平均准确率分别为79.4%、81.0%和79.3%,处理时间每例20-42秒,远低于人工审查的每例2小时。
要点
- TRACER框架在跨机构、跨病种验证中平均准确率约79%-81%。
- DeepSeek-R1在内部验证集准确率最高达83.4%-85.2%。
- Llama-3.3在外部验证集准确率达85.5%。
核心结论
- TRACER作为一种本地部署的LLM框架,能跨机构和疾病部位准确提取心脏事件,显著减少心脏事件识别所需资源,支持可扩展的心脏肿瘤学研究。
研究设计与样本
- 研究纳入两个机构共411例肺癌和乳腺癌患者,机构1数据(n=266)分为开发队列(肺癌,n=178)和内部验证队列(乳腺癌,n=88),外部验证使用机构2数据(肺癌,n=145)。
- 心脏事件由医师根据全部电子健康记录病史裁定,提取数据包括结构化问题列表和非结构化临床笔记。
- TRACER框架结合结构化术语匹配与基于LLM的非结构化笔记分析,包括专科过滤、时间感知查询和少样本示例;初始评估六种候选模型后,验证了三种表现最佳的开源LLM(DeepSeek-R1、Llama-3.3、Mistral-Large)。
主要发现
- 411例患者中220例至少有一次心脏事件。
- 三种最佳模型在开发、内部验证和外部验证队列的平均准确率分别为79.4%、81.0%和79.3%。
- DeepSeek-R1在内部队列准确率最高(83.4%-85.2%),Llama-3.3在外部验证准确率达85.5%。
- TRACER处理时间每例20-42秒(总计2.3-4.8小时),人工审查每例2小时(总计822人时)。
关键数值与推荐
- 性能评估以医师裁定结果为金标准,采用准确率和处理时间作为指标。
临床怎么用
- TRACER可本地部署,用于自动化提取电子健康记录中的心脏事件,减少人工审查负担,助力心脏肿瘤学研究。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
PURPOSE: Cardiac toxicity is a significant concern for cancer patients undergoing radiotherapy. The complexity of evaluating cardiac events, the time required to parse the Electronic Health Record (EHR), and the need for large datasets make cardiotoxicity studies time- and resource-intensive, limiting progress of cardio-oncology research. Large-language models (LLMs) have the potential to automatically identify cardiac events. This work aimed to develop and validate a novel LLM-based automated cardiac event identification framework using a two-institution dataset. METHODS: 411 lung and breast cancer patients from two institutions were analyzed. [Institution 1] data (n=266) were divided into a development cohort (lung, n=178) and an internal validation cohort (breast, n=88). External validation used [Institution 2]data (lung, n=145). Cardiac events were physician-adjudicated from the entire EHR patient history. Extracted EHR data comprised structured problem lists and unstructured clinical notes. We introduced the Two-phase Reasoning for Automated Cardiac Event Recognition (TRACER) framework, which combines structured term matching with LLM-based analysis of unstructured notes (including specialty-filtering, temporally aware queries, and few-shot examples). Following initial evaluation of six candidate models, the three top-performing open-source LLMs (DeepSeek-R1, Llama-3.3, Mistral-Large) were validated. Performance was evaluated against physician-adjudicated ground truth using accuracy and processing time. RESULTS: Of 411 patients, 220 had at least one cardiac event. The top three models achieved mean accuracies of 79.4%, 81.0%, and 79.3% for the development, internal validation, and external validation cohorts, respectively. DeepSeek-R1 achieved the highest accuracy on internal cohorts (83.4-85.2%), while Llama-3.3 reached 85.5% accuracy on external validation. TRACER processing time was 20-42 seconds per patient (2.3-4.8 hours total) versus 2 hours per patient for manual review (822 person-hours total). CONCLUSION: TRACER, a locally deployed LLM framework, accurately extracted cardiac events across institutions and disease sites. This approach enables scalable cardio-oncology research by substantially reducing the resources required for cardiac event identification.
引用
- Cao W, Lloyd I, Dichmann M, et al. Cross-Institutional Validation of a novel LLM-Based Cardiac Event Extraction framework from Electronic Health Records. Int J Radiat Oncol Biol Phys. 2026. PMID: 42373015. doi:10.1016/j.ijrobp.2026.06.3060
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。