文献速递技术与安全

利用微调大语言模型识别含意外发现的高优先级放射学报告

Identification of high-priority radiology reports with unexpected findings using fine-tuned large language models.
PubMed 文献 · Eur Radiol · 2025-12-24

微调后的大语言模型能准确预测HP放射学报告,提示其可能有助于支持高紧急或严重放射学报告的沟通。

要点

  • Llama3 Elyza 8B以「所见+转诊科室」输入表现最佳,PRAUC=0.962、ROCAUC=0.968。
  • 最佳模型准确率=0.915、敏感度/召回率=0.932、特异度=0.898、F1=0.916。
  • 增加检查前临床诊断和检查申请细节不一定提升模型性能。

核心结论

  • 微调后的大语言模型能准确预测HP放射学报告,提示其可能有助于支持高紧急或严重放射学报告的沟通。

诊断标准与关键数值

  • HP放射学报告定义为含高紧急或严重意外发现的报告;最佳模型Llama3 Elyza 8B(输入为所见+转诊科室)PRAUC=0.962、ROCAUC=0.968、准确率=0.915、敏感度/召回率=0.932、特异度=0.898、F1=0.916。

临床应用与阅片要点

  • 原文结论指出微调大语言模型能准确预测HP放射学报告,提示其可能有助于支持高紧急或严重放射学报告的沟通。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVE: This study aims to evaluate whether large language models (LLMs) can accurately predict the urgency and severity of radiology reports.

MATERIALS AND METHODS: Based on the recommendations of the Academy of Royal Colleges, we defined radiology reports that include unexpected findings of high urgency or severity as "high-priority (HP) radiology reports." Overall, 1906 radiology reports were used as the training set, and 176 radiology reports were used as the test set, with a balanced ratio of HP to non-HP radiology reports (1:1) in both sets. Four types of LLMs (Llama2 7B, Llama3 8B, Llama3 Elyza 8B, and Llama 3.1 8B) were fine-tuned using four different input settings: (1) findings only, (2) findings + referring department, (3) findings + referring department + clinical diagnosis before examination, and (4) findings + referring department + clinical diagnosis before examination + details of examination request. The fine-tuned LLMs predicted whether each radiology report was HP or not.

RESULTS: Among the four LLMs, Llama3 Elyza 8B, with inputs comprising findings and the referring department, demonstrated the best performance, achieving PRAUC = 0.962, ROCAUC = 0.968, accuracy = 0.915, sensitivity/recall = 0.932, specificity = 0.898, and F1 = 0.916. Adding a clinical diagnosis before the examination and details of examination requests did not necessarily lead to performance improvement.

CONCLUSION: The fine-tuned LLMs accurately predicted HP radiology reports, suggesting their potential utility in supporting communication regarding radiology reports with high urgency or severity.

KEY POINTS: Question This study aims to evaluate whether large language models (LLMs) can accurately predict the high-priority (HP) radiology reports. Findings The fine-tuned best LLM accurately HP radiology reports, achieving PRAUC of 0.962 and ROCAUC of 0.968. Clinical relevance This study demonstrates that fine-tuned LLMs can accurately identify HP radiology reports, potentially improving timely clinical decision-making and enhancing patient safety through faster communication of critical findings.

来源:Eur Radiol,查看原文。版权归原作者所有。

引用

  1. Umeno A, Nishio M, Matsuo H, et al. Identification of high-priority radiology reports with unexpected findings using fine-tuned large language models. Eur Radiol. 2025. PMID: 41441998. doi:10.1007/s00330-025-12252-2

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。