利用微调大语言模型识别含意外发现的高优先级放射学报告
微调后的大语言模型能准确预测HP放射学报告,提示其可能有助于支持高紧急或严重放射学报告的沟通。
要点
- Llama3 Elyza 8B以「所见+转诊科室」输入表现最佳,PRAUC=0.962、ROCAUC=0.968。
- 最佳模型准确率=0.915、敏感度/召回率=0.932、特异度=0.898、F1=0.916。
- 增加检查前临床诊断和检查申请细节不一定提升模型性能。
核心结论
- 微调后的大语言模型能准确预测HP放射学报告,提示其可能有助于支持高紧急或严重放射学报告的沟通。
诊断标准与关键数值
- HP放射学报告定义为含高紧急或严重意外发现的报告;最佳模型Llama3 Elyza 8B(输入为所见+转诊科室)PRAUC=0.962、ROCAUC=0.968、准确率=0.915、敏感度/召回率=0.932、特异度=0.898、F1=0.916。
临床应用与阅片要点
- 原文结论指出微调大语言模型能准确预测HP放射学报告,提示其可能有助于支持高紧急或严重放射学报告的沟通。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVE: This study aims to evaluate whether large language models (LLMs) can accurately predict the urgency and severity of radiology reports.
MATERIALS AND METHODS: Based on the recommendations of the Academy of Royal Colleges, we defined radiology reports that include unexpected findings of high urgency or severity as "high-priority (HP) radiology reports." Overall, 1906 radiology reports were used as the training set, and 176 radiology reports were used as the test set, with a balanced ratio of HP to non-HP radiology reports (1:1) in both sets. Four types of LLMs (Llama2 7B, Llama3 8B, Llama3 Elyza 8B, and Llama 3.1 8B) were fine-tuned using four different input settings: (1) findings only, (2) findings + referring department, (3) findings + referring department + clinical diagnosis before examination, and (4) findings + referring department + clinical diagnosis before examination + details of examination request. The fine-tuned LLMs predicted whether each radiology report was HP or not.
RESULTS: Among the four LLMs, Llama3 Elyza 8B, with inputs comprising findings and the referring department, demonstrated the best performance, achieving PRAUC = 0.962, ROCAUC = 0.968, accuracy = 0.915, sensitivity/recall = 0.932, specificity = 0.898, and F1 = 0.916. Adding a clinical diagnosis before the examination and details of examination requests did not necessarily lead to performance improvement.
CONCLUSION: The fine-tuned LLMs accurately predicted HP radiology reports, suggesting their potential utility in supporting communication regarding radiology reports with high urgency or severity.
KEY POINTS: Question This study aims to evaluate whether large language models (LLMs) can accurately predict the high-priority (HP) radiology reports. Findings The fine-tuned best LLM accurately HP radiology reports, achieving PRAUC of 0.962 and ROCAUC of 0.968. Clinical relevance This study demonstrates that fine-tuned LLMs can accurately identify HP radiology reports, potentially improving timely clinical decision-making and enhancing patient safety through faster communication of critical findings.
引用
- Umeno A, Nishio M, Matsuo H, et al. Identification of high-priority radiology reports with unexpected findings using fine-tuned large language models. Eur Radiol. 2025. PMID: 41441998. doi:10.1007/s00330-025-12252-2
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。