文献速递神经MR

开放权重大语言模型自动提取脑MRI报告结构化信息

Automatic extraction of structured information from brain MRI reports using an open-weight large language model.
PubMed 文献 · Eur Radiol · 2026-08-18

本研究分析947份荷兰语脑MRI报告,评估开放权重大语言模型LLaMA 3.1自动提取30个变量的性能。结果显示视觉评分提取准确性较高,少样本提示可显著提升数值变量提取表现。开放权重LLM可用于放射报告的事后结构化,支持痴呆诊疗相关研究。

要点

  • LLaMA 3.1可高准确率提取脑MRI报告中的视觉评分与文字描述
  • 少样本提示尤其基于结构相似性选择示例可提升数值变量提取
  • 该工具适合回顾性数据再利用,不干扰现有临床报告流程

核心结论

  • LLaMA 3.1对荷兰语脑MRI报告的结构化信息提取具有较强潜力
  • 少样本提示可提升数值变量的提取性能,位置相关变量仍存在挑战

研究设计与样本

  • 回顾性分析947份三级记忆门诊脑MRI报告,时间范围为2016至2021年
  • 由神经放射学顾问撰写,训练医学生标注30个变量,其中100份报告双标注评估一致性
  • 评估LLaMA 3.1在不同语言和不同示例选择策略下少样本提示的表现
  • 采用10次随机划分的947份报告来计算各项指标

关键影像表现/征象

  • 内侧颞叶萎缩视觉评分左侧准确性均值0.90,右侧0.96
  • 全脑皮质萎缩评分准确性0.87,Fazekas评分准确性0.94
  • 微出血提及检测准确性0.93,梗死提及检测准确性0.82
  • 病灶位置的文本相似度达到0.95

诊断标准与关键数值

  • 分类变量采用平衡准确性,计数采用准确性与平均绝对误差,自由文本采用文本相似度
  • 内侧颞叶萎缩评分左侧0.90 [0.77-1.00],右侧0.96 [0.94-0.99]
  • 全脑皮质萎缩评分0.87 [0.83-0.91],Fazekas评分0.94 [0.93-0.96]
  • 梗死提及检测准确性0.82 [0.80-0.84]
  • 数值变量微出血数量提取准确性0.80 [0.78-0.82],梗死数量0.66 [0.63-0.68]

临床应用与阅片要点

  • 开放权重LLM可用于回顾性队列研究中脑MRI报告的大规模结构化提取
  • 对视觉评分类变量可直接应用零样本提取,数值类变量建议使用少样本提示
  • 用于数据再利用和报告一致性改善,无需改变现有工作流程

局限性

  • 研究纳入的报告仅来自单一三级记忆门诊
  • 数据来源为回顾性报告,双重标注仅在100份报告中进行
  • 位置相关变量的提取性能仍存在挑战

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVES: Automatic data extraction from free-text radiology reports enables large-scale research, but few studies assessed the performance of large language models (LLMs) on Dutch neuroradiology reports.

MATERIALS AND METHODS: We analyzed 947 brain MRI reports from a tertiary memory clinic (2016-2021), authored by consultant neuroradiologists. Trained medical students annotated thirty variables; 100 reports were double annotated to assess inter-rater reliability. We evaluated the performance of the open-weight LLM LLaMA 3.1 using different languages (Dutch vs English translation) and few-shot prompting with different example selection strategies. Performance was evaluated using balanced accuracy for categorical variables, accuracy and mean absolute error for counts, and text similarity for free text. Metrics were computed across 10 random splits of the 947 reports.

RESULTS: LLaMA 3.1 demonstrated high zero-shot performance for visual rating scores (mean [95% CI]): medial temporal atrophy: 0.90 [0.77-1.00] on the left and 0.96 [0.94-0.99] on the right, Global Cortical Atrophy: 0.87 [0.83-0.91], and Fazekas: 0.94 [0.93-0.96]. Microbleed mentions were detected with 0.93 [0.92-0.95] accuracy, and infarct mentions with 0.82 [0.80-0.84]. Text similarity for lesion location reached 0.95 [0.95-0.96]. Performance was lower for numerical variables: 0.80 [0.78-0.82] for the number of microbleeds and 0.66 [0.63-0.68] for infarcts. English translation yielded comparable results. Few-shot prompting improved performance for numerical variables, achieving 0.92 [0.90-0.93] for microbleeds and 0.81 [0.77-0.85] for infarcts using structural similarity-based selection.

CONCLUSION: LLaMA 3.1 shows strong potential for extracting data from Dutch neuroradiology reports. Few-shot prompting enhances performance for numerical variables, whereas challenges remain for location-specific variables.

KEY POINTS: Question The free-text format of radiology reports limits the accessibility of structured data; can LLM LLaMA 3.1 automatically structure free-text Dutch neuroradiology reports? Findings LLaMA 3.1 accurately extracted 30 variables from Dutch neuroradiology reports, with few-shot prompting significantly improving performance compared to zero-shot prompting, particularly with similarity-based example selection. Clinical relevance By enabling accurate, automated post-hoc structuring of free-text radiology reports, open-weight LLMs facilitate large-scale data reuse, improve consistency in reporting, and support clinical research and decision-making in dementia care without disrupting existing clinical workflows.

来源:Eur Radiol,查看原文。版权归原作者所有。

引用

  1. Mouheb, Pomp, Manenti, et al. Automatic extraction of structured information from brain MRI reports using an open-weight large language model. Eur Radiol. 2026. PMID: 42613470. doi:10.1007/s00330-026-12821-z

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。