文献速递腹部技术与安全CTMR

大语言模型在土耳其语住院医师腹部CT/MRI报告差异检测中的诊断性能

Diagnostic performance of large language models for discrepancy detection in non-English resident-authored abdominal CT and MRI reports: a Turkish-language evaluation.
PubMed 文献 · Abdom Radiol (NY) · 2026-09-26

本回顾性单中心研究以488份土耳其语腹部影像报告(317份CT、171份MRI)为对象,由两名腹部放射科医师建立专家共识参考标准,评估三种大语言模型在无图像访问条件下检测住院医师初稿与最终报告间临床意义差异的性能。Claude 4.6 Sonnet总体敏感度最高(93.4%),对重大差异敏感度97.8%且假阳性率最低(0.6%);Gemini 3.0 Pro教育反馈评分最高(4.60±0.65)。作者指出单中心文本比较结果尚不能确立基于图像的诊断准确性或自主临床使用条件。

要点

  • LLM仅基于文本比较报告对,未访问图像,不能替代影像本身判读。
  • Claude 4.6 Sonnet对重大差异敏感度97.8%、假阳性率0.6%。
  • 不同模型差异检测敏感度显著不同(p<0.001),性能具模型依赖性。
英文原文(摘要)· 点击展开

PURPOSE: To evaluate the diagnostic performance of large language models (LLMs) for detecting clinically meaningful discrepancies in non-English, resident-authored abdominal CT and MRI reports, using Turkish-language reports as a test setting, and to compare the educational quality of model-generated feedback.

METHODS: This retrospective study included 488 abdominal imaging reports, comprising 317 CT and 171 MRI examinations, each with a resident-authored preliminary report and an attending-approved final report. Two abdominal radiologists established an expert consensus reference standard using both report pairs and corresponding imaging studies. Three LLMs-GPT-5.3, Claude 4.6 Sonnet, and Gemini 3.0 Pro-were prompted to detect and classify discrepancies and generate brief educational feedback. Importantly, models evaluated text-based report pairs without direct image access. Primary outcomes were event-level discrepancy detection sensitivity and false-positive behavior; educational feedback was assessed using a 5-point Likert scale.

RESULTS: The reference standard identified 776 clinically meaningful discrepancies in 390 reports; 98 reports were error-free. Event-level sensitivity differed significantly among models (p < 0.001). Claude 4.6 Sonnet achieved the highest overall sensitivity (93.4%), followed by GPT-5.3 (91.9%) and Gemini 3.0 Pro (72.6%). Claude also showed the highest sensitivity for major discrepancies (97.8%) and the lowest false-positive rate (0.6%). Gemini 3.0 Pro achieved the highest educational feedback score (4.60 ± 0.65), followed by GPT-5.3 (4.41 ± 0.68) and Claude 4.6 Sonnet (3.95 ± 0.80; p < 0.001).

CONCLUSION: Under a single structured prompt, LLMs showed model-dependent performance in the text-based comparison of Turkish preliminary and final abdominal radiology reports. These single-center findings do not establish image-based diagnostic accuracy or readiness for autonomous clinical use and require prospective multicenter validation.

来源:Abdom Radiol (NY),查看原文。版权归原作者所有。

引用

  1. Ozel MA, Naldemir IF, Unal Ozen A, et al. Diagnostic performance of large language models for discrepancy detection in non-English resident-authored abdominal CT and MRI reports: a Turkish-language evaluation. Abdom Radiol (NY). 2026. PMID: 42799750. doi:10.1007/s00261-026-05807-4

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。