ChatGPT-4 Turbo与Llama 3的零样本PI-RADS 2.1评分:诊断性能及与腹部放射科医师的一致性
本回顾性单中心研究用标准化提示词让ChatGPT-4 Turbo和Llama 3对500份去除原评分的前列腺MRI结构化报告行零样本PI-RADS v2.1评分,并与两位腹部放射科医师比较,以前列腺活检为参考标准。两种模型与医师间一致性高(κ=0.89-0.91),ChatGPT-4 Turbo评分显著高于医师(P<.005),ChatGPT-4 Turbo与原始报告预测前列腺癌的AUC均为0.79,放射科医师与Llama 3的AUC各为0.78。
要点
- 两种大语言模型与放射科医师的PI-RADS评分一致性高(κ=0.89-0.91)。
- ChatGPT-4 Turbo评分显著高于放射科医师(P<.005),Llama 3与医师差异无统计学意义(P=.08)。
- ChatGPT-4 Turbo和原始MRI报告预测前列腺癌的AUC均为0.79,放射科医师和Llama 3的AUC各为0.78。
核心结论
- 大语言模型可能通过准确且一致的PI-RADS评分改善前列腺MRI报告。
研究设计与样本
- 回顾性单中心研究,纳入2022年1月至12月间500份去除原始PI-RADS评分的前列腺MRI结构化报告,使用标准化提示词由ChatGPT-4 Turbo和Llama 3行零样本PI-RADS v2.1评分,两位腹部放射科医师独立评分、第三位医师裁决分歧,以前列腺活检结果为参考标准。
诊断标准与关键数值
- ChatGPT-4 Turbo与Llama 3间一致性97.7%(κ=0.95);大语言模型与放射科医师间一致性94.7%-95.7%(κ=0.89-0.91);放射科医师间一致性94.4%(κ=0.88);ChatGPT-4 Turbo和原始MRI报告预测前列腺癌的AUC均为0.79,放射科医师和Llama 3的AUC各为0.78。
临床应用与阅片要点
- 作者认为大语言模型可能通过准确且一致的PI-RADS评分改善前列腺MRI报告;ChatGPT-4 Turbo评分显著高于放射科医师(P<.005)。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
This retrospective, single-center study aimed to assess the diagnostic performance and agreement of two large language models (LLMs), ChatGPT-4 Turbo (OpenAI) and Llama 3 (Meta AI), in assigning Prostate Imaging Reporting and Data System (PI-RADS) scores to prostate MRI reports and to compare their performance with two abdominal radiologists. Structured prostate MRI reports (n = 500) obtained between January and December 2022, with original PI-RADS scores removed, were processed with LLMs using a standardized prompt to extract PI-RADS version 2.1 scores. Two abdominal radiologists independently assigned scores, with a third adjudicating discrepancies. Prostate biopsy results served as the reference standard for diagnostic performance assessment. There was high agreement between both models and radiologists: ChatGPT-4 Turbo and Llama 3 achieved 97.7% agreement (κ = 0.95), agreement between the LLM and radiologists ranged from 94.7% to 95.7% (κ = 0.89-0.91), and interradiologist agreement was 94.4% (κ = 0.88). ChatGPT-4 Turbo assigned significantly higher scores than radiologists (P < .005), while differences with Llama 3 were not statistically significant (P = .08). ChatGPT-4 Turbo and the original MRI reports achieved an area under the receiver operating characteristic curve (AUC) of 0.79 for predicting prostate cancer, and radiologists and Llama 3 achieved AUCs of 0.78 each. These results suggest that LLMs could improve prostate MRI reporting through accurate and consistent PI-RADS scoring. Keywords: Prostate, Oncology, Large Language Model Supplemental material is available for this article. © RSNA, 2025.
引用
- Firoozeh N, Mastrodicasa D, Behr S, et al. Zero-Shot PI-RADS Version 2.1 Scoring with ChatGPT-4 Turbo and Llama 3: Diagnostic Performance and Agreement with Abdominal Radiologists. Radiol Imaging Cancer. 2026. PMID: 41347921. doi:10.1148/rycan.250119
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。