文献速递技术与安全

人工智能在放射学中的应用:ChatGPT-4v与GPT-4o在放射科住院医师培训诊断考试(DXIT)题目中的表现

Artificial Intelligence in Radiology: Performance of ChatGPT-4v and GPT-4o on Diagnostic Radiology in-Training (DXIT) Examination Questions.
PubMed 文献 · J Am Coll Radiol · 2025-10-30

本研究将1136道公开的DXIT考试题输入GPT-4v与GPT-4o,评估其答题准确率与置信度。GPT-4o与GPT-4v总准确率分别为73.5%与69.3%,图像题准确率分别为55.6%与50.3%;两者均高于2022年DXIT考试放射科住院医师第3年(PGY-3)的全国平均成绩61.9%,但图像题表现显著差于纯文本题,且利用内置置信度预测答案正确性的能力有限。

要点

  • GPT-4o总准确率73.5%,GPT-4v为69.3%,均高于PGY-3住院医师全国平均61.9%。
  • 图像题准确率GPT-4o为55.6%、GPT-4v为50.3%,显著低于纯文本题。
  • 置信度预测正确性的AUC:GPT-4o为0.64,GPT-4v为0.66,能力有限。

核心结论

  • GPT-4o在几乎所有指标上优于GPT-4v,两模型总成绩均超过2022年DXIT考试PGY-3住院医师全国平均61.9%,但图像题表现显著更差且低于同队列放射科受训者,内置置信度预测正确性的能力有限,用于备考和图像解读需谨慎。

研究设计与样本

  • 将1136道公开的ACR DXIT考试题输入GPT-4v和GPT-4o,要求模型给出答案、理由和0-100置信度,采用χ2检验比较比例、t检验比较均值、ROC曲线评估置信度。

诊断标准与关键数值

  • GPT-4o总准确率73.5%,GPT-4v为69.3%(P < .0001)。
  • 图像题准确率GPT-4o为55.6%,GPT-4v为50.3%(P < .0001)。
  • 置信度与正确性的ROC曲线下面积:GPT-4o为0.64,GPT-4v为0.66。
  • 2022年DXIT考试PGY-3放射科住院医师全国平均成绩为61.9%。

临床应用与阅片要点

  • 作者认为,使用ChatGPT进行备考和图像解读必须谨慎。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVE: The purpose of this study is to examine the performance of Chat Generative Pre-trained Transformer (GPT)-4vision (GPT-4v) and GPT-4omni (GPT-4o) on the ACR's Diagnostic Radiology in-Training (DXIT) examination, comparing performance on image-based and text-only questions.

METHODS: In all, 1,136 publicly available DXIT examination questions were input into GPT-4v and GPT-4o with a prompt asking the large language model to provide its answer, rationale, and confidence level (0-100). Accuracy of each model across different categories was then analyzed, with χ2 tests to compare proportions, t tests to compare means, and receiver operating characteristic curves to evaluate confidence levels.

RESULTS: GPT-4o and GPT-4v achieved accuracies of 73.5% and 69.3%, respectively (P < .0001) while scoring 55.6% and 50.3% on image-based questions (P < .0001). Receiver operating characteristic curves of confidence levels and correctness produced areas under the curve of 0.64 and 0.66 for GPT-4o and GPT-4v, respectively.

DISCUSSION: GPT-4o outperformed GPT-4v on nearly every metric, with both models outperforming the national average performance of postgraduate year 3 radiology residents (61.9%) on the 2022 DXIT examination. However, performance on image-based questions remains significantly worse than text-only questions, and both models score below radiology trainees from the same cohort. Both models exhibit limited ability to predict correctness using an intrinsic confidence level. Use of ChatGPT for test preparation and image interpretation must therefore be approached with caution.

来源:J Am Coll Radiol,查看原文。版权归原作者所有。

引用

  1. Martini RS, Sang A, Saunders P, et al. Artificial Intelligence in Radiology: Performance of ChatGPT-4v and GPT-4o on Diagnostic Radiology in-Training (DXIT) Examination Questions. J Am Coll Radiol. 2025. PMID: 41173086. doi:10.1016/j.jacr.2025.10.026

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。