前沿大语言模型在FRCR Part 1风格考试中的表现比较
本研究比较了三种前沿大语言模型在模拟FRCR Part 1考试中的表现。物理模块含40道判断题(200个评分点),解剖模块含100道图像题(0-2分制)。结果显示模型在文本型物理题上表现强劲,在图像解剖题上差异显著。
要点
- 大语言模型在文本型物理题上表现优异,可用于放射科医师备考辅助。
- 图像解剖题上模型间差异显著,阅片与精确定位仍是短板。
- 可将此类模型用于教学场景,但不宜替代系统解剖训练。
核心结论
- 大语言模型在文本型物理题上表现强劲,但在图像解剖题上表现差异较大,提示可用于教学辅助但存在解剖定位局限。
研究设计与样本
- 研究类型为基于题库的横断面比较研究,使用定制的FRCR Part 1风格考试。
- 物理模块含40道判断题共200个独立评分点,解剖模块含100道基于图像的题目按0-2分制评分。
- 试题结构按照已发表的FRCR Part 1主题与成像方式分布设计,由两名已通过该考试的放射科医师审校。
- 评估对象为GPT-5.4 Thinking、Gemini 3.1 Pro和Claude Opus 4.6三种大语言模型。
- 物理模块比较采用Cochran Q检验与精确McNemar检验,解剖模块采用Friedman检验与Wilcoxon符号秩检验。
诊断标准与关键数值
- 物理模块得分为Claude 195/200、GPT 189/200、Gemini 187/200,总体P=0.018。
- 物理模块Holm校正后Claude优于Gemini(校正P=0.023),其余两两比较差异无统计学意义。
- 解剖模块得分为Gemini 160/200、GPT 137/200、Claude 83/200,总体P<0.001。
- 解剖模块所有两两比较经Holm校正后仍具有统计学意义(校正P≤0.038)。
临床应用与阅片要点
- 大语言模型可作为放射科医师备考FRCR Part 1物理模块的辅助工具。
- 在涉及图像解剖定位的训练中不宜依赖此类模型,需结合真实影像与解剖教学。
局限性
- 研究使用自定义模拟试题而非官方FRCR Part 1考试真题,结论外推需谨慎。
- 试题由两名放射科医师审校,但样本量和审校流程的稳健性有限。
- 研究仅评估三种模型,未覆盖其他主流大语言模型版本。
- 图像题评分为0-2分制,可能无法完全反映临床影像解读的细微差异。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
RATIONALE AND OBJECTIVES: This study aims to compare the performance of three frontier large language models on an Fellowship of the Royal College of Radiologists (FRCR) Part 1-style examination comprising physics and image-based anatomy modules.
MATERIALS AND METHODS: GPT-5.4 Thinking, Gemini 3.1 Pro, and Claude Opus 4.6 were evaluated using a custom FRCR Part 1-style examination. The physics module included 40 true/false questions comprising 200 independently scored statements, while the anatomy module included 100 image-based questions scored on a 0-2 scale. Both modules were structured according to the published FRCR Part 1 topic and modality distributions and were reviewed by two radiologists who had previously passed the examination. Model performance was compared using Cochran's Q and exact McNemar tests for physics and Friedman and Wilcoxon signed-rank tests for anatomy.
RESULTS: Physics scores were 195/200 for Claude, 189/200 for GPT, and 187/200 for Gemini (overall P = 0.018). After Holm correction, Claude outperformed Gemini (adjusted P = 0.023), whereas the other pairwise differences were not significant. Anatomy scores were 160/200 for Gemini, 137/200 for GPT, and 83/200 for Claude (overall P < 0.001); all pairwise comparisons remained significant after Holm correction (adjusted P ≤ 0.038).
CONCLUSION: In this custom FRCR Part 1-style assessment, large language models performed strongly on text-based physics questions but showed more variable performance on image-based anatomy, supporting their potential educational role while highlighting persistent limitations in precise anatomical localization.
引用
- Güzel, Ünverdi. Performance of Frontier Large Language Models on an FRCR Part 1-Style Examination: A Comparative Study of Physics and Anatomy Modules. Acad Radiol. 2026. PMID: 42674933. doi:10.1016/j.acra.2026.08.077
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。