评估生成式人工智能作为放射科住院医师报告起草教育工具的价值
本研究分析了美国多站点健康系统常规实践中5000对住院医师-主治医师报告对,使用符合HIPAA的GPT-4o系统提供自动反馈,并在100对报告上进行阅读者研究。GPT-4o与主治医师共识在三种错误类型上一致率分别为90.5%、78.3%、90.4%,反馈在多数案例中被评为有帮助。
要点
- GPT-4o在识别关键发现遗漏或添加、技术描述词错误、最终评估与发现不一致三类错误上与主治医师共识一致率较高。
- GPT-4o反馈在大多数案例中被评为有帮助,比例分别为89.8%、83.0%、92.0%。
- 将GPT-4o替换单个阅读者后,阅读者间一致性变化无统计学显著差异。
核心结论
- ChatGPT-4o能可靠识别关键教育错误,可能作为支持放射学教育的可扩展工具。
研究设计与样本
- 研究分析了美国多站点健康系统常规实践中的5000对住院医师-主治医师报告对,使用GPT-4o系统提供反馈,并进行100对报告的阅读者研究,由4名主治放射科医师和4名住院医师独立评审。
诊断标准与关键数值
- GPT-4o与主治医师共识在错误类型1(关键发现遗漏或添加)的一致率为90.5%(Cohen's κ=0.790)。
- GPT-4o与主治医师共识在错误类型2(技术描述词错误使用或遗漏)的一致率为78.3%(Cohen's κ=0.550)。
- GPT-4o与主治医师共识在错误类型3(最终评估与发现不一致)的一致率为90.4%(Cohen's κ=0.615)。
- 所有8名阅读者间可靠性为中等至较大变异(Krippendorff's α=0.767、0.595、0.567)。
- 将每个阅读者 individually 替换为GPT-4o后,7名阅读者与GPT的阅读者间一致性重算,效应无统计学显著差异(Δ=-0.004至0.002,所有P>.05)。
临床应用与阅片要点
- 作者认为GPT-4o可能作为支持放射学教育的可扩展工具。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVE: Radiology residents require timely, personalized feedback to develop accurate image analysis and reporting skills. Increasing clinical workload often limits attendings' ability to provide guidance. This study evaluates a HIPAA-compliant Generative Pretrained Transformer (GPT)-4o system that delivers automated feedback on breast imaging reports drafted by residents in real clinical settings.
METHODS: We analyzed 5,000 resident-attending report pairs from routine practice at a multisite US health system. GPT-4o was prompted with clinical instructions to identify common errors and provide feedback. A reader study using 100 report pairs was conducted. Four attending radiologists and four residents independently reviewed each pair, determined whether predefined error types were present, and rated GPT-4o's feedback as helpful or not. Agreement between GPT and readers was assessed using percent match. Interreader reliability was measured with Krippendorff's α. Educational value was measured as the proportion of cases rated helpful.
RESULTS: Three common error types were identified: (1) omission or addition of key findings, (2) incorrect use or omission of technical descriptors, and (3) final assessment inconsistent with findings. GPT-4o showed strong agreement with attending consensus: 90.5%, 78.3%, and 90.4% (Cohen's κ: 0.790, 0.550, and 0.615) across error types. Interreader reliability among all eight readers showed moderate to substantial variability (α = 0.767, 0.595, 0.567). When each reader was individually replaced with GPT-4o and interreader agreement among seven readers and GPT was recalculated, the effect was not statistically significant (Δ = -0.004 to 0.002, all P > .05). GPT's feedback was rated helpful in most cases: 89.8%, 83.0%, and 92.0%.
DISCUSSION: ChatGPT-4o can reliably identify key educational errors. It may serve as a scalable tool to support radiology education.
引用
- Verdone A, Cardall A, Siddiqui F, et al. Evaluating Generative Artificial Intelligence as an Educational Tool for Radiology Resident Report Drafting. J Am Coll Radiol. 2025. PMID: 41453630. doi:10.1016/j.jacr.2025.12.024
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。