利用ChatGPT增强放射科住院医师教育中的学习
本研究评估ChatGPT基于住院医师讲座转录文本生成委员会风格单选题的效果。21名放射科住院医师对17道ChatGPT生成题与11道主治医师编写题进行作答、质量评分和来源判断。结果显示两类题目在感知质量和正确率上无显著差异,但ChatGPT生成题被误判为主治医师所写的比例显著较低;作者认为大语言模型在放射科教育材料生成方面具有潜力,需在更大规模上进一步探索。
要点
- ChatGPT生成题与主治医师编写题的感知质量评分无显著差异(M=6.93 vs M=7.08,p=0.15)。
- 两类题目的正确率无显著差异(ChatGPT生成题M=57% vs 主治医师编写题M=59%,p=0.78)。
- ChatGPT生成题被误判为主治医师所写的比例显著低于主治医师编写题(M=57% vs M=71%,p=0.04)。
核心结论
- ChatGPT等大语言模型在生成和呈现放射科教育材料方面显示出潜力,作者认为需在更大规模上进一步探索其应用。
研究设计与样本
- 研究论文;使用自定义提示词由ChatGPT从住院医师讲座转录文本生成单选题,选取17道ChatGPT生成题与11道主治放射科医师编写题随机组合,21名放射科住院医师参与作答、评分和来源判断。
诊断标准与关键数值
- 感知题目质量评分:ChatGPT生成题M=6.93,SD=0.29;主治医师编写题M=7.08,SD=0.51;p=0.15。
- 题目正确率:ChatGPT生成题M=57%,SD=20%;主治医师编写题M=59%,SD=25%;p=0.78。
- 被判断为主治医师所写的题目比例:ChatGPT生成题M=57%,SD=13%;主治医师编写题M=71%,SD=20%;p=0.04。
局限性
- 作者在结论中提出需在更大规模上进一步探索大语言模型在放射科教育中的应用。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
RATIONALE AND OBJECTIVES: Chat generative pre-trained transformer (ChatGPT) is a generative artificial intelligence chatbot based on a LLM at the forefront of technological development with promising applications in medical education. This study aims to evaluate the use of ChatGPT in generating board-style practice questions for radiology resident education.
MATERIALS AND METHODS: Multiple-choice questions (MCQs) were generated by ChatGPT from resident lecture transcripts using a custom prompt. 17 of the ChatGPT-generated MCQs were selected for inclusion in the study and randomly combined with 11 attending radiologist-written MCQs. For each MCQ, the 21 participating radiology residents answered the MCQ, rated the MCQ from 1-10 on effectiveness in reinforcing lecture material, and responded whether they thought an attending radiologist at their institution wrote the MCQ versus an alternative source.
RESULTS: Perceived MCQ quality was not significantly different between ChatGPT-generated (M=6.93, SD=0.29) and attending radiologist-written MCQs (M=7.08, SD=0.51) (p=0.15). MCQ correct answer percentages did not significantly differ between ChatGPT-generated (M=57%, SD=20%) and attending radiologist-written MCQs (M=59%, SD=25%) (p=0.78). The percentage of MCQs thought to be written by an attending radiologist was significantly different between ChatGPT-generated (M=57%, SD=13%) and attending radiologist-written MCQs (M=71%, SD=20%) (p=0.04).
CONCLUSION: LLMs such as ChatGPT demonstrate potential in generating and presenting educational material for radiology education, and their use should be explored further on a larger scale.
引用
- Zheng A, Barker CJ, Ferrante SS, et al. Leveraging ChatGPT for Enhancing Learning in Radiology Resident Education. Acad Radiol. 2025. PMID: 40628645. doi:10.1016/j.acra.2025.06.019
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。