大型语言模型在放射科住院医师申请中的使用:不受欢迎但不可避免
本研究通过混合方法调查8名放射科住院医师项目主任对申请人使用大型语言模型撰写个人陈述的看法,发现GPT生成陈述的质量评级更多为一般或更差(56%,268/475)而非人工撰写(29%,45/160),评审者能以95%(38/40)可靠识别人工陈述,讨论强调AI使用的不可避免性及对真实性的担忧。
要点
- 项目主任对GPT生成个人陈述的质量评级更多为一般或更差(56% vs 29%),提示AI文本质量不被认可。
- 评审者虽不自信但能可靠识别人工陈述(95%判为可能或确定原创),说明AI文本有可辨识特征。
- 讨论认为AI在申请陈述中的使用不可避免,但对真实性和选拔价值存在担忧,监管意见分歧。
核心结论
- 放射科住院医师项目主任将LLM生成的个人陈述评为较低质量,担忧申请人声音的丢失,但承认AI在申请陈述生成中的使用增加是不可避免的。
研究设计与样本
- 混合方法研究,8名来自放射科住院医师教育研究联盟的项目主任参与,包括调查和焦点小组讨论(2023年7月),每位主任审阅5名申请人的4个个人陈述变体(原始版和3个GPT-4.0生成版),匿名化后汇总分析。
关键影像表现/征象
- GPT生成陈述的质量评级更多为一般或更差(56%,268/475),而人工撰写陈述为29%(45/160)。
- 评审者识别人工撰写个人陈述为可能或确定原创的准确率为95%(38/40)。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVE: This study explores radiology program directors' perspectives on the impact of large language model (LLM) use among residency applicants to craft personal statements.
METHODS: Eight program directors from the Radiology Residency Education Research Alliance participated in a mixed-methods study, which included a survey regarding impressions of artificial intelligence (AI)-generated personal statements and focus group discussions (July 2023). Each director reviewed four personal statement variations for five applicants, anonymized to author type: the original and three Chat Generative Pre-trained Transformer-4.0 (GPT) versions generated with varying prompts, aggregated for analysis. A 5-point Likert scale surveyed the writing quality, including voice, clarity, engagement, organization, and perceived origin of each statement. An experienced qualitative researcher facilitated focus group discussions. Data analysis was performed using a rapid analytic approach with a coding template capturing key areas related to residency applications.
RESULTS: GPT-generated statement ratings were more often average or worse in quality (56%, 268 of 475) than ratings of human-authored statements (29%, 45 of 160). Although reviewers were not confident in their ability to distinguish the origin of personal statements, they did so reliably and consistently, identifying the human-authored personal statements at 95% (38 of 40) as probably or definitely original. Focus group discussions highlighted the inevitable use of AI in crafting personal statements and concerns about its impact on the authenticity and the value of the personal statement in residency selections. Program directors were divided on the appropriate use and regulation of AI.
DISCUSSION: Radiology residency program directors rated LLM-generated personal statements as lower in quality and expressed concern about the loss of the applicant's voice but acknowledged the inevitability of increased AI use in the generation of application statements.
引用
- Gordon EB, Maxfield CM, French R, et al. Large Language Model Use in Radiology Residency Applications: Unwelcomed but Inevitable. J Am Coll Radiol. 2024. PMID: 39299618. doi:10.1016/j.jacr.2024.08.027
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。