人工智能聊天机器人参加美国牙体牙髓病学委员会模拟口试的表现评估
本研究由2名学术委员会认证的牙体牙髓病学家构建3个口试病例,每个病例含患者病史、牙科史和临床检查结果及20道开放式口试问题,评估GPT-4o和Gemini-2.5 Pro两款AI聊天机器人的表现。结果显示Gemini-2.5 Pro和GPT-4o的平均总体表现得分分别为2.83(±0.42)和2.73(±0.51),两者在获得优秀评分的概率上无显著差异。结论支持AI聊天机器人可作为牙体牙髓病学教育的辅助工具。
要点
- 两款AI聊天机器人在模拟口试中总体表现得分较高,均接近优秀水平。
- GPT-4o与Gemini-2.5 Pro在回答有效性和总体表现上无显著差异。
- AI聊天机器人可作为牙体牙髓病学教育的辅助工具。
核心结论
- 两款聊天机器人在模拟美国牙体牙髓病学委员会口试中得分相当高,支持AI聊天机器人作为牙体牙髓病学教育辅助工具的理念。
研究设计与样本
- 由2名学术委员会认证的牙体牙髓病学家构建3个口试病例,每个病例含完整患者档案(病史、牙科史、临床检查结果)及20道连续开放式口试问题。
- 选取GPT-4o和Gemini-2.5 Pro两款公开可用的AI聊天机器人参加考试,由同一2名考官独立使用4分制序数量表(0-3分)评分,评估回答有效性、引用有效性和总体表现。
- 采用累积链接混合模型(比例优势模型)分析,固定效应为聊天机器人和病例,随机截距为评审者和问题。
主要发现
- Gemini-2.5 Pro和GPT-4o的平均总体表现得分分别为2.83(±0.42)和2.73(±0.51)。
关键数值与推荐
- 评分采用4分制序数量表(0-3分),评估维度包括回答有效性、引用有效性和总体表现。
临床怎么用
- AI聊天机器人可作为牙体牙髓病学教育的辅助工具,用于模拟口试训练和教学支持。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
INTRODUCTION: The aim of this study was to assess the overall performance of artificial intelligence (AI) chatbots in taking the American Board of Endodontics simulated Oral Board Examination. METHODS: Three oral board cases were constructed by 2 academic board-certified endodontists. Each case included a comprehensive patient profile consisting of medical history, dental history, and results of clinical testing, followed by 20 consecutive open-ended oral board-style questions. Two publicly accessible AI chatbots were selected to take the exam: GPT-4o and Gemini-2.5 Pro. Responses were scored based on a comprehensive rubric on a 4-point ordinal scale (0-3) by the same 2 examiners independently: response validity, citation validity, and overall performance score. A Cumulative Link Mixed Model (proportional odds) was used with fixed effects for chatbot and case, and random intercepts for reviewer and question to analyze and compare the performance of models, that is, inter- and intra-chatbot comparisons. RESULTS: Gemini-2.5 Pro and GPT-4o achieved high mean overall performance scores of 2.83 (±0.42) and 2.73 (±0.51), respectively. Cumulative Link Mixed Model showed no significant difference between the 2 chatbots in probability of receiving an excellent score (ie, 3) in response validity (odds ratio = 2.44, 95% confidence interval [0.98-6.06], P = .054) or in overall performance (odds ratio = 2.04, 95% confidence interval 0.97-4.30, P = .061). There was a positive correlation between response validity and citation validity for GPT-4o (ρ = 0.21, P = .019). CONCLUSIONS: Both chatbots scored considerably high in the simulated American Board of Endodontics Oral Board Examination. Results of this study support the concept of using AI chatbots as aid in endodontic education.
引用
- Jalali P, Wang FM, Ourang SA, et al. Artificial Intelligence Chatbots Taking American Board of Endodontics Simulated Oral Board Examination. J Endod. 2026. PMID: 41759779. doi:10.1016/j.joen.2026.02.014
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。