基于GPT-4的内容感知聊天机器人可为牙科影像中的锥形束CT指南提供可信推荐
研究开发基于GPT-3.5-Turbo和GPT-4、整合德国S2锥形束CT牙科影像指南知识的内容感知聊天机器人,用40个指南相关问题测试并与早期职业及资深执业者对照。GPT-4聊天机器人推荐准确率100%,解释质量优于GPT-3.5-Turbo版本,且正确回答优于早期职业执业者,可信度更高。
要点
- GPT-4聊天机器人对40个CBCT指南问题给出100%正确推荐。
- GPT-3.5-Turbo聊天机器人正确率为87.5%。
- GPT-4版本解释质量优于GPT-3.5-Turbo版本(P = .003)。
核心结论
- 基于GPT-4的内容感知聊天机器人能依据当前共识指南可靠地提供推荐,其回答被认为可信且透明,有助于将人工智能整合进临床决策。
研究设计与样本
- 研究使用LlamaIndex软件库将德国S2锥形束CT牙科影像指南上下文整合进基于GPT-3.5-Turbo和GPT-4的聊天机器人,基于该指南提出40个问题,以不同经验水平的早期职业和资深执业者作为参照,比较推荐准确率和解释质量,采用卡方检验和单侧Wilcoxon符号秩检验分别评估准确率和解释质量。
诊断标准与关键数值
- GPT-4聊天机器人提供100%正确推荐。
- GPT-3.5-Turbo聊天机器人正确率为87.5%。
- GPT-4聊天机器人解释质量优于GPT-3.5-Turbo版本(P = .003)。
- GPT-4聊天机器人在正确回答上优于早期职业执业者(P = .002和P = .032)。
- GPT-4聊天机器人可信度高于GPT-3.5-Turbo聊天机器人(P = 0.006)。
临床应用与阅片要点
- 原文结论认为基于GPT-4的内容感知聊天机器人能依据当前共识指南可靠地提供推荐。
- 原文认为其回答被认为可信且透明,有助于将人工智能整合进临床决策。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVES: To develop a content-aware chatbot based on GPT-3.5-Turbo and GPT-4 with specialized knowledge on the German S2 Cone-Beam CT (CBCT) dental imaging guideline and to compare the performance against humans.
METHODS: The LlamaIndex software library was used to integrate the guideline context into the chatbots. Based on the CBCT S2 guideline, 40 questions were posed to content-aware chatbots and early career and senior practitioners with different levels of experience served as reference. The chatbots' performance was compared in terms of recommendation accuracy and explanation quality. Chi-square test and one-tailed Wilcoxon signed rank test evaluated accuracy and explanation quality, respectively.
RESULTS: The GPT-4 based chatbot provided 100% correct recommendations and superior explanation quality compared to the one based on GPT3.5-Turbo (87.5% vs. 57.5% for GPT-3.5-Turbo; P = .003). Moreover, it outperformed early career practitioners in correct answers (P = .002 and P = .032) and earned higher trust than the chatbot using GPT-3.5-Turbo (P = 0.006).
CONCLUSIONS: A content-aware chatbot using GPT-4 reliably provided recommendations according to current consensus guidelines. The responses were deemed trustworthy and transparent, and therefore facilitate the integration of artificial intelligence into clinical decision-making.
引用
- Russe MF, Rau A, Ermer MA, et al. A content-aware chatbot based on GPT 4 provides trustworthy recommendations for Cone-Beam CT guidelines in dental imaging. Dentomaxillofac Radiol. 2024. PMID: 38180877. doi:10.1093/dmfr/twad015
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。