四种AI聊天机器人在创伤性牙髓后遗症诊疗中的准确性与完整性评估
本研究评估四种免费AI聊天机器人(ChatGPT 3.5、Copilot、Perplexity、DeepSeek)对14个牙髓病专家设计问题的回答准确性与完整性。结果显示DeepSeek在诊断方面表现更优,Perplexity在治疗领域表现更佳,所有机器人在再生牙髓程序概念题上准确率高,但均未在所有场景中达到专家基准。
要点
- DeepSeek在诊断类问题上准确性优于其他机器人。
- Perplexity在治疗相关领域表现最佳。
- 所有机器人对再生牙髓程序概念题回答准确率高。
核心结论
- DeepSeek在诊断方面表现更优,Perplexity在治疗领域表现更佳。
- 所有AI聊天机器人在部分领域表现中到高度,但均未在所有评估场景中持续匹配专家基准。
- 聊天机器人生成的信息应谨慎解读,临床使用前需对照当前循证指南验证。
研究设计与样本
- 使用14个由牙髓病专家开发的结构化问题评估四种AI聊天机器人(ChatGPT 3.5、Copilot、Perplexity、DeepSeek)。
- 问题包括9个关于创伤性未成熟和成熟恒牙临床病例的问题(诊断、治疗、先诊断后治疗)和5个关于再生牙髓程序(REP)的概念性问题。
- 在基线(第1天)和7天后(第8天)重复应用,两名盲法牙髓病专家独立评估回答的准确性(1-6分)和完整性(1-3分)。
主要发现
- 基线时,DeepSeek在诊断方面超过其他机器人,Perplexity在治疗相关领域持续优于其他机器人。
- 在回答准确性上,Perplexity得分高于ChatGPT和Microsoft Copilot,与DeepSeek相似。
- 在完整性上,各AI聊天机器人之间无显著差异(P > .05)。
- 随时间变化,仅DeepSeek在第8天完整性得分(均值±标准差:1.56 ± 0.8)较基线(均值±标准差:1.34 ± 0.55)有显著增加,P < .05。
- 概念性REP问题在所有聊天机器人中均达到高准确性。
关键数值与推荐
- 准确性评分范围为1-6分,完整性评分范围为1-3分,基于与专家基准的一致性。
- 完整性比较无显著差异(P > .05);DeepSeek完整性得分从基线1.34 ± 0.55增至第8天1.56 ± 0.8(P < .05)。
临床怎么用
- 临床医生可参考AI聊天机器人在诊断和治疗方面的辅助信息,但需谨慎解读。
- 在临床应用前,应将聊天机器人生成的信息与当前循证指南进行验证。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
INTRODUCTION: This study aimed to evaluate the accuracy and completeness of the answers generated by four freely available artificial intelligence (AI) chatbots regarding the management of endodontic sequelae in traumatized permanent teeth and the regenerative endodontic procedure, analyzing their performance by question type and over time. METHODS: Four AI chatbots (ChatGPT 3.5, Copilot, Perplexity, and DeepSeek) were evaluated with 14 structured questions developed by endodontists, applied at baseline (day 1) and repeated after 7 days (day 8). Nine questions involved clinical cases of traumatized immature and mature permanent teeth (diagnosis, treatment, and treatment with prior diagnosis), and five were conceptual about regenerative endodontic procedures (REPs). Two blinded endodontists independently evaluated response accuracy (1-6 points) and completeness (1-3 points) based on agreement with the expert benchmark, with results reported as median ± interquartile range. Reproducibility was analyzed with weighted Kappa, while Kruskal-Wallis with Dunn's post hoc and Wilcoxon signed-rank tests were used for comparisons. RESULTS: On baseline, DeepSeek exceeded in diagnosis while Perplexity consistently outperformed others in treatment-related domains. Regarding response accuracy, Perplexity showed higher score values compared to ChatGPT and Microsoft Copilot, and similar scores to DeepSeek. In terms of completeness, no significant differences were observed among the AI chatbots (P > .05). Over time, only DeepSeek demonstrated a significant increase in completeness scores on day 8 (mean ± standard deviation: 1.56 ± 0.8) compared to baseline (mean ± standard deviation: 1.34 ± 0.55), with P < .05. Conceptual REP questions achieved high accuracy across chatbots. CONCLUSIONS: DeepSeek showed superior performance in diagnosis, while Perplexity outperformed others in treatment domains. All chatbots performed well on conceptual REP questions. Although chatbots demonstrated moderate-to-high performance in selected domains, none consistently matched the expert benchmark across all evaluated scenarios. Therefore, chatbot-generated information should be interpreted with caution and verified against current evidence-based guidelines before clinical application.
引用
- Portes-Zeno AP, Caetano BP, da Conceição GASG, et al. Accuracy and Completeness of Four Artificial Intelligence Chatbots in Assisting Diagnosis and Treatment of Post-traumatic Endodontic Sequelae. J Endod. 2026. PMID: 42526596. doi:10.1016/j.joen.2026.07.017
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。