指南与共识口腔修复与种植

四种大语言模型对无牙颌修复方案推荐的评价与比较

[Recommendation and comparison of four large language models for prosthetic treatment options in edentulous patients].
指南 / 共识 · Zhonghua Kou Qiang Yi Xue Za Zhi · 2026-01-01

该研究构建12份无牙颌标准化病历,评估ChatGPT、通义千问、文心一言、DeepSeek四种大语言模型推荐修复方案的全面性与一致性。ChatGPT综合评分最高(3.13分),与专家意见一致性最佳(12/12),四种模型总体差异有统计学意义(H=22.80,P<0.001)。各模型对非医学因素提及率较低,尚不能替代临床医师判断。

要点

  • ChatGPT综合评分最高(3.13分),与专家意见一致性最佳(12/12)。
  • 四种模型对就诊便利性和长期维护能力提及率均≤6/12。
  • 当前大语言模型不能替代临床医师判断,仅有一定辅助价值。

核心结论

  • 四种大语言模型在无牙颌修复决策中表现存在显著差异(H=22.80,P<0.001)。
  • ChatGPT综合评分最高且与专家意见一致性最佳(12/12)。
  • 文心一言、通义千问和DeepSeek对非医学因素提及率较低,不能替代临床医师判断。

研究设计与样本

  • 构建12份模拟无牙颌患者标准化文本病历,每份包含六大维度信息。
  • 四种大语言模型(ChatGPT、通义千问、文心一言、DeepSeek)各对12份病历回答3次,每次清除历史对话。
  • 3名主任医师组成专家组提供意见,2名高年资主治医师使用多维评价量表独立评分。
  • 采用Kruskal-Wallis检验分析各模型间差异。

主要发现

  • 高年资主治医师间评分一致性良好(Spearman系数=0.906,P<0.001)。
  • ChatGPT综合评分最高,为3.13(2.75,3.44)分;DeepSeek为2.75(2.50,2.94)分;通义千问为2.38(2.00,2.50)分;文心一言为2.13(1.56,2.43)分。
  • ChatGPT与专家意见基本一致和完全一致合计占比为12/12。
  • 各模型对口腔局部条件、全身健康状况和社会经济因素的提及率均≥6/12(ChatGPT≥11/12)。
  • 文心一言、通义千问和DeepSeek对就诊便利性及长期维护能力的提及率均≤6/12。

临床怎么用

  • 大语言模型可在结构化信息呈现和方案一致性方面提供辅助价值。
  • 临床决策仍需医师综合权衡所有关键因素,尤其非医学因素。

局限性

  • 当前大语言模型尚不能全面权衡所有关键决策因素,对非医学因素评估存在明显短板。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

Objective: To evaluate the comprehensiveness and consistency of treatment plan recommendations provided by four large language models (LLM) for edentulous patients, and to explore their feasibility and limitations as clinical decision-making support tools. Methods: Twelve standardized simulated clinical records of edentulous patients were constructed, each fully incorporating information on six dimensions influencing treatment plan selection (local oral conditions, general health status, socioeconomic factors, patient preferences, convenience of maintenance, and long-term maintenance capability). Four LLM (ChatGPT, Tongyi Qianwen, Wenxin Yiyan, DeepSeek) were used to provide treatment recommendations and corresponding rationales for implant-supported prostheses and complete dentures (single-turn question-and-answer, fixed prompts, each LLM responded three times to each of the 12 standardized records, with the conversation history cleared before each response). Meanwhile, a panel of three chief physicians specializing in prosthodontics and implant dentistry was invited to provide expert opinions for each scenario. Subsequently, two senior attending physicians independently evaluated the responses of each LLM using a multidimensional scoring system (consistency of recommendations, completeness of rationales, risk identification capability, and decision-making transparency). Finally, the Kruskal-Wallis test was used to analyze differences among the LLM. Results: The inter-rater reliability between the two senior attending physicians was excellent (Spearman's ρ=0.906, P0.001). The performance of LLM varied across dimensions. ChatGPT achieved the highest overall score of 3.13 (2.75, 3.44), followed by DeepSeek 2.75 (2.50, 2.94), Tongyi Qianwen 2.38 (2.00, 2.50), and Wenxin Yiyan 2.13 (1.56, 2.43). The overall difference among the four models was statistically significant (H=22.80, P0.001). Regarding consistency of recommendations, ChatGPT showed the highest concordance with expert opinions (the combined proportion of basic and complete consistency was 12/12). In terms of completeness of rationales, the mention rates of local oral conditions, general health status, and socioeconomic factors were all ≥6/12 for each model (with ChatGPT achieving ≥11/12). However, for convenience of maintenance and long-term maintenance capability, the mention rates for Wenxin Yiyan, Tongyi Qianwen, and DeepSeek were ≤6/12. Conclusions: The four LLM showed significant differences in their performance in edentulous restoration decision-making. ChatGPT achieved the highest overall score and the best concordance with expert opinions. Regarding non-medical factors such as convenience of maintenance and long-term maintenance capability, the mention rates of Wenxin Yiyan, Tongyi Qianwen, and DeepSeek were relatively low. These findings suggest that current LLM are not yet capable of comprehensively weighing all key decision-making factors, with a notable deficiency in assessing non-medical factors in particular. They cannot replace the judgment of clinical practitioners; however, they demonstrate certain supportive value in structured information presentation and consistency of recommendations. 目的: 评估4种大语言模型(LLM)对无牙颌患者修复推荐方案的全面性和一致性,探讨其作为临床决策辅助工具的可行性及局限性。 方法: 构建12份模拟无牙颌患者的标准化文本病历,每份病历均完整包含影响修复方案选择的六大维度(口腔局部条件、全身健康状况、社会经济因素、患者主观意愿、就诊便利性、长期维护能力)信息,由4种LLM(ChatGPT、通义千问、文心一言、DeepSeek)分别给出种植体支持式修复及全口义齿修复的推荐方案及相应理由(单轮问答,固定提示词,每种LLM对上述12份标准化文本病历回答3次,每次回答前均清除历史对话)。同时邀请口腔修复、种植领域的3名主任医师组成专家组,获得针对各标准化文本病历的专家意见。由2名高年资主治医师使用多维评价量表(方案一致性、理由完整性、风险识别能力、决策透明度)对各LLM的回答进行独立评分,最后采用Kruskal-Wallis检验分析各LLM间差异。 结果: 高年资主治医师间评分一致性良好(Spearman系数=0.906,P0.001)。4种LLM各维度评分表现各异:ChatGPT综合评分最高,为3.13(2.75,3.44)分,其次是DeepSeek[2.75(2.50,2.94)分],通义千问为2.38(2.00,2.50)分,最后为文心一言[2.13(1.56,2.43)分];4种模型综合评分总体差异有统计学意义(H=22.80,P0.001)。具体而言,方案一致性方面,ChatGPT与专家意见吻合度最高(基本一致和完全一致合计占比为12/12)。理由完整性方面,各LLM对口腔局部条件、全身健康状况和社会经济因素的提及率均≥6/12(其中ChatGPT≥11/12);但对于就诊便利性及长期维护能力,文心一言、通义千问和DeepSeek的提及率均≤6/12。 结论: 4种LLM在无牙颌修复决策中的表现存在显著差异,ChatGPT综合评分最高且与专家意见一致性最佳。在就诊便利性、长期维护能力等非医学因素方面,文心一言、通义千问和DeepSeek的提及率较低;提示当前LLM尚不能全面权衡所有关键决策因素,尤其对非医学因素的评估存在明显短板,不能替代临床医师判断,但在结构化信息呈现和方案一致性方面显示出一定的辅助价值。.

来源:Zhonghua Kou Qiang Yi Xue Za Zhi,查看原文。版权归原作者所有。

引用

  1. Li X, Feng Y, Li J, et al. [Recommendation and comparison of four large language models for prosthetic treatment options in edentulous patients]. Zhonghua Kou Qiang Yi Xue Za Zhi. 2026. PMID: 42394556. doi:10.3760/cma.j.cn112144-20260405-00214

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。