大型语言模型在复杂牙髓病例中的诊断预测与临床推理比较分析
该研究将9例复杂牙髓病例报告转为标准化叙述并隐藏诊断,比较GPT-4o、Claude 3.7 Sonnet、DeepSeek R1和Gemini 2.0 Flash四种大语言模型在结构化提示下的诊断预测准确性与临床推理。结果显示Gemini 2.0 Flash和Claude 3.7 Sonnet的解读推理质量总体高于DeepSeek R1,但与GPT-4o的差异未达统计学显著。
要点
- Gemini 2.0 Flash与Claude 3.7 Sonnet的解读推理质量总体高于DeepSeek R1。
- Claude和Gemini与GPT-4o的差异经Bonferroni校正后无统计学显著性。
- 模型选择与结构化提示影响AI辅助牙髓诊断的可靠性。
核心结论
- 当代大语言模型的解读推理表现存在显著差异,Claude 3.7 Sonnet和Gemini 2.0 Flash优于DeepSeek R1,但与GPT-4o无显著差异。
研究设计与样本
- 将9例来自同行评审期刊的复杂牙髓病例报告转换为隐藏诊断的标准化叙述,每种模型在结构化提示下生成三个按贝叶斯排序的诊断可能及理由。
- 由两名牙髓专科医生独立将金标准诊断抽象为标准化参考格式,并使用评估临床合理性、影像学依据和术语准确性的评分表评价模型输出。
- 统计分析包括Kruskal-Wallis检验、Bonferroni校正两两比较和诊断一致性指数。
主要发现
- Gemini 2.0 Flash和Claude 3.7 Sonnet在所有诊断等级上的解读推理质量总体高于DeepSeek R1。
- Bonferroni校正后,Claude和Gemini与GPT-4o的差异无统计学显著性(校正P=1.000)。
- DeepSeek R1与Claude和Gemini的差异有统计学显著性(校正P<0.05),与GPT-4o的比较接近显著性(校正P=0.059)。
临床怎么用
- 模型选择和结构化提示对AI辅助牙髓诊断的可靠性有重要影响。
- Gemini 2.0和Claude 3.7显示出更优的推理质量,临床整合时需专门训练和人工监督。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVE: To compare the diagnostic prediction accuracy and clinical reasoning of four state-of-the-art large language models (LLMs) - GPT-4o, Claude 3.7 Sonnet, DeepSeek R1, and Gemini 2.0 Flash - in complex endodontic case analysis using structured prompts. METHODS: Nine complex endodontic case reports from peer-reviewed journals were converted into standardized narratives with concealed diagnoses. Each LLM generated three prioritized (Bayesian-ranked) diagnostic possibilities with corresponding justifications using structured prompts. Two endodontic specialists independently abstracted the gold-standard diagnoses into a standardized reference format and evaluated the LLM outputs using a rubric assessing clinical plausibility, radiographic justification, and terminological accuracy. Statistical analysis included Kruskal-Wallis testing, Bonferroni-adjusted pairwise comparisons, and the Diagnostic Agreement Index. RESULTS: Gemini 2.0 Flash and Claude 3.7 Sonnet generally achieved higher interpretive reasoning quality than DeepSeek R1 across all diagnostic ranks. However, post-hoc Bonferroni-adjusted comparisons revealed that their differences relative to GPT-4o were not statistically significant (adjusted p = 1.000), whereas DeepSeek R1 differed significantly from both Claude and Gemini (adjusted p < 0.05) and approached significance compared with GPT-4o (adjusted p = 0.059). CONCLUSIONS: This study demonstrates significant variation in the interpretive reasoning performance of contemporary LLMs. Claude 3.7 Sonnet and Gemini 2.0 Flash outperformed DeepSeek R1, although their performance did not differ significantly from GPT-4o. CLINICAL SIGNIFICANCE: This study demonstrates that model selection and structured prompting critically influence the reliability of AI-assisted endodontic diagnoses, with Gemini 2.0 and Claude 3.7 showing superior reasoning quality, highlighting the need for specialized training and human oversight for safe clinical integration.
引用
- Sahoo HS, Kumar ARP, Durvasulu A, et al. Comparative analysis of diagnostic prediction and clinical reasoning of large language models in complex endodontic case scenarios. J Dent. 2025. PMID: 41435997. doi:10.1016/j.jdent.2025.106309
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。