文献速递口腔修复与种植

人工智能大语言模型在放射影像中识别牙种植体的准确性与幻觉行为

Accuracy and hallucination behaviors of artificial intelligence-based large language models for dental implant identification on radiographs.
PubMed 文献 · J Dent · 2026-05-15

本研究评估了四种人工智能大语言模型(ChatGPT-5、ChatGPT-4o、Gemini 2.5 Pro、Grok 4)在120张标准化放射影像(60张根尖片和60张全景片)上识别六种种植体系统的准确性,主要结局为响应级准确率。结果显示结构特征识别准确率为51.67%至82.5%,而品牌和制造商识别准确率仅为4.17%至32.78%,幻觉是所有模型的主要错误类型。结论认为当前大语言模型在种植体品牌识别方面能力有限,仅可作为需临床医生验证的辅助工具。

要点

  • 大语言模型对种植体品牌和制造商的识别准确率仅4.17%至32.78%。
  • 结构特征(种植体水平和连接类型)识别准确率为51.67%至82.5%。
  • 幻觉是主要错误类型,但ChatGPT-5弃权率较高且幻觉频率较低。

核心结论

  • 当前大语言模型在放射影像中确定性识别牙种植体系统的能力有限,尤其是特定品牌。
  • 近期模型迭代显示出减少幻觉并通过弃权提高可靠性的趋势,但高错误率表明其仅可作为需临床医生验证的辅助工具。

研究设计与样本

  • 回顾性收集120张标准化放射影像(60张根尖片和60张全景片),代表六种种植体系统(Straumann BL、Straumann TL、Osstem TS [Hiossen ET]、Osstem [Hiossen] US、Osstem [Hiossen] SS、Dentium SuperLine)。
  • 对每张影像,使用标准化提示词向四种多模态大语言模型(ChatGPT-5、ChatGPT-4o、Gemini 2.5 Pro、Grok 4)各查询六次,要求识别种植体水平、连接类型、品牌和制造商。
  • 主要结局为响应级准确率;统计使用广义估计方程分析准确率,卡方检验分析错误类型(α=0.05)。

主要发现

  • 结构特征(种植体水平和连接类型)识别准确率为51.67%至82.5%。
  • 品牌和制造商识别准确率持续较低,为4.17%至32.78%。
  • 幻觉是所有模型的主要错误类型(占所有模型错误的>87%)。
  • ChatGPT-5相比其他模型弃权率较高且幻觉频率较低。

关键数值与推荐

  • 响应分类为成功或未成功识别;未成功响应进一步分为确定性错误(幻觉)、试探性错误、模糊或弃权。
  • 准确率比较采用广义估计方程,错误类型比较采用卡方检验,显著性水平α=0.05。

临床怎么用

  • 当前大语言模型输出应经临床医生验证,不能用于确定性放射影像种植体识别。
  • 近期模型通过弃权表达不确定性的趋势有助于更安全的临床决策支持。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: This study aimed to evaluate the accuracy and error patterns of four artificial intelligence-based large language models (LLMs) in identifying dental implant characteristics including implant level, connection type, brand, and manufacturer, on radiographic images. METHODS: In total, 120 standardized radiographic images (60 periapical and 60 panoramic) representing six implant systems (Straumann BL, Straumann TL, Osstem TS [Hiossen ET], Osstem [Hiossen] US, Osstem [Hiossen] SS, and Dentium SuperLine) were retrospectively collected and analyzed. For each image, four multimodal LLMs (ChatGPT-5, ChatGPT-4o, Gemini 2.5 Pro, and Grok 4) were queried six times with a standardized prompt that requested the implant level, connection type, brand, and manufacturer. The primary outcome was response-level accuracy. Responses were classified as successful or unsuccessful identification; unsuccessful responses were further categorized as definitive incorrect (hallucination), tentative incorrect, ambiguous, or abstention. Statistical analysis used generalized estimating equations for accuracy and Chi-square tests for error types (α=0.05). RESULTS: Structural characteristics (implant level and connection type) were identified with accuracies ranging from 51.67% to 82.5%, whereas brand and manufacturer recognition remained consistently poor (4.17%-32.78%). Hallucinations represented the predominant error type (>87% of errors in all models), while ChatGPT-5 showed a comparatively higher rate of abstention and lower hallucination frequency. CONCLUSIONS: Current LLMs have limited capability for the definitive radiographic identification of dental implant systems, particularly for specific brands. Recent model iterations show a promising trend toward reduced hallucinations and increased reliability through abstention; however, their high error rates indicate that they can only be considered as supplementary tools requiring clinician verification. CLINICAL SIGNIFICANCE: Current LLMs are not sufficiently reliable for definitive radiographic implant identification, and their outputs should be interpreted with clinician verification. However, the increasing tendency of recent models to acknowledge uncertainty indicates promising progress toward safer clinical decision support systems.

来源:J Dent,查看原文。版权归原作者所有。

引用

  1. Hlaing NHMM, Park K, Wan Q, et al. Accuracy and hallucination behaviors of artificial intelligence-based large language models for dental implant identification on radiographs. J Dent. 2026. PMID: 42142612. doi:10.1016/j.jdent.2026.106754

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。