文献速递艾滋病与性传播感染新发突发、热带与寄生虫病

商业大语言模型对肛门生殖器皮损图像的诊断准确性比较

Diagnostic Accuracy of Commercial Large Language Models for Anogenital Skin Lesion Images: A Comparative Study of Gemini, Claude, and ChatGPT.
PubMed 文献 · J Infect Dis · 2026-01-01

Gemini表现最佳,STI二分类准确率为76.2%(95%CI 70.5%–81.9%),top-1诊断准确率仅39.0%(95%CI 32.7%–45.7%);所有模型对挑战性图像准确性显著降低,且均未能正确识别任何猴痘病例。

要点

  • Gemini对STI二分类准确率76.2%,高于ChatGPT和Claude。
  • Gemini top-1、top-3、top-5诊断准确率分别为39.0%、54.6%、60.6%。
  • 所有模型对挑战性图像top-5准确率仅29.2%–40.0%,且均未识别猴痘。

核心结论

  • 当前大语言模型对肛门生殖器皮损的诊断准确性有限,最佳模型Gemini的top-1准确率仅39.0%,不能可靠用于临床诊断。

主要发现

  • 在218张图像中,Gemini的STI二分类准确率最高,为76.2%(95%CI 70.5%–81.9%)。
  • Gemini的鉴别诊断top-1、top-3、top-5准确率分别为39.0%(95%CI 32.7%–45.7%)、54.6%(95%CI 47.9%–61.1%)、60.6%(95%CI 53.9%–66.9%)。
  • 在亚组分析中,所有模型对诊断挑战性图像的top-5准确率范围为29.2%–40.0%。
  • Gemini在多数亚组中优于Claude(P < .05)。
  • 所有模型均未能正确识别任何猴痘病例。

临床怎么用

  • 当前大语言模型或可在监督下辅助临床分诊,但常规临床使用前需进一步验证。

局限性

  • 未来研究需将大语言模型与临床医生比较,并探索其辅助临床诊断的方式。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: Diagnosing anogenital dermatological conditions often requires specialist expertise that is unavailable in many clinical settings. Large language models (LLMs) are increasingly accessible to clinicians, but their diagnostic accuracy for anogenital dermatology has not been evaluated. We evaluated the diagnostic accuracy of 3 LLMs (Gemini 2.5 Pro, Claude Opus 4.1, and ChatGPT 5 Thinking) for anogenital dermatological conditions. METHODS: This study was conducted between September and November 2025, using deidentified clinical images of anogenital conditions from the STI Atlas (stiatlas.org) and other publicly available sources. Primary outcomes were correct classification of images identified as sexually transmitted infections (STIs) vs non-STIs and the inclusion of the correct diagnosis among the LLMs' top-ranked (top-1), top-3, or top-5 differential diagnoses. RESULTS: Among 218 images, Gemini achieved the highest accuracy for STI binary classification (76.2% [95% CI, 70.5%-81.9%]) and differential diagnosis (top-1, 39.0% [95% CI, 32.7%-45.7%]; top-3, 54.6% [95% CI, 47.9%-61.1%]; top-5, 60.6% [95% CI, 53.9%-66.9%]), followed by ChatGPT and Claude. In subgroup analysis, all LLMs showed substantially reduced accuracy for diagnostically challenging images (top-5 accuracy range, 29.2%-40.0%). Gemini consistently outperformed Claude across most subgroups (P < .05). None of the LLMs could identify any mpox correctly. CONCLUSIONS: LLMs showed limited accuracy for diagnosing anogenital dermatological conditions, particularly for challenging images. The best-performing model (Gemini) achieved only 39.0% for top-1 diagnosis, indicating that current LLMs cannot reliably diagnose anogenital conditions. These tools may support supervised clinical triage but need further validation before routine clinical use. Future studies should compare LLMs with clinicians and explore how they can assist clinical diagnosis.

来源:J Infect Dis,查看原文。版权归原作者所有。

引用

  1. Soe NN, Latt PM, Lee D, et al. Diagnostic Accuracy of Commercial Large Language Models for Anogenital Skin Lesion Images: A Comparative Study of Gemini, Claude, and ChatGPT. J Infect Dis. 2026. PMID: 42105020. doi:10.1093/infdis/jiag258

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。