文献速递眼底病与葡萄膜炎青光眼角膜与眼表疾病

多模态大语言模型眼科推理能力的双语基准研究

Evaluating reasoning in multimodal large language models for ophthalmology: a bilingual benchmark study using clinical vignettes and imaging.
PubMed 文献 · Br J Ophthalmol · 2026-06-10

该研究在316道中英双语眼科题目(含临床病例与图像)上评估了三种多模态大语言模型(CLM-V、ChatGPT-5、MiniCPM-V 4.5)的准确性与推理质量,并比较启用与禁用推理提示的差异。结果显示,启用推理提示通常使各项评分提高,ChatGPT-5得分最高;但模型在亚专业稳健性和图像解读方面仍存在局限。

要点

  • 启用推理提示可提高模型评分,但改善幅度因模型和数据集而异。
  • ChatGPT-5在人工评价中排名最高,评分者间一致性较高(κ=0.87)。
  • 模型在亚专业稳健性和图像解读上仍有局限,需严格评估推理质量。

核心结论

  • 多模态大语言模型在眼科问答中具有潜力,启用推理提示与可解释性提高相关,并在多数情形下数值评分更高。
  • 模型在亚专业稳健性和图像解读方面存在局限,用于教育与临床前需严格评估推理质量。

研究设计与样本

  • 研究评估三种多模态大语言模型(CLM-V、ChatGPT-5、MiniCPM-V 4.5),使用316道中英双语眼科题目。
  • 题目包括175道英文基础与临床科学课程单选题和141道中文高级职称多选题,覆盖角膜、葡萄膜、青光眼、视网膜和眼眶。
  • 每题由临床病例与图像配对,模型分别在启用和禁用推理提示下测试,采用自动评分量表和专家评审评价推理质量。
  • 对4个病例进行定性分析。

主要发现

  • 启用推理提示后,英文数据集平均人工智能辅助总分:CLM-V从14.97升至16.07,ChatGPT-5从20.77升至23.97,MiniCPM-V 4.5从10.83升至12.60。
  • 中文数据集对应分数:CLM-V从9.03升至10.27,ChatGPT-5从19.95升至22.00,MiniCPM-V 4.5从11.05升至13.30。
  • 人工评价显示评分者间一致性较高(κ=0.87),ChatGPT-5排名最高。
  • 定性病例分析显示,启用推理的输出通常更具临床可解释性,但获益幅度因模型和数据集而异。

关键数值与推荐

  • 自动评分量表维度包括准确性、数据综合、逻辑、选项分析和安全性。
  • 人工评价评分者间一致性κ=0.87。

临床怎么用

  • 在眼科教育与临床辅助场景使用多模态大语言模型时,可考虑启用推理提示以提升输出可解释性。
  • 应结合亚专业与图像解读的稳健性评估,不能仅依据总体准确率判断安全性。

局限性

  • 模型在亚专业稳健性和图像解读方面存在局限,可能影响安全的教育与临床应用。
  • 推理提示的获益幅度因模型和数据集而异,需谨慎外推。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: Large language models (LLMs) excel in text-based medical exams, but their ability to integrate multimodal data, critical for ophthalmology, is underexplored. This study evaluates vision-language LLMs' accuracy and reasoning in complex ophthalmic questions. METHODS: We assessed three multimodal LLMs (CLM-V, ChatGPT-5, MiniCPM-V 4.5) on 316 bilingual ophthalmology questions (175 English Basic and Clinical Science Course single-choice, 141 Chinese senior professional title multiple-choice questions) across cornea, uvea, glaucoma, retina and orbit. Each question paired a clinical vignette with an image. Models were tested with reasoning-enabled and reasoning-disabled prompts. Accuracy was measured against reference standards, and reasoning quality was evaluated using automated rubric scoring (accuracy, data synthesis, logic, option analysis, safety) and expert review. Four cases were analysed qualitatively. RESULTS: Reasoning-enabled prompting increased mean artificial intelligence-assisted total scores in the English dataset from 14.97 to 16.07 for CLM-V, 20.77 to 23.97 for ChatGPT-5 and 10.83 to 12.60 for MiniCPM-V 4.5; in the Chinese dataset, the corresponding scores were 9.03 to 10.27, 19.95 to 22.00 and 11.05 to 13.30, respectively. Human evaluation showed substantial inter-rater agreement (κ=0.87) and again ranked ChatGPT-5 highest. Qualitative case analyses illustrated that reasoning-enabled outputs were often more clinically interpretable, although the magnitude of benefit was model-dependent and dataset-dependent. CONCLUSION: Multimodal LLMs demonstrate potential in ophthalmic question-answering, with reasoning-enabled prompting being associated with improved interpretability and, in most settings, numerically higher performance. However, limitations in subspecialty robustness and image interpretation necessitate rigorous reasoning evaluation for safe educational and clinical applications.

来源:Br J Ophthalmol,查看原文。版权归原作者所有。

引用

  1. Yin H, Zhao K, Shi D, et al. Evaluating reasoning in multimodal large language models for ophthalmology: a bilingual benchmark study using clinical vignettes and imaging. Br J Ophthalmol. 2026. PMID: 42270291. doi:10.1136/bjo-2025-328992

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。