文献速递眼底病与葡萄膜炎神经眼科

人工智能聊天机器人对临床眼科图像解读的准确性

Accuracy of an Artificial Intelligence Chatbot's Interpretation of Clinical Ophthalmic Images.
PubMed 文献 · JAMA Ophthalmol · 2024-01-01

这项横断面研究使用多伦多大学眼科教育平台OCTCases的136个眼科病例(448张图像、429道选择题),评估ChatGPT-4在2023年10月16日至23日回答需多模态图像输入问题的准确性。聊天机器人总体正确回答299题(70%),视网膜类问题正确率77%优于神经眼科的58%,非图像题正确率82%优于图像题65%,结论认为其正确回答约三分之二基于图像解读的选择题且非图像题表现更好。

要点

  • ChatGPT-4在429道眼科病例选择题中正确回答299道(70%)。
  • 视网膜类正确率77% vs 神经眼科58%,差异18%(95%CI 7.5%-29.4%,P<.001)。
  • 非图像题正确率82% vs 图像题65%,差异17%(95%CI 7.8%-25.1%,P<.001)。

核心结论

  • ChatGPT-4准确回答约三分之二基于图像解读的眼科病例选择题,在非图像题上表现更好,作者强调需注意其在医疗场景中的适当整合。

研究设计与样本

  • 横断面研究,使用多伦多大学眼科与视觉科学系OCTCases平台的公开数据集,137个病例中136个(99%)含选择题。
  • 聊天机器人在2023年10月16日至10月23日回答需多模态输入的问题,共分析136个病例的429道选择题和448张图像。

主要发现

  • 聊天机器人总体正确回答299道选择题(70%)。
  • 视网膜类正确率77% vs 神经眼科类58%,差异18%(95%CI 7.5%-29.4%;χ21=11.4;P<.001)。
  • 非图像题正确率82% vs 图像题65%,差异17%(95%CI 7.8%-25.1%;χ21=12.2;P<.001)。
  • 眼肿瘤类正确率72%,小儿眼科类68%,葡萄膜炎类67%,青光眼类61%。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

IMPORTANCE: Ophthalmology is reliant on effective interpretation of multimodal imaging to ensure diagnostic accuracy. The new ability of ChatGPT-4 (OpenAI) to interpret ophthalmic images has not yet been explored. OBJECTIVE: To evaluate the performance of the novel release of an artificial intelligence chatbot that is capable of processing imaging data. DESIGN, SETTING, AND PARTICIPANTS: This cross-sectional study used a publicly available dataset of ophthalmic cases from OCTCases, a medical education platform based out of the Department of Ophthalmology and Vision Sciences at the University of Toronto, with accompanying clinical multimodal imaging and multiple-choice questions. Across 137 available cases, 136 contained multiple-choice questions (99%). EXPOSURES: The chatbot answered questions requiring multimodal input from October 16 to October 23, 2023. MAIN OUTCOMES AND MEASURES: The primary outcome was the accuracy of the chatbot in answering multiple-choice questions pertaining to image recognition in ophthalmic cases, measured as the proportion of correct responses. χ2 Tests were conducted to compare the proportion of correct responses across different ophthalmic subspecialties. RESULTS: A total of 429 multiple-choice questions from 136 ophthalmic cases and 448 images were included in the analysis. The chatbot answered 299 of multiple-choice questions correctly across all cases (70%). The chatbot's performance was better on retina questions than neuro-ophthalmology questions (77% vs 58%; difference = 18%; 95% CI, 7.5%-29.4%; χ21 = 11.4; P < .001). The chatbot achieved a better performance on nonimage-based questions compared with image-based questions (82% vs 65%; difference = 17%; 95% CI, 7.8%-25.1%; χ21 = 12.2; P < .001).The chatbot performed best on questions in the retina category (77% correct) and poorest in the neuro-ophthalmology category (58% correct). The chatbot demonstrated intermediate performance on questions from the ocular oncology (72% correct), pediatric ophthalmology (68% correct), uveitis (67% correct), and glaucoma (61% correct) categories. CONCLUSIONS AND RELEVANCE: In this study, the recent version of the chatbot accurately responded to approximately two-thirds of multiple-choice questions pertaining to ophthalmic cases based on imaging interpretation. The multimodal chatbot performed better on questions that did not rely on the interpretation of imaging modalities. As the use of multimodal chatbots becomes increasingly widespread, it is imperative to stress their appropriate integration within medical contexts.

来源:JAMA Ophthalmol,查看原文。版权归原作者所有。

引用

  1. Mihalache A, Huang RS, Popovic MM, et al. Accuracy of an Artificial Intelligence Chatbot's Interpretation of Clinical Ophthalmic Images. JAMA Ophthalmol. 2024. PMID: 38421670. doi:10.1001/jamaophthalmol.2024.0017

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。