多模态生成式人工智能在真实视网膜门诊环境中的应用
本研究在Bascom Palmer眼科诊所回顾性分析2010年1月至2023年3月患者数据,评估GPT-4V在256只眼(143例患者)中诊断玻璃体视网膜疾病及ICD-10编码的表现。GPT-4V在开放式问题中准确率13.7%、多项选择题31.3%,ICD-10编码准确率分别为5.5%和31.3%;简单病例组与复杂病例组间无显著差异。结论认为GPT-4V在标准化问题下具潜力,但开放式场景效果有限。
要点
- GPT-4V在256只眼中开放式问题诊断准确率13.7%,多项选择题31.3%。
- ICD-10编码准确率在开放式问题为5.5%,多项选择题为31.3%。
- 简单病例组与复杂病例组间诊断或编码准确率无显著差异。
核心结论
- GPT-4V在临床诊疗和病历记录方面具潜力,尤其在标准化问题下。
- GPT-4V在开放式场景中的效果有限,表明在提供复杂医疗建议方面存在显著局限。
研究设计与样本
- 回顾性横断面研究,于Bascom Palmer眼科诊所分析2010年1月至2023年3月患者数据。
- 评估GPT-4V在视网膜图像分析及ICD-10编码中的表现,分为简单病例组(A组)和需更深入分析的复杂病例组(B组)。
- 诊断准确性通过开放式问题和多项选择题评估,由三位视网膜专家独立核实。
- 共纳入143例患者的256只眼。
主要发现
- GPT-4V在开放式问题中准确率为13.7%,在多项选择题中准确率为31.3%。
- ICD-10编码准确率在开放式问题中为5.5%,在多项选择题中为31.3%。
- GPT-4V能准确诊断玻璃体后脱离、非渗出性年龄相关性黄斑变性和视网膜脱离。
- ICD-10编码在开放式问题中对非渗出性年龄相关性黄斑变性、视网膜中央静脉阻塞和黄斑裂孔最准确;在多项选择题中对玻璃体后脱离、非渗出性年龄相关性黄斑变性和视网膜脱离最准确。
- A组与B组在诊断或编码准确率上无显著差异。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
PURPOSE: This study evaluates a large language model, Generative Pre-trained Transformer 4 with vision, for diagnosing vitreoretinal diseases in real-world ophthalmology settings. METHODS: A retrospective cross-sectional study at Bascom Palmer Eye Clinic, analyzing patient data from January 2010 to March 2023, assesses Generative Pre-trained Transformer 4 with vision's performance on retinal image analysis and International Classification of Diseases 10th revision coding across 2 patient groups: simpler cases (Group A) and complex cases (Group B) requiring more in-depth analysis. Diagnostic accuracy was assessed through open-ended questions and multiple-choice questions independently verified by three retina specialists. RESULTS: In 256 eyes from 143 patients, Generative Pre-trained Transformer 4-V demonstrated a 13.7% accuracy for open-ended questions and 31.3% for multiple-choice questions, with International Classification of Diseases 10th revision code accuracies at 5.5% and 31.3%, respectively. Accurately diagnosed posterior vitreous detachment, nonexudative age-related macular degeneration, and retinal detachment. International Classification of Diseases 10th revision coding was most accurate for nonexudative age-related macular degeneration, central retinal vein occlusion, and macular hole in OEQs, and for posterior vitreous detachment, nonexudative age-related macular degeneration, and retinal detachment in multiple-choice questions. No significant difference in diagnostic or coding accuracy was found in Groups A and B. CONCLUSION: Generative Pre-trained Transformer 4 with vision has potential in clinical care and record keeping, particularly with standardized questions. Its effectiveness in open-ended scenarios is limited, indicating a significant limitation in providing complex medical advice.
引用
- Ghalibafan S, Taylor Gonzalez DJ, Cai LZ, et al. APPLICATIONS OF MULTIMODAL GENERATIVE ARTIFICIAL INTELLIGENCE IN A REAL-WORLD RETINA CLINIC SETTING. Retina. 2024. PMID: 39287535. doi:10.1097/IAE.0000000000004204
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。