多模态提示元素对GPT-4V在挑战性脑MRI病例中诊断性能的影响
研究纳入60例诊断困难但已证实的脑MRI病例,比较7种提示组合对GPT-4V鉴别诊断准确性的影响。含图像描述、标注、病史和图像的组合准确率最高(180次回答中124次正确,69%);仅图像(2.2%)或仅图像加标注(1.1%)准确率极低。回归分析显示图像描述贡献最大(OR 68.03,P < .001),其次为病史(OR 4.18,P < .001)。
要点
- 含图像描述的提示组合诊断准确率最高,达69%(124/180)。
- 仅图像或仅图像加标注的准确率极低(分别为2.2%和1.1%)。
- 图像描述对诊断准确性的贡献最大(OR 68.03,P < .001)。
核心结论
- 放射学图像发现的文本描述是GPT-4V在脑MRI鉴别诊断中性能的最强贡献因素,其次为病史;仅图像或仅图像加标注的诊断性能极低。
研究设计与样本
- 研究选取60例诊断困难但已证实的脑MRI病例,定义7种提示组合,每组组合对每个病例用基于GPT-4V的LLM搜索引擎(Perplexity AI)进行三次相同查询,采用二分类和数值评分系统评估准确性,使用χ2检验和Kruskal-Wallis检验分析,并用Benjamini-Hochberg法校正错误发现率,回归分析确定各输入元素的贡献。
诊断标准与关键数值
- 含图像(I)、标注(A)、病史(H)和图像描述(D)的提示组合准确率最高,为69%(124/180次回答)。
- 仅图像(I)准确率为2.2%(4/180次回答)。
- 仅图像加标注(I和A)准确率为1.1%(2/180次回答)。
- 图像描述(D)对诊断准确性的效应量OR为68.03(P < .001)。
- 病史(H)对诊断准确性的效应量OR为4.18(P < .001)。
临床应用与阅片要点
- 在脑MRI鉴别诊断中,向GPT-4V提供图像发现的文本描述可显著提高诊断准确性。
- 仅提供图像或图像加标注不足以获得可靠的鉴别诊断性能。
- 提供病史对诊断准确性有中等程度的正向贡献。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
Background Studies have explored the application of multimodal large language models (LLMs) in radiologic differential diagnosis. Yet, how different multimodal input combinations affect diagnostic performance is not well understood. Purpose To evaluate the impact of varying multimodal input elements on the accuracy of OpenAI's GPT-4 with vision (GPT-4V)-based brain MRI differential diagnosis. Materials and Methods Sixty brain MRI cases with a challenging yet verified diagnosis were selected. Seven prompt groups with variations of four input elements (image without modifiers [I], annotation [A], medical history [H], and image description [D]) were defined. For each MRI case and prompt group, three identical queries were performed using an LLM-based search engine (Perplexity AI, powered by GPT-4V). The accuracy of LLM-generated differential diagnoses was rated using a binary and a numeric scoring system and analyzed using a χ2 test and a Kruskal-Wallis test. Results were corrected for false-discovery rate with use of the Benjamini-Hochberg procedure. Regression analyses were performed to determine the contribution of each input element to diagnostic performance. Results The prompt group containing I, A, H, and D as input exhibited the highest diagnostic accuracy (124 of 180 responses [69%]). Significant differences were observed between prompt groups that contained D among their inputs and those that did not. Unannotated (I) (four of 180 responses [2.2%]) or annotated radiologic images alone (I and A) (two of 180 responses [1.1%]) yielded very low diagnostic accuracy. Regression analyses confirmed a large positive effect of D on diagnostic accuracy (odds ratio [OR], 68.03; P < .001), as well as a moderate positive effect of H (OR, 4.18; P < .001). Conclusion The textual description of radiologic image findings was identified as the strongest contributor to the performance of GPT-4V in brain MRI differential diagnosis, followed by the medical history; unannotated or annotated images alone yielded very low diagnostic performance. © RSNA, 2025 Supplemental material is available for this article.
引用
- Schramm S, Preis S, Metz MC, et al. Impact of Multimodal Prompt Elements on Diagnostic Performance of GPT-4V in Challenging Brain MRI Cases. Radiology. 2025. PMID: 39835982. doi:10.1148/radiol.240689
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。