文献速递儿科影像技术与安全

多模态大型语言模型解读儿科放射影像的能力评估

Capability of multimodal large language models to interpret pediatric radiological images.
PubMed 文献 · Pediatr Radiol · 2024-08-12

研究将30例有临床意义的儿科影像病例共90幅图像提交给GPT-4、Gemini 1.5 Pro和Claude 3 Opus进行解读,由住院医师和主治医师独立评估准确性。模型总体正确诊断率为27.8%(25/90,95%CI=19.5-37.8%),部分正确13.3%(12/90,95%CI=2.7-26.4%),错误58.9%(53/90,95%CI=48.6-68.5%),结论认为多模态大型语言模型尚不能解读儿科放射影像。

要点

  • 模型总体正确诊断率为27.8%(25/90,95%CI=19.5-37.8%)。
  • 模型错误诊断率为58.9%(53/90,95%CI=48.6-68.5%)。
  • 作者结论认为多模态大型语言模型尚不能解读儿科放射影像。

核心结论

  • 多模态大型语言模型尚不能解读儿科放射影像。

研究设计与样本

  • 研究收集30例有临床意义的儿科病例共90幅图像,提交给GPT-4、Gemini 1.5 Pro和Claude 3 Opus并附简短病史,由一名住院医师和一名主治医师独立评估AI回答的准确性,95%置信区间采用调整Wald法确定。

诊断标准与关键数值

  • 模型总体正确诊断27.8%(25/90,95%CI=19.5-37.8%)。
  • 模型部分正确13.3%(12/90,95%CI=2.7-26.4%)。
  • 模型错误58.9%(53/90,95%CI=48.6-68.5%)。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

BACKGROUND: There is a dearth of artificial intelligence (AI) development and research dedicated to pediatric radiology. The newest iterations of large language models (LLMs) like ChatGPT can process image and video input in addition to text. They are thus theoretically capable of providing impressions of input radiological images.

OBJECTIVE: To assess the ability of multimodal LLMs to interpret pediatric radiological images.

MATERIALS AND METHODS: Thirty medically significant cases were collected and submitted to GPT-4 (OpenAI, San Francisco, CA), Gemini 1.5 Pro (Google, Mountain View, CA), and Claude 3 Opus (Anthropic, San Francisco, CA) with a short history for a total of 90 images. AI responses were recorded and independently assessed for accuracy by a resident and attending physician. 95% confidence intervals were determined using the adjusted Wald method.

RESULTS: Overall, the models correctly diagnosed 27.8% (25/90) of images (95% CI=19.5-37.8%), were partially correct for 13.3% (12/90) of images (95% CI=2.7-26.4%), and were incorrect for 58.9% (53/90) of images (95% CI=48.6-68.5%).

CONCLUSION: Multimodal LLMs are not yet capable of interpreting pediatric radiological images.

来源:Pediatr Radiol,查看原文。版权归原作者所有。

引用

  1. Reith TP, D'Alessandro DM, D'Alessandro MP, et al. Capability of multimodal large language models to interpret pediatric radiological images. Pediatr Radiol. 2024. PMID: 39133401. doi:10.1007/s00247-024-06025-0

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。