文献速递技术与安全CTMRDR

多模态大语言模型在医学影像中的应用:现状与未来方向

Multimodal Large Language Models in Medical Imaging: Current State and Future Directions.
PubMed 文献 · Korean J Radiol · 2025-01-01

本文综述多模态大语言模型(MLLMs)在医学影像中的应用现状,其整合大语言模型与多种模态数据(临床信息、文本及2D胸片至3D CT/MRI),已用于自动生成初步放射报告、视觉问答和交互式诊断支持。但大规模高质量多模态数据集稀缺、幻觉发现风险、决策不透明及高算力需求阻碍临床推广,未来需聚焦区域锚定推理、医学基础模型及安全有效整合策略。

要点

  • MLLMs可整合临床文本与2D胸片至3D CT/MRI等多模态影像数据。
  • 当前应用包括自动生成初步放射报告、视觉问答和交互式诊断支持。
  • 临床推广面临数据稀缺、幻觉发现、决策不透明和高算力需求等挑战。

核心结论

  • MLLMs有望成为临床医生可信的AI伙伴,但需解决数据、幻觉、透明度和算力等挑战,未来研究方向包括区域锚定推理、医学基础模型及临床整合策略。

研究设计与样本

  • 综述,总结MLLMs在医学尤其是放射学中的当前能力与局限,并概述未来研究方向。

局限性

  • MLLMs需要大规模高质量多模态数据集,而医学领域此类数据稀缺。
  • 存在幻觉发现的风险。
  • 决策过程缺乏透明度。
  • 计算需求高。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

Multimodal large language models (MLLMs) are emerging as powerful tools in medicine, particularly in radiology, with the potential to serve as trusted artificial intelligence (AI) partners for clinicians. In radiology, these models integrate large language models (LLMs) with diverse multimodal data sources by combining clinical information and text with radiologic images of various modalities, ranging from 2D chest X-rays to 3D CT/MRI. Methods for achieving this multimodal integration are rapidly evolving, and the high performance of freely available LLMs may further accelerate MLLM development. Current applications of MLLMs now span automatic generation of preliminary radiology report, visual question answering, and interactive diagnostic support. Despite these promising capabilities, several significant challenges hinder widespread clinical adoption. MLLMs require access to large-scale, high-quality multimodal datasets, which are scarce in the medical domain. Risks of hallucinated findings, lack of transparency in decision-making processes, and high computational demands further complicate implementation. This review summarizes the current capabilities and limitations of MLLMs in medicine-particularly in radiology-and outlines key directions for future research. Critical areas include incorporating region-grounded reasoning to link model outputs to specific image regions, developing robust foundation models pre-trained on large-scale medical datasets, and establishing strategies for the safe and effective integration of MLLMs into clinical practice.

来源:Korean J Radiol,查看原文。版权归原作者所有。

引用

  1. Nam Y, Kim DY, Kyung S, et al. Multimodal Large Language Models in Medical Imaging: Current State and Future Directions. Korean J Radiol. 2025. PMID: 41015856. doi:10.3348/kjr.2025.0599

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。