基于医学文献的图像-文本基础模型实现透明医学图像人工智能
研究提出名为MONET(医学概念检索器)的基础模型方法,学习将医学图像与文本连接并对概念存在进行密集评分,以支持数据审计、模型审计和模型解释。MONET基于105,550张皮肤科图像与来自大量医学文献的自然语言描述配对训练,经认证皮肤科医生验证可准确标注图像概念,并与基于先前概念标注临床图像数据集构建的监督模型具有竞争力。研究展示了MONET在整个人工智能系统开发流程中实现透明性,包括构建可解释模型、数据集和模型审计,以及剖析一项人工智能临床试验结果的案例研究。
要点
- MONET基于105,550张皮肤科图像与医学文献文本配对训练,实现图像概念密集评分。
- 经认证皮肤科医生验证,MONET概念标注准确,与监督模型具竞争力。
- MONET支持数据审计、模型审计和模型解释,覆盖AI开发全流程。
核心结论
- MONET可连接医学图像与文本并对概念存在密集评分,支持数据审计、模型审计和模型解释,实现AI系统开发与部署全流程透明。
研究设计与样本
- 基础模型方法研究,训练MONET基于105,550张皮肤科图像与来自大量医学文献的自然语言描述配对。
- 以皮肤科为 demanding 用例,因疾病、肤色和成像模态异质性。
主要发现
- MONET可准确标注皮肤科图像概念,经认证皮肤科医生验证,与基于先前概念标注临床图像数据集构建的监督模型具竞争力。
- MONET支持从构建可解释模型到数据集和模型审计的AI透明性,包括剖析一项AI临床试验结果的案例研究。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Building trustworthy and transparent image-based medical artificial intelligence (AI) systems requires the ability to interrogate data and models at all stages of the development pipeline, from training models to post-deployment monitoring. Ideally, the data and associated AI systems could be described using terms already familiar to physicians, but this requires medical datasets densely annotated with semantically meaningful concepts. In the present study, we present a foundation model approach, named MONET (medical concept retriever), which learns how to connect medical images with text and densely scores images on concept presence to enable important tasks in medical AI development and deployment such as data auditing, model auditing and model interpretation. Dermatology provides a demanding use case for the versatility of MONET, due to the heterogeneity in diseases, skin tones and imaging modalities. We trained MONET based on 105,550 dermatological images paired with natural language descriptions from a large collection of medical literature. MONET can accurately annotate concepts across dermatology images as verified by board-certified dermatologists, competitively with supervised models built on previously concept-annotated dermatology datasets of clinical images. We demonstrate how MONET enables AI transparency across the entire AI system development pipeline, from building inherently interpretable models to dataset and model auditing, including a case study dissecting the results of an AI clinical trial.
引用
- Kim C, Gadgil SU, DeGrave AJ, et al. Transparent medical image AI via an image-text foundation model grounded in medical literature. Nat Med. 2024. PMID: 38627560. doi:10.1038/s41591-024-02887-x
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。