文献速递医学人工智能与信息学组学、生物样本库与精准医学

面向病理学的多模态全切片基础模型

A multimodal whole-slide foundation model for pathology.
PubMed 文献 · Nat Med · 2025-11-05

研究提出基于Transformer的病理图像与文本对齐网络(TITAN),利用335,645张全切片图像经视觉自监督学习和与病理报告及423,122条合成描述进行视觉-语言对齐预训练,无需微调或临床标签即可提取通用切片表征并生成病理报告,在多种机器学习设置下优于ROI和全切片基础模型。

要点

  • TITAN是多模态全切片基础模型,预训练使用335,645张全切片图像。
  • 预训练结合视觉自监督学习与病理报告及423,122条合成描述的视觉-语言对齐。
  • 无需微调或临床标签即可提取通用切片表征并生成病理报告,在多项任务上优于ROI和全切片基础模型。

核心结论

  • TITAN无需微调或临床标签即可提取通用切片表征并生成病理报告,可泛化至罕见病检索和癌症预后等资源有限临床场景。
  • TITAN在多种机器学习设置下优于ROI和全切片基础模型。

研究设计与样本

  • 提出基于Transformer的病理图像与文本对齐网络(TITAN),为多模态全切片基础模型。
  • 预训练使用335,645张全切片图像,通过视觉自监督学习和与病理报告及423,122条合成描述的视觉-语言对齐进行。
  • 合成描述由病理多模态生成式AI助手生成。

主要发现

  • TITAN在机器学习设置(包括线性探测、少样本和零样本分类、罕见癌症检索、跨模态检索和病理报告生成)下优于ROI和全切片基础模型。
  • TITAN无需微调或临床标签即可提取通用切片表征并生成病理报告,泛化至罕见病检索和癌症预后等资源有限临床场景。

局限性

  • 原文未报告局限性

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

The field of computational pathology has been transformed with recent advances in foundation models that encode histopathology region-of-interests (ROIs) into versatile and transferable feature representations via self-supervised learning. However, translating these advancements to address complex clinical challenges at the patient and slide level remains constrained by limited clinical data in disease-specific cohorts, especially for rare clinical conditions. We propose Transformer-based pathology Image and Text Alignment Network (TITAN), a multimodal whole-slide foundation model pretrained using 335,645 whole-slide images via visual self-supervised learning and vision-language alignment with corresponding pathology reports and 423,122 synthetic captions generated from a multimodal generative AI copilot for pathology. Without any fine-tuning or requiring clinical labels, TITAN can extract general-purpose slide representations and generate pathology reports that generalize to resource-limited clinical scenarios such as rare disease retrieval and cancer prognosis. We evaluate TITAN on diverse clinical tasks and find that it outperforms both ROI and slide foundation models across machine learning settings, including linear probing, few-shot and zero-shot classification, rare cancer retrieval, cross-modal retrieval and pathology report generation.

来源:Nat Med,查看原文。版权归原作者所有。

引用

  1. Ding T, Wagner SJ, Song AH, et al. A multimodal whole-slide foundation model for pathology. Nat Med. 2025. PMID: 41193692. doi:10.1038/s41591-025-03982-3

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。