文献速递腹部技术与安全CT

一种基于迁移学习的新型CLIP模型联合自注意力机制用于鉴别胰腺导管腺癌的肿瘤-间质比

One novel transfer learning-based CLIP model combined with self-attention mechanism for differentiating the tumor-stroma ratio in pancreatic ductal adenocarcinoma.
PubMed 文献 · Radiol Med · 2024-10-16

本研究回顾性纳入3家医院的207例胰腺导管腺癌患者,构建基于迁移学习和自注意力机制的CLIP-adapter模型,在术前增强CT图像上预测肿瘤-间质比。多期相特征融合的CLIP-adapter模型优于任一单期相,且优于传统影像组学和深度学习模型,其中CLIP-adapter_ViT_Base_32在测试集AUC最高(0.978)、准确率最高(0.921)。低肿瘤-间质比患者总生存期较长。

要点

  • CLIP-adapter模型在测试集预测肿瘤-间质比的AUC达0.978、准确率0.921。
  • 多期相(动脉期+静脉期)特征融合优于任一单期相。
  • 低肿瘤-间质比患者总生存期较高肿瘤-间质比患者长。

核心结论

  • CLIP-adapter模型为预测胰腺导管腺癌肿瘤-间质比提供了一种安全准确的方法,基于多模态(图像与文本)和多期相(动脉期与静脉期)的特征融合模块显著提升模型性能。

研究设计与样本

  • 回顾性研究,纳入3家医院共207例胰腺导管腺癌患者,以手术标本病理学家评估的肿瘤-间质比为参考标准分为高肿瘤-间质比组和低肿瘤-间质比组,基于术前增强CT构建CLIP-adapter模型并与临床变量、传统影像组学模型及深度学习模型(ResNet50、ResNet101、ViT_Base_32、ViT_Base_16)比较。

诊断标准与关键数值

  • 基于多期相特征融合的CLIP-adapter模型优于基于任一单期相(动脉期或静脉期)的模型;CLIP-adapter模型优于传统影像组学模型和深度学习模型,CLIP-adapter_ViT_Base_32表现最佳,测试集AUC为0.978、准确率为0.921。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

PURPOSE: To develop a contrastive language-image pretraining (CLIP) model based on transfer learning and combined with self-attention mechanism to predict the tumor-stroma ratio (TSR) in pancreatic ductal adenocarcinoma on preoperative enhanced CT images, in order to understand the biological characteristics of tumors for risk stratification and guiding feature fusion during artificial intelligence-based model representation.

MATERIAL AND METHODS: This retrospective study collected a total of 207 PDAC patients from three hospitals. TSR assessments were performed on surgical specimens by pathologists and divided into high TSR and low TSR groups. This study developed one novel CLIP-adapter model that integrates the CLIP paradigm with a self-attention mechanism for better utilizing features from multi-phase imaging, thereby enhancing the accuracy and reliability of tumor-stroma ratio predictions. Additionally, clinical variables, traditional radiomics model and deep learning models (ResNet50, ResNet101, ViT_Base_32, ViT_Base_16) were constructed for comparison.

RESULTS: The models showed significant efficacy in predicting TSR in PDAC. The performance of the CLIP-adapter model based on multi-phase feature fusion was superior to that based on any single phase (arterial or venous phase). The CLIP-adapter model outperformed traditional radiomics models and deep learning models, with CLIP-adapter_ViT_Base_32 performing the best, achieving the highest AUC (0.978) and accuracy (0.921) in the test set. Kaplan-Meier survival analysis showed longer overall survival in patients with low TSR compared to those with high TSR.

CONCLUSION: The CLIP-adapter model designed in this study provides a safe and accurate method for predicting the TSR in PDAC. The feature fusion module based on multi-modal (image and text) and multi-phase (arterial and venous phase) significantly improves model performance.

来源:Radiol Med,查看原文。版权归原作者所有。

引用

  1. Liao H, Yuan J, Liu C, et al. One novel transfer learning-based CLIP model combined with self-attention mechanism for differentiating the tumor-stroma ratio in pancreatic ductal adenocarcinoma. Radiol Med. 2024. PMID: 39412688. doi:10.1007/s11547-024-01902-y

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。