基于跨模态迁移学习的视网膜疾病OCTA分类
本研究评估了使用眼科基础深度学习模型与跨模态迁移学习在有限样本量下对光学相干断层扫描血管成像(OCTA)进行多疾病分类。OCTA-500数据集(n=500受试者)按85%训练/验证与15%测试划分,采用中间融合结合浅层与深层投影。VisionFM和RETFound域特异性基础模型在跨模态迁移学习下分类准确率分别为0.8133±0.0470和0.8000±0.0507,均优于非域特异性ImageNet模型的0.7600±0.0502。显著性图显示域特异性模型注意力定位于解剖相关视网膜结构。
要点
- VisionFM跨模态迁移学习在OCTA多疾病分类中准确率0.8133±0.0470,优于ImageNet的0.7600±0.0502。
- RETFound跨模态迁移学习准确率0.8000±0.0507,同样优于ImageNet的0.7600±0.0521。
- 域特异性基础模型显著性图注意力定位于OCTA浅层和深层投影的解剖相关视网膜结构。
核心结论
- 视网膜基础模型结合跨模态迁移学习可在小样本OCTA数据上实现准确的多类分类。
- 域特异性基础模型性能优于非域特异性模型,显著性分析显示病理注意力定位于解剖相关结构。
研究设计与样本
- 使用OCTA-500数据集(n=500受试者),85%用于五折交叉验证训练/验证,15%为保留测试集。
- OCTA浅层与深层投影采用中间融合,从彩色眼底照相进行迁移学习以克服小样本限制。
- 评估Vision-Transformer-VisionFM和Vision-Transformer-RETFound域特异性基础模型与Vision-Transformer-ImageNet非域特异性模型。
- 性能指标包括准确率、F1分数、精确率、召回率和受试者工作特征曲线下面积,并计算显著性图。
主要发现
- VisionFM跨模态迁移学习分类准确率0.8133±0.0470,优于ImageNet的0.7600±0.0502。
- RETFound跨模态迁移学习分类准确率0.8000±0.0507,优于ImageNet的0.7600±0.0521。
- 其他性能指标得到相似结论。
- VisionFM和RETFound显著性图注意力定位于OCTA浅层和深层投影的相关视网膜结构,优于ImageNet模型。
临床怎么用
- 在OCTA样本量有限时,可考虑采用眼科域特异性基础模型结合跨模态迁移学习进行多疾病分类。
- 模型显著性图有助于定位病理至解剖相关视网膜结构,辅助临床解读。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
AIMS: This study evaluated the use of ophthalmic foundation deep-learning models with cross-modal transfer learning to classify multiple diseases on optical coherence tomography angiography (OCTA) with limited sample size. METHODS: The OCTA-500 dataset (n=500 subjects) was split into an 85% training/validation set for fivefold cross-validation and a 15% held-out test set. Superficial and deep projections from OCTA were combined using intermediate fusion. Outcomes were multi-disease classification of normal, diabetic retinopathy, age-related macular degeneration and 'other'. Transfer-learning from colour fundus photography was used to overcome the limitation of small sample sizes. Vision-Transformer-VisionFM and Vision-Transformer-RETFound domain-specific foundation models with cross-modal transfer learning were evaluated. Comparison was made with Vision-Transformer-ImageNet, a non-domain-specific model. Performance was evaluated using accuracy, F1-score, precision, recall and area under the receiver operating characteristic curve. Saliency maps were also computed. RESULTS: VisionFM with cross-modal transfer learning outperformed ImageNet in classifying different diseases on OCTA (accuracy: 0.8133±0.0470 vs 0.7600±0.0502). RETFound with cross-modal transfer learning outperformed ImageNet in classifying different diseases on OCTA (accuracy: 0.8000±0.0507 vs 0.7600±0.0521). Similar conclusions were reached with other performance metrics. Saliency maps from VisionFM and RETFound yielded attention patterns that localised pathology to relevant retinal structures on superficial and deep projections from OCTA, comparing favourably with those from ImageNet models. CONCLUSIONS: Retinal foundation models with cross-modal transfer learning enable accurate multi-class classification using OCTA data, which had small sample size. Results from domain-specific foundation models compared favourably with a non-domain-specific model. Saliency analysis showed attention patterns of pathology localised to anatomically relevant retinal structures.
引用
- Shah P, Farah HA, Wisotsky DJ, et al. Classification of retinal diseases based on optical coherence tomography angiography using cross-modal transfer learning of domain-specific foundation AI models. Br J Ophthalmol. 2026. PMID: 42086316. doi:10.1136/bjo-2025-329249
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。