使用大语言模型辅助的多模态学习对放射治疗靶区进行自动勾画
本研究开发了视觉语言模型Medformer,将靶区勾画建模为临床决策问题,利用大语言模型提取文本特征并与视觉特征融合。在院内前列腺癌数据集和公开口咽癌数据集共668例受试者上评估,勾画大体肿瘤体积时Dice相似系数等指标优于传统方法,勾画临床靶区体积时与其他先进算法相当。
要点
- Medformer在前列腺癌数据集上勾画大体肿瘤体积的Dice相似系数为0.81±0.10,传统方法为0.72±0.10。
- 在口咽癌数据集上,Medformer勾画大体肿瘤体积的Dice相似系数为0.77±0.11,传统方法为0.72±0.09,差异有统计学意义(P<0.05)。
- 勾画临床靶区体积时,Medformer的Dice相似系数为0.91±0.04,与其他先进算法相当。
核心结论
- 基于多模态学习的治疗靶区自动勾画优于仅依赖视觉特征的传统方法。
- 作者认为该方法可被采纳到常规实践中,以快速勾画临床靶区体积/大体肿瘤体积。
研究设计与样本
- 开发了名为Medformer的视觉语言模型,以分层视觉Transformer为骨干,结合大语言模型提取文本特征,并通过视觉语言注意力模块将语言特征融入视觉特征进行语言感知的视觉编码。
- 采用Dice相似系数(DSC)、交并比(IOU)和第95百分位Hausdorff距离(HD95)定量评估模型性能。
- 评估在院内前列腺癌数据集和公开口咽癌数据集上进行,共计668例受试者。
主要发现
- 在前列腺癌数据集上勾画大体肿瘤体积,Medformer的DSC为0.81±0.10,对照方法为0.72±0.10;IOU为0.73±0.12,对照方法为0.65±0.09;HD95为9.86±9.77 mm,对照方法为19.13±12.96 mm。
- 在口咽癌数据集上勾画大体肿瘤体积,Medformer的DSC为0.77±0.11,对照方法为0.72±0.09;IOU为0.70±0.09,对照方法为0.65±0.07;HD95为7.52±4.8 mm,对照方法为13.63±7.13 mm;上述改善均具有统计学意义(P<0.05)。
- 勾画临床靶区体积时,Medformer的DSC为0.91±0.04,IOU为0.85±0.05,HD95为2.98±1.60 mm,与其他先进算法相当。
临床怎么用
- 作者认为,基于多模态学习的靶区自动勾画方法可被采纳到常规实践中,以快速勾画临床靶区体积/大体肿瘤体积。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
PURPOSE: Artificial intelligence-aided methods have made significant progress in the auto-delineation of normal tissues. However, these approaches struggle with the auto-contouring of radiation therapy target volume. Our goal was to model the delineation of target volume as a clinical decision-making problem, resolved by leveraging large language model-aided multimodal learning approaches. METHODS AND MATERIALS: A vision-language model, termed Medformer, has been developed, employing the hierarchical vision transformer as its backbone and incorporating large language models to extract text-rich features. The contextually embedded linguistic features are seamlessly integrated into visual features for language-aware visual encoding through the visual language attention module. Metrics, including Dice similarity coefficient (DSC), intersection over union (IOU), and 95th percentile Hausdorff distance (HD95), were used to quantitatively evaluate the performance of our model. The evaluation was conducted on an in-house prostate cancer data set and a public oropharyngeal carcinoma data set, totaling 668 subjects. RESULTS: Our Medformer achieved a DSC of 0.81 ± 0.10 versus 0.72 ± 0.10, IOU of 0.73 ± 0.12 versus 0.65 ± 0.09, and HD95 of 9.86 ± 9.77 mm versus 19.13 ± 12.96 mm for delineation of gross tumor volume on the prostate cancer dataset. Similarly, on the oropharyngeal carcinoma dataset, it achieved a DSC of 0.77 ± 0.11 versus 0.72 ± 0.09, IOU of 0.70 ± 0.09 versus 0.65 ± 0.07, and HD95 of 7.52 ± 4.8 mm versus 13.63 ± 7.13 mm, representing significant improvements (P < 0.05). For delineating the clinical target volume, Medformer achieved a DSC of 0.91 ± 0.04, IOU of 0.85 ± 0.05, and HD95 of 2.98 ± 1.60 mm, comparable with other state-of-the-art algorithms. CONCLUSIONS: Auto-delineation of the treatment target based on multimodal learning outperforms conventional approaches that rely purely on visual features. Our method could be adopted into routine practice to rapidly contour clinical target volume/gross tumor volume.
引用
- Rajendran P, Chen Y, Qiu L, et al. Autodelineation of Treatment Target Volume for Radiation Therapy Using Large Language Model-Aided Multimodal Learning. Int J Radiat Oncol Biol Phys. 2024. PMID: 39117164. doi:10.1016/j.ijrobp.2024.07.2149
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。