文献速递乳腺技术与安全乳腺X线

基于定量评估的显著性可解释人工智能方法在深度学习乳腺X线摄影分析中的表现

Quantitative evaluation of Saliency-Based Explainable artificial intelligence (XAI) methods in Deep Learning-Based mammogram analysis.
PubMed 文献 · Eur J Radiol · 2024-02-05

本研究在三个中心共1496例(癌症阳性与阴性扫描)平衡乳腺X线数据集上,由三名放射科医生绘制真值框,用改良预训练深度学习模型检测乳腺癌,并以Pointing Game定量评估Grad-CAM、Grad-CAM++和Eigen-CAM三种显著性可解释AI方法。测试集中模型识别癌症的召回率69%、精确率88%、准确率80%、F1分数0.77;Grad-CAM、Grad-CAM++和Eigen-CAM的Pointing Game评分在癌症女性中分别为0.41、0.30、0.35,仅考虑真阳性样本时略升至0.41、0.31、0.36。作者认为显著性方法虽提供一定可解释性,但在相当多情况下难以描绘深度学习模型的决策依据。

要点

  • 在癌症女性中Grad-CAM的Pointing Game评分最高(0.41),高于Eigen-CAM的0.35和Grad-CAM++的0.30。
  • 仅纳入真阳性样本时三种方法评分仅边际升高:Grad-CAM 0.41、Grad-CAM++ 0.31、Eigen-CAM 0.36。
  • 测试集深度学习模型检测癌症的召回率69%、精确率88%、准确率80%、F1分数0.77。

核心结论

  • 基于显著性的方法虽提供一定程度的可解释性,但在相当多情况下难以描绘深度学习模型作出决策的依据。

研究设计与样本

  • 研究为定量评估显著性可解释AI方法的性能,采用平衡乳腺X线数据集(癌症阳性与阴性扫描共1496例),来自三个中心,由三名放射科医生绘制真值框。
  • 采用改良的预训练深度学习模型进行乳腺癌检测,使用MLO和CC图像。
  • 数据集划分为开发样本2244名女性(75%)和测试集748名女性(25%),用于无偏的可解释AI评估。
  • 评估的显著性可解释AI方法包括Grad-CAM、Grad-CAM++和Eigen-CAM,采用Pointing Game评估显著性图最大值是否与边界框对齐。

诊断标准与关键数值

  • Pointing Game定义为显著性图最大值是否与代表病灶的边界框对齐,即所有癌症患者中正确识别病灶的比例,取值0至1。
  • 测试集深度学习模型识别癌症的召回率69%、精确率88%、准确率80%、F1分数0.77。
  • 癌症女性中Grad-CAM、Grad-CAM++、Eigen-CAM的Pointing Game评分分别为0.41、0.30、0.35。
  • 仅考虑真阳性样本时,Grad-CAM、Grad-CAM++、Eigen-CAM的Pointing Game评分分别为0.41、0.31、0.36。

临床应用与阅片要点

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

BACKGROUND: Explainable Artificial Intelligence (XAI) is prominent in the diagnostics of opaque deep learning (DL) models, especially in medical imaging. Saliency methods are commonly used, yet there's a lack of quantitative evidence regarding their performance.

OBJECTIVES: To quantitatively evaluate the performance of widely utilized saliency XAI methods in the task of breast cancer detection on mammograms.

METHODS: Three radiologists drew ground-truth boxes on a balanced mammogram dataset of women (n = 1496 cancer-positive and negative scans) from three centers. A modified, pre-trained DL model was employed for breast cancer detection, using MLO and CC images. Saliency XAI methods, including Gradient-weighted Class Activation Mapping (Grad-CAM), Grad-CAM++, and Eigen-CAM, were evaluated. We utilized the Pointing Game to assess these methods, determining if the maximum value of a saliency map aligned with the bounding boxes, representing the ratio of correctly identified lesions among all cancer patients, with a value ranging from 0 to 1.

RESULTS: The development sample included 2,244 women (75%), with the remaining 748 women (25%) in the testing set for unbiased XAI evaluation. The model's recall, precision, accuracy, and F1-Score in identifying cancer in the testing set were 69%, 88%, 80%, and 0.77, respectively. The Pointing Game Scores for Grad-CAM, Grad-CAM++, and Eigen-CAM were 0.41, 0.30, and 0.35 in women with cancer and marginally increased to 0.41, 0.31, and 0.36 when considering only true-positive samples.

CONCLUSIONS: While saliency-based methods provide some degree of explainability, they frequently fall short in delineating how DL models arrive at decisions in a considerable number of instances.

来源:Eur J Radiol,查看原文。版权归原作者所有。

引用

  1. Cerekci E, Alis D, Denizoglu N, et al. Quantitative evaluation of Saliency-Based Explainable artificial intelligence (XAI) methods in Deep Learning-Based mammogram analysis. Eur J Radiol. 2024. PMID: 38364587. doi:10.1016/j.ejrad.2024.111356

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。