深度学习看到了什么?从训练用于预测CT图像对比增强期的分类器中获得的启示
本研究开发了一个从全切片CT数据中识别对比增强期的深度学习分类器,并利用该分类器作为可解释系统,评估类激活图、梯度加权类激活图、显著性图、引导反向传播图和新型显著性激活图在识别模型预测所用图像特征方面的效用。结果显示所有技术均能识别分类器使用的体素,且显著性激活图在识别模型预测所用体素方面具有高于引导反向传播图、类激活图和梯度加权类激活图的特异性。
要点
- 开发了从全切片CT数据识别对比增强期的深度学习分类器,作为可解释系统。
- 所有可视化技术均能识别分类器预测所使用的体素。
- 显著性激活图在识别模型预测所用体素方面特异性高于引导反向传播图、类激活图和梯度加权类激活图。
核心结论
- 体素级可视化及激活浅层网络层的影像特征可视化是识别深度学习模型预测所用特征的有力技术。
研究设计与样本
- 研究开发了一个从全切片CT数据识别对比增强期的深度学习分类器,并作为可解释系统评估多种可视化技术。
局限性
- 原文未报告局限性
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVE: Deep learning has shown great promise for improving medical image classification tasks. However, knowing what aspects of an image the deep learning system uses or, in a manner of speaking, sees to make its prediction is difficult.
MATERIALS AND METHODS: Within a radiologic imaging context, we investigated the utility of methods designed to identify features within images on which deep learning activates. In this study, we developed a classifier to identify contrast enhancement phase from whole-slice CT data. We then used this classifier as an easily interpretable system to explore the utility of class activation map (CAMs), gradient-weighted class activation maps (Grad-CAMs), saliency maps, guided backpropagation maps, and the saliency activation map, a novel map reported here, to identify image features the model used when performing prediction.
RESULTS: All techniques identified voxels within imaging that the classifier used. SAMs had greater specificity than did guided backpropagation maps, CAMs, and Grad-CAMs at identifying voxels within imaging that the model used to perform prediction. At shallow network layers, SAMs had greater specificity than Grad-CAMs at identifying input voxels that the layers within the model used to perform prediction.
CONCLUSION: As a whole, voxel-level visualizations and visualizations of the imaging features that activate shallow network layers are powerful techniques to identify features that deep learning models use when performing prediction.
引用
- Philbrick KA, Yoshida K, Inoue D, et al. What Does Deep Learning See? Insights From a Classifier Trained to Predict Contrast Enhancement Phase From CT Images. AJR Am J Roentgenol. 2018. PMID: 30403527. doi:10.2214/AJR.18.20331
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。