更进一步深入黑箱:一项关于如何在二维超声乳腺结节评估的AI决策系统中建立更多信心的初步研究
研究采用DenseNet-121对367名女性的785张二维乳腺超声图像进行良恶性分类,比较粗ROI与细ROI两种输入的性能。粗ROI模型AUC为0.899,细ROI模型AUC为0.869;粗ROI模型准确率、敏感度、特异度分别为88.4%、87.9%、89.2%,细ROI模型分别为86.1%、87.9%、83.8%。研究提出基于证据区域(ROE)的度量系统,以帮助理解AI决策。
要点
- 粗ROI模型AUC 0.899,细ROI模型AUC 0.869,粗ROI模型略优。
- 粗ROI模型准确率88.4%、敏感度87.9%、特异度89.2%。
- 细ROI模型准确率86.1%、敏感度87.9%、特异度83.8%。
关键影像表现/征象
- 深度学习模型捕获的ROE与医生评估图像时考虑的区域高度相似。
- 粗ROI与细ROI的结果一致,周边组织也是乳腺病变分类的影响因素。
诊断标准与关键数值
- 粗ROI模型:AUC 0.899,准确率88.4%,敏感度87.9%,特异度89.2%。
- 细ROI模型:AUC 0.869,准确率86.1%,敏感度87.9%,特异度83.8%。
- 根据这些评估指标,粗ROI模型略优于细ROI模型。
- 报告了测试集中真阳性、假阳性、真阴性、假阴性结果按高、中、低相似度的细分。
临床应用与阅片要点
- 作者提出ROE度量系统可帮助医生和患者更好理解AI如何在阅片中做出决策,可能整合为乳腺超声检查早期筛查或分诊的证据部分。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVES: To investigate how a DL model makes decisions in lesion classification with a newly defined region of evidence (ROE) by incorporating "explainable AI" (xAI) techniques.
METHODS: A data set of 785 2D breast ultrasound images acquired from 367 females. The DenseNet-121 was used to classify whether the lesion is benign or malignant. For performance assessment, classification results are evaluated by calculating accuracy, sensitivity, specificity, and receiver operating characteristic for experiments of both coarse and fine regions of interest (ROIs). The area under the curve (AUC) was evaluated, and the true-positive, false-positive, true-negative, and false-negative results with breakdown in high, medium, and low resemblance on test sets were also reported.
RESULTS: The two models with coarse and fine ROIs of ultrasound images as input achieve an AUC of 0.899 and 0.869, respectively. The accuracy, sensitivity, and specificity of the model with coarse ROIs are 88.4%, 87.9%, and 89.2%, and with fine ROIs are 86.1%, 87.9%, and 83.8%, respectively. The DL model captures ROE with high resemblance of physicians' consideration as they assess the image.
CONCLUSIONS: We have demonstrated the effectiveness of using DenseNet to classify breast lesions with limited quantity of 2D grayscale ultrasound image data. We have also proposed a new ROE-based metric system that can help physicians and patients better understand how AI makes decisions in reading images, which can potentially be integrated as a part of evidence in early screening or triaging of patients undergoing breast ultrasound examinations.
KEY POINTS: • The two models with coarse and fine ROIs of ultrasound images as input achieve an AUC of 0.899 and 0.869, respectively. The accuracy, sensitivity, and specificity of the model with coarse ROIs are 88.4%, 87.9%, and 89.2%, and with fine ROIs are 86.1%, 87.9%, and 83.8%, respectively. • The first model with coarse ROIs is slightly better than the second model with fine ROIs according to these evaluation metrics. • The results from coarse ROI and fine ROI are consistent and the peripheral tissue is also an impact factor in breast lesion classification.
引用
- Dong F, She R, Cui C, et al. One step further into the blackbox: a pilot study of how to build more confidence around an AI-based decision system of breast nodule assessment in 2D ultrasound. Eur Radiol. 2021. PMID: 33404698. doi:10.1007/s00330-020-07561-7
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。