文献速递乳腺技术与安全乳腺X线

让AI保持正轨:乳腺X线摄影中算法更新的定期监测

Keeping AI on Track: Regular monitoring of algorithmic updates in mammography.
PubMed 文献 · Eur J Radiol · 2025-04-09

本研究利用PERFORMS外部质量保证方案,回顾性比较同一商业AI模型两个连续版本与人类阅片者在乳腺X线筛查中的诊断性能。结果显示AI V1与V2的AUC无显著差异(0.93 vs 0.94,p=0.13),V2敏感度(88.7%)优于人类(83.2%,p=0.04),两版AI特异度均高于人类(87.4%、88.2% vs 79.0%,p<0.01)。

要点

  • AI V2与V1的AUC无显著差异(0.94 vs 0.93,p=0.13)。
  • AI V2敏感度88.7%高于人类83.2%(p=0.04),V1与人类无差异(87.5% vs 83.2%,p=0.12)。
  • AI V1和V2特异度分别为87.4%和88.2%,均高于人类79.0%(p<0.01)。

核心结论

  • 升级版AI模型在评估PERFORMS测试集乳腺X线图像时,诊断性能与前一版本无显著差异。

研究设计与样本

  • 回顾性研究,使用10个PERFORMS测试集(每个含60例挑战性病例),由1254名人类阅片者在2012至2023年间评估,同一AI模型V1和V2分别于2022年和2023年评估;以最高恶性可疑度评分(非恶性病例按乳腺、恶性病例按病灶)计算敏感度、特异度和AUC。

诊断标准与关键数值

  • 共分析328个恶性病灶、823个正常乳腺和55个良性乳腺;AI V1的AUC为0.93,V2为0.94(p=0.13);敏感度:人类83.2% vs AI V1 87.5%(p=0.12),人类83.2% vs AI V2 88.7%(p=0.04);特异度:人类79.0% vs AI V1 87.4%(p<0.01),人类79.0% vs AI V2 88.2%(p<0.01)。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

PURPOSE: To demonstrate a method of benchmarking the performance of two consecutive software releases of the same commercial artificial intelligence (AI) product to trained human readers using the Personal Performance in Mammographic Screening scheme (PERFORMS) external quality assurance scheme.

METHODS: In this retrospective study, ten PERFORMS test sets, each consisting of 60 challenging cases, were evaluated by human readers between 2012 and 2023 and were evaluated by Version 1 (V1) and Version 2 (V2) of the same AI model in 2022 and 2023 respectively. Both AI and humans considered each breast independently. Both AI and humans considered the highest suspicion of malignancy score per breast for non-malignant cases and per lesion for breasts with malignancy. Sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were calculated for comparison, with the study powered to detect a medium-sized effect (odds ratio, 3.5 or 0.29) for sensitivity.

RESULTS: The study included 1,254 human readers, with a total of 328 malignant lesions, 823 normal, and 55 benign breasts analysed. No significant difference was found between the AUCs for AI V1 (0.93) and V2 (0.94) (p = 0.13). In terms of sensitivity, no difference was observed between human readers and AI V1 (83.2 % vs 87.5 % respectively, p = 0.12), however V2 outperformed humans (88.7 %, p = 0.04). Specificity was higher for AI V1 (87.4 %) and V2 (88.2 %) compared to human readers (79.0 %, p < 0.01 respectively).

CONCLUSION: The upgraded AI model showed no significant difference in diagnostic performance compared to its predecessor when evaluating mammograms from PERFORMS test sets.

来源:Eur J Radiol,查看原文。版权归原作者所有。

引用

  1. Taib AG, James JJ, Partridge GJW, et al. Keeping AI on Track: Regular monitoring of algorithmic updates in mammography. Eur J Radiol. 2025. PMID: 40252280. doi:10.1016/j.ejrad.2025.112100

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。