文献速递乳腺乳腺X线

2023年RSNA筛查乳腺X线摄影乳腺癌检测AI挑战赛提交算法的性能表现

Performance of Algorithms Submitted in the 2023 RSNA Screening Mammography Breast Cancer Detection AI Challenge.
PubMed 文献 · Radiology · 2025-01-01

该研究评估了2023年RSNA筛查乳腺X线摄影乳腺癌检测AI挑战赛中提交的1537个算法,在来自美国和澳大利亚两个中心、共5415名女性的评估数据集上的表现。中位召回率、敏感度、特异度和阳性预测值分别为1.7%、27.6%、98.7%和36.9%;排名第一的算法分别为1.5%、48.6%、99.5%和64.6%。前3和前10集成模型敏感度分别为60.7%和67.8%,召回率分别为2.4%和3.5%,特异度分别为98.8%和97.8%。

要点

  • 不同AI算法在筛查乳腺X线摄影中检出的癌症各不相同,集成模型可在保持低召回率的同时提高敏感度。
  • 排名第一的算法召回率1.5%、敏感度48.6%、特异度99.5%、阳性预测值64.6%。
  • 前3集成模型对美国数据集的敏感度低于澳大利亚数据集(52.0% vs 68.1%;OR=0.51;P=.02)。

核心结论

  • 不同AI算法在筛查乳腺X线摄影中识别出不同的癌症,集成模型在保持低召回率的同时提高了敏感度。

研究设计与样本

  • 评估2022年11月至2023年2月提交的1687个AI算法中的1537个,使用来自美国一个中心和澳大利亚一个中心共两个中心的评估数据集;癌症病例在筛查时发现并经病理检查确认,非癌症病例随访至少1年;前算法集成模型的结果按任一纳入算法提示召回即召回该病例计算;采用比值比(OR)分析按临床或人口学特征分层时AI性能的差异。

诊断标准与关键数值

  • 评估数据集共5415名女性(中位年龄59岁,IQR 52-66岁);1537个算法的中位召回率1.7%、敏感度27.6%、特异度98.7%、阳性预测值36.9%;排名第一的算法召回率1.5%、敏感度48.6%、特异度99.5%、阳性预测值64.6%;前3集成模型敏感度60.7%、召回率2.4%、特异度98.8%;前10集成模型敏感度67.8%、召回率3.5%、特异度97.8%。

临床应用与阅片要点

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

Background The 2023 RSNA Screening Mammography Breast Cancer Detection AI Challenge invited participants to develop artificial intelligence (AI) models capable of independently interpreting mammograms. Purpose To assess the performance of the submitted algorithms, explore the potential for improving performance by combining the best-performing AI algorithms, and investigate how performance was influenced by the demographic and clinical characteristics of the evaluation cohort. Materials and Methods A total of 1687 AI algorithms were submitted from November 2022 to February 2023. Of these, 1537 algorithms were assessed using an evaluation dataset from two sites-one in the United States and one in Australia. Cancer cases were identified at screening and confirmed with pathologic examination; noncancer cases were followed up for at least 1 year. Results for ensemble models of top algorithms were computed by recalling a case when any of the included algorithms indicated recall. Odds ratios (ORs) were used to investigate differences in AI performance when the dataset was stratified by clinical or demographic characteristics. Results The evaluation dataset consisted of 5415 women (median age, 59 years [IQR, 52-66 years]). Among the 1537 AI algorithms, the median recall rate, sensitivity, specificity, and positive predictive value (PPV) were 1.7%, 27.6%, 98.7%, and 36.9%, respectively. For the top-ranked algorithm, the recall rate, sensitivity, specificity, and PPV were 1.5%, 48.6%, 99.5%, and 64.6%, respectively. Ensemble models of the top 3 and top 10 algorithms had a sensitivity of 60.7% and 67.8%, respectively; the corresponding recall rates were 2.4% and 3.5%, and the corresponding specificities were 98.8% and 97.8%. Lower sensitivity was observed for the U.S. dataset than for the Australian dataset (top 3 ensemble model: 52.0% vs 68.1%; OR = 0.51; P = .02), and greater sensitivity was observed for invasive cancers than for noninvasive cancers (top 3 ensemble model: 68.0% vs 43.8%; OR = 2.73; P = .001). Conclusion The different AI algorithms identified different cancers during screening mammography, and ensemble models had increased sensitivity while maintaining low recall rates. © RSNA, 2025 Supplemental material is available for this article.

来源:Radiology,查看原文。版权归原作者所有。

引用

  1. Chen Y, Partridge GJW, Vazirabad M, et al. Performance of Algorithms Submitted in the 2023 RSNA Screening Mammography Breast Cancer Detection AI Challenge. Radiology. 2025. PMID: 40793948. doi:10.1148/radiol.241447

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。