三种商用人工智能算法独立评估乳腺X线筛查的外部验证
这项回顾性病例对照研究基于瑞典2008至2015年双读片乳腺X线筛查队列,纳入739例乳腺癌患者和8066例健康对照,评估三种商用AI算法独立读片性能。AI-1的AUC为0.956,在放射科医生特异性下敏感度81.9%;AI-1与第一读片医生联合敏感度达88.6%,高于第一与第二读片医生联合。
要点
- AI-1独立读片AUC 0.956,敏感度81.9%,接近放射科医生水平。
- AI-1联合第一读片医生敏感度88.6%,优于双读片医生组合。
- AI算法作为独立读片者需前瞻性临床试验进一步验证。
核心结论
- 商用AI算法AI-1在乳腺X线筛查中展现出足够诊断性能,可进一步作为独立读片者在前瞻性试验中评估。
- 第一读片医生联合最佳AI算法比联合第二读片医生发现更多癌症阳性病例。
研究设计与样本
- 回顾性病例对照研究,基于瑞典斯德哥尔摩一家学术医院2008至2015年双读片人群乳腺X线筛查队列。
- 纳入8805名40至74岁接受乳腺X线筛查且无植入物或既往乳腺癌的女性,包括739例乳腺癌患者和8066例健康对照。
主要发现
- 癌症检测AUC:AI-1为0.956(95%CI 0.948-0.965),AI-2为0.922(95%CI 0.910-0.934),AI-3为0.920(95%CI 0.909-0.931)。
- 在放射科医生特异性(96.6%)下,敏感度:AI-1为81.9%,AI-2为67.0%,AI-3为67.4%,第一读片医生为77.4%,第二读片医生为80.1%。
- AI-1联合第一读片医生(任一评估异常即判为异常)达到敏感度88.6%、特异性93.0%,无其他AI与医生组合超过此敏感度。
关键数值与推荐
- 阳性随访结果定义为筛查时或筛查后12个月内病理证实的诊断;阴性随访定义为2年无癌随访。
- 正常或异常的决策切点定义为第一读片放射科医生的平均特异性96.6%。
临床怎么用
- AI算法可作为独立读片者用于乳腺X线筛查评估,AI-1联合第一读片医生可提高癌症检出敏感度。
- 在临床推广前,AI作为独立读片者需在前瞻性临床试验中进一步评估。
局限性
- 回顾性病例对照设计,可能引入选择偏倚。
- 研究基于单中心筛查队列,外部泛化性需验证。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
IMPORTANCE: A computer algorithm that performs at or above the level of radiologists in mammography screening assessment could improve the effectiveness of breast cancer screening. OBJECTIVE: To perform an external evaluation of 3 commercially available artificial intelligence (AI) computer-aided detection algorithms as independent mammography readers and to assess the screening performance when combined with radiologists. DESIGN, SETTING, AND PARTICIPANTS: This retrospective case-control study was based on a double-reader population-based mammography screening cohort of women screened at an academic hospital in Stockholm, Sweden, from 2008 to 2015. The study included 8805 women aged 40 to 74 years who underwent mammography screening and who did not have implants or prior breast cancer. The study sample included 739 women who were diagnosed as having breast cancer (positive) and a random sample of 8066 healthy controls (negative for breast cancer). MAIN OUTCOMES AND MEASURES: Positive follow-up findings were determined by pathology-verified diagnosis at screening or within 12 months thereafter. Negative follow-up findings were determined by a 2-year cancer-free follow-up. Three AI computer-aided detection algorithms (AI-1, AI-2, and AI-3), sourced from different vendors, yielded a continuous score for the suspicion of cancer in each mammography examination. For a decision of normal or abnormal, the cut point was defined by the mean specificity of the first-reader radiologists (96.6%). RESULTS: The median age of study participants was 60 years (interquartile range, 50-66 years) for 739 women who received a diagnosis of breast cancer and 54 years (interquartile range, 47-63 years) for 8066 healthy controls. The cases positive for cancer comprised 618 (84%) screen detected and 121 (16%) clinically detected within 12 months of the screening examination. The area under the receiver operating curve for cancer detection was 0.956 (95% CI, 0.948-0.965) for AI-1, 0.922 (95% CI, 0.910-0.934) for AI-2, and 0.920 (95% CI, 0.909-0.931) for AI-3. At the specificity of the radiologists, the sensitivities were 81.9% for AI-1, 67.0% for AI-2, 67.4% for AI-3, 77.4% for first-reader radiologist, and 80.1% for second-reader radiologist. Combining AI-1 with first-reader radiologists achieved 88.6% sensitivity at 93.0% specificity (abnormal defined by either of the 2 making an abnormal assessment). No other examined combination of AI algorithms and radiologists surpassed this sensitivity level. CONCLUSIONS AND RELEVANCE: To our knowledge, this study is the first independent evaluation of several AI computer-aided detection algorithms for screening mammography. The results of this study indicated that a commercially available AI computer-aided detection algorithm can assess screening mammograms with a sufficient diagnostic performance to be further evaluated as an independent reader in prospective clinical trials. Combining the first readers with the best algorithm identified more cases positive for cancer than combining the first readers with second readers.
引用
- Salim M, Wåhlin E, Dembrower K, et al. External Evaluation of 3 Commercial Artificial Intelligence Algorithms for Independent Assessment of Screening Mammograms. JAMA Oncol. 2020. PMID: 32852536. doi:10.1001/jamaoncol.2020.3321
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。