前瞻性影子模式评估AI在CT血管造影中检测颅内动脉瘤的增量价值与运行影响
本研究以前瞻性影子模式连续纳入3856例脑CTA检查,放射科医生对AI结果设盲,评估FDA批准的Aidoc AI算法检测颅内动脉瘤的运行表现。AI单独敏感度高于放射科医生,AI额外检出55例放射科医生漏诊的真阳性动脉瘤,相对增强检出率为39%。运行指标总体有利但随场景变化,住院与急诊场景获益最大,门诊场景不利。
要点
- AI单独敏感度0.846高于放射科医生0.718,特异度相似,提示AI可作为脑CTA动脉瘤筛查的辅助手段。
- AI额外检出的动脉瘤多为小于3 mm(56.4%),对发现微小动脉瘤有增量价值。
- 运行指标在住院和急诊场景最有利,门诊场景收益有限,部署时应考虑场景差异。
核心结论
- AI工具显示出有利的运行性能,支持在真实世界临床实践中部署,但不同场景的运行指标存在差异。
- AI与放射科医生联合可提高脑CTA颅内动脉瘤的综合检出敏感度。
研究设计与样本
- 放射科医生在阅片时对AI结果设盲,AI为Aidoc公司(以色列特拉维夫)算法,自然语言处理提取放射学报告结果。
- 不一致病例经神经放射科医生判定并对AI解除设盲,采用混合参考标准仅对不一致病例进行判定。
关键影像表现/征象
- AI单独检出的动脉瘤大多小于3 mm(55例中有31例,占56.4%)。
- 放射科医生单独检出的动脉瘤大多为3至5 mm(33例中有18例,占54.5%)。
- 总体检查阳性率为5.1%(3856例中195例),放射科医生与AI一致率为96.3%(3856例中3714例)。
诊断标准与关键数值
- AI单独敏感度为0.846(95%CI:0.787-0.894),高于放射科医生单独的0.718(95%CI:0.649-0.779)。
- AI特异度为0.987(95%CI:0.983-0.991),放射科医生特异度为0.985(95%CI:0.981-0.989)。
- 相对增强检出率为39%(AI独有的真阳性55例/放射科医生真阳性140例),AI与放射科医生增量检出比值为1.83(55/30)。
- 运行指标中,增益疼痛比为1.20(55/46),需检查数为70.1(3856/55),住院场景相对增强检出率78.3%、增益疼痛比2.57、需检查数28.9。
- 急诊场景相对增强检出率37.1%、增益疼痛比1.0、需检查数85.3;门诊场景相对增强检出率14.1%、增益疼痛比0.67、需检查数130.3。
临床应用与阅片要点
- AI对小于3 mm动脉瘤有突出检出能力,临床中可作为放射科医生的辅助工具以降低微小动脉瘤漏诊风险。
- 住院和急诊脑CTA检查中AI的运行获益最明显,可优先在这些场景部署。
- 门诊场景运行指标不利,提示该场景下AI的假阳性负担相对较高,需谨慎使用。
- 放射科医生与AI一致率为96.3%,说明多数情况下AI结果与阅片结论相符,AI异常提示需要引起关注。
局限性
- 采用混合参考标准,仅对不一致病例进行判定,一致性病例未做独立验证,可能导致敏感度和特异度估计偏倚。
- 本研究为单中心数据,未说明设备与扫描序列的标准化情况,结果的外推性需进一步验证。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVE: To evaluate the operational performance of an FDA-cleared artificial intelligence (AI) algorithm for brain CT angiography (CTA) aneurysm detection and the incremental value of combining AI with radiologists.
METHODS: Prospective shadow-mode study of consecutive brain CTAs (November 7, 2023, to December 19, 2023) was performed with radiologists blinded to AI. Aidoc's (Tel Aviv, Israel) AI algorithm processed CTAs, and natural language processing extracted radiology report results. Discordances underwent neuroradiologist adjudication with AI unblinding. Performance metrics included AI:radiologist incremental detection ratio, relative enhanced detection rate (rEDR), gain-to-pain ratio (GPR), and number-needed-to-examine (NNE). Performance metrics are estimated under a hybrid reference standard wherein only discordances are adjudicated.
RESULTS: Among 3,856 CTAs, examination positive rate was 5.1% (195 of 3,856). Radiologist-AI concordance was 96.3% (3,714 of 3,856). Sensitivity of AI alone (0.846, 0.787-0.894) exceeded radiologist alone (0.718, 0.649-0.779), with similar specificity (radiologist: 0.985, 0.981-0.989; AI: 0.987, 0.983-0.991). AI surfaced additional aneurysms not identified by radiologists, yielding an rEDR of 39% (55 AI-only true-positives per 140 radiologist true-positives) and projecting a higher combined sensitivity for radiologists with AI. AI:radiologist incremental detection ratio was 1.83 (55 of 30), favorable for AI. Operationally, GPR was favorable at 1.20 (55 of 46), and NNE was 70.1 (3,856 of 55). Operational metrics were most favorable in inpatients (rEDR 78.3%, GPR 2.57, NNE 28.9) and emergency (rEDR 37.1%, GPR 1.0, NNE 85.3) and were unfavorable in outpatients (rEDR 14.1%, GPR 0.67, NNE 130.3). Most AI-only aneurysms were <3 mm (31 of 55, 56.4%); radiologist-only were mostly 3 to 5 mm (18 of 33, 54.5%).
CONCLUSIONS: The AI tool demonstrated favorable operational performance, supporting clinical deployment in real-world practice, although setting-specific metrics were variable.
引用
- Goldberg-Stein, Sanmartin, Saks, et al. Prospective Shadow-Mode Evaluation of an Artificial Intelligence Tool for Intracranial Aneurysm Detection on CT Angiography: Incremental Yield and Operational Impact. J Am Coll Radiol. 2026. PMID: 42742498. doi:10.1016/j.jacr.2026.07.018
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。