评估临床可用人工智能模型检测颅内动脉瘤:多阅片者研究与算法审计
本研究采用两个头CTA队列验证临床可用AI模型检测颅内动脉瘤,队列1(131例)用于六名普通放射科医师参与的多阅片者多病例研究,队列2(515例)模拟常规临床场景。AI辅助使动脉瘤诊断AUC从0.815升至0.875(p=0.008),AI初筛判为阴性的病例占60.4%,阴性预测值0.994。算法审计发现微小动脉瘤检测和假阳性排除有待改进。
要点
- AI辅助使普通放射科医师诊断颅内动脉瘤的AUC从0.815提高至0.875(p=0.008)。
- AI初筛将60.4%的CTA病例判为阴性,阴性预测值达0.994(95%CI 0.977-0.999)。
- 算法审计提示微小动脉瘤的准确检测和假阳性病灶的有效排除是需改进的两个问题。
核心结论
- 高性能AI模型检测颅内动脉瘤具有临床实用性,可显著提高普通放射科医师的诊断效能,并可能减少其常规临床实践中的工作量;算法审计为未来AI模型的开发与验证提供了指导。
研究设计与样本
- 多阅片者多病例(MRMC)研究加算法审计,两个头CTA队列:队列1为金标准连续CTA病例131例,由六名委员会认证的普通放射科医师参与;队列2为临床CTA病例515例,用于模拟常规临床场景。
诊断标准与关键数值
- MRMC研究中,AI辅助策略使动脉瘤诊断的受试者工作特征曲线下面积从0.815(95%CI 0.754-0.875)提高至0.875(95%CI 0.831-0.921;p=0.008)。
- AI初筛研究中,60.4%的CTA病例被AI判为阴性,阴性预测值为0.994(95%CI 0.977-0.999)。
临床应用与阅片要点
- 作者认为该高性能AI模型检测颅内动脉瘤具有临床实用性,可显著提高普通放射科医师的诊断效能,并可能减少其常规临床实践中的工作量。
- 作者指出算法审计发现的微小动脉瘤准确检测和假阳性病灶有效排除两个问题,可为未来AI模型的开发与验证提供指导。
局限性
- 算法审计发现模型在微小动脉瘤的准确检测方面存在需改进的问题。
- 算法审计发现模型在假阳性病灶的有效排除方面存在需改进的问题。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
PURPOSE: We aimed to validate a clinically available artificial intelligence (AI) model to assist general radiologists in the detection of intracranial aneurysm (IA) in a multi-reader multi-case (MRMC) study, and to explore its performance in routine clinical settings.
METHODS: Two distinct cohorts of head CT angiography (CTA) data were assembled to validate an AI model. Cohort 1, comprising gold-standard consecutive CTA cases, was used in an MRMC study involving six board-certified general radiologists. Cohort 2, representing clinical CTA cases, was used to simulate a routine clinical setting. Following these evaluations, an algorithmic audit was conducted to identify any unusual or unexpected behaviors exhibited by the model.
RESULTS: Cohort 1 consisted of 131 CTA cases, while Cohort 2 included 515 CTA cases. In the MRMC study, the AI-assisted strategy demonstrated a significant improvement in aneurysm diagnostic performance, with the area under the receiver operating characteristic curve increasing from 0.815 (95%CI: 0.754-0.875) to 0.875 (95%CI: 0.831-0.921; p = 0.008). In the AI-based first-reader study, 60.4% of the CTA cases were identified as negative by the AI, with a high negative predictive value of 0.994 (95%CI: 0.977-0.999). The algorithmic audit highlighted two issues for improvement: the accurate detection of tiny aneurysms and the effective exclusion of false-positive lesions.
CONCLUSION: This study highlights the clinical utility of a high-performance AI model in detecting IAs, significantly improving general radiologists' diagnostic performance with the potential to reduce their workload in routine clinical practice. The algorithmic audit offers insights to guide the development and validation of future AI models.
引用
- Hu B, He H, Shi Z, et al. Evaluating a clinically available artificial intelligence model for intracranial aneurysm detection: a multi-reader study and algorithmic audit. Neuroradiology. 2025. PMID: 39812775. doi:10.1007/s00234-024-03536-3
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。