影响深度学习检测肺栓塞性能的因素
本回顾性研究纳入单一一医疗系统2023年4月至2024年6月间11,144例CT肺动脉造影检查,评估一款FDA清关的商业肺栓塞检测模型的真实世界性能,结果显示总体敏感度为83.5%、阳性预测值为90.5%,扫描仪厂商、层厚增加、成像伪影、心力衰竭和肺动脉高压与敏感度下降显著相关,人口学因素无显著关联。
要点
- 商业肺栓塞AI模型总体敏感度83.5%、阳性预测值90.5%,真实世界性能存在显著变异。
- 扫描仪厂商不同、层厚增加、存在成像伪影与敏感度下降显著相关。
- 心力衰竭和肺动脉高压患者中模型敏感度显著降低,年龄、性别、种族、BMI无显著关联。
核心结论
- AI在临床实践中的性能因技术成像参数和患者合并症而显著变化,理解这些因素对产品选择和部署后监测至关重要,研究结果强调需要建立考虑机构特定技术基础设施和患者人群的本地验证框架,以确保AI在不同临床环境中的安全部署。
研究设计与样本
- 回顾性研究,纳入单一一医疗系统2023年4月至2024年6月间11,144例CT肺动脉造影检查,由一款商业肺栓塞检测模型处理;从DICOM头和电子健康记录中提取技术参数、人口学因素和临床合并症,采用单因素和多因素逻辑回归分析识别与性能下降相关的因素。
诊断标准与关键数值
- 11,144例中肺栓塞阳性1,193例(10.7%);模型总体敏感度83.5%(95%CI 81.3%-85.5%),阳性预测值90.5%(95%CI 88.7%-92.1%);多因素分析显示敏感度下降与扫描仪厂商(不同厂商 vs 参照,OR 0.25,95%CI 0.14-0.46 和 OR 0.34,95%CI 0.17-0.69,P < .003)、层厚每增加1 mm(OR 0.74,95%CI 0.57-0.95,P = .018)、存在成像伪影(OR 0.33,95%CI 0.23-0.48,P < .001)、心力衰竭(OR 0.58,95%CI 0.38-0.88,P = .010)、肺动脉高压(OR 0.44,95%CI 0.25-0.77,P = .004)显著相关;年龄、性别、种族、体重指数与模型性能无显著关联。
临床应用与阅片要点
- 作者认为,理解影响AI性能的技术成像参数和患者合并症因素对最佳产品选择和有效部署后监测至关重要,研究结果强调需要建立考虑机构特定技术基础设施和患者人群的本地验证框架,以确保AI在不同临床环境中的安全部署。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVE: AI models are increasingly adopted in clinical practice, yet their generalizability outside controlled validation settings remains unclear. We aimed to evaluate the real-world performance of an FDA-cleared commercial pulmonary embolism (PE) detection model and identify technical, demographic, and clinical factors associated with performance variation, to inform postproduction monitoring and deployment strategies.
METHODS: This retrospective study included 11,144 CT pulmonary angiography examinations performed in a single health system between April 2023 and June 2024, processed by a commercial PE detection model. Technical parameters (scanner manufacturer, slice thickness, dose index volume, contrast enhancement of pulmonary artery), demographic factors (age, gender, race, body mass index), and clinical comorbidities (heart failure, pulmonary hypertension, cancer) were extracted from DICOM headers and electronic health records. Univariate and multivariable logistic regression analyses identified factors associated with decreased performance.
RESULTS: There were 1,193 of 11,144 (10.7%) PE-positive cases. The model had an overall 83.5% (95% confidence interval [CI] 81.3%-85.5%) sensitivity and positive predictive value was 90.5% (95% CI 88.7%-92.1%). Multivariable analysis showed significant associations between decreased sensitivity and scanner manufacturer (odds ratio [OR] 0.25, 95% CI 0.14-0.46 and OR 0.34, 95% CI 0.17-0.69, for different vendors versus reference, P < .003), increased slice thickness (OR 0.74, 95% CI 0.57-0.95 per 1-mm increase, P = .018), presence of imaging artifacts (OR 0.33, 95% CI 0.23-0.48, P < .001), heart failure (OR 0.58, 95% CI 0.38-0.88, P = .010), and pulmonary hypertension (OR 0.44, 95% CI 0.25-0.77, P = .004). Demographic factors including age, gender, race, and body mass index showed no significant associations with model performance.
CONCLUSION: AI performance in clinical practice varies significantly based on technical imaging parameters and patient comorbidities. Understanding these factors is essential for optimal product selection and for effective postdeployment monitoring, enabling investigation of model drift in evolving clinical settings. The findings highlight the need for local validation frameworks that account for institution-specific technical infrastructure and patient populations, to ensure safe AI deployment across diverse clinical environments.
引用
- Sorin V, Korfiatis P, Langer SG, et al. Factors Impacting the Performance of Deep Learning Detection of Pulmonary Emboli. J Am Coll Radiol. 2025. PMID: 41456841. doi:10.1016/j.jacr.2025.12.028
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。