FDA批准的深度学习软件应用与不同专业水平放射科医师的比较:一项回顾性单中心研究中的颅内出血检测
该回顾性单中心研究在2020年1月至5月的20周内,对2210例成人非增强头部CT扫描进行自动AI分析,排除22例严重运动伪影后,由第二年住院医师和认证放射科医师在模拟时间压力下回顾性评估颅内出血,分歧由神经放射学专科医师裁决作为参考标准。结果显示住院医师与认证放射科医师间一致性很高(κ=0.89),住院医师与参考标准间更高(κ=0.93)。软件假阳性64例、假阴性68例,总体敏感度68.2%、特异度96.8%、阳性预测值69.5%、阴性预测值96.6%、准确率94.0%;住院医师对应值分别为94.9%、99.2%、93.1%、99.4%、98.8%。深度学习应用的准确率低于住院医师和认证神经放射科医师(p < 0.001)。
要点
- FDA批准的深度学习软件检测颅内出血的敏感度为68.2%,假阴性68例。
- 住院医师在时间压力下敏感度94.9%,准确率98.8%,高于软件。
- 软件准确率低于住院医师和认证神经放射科医师(p < 0.001)。
核心结论
- 在模拟时间压力下,住院医师检测CT扫描颅内出血的表现优于FDA批准的深度学习程序。
研究设计与样本
- 回顾性单中心研究,2020年1月至5月20周内共2210例成人非增强头部CT扫描,排除22例严重运动伪影后分析2188例;由第二年住院医师和认证放射科医师在模拟时间压力下评估,分歧由神经放射学专科医师裁决作为参考标准。
诊断标准与关键数值
- 颅内出血存在于2188例扫描中的214例;软件敏感度68.2%、特异度96.8%、阳性预测值69.5%、阴性预测值96.6%、准确率94.0%;住院医师对应值分别为94.9%、99.2%、93.1%、99.4%、98.8%;住院医师与认证放射科医师间κ=0.89,住院医师与参考标准间κ=0.93;软件准确率低于住院医师和认证神经放射科医师(p < 0.001)。
临床应用与阅片要点
- 作者结论强调 thoughtful workflow integration 和AI应用在不同临床环境中获批后验证的重要性。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
PURPOSE: To assess an FDA-approved and CE-certified deep learning (DL) software application compared to the performance of human radiologists in detecting intracranial hemorrhages (ICH).
METHODS: Within a 20-week trial from January to May 2020, 2210 adult non-contrast head CT scans were performed in a single center and automatically analyzed by an artificial intelligence (AI) solution with workflow integration. After excluding 22 scans due to severe motion artifacts, images were retrospectively assessed for the presence of ICHs by a second-year resident and a certified radiologist under simulated time pressure. Disagreements were resolved by a subspecialized neuroradiologist serving as the reference standard. We calculated interrater agreement and diagnostic performance parameters, including the Breslow-Day and Cochran-Mantel-Haenszel tests.
RESULTS: An ICH was present in 214 out of 2188 scans. The interrater agreement between the resident and the certified radiologist was very high (κ = 0.89) and even higher (κ = 0.93) between the resident and the reference standard. The software has delivered 64 false-positive and 68 false-negative results giving an overall sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of 68.2%, 96.8%, 69.5%, 96.6%, and 94.0%, respectively. Corresponding values for the resident were 94.9%, 99.2%, 93.1%, 99.4%, and 98.8%. The accuracy of the DL application was inferior (p < 0.001) to that of both the resident and the certified neuroradiologist.
CONCLUSION: A resident under time pressure outperformed an FDA-approved DL program in detecting ICH in CT scans. Our results underline the importance of thoughtful workflow integration and post-approval validation of AI applications in various clinical environments.
引用
- Kau, Ziurlys, Taschwer, et al. FDA-approved deep learning software application versus radiologists with different levels of expertise: detection of intracranial hemorrhage in a retrospective single-center study. Neuroradiology. 2022. PMID: 34988593. doi:10.1007/s00234-021-02874-w
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。