文献速递神经技术与安全CT

非增强头颅CT颅内出血人工智能分诊的前瞻性评估

Prospective Evaluation of Artificial Intelligence Triage of Intracranial Hemorrhage on Noncontrast Head CT Examinations.
PubMed 文献 · AJR Am J Roentgenol · 2024-09-04

这项前瞻性单中心研究纳入9954例成人头颅非增强CT检查,对比放射科医生在无AI辅助与有AI分诊通知系统辅助下的颅内出血诊断表现和报告周转时间。结果显示两组准确度、敏感度、PPV、NPV无显著差异,无AI辅助组特异度更高(99.8% vs 99.3%,P=0.004),出血阳性报告平均周转时间无显著差异(147.1分钟 vs 149.9分钟,P=0.11)。结论认为该AI分诊系统未改善诊断表现或报告周转时间。

要点

  • AI辅助组特异度低于无AI辅助组(99.3% vs 99.8%,P=0.004),阅片时需警惕AI提示阳性但实际无出血的假阳性。
  • 有无AI辅助在准确度、敏感度、PPV、NPV上均无显著差异(均P>0.01)。
  • 出血阳性报告平均周转时间无AI辅助147.1分钟 vs 有AI辅助149.9分钟,差异未达统计学显著(P=0.11)。

核心结论

  • 颅内出血检测AI分诊系统未改善放射科医生的诊断表现或报告周转时间。

研究设计与样本

  • 前瞻性单中心研究,纳入2021年5月12日至2021年6月30日(第1阶段)或2021年9月30日至2021年12月4日(第2阶段)接受头颅非增强CT的成人患者,共9954例检查、7371例患者(平均年龄54.8±19.8岁;女3773例,男3598例)。

诊断标准与关键数值

  • 第1阶段无AI辅助阅片,第2阶段有AI辅助阅片,由神经放射科或急诊放射科医生判读。
  • 参考标准由放射科医生小组对报告与AI不一致的全部检查及部分其余检查复核建立。
  • 无AI辅助 vs 有AI辅助:准确度99.5% vs 99.2%,敏感度98.6% vs 98.9%,PPV 99.0% vs 97.5%,NPV 99.7% vs 99.7%,均P>0.01(Bonferroni校正后显著性阈值为0.01)。
  • 无AI辅助 vs 有AI辅助:特异度99.8% vs 99.3%,P=0.004。
  • 第1阶段出血阳性率19.8%(735/3716)vs 第2阶段21.9%(1368/6238),P=0.01。

临床应用与阅片要点

  • 作者结论认为这项大型前瞻性真实世界研究不支持使用AI辅助进行颅内出血检测。
  • AI辅助组特异度较低,阅片时对AI阳性提示需结合图像自行确认。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

BACKGROUND. Retrospective studies evaluating artificial intelligence (AI) algorithms for intracranial hemorrhage (ICH) detection on noncontrast CT (NCCT) have shown promising results but lack prospective validation. OBJECTIVE. The purpose of this article was to evaluate the impact on radiologists' real-world aggregate performance for ICH detection and report turnaround times for ICH-positive examinations of a radiology department's implementation of an AI triage and notification system for ICH detection on head NCCT examinations. METHODS. This prospective single-center study included adult patients who underwent head NCCT examinations from May 12, 2021, to June 30, 2021 (phase 1), or from September 30, 2021, to December 4, 2021 (phase 2). Before phase 1, the radiology department implemented a commercial AI triage system for ICH detection that processed head NCCT examinations and notified radiologists of positive results through a widget with a floating pop-up display. Examinations were interpreted by neuroradiologists or emergency radiologists, who evaluated examinations without and with AI assistance in phases 1 and 2, respectively. A panel of radiologists conducted a review process for all examinations with discordance between the radiology report and AI and a subset of remaining examinations to establish the reference standard. Diagnostic performance and report turnaround times were compared using the Pearson chi-square test and Wilcoxon rank sum test, respectively. Bonferroni correction was used to account for five diagnostic performance metrics (adjusted significance threshold, .01 [α = .05/5]). RESULTS. A total of 9954 examinations from 7371 patients (mean age, 54.8 ± 19.8 [SD] years; 3773 women, 3598 men) were included. In phases 1 and 2, 19.8% (735/3716) and 21.9% (1368/6238) of examinations, respectively, were positive for ICH (p = .01). Radiologists without versus with AI showed no significant difference in accuracy (99.5% vs 99.2%), sensitivity (98.6% vs 98.9%), PPV (99.0% vs 97.5%), or NPV (99.7% vs 99.7%) (all p > .01); specificity was higher for radiologists without than with AI (99.8% vs 99.3%, respectively, p = .004). Mean report turnaround time for ICH-positive examinations was 147.1 minutes without AI versus 149.9 minutes with AI (p = .11). CONCLUSION. An AI triage system for ICH detection did not improve radiologists' diagnostic performance or report turnaround times. CLINICAL IMPACT. This large prospective real-world study does not support use of AI assistance for ICH detection.

来源:AJR Am J Roentgenol,查看原文。版权归原作者所有。

引用

  1. Savage CH, Tanwar M, Elkassem AA, et al. Prospective Evaluation of Artificial Intelligence Triage of Intracranial Hemorrhage on Noncontrast Head CT Examinations. AJR Am J Roentgenol. 2024. PMID: 39230402. doi:10.2214/AJR.24.31639

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。