时间飞跃法MR血管成像上狭窄闭塞性病变的自动检测:观察者性能研究
本研究从2家机构收集138例TOF-MRA图像作为内部(n=62)和外部(n=76)测试集,由5名放射科医生(2名神经放射科医生和3名放射科住院医师)分别在使用和不使用AI模型的情况下识别狭窄闭塞性病变并记录阅读时间。结果显示,5名放射科医生的平均AUFROC从无AI时的0.70提高到有AI时的0.76(P = .027),其中3名住院医师的提高最为明显(从0.68到0.76,P = .002);使用AI增加了阅读时间,但住院医师的阅读时间无显著变化,且观察者间一致性有所提高(组内相关系数从0.734增至0.752)。作者结论认为,该AI模型为放射科医生提供了辅助工具,可能提高TOF-MRA上颅内狭窄闭塞病变的检测准确性,经验较少的读者可能获益最大。
要点
- AI辅助使5名放射科医生平均AUFROC从0.70升至0.76(P = .027),住院医师从0.68升至0.76(P = .002)。
- 使用AI增加了阅读时间,但住院医师的阅读时间无显著变化。
- 使用AI后观察者间组内相关系数从0.734升至0.752。
核心结论
- 所提出的AI模型为放射科医生提供了辅助工具,可能提高TOF-MRA上颅内狭窄闭塞病变的检测准确性,经验较少的读者可能获益最大。
研究设计与样本
- 观察者性能研究,从2家机构收集138例TOF-MRA图像,分别作为内部测试集(n = 62)和外部测试集(n = 76);由5名放射科医生(2名神经放射科医生和3名放射科住院医师)分别在使用和不使用AI模型的情况下判读,识别狭窄闭塞性病变并记录阅读时间。
诊断标准与关键数值
- 采用折刀自由响应受试者工作特征曲线下面积(AUFROC)评估观察者性能;5名放射科医生平均AUFROC无AI时为0.70,有AI时为0.76(P = .027);3名放射科住院医师无AI时为0.68,有AI时为0.76(P = .002);观察者间组内相关系数无AI时为0.734,有AI时为0.752。
临床应用与阅片要点
- 作者结论认为该AI模型可作为放射科医生的辅助工具,可能提高TOF-MRA上颅内狭窄闭塞病变的检测准确性,经验较少的读者可能获益最大。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
BACKGROUND AND PURPOSE: Intracranial steno-occlusive lesions are responsible for acute ischemic stroke. However, the clinical benefits of artificial intelligence (AI)-based methods for detecting pathologic lesions in intracranial arteries have not been evaluated. We aimed to validate the clinical utility of an AI model for detecting steno-occlusive lesions in the intracranial arteries.
MATERIALS AND METHODS: Overall, 138 TOF-MRA images were collected from 2 institutions, which served as internal (n = 62) and external (n = 76) test sets, respectively. Each study was reviewed by 5 radiologists (2 neuroradiologists and 3 radiology residents) to compare the usage and nonusage of our proposed AI model for TOF-MRA interpretation. They identified the steno-occlusive lesions and recorded their reading time. Observer performance was assessed by using the area under the jackknife free-response receiver operating characteristic curve (AUFROC) and reading time for comparison.
RESULTS: The average AUFROC for the 5 radiologists demonstrated an improvement from 0.70 without AI to 0.76 with AI (P = .027). Notably, this improvement was most pronounced among the 3 radiology residents, whose performance metrics increased from 0.68 to 0.76 (P = .002). Despite an increased reading time by using AI, there was no significant change among the readings by radiology residents. Moreover, the use of AI resulted in improved interobserver agreement among the reviewers (the intraclass correlation coefficient increased from 0.734 to 0.752).
CONCLUSIONS: Our proposed AI model offers a supportive tool for radiologists, potentially enhancing the accuracy of detecting intracranial steno-occlusion lesions on TOF-MRA. Less experienced readers may benefit the most from this model.
引用
- Lim H, Choi D, Sunwoo L, et al. Automated Detection of Steno-Occlusive Lesion on Time-of-Flight MR Angiography: An Observer Performance Study. AJNR Am J Neuroradiol. 2024. PMID: 38719612. doi:10.3174/ajnr.A8334
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。