人类与人工智能在局灶性皮质发育不良检测中的定量比较
该前瞻性研究记录28名阅片者(20名非专家和5名专家)对180例(146例局灶性皮质发育不良患者和34例健康对照)的检测表现,并与3种人工智能模型比较。非专家检出47%,专家检出68%,3种人工智能模型分别检出32%、51%和72%,但后者平均每例受试者预测超过13个假阳性簇。专家+人工智能组合可将专家检出率提高至多19%。
要点
- 专家对局灶性皮质发育不良的检出率为68%,非专家为47%。
- 表现最好的人工智能模型检出72%,但平均每例受试者超过13个假阳性簇。
- 经皮质征和皮质增厚可提高人类检出率,异常脑回和灰白质分界模糊则影响人工智能模型。
核心结论
- 人工智能与人类对局灶性皮质发育不良的预测存在差异,尤其针对某些MRI特征,人工智能可能补充诊断流程。
研究设计与样本
- 前瞻性记录阅片者使用单点和三维边界框检测局灶性皮质发育不良的表现。
关键影像表现/征象
- 人类检出率在存在经皮质征(P < 0.001)和皮质增厚(P < 0.001)时提高。
- 人工智能模型对异常脑回(P < 0.01)或灰白质分界模糊(P < 0.01)敏感。
诊断标准与关键数值
- 非专家检出47%(95%CI:46, 49)的局灶性皮质发育不良,专家检出68%(95%CI:65, 71)。
- 3种人工智能模型分别检出32%、51%和72%的局灶性皮质发育不良。
- 表现最好的人工智能模型平均每例受试者预测超过13个假阳性簇。
- 与单个专家相比,专家-专家配对多检出13%(95%CI:9, 18)的局灶性皮质发育不良(P < 0.001)。
- 所有人工智能模型使专家检出率提高至多19%(95%CI:15, 24)(P < 0.001);非专家+人工智能配对仍可超过单个专家至多13%(95%CI:10, 17)。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVES: Artificial intelligence (AI) is thought to improve lesion detection. However, a lack of knowledge about human performance prevents a comparative evaluation of AI and an accurate assessment of its impact on clinical decision-making. The objective of this work is to quantitatively evaluate the ability of humans to detect focal cortical dysplasia (FCD), compare it to state-of-the-art AI, and determine how it may aid diagnostics.
MATERIALS AND METHODS: We prospectively recorded the performance of readers in detecting FCDs using single points and 3-dimensional bounding boxes. We acquired predictions of 3 AI models for the same dataset and compared these to readers. Finally, we analyzed pairwise combinations of readers and models.
RESULTS: Twenty-eight readers, including 20 nonexpert and 5 expert physicians, reviewed 180 cases: 146 subjects with FCD (median age: 25, interquartile range: 18) and 34 healthy control subjects (median age: 43, interquartile range: 19). Nonexpert readers detected 47% (95% confidence interval [CI]: 46, 49) of FCDs, whereas experts detected 68% (95% CI: 65, 71). The 3 AI models detected 32%, 51%, and 72% of FCDs, respectively. The latter, however, also predicted more than 13 false-positive clusters per subject on average. Human performance was improved in the presence of a transmantle sign ( P < 0.001) and cortical thickening ( P < 0.001). In contrast, AI models were sensitive to abnormal gyration ( P < 0.01) or gray-white matter blurring ( P < 0.01). Compared with single experts, expert-expert pairs detected 13% (95% CI: 9, 18) more FCDs ( P < 0.001). All AI models increased expert detection rates by up to 19% (95% CI: 15, 24) ( P < 0.001). Nonexpert+AI pairs could still outperform single experts by up to 13% (95% CI: 10, 17).
CONCLUSIONS: This study pioneers the comparative evaluation of humans and AI for FCD lesion detection. It shows that AI and human predictions differ, especially for certain MRI features of FCD, and, thus, how AI may complement the diagnostic workup.
引用
- Walger L, Bauer T, Kügler D, et al. A Quantitative Comparison Between Human and Artificial Intelligence in the Detection of Focal Cortical Dysplasia. Invest Radiol. 2024. PMID: 39437019. doi:10.1097/RLI.0000000000001125
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。