人机交互对胸片恶性肺结节检出的影响
本回顾性研究于2021年4月至6月开展两轮阅片,30名阅片者按第一轮无AI辅助时AUFROC相当分为两组,第二轮分别使用高或低准确度AI模型辅助重读胸片。高准确度AI使阅片者检出肺癌的效能提升(AUC 0.77→0.82,AUFROC 0.71→0.79),也使其更易受AI建议影响而改变判读。
要点
- 高准确度AI辅助使胸片肺癌检出AUC由0.77升至0.82,AUFROC由0.71升至0.79,低准确度AI无提升。
- 使用高准确度AI的阅片者改变原判读的比例为67%(224/334),高于低准确度AI组的59%(229/386)。
- 首轮判读准确、AI建议正确、AI准确度高及诊断难度与AI辅助判读准确相关,阅片者特征无关。
核心结论
- 高诊断准确度的AI模型可提升放射科医生在胸片上检出肺癌的效能,并增加其对AI建议的易感性。
研究设计与样本
- 回顾性研究,两轮阅片时间为2021年4月至2021年6月,共评估120张胸片,其中60张来自肺癌患者、60张来自对照。
- 肺癌患者平均年龄67岁±12(标准差),32名男性,共63个癌灶;对照组平均年龄67岁±12,36名男性。
- 30名阅片者按第一轮无AI辅助的AUFROC相当分为两组,包括20名胸科放射科医生(5-18年经验)和10名放射科住院医师(2-3年经验)。
- 第二轮每组分别使用高准确度或低准确度AI模型辅助重读,阅片者不知使用了两种不同AI模型。
- 采用广义线性混合模型分析影响AI辅助检出效能的因素,包括阅片者对AI的态度与经验及Grit评分。
诊断标准与关键数值
- 高准确度AI组AUC由0.77升至0.82,AUFROC由0.71升至0.79;低准确度AI组AUC为0.75至0.75,AUFROC为0.7至0.72。
- 使用高准确度AI的阅片者改变原判读的比例为67%(224/334),使用低准确度AI者为59%(229/386)。
临床应用与阅片要点
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
Background The factors affecting radiologists' diagnostic determinations in artificial intelligence (AI)-assisted image reading remain underexplored. Purpose To assess how AI diagnostic performance and reader characteristics influence detection of malignant lung nodules during AI-assisted reading of chest radiographs. Materials and Methods This retrospective study consisted of two reading sessions from April 2021 to June 2021. Based on the first session without AI assistance, 30 readers were assigned into two groups with equivalent areas under the free-response receiver operating characteristic curve (AUFROCs). In the second session, each group reinterpreted radiographs assisted by either a high or low accuracy AI model (blinded to the fact that two different AI models were used). Reader performance for detecting lung cancer and reader susceptibility (changing the original reading following the AI suggestion) were compared. A generalized linear mixed model was used to identify the factors influencing AI-assisted detection performance, including readers' attitudes and experiences of AI and Grit score. Results Of the 120 chest radiographs assessed, 60 were obtained in patients with lung cancer (mean age, 67 years ± 12 [SD]; 32 male; 63 cancers) and 60 in controls (mean age, 67 years ± 12; 36 male). Readers included 20 thoracic radiologists (5-18 years of experience) and 10 radiology residents (2-3 years of experience). Use of the high accuracy AI model improved readers' detection performance to a greater extent than use of the low accuracy AI model (area under the receiver operating characteristic curve, 0.77 to 0.82 vs 0.75 to 0.75; AUFROC, 0.71 to 0.79 vs 0.7 to 0.72). Readers who used the high accuracy AI showed a higher susceptibility (67%, 224 of 334 cases) to changing their diagnosis based on the AI suggestions than those using the low accuracy AI (59%, 229 of 386 cases). Accurate readings at the first session, correct AI suggestions, high accuracy Al, and diagnostic difficulty were associated with accurate AI-assisted readings, but readers' characteristics were not. Conclusion An AI model with high diagnostic accuracy led to improved performance of radiologists in detecting lung cancer on chest radiographs and increased radiologists' susceptibility to AI suggestions. © RSNA, 2023 Supplemental material is available for this article.
引用
- Lee JH, Hong H, Nam G, et al. Effect of Human-AI Interaction on Detection of Malignant Lung Nodules on Chest Radiographs. Radiology. 2023. PMID: 37367443. doi:10.1148/radiol.222976
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。