文献速递胸部技术与安全DR

向机器学习的局限:人工智能辅助并非住院医师胸部X线判读的有效学习工具

Learning from the machine: AI assistance is not an effective learning tool for resident education in chest x-ray interpretation.
PubMed 文献 · Eur Radiol · 2023-08-12

八名放射科住院医师判读500张胸片以检出五种异常,在完成150张后被分为两组,一组在200张判读中以计算机辅助检测(CADe)作为第二读者,另一组无CADe,最后两组再各判读150张无CADe的胸片。使用CADe期间该组敏感度、特异度和准确率均显著高于对照组,但去除CADe后两组间无显著差异。

要点

  • 以CADe作为第二读者时,住院医师整体敏感度53%(n=431/816)高于无CADe组的43%(n=349/816),p<0.001。
  • 使用CADe期间特异度94%(n=3206/3428)高于对照组90%(n=3127/3477),准确率86%(n=3637/4244)高于对照组81%(n=3476/4293),均p<0.001。
  • 撤除CADe后两组敏感度44%(n=309/696)vs 46%(n=317/696)(p=0.666)、特异度90% vs 90%(p=0.642)、准确率80% vs 80%(p=0.955)均无显著差异。

核心结论

  • CADe系统虽能提高放射科住院医师判读胸片的表现,但单独使用似乎并非有效的学习工具,且不应取代教学。

研究设计与样本

  • 八名放射科住院医师判读500张胸片,检测气胸、胸腔积液、肺泡综合征、肺结节和纵隔肿块五种异常;在完成150张后按表现和经验分为两组,组1在200张判读中使用CADe作为第二读者,组2为对照,随后两组再各判读150张无CADe的胸片。

诊断标准与关键数值

  • 干预前八名住院医师个体敏感度中位数43%(范围35-57%)、特异度90%(范围82-96%)、准确率81%(范围76-84%)。
  • 使用CADe期间组1整体敏感度53%(n=431/816)vs 对照组43%(n=349/816),p<0.001。
  • 使用CADe期间组1整体特异度94%(n=3206/3428)vs 对照组90%(n=3127/3477),p<0.001。
  • 使用CADe期间组1整体准确率86%(n=3637/4244)vs 对照组81%(n=3476/4293),p<0.001。
  • 撤除CADe后组1与组2整体敏感度44%(n=309/696)vs 46%(n=317/696)(p=0.666)、特异度90%(n=2294/2541)vs 90%(n=2285/2542)(p=0.642)、准确率80%(n=2603/3237)vs 80%(n=2602/3238)(p=0.955),均无显著差异。

临床应用与阅片要点

  • 作者结论认为,CADe系统单独使用似乎并非有效的学习工具,且不应取代教学。
  • 临床相关性声明指出,人工智能虽提高住院医师胸片判读表现,但不能单独用作学习工具,且不应取代专门教学。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

OBJECTIVES: To assess whether a computer-aided detection (CADe) system could serve as a learning tool for radiology residents in chest X-ray (CXR) interpretation.

METHODS: Eight radiology residents were asked to interpret 500 CXRs for the detection of five abnormalities, namely pneumothorax, pleural effusion, alveolar syndrome, lung nodule, and mediastinal mass. After interpreting 150 CXRs, the residents were divided into 2 groups of equivalent performance and experience. Subsequently, group 1 interpreted 200 CXRs from the "intervention dataset" using a CADe as a second reader, while group 2 served as a control by interpreting the same CXRs without the use of CADe. Finally, the 2 groups interpreted another 150 CXRs without the use of CADe. The sensitivity, specificity, and accuracy before, during, and after the intervention were compared.

RESULTS: Before the intervention, the median individual sensitivity, specificity, and accuracy of the eight radiology residents were 43% (range: 35-57%), 90% (range: 82-96%), and 81% (range: 76-84%), respectively. With the use of CADe, residents from group 1 had a significantly higher overall sensitivity (53% [n = 431/816] vs 43% [n = 349/816], p < 0.001), specificity (94% [i = 3206/3428] vs 90% [n = 3127/3477], p < 0.001), and accuracy (86% [n = 3637/4244] vs 81% [n = 3476/4293], p < 0.001), compared to the control group. After the intervention, there were no significant differences between group 1 and group 2 regarding the overall sensitivity (44% [n = 309/696] vs 46% [n = 317/696], p = 0.666), specificity (90% [n = 2294/2541] vs 90% [n = 2285/2542], p = 0.642), or accuracy (80% [n = 2603/3237] vs 80% [n = 2602/3238], p = 0.955).

CONCLUSIONS: Although it improves radiology residents' performances for interpreting CXRs, a CADe system alone did not appear to be an effective learning tool and should not replace teaching.

CLINICAL RELEVANCE STATEMENT: Although the use of artificial intelligence improves radiology residents' performance in chest X-rays interpretation, artificial intelligence cannot be used alone as a learning tool and should not replace dedicated teaching.

KEY POINTS: • With CADe as a second reader, residents had a significantly higher sensitivity (53% vs 43%, p < 0.001), specificity (94% vs 90%, p < 0.001), and accuracy (86% vs 81%, p < 0.001), compared to residents without CADe. • After removing access to the CADe system, residents' sensitivity (44% vs 46%, p = 0.666), specificity (90% vs 90%, p = 0.642), and accuracy (80% vs 80%, p = 0.955) returned to that of the level for the group without CADe.

来源:Eur Radiol,查看原文。版权归原作者所有。

引用

  1. Chassagnon G, Billet N, Rutten C, et al. Learning from the machine: AI assistance is not an effective learning tool for resident education in chest x-ray interpretation. Eur Radiol. 2023. PMID: 37572190. doi:10.1007/s00330-023-10043-1

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。