文献速递区域麻醉与超声

人类专家与人工智能在区域麻醉超声解剖结构识别中的变异性:辅助人工智能评估框架

Variability between human experts and artificial intelligence in identification of anatomical structures by ultrasound in regional anaesthesia: a framework for evaluation of assistive artificial intelligence.
PubMed 文献 · Br J Anaesth · 2023-10-26

结果显示,人类专家与AI在动脉识别上一致性最高(平均Dice评分0.88/0.86),肌肉次之(0.80/0.77),神经最低(0.48/0.41);不同结构的一致性差异很大。人类专家与AI系统在超声解剖结构识别中表现出相同的 agreement 模式,但人类专家意见的均匀性受到挑战。

要点

  • 人类专家与AI在动脉识别上一致性最高(Dice 0.88/0.86),神经最低(0.48/0.41)。
  • 不同解剖结构的一致性差异大,最高为缝匠肌(0.91/0.92),最低为桡神经(0.21/0.27)。
  • 人类专家意见并非均匀一致,临床意义需进一步探索。

核心结论

  • 人类专家和AI系统在超声解剖结构识别中表现出相同的一致性模式。
  • 人类专家意见的均匀性受到挑战,需探索所呈现差异的临床意义。

研究设计与样本

  • 将专家标注相互比较以量化人类专家间的一致性,并将AI彩色叠加与所有专家标注进行比较。

主要发现

  • 人类-人类和人类-AI标注的一致性最高为动脉(平均Dice评分0.88/0.86),其次为肌肉(0.80/0.77),最低为神经(0.48/0.41)。
  • 不同结构的一致性差异很大,最高为缝匠肌(0.91/0.92),最低为桡神经(0.21/0.27)。

临床怎么用

  • 该评估框架可用于其他设备,以进行一致的评估,为临床培训和实践提供信息。
  • 麻醉医师应积极参与新AI技术的开发和采用。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: ScanNavTMAnatomy Peripheral Nerve Block (ScanNav™) is an artificial intelligence (AI)-based device that produces a colour overlay on real-time B-mode ultrasound to highlight key anatomical structures for regional anaesthesia. This study compares consistency of identification of sono-anatomical structures between expert ultrasonographers and ScanNav™. METHODS: Nineteen experts in ultrasound-guided regional anaesthesia (UGRA) annotated 100 structures in 30 ultrasound videos across six anatomical regions. These annotations were compared with each other to produce a quantitative assessment of the level of agreement amongst human experts. The AI colour overlay was then compared with all expert annotations. Differences in human-human and human-AI agreement are presented for each structure class (artery, muscle, nerve, fascia/serosal plane) and structure. Clinical context is provided through subjective assessment data from UGRA experts. RESULTS: For human-human and human-AI annotations, agreement was highest for arteries (mean Dice score 0.88/0.86), then muscles (0.80/0.77), and lowest for nerves (0.48/0.41). Wide discrepancy exists in consistency for different structures, both with human-human and human-AI comparisons; highest for sartorius muscle (0.91/0.92) and lowest for the radial nerve (0.21/0.27). CONCLUSIONS: Human experts and the AI system both showed the same pattern of agreement in sono-anatomical structure identification. The clinical significance of the differences presented must be explored; however the perception that human expert opinion is uniform must be challenged. Elements of this assessment framework could be used for other devices to allow consistent evaluations that inform clinical training and practice. Anaesthetists should be actively engaged in the development and adoption of new AI technology.

来源:Br J Anaesth,查看原文。版权归原作者所有。

引用

  1. Bowness JS, Morse R, Lewis O, et al. Variability between human experts and artificial intelligence in identification of anatomical structures by ultrasound in regional anaesthesia: a framework for evaluation of assistive artificial intelligence. Br J Anaesth. 2023. PMID: 39492288. doi:10.1016/j.bja.2023.09.023

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。