指南与共识骨肌技术与安全US

超声股神经分割AI与人类专家的共识校准框架

Calibrating artificial intelligence against human expertise using femoral nerve segmentation on ultrasound: a consensus framework.
指南 / 共识 · Anaesthesia · 2026-09-01

针对超声图像上股神经分割缺乏统一验证标准的问题,本研究建立多阅片者共识框架,由9名临床医生独立标注100幅超声图像,以多数投票形成共识标准。以该共识评估AI模型,并与个体人类表现比较,AI达到了人类专家表现上限区间。该框架不依赖特定AI系统,可用于不存在单一正确解读的临床影像场景。

要点

  • 该AI对股神经检出敏感度很高,但精确度偏低,常出现超出目标范围的过度分割,临床应用时需逐幅核对。
  • 超声股神经边界即使在专家之间也存在明显分歧,报告或教学时不宜视为唯一绝对轮廓。
  • 用多阅片者共识而非单一标注评估AI更可靠,可作为放射科验证影像AI的参考方法。

核心结论

  • AI模型相对联合共识的中位Dice系数为0.72,匹配中位人类表现并优于9名标注者中的5名。
  • AI敏感度为0.94、精确度为0.60,提示神经检出可靠但存在仍可临床解释的过度分割。

研究设计与样本

  • 多阅片者共识研究,9名临床医生对100幅股神经超声图像独立分割,共产生900个标注。
  • 以多数投票形成的联合共识作为参考标准,评估一个学术深度学习模型并与个体人类表现比较。
  • 通过留一标注者分析检验共识稳定性,并以Fleiss κ评估阅片者间一致性。

关键影像表现/征象

  • AI模型在超声图像上几乎总能检出股神经,但常勾画为略大于所需的区域,表现为过度分割。
  • 股神经超声分割存在固有主观性,专家间整体一致性为中度,神经起止边界无法形成唯一确定轮廓。

诊断标准与关键数值

  • AI相对联合共识的中位Dice系数为0.72,四分位距0.56至0.84,范围0.00至0.91。
  • AI敏感度为0.94,四分位距0.88至0.97,范围0.33至1.00;精确度为0.60,四分位距0.44至0.76,范围0.00至0.89。
  • 人类个体Dice系数范围为0.32至0.73,中位数为0.60。
  • AI较5名标注者的相对改善幅度为31%至125%,对表现最低者获益最大。
  • 留一标注者分析中AI Dice中位数为0.749,四分位距0.745至0.752,范围0.742至0.769;Fleiss κ为0.54,p<0.001。

临床应用与阅片要点

  • 使用此类AI辅助股神经勾画时,超声医师必须检查AI所示范围,避免直接采纳过度分割边界。
  • 神经阻滞或教学标注应认识股神经超声分割的中度观察者间差异,避免以单一操作者轮廓作为金标准。
  • 该共识校准框架可迁移到其他缺乏单一正确解读的放射与超声影像AI性能评估中。

局限性

  • 以标注者共同定义的共识评价标注者本身存在循环性问题,限制了AI与人类得分的直接比较。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

INTRODUCTION: Meaningful validation of artificial intelligence for medical image interpretation requires comparison against human expert performance, yet multi-rater frameworks establishing such comparisons remain uncommon.

METHODS: We developed and applied a consensus framework using nine clinicians who independently segmented the femoral nerve on 100 ultrasound images, yielding 900 annotations and a combined consensus standard established by majority voting. We then evaluated an academic deep learning model against this consensus and individual human performance.

RESULTS: The artificial intelligence model achieved a median (IQR [range]) Dice coefficient of 0.72 (0.56-0.84 [0.00-0.91]) against combined consensus. Sensitivity was 0.94 (0.88-0.97 [0.33-1.00]) and precision 0.60 (0.44-0.76 [0.00-0.89]). Individual human Dice scores ranged from 0.32 to 0.73 (median 0.60). The artificial intelligence model matched median human performance and outperformed five of nine annotators (31%-125% relative improvement), with the greatest benefit for the lowest-performing practitioners. Leave-one-annotator-out analysis confirmed consensus stability (median (IQR [range]) artificial intelligence Dice 0.749 (0.745-0.752 [0.742-0.769])). Inter-rater reliability was moderate overall (Fleiss's κ 0.54, p < 0.001).

DISCUSSION: The sensitivity and precision profile of the artificial intelligence model indicated reliable nerve detection with over-segmentation that remained clinically interpretable. The moderate inter-rater reliability is consistent with the inherent subjectivity of nerve delineation on ultrasound. The circularity inherent in evaluating annotators against a consensus they helped define limits direct comparison of artificial intelligence and human scores. A Dice score of 0.72 represents the upper range of human expert performance rather than moderate accuracy. The framework methodology is independent of the specific artificial intelligence system evaluated and offers a transferable approach for calibrating artificial intelligence performance in clinical imaging where no single correct interpretation exists.

WHAT WE DID: We tested our own created artificial intelligence program that finds a leg nerve on ultrasound scans. Doctors use this nerve, called the femoral nerve, to guide a small injection of numbing medicine. The injection blocks pain after a broken hip or leg surgery. We collected 100 ultrasound images of this nerve. Doctors and experienced specialists then outlined the nerve on every image. The artificial intelligence also outlined the nerve on each image. We compared the outlines created by the artificial intelligence and the doctors.

WHY DID WE DO IT: Finding a nerve on ultrasound is difficult. Even experienced doctors disagree about exactly where a nerve begins and ends. For doctors who are in training, this can mean the injection works less reliably. Artificial intelligence tools may help, but until now there is no one way to test them. Most studies compare artificial intelligence to a single outline made by a doctor, which does not allow for the natural differences between doctors. We wanted another method, so we tested the artificial intelligence against the combined judgement of nine doctors, rather than just one.

WHAT WE FOUND: The artificial intelligence matched the typical doctor in our group and performed better than five of the nine. The benefit was greatest for the doctors in training. The artificial intelligence nearly always found the nerve, but it often outlined a slightly larger area than needed. Doctors should still check what the artificial intelligence shows before using it on a patient. Our method can also be used to test other medical artificial intelligence tools in the same way.

来源:Anaesthesia,查看原文。版权归原作者所有。

引用

  1. Berggreen, Johansson, Möller, et al. Calibrating artificial intelligence against human expertise using femoral nerve segmentation on ultrasound: a consensus framework. Anaesthesia. 2026. PMID: 42680190. doi:10.1111/anae.70364

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。