人工智能的压力测试:基于正式超声心动图训练的深度学习模型能否准确解读床旁超声
本研究用急诊科获得的333段心脏床旁超声视频,测试基于正式超声心动图训练的最佳深度学习模型EchoNet-Dynamic能否准确分割左心室并预测射血分数。结果显示左心室分割平均Dice相似系数为0.72,射血分数分类的Cohen kappa系数为0.16,诊断心力衰竭的AUC为0.74,所有任务准确率均低于该模型已发表的基准。作者结论认为用于床旁超声的深度学习模型应使用床旁超声图像数据集训练,而此类数据集尚未公开。
要点
- 基于正式超声心动图训练的模型用于床旁超声时,射血分数分类一致性仅kappa 0.16。
- 左心室分割平均Dice相似系数0.72,诊断心力衰竭AUC 0.74。
- 视频质量改善可提升左心室分割和心衰诊断表现,但射血分数分类未变。
核心结论
- 基于正式超声心动图训练的深度学习模型在用于复苏期间采集的床旁超声图像时表现变差,所有任务准确率均低于已发表的EchoNet-Dynamic基准。
研究设计与样本
- 研究纳入急诊科心脏床旁超声检查的333段视频,对每段视频从一帧图像分割左心室并标注射血分数分类(正常、减低、严重减低),同时将图像质量分为最佳、合格、不合格,以此测试EchoNet-Dynamic模型的自动左心室分割和射血分数分类准确性。
诊断标准与关键数值
- 左心室分割平均Dice相似系数0.72(N=333;95%CI 0.70-0.74)。
- 预测与真实射血分数分类一致性Cohen kappa系数0.16(N=333)。
- 诊断心力衰竭的受试者工作特征曲线下面积0.74(N=333)。
- 射血分数分类标准为正常、减低或严重减低。
- 图像质量分级为最佳、合格或不合格。
临床应用与阅片要点
- 作者认为用于评估床旁超声的深度学习模型应使用床旁超声图像组成的数据集进行训练。
- 此类床旁超声数据集尚未公开可用。
局限性
- 所有任务上模型准确率均低于EchoNet-Dynamic已发表的基准。
- 模型性能在视频质量改善时提升,但射血分数分类性能未随质量改变。
- 用于评估床旁超声的深度学习模型所需的数据集尚未公开可用。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVES: To test if a deep learning (DL) model trained on echocardiography images could accurately segment the left ventricle (LV) and predict ejection fraction on apical 4-chamber images acquired by point-of-care ultrasound (POCUS).
METHODS: We created a dataset of 333 videos from cardiac POCUS exams acquired in the emergency department. For each video we derived two ground-truth labels. First, we segmented the LV from one image frame and second, we classified the EF as normal, reduced, or severely reduced. We then classified the media's quality as optimal, adequate, or inadequate. With this dataset we tested the accuracy of automated LV segmentation and EF classification by the best-in-class echocardiography trained DL model EchoNet-Dynamic.
RESULTS: The mean Dice similarity coefficient for LV segmentation was 0.72 (N = 333; 95% CI 0.70-0.74). Cohen's kappa coefficient for agreement between predicted and ground-truth EF classification was 0.16 (N = 333). The area under the receiver-operating curve for the diagnosis of heart failure was 0.74 (N = 333). Model performance improved with video quality for the tasks of LV segmentation and diagnosis of heart failure, but was unchanged with EF classification. For all tasks the model was less accurate than the published benchmarks for EchoNet-Dynamic.
CONCLUSIONS: Performance of a DL model trained on formal echocardiography worsened when challenged with images captured during resuscitations. DL models intended for assessing bedside ultrasound should be trained on datasets composed of POCUS images. Such datasets have yet to be made publicly available.
引用
- Crockett, Kelly, Brundage, et al. A Stress Test of Artificial Intelligence: Can Deep Learning Models Trained From Formal Echocardiography Accurately Interpret Point-of-Care Ultrasound? J Ultrasound Med. 2022. PMID: 35560254. doi:10.1002/jum.16007
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。