文献速递神经核医学

在训练卷积神经网络时纳入标签不确定性可提高多巴胺转运体SPECT中确定与不确定病例的判别性能

Incorporating label uncertainty during the training of convolutional neural networks improves performance for the discrimination between certain and inconclusive cases in dopamine transporter SPECT.
PubMed 文献 · Eur J Nucl Med Mol Imaging · 2024-11-27

本研究回顾性使用三个数据集,开发集1740例用于CNN训练、验证和测试,两个独立分布外数据集640例和645例仅用于测试。以三名训练有素的读者视觉判读为参考,分别采用随机投票、平均投票或多数投票训练ResNet-18进行二分类,评估在确定和不确定sigmoid输出上的平衡准确率。结果显示随机投票和平均投票训练较多数投票训练在保证确定病例平衡准确率时需接受的不确定测试病例比例更低,且整体平衡准确率略高。

要点

  • 随机投票和平均投票训练较多数投票训练降低需接受的不确定测试病例比例。
  • 随机投票和平均投票训练在所有测试病例中平衡准确率略高于多数投票训练。
  • 将读者间分歧纳入训练可提高sigmoid输出判别确定与不确定病例的效用,且不损害整体准确率。

核心结论

  • 在训练中让CNN知晓读者间分歧可提高其sigmoid输出判别确定与不确定病例的效用,这些不确定病例在严格应用预设决策阈值时可能被CNN误分类。
  • 该方法不损害整体准确率。

研究设计与样本

  • 回顾性研究,使用三个数据集:开发集n=1740用于CNN训练、验证和测试,两个独立分布外数据集n=640和n=645仅用于测试。
  • 开发集中由三名训练有素的读者仔细进行基于视觉判读的二分类。
  • 训练ResNet-18架构进行DAT-SPECT二分类,参考标准分别为三名读者中随机选择的投票(随机投票训练)、「减低」投票的比例(平均投票训练)或多数投票(多数投票训练)。

主要发现

  • 在开发集确定测试病例达到98%平衡准确率时,需接受的不确定测试病例比例:随机投票训练1.9%和平均投票训练1.2%,均低于多数投票训练的2.8%。
  • 所有测试病例(无论确定性)的平衡准确率:随机投票训练97.3%和平均投票训练97.5%,均略高于多数投票训练的97.0%(开发集)。
  • 在所有数据集中,随机投票训练和平均投票训练需接受的不确定测试病例比例均低于多数投票训练。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

PURPOSE: Deep convolutional neural networks (CNN) hold promise for assisting the interpretation of dopamine transporter (DAT)-SPECT. For improved communication of uncertainty to the user it is crucial to reliably discriminate certain from inconclusive cases that might be misclassified by strict application of a predefined decision threshold on the CNN output. This study tested two methods to incorporate existing label uncertainty during the training to improve the utility of the CNN sigmoid output for this task. METHODS: Three datasets were used retrospectively: a "development" dataset (n = 1740) for CNN training, validation and testing, two independent out-of-distribution datasets (n = 640, 645) for testing only. In the development dataset, binary classification based on visual inspection was performed carefully by three well-trained readers. A ResNet-18 architecture was trained for binary classification of DAT-SPECT using either a randomly selected vote ("random vote training", RVT), the proportion of "reduced" votes ( "average vote training", AVT) or the majority vote (MVT) across the three readers as reference standard. Balanced accuracy was computed separately for "inconclusive" sigmoid outputs (within a predefined interval around the 0.5 decision threshold) and for "certain" (non-inconclusive) sigmoid outputs. RESULTS: The proportion of "inconclusive" test cases that had to be accepted to achieve a given balanced accuracy in the "certain" test case was lower with RVT and AVT than with MVT in all datasets (e.g., 1.9% and 1.2% versus 2.8% for 98% balanced accuracy in "certain" test cases from the development dataset). In addition, RVT and AVT resulted in slightly higher balanced accuracy in all test cases independent of their certainty (97.3% and 97.5% versus 97.0% in the development dataset). CONCLUSION: Making between-readers-discrepancy known to CNN during the training improves the utility of their sigmoid output to discriminate certain from inconclusive cases that might be misclassified by the CNN when the predefined decision threshold is strictly applied. This does not compromise on overall accuracy.

来源:Eur J Nucl Med Mol Imaging,查看原文。版权归原作者所有。

引用

  1. Kucerenko A, Buddenkotte T, Apostolova I, et al. Incorporating label uncertainty during the training of convolutional neural networks improves performance for the discrimination between certain and inconclusive cases in dopamine transporter SPECT. Eur J Nucl Med Mol Imaging. 2024. PMID: 39592475. doi:10.1007/s00259-024-06988-0

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。