评估现代学习方法的分类准确性
本文针对多层感知机(MLP)和卷积神经网络(CNN)等深度学习模型,提供评估分类准确性的教程,重点讨论多类别响应变量下传统指标(如敏感度、特异度、ROC曲线下面积)的扩展,并用真实医学示例展示统计计算步骤与R代码。
要点
- 深度学习模型分类准确性评估对实践者尚不便捷,需专门统计方法。
- 多类别分类时敏感度、特异度、AUC等传统指标不适用,需用扩展指标。
- 提供基于问题的R代码逐步演示多类别分类准确性统计计算。
核心结论
- 多类别分类准确性评估需采用扩展的统计指标,传统二分类指标不适用。
- 提供的R代码教程有助于实践者掌握深度学习分类准确性评估。
研究设计与样本
- 教程性文章,回顾多类别分类准确性的重要统计概念。
- 以真实医学示例演示多种学习算法的统计计算,并提供R代码。
主要发现
- 多层感知机(MLP)和卷积神经网络(CNN)是新颖且有吸引力的人工智能计算工具。
- 对于多于两类的定性响应变量,敏感度、特异度和ROC曲线下面积等传统指标不适用,需考虑其扩展。
临床怎么用
- 实践者可利用文中R代码逐步进行多类别分类准确性的统计计算。
- 评估深度学习模型分类性能时应选用适当扩展的准确性指标。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Deep learning neural network models such as multilayer perceptron (MLP) and convolutional neural network (CNN) are novel and attractive artificial intelligence computing tools. However, evaluation of the performance of these methods is not readily available for practitioners yet. We provide a tutorial for evaluating classification accuracy for various state-of-the-art learning approaches, including familiar shallow and deep learning methods. For qualitative response variables with more than two categories, many traditional accuracy measures such as sensitivity, specificity, and area under the receiver operating characteristic curve are not applicable and we have to consider their extensions properly. In this paper, a few important statistical concepts for multicategory classification accuracy are reviewed and their utilities for various learning algorithms are demonstrated with real medical examples. We offer problem-based R code to illustrate how to perform these statistical computations step by step. We expect that such analysis tools will become more familiar to practitioners and receive broader applications in biostatistics.
引用
- Li J, Gao M, D'Agostino R, et al. Evaluating classification accuracy for modern learning approaches. Stat Med. 2019. PMID: 30701585. doi:10.1002/sim.8103
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。