集成神经网络模型利用外部照片检测甲状腺眼病
本研究开发了一种深度学习集成模型,使用1944张临床数据库照片训练,344张独立测试集评估性能。模型在测试集上准确率89.2%、特异度86.9%、召回率93.4%、精确率79.7%、F1分数86.0%。在50张调查集上,模型各项指标均高于27名眼科医生的平均表现,且对中重度和活动期疾病的召回率更高。
要点
- 集成模型在测试集上准确率89.2%、特异度86.9%、召回率93.4%。
- 调查集上模型准确率86%、特异度84%、召回率89%,均高于27名眼科医生平均的75%、82%、63%。
- 模型对中重度(98.2%,n=55)和活动期(98.3%,n=60)疾病的召回率高于轻度(86.8%,n=68)和稳定期(85.7%,n=63)。
核心结论
- 深度学习分类器是识别甲状腺眼病的一种新方法,是开发提高诊断准确性、降低专科评估障碍工具的第一步。
研究设计与样本
- 开发深度学习模型,使用1944张来自临床数据库的照片进行训练,344张额外图像作为测试集计算性能指标。
- 从测试集中随机选取50张图像作为调查集,比较模型与27名眼科医生的表现。
- 另用来自独立临床数据库的222张图像评估模型召回率,并量化模型性能与疾病分期和分级的关系。
主要发现
- 模型在测试集上准确率89.2%、特异度86.9%、召回率93.4%、精确率79.7%、F1分数86.0%。
- 热力图显示模型识别出对应甲状腺眼病临床特征的像素。
- 调查集上集成模型准确率86%、特异度84%、召回率89%、精确率77%、F1分数82%;27名眼科医生平均分别为75%、82%、63%、72%、66%。
- 第二测试集上模型召回率91.9%,中重度疾病召回率98.2%(n=55)、活动期疾病98.3%(n=60),高于轻度86.8%(n=68)和稳定期85.7%(n=63)。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
PURPOSE: To describe an artificial intelligence platform that detects thyroid eye disease (TED). DESIGN: Development of a deep learning model. METHODS: 1944 photographs from a clinical database were used to train a deep learning model. 344 additional images ('test set') were used to calculate performance metrics. Receiver operating characteristic, precision-recall curves and heatmaps were generated. From the test set, 50 images were randomly selected ('survey set') and used to compare model performance with ophthalmologist performance. 222 images obtained from a separate clinical database were used to assess model recall and to quantitate model performance with respect to disease stage and grade. RESULTS: The model achieved test set accuracy of 89.2%, specificity 86.9%, recall 93.4%, precision 79.7% and an F1 score of 86.0%. Heatmaps demonstrated that the model identified pixels corresponding to clinical features of TED. On the survey set, the ensemble model achieved accuracy, specificity, recall, precision and F1 score of 86%, 84%, 89%, 77% and 82%, respectively. 27 ophthalmologists achieved mean performance of 75%, 82%, 63%, 72% and 66%, respectively. On the second test set, the model achieved recall of 91.9%, with higher recall for moderate to severe (98.2%, n=55) and active disease (98.3%, n=60), as compared with mild (86.8%, n=68) or stable disease (85.7%, n=63). CONCLUSIONS: The deep learning classifier is a novel approach to identify TED and is a first step in the development of tools to improve diagnostic accuracy and lower barriers to specialist evaluation.
引用
- Karlin J, Gai L, LaPierre N, et al. Ensemble neural network model for detecting thyroid eye disease using external photographs. Br J Ophthalmol. 2022. PMID: 36126104. doi:10.1136/bjo-2022-321833
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。