甲状腺超声人工智能模型的泛化性与诊断性能
这项回顾性研究纳入中国208家医院、12个厂商设备的10023例病理确诊甲状腺结节患者,开发检测、分割与分类AI模型;分类模型AUC为0.90,基于规则AI辅助可提升各级放射科医师诊断准确率。
要点
- 检测、分割、分类模型平均精确率0.98、Dice系数0.86、AUC 0.90。
- 全国数据训练的分割模型Dice系数0.91,混合厂商数据训练的分类模型AUC 0.98。
- 基于规则AI辅助可提升所有放射科医师诊断准确率(P < .05)。
核心结论
- 基于多样数据开发的甲状腺超声AI模型在中国人群中具有高诊断性能,基于规则的AI辅助可提升放射科医师诊断甲状腺癌的准确率。
研究设计与样本
- 回顾性研究,纳入2017年11月至2019年1月中国208家医院、12个厂商超声设备的连续病理确诊甲状腺结节患者10023例(中位年龄46岁,IQR 37-55岁;女性7669例)。
- 基于部分或全部图像开发检测、分割和分类模型,以精确率与召回率、Dice系数、AUC评价性能。
- 比较无AI辅助、自由式AI辅助和基于规则AI辅助三种场景下3名高年资与3名低年资放射科医师的诊断表现。
诊断标准与关键数值
- 检测模型平均精确率0.98(95% CI:0.96,0.99)。
- 分割模型Dice系数0.86(95% CI:0.86,0.87);全国数据训练的分割模型Dice系数0.91(95% CI:0.90,0.91)。
- 分类模型AUC 0.90(95% CI:0.88,0.92);混合厂商数据训练的分类模型AUC 0.98(95% CI:0.97,1.00)。
临床应用与阅片要点
- AI模型在所有比较中均优于高年资和低年资放射科医师(P < .05)。
- 基于规则的AI辅助使所有放射科医师的诊断准确率均得到提升(P < .05)。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
Background Artificial intelligence (AI) models have improved US assessment of thyroid nodules; however, the lack of generalizability limits the application of these models. Purpose To develop AI models for segmentation and classification of thyroid nodules in US using diverse data sets from nationwide hospitals and multiple vendors, and to measure the impact of the AI models on diagnostic performance. Materials and Methods This retrospective study included consecutive patients with pathologically confirmed thyroid nodules who underwent US using equipment from 12 vendors at 208 hospitals across China from November 2017 to January 2019. The detection, segmentation, and classification models were developed based on the subset or complete set of images. Model performance was evaluated by precision and recall, Dice coefficient, and area under the receiver operating characteristic curve (AUC) analyses. Three scenarios (diagnosis without AI assistance, with freestyle AI assistance, and with rule-based AI assistance) were compared with three senior and three junior radiologists to optimize incorporation of AI into clinical practice. Results A total of 10 023 patients (median age, 46 years [IQR 37-55 years]; 7669 female) were included. The detection, segmentation, and classification models had an average precision, Dice coefficient, and AUC of 0.98 (95% CI: 0.96, 0.99), 0.86 (95% CI: 0.86, 0.87), and 0.90 (95% CI: 0.88, 0.92), respectively. The segmentation model trained on the nationwide data and classification model trained on the mixed vendor data exhibited the best performance, with a Dice coefficient of 0.91 (95% CI: 0.90, 0.91) and AUC of 0.98 (95% CI: 0.97, 1.00), respectively. The AI model outperformed all senior and junior radiologists (P < .05 for all comparisons), and the diagnostic accuracies of all radiologists were improved (P < .05 for all comparisons) with rule-based AI assistance. Conclusion Thyroid US AI models developed from diverse data sets had high diagnostic performance among the Chinese population. Rule-based AI assistance improved the performance of radiologists in thyroid cancer diagnosis. © RSNA, 2023 Supplemental material is available for this article.
引用
- Xu W, Jia X, Mei Z, et al. Generalizability and Diagnostic Performance of AI Models for Thyroid US. Radiology. 2023. PMID: 37338356. doi:10.1148/radiol.221157
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。