患者背景皮肤图像对黑色素瘤诊断的影响
该研究基于2020年SIIM-ISIC黑色素瘤分类挑战赛,评估患者背景病灶图像能否提升黑色素瘤诊断准确率。3308份AI参赛方案中前50名AUROC为0.943至0.949,多数算法独立评估图像而未利用患者内病灶模式。人类读片者获得7张背景图像前后中位敏感度均为60.0%,特异度由86.7%降至85.7%,提示患者背景图像未带来正向影响。
要点
- AI算法AUROC达0.943至0.949,但多数未利用患者内病灶模式。
- 人类读片者获得背景图像前后敏感度均为60.0%。
- 提供7张背景图像而未提供全身图像,可能不足以检验该策略。
核心结论
- 患者背景病灶图像未提升AI算法或人类读片者的黑色素瘤诊断表现。
- 该研究提供了经多中心评估的开源先进黑色素瘤检测算法。
研究设计与样本
- 2020年国际在线AI竞赛,任务为将皮肤镜图像分类为黑色素瘤或良性病变。
- 提供按患者分组的多来源皮肤镜图像数据集,并允许使用公共数据集。
- 同期使用部分测试数据开展人类读片研究,读片者先诊断索引病例,再查看该患者7张其他病灶图像后再次预测。
主要发现
- 3308份AI参赛方案中前50名在私榜上的AUROC为0.943至0.949。
- 少数算法考虑患者内病灶模式,多数算法独立评估图像。
- 人类读片者获得背景图像前中位敏感度和特异度分别为60.0%和86.7%,获得后分别为60.0%和85.7%。
- 人类与AI算法表现随图像来源不同而变化。
临床怎么用
- 当前证据不支持仅凭额外患者背景病灶图像来提升黑色素瘤诊断准确率。
- AI辅助诊断工具开发时应重视患者整体信息的整合方式而非单纯增加图像数量。
局限性
- 提供7张背景图像而未提供全身图像,可能不足以检验患者内病灶模式策略的适用性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: While the high accuracy of reported AI tools for melanoma detection is promising, the lack of holistic consideration of the patient is often criticized. Along with medical history, a dermatologist would also consider intra-patient nevi patterns, such that nevi that are different from others on a given patient are treated with suspicion. OBJECTIVE: To evaluate whether patient-contextual lesion-images improves diagnostic accuracy for melanoma in a dermoscopic image-based AI competition and a human reader study. METHODS: An international online AI competition was held in 2020. The task was to classify dermoscopy images as melanoma or benign lesions. A multi-source dataset of dermoscopy images grouped by patient were provided, and additional use of public datasets was permitted. Competitors were judged on area under the receiver operating characteristic (AUROC) on a private leaderboard. Concurrently, a human reader study was hosted using a subset of the test data. Participants gave their initial diagnosis of an index case (melanoma vs. benign) and were then presented with seven additional lesion-images of that patient before giving a second prediction of the index case. Outcome measures were sensitivity and specificity. RESULTS: The top 50 of 3308 AI competition entries achieved AUROC scores ranging from 0.943 to 0.949. Few algorithms considered intra-patient lesion patterns and instead most evaluated images independently. The median sensitivity and specificity of human readers before receiving contextual images were 60.0% and 86.7%, and after were 60.0% and 85.7%. Human and AI algorithm performance varied by image source. CONCLUSIONS: This study provided an open-source state-of-the-art algorithm for melanoma detection that has been evaluated at multiple centres. Patient-contextual images did not positively impact performance of AI algorithms or human readers. Providing seven contextual images and no total body image may have been insufficient to test the applicability of the intra-patient lesion patterns.
引用
- Kurtansky NR, Primiero CA, Betz-Stablein B, et al. Effect of patient-contextual skin images in human- and artificial intelligence-based diagnosis of melanoma: Results from the 2020 SIIM-ISIC melanoma classification challenge. J Eur Acad Dermatol Venereol. 2024. PMID: 39648687. doi:10.1111/jdv.20479
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。