文献速递骨软组织肉瘤与黑色素瘤

比较人工智能算法与157名德国皮肤科医生:黑色素瘤分类基准

Comparing artificial intelligence algorithms to 157 German dermatologists: the melanoma classification benchmark.
PubMed 文献 · Eur J Cancer · 2019-02-22

该研究向12家德国大学医院的皮肤科医生发放电子问卷,每份含100张皮肤镜和100张临床图像(各80张痣和20张活检确诊黑色素瘤图像),要求给出处理决策。157名医生评估皮肤镜图像,总体敏感度74.1%、特异度60.0%、ROC 0.67;145名医生评估临床图像,敏感度89.4%、特异度64.4%、ROC 0.769,两种图像结果差异有统计学意义(P < 0.05)。作者提出首个公开黑色素瘤分类基准,用于比较人工智能算法与皮肤科医生的诊断性能。

要点

  • 157名皮肤科医生评估皮肤镜图像:敏感度74.1%、特异度60.0%、ROC 0.67(范围0.538-0.769)。
  • 145名皮肤科医生评估临床图像:敏感度89.4%、特异度64.4%、ROC 0.769(范围0.613-0.9)。
  • 两种图像测试集结果差异有统计学意义(P < 0.05),支持建立标准化基准的必要性。

核心结论

  • 作者提出首个公开的黑色素瘤分类基准,涵盖非皮肤镜和皮肤镜图像,用于比较人工智能算法与145名或157名皮肤科医生的诊断性能。
  • 作者认为该黑色素瘤分类基准应被视为白皮肤西方人群二分类算法黑色素瘤分类领域的参考标准。

研究设计与样本

  • 电子问卷发送至12家德国大学医院的皮肤科医生,每份问卷包含100张皮肤镜和100张临床图像(各80张痣和20张活检确诊黑色素瘤图像),均为开源图像。
  • 问卷记录皮肤科工作经验年数、是否进行皮肤检查、年龄、性别及在大学医院的职位或住院医师身份。
  • 对每张图像,要求皮肤科医生给出处理决策(治疗/活检病变或安抚患者)。
  • 主要结局指标为敏感度、特异度和受试者工作特征(ROC)。

主要发现

  • 157名皮肤科医生评估全部100张皮肤镜图像,总体敏感度74.1%、特异度60.0%、ROC 0.67(范围0.538-0.769)。
  • 145名皮肤科医生评估全部100张临床图像,总体敏感度89.4%、特异度64.4%、ROC 0.769(范围0.613-0.9)。
  • 两种测试集结果差异有统计学意义(P < 0.05),证实需要标准化基准。

关键数值与推荐

  • 皮肤镜图像:敏感度74.1%、特异度60.0%、ROC 0.67(范围0.538-0.769),评估者为157名皮肤科医生。
  • 临床图像:敏感度89.4%、特异度64.4%、ROC 0.769(范围0.613-0.9),评估者为145名皮肤科医生。
  • 两种测试集结果差异有统计学意义(P < 0.05)。

临床怎么用

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: Several recent publications have demonstrated the use of convolutional neural networks to classify images of melanoma at par with board-certified dermatologists. However, the non-availability of a public human benchmark restricts the comparability of the performance of these algorithms and thereby the technical progress in this field. METHODS: An electronic questionnaire was sent to dermatologists at 12 German university hospitals. Each questionnaire comprised 100 dermoscopic and 100 clinical images (80 nevi images and 20 biopsy-verified melanoma images, each), all open-source. The questionnaire recorded factors such as the years of experience in dermatology, performed skin checks, age, sex and the rank within the university hospital or the status as resident physician. For each image, the dermatologists were asked to provide a management decision (treat/biopsy lesion or reassure the patient). Main outcome measures were sensitivity, specificity and the receiver operating characteristics (ROC). RESULTS: Total 157 dermatologists assessed all 100 dermoscopic images with an overall sensitivity of 74.1%, specificity of 60.0% and an ROC of 0.67 (range = 0.538-0.769); 145 dermatologists assessed all 100 clinical images with an overall sensitivity of 89.4%, specificity of 64.4% and an ROC of 0.769 (range = 0.613-0.9). Results between test-sets were significantly different (P < 0.05) confirming the need for a standardised benchmark. CONCLUSIONS: We present the first public melanoma classification benchmark for both non-dermoscopic and dermoscopic images for comparing artificial intelligence algorithms with diagnostic performance of 145 or 157 dermatologists. Melanoma Classification Benchmark should be considered as a reference standard for white-skinned Western populations in the field of binary algorithmic melanoma classification.

来源:Eur J Cancer,查看原文。版权归原作者所有。

引用

  1. Brinker TJ, Hekler A, Hauschild A, et al. Comparing artificial intelligence algorithms to 157 German dermatologists: the melanoma classification benchmark. Eur J Cancer. 2019. PMID: 30802784. doi:10.1016/j.ejca.2018.12.016

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。