文献速递皮肤肿瘤

通过评估不同皮肤光型和罕见皮肤癌类型的报告情况来确定机器学习模型的临床适用性:一项系统综述

Determining the clinical applicability of machine learning models through assessment of reporting across skin phototypes and rarer skin cancer types: A systematic review.
PubMed 文献 · J Eur Acad Dermatol Venereol · 2023-01-02

该系统综述检索了PubMed、Embase和CENTRAL,纳入2012年1月1日至2021年9月22日期间使用临床或皮肤镜图像评估皮肤癌诊断机器学习技术的114项研究。仅8/114项研究(7.0%)纳入了罕见皮肤癌,1/114项研究(0.9%)分别报告了罕见皮肤癌的结果,1/114项研究(0.9%)报告了所有皮肤光型的性能。大多数机器学习模型未按罕见皮肤癌和不同皮肤光型分别报告性能。

要点

  • 仅7.0%(8/114)的研究纳入了罕见皮肤癌,0.9%(1/114)分别报告其结果。
  • 仅0.9%(1/114)的研究报告了所有皮肤光型的性能。
  • 最广泛使用的公开数据集是ISIC档案(57.0%,65/114),但最大数据集为私有。

核心结论

  • 本综述发现大多数机器学习模型未按罕见皮肤癌和不同皮肤光型分别报告性能。
  • 当前缺乏透明度不可接受,存在模型在准确率低的人群中被不当使用的风险。

研究设计与样本

  • 系统综述,检索PubMed、Embase和CENTRAL,纳入2012年1月1日至2021年9月22日期间使用临床或皮肤镜图像评估皮肤癌诊断机器学习技术的研究,无语言限制,共纳入114项研究。

主要发现

  • 罕见皮肤癌被8/114项研究(7.0%)纳入,罕见皮肤癌结果被1/114项研究(0.9%)分别报告。
  • 性能在所有皮肤光型中报告见于1/114项研究(0.9%),但皮肤光型I和VI的性能因测试数据集代表性极少而不确定(分别为9/3756和1/3756)。
  • 训练数据集方面,公开数据集最常用,最广泛使用的是国际皮肤成像协作组织(ISIC)档案(65/114项研究,57.0%),但最大数据集为私有。

临床怎么用

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

Machine learning (ML) models for skin cancer recognition may have variable performance across different skin phototypes and skin cancer types. Overall performance metrics alone are insufficient to detect poor subgroup performance. We aimed (1) to assess whether studies of ML models reported results separately for different skin phototypes and rarer skin cancers, and (2) to graphically represent the skin cancer training datasets used by current ML models. In this systematic review, we searched PubMed, Embase and CENTRAL. We included all studies in medical journals assessing an ML technique for skin cancer diagnosis that used clinical or dermoscopic images from 1 January 2012 to 22 September 2021. No language restrictions were applied. We considered rarer skin cancers to be skin cancers other than pigmented melanoma, basal cell carcinoma and squamous cell carcinoma. We identified 114 studies for inclusion. Rarer skin cancers were included by 8/114 studies (7.0%), and results for a rarer skin cancer were reported separately in 1/114 studies (0.9%). Performance was reported across all skin phototypes in 1/114 studies (0.9%), but performance was uncertain in skin phototypes I and VI from minimal representation of the skin phototypes in the test dataset (9/3756 and 1/3756, respectively). For training datasets, although public datasets were most frequently used, with the most widely used being the International Skin Imaging Collaboration (ISIC) archive (65/114 studies, 57.0%), the largest datasets were private. Our review identified that most ML models did not report performance separately for rarer skin cancers and different skin phototypes. A degree of variability in ML model performance across subgroups is expected, but the current lack of transparency is not justifiable and risks models being used inappropriately in populations in whom accuracy is low.

来源:J Eur Acad Dermatol Venereol,查看原文。版权归原作者所有。

引用

  1. Steele L, Tan XL, Olabi B, et al. Determining the clinical applicability of machine learning models through assessment of reporting across skin phototypes and rarer skin cancer types: A systematic review. J Eur Acad Dermatol Venereol. 2023. PMID: 36514990. doi:10.1111/jdv.18814

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。