市场批准CNN与96名皮肤科医生在广泛皮肤病变分类中的对比研究
临床近照+皮肤镜+文本信息)判读,并与欧洲市场批准的CNN(Moleanalyzer Pro)对比。CNN灵敏度95.0%、特异度76.7%、AUC 0.918;在二级信息下医生灵敏度94.1%、特异度80.4%,CNN特异度固定为80.4%时灵敏度与医生几乎相等(95% vs 94.1%,P=0.1)。结论:在更接近真实的条件下,CNN与多数皮肤科医生表现相当,经验丰富的医生常超越CNN。
要点
- CNN灵敏度95.0%、特异度76.7%、AUC 0.918,与医生二级管理决策相当。
- 医生二级信息下灵敏度94.1%、特异度80.4%,特异度固定后CNN灵敏度与医生无显著差异。
- 经验丰富的皮肤科医生常超越CNN表现,单图对比研究不足以评价医生综合能力。
核心结论
- 在更接近真实的条件下和更广泛的诊断谱中,CNN与多数皮肤科医生表现处于同一水平。
- 皮肤科医生善于整合多源信息,仅基于单张病例图像的对比研究不足以评价其能力。
研究设计与样本
- 一级仅用皮肤镜;二级用临床近照、皮肤镜和文本信息。
- CNN为欧洲市场批准的医疗器械Moleanalyzer Pro(FotoFinder Systems,德国Bad Birnbach),对皮肤镜图像进行分类。
主要发现
- CNN灵敏度95.0%(95%CI 83.5%–98.6%)、特异度76.7%(95%CI 64.6%–85.6%)、ROC AUC 0.918(95%CI 0.866–0.970)。
- 医生一级管理决策平均灵敏度89.0%(95%CI 87.4%–90.6%)、特异度80.7%(95%CI 78.8%–82.6%)。
- 医生二级信息下灵敏度显著提高至94.1%(95%CI 93.1%–95.1%;P<0.001),特异度保持80.4%(95%CI 78.4%–82.4%;P=0.97)。
- CNN特异度固定为医生二级平均特异度80.4%时,CNN灵敏度95%(95%CI 83.5%–98.6%)与医生94.1%(95%CI 93.1%–95.1%)几乎相等(P=0.1)。
- 医生在一级管理决策及一级、二级诊断决策中表现不及CNN;经验丰富的医生常超越CNN。
临床怎么用
- 市场批准的CNN可作为皮肤病变分类的辅助工具,其灵敏度与特异度与多数皮肤科医生相当。
- 临床判读应整合临床近照、皮肤镜和文本信息等多源资料,而非仅依赖单张图像。
- 经验丰富的医生可结合CNN辅助提升决策质量。
局限性
- 对比条件仍非完全真实临床场景,医生分级信息设置可能影响结果外推。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: Convolutional neural networks (CNNs) efficiently differentiate skin lesions by image analysis. Studies comparing a market-approved CNN in a broad range of diagnoses to dermatologists working under less artificial conditions are lacking. MATERIALS AND METHODS: One hundred cases of pigmented/non-pigmented skin cancers and benign lesions were used for a two-level reader study in 96 dermatologists (level I: dermoscopy only; level II: clinical close-up images, dermoscopy, and textual information). Additionally, dermoscopic images were classified by a CNN approved for the European market as a medical device (Moleanalyzer Pro, FotoFinder Systems, Bad Birnbach, Germany). Primary endpoints were the sensitivity and specificity of the CNN's dichotomous classification in comparison with the dermatologists' management decisions. Secondary endpoints included the dermatologists' diagnostic decisions, their performance according to their level of experience, and the CNN's area under the curve (AUC) of receiver operating characteristics (ROC). RESULTS: The CNN revealed a sensitivity, specificity, and ROC AUC with corresponding 95% confidence intervals (CI) of 95.0% (95% CI 83.5% to 98.6%), 76.7% (95% CI 64.6% to 85.6%), and 0.918 (95% CI 0.866-0.970), respectively. In level I, the dermatologists' management decisions showed a mean sensitivity and specificity of 89.0% (95% CI 87.4% to 90.6%) and 80.7% (95% CI 78.8% to 82.6%). With level II information, the sensitivity significantly improved to 94.1% (95% CI 93.1% to 95.1%; P < 0.001), while the specificity remained unchanged at 80.4% (95% CI 78.4% to 82.4%; P = 0.97). When fixing the CNN's specificity at the mean specificity of the dermatologists' management decision in level II (80.4%), the CNN's sensitivity was almost equal to that of human raters, at 95% (95% CI 83.5% to 98.6%) versus 94.1% (95% CI 93.1% to 95.1%); P = 0.1. In contrast, dermatologists were outperformed by the CNN in their level I management decisions and level I and II diagnostic decisions. More experienced dermatologists frequently surpassed the CNN's performance. CONCLUSIONS: Under less artificial conditions and in a broader spectrum of diagnoses, the CNN and most dermatologists performed on the same level. Dermatologists are trained to integrate information from a range of sources rendering comparative studies that are solely based on one single case image inadequate.
引用
- Haenssle HA, Fink C, Toberer F, et al. Man against machine reloaded: performance of a market-approved convolutional neural network in classifying a broad spectrum of skin lesions in comparison with 96 dermatologists working under less artificial conditions. Ann Oncol. 2020. PMID: 31912788. doi:10.1016/j.annonc.2019.10.013
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。