模型汤可提升皮肤镜皮肤癌分类器性能
研究构建模型汤(对七种图像分辨率微调后的预训练模型权重取平均),在包含内部留出集与外部组件的多源数据集上评估皮肤镜黑色素瘤-痣分类任务的泛化性、鲁棒性与校准。结果显示模型汤在外部组件上改善泛化与校准,在留出组件上保持性能;对扰动测试图像性能改善,对损坏测试图像性能持平。
要点
- 模型汤在外部组件上改善泛化与校准,在留出组件上保持性能。
- 对扰动测试图像性能改善,对损坏测试图像性能持平。
- 模型汤易于实践者实现,将多模型合并为单模型降低复杂性。
核心结论
- 模型汤对皮肤癌分类器的泛化性、鲁棒性与校准均有正面影响,且易于实现并降低复杂性。
研究设计与样本
- 在七种不同图像分辨率上微调预训练模型,随后对其权重取平均构建模型汤。
- 在包含留出组件与外部组件的多源数据集上评估性能。
主要发现
- 模型汤在外部组件上改善泛化与校准,同时在留出组件上保持性能。
- 对扰动测试图像观察到性能改善,对损坏测试图像性能持平。
临床怎么用
- 作者认为模型汤易于实践者实现,合并多模型为单模型可降低复杂性,可能有助于实现临床适用性。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: Image-based cancer classifiers suffer from a variety of problems which negatively affect their performance. For example, variation in image brightness or different cameras can already suffice to diminish performance. Ensemble solutions, where multiple model predictions are combined into one, can improve these problems. However, ensembles are computationally intensive and less transparent to practitioners than single model solutions. Constructing model soups, by averaging the weights of multiple models into a single model, could circumvent these limitations while still improving performance. OBJECTIVE: To investigate the performance of model soups for a dermoscopic melanoma-nevus skin cancer classification task with respect to (1) generalisation to images from other clinics, (2) robustness against small image changes and (3) calibration such that the confidences correspond closely to the actual predictive uncertainties. METHODS: We construct model soups by fine-tuning pre-trained models on seven different image resolutions and subsequently averaging their weights. Performance is evaluated on a multi-source dataset including holdout and external components. RESULTS: We find that model soups improve generalisation and calibration on the external component while maintaining performance on the holdout component. For robustness, we observe performance improvements for pertubated test images, while the performance on corrupted test images remains on par. CONCLUSIONS: Overall, souping for skin cancer classifiers has a positive effect on generalisation, robustness and calibration. It is easy for practitioners to implement and by combining multiple models into a single model, complexity is reduced. This could be an important factor in achieving clinical applicability, as less complexity generally means more transparency.
引用
- Maron RC, Hekler A, Haggenmüller S, et al. Model soups improve performance of dermoscopic skin cancer classifiers. Eur J Cancer. 2022. PMID: 35973360. doi:10.1016/j.ejca.2022.07.002
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。