计算机辅助诊断黑色素瘤的准确性:一项荟萃分析
该荟萃分析检索2002至2018年文献,纳入132项研究(70项可定量分析),评估自动化系统诊断黑色素瘤的准确性。汇总敏感度为0.74(95%CI 0.66-0.80),特异度为0.84(95%CI 0.79-0.88);使用独立测试集的研究敏感度更低(0.51 vs 0.82,P<.001)。与皮肤科医生相比,计算机辅助诊断敏感度相似,特异度低10个百分点但无统计学差异。
要点
- 自动化系统诊断黑色素瘤汇总敏感度0.74,特异度0.84。
- 独立测试集研究中敏感度显著更低(0.51 vs 0.82,P<.001)。
- 与专家相比敏感度相似,特异度低10个百分点但无统计学差异。
核心结论
- 计算机辅助诊断黑色素瘤的准确性可与专家相比,但真实世界适用性未知且可能受过拟合和偏倚风险限制。
研究设计与样本
- 使用QUADAS-2工具评估偏倚风险,质量评估基于预设标准,数据分析时间为2019年2月1日至3月10日。
主要发现
- 自动化系统汇总敏感度为0.74(95%CI 0.66-0.80),特异度为0.84(95%CI 0.79-0.88)。
- 使用独立测试集的研究敏感度为0.51(95%CI 0.34-0.69),低于未使用独立测试集研究的0.82(95%CI 0.77-0.86),差异有统计学意义(P < .001)。
- 两组特异度相似:独立测试集组0.83(95%CI 0.71-0.91),非独立测试集组0.85(95%CI 0.80-0.88),差异无统计学意义(P = .67)。
- 与皮肤科医生诊断相比,计算机辅助诊断敏感度相似,特异度低10个百分点,但差异无统计学意义。
- 纳入定量分析的70项研究中,除4项外均存在实质性偏倚风险,研究间异质性大。
临床怎么用
- 当前证据尚不支持将计算机辅助诊断作为黑色素瘤诊断的标准,临床应用中需谨慎解读其输出。
- 独立测试集下敏感度明显下降,提示实际应用中需关注过拟合问题。
局限性
- 多数研究来自计算机科学领域,前瞻性临床研究罕见。
- 纳入定量分析的研究中仅4项无实质性偏倚风险,研究间异质性大。
- 真实世界适用性未知,可能受过拟合和偏倚风险限制。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
IMPORTANCE: The recent advances in the field of machine learning have raised expectations that computer-aided diagnosis will become the standard for the diagnosis of melanoma. OBJECTIVE: To critically review the current literature and compare the diagnostic accuracy of computer-aided diagnosis with that of human experts. DATA SOURCES: The MEDLINE, arXiv, and PubMed Central databases were searched to identify eligible studies published between January 1, 2002, and December 31, 2018. STUDY SELECTION: Studies that reported on the accuracy of automated systems for melanoma were selected. Search terms included melanoma, diagnosis, detection, computer aided, and artificial intelligence. DATA EXTRACTION AND SYNTHESIS: Evaluation of the risk of bias was performed using the QUADAS-2 tool, and quality assessment was based on predefined criteria. Data were analyzed from February 1 to March 10, 2019. MAIN OUTCOMES AND MEASURES: Summary estimates of sensitivity and specificity and summary receiver operating characteristic curves were the primary outcomes. RESULTS: The literature search yielded 1694 potentially eligible studies, of which 132 were included and 70 offered sufficient information for a quantitative analysis. Most studies came from the field of computer science. Prospective clinical studies were rare. Combining the results for automated systems gave a melanoma sensitivity of 0.74 (95% CI, 0.66-0.80) and a specificity of 0.84 (95% CI, 0.79-0.88). Sensitivity was lower in studies that used independent test sets than in those that did not (0.51; 95% CI, 0.34-0.69 vs 0.82; 95% CI, 0.77-0.86; P < .001); however, the specificity was similar (0.83; 95% CI, 0.71-0.91 vs 0.85; 95% CI, 0.80-0.88; P = .67). In comparison with dermatologists' diagnosis, computer-aided diagnosis showed similar sensitivities and a 10 percentage points lower specificity, but the difference was not statistically significant. Studies were heterogeneous and substantial risk of bias was found in all but 4 of the 70 studies included in the quantitative analysis. CONCLUSIONS AND RELEVANCE: Although the accuracy of computer-aided diagnosis for melanoma detection is comparable to that of experts, the real-world applicability of these systems is unknown and potentially limited owing to overfitting and the risk of bias of the studies at hand.
引用
- Dick V, Sinz C, Mittlböck M, et al. Accuracy of Computer-Aided Diagnosis of Melanoma: A Meta-analysis. JAMA Dermatol. 2019. PMID: 31215969. doi:10.1001/jamadermatol.2019.1375
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。