膝关节影像深度学习研究的系统综述:基于CLAIM清单的方法学质量评价
本系统综述检索2015年1月至2020年6月PubMed、EMBASE和Web of Science数据库,纳入36篇膝关节影像深度学习研究,按成像方式、任务、数据来源、算法类型和结局指标分类,并使用CLAIM清单评估方法学质量。结果显示平均CLAIM得分为27.94分(标准差4.26),仅占理想分42.00的66.53%,仅4项研究进行了外部验证,提示整体科学质量不足但深度学习仍具潜力。
要点
- 膝关节深度学习研究平均CLAIM得分仅27.94/42.00,方法学质量不足。
- 仅4/36项研究进行外部验证,多数仅内部交叉验证,泛化性证据有限。
- CLAIM清单评分者间一致性良好(ICC 0.815,95%CI 0.660-0.902)。
核心结论
- 膝关节影像深度学习研究的整体科学质量不足,但深度学习仍是有前景的诊断或预测技术。
- 需改进研究设计、验证和开放科学以证明发现的可泛化性并实现临床应用。
研究设计与样本
- 系统综述,检索2015年1月1日至2020年6月1日PubMed、EMBASE和Web of Science数据库。
- 共纳入36篇讨论膝关节影像深度学习应用的文章,按成像方式、影像任务、数据来源、算法类型和结局指标分类。
- 使用CLAIM清单评估方法学质量,平均CLAIM得分为27.94(标准差4.26),为理想分42.00的66.53%。
诊断标准与关键数值
- CLAIM评分者间一致性ICC为0.815(95%CI 0.660-0.902)。
- 36项研究中32项进行内部交叉验证,仅4项进行外部验证。
- 研究按成像方式分为X线(44.44%)和MRI(55.56%)。
临床应用与阅片要点
- 阅读膝关节深度学习研究时,应关注是否进行外部验证及CLAIM清单报告完整性。
- CLAIM评分可作为评估AI研究的方法学质量工具,但清单条目和结果报告仍需改进。
局限性
- 仅纳入36项研究,检索截止2020年6月,未涵盖此后文献。
- CLAIM清单本身在评分条目和结果报告方面需改进,可能影响评估全面性。
- 多数研究为回顾性设计且缺乏外部验证,限制结论的泛化性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
PURPOSE: Our purposes were (1) to explore the methodologic quality of the studies on the deep learning in knee imaging with CLAIM criterion and (2) to offer our vision for the development of CLAIM to assure high-quality reports about the application of AI to medical imaging in knee joint.
MATERIALS AND METHODS: A Checklist for Artificial Intelligence in Medical Imaging systematic review was conducted from January 1, 2015, to June 1, 2020, using PubMed, EMBASE, and Web of Science databases. A total of 36 articles discussing deep learning applications in knee joint imaging were identified, divided by imaging modality, and characterized by imaging task, data source, algorithm type, and outcome metrics.
RESULTS: A total of 36 studies were identified and divided into: X-ray (44.44%) and MRI (55.56%). The mean CLAIM score of the 36 studies was 27.94 (standard deviation, 4.26), which was 66.53% of the ideal score of 42.00. The CLAIM items achieved an average good inter-rater agreement (ICC 0.815, 95% CI 0.660-0.902). In total, 32 studies performed internal cross-validation on the data set, while only 4 studies conducted external validation of the data set.
CONCLUSIONS: The overall scientific quality of deep learning in knee imaging is insufficient; however, deep learning remains a promising technology for diagnostic or predictive purpose. Improvements in study design, validation, and open science need to be made to demonstrate the generalizability of findings and to achieve clinical applications. Widespread application, pre-trained scoring procedure, and modification of CLAIM in response to clinical needs are necessary in the future.
KEY POINTS: • Limited deep learning studies were established in knee imaging with mean score of 27.94, which was 66.53% of the ideal score of 42.00, commonly due to invalidated results, retrospective study design, and absence of a clear definition of the CLAIM items in detail. • A previous trained data extraction instrument allowed reaching moderate inter-rater agreement in the application of the CLAIM, while CLAIM still needs improvement in scoring items and result reporting to become a wide adaptive tool in reviews of deep learning studies.
引用
- Si L, Zhong J, Huo J, et al. Deep learning in knee imaging: a systematic review utilizing a Checklist for Artificial Intelligence in Medical Imaging (CLAIM). Eur Radiol. 2021. PMID: 34347157. doi:10.1007/s00330-021-08190-4
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。