医学影像人工智能研究中清单自我报告:基于CLAIM引文的系统综述
该研究通过系统分析CLAIM清单的引文,评估其在医学影像人工智能研究中的自我报告使用情况。共纳入118篇论文,仅12篇(10%)提供了填写完整的清单作为证明,70篇(59%)仅提及遵循但无证明。逐项确认分析显示各文献57%至93%的条目声明被证实,均值和标准差分别为81%和10%。结论认为自我报告清单可能包含错误,应谨慎对待。
要点
- 仅10%(12/118)的论文提供了填写完整的CLAIM清单作为自我报告证明。
- 59%(70/118)的论文仅提及遵循CLAIM但未提供任何清单证明。
- 逐项确认分析中各文献57%至93%的条目声明被证实,均值81%、标准差10%。
核心结论
- 仅小部分出版物将CLAIM用作清单并提供填写完整的文档;但自我报告的清单可能包含错误,应谨慎对待。
研究设计与样本
- 系统综述,通过Google Scholar、Web of Science和Scopus检索CLAIM引文(检索日期2023年4月29日),共纳入118篇研究论文,由两名读者在自动标注辅助下独立评估。
诊断标准与关键数值
- 118篇论文中12篇(10%)提供了填写完整的CLAIM清单作为自我报告证明。
- 118篇论文中70篇(59%)仅提及某种程度的遵循但未提供任何清单形式的证明。
- 118篇论文中36篇(31%)引用CLAIM的原因与报告或方法学遵循无关。
- 逐项确认分析中各文献57%至93%的条目声明被证实,均值和标准差分别为81%和10%。
临床应用与阅片要点
- 作者希望这一系统引文分析能促使人工智能界认识恰当自我报告的重要性,并鼓励研究者、期刊、编辑和审稿人采取行动确保清单的恰当使用。
- 作者指出即使有清单证明,所声明的条目也可能包含错误(如虚假的遵循声明),应谨慎对待;这些可能表明清单使用不当,需要权威机构采取进一步行动。
局限性
- 自我报告的清单可能包含错误,应谨慎对待。
- 即使有清单证明,所声明的条目也可能包含错误(如虚假的遵循声明),应谨慎对待。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVE: To evaluate the usage of a well-known and widely adopted checklist, Checklist for Artificial Intelligence in Medical imaging (CLAIM), for self-reporting through a systematic analysis of its citations.
METHODS: Google Scholar, Web of Science, and Scopus were used to search for citations (date, 29 April 2023). CLAIM's use for self-reporting with proof (i.e., filled-out checklist) and other potential use cases were systematically assessed in research papers. Eligible papers were evaluated independently by two readers, with the help of automatic annotation. Item-by-item confirmation analysis on papers with checklist proof was subsequently performed.
RESULTS: A total of 391 unique citations were identified from three databases. Of the 118 papers included in this study, 12 (10%) provided a proof of self-reported CLAIM checklist. More than half (70; 59%) only mentioned some sort of adherence to CLAIM without providing any proof in the form of a checklist. Approximately one-third (36; 31%) cited the CLAIM for reasons unrelated to their reporting or methodological adherence. Overall, the claims on 57 to 93% of the items per publication were confirmed in the item-by-item analysis, with a mean and standard deviation of 81% and 10%, respectively.
CONCLUSION: Only a small proportion of the publications used CLAIM as checklist and supplied filled-out documentation; however, the self-reported checklists may contain errors and should be approached cautiously. We hope that this systematic citation analysis would motivate artificial intelligence community about the importance of proper self-reporting, and encourage researchers, journals, editors, and reviewers to take action to ensure the proper usage of checklists.
CLINICAL RELEVANCE STATEMENT: Only a small percentage of the publications used CLAIM for self-reporting with proof (i.e., filled-out checklist). However, the filled-out checklist proofs may contain errors, e.g., false claims of adherence, and should be approached cautiously. These may indicate inappropriate usage of checklists and necessitate further action by authorities.
KEY POINTS: • Of 118 eligible papers, only 12 (10%) followed the CLAIM checklist for self-reporting with proof (i.e., filled-out checklist). More than half (70; 59%) only mentioned some kind of adherence without providing any proof. • Overall, claims on 57 to 93% of the items were valid in item-by-item confirmation analysis, with a mean and standard deviation of 81% and 10%, respectively. • Even with the checklist proof, the items declared may contain errors and should be approached cautiously.
引用
- Kocak B, Keles A, Akinci D'Antonoli T, et al. Self-reporting with checklists in artificial intelligence research on medical imaging: a systematic review based on citations of CLAIM. Eur Radiol. 2023. PMID: 37740080. doi:10.1007/s00330-023-10243-9
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。