肿瘤学领域机器学习预测模型研究中对研究结果的过度解读:一项系统综述
该系统综述检索2019年1月1日至2019年9月5日MEDLINE和EMBASE中肿瘤学领域开发与验证机器学习预后模型的研究,共纳入62篇出版物(含152个开发模型、37个验证模型)。27%的研究因额外分析和选择性报告导致方法与结果报告不一致,35项研究(56%)在标题、摘要、结果、讨论或结论中使用过强或诱导性措辞。作者认为阅读、解读和使用此类研究时需考虑过度解读的可能性,研究者应使用反映实际结果和证据强度的措辞进行报告。
要点
- 27%的肿瘤学机器学习预后模型研究因额外分析和选择性报告导致方法与结果报告不一致。
- 35项研究(56%)在标题、摘要、结果、讨论或结论中使用过强或诱导性措辞。
- 36项适用研究中有32项在讨论中比较开发模型,78%主要使用区分度指标支持其主张。
核心结论
- 阅读、解读和使用肿瘤学领域开发与验证预后模型的研究时需考虑过度解读的可能性。
- 研究者应使用反映实际结果和证据强度的措辞仔细报告其预后模型研究。
研究设计与样本
- 系统综述,检索MEDLINE和EMBASE中2019年1月1日至2019年9月5日发表的肿瘤学相关机器学习预后模型开发与验证研究。
- 共纳入62篇出版物,包含152个开发模型和37个验证模型。
- 使用现有过度解读框架描述高度提示性过度解读实践领域。
主要发现
- 27%的研究因额外分析和选择性报告导致方法与结果报告不一致。
- 36项适用研究中有32项在讨论中报告开发模型之间的比较,其中78%主要使用区分度指标支持其主张。
- 35项研究(56%)在标题、摘要、结果、讨论或结论中使用过强或诱导性措辞。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction. STUDY DESIGN AND SETTING: We conducted a systematic review, searching MEDLINE and EMBASE for oncology-related studies that developed and validated a prognostic model using machine learning published between 1st January, 2019, and 5th September, 2019. We used existing spin frameworks and described areas of highly suggestive spin practices. RESULTS: We included 62 publications (including 152 developed models; 37 validated models). Reporting was inconsistent between methods and the results in 27% of studies due to additional analysis and selective reporting. Thirty-two studies (out of 36 applicable studies) reported comparisons between developed models in their discussion and predominantly used discrimination measures to support their claims (78%). Thirty-five studies (56%) used an overly strong or leading word in their title, abstract, results, discussion, or conclusion. CONCLUSION: The potential for spin needs to be considered when reading, interpreting, and using studies that developed and validated prognostic models in oncology. Researchers should carefully report their prognostic model research using words that reflect their actual results and strength of evidence.
引用
- Dhiman P, Ma J, Andaur Navarro CL, et al. Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review. J Clin Epidemiol. 2023. PMID: 36935090. doi:10.1016/j.jclinepi.2023.03.012
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。