机器学习工具辅助EULAR类风湿关节炎系统文献综述标题摘要筛选的评估
该研究基于2025年EULAR类风湿关节炎管理推荐更新的4项系统文献综述,用不同机器学习模型复现人工标题摘要筛选以评估半自动化筛选性能,发现平均减少77.8%工作量且捕获超过95%相关记录,机器学习辅助随机对照试验分类过滤使待筛摘要平均减少57.3%。
要点
- 机器学习工具可平均减少77.8%的标题摘要筛选工作量,同时捕获超过95%的相关记录。
- 机器学习辅助随机对照试验分类过滤使待筛摘要平均减少57.3%。
- 研究复现人工筛选验证机器学习潜力,但强调需前瞻性评估以优化整合策略。
核心结论
- 复现人工标题摘要筛选表明机器学习工具支持系统文献综述具有可观潜力,同时存在重要局限和陷阱。
- 前瞻性评估将确定在保持高方法学标准下将这些工具整合到未来综述流程中的最佳策略。
研究设计与样本
- 研究复现了为2025年EULAR类风湿关节炎管理推荐更新而进行的4项系统文献综述的标题摘要筛选,以人工筛选为参考标准,使用不同机器学习模型进行。
- 通过Systematic Review Toolbox、在线和基于机器学习的检索及参考文献检查识别合格软件,纳入标准为积极维护、可访问且提供超出常用参考文献管理器的筛选功能。
主要发现
- 识别出9个采用基于机器学习的记录优先级排序和相关预测工具,其中3个符合所有纳入标准并被详细评估。
- 这些工具应用于3项随机对照试验系统文献综述和1项还包括观察性研究的综述的检索结果。
- 在各综述中,实现了显著的工作量减少(平均77.8%,标准差12.8%),同时始终捕获超过95%的所有相关记录。
- 机器学习支持的随机对照试验分类和筛选使待筛摘要平均减少57.3%(标准差7.4%)。
临床怎么用
- 临床医生和研究人员可利用机器学习工具辅助系统文献综述的标题摘要筛选,以减少工作量并保持高相关记录捕获率。
- 在整合机器学习工具时,需注意其局限性和陷阱,并考虑前瞻性评估以优化未来综述流程。
局限性
- 研究强调了机器学习工具的重要局限和陷阱。
- 需前瞻性评估以确定最佳整合策略并保持高方法学标准。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: Systematic literature reviews (SLRs) provide the scientific basis for European Alliance of Associations for Rheumatology (EULAR) task force projects, but they are highly time- and labour-intensive in an ever-growing research landscape, repetitive, and susceptible to human error. In this study, we evaluate the performance of machine learning (ML) models developed for semiautomated title and abstract screening, aiming at supporting and accelerating future review processes. METHODS: Title and abstract screening of 4 SLRs, conducted to inform the 2025 update of the EULAR management recommendations for rheumatoid arthritis, was replicated using different ML models, with manual screening serving as reference standard. Eligible software was identified via the Systematic Review Toolbox, online and ML-based searches, and reference checking. Tools were included if they were actively maintained, accessible, and offered screening functionality beyond commonly used reference managers. RESULTS: Nine tools employing ML-based record prioritisation and relevance prediction were identified, of which 3 met all inclusion criteria and were evaluated in detail. These were applied to search results from 3 SLRs on randomised controlled trials (RCTs) and 1 that also included observational studies. Across reviews, substantial workload reductions were achieved (mean 77.8%, SD 12.8%) while consistently capturing over 95% of all relevant records. Additionally, ML-supported classification and filtering of RCTs prior to screening resulted in a mean reduction of abstracts to screen by 57.3% (SD 7.4%). CONCLUSIONS: Replication of manual title and abstract screening demonstrates the considerable potential of ML tools to support systematic reviews, while highlighting important limitations and pitfalls. Prospective evaluations will identify optimal strategies for integrating these tools into future review processes while maintaining high methodological standards.
引用
- Konzett V, Laskou F, Smolen JS, et al. Evaluating machine learning tools to assist title and abstract screening in systematic literature reviews: a report based on the EULAR RA Management Recommendations Task Force. Ann Rheum Dis. 2026. PMID: 42230212. doi:10.1016/j.ard.2026.05.006
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。