文献速递医学人工智能与信息学医学教育与科研方法

在一项先导研究中,自动化实时系统综述更新可行、准确且节省人力

In a pilot study, automated real-time systematic review updates were feasible, accurate, and work-saving.
PubMed 文献 · J Clin Epidemiol · 2022-09-20

该先导研究将人工智能与人工专业判断结合,使用RobotReviewer LIVE从2021年2月至8月持续更新一项COVID-19疫苗接种试验的系统综述,并与传统人工更新流程比较。截至2021年7月,人工检索获得135篇摘要、纳入31篇(精确率23%,召回率100%),自动化系统检索获得56篇摘要、人工筛选后纳入31篇(精确率55%,召回率100%)。结论认为该系统识别了全部相关研究、减少了人工筛选工作量,并实现了随新原始研究发表而滚动更新。

要点

  • 自动化系统检索56篇摘要,人工筛选后纳入31篇,精确率55%,召回率100%。
  • 传统人工流程检索135篇摘要,筛选后纳入31篇,精确率23%,召回率100%。
  • 该系统仅限检索PubMed/MEDLINE,且仅考虑随机对照试验报告。

核心结论

  • 该系统识别了全部相关研究,减少了人工筛选工作量,并实现了随新原始研究发表而滚动更新。

研究设计与样本

  • 研究描述并先导了一种将人工智能与人工专业判断相结合、持续识别与系统综述相关新发表试验的新方法。
  • 使用RobotReviewer LIVE从2021年2月至8月持续更新一项COVID-19疫苗接种试验的系统综述。
  • 将系统识别的论文与综述团队传统人工流程发现的论文进行比较。

主要发现

  • 人工更新检索(最后检索日期2021年7月)获得135篇摘要,筛选后纳入31篇(精确率23%,召回率100%)。
  • 截至同一日期,自动化系统获得56篇摘要,人工筛选后纳入31篇(精确率55%,召回率100%)。

临床怎么用

  • 原文未据此给出推荐。

局限性

  • 该系统仅限于检索PubMed/MEDLINE。
  • 该系统仅考虑随机对照试验报告。
  • 作者旨在未来解决这些局限性。
  • 该系统作为开源软件提供,以供进一步先导研究和评估。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: The aim of this study is to describe and pilot a novel method for continuously identifying newly published trials relevant to a systematic review, enabled by combining artificial intelligence (AI) with human expertise. STUDY DESIGN AND SETTING: We used RobotReviewer LIVE to keep a review of COVID-19 vaccination trials updated from February to August 2021. We compared the papers identified by the system with those found by the conventional manual process by the review team. RESULTS: The manual update searches (last search date July 2021) retrieved 135 abstracts, of which 31 were included after screening (23% precision, 100% recall). By the same date, the automated system retrieved 56 abstracts, of which 31 were included after manual screening (55% precision, 100% recall). Key limitations of the system include that it is limited to searches of PubMed/MEDLINE, and considers only randomized controlled trial reports. We aim to address these limitations in future. The system is available as open-source software for further piloting and evaluation. CONCLUSION: Our system identified all relevant studies, reduced manual screening work, and enabled rolling updates on publication of new primary research.

来源:J Clin Epidemiol,查看原文。版权归原作者所有。

引用

  1. Marshall IJ, Trikalinos TA, Soboczenski F, et al. In a pilot study, automated real-time systematic review updates were feasible, accurate, and work-saving. J Clin Epidemiol. 2022. PMID: 36150548. doi:10.1016/j.jclinepi.2022.08.013

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。