文献速递医学教育与科研方法流行病学与生物统计

单审稿人摘要筛查漏检13%相关研究:一项基于众包的随机对照试验

Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial.
PubMed 文献 · J Clin Epidemiol · 2020-01-21

本研究通过众包平台开展平行组随机对照试验,比较单审稿人与双审稿人摘要筛查的准确性,以两篇已发表系统综述为参考标准。结果显示单审稿人筛查漏检13%相关研究(敏感性86.6%,95%CI 80.6%-91.2%),双审稿人漏检3%(敏感性97.5%,95%CI 95.1%-98.8%)。结论认为单审稿人筛查不符合系统综述的高方法学标准,或可用于快速综述。

要点

  • 单审稿人摘要筛查敏感性86.6%,漏检13%相关研究。
  • 双审稿人筛查敏感性97.5%,漏检3%相关研究。
  • 单审稿人筛查或仅适用于刻意降低方法学标准的快速综述。

核心结论

  • 单审稿人摘要筛查漏检13%相关研究,不符合决策者对系统综述的高方法学期望。
  • 单审稿人筛查可作为快速综述的可选方案,因快速综述刻意降低方法学标准以加速证据合成。

研究设计与样本

  • 采用基于众包的平行组随机对照试验设计。
  • 通过Cochrane Crowd平台将合格参与者随机分配,每人筛查药物或公共卫生主题的100篇摘要。
  • 以两篇已发表系统综述为参考标准,计算单审稿人和双审稿人筛查的敏感性与特异性。

主要发现

  • 单审稿人摘要筛查漏检13%相关研究(敏感性86.6%;95%CI 80.6%-91.2%)。
  • 双审稿人摘要筛查漏检3%相关研究(敏感性97.5%;95%CI 95.1%-98.8%)。
  • 单审稿人筛查特异性为79.2%(95%CI 77.4%-80.9%),双审稿人为68.7%(95%CI 66.4%-71.0%)。

临床怎么用

  • 在系统综述中不宜采用单审稿人摘要筛查,因其漏检率较高。
  • 在快速综述中可考虑单审稿人筛查,但需明确其方法学标准较低。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVES: To determine the accuracy of single-reviewer screening in correctly classifying abstracts as relevant or irrelevant for literature reviews. STUDY DESIGN AND SETTING: We conducted a crowd-based, parallel-group randomized controlled trial. Using the Cochrane Crowd platform, we randomly assigned eligible participants to 100 abstracts each of a pharmacological or a public health topic. After completing a training exercise, participants screened abstracts online based on predefined inclusion and exclusion criteria. We calculated sensitivities and specificities of single- and dual-reviewer screening using two published systematic reviews as reference standards. RESULTS: Two hundred and eighty participants made 24,942 screening decisions on 2,000 randomly selected abstracts from the reference standard reviews. On average, each abstract was screened 12 times. Overall, single-reviewer abstract screening missed 13% of relevant studies (sensitivity: 86.6%; 95% confidence interval [CI], 80.6%-91.2%). By comparison, dual-reviewer abstract screening missed 3% of relevant studies (sensitivity: 97.5%; 95% CI, 95.1%-98.8%). The corresponding specificities were 79.2% (95% CI, 77.4%-80.9%) and 68.7% (95% CI, 66.4%-71.0%), respectively. CONCLUSIONS: Single-reviewer abstract screening does not appear to fulfill the high methodological standards that decisionmakers expect from systematic reviews. It may be a viable option for rapid reviews, which deliberately lower methodological standards to provide decision makers with accelerated evidence synthesis products.

来源:J Clin Epidemiol,查看原文。版权归原作者所有。

引用

  1. Gartlehner G, Affengruber L, Titscher V, et al. Single-reviewer abstract screening missed 13 percent of relevant studies: a crowd-based, randomized controlled trial. J Clin Epidemiol. 2020. PMID: 31972274. doi:10.1016/j.jclinepi.2020.01.005

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。