文献速递医学教育与科研方法

评估临床试验可信度检查的可行性与影响:应用于Cochrane综述的INSPECT-SR项目第二阶段

Assessing the feasibility and impact of clinical trial trustworthiness checks via an application to Cochrane Reviews: Stage 2 of the INSPECT-SR project.
PubMed 文献 · J Clin Epidemiol · 2025-05-09

本研究将72项可信度检查应用于50篇Cochrane综述中的95项随机对照试验(RCT),评估检查的可行性及对Meta分析的影响。结果显示,评估者对25%的RCT存在一些担忧、对6%存在严重担忧;剔除这些RCT后,22%的Meta分析无剩余RCT。许多检查难以理解或实施,且可信度评估与偏倚风险或GRADE评估无关联。

要点

  • 72项可信度检查应用于50篇Cochrane综述中的95项RCT。
  • 评估者对25%的RCT有一些担忧,对6%的RCT有严重担忧。
  • 剔除有担忧的RCT后,22%的Meta分析无剩余RCT。

核心结论

  • 本研究支持在系统综述中常规开展可信度评估,因为有问题的研究似乎未被偏倚风险评估标记。
  • 研究结果为可信度检查的可行性和影响提供了证据。
  • 这些结果将与后续Delphi流程的结果一起,用于确定哪些检查应纳入INSPECT-SR工具。

研究设计与样本

  • 将72项可信度检查应用于50篇Cochrane综述中的随机对照试验(RCT)。
  • 对每项检查记录通过、失败、可能失败或无法完成检查。
  • 应用检查后记录对每项RCT真实性的担忧情况。
  • 剔除每项检查标记的RCT后重复每个Meta分析,再次剔除对真实性有担忧的RCT后重复分析,以估计可信度评估的影响。
  • 将可信度评估与综述中的偏倚风险评估和GRADE评估进行比较。

主要发现

  • 共评估95项RCT。
  • 应用检查后,评估者对25%的RCT存在一些担忧,对6%的RCT存在严重担忧。
  • 剔除存在一些或严重担忧的RCT后,22%的Meta分析无剩余RCT。
  • 许多检查难以理解或实施,可能在某些情况下导致不必要的怀疑。
  • 评估仅限于纳入不超过5项RCT的Meta分析(其中54%仅含1项RCT),这将扭曲对结果的影响。

临床怎么用

  • 作者认为,本研究支持在系统综述中常规开展可信度评估,因为有问题的研究似乎未被偏倚风险评估标记。
  • 研究结果将与后续Delphi流程的结果一起,用于确定哪些检查应纳入INSPECT-SR工具。

局限性

  • 许多检查难以理解或实施,可能在某些情况下导致不必要的怀疑。
  • 评估仅限于纳入不超过5项RCT的Meta分析(其中54%仅含1项RCT),这将扭曲对结果的影响。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND AND OBJECTIVES: The aim of the INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project is to develop a tool to identify problematic RCTs in systematic reviews. In stage 1 of the project, a list of potential trustworthiness checks was created. The checks on this list must be evaluated to determine which should be included in the INSPECT-SR tool. METHODS: We attempted to apply 72 trustworthiness checks to randomized controlled trials (RCTs) in 50 Cochrane reviews. For each, we recorded whether the check was passed, failed, or possibly failed or whether it was not feasible to complete the check. Following application of the checks, we recorded whether we had concerns about the authenticity of each RCT. We repeated each meta-analysis after removing RCTs flagged by each check and again after removing RCTs where we had concerns about authenticity to estimate the impact of trustworthiness assessment. Trustworthiness assessments were compared to Risk of Bias and Grading of Recommendations Assessment, Development and Evaluation (GRADE) assessments in the reviews. RESULTS: Ninety-five RCTs were assessed. Following application of the checks, assessors had some or serious concerns about the authenticity of 25% and 6% of the RCTs, respectively. Removing RCTs with either some or serious concerns resulted in 22% of meta-analyses having no remaining RCTs. However, many checks proved difficult to understand or implement, which may have led to unwarranted skepticism in some instances. Furthermore, we restricted assessment to meta-analyses with no more than five RCTs (54% contained only 1 RCT), which will distort the impact on results. No relationship was identified between trustworthiness assessment and Risk of Bias or GRADE. CONCLUSION: This study supports the case for routine trustworthiness assessment in systematic reviews, as problematic studies do not appear to be flagged by Risk of Bias assessment. The study produced evidence on the feasibility and impact of trustworthiness checks. These results will be used, in conjunction with those from a subsequent Delphi process, to determine which checks should be included in the INSPECT-SR tool. PLAIN LANGUAGE SUMMARY: Systematic reviews collate evidence from randomized controlled trials (RCTs) to find out whether health interventions are safe and effective. However, it is now recognized that the findings of some RCTs are not genuine, and some of these studies appear to have been fabricated. Various checks for these "problematic" RCTs have been proposed, but it is necessary to evaluate these checks to find out which are useful and which are feasible. We applied a comprehensive list of "trustworthiness checks" to 95 RCTs in 50 systematic reviews to learn more about them and to see how often performing the checks would lead us to classify RCTs as being potentially inauthentic. We found that applying the checks led to concerns about the authenticity of around 1 in three RCTs. However, we found that many of the checks were difficult to perform and could have been misinterpreted. This might have led us to be overly skeptical in some cases. The findings from this study will be used, alongside other evidence, to decide which of these checks should be performed routinely to try to identify problematic RCTs, to stop them from being mistaken for genuine studies and potentially being used to inform health care decisions.

来源:J Clin Epidemiol,查看原文。版权归原作者所有。

引用

  1. Wilkinson J, Heal C, Antoniou GA, et al. Assessing the feasibility and impact of clinical trial trustworthiness checks via an application to Cochrane Reviews: Stage 2 of the INSPECT-SR project. J Clin Epidemiol. 2025. PMID: 40349737. doi:10.1016/j.jclinepi.2025.111824

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。