新冠疫情期间加速确证性试验的可靠性:低试验前可信度的后果
该研究评估新冠疫情期间早期确证性试验的试验前可信度及对结果可靠性的影响。基于463项早期随机试验和150种候选分子,试验级试验前可信度经验值为11%,新冠试验中位把握度仅9%,统计显著结果反映真实疗效的概率为31%,远低于非新冠试验的97%。结论提示需加强试验前证据、协调试验优先排序并采用重复验证策略。
要点
- 新冠早期确证性试验试验前可信度经验值仅11%,中位把握度9%,显著结果真实有效概率31%。
- 非新冠试验在常规II至III期50%成功率基准下,显著结果真实有效概率达97%。
- 两项独立80%把握度试验的重复框架阳性预测值为99.2%,优于单項90%至95%把握度试验的81%至82%。
核心结论
- 新冠早期试验生态系统常把握度不足且试验前可信度低,导致试验后真实疗效概率低于常规研发路径。
- 未来大流行应对需更强试验前证据、协调试验优先排序及以重复验证为导向的策略。
研究设计与样本
- 研究在试验层面估计经验性试验前可信度,即早期随机试验中评估的干预措施最终被参考网络荟萃分析认定为有效的比例。
- 针对每项试验,利用报告样本量和事件数估计反事实统计把握度。
- 采用贝叶斯定理基于敏感性(试验把握度)、特异性(1减0.025)和经验性试验前可信度计算试验后疗效概率。
- 比较新冠与非新冠研发管线的阳性预测值。
主要发现
- 463项早期试验评估150种候选分子,其中51项评估的干预措施最终获参考网络荟萃分析支持,对应试验级经验性试验前可信度为11%。
- 新冠试验中位把握度为9%(最小至最大:6%至100%)。
- 统计显著结果反映真实疗效的概率为31%(20%至83%),而非新冠试验在常规II至III期50%成功基准下为97%。
- 在经验性试验前可信度11%下,由两项独立、80%把握度试验组成的重复框架阳性预测值为99.2%,而单項90%至95%把握度试验阳性预测值为81%至82%。
临床怎么用
- 启动大型确证性试验前应审慎选择候选治疗,要求更强的试验前证据支持。
- 未来大流行中可考虑协调国际研究策略、适应性平台试验及简化大规模研究(如RECOVERY试验)以提高证据可靠性并避免资源浪费。
- 在试验前可信度低时,优先采用重复验证框架而非依赖单項高把握度试验。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVES: The reliability of confirmatory clinical trials (ie, the post-trial probability) depends on the pretrial probability. During the COVID-19 pandemic, many early COVID-19 confirmatory trials were launched despite limited phase II evidence and weak mechanistic or preclinical support. Whether this accelerated development strategy reduced the reliability of therapeutic evidence remains uncertain. STUDY DESIGN AND SETTING: Overall empirical pretrial plausibility was estimated at the trial level as the proportion of early randomized trials evaluating interventions that were ultimately considered effective in the reference network meta-analysis. For each trial, counterfactual statistical power was estimated using the reported sample size and the number of events. Bayes' theorem was used to calculate the post-trial probability of efficacy, based on sensitivity (ie, trial power), specificity (ie, 1 minus 0.025), and empirical pretrial plausibility. We then compared the positive predictive values (PPVs) of COVID-19 and non-COVID-19 development pipelines. RESULTS: Of the 463 early trials evaluating 150 candidate molecules, 51 assessed interventions ultimately supported by the reference network meta-analysis, corresponding to an empirical trial-level pretrial plausibility of 11%. The median power of COVID-19 trials was 9% (min-max: 6%-100%). Accordingly, the probability that a statistically significant result reflected true efficacy was 31% (20%-83%), compared to 97% for non-COVID-19 trials characterized by the conventional phase II to phase III success benchmark of 50%. With an empirical pretrial plausibility of 11%, a replication framework consisting of two independent, 80% powered trials yielded a PPV of 99.2%. This contrasts with a single 90% to 95% powered trial, which yielded a PPV of 81% to 82%. CONCLUSION: The early COVID-19 trial ecosystem was frequently underpowered and conducted under conditions of low pretrial plausibility, resulting in lower post-trial probabilities of true efficacy than in conventional development pathways. These findings highlight the need for stronger pretrial evidence, coordinated trial prioritization, and replication-oriented strategies in future pandemic responses. PLAIN LANGUAGE SUMMARY: During the COVID-19 pandemic, hundreds of clinical trials were launched within a very short period to identify effective treatments. Many of these studies started before strong laboratory data, animal studies, or early clinical trial results were available. At the same time, many trials enrolled only small numbers of patients, making them less able to reliably detect true treatment effects. In this study, we examined how these conditions may have affected the reliability of COVID-19 treatment research. Using data from more than 400 randomized trials, we estimated how often a statistically significant result was likely to reflect a truly effective treatment. We found that when treatments are tested with limited supporting evidence and low-powered studies, positive trial results are more likely to be misleading or difficult to reproduce. We also explored alternative research strategies. Our findings suggest that repeating smaller but well-designed independent trials may provide more reliable evidence than relying on a single large trial, especially when the initial evidence supporting a treatment is weak. These results highlight the importance of carefully selecting candidate treatments before launching large confirmatory trials during future pandemics. Coordinated international research strategies, adaptive platform trials, and simplified large-scale studies such as the RECOVERY trial may help produce more reliable evidence while avoiding unnecessary duplication and waste of resources.
引用
- Gougeon A, Massy E, Michal C, et al. Reliability of accelerated confirmatory trials during the COVID-19 pandemic: the consequences of low pretrial plausibility. J Clin Epidemiol. 2026. PMID: 42600710. doi:10.1016/j.jclinepi.2026.112465
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。