识别与缓解药物审批预测中的潜在偏倚
研究采用Debiasing变分自编码器在Citeline数据集上预测II期试验后的药物开发结局,去偏后F1分数从0.25升至0.48,真阳性率从15%升至60%,真阴性率从99%降至88%。
要点
- 去偏模型F1分数0.48优于基线0.25,真阳性率60%高于基线15%。
- 去偏模型真阴性率88%低于基线99%,可能高估有潜在不良反应药物的成功率。
- 模型预测受既往其他适应症获批、试验终点是否达到及试验完成年份等因素影响。
核心结论
- 去偏可提高晚期药物开发的财务效率,去偏模型更可能识别安全有效的药物。
- 去偏模型真阴性率较低,可能对有潜在不良反应的药物预测更高成功概率,用于在研候选药物时需谨慎。
研究设计与样本
- 研究实例化Debiasing Variational Autoencoder(最先进的自动去偏模型),在Informa Pharma Intelligence提供的Citeline数据集上训练和评估,以II期试验结果预测最终药物开发结局。
主要发现
- 去偏模型F1分数0.48,优于未去偏基线的0.25。
- 去偏模型真阳性率60%,远高于基线的15%;真阴性率88%,略低于基线的99%。
- 模型能区分大型制药企业与小型生物技术公司开发的药物。
- 模型预测受药物既往其他适应症获批、试验是否达到阳性/阴性终点、试验完成年份等多因素强烈影响。
临床怎么用
- 利用去偏模型辅助识别安全有效药物,提高晚期开发财务效率。
- 对在研候选药物预测开发结局时需谨慎,因模型可能高估有潜在不良反应药物的成功概率。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
INTRODUCTION: Machine learning models are increasingly applied to predict the drug development outcomes based on intermediary clinical trial results. A key challenge to this task is to address various forms of bias in the historical drug approval data. OBJECTIVE: We aimed to identify and mitigate the bias in drug approval predictions and quantify the impacts of debiasing in terms of financial value and drug safety. METHODS: We instantiated the Debiasing Variational Autoencoder, the state-of-the-art model for automated debiasing. We trained and evaluated the model on the Citeline dataset provided by Informa Pharma Intelligence to predict the final drug development outcome from phase II trial results. RESULTS: The debiased Debiasing Variational Autoencoder model achieved better performance (measured by the [Formula: see text] score 0.48) in predicting the drug development outcomes than its un-debiased baseline ([Formula: see text] score 0.25). It had a much higher true-positive rate than baseline (60% vs 15%), while its true-negative rate was slightly lower (88% vs 99%). The Debiasing Variational Autoencoder distinguished between drugs developed by large pharmaceutical firms and those by small biotech companies. The model prediction is strongly influenced by multiple factors such as prior approval of the drug for another indication, whether the trial meets the positive/negative endpoints, and the year when the trial is completed. We estimate that the debiased model generates financial value for the drug developer in six major therapeutic areas, with a range of US$763-1,365 million. CONCLUSIONS: Our analysis shows that debiasing improves the financial efficiency of late-stage drug development. From the pharmacovigilance perspective, the debiased model is more likely to identify drugs that are both safe and effective. Meanwhile, it may predict a higher probability of success for drugs with potential adverse effects (because of its lower true-negative rate), thus it must be used with caution to predict the development outcomes of drug candidates currently in the pipeline.
引用
- Xu Q, Ahmadi E, Amini A, et al. Identifying and Mitigating Potential Biases in Predicting Drug Approvals. Drug Saf. 2022. PMID: 35579815. doi:10.1007/s40264-022-01160-9
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。