基于深度强化学习的个性化治疗推荐
本文提出一种基于深度强化学习(DRL)的个性化排序系统PPORank,以马尔可夫决策过程建模,按细胞系(或患者)的分子与临床特征预测药物效果并排序,并在两个大规模癌细胞系数据集及模拟数据上验证,结果显示其优于基于监督学习的现有方法。
要点
- PPORank在深度强化学习框架下按预测效果对药物排序,可随时间连续学习并推荐最合适药物。
- 在两个大规模癌细胞系数据集及模拟数据上,PPORank优于基于监督学习的现有方法。
- 作者认为深度强化学习框架下的新方法在精准医学中具有较大潜力,应进一步研究。
核心结论
- 深度强化学习框架下的PPORank在实验中优于基于监督学习的现有方法,作者认为该类方法在精准医学中具有较大潜力并应进一步研究。
研究设计与样本
- 方法学研究:提出基于深度强化学习的个性化排序系统PPORank,以马尔可夫决策过程建模。
- 在模拟数据及两个大规模癌细胞系数据集上进行概念验证实验。
主要发现
- 在模拟数据和两个大规模癌细胞系数据集上,基于深度强化学习的PPORank优于基于监督学习的现有方法。
局限性
- 原文结论指出深度强化学习框架下的新方法在精准医学中具有较大潜力,应进一步研究。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
In precision medicine, the ultimate goal is to recommend the most effective treatment to an individual patient based on patient-specific molecular and clinical profiles, possibly high-dimensional. To advance cancer treatment, large-scale screenings of cancer cell lines against chemical compounds have been performed to help better understand the relationship between genomic features and drug response; existing machine learning approaches use exclusively supervised learning, including penalized regression and recommender systems. However, it would be more efficient to apply reinforcement learning to sequentially learn as data accrue, including selecting the most promising therapy for a patient given individual molecular and clinical features and then collecting and learning from the corresponding data. In this article, we propose a novel personalized ranking system called Proximal Policy Optimization Ranking (PPORank), which ranks the drugs based on their predicted effects per cell line (or patient) in the framework of deep reinforcement learning (DRL). Modeled as a Markov decision process, the proposed method learns to recommend the most suitable drugs sequentially and continuously over time. As a proof-of-concept, we conduct experiments on two large-scale cancer cell line data sets in addition to simulated data. The results demonstrate that the proposed DRL-based PPORank outperforms the state-of-the-art competitors based on supervised learning. Taken together, we conclude that novel methods in the framework of DRL have great potential for precision medicine and should be further studied.
引用
- Liu M, Shen X, Pan W, et al. Deep reinforcement learning for personalized treatment recommendation. Stat Med. 2022. PMID: 35716038. doi:10.1002/sim.9491
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。