模型引导的强化学习实现适应性精准给药
本文探讨将强化学习与群体PK/PD模型耦合开发精准给药算法的潜力,综述了丙泊酚麻醉、华法林抗凝及多种抗癌治疗等案例研究。结果显示各方法在精准给药途径、方法和临床贴合度上存在异质性,且PK/PD模型整合入强化学习框架前景广阔但仍需进一步研究。
要点
- 强化学习天然契合基于患者反应进行序贯剂量调整的适应性给药范式。
- PK/PD模型可作为强化学习的模拟引擎预测给药后果。
- 现有案例在方法学和临床贴合度上异质,需更多研究验证。
核心结论
- 将PK/PD模型整合入强化学习框架对精准给药前景广阔,但尚需进一步研究以解决当前局限并扩展至需适应性给药的药物。
研究设计与样本
- 本文为综述性文章,旨在探究强化学习与群体PK/PD模型耦合开发精准给药算法的潜力,并回顾领域内最相关的工作。
主要发现
- 已考虑的案例中PK/PD模型作为模拟引擎整合入强化学习算法以预测给药行动的后果,主要涉及丙泊酚诱导麻醉、华法林抗凝治疗及多种抗癌治疗。
- 结果突显了精准给药方法、应用方法学及临床领域贴合度方面存在一定异质性。
临床怎么用
- 临床可参考本文提出的教程,按强化学习框架关键要素(系统状态、智能体行动和奖励函数)来形式化精准给药问题。
- 可关注PK/PD模型如何增强强化学习方法,以支持基于患者反应和演变的序贯剂量调整。
局限性
- 当前方法在精准给药途径、方法学和临床贴合度上存在异质性。
- 仍需进一步研究和进展以解决当前局限并扩展该方法学至需适应性给药策略的药物。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Precision dosing, the tailoring of drug doses to optimize therapeutic benefits and minimize risks in each patient, is essential for drugs with a narrow therapeutic window and severe adverse effects. Adaptive dosing strategies extend the precision dosing concept to time-varying treatments which require sequential dose adjustments based on evolving patient conditions. Reinforcement learning (RL) naturally fits this paradigm: it perfectly mimics the sequential decision-making process where clinicians adapt dose administration based on patient response and evolution monitoring. This paper aims to investigate the potentiality of coupling RL with population PK/PD models to develop precision dosing algorithms, reviewing the most relevant works in the field. Case studies in which PK/PD models were integrated within RL algorithms as simulation engine to predict consequences of any dosing action have been considered and discussed. They mainly concern propofol-induced anesthesia, anticoagulant therapy with warfarin and a variety of anticancer treatments differing for administered agents and/or monitored biomarkers. The resulted picture highlights a certain heterogeneity in terms of precision dosing approaches, applied methodologies, and degree of adherence to the clinical domain. In addition, a tutorial on how a precision dosing problem should be formulated in terms of the key elements composing the RL framework (i.e., system state, agent actions and reward function), and on how PK/PD models could enhance RL approaches is proposed for readers interested in delving in this field. Overall, the integration of PK/PD models into a RL-framework holds great promise for precision dosing, but further investigations and advancements are still needed to address current limitations and extend the applicability of this methodology to drugs requiring adaptive dosing strategies.
引用
- Tosca EM, De Carlo A, Ronchi D, et al. Model-Informed Reinforcement Learning for Enabling Precision Dosing Via Adaptive Dosing. Clin Pharmacol Ther. 2024. PMID: 38989560. doi:10.1002/cpt.3356
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。