在剂量反应研究中使用深度强化学习的最优自适应分配
本研究旨在构建一种直接优化性能指标的自适应分配规则,采用深度强化学习定义状态与奖励,并在模拟研究中与等量分配、D-最优和TD-最优方法比较,结果显示所提方法能成功改善待优化的性能指标,尤其在以平均绝对误差为优化指标时可构建对多种指标均占优的规则。
要点
- 深度强化学习可用于构建直接优化性能指标的自适应分配规则。
- 模拟研究中,所提方法较等量分配、D-最优和TD-最优方法改善了待优化指标。
- 以平均绝对误差为优化指标时,可构建对多种指标均占优的规则。
核心结论
- 深度强化学习在适当定义状态和奖励后,可用于构建直接优化功效、模型选择准确性、目标剂量估计准确性或平均绝对误差等性能指标的自适应分配规则。
研究设计与样本
- 本研究通过模拟研究构建并评估自适应分配规则,采用深度强化学习定义状态与奖励,并与等量分配、D-最优和TD-最优方法进行比较。
主要发现
- 模拟研究显示,与等量分配、D-最优和TD-最优方法相比,所提方法能成功改善待优化的性能指标。
- 当以平均绝对误差为优化指标时,可构建对多种指标均占优的规则。
临床怎么用
- 原文未据此给出临床推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Estimation of the dose-response curve for efficacy and subsequent selection of an appropriate dose in phase II trials are important processes in drug development. Various methods have been investigated to estimate dose-response curves. Generally, these methods are used with equal allocation of subjects for simplicity; nevertheless, they may not fully optimize performance metrics because of nonoptimal allocation. Optimal allocation methods, which include adaptive allocation methods, have been proposed to overcome the limitations of equal allocation. However, they rely on asymptotics, and thus sometimes cannot efficiently optimize the performance metric with the sample size in an actual clinical trial. The purpose of this study is to construct an adaptive allocation rule that directly optimizes a performance metric, such as power, accuracy of model selection, accuracy of the estimated target dose, or mean absolute error over the estimated dose-response curve. We demonstrate that deep reinforcement learning with an appropriately defined state and reward can be used to construct such an adaptive allocation rule. The simulation study shows that the proposed method can successfully improve the performance metric to be optimized when compared with the equal allocation, D-optimal, and TD-optimal methods. In particular, when the mean absolute error was set to the metric to be optimized, it is possible to construct a rule that is superior for many metrics.
引用
- Matsuura K, Honda J, El Hanafi I, et al. Optimal adaptive allocation using deep reinforcement learning in a dose-response study. Stat Med. 2021. PMID: 34747043. doi:10.1002/sim.9247
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。