文献速递重症综合管理与镇静镇痛

强化学习能改善危重症患者的结局吗?系统综述与就绪度评估

Does Reinforcement Learning Improve Outcomes for Critically Ill Patients? A Systematic Review and Level-of-Readiness Assessment.
PubMed 文献 · Crit Care Med · 2023-11-08
一句话结论系统综述未发现任何研究证明基于强化学习的技术能改善危重症患者结局,所有显示的优于临床医生的结果均基于回顾性off-policy评估。

该系统综述检索PubMed、Embase.com、Web of Science核心合集、SCOPUS和IEEE Xplore数据库(建库至2022年3月25日),共筛选1033篇文章,纳入18篇期刊论文和18篇会议论文。其中30篇为原型或建模文章,6篇为验证文章;所有文章均报告强化学习算法优于ICU专业人员的临床决策,但仅基于回顾性数据。全部文章偏倚风险高,主要因评估程序。作者结论为:未发现任何研究证明基于强化学习的技术能改善患者结局。

要点

  • 系统综述纳入18篇期刊论文和18篇会议论文,30篇为原型或建模文章,6篇为验证文章。
  • 所有文章均报告强化学习算法优于ICU专业人员的临床决策,但仅基于回顾性数据。
  • 全部文章偏倚风险高,主要因评估程序。
  • 未发现任何研究证明基于强化学习的技术能改善危重症患者结局。
  • 状态空间、动作空间、奖励函数、模型训练和评估的建模技术差异很大。

适用人群与场景

  • ICU人群(纳入标准:使用强化学习技术且报告患者健康相关结局的期刊文章;会议论文仅纳入就绪度评估)。

研究设计与关键结果

  • 研究类型:系统综述与Meta分析(原文称meta-analyze),检索PubMed、Embase.com、Clarivate Analytics/Web of Science核心合集、Elsevier/SCOPUS和IEEE Xplore数字图书馆,检索时间自建库至2022年3月25日,并进行后续引文追踪。
  • 共筛选1033篇文章,纳入18篇期刊论文和18篇会议论文;30篇为原型或建模文章,6篇为验证文章。
  • 所有文章均报告强化学习算法优于ICU专业人员的临床决策,但仅基于回顾性数据。
  • 状态空间、动作空间、奖励函数、强化学习模型训练和评估的建模技术差异很大。
  • 全部文章偏倚风险高,主要因评估程序。
  • 未发现任何研究证明基于强化学习的技术能改善患者结局。

证据类型、等级与局限

  • 证据类型:系统综述与Meta分析。
  • 全部文章偏倚风险高,主要因评估程序。
  • 所有报告强化学习优于临床医生的评估均基于回顾性off-policy评估。
  • 局限:原文未报告局限性。

对床旁实践的改变

  • 原文未据此给出推荐。

提炼说明

  • 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开

OBJECTIVE: Reinforcement learning (RL) is a machine learning technique uniquely effective at sequential decision-making, which makes it potentially relevant to ICU treatment challenges. We set out to systematically review, assess level-of-readiness and meta-analyze the effect of RL on outcomes for critically ill patients.

DATA SOURCES: A systematic search was performed in PubMed, Embase.com, Clarivate Analytics/Web of Science Core Collection, Elsevier/SCOPUS and the Institute of Electrical and Electronics Engineers Xplore Digital Library from inception to March 25, 2022, with subsequent citation tracking.

DATA EXTRACTION: Journal articles that used an RL technique in an ICU population and reported on patient health-related outcomes were included for full analysis. Conference papers were included for level-of-readiness assessment only. Descriptive statistics, characteristics of the models, outcome compared with clinician's policy and level-of-readiness were collected. RL-health risk of bias and applicability assessment was performed.

DATA SYNTHESIS: A total of 1,033 articles were screened, of which 18 journal articles and 18 conference papers, were included. Thirty of those were prototyping or modeling articles and six were validation articles. All articles reported RL algorithms to outperform clinical decision-making by ICU professionals, but only in retrospective data. The modeling techniques for the state-space, action-space, reward function, RL model training, and evaluation varied widely. The risk of bias was high in all articles, mainly due to the evaluation procedure.

CONCLUSION: In this first systematic review on the application of RL in intensive care medicine we found no studies that demonstrated improved patient outcomes from RL-based technologies. All studies reported that RL-agent policies outperformed clinician policies, but such assessments were all based on retrospective off-policy evaluation.

来源:Crit Care Med,查看原文。版权归原作者所有。

参考来源

  1. Otten M, Jagesar AR, Dam TA, et al. Does Reinforcement Learning Improve Outcomes for Critically Ill Patients? A Systematic Review and Level-of-Readiness Assessment. Crit Care Med. 2023. PMID: 37938042. doi:10.1097/CCM.0000000000006100

供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。