重症监护患者院内死亡风险调整的自动化机器学习算法的开发与评估
该研究使用现代机器学习方法和开源工具,开发并评估了一个针对ICU患者院内死亡的回顾性风险调整算法——风险院内死亡评分(Risk of Inpatient Death score),该评分可完全自动化并依赖医院常规流程中产生的数据元素。数据来自Tenet Healthcare运营的53家医院中的131个ICU,纳入2014年1月至2016年12月出院的237173例ICU患者,随机分为训练集(36家医院)和验证集(17家医院)。最终模型保留17个特征,包括临床和行政数据元素;验证集中风险院内死亡评分展现出优秀的区分度(AUC=0.94)和校准度(调整Brier分数=52.8%),这些结果优于最常用死亡风险调整算法的已发表性能统计。作者认为,自动化风险调整算法有潜力消除已知的采用障碍,如高昂的许可费和显著直接人力成本,但需要进一步评估以确保在独立站点能达到本研究观察到的性能水平。
要点
- 风险院内死亡评分可完全自动化,仅依赖医院常规流程中产生的数据元素,无需额外人力数据收集。
- 在237173例ICU患者中,验证集区分度AUC为0.94,校准度调整Brier分数为52.8%。
- 最终模型仅保留17个特征,包括临床和行政数据元素的混合。
- 该评分性能与最常用死亡风险调整算法的已发表性能统计相比更优。
- 作者认为自动化算法可消除许可费和直接人力成本等采用障碍,但需在独立站点进一步评估性能。
适用人群与场景
- 2014年1月至2016年12月出院的237173例ICU患者,来自Tenet Healthcare运营的53家医院中的131个ICU。
- 训练集来自36家医院,验证集来自17家医院。
研究设计与关键结果
- 研究类型为回顾性风险调整算法开发与评估,数据随机分为训练集(36家医院)和验证集(17家医院)。
- 最终模型保留17个特征,包括临床和行政数据元素的混合。
- 验证集中风险院内死亡评分的AUC为0.94,调整Brier分数为52.8%。
- 上述结果与最常用死亡风险调整算法的已发表性能统计相比更优。
证据类型、等级与局限
- 证据类型:回顾性算法开发与验证研究。
- 局限:作者指出需要进一步评估以确保在本研究观察到的性能水平能在独立站点实现。
对床旁实践的改变
- 风险院内死亡评分可完全自动化并依赖常规医院流程数据,可能减少与人力数据收集相关的劳动成本。
- 作者认为自动化风险调整算法有潜力消除已知采用障碍,如高昂的许可费和显著直接人力成本。
- 作者指出需要进一步评估以确保在独立站点能达到本研究观察到的性能水平。
提炼说明
- 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开
OBJECTIVES: Risk adjustment algorithms for ICU mortality are necessary for measuring and improving ICU performance. Existing risk adjustment algorithms are not widely adopted. Key barriers to adoption include licensing and implementation costs as well as labor costs associated with human-intensive data collection. Widespread adoption of electronic health records makes automated risk adjustment feasible. Using modern machine learning methods and open source tools, we developed and evaluated a retrospective risk adjustment algorithm for in-hospital mortality among ICU patients. The Risk of Inpatient Death score can be fully automated and is reliant upon data elements that are generated in the course of usual hospital processes.
SETTING: One hundred thirty-one ICUs in 53 hospitals operated by Tenet Healthcare.
PATIENTS: A cohort of 237,173 ICU patients discharged between January 2014 and December 2016.
DESIGN: The data were randomly split into training (36 hospitals), and validation (17 hospitals) data sets. Feature selection and model training were carried out using the training set while the discrimination, calibration, and accuracy of the model were assessed in the validation data set.
MEASUREMENTS AND MAIN RESULTS: Model discrimination was evaluated based on the area under receiver operating characteristic curve; accuracy and calibration were assessed via adjusted Brier scores and visual analysis of calibration curves. Seventeen features, including a mix of clinical and administrative data elements, were retained in the final model. The Risk of Inpatient Death score demonstrated excellent discrimination (area under receiver operating characteristic curve = 0.94) and calibration (adjusted Brier score = 52.8%) in the validation dataset; these results compare favorably to the published performance statistics for the most commonly used mortality risk adjustment algorithms.
CONCLUSIONS: Low adoption of ICU mortality risk adjustment algorithms impedes progress toward increasing the value of the healthcare delivered in ICUs. The Risk of Inpatient Death score has many attractive attributes that address the key barriers to adoption of ICU risk adjustment algorithms and performs comparably to existing human-intensive algorithms. Automated risk adjustment algorithms have the potential to obviate known barriers to adoption such as cost-prohibitive licensing fees and significant direct labor costs. Further evaluation is needed to ensure that the level of performance observed in this study could be achieved at independent sites.
参考来源
- Delahanty RJ, Kaufman D, Jones SS, et al. Development and Evaluation of an Automated Machine Learning Algorithm for In-Hospital Mortality Risk Adjustment Among Critical Care Patients. Crit Care Med. 2018. PMID: 29419557. doi:10.1097/CCM.0000000000003011
供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。