利用电子健康记录数据的自然语言处理验证重症监护结局预测模型
这项回顾性队列研究纳入2个学术医学中心(UCSF、BIDMC)和1家社区医院(MPMC)共20个ICU的101196例首次入住ICU且停留≥4小时的患者,评估在仅用实验室检查和生命体征最高/最低值的基线模型上,加入临床轨迹指标及NLP提取的文本项对院内死亡预测效力的提升。基线模型交叉验证AUC为0.831(95%CI 0.830-0.832),加入临床轨迹后升至0.899(95%CI 0.896-0.902,P<.001),再加入NLP衍生项后升至0.922(95%CI 0.916-0.924,P<.001)。跨院应用时NLP项仍改善模型表现(各院AUC差值P<.001)。作者认为这类自动化算法在临床研究和质量改进中的作用似乎值得进一步研究。
要点
- 基线模型仅用实验室检查和生命体征的最高/最低值,交叉验证AUC为0.831(95%CI 0.830-0.832)。
- 加入临床轨迹指标后AUC升至0.899(95%CI 0.896-0.902),较基线差异P<.001。
- 再加入NLP衍生文本项后AUC进一步升至0.922(95%CI 0.916-0.924),差异P<.001。
- 跨院应用NLP项时各院AUC差值:UCSF 0.077至0.021、MPMC 0.071至0.051、BIDMC 0.035至0.043,P<.001。
- 研究共纳入101196例患者,院内死亡率10.4%(n=10505),平均年龄61.3(17.1)岁,男性51.3%(n=51899)。
适用人群与场景
- 首次入住ICU且ICU停留时间至少4小时的患者,数据来自UCSF、BIDMC和MPMC共20个ICU,时间范围2001年1月1日至2017年6月1日。
研究设计与关键结果
- 研究类型为回顾性队列研究,纳入101196例首次入住ICU且停留≥4小时的患者,来自2个学术医学中心和1家社区医院共20个ICU。
- 基线模型(仅用每项实验室检查和生命体征的最高与最低观测值)交叉验证AUC为0.831(95%CI 0.830-0.832)。
- 基线模型加入临床轨迹指标后AUC为0.899(95%CI 0.896-0.902),与基线AUC差异P<.001。
- 进一步加入与死亡相关的NLP衍生项后AUC升至0.922(95%CI 0.916-0.924),P<.001。
- 跨院应用NLP项时各院AUC差值:UCSF为0.077至0.021,MPMC为0.071至0.051,BIDMC为0.035至0.043,各院加入NLP时P<.001。
- 患者中男性51.3%(n=51899),平均(SD)年龄61.3(17.1)岁,院内死亡率为10.4%(n=10505)。
- 模型校准采用改良Hosmer-Lemeshow统计量评估。
证据类型、等级与局限
- 证据类型:回顾性队列研究。
- 局限:原文未报告局限性。
对床旁实践的改变
- 作者认为,这些自动化算法(尤其是使用来自病历记录和其他来源的非结构化数据的算法)在临床研究和质量改进中的作用似乎值得进一步研究。
提炼说明
- 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开
IMPORTANCE: Accurate prediction of outcomes among patients in intensive care units (ICUs) is important for clinical research and monitoring care quality. Most existing prediction models do not take full advantage of the electronic health record, using only the single worst value of laboratory tests and vital signs and largely ignoring information present in free-text notes. Whether capturing more of the available data and applying machine learning and natural language processing (NLP) can improve and automate the prediction of outcomes among patients in the ICU remains unknown.
OBJECTIVES: To evaluate the change in power for a mortality prediction model among patients in the ICU achieved by incorporating measures of clinical trajectory together with NLP of clinical text and to assess the generalizability of this approach.
DESIGN, SETTING, AND PARTICIPANTS: This retrospective cohort study included 101 196 patients with a first-time admission to the ICU and a length of stay of at least 4 hours. Twenty ICUs at 2 academic medical centers (University of California, San Francisco [UCSF], and Beth Israel Deaconess Medical Center [BIDMC], Boston, Massachusetts) and 1 community hospital (Mills-Peninsula Medical Center [MPMC], Burlingame, California) contributed data from January 1, 2001, through June 1, 2017. Data were analyzed from July 1, 2017, through August 1, 2018.
MAIN OUTCOMES AND MEASURES: In-hospital mortality and model discrimination as assessed by the area under the receiver operating characteristic curve (AUC) and model calibration as assessed by the modified Hosmer-Lemeshow statistic.
RESULTS: Among 101 196 patients included in the analysis, 51.3% (n = 51 899) were male, with a mean (SD) age of 61.3 (17.1) years; their in-hospital mortality rate was 10.4% (n = 10 505). A baseline model using only the highest and lowest observed values for each laboratory test result or vital sign achieved a cross-validated AUC of 0.831 (95% CI, 0.830-0.832). In contrast, that model augmented with measures of clinical trajectory achieved an AUC of 0.899 (95% CI, 0.896-0.902; P < .001 for AUC difference). Further augmenting this model with NLP-derived terms associated with mortality further increased the AUC to 0.922 (95% CI, 0.916-0.924; P < .001). These NLP-derived terms were associated with improved model performance even when applied across sites (AUC difference for UCSF: 0.077 to 0.021; AUC difference for MPMC: 0.071 to 0.051; AUC difference for BIDMC: 0.035 to 0.043; P < .001) when augmenting with NLP at each site.
CONCLUSIONS AND RELEVANCE: Intensive care unit mortality prediction models incorporating measures of clinical trajectory and NLP-derived terms yielded excellent predictive performance and generalized well in this sample of hospitals. The role of these automated algorithms, particularly those using unstructured data from notes and other sources, in clinical research and quality improvement seems to merit additional investigation.
参考来源
- Marafino BJ, Park M, Davies JM, et al. Validation of Prediction Models for Critical Care Outcomes Using Natural Language Processing of Electronic Health Record Data. JAMA Netw Open. 2018. PMID: 30646310. doi:10.1001/jamanetworkopen.2018.5097
供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。