大语言模型在围术期风险预测与预后判断中的能力
该预后研究基于2年回顾性电子健康记录数据,构建任务特异性数据集,评估GPT-4 Turbo在8项围术期预测任务上的表现。结果显示,模型在分类任务(如ICU入住、院内死亡)上表现尚可,但对数值时长预测普遍较差。
要点
- GPT-4 Turbo在分类任务中表现中等,ICU入住F1为0.81,院内死亡F1为0.86。
- 数值时长预测误差大,PACU第一阶段时长平均绝对误差达49分钟。
- 模型生成的解释可能辅助临床工作流,但不足以替代传统风险模型。
核心结论
- 当前通用大语言模型可辅助分类任务的围术期风险分层,但对数值时长预测不足。
研究设计与样本
- 预后研究,基于2年回顾性电子健康记录数据构建任务特异性数据集,各任务1000例(非计划入院949例,院内死亡576例)。
主要发现
- 最佳F1:ASA-PS 0.50(95%CI 0.47-0.53),入院0.64(95%CI 0.61-0.67),ICU入住0.81(95%CI 0.78-0.83),非计划入院0.61(95%CI 0.58-0.64),院内死亡0.86(95%CI 0.83-0.89)。
- 时长预测平均绝对误差:PACU第一阶段49分钟(95%CI 46-51分钟),住院4.5天(95%CI 4.2-5.0天),ICU 1.1天(95%CI 0.9-1.3天)。
临床怎么用
- 可将通用大语言模型用于围术期分类风险分层(如ICU入住、院内死亡),但不应依赖其预测具体时长。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
IMPORTANCE: General-domain large language models may be able to perform risk stratification and predict postoperative outcome measures using a description of the procedure and a patient's electronic health record notes. OBJECTIVE: To examine predictive performance on 8 different tasks: prediction of American Society of Anesthesiologists Physical Status (ASA-PS), hospital admission, intensive care unit (ICU) admission, unplanned admission, hospital mortality, postanesthesia care unit (PACU) phase 1 duration, hospital duration, and ICU duration. DESIGN, SETTING, AND PARTICIPANTS: This prognostic study included task-specific datasets constructed from 2 years of retrospective electronic health records data collected during routine clinical care. Case and note data were formatted into prompts and given to the large language model GPT-4 Turbo (OpenAI) to generate a prediction and explanation. The setting included a quaternary care center comprising 3 academic hospitals and affiliated clinics in a single metropolitan area. Patients who had a surgery or procedure with anesthesia and at least 1 clinician-written note filed in the electronic health record before surgery were included in the study. Data were analyzed from November to December 2023. EXPOSURES: Compared original notes, note summaries, few-shot prompting, and chain-of-thought prompting strategies. MAIN OUTCOMES AND MEASURES: F1 score for binary and categorical outcomes. Mean absolute error for numerical duration outcomes. RESULTS: Study results were measured on task-specific datasets, each with 1000 cases with the exception of unplanned admission, which had 949 cases, and hospital mortality, which had 576 cases. The best results for each task included an F1 score of 0.50 (95% CI, 0.47-0.53) for ASA-PS, 0.64 (95% CI, 0.61-0.67) for hospital admission, 0.81 (95% CI, 0.78-0.83) for ICU admission, 0.61 (95% CI, 0.58-0.64) for unplanned admission, and 0.86 (95% CI, 0.83-0.89) for hospital mortality prediction. Performance on duration prediction tasks was universally poor across all prompt strategies for which the large language model achieved a mean absolute error of 49 minutes (95% CI, 46-51 minutes) for PACU phase 1 duration, 4.5 days (95% CI, 4.2-5.0 days) for hospital duration, and 1.1 days (95% CI, 0.9-1.3 days) for ICU duration prediction. CONCLUSIONS AND RELEVANCE: Current general-domain large language models may assist clinicians in perioperative risk stratification on classification tasks but are inadequate for numerical duration predictions. Their ability to produce high-quality natural language explanations for the predictions may make them useful tools in clinical workflows and may be complementary to traditional risk prediction models.
引用
- Chung P, Fong CT, Walters AM, et al. Large Language Model Capabilities in Perioperative Risk Prediction and Prognostication. JAMA Surg. 2024. PMID: 38837145. doi:10.1001/jamasurg.2024.1621
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。