大型语言模型在非结构化电子健康记录中识别卒中分型的准确性
本研究评估GPT-4o从非结构化临床笔记中识别卒中类型和缺血性卒中亚型的准确性,以Get With The Guidelines-Stroke登记库为金标准,纳入4123例卒中住院患者。外部验证中,GPT-4o分类卒中类型的准确率达98%,但对缺血性卒中亚型的敏感性差异大,从隐源性0.40到小血管闭塞0.95。零样本思维链提示策略表现与更费力策略相当,重复查询一致性>99%。
要点
- GPT-4o分类卒中类型(缺血vs出血)准确率98%,敏感性0.98,特异性0.97。
- 缺血性卒中亚型敏感性差异大:隐源性0.40,小血管闭塞0.95。
- 零样本思维链提示即可达到与复杂策略相当的性能,重复查询一致性>99%。
核心结论
- GPT-4o在分类卒中类型方面表现出高准确性,但在缺血性卒中亚型分类上面临挑战。
研究设计与样本
- 研究采用检索增强生成框架与GPT-4o,以Get With The Guidelines-Stroke登记库为金标准,使用UT Southwestern Medical Center 20%数据开发,80%测试,并用Parkland Health and Hospital System数据外部验证。
- 共纳入2019年1月至2023年8月的4123例卒中住院患者(UTSW: n=2047; PHHS: n=2076)。
- 评估了三种提示策略:零样本思维链、专家引导和基于指令的提示。
主要发现
- 外部验证集中,79.6%的患者为缺血性卒中,20.4%为出血性卒中。
- GPT-4o分类卒中类型的准确率为98%(95% CI, 0.97-0.99),敏感性为0.98(95% CI, 0.97-0.99),特异性为0.97(95% CI, 0.96-0.98)。
- 缺血性卒中亚型敏感性从隐源性的0.40(95% CI, 0.31-0.49)到小血管闭塞的0.95(95% CI, 0.93-0.97)。
- 缺血性卒中亚型特异性从大动脉粥样硬化的0.94(95% CI, 0.92-0.96)到心源性栓塞的0.98(95% CI, 0.97-0.99)。
- 零样本思维链提示策略表现与更费力的策略相当,重复查询一致性超过99%。
临床怎么用
- GPT-4o可高精度识别卒中类型,但缺血性卒中亚型分类需谨慎,尤其隐源性卒中敏感性低。
- 零样本思维链提示策略高效,可减少人工输入,适合初步筛查。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: While International Classification of Diseases, Tenth Revision codes suffice for identifying stroke events in surveillance, accurately classifying stroke types and subtypes using electronic health records remains challenging due to limitations in structured data. This often necessitates manual review of clinical documentation. This study evaluated whether a large language model, Generative Pre-Trained Transformer 4 Omni (GPT-4o), can accurately identify stroke types and subtypes from unstructured clinical notes. METHODS: We implemented a retrieval-augmented generation framework with GPT-4o to classify stroke types (ischemic versus hemorrhagic) and ischemic stroke subtypes using electronic health records data. The American Heart Association Get With The Guidelines-Stroke registry served as the gold standard. Model development used a 20% subset of Get With The Guidelines-Stroke-linked data from UT Southwestern Medical Center (UTSW), with the remaining 80% reserved for testing. External validation used data from the Parkland Health and Hospital System (PHHS). A total of 4123 stroke hospitalizations from January 2019 to August 2023 were included (UTSW: n=2047; PHHS: n=2076). Three prompting strategies-zero-shot chain-of-thought, expert-guided, and instruction-based-were evaluated. Predictions of GPT-4os were compared with classifications made by trained abstractors contributing to the Get With The Guidelines-Stroke registry. RESULTS: In the external validation set, 79.6% of patients had ischemic stroke and 20.4% hemorrhagic. GPT-4o achieved 98% accuracy (95% CI, 0.97-0.99) in classifying stroke type, where accuracy reflects the overall proportion of correctly classified patients. Sensitivity was 0.98 (95% CI, 0.97-0.99), and specificity was 0.97 (95% CI, 0.96-0.98). For ischemic stroke subtypes, sensitivity ranged from 0.40 (95% CI, 0.31-0.49) for cryptogenic to 0.95 (95% CI, 0.93-0.97) for small-vessel occlusion. Specificity ranged from 0.94 (95% CI, 0.92-0.96) for large-artery atherosclerosis to 0.98 (95% CI, 0.97-0.99) for cardioembolism. Zero-shot chain-of-thought prompting-requiring minimal human input-performed comparably to more labor-intensive strategies. Consistency analysis revealed >99% agreement across repeated queries. CONCLUSIONS: GPT-4o demonstrated strong accuracy in classifying stroke types but faced challenges with ischemic subtypes.
引用
- Owens D, Nguyen DQ, Dohopolski M, et al. Accuracy of Large Language Models to Identify Stroke Subtypes Within Unstructured Electronic Health Record Data. Stroke. 2025. PMID: 40709446. doi:10.1161/STROKEAHA.125.051993
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。