文献速递神经介入DSA

使用ChatGPT从急性缺血性卒中机械取栓自由文本报告中提取数据:一项回顾性分析

Data Extraction from Free-Text Reports on Mechanical Thrombectomy in Acute Ischemic Stroke Using ChatGPT: A Retrospective Analysis.
PubMed 文献 · Radiology · 2024-01-01

GPT-4正确提取94.0%(2631/2800)的数据条目,GPT-3.5为63.9%(1788/2800),差异有统计学意义(P < .001)。

要点

  • GPT-4从取栓报告中正确提取94.0%的数据条目,GPT-3.5为63.9%。
  • 外部报告验证中GPT-4正确率90.5%,GPT-3.5为64.2%。
  • 两模型均能成功处理全部报告,但GPT-4无需进一步后处理。

核心结论

  • 与GPT-3.5相比,GPT-4更常从缺血性卒中机械取栓自由文本报告中正确提取操作数据。

研究设计与样本

  • 回顾性研究,纳入机构1在2022年11月至2023年9月、机构2在2016年9月至2019年12月期间接受机械取栓的连续缺血性卒中患者报告,共100份内部报告(100例患者,平均年龄74.7岁±13.2,53例女性)和30份外部报告(30例患者,平均年龄72.7岁±13.5,18例男性);以介入神经放射科医生手动提取数据为参考标准;用20份报告优化提示词;用McNemar检验比较两模型。

诊断标准与关键数值

  • GPT-4从2800个数据条目中正确提取2631个(94.0% [95% CI: 93.0, 94.8];各类别范围61%-100%),GPT-3.5正确提取1788个(63.9% [95% CI: 62.0, 65.6];各类别范围14%-99%;P < .001);外部报告中GPT-4正确提取760/840个(90.5% [95% CI: 88.3, 92.4]),GPT-3.5正确提取539/840个(64.2% [95% CI: 60.8, 67.4];P < .001)。

临床应用与阅片要点

  • 作者结论认为,与GPT-3.5相比,GPT-4更常从缺血性卒中机械取栓自由文本报告中正确提取操作数据。

局限性

  • 原文未报告局限性

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

Background Procedural details of mechanical thrombectomy in patients with ischemic stroke are important predictors of clinical outcome and are collected for prospective studies or national stroke registries. To date, these data are collected manually by human readers, a labor-intensive task that is prone to errors. Purpose To evaluate the use of the large language models (LLMs) GPT-4 and GPT-3.5 to extract data from neuroradiology reports on mechanical thrombectomy in patients with ischemic stroke. Materials and Methods This retrospective study included consecutive reports from patients with ischemic stroke who underwent mechanical thrombectomy between November 2022 and September 2023 at institution 1 and between September 2016 and December 2019 at institution 2. A set of 20 reports was used to optimize the prompt, and the ability of the LLMs to extract procedural data from the reports was compared using the McNemar test. Data manually extracted by an interventional neuroradiologist served as the reference standard. Results A total of 100 internal reports from 100 patients (mean age, 74.7 years ± 13.2 [SD]; 53 female) and 30 external reports from 30 patients (mean age, 72.7 years ± 13.5; 18 male) were included. All reports were successfully processed by GPT-4 and GPT-3.5. Of 2800 data entries, 2631 (94.0% [95% CI: 93.0, 94.8]; range per category, 61%-100%) data points were correctly extracted by GPT-4 without the need for further postprocessing. With 1788 of 2800 correct data entries, GPT-3.5 produced fewer correct data entries than did GPT-4 (63.9% [95% CI: 62.0, 65.6]; range per category, 14%-99%; P < .001). For the external reports, GPT-4 extracted 760 of 840 (90.5% [95% CI: 88.3, 92.4]) correct data entries, while GPT-3.5 extracted 539 of 840 (64.2% [95% CI: 60.8, 67.4]; P < .001). Conclusion Compared with GPT-3.5, GPT-4 more frequently extracted correct procedural data from free-text reports on mechanical thrombectomy performed in patients with ischemic stroke. © RSNA, 2024 Supplemental material is available for this article.

来源:Radiology,查看原文。版权归原作者所有。

引用

  1. Lehnen NC, Dorn F, Wiest IC, et al. Data Extraction from Free-Text Reports on Mechanical Thrombectomy in Acute Ischemic Stroke Using ChatGPT: A Retrospective Analysis. Radiology. 2024. PMID: 38625006. doi:10.1148/radiol.232741

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。