文献速递心脑血管急症重症综合管理与镇静镇痛

头部CT报告自然语言处理识别颅内占位效应:CTIME算法

Natural language processing of head CT reports to identify intracranial mass effect: CTIME algorithm.
PubMed 文献 · Am J Emerg Med · 2021-11-09
一句话结论NLP算法可从约80%头部CT报告中准确识别颅内占位效应,支持自动计算MPM。

研究开发并测试自然语言处理算法,从成人ICU患者初始头部CT报告中识别「颅内占位效应」。报告经至少两名标注者标记,随机分为80%训练集和20%测试集,比较TF-IDF、Word2Vec和通用句子编码器三种向量化策略,并用XGBoost分类。TF-IDF策略AUROC最高,为0.9625(95%CI 0.9443-0.9807),82%报告可明确分类为阳性或阴性,有助于基于电子健康记录自动计算死亡概率模型。

要点

  • TF-IDF向量化联合XGBoost识别颅内占位效应,测试集AUROC为0.9625(95%CI 0.9443-0.9807)。
  • TF-IDF评分>0.5判为「阳性」,似然比24.59;0.05-0.5为「不确定」,似然比0.99;<0.05为「阴性」,似然比0.05。
  • 82%的报告可被分类为「阳性」或「阴性」;测试集中「阴性」报告假阴性仅4/199(2.0%),「阳性」报告假阳性仅8/93(8.6%)。
  • 训练集1202份报告中308份(25.6%)人工标注存在颅内占位效应;测试集355份中108份(30.4%)标注存在。
  • 该算法识别颅内占位效应可支持基于电子健康记录数据自动计算死亡概率模型,用于许多应用场景。

适用人群与场景

  • 2013年10月至2016年9月期间从急诊科收入ICU的成人患者的初始头部CT报告。

诊断标准与关键阈值

  • 颅内占位效应标注:每份头部CT报告由五名独立标注者中至少两名标记为「是/否」存在颅内占位效应。
  • TF-IDF评分分类:>0.5为「阳性」(似然比24.59);0.05-0.5为「不确定」(似然比0.99);<0.05为「阴性」(似然比0.05)。

研究设计与关键结果

  • 研究类型为算法开发与测试;训练集1202份CT报告中308份(25.6%)人工标注为存在颅内占位效应,测试集355份中108份(30.4%)标注为存在。
  • 以TF-IDF向量化作为XGBoost模型输入的AUROC最高,为0.9625(95%CI 0.9443-0.9807)。
  • 测试集中,分类为「阴性」的报告仅4/199(2.0%)为假阴性;分类为「阳性」的报告仅8/93(8.6%)为假阳性。
  • 82%的报告被分类为「阳性」或「阴性」。

证据类型、等级与局限

  • 证据类型:算法开发与验证研究(训练集/测试集划分)。
  • 局限:原文未报告局限性。

对床旁实践的改变

  • 作者认为自然语言处理可从头部CT自由文本报告中准确识别颅内占位效应,覆盖约80%的记录,足以在许多应用中基于电子健康记录数据自动计算死亡概率模型。

提炼说明

  • 本条依据论文摘要整理;摘要未涉及「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开

BACKGROUND: The Mortality Probability Model (MPM) is used in research and quality improvement to adjust for severity of illness and can also inform triage decisions. However, a limitation for its automated use or application is that it includes the variable "intracranial mass effect" (IME), which requires human engagement with the electronic health record (EHR). We developed and tested a natural language processing (NLP) algorithm to identify IME from CT head reports.

METHODS: We obtained initial CT head reports from adult patients who were admitted to the ICU from our ED between 10/2013 and 9/2016. Each head CT head report was labeled yes/no IME by at least two of five independent labelers. The reports were then randomly divided 80/20 into training and test sets. All reports were preprocessed to remove linguistic and style variability, and a dictionary was created to map similar common terms. We tested three vectorization strategies: Term Frequency-Inverse Document frequency (TF-IDF), Word2Vec, and Universal Sentence Encoder to convert the report text to a numerical vector. This vector served as the input to a classification-tree-based ensemble machine learning algorithm (XGBoost). After training, model performance was assessed in the test set using the area under the receiver operating characteristic curve (AUROC). We also divided the continuous range of scores into positive/inconclusive/negative categories for IME.

RESULTS: Of the 1202 CT reports in the training set, 308 (25.6%) reports were manually labeled as "yes" for IME. Of the 355 reports in the test set, 108 (30.4%) were labeled as "yes" for IME. The TF-IDF vectorization strategy as an input for the XGBoost model had the best AUROC:-- 0.9625 (95% CI 0.9443-0.9807). TF-IDF score categories were defined and had the following likelihood ratios: "positive" (TF-IDF score > 0.5) LR = 24.59; "inconclusive" (TF-IDF 0.05-0.5) LR = 0.99; and "negative" (TF-IDF < 0.05) LR = 0.05. 82% of reports were classified as either "positive" or "negative". In the test set, only 4 of 199 (2.0%) reports with a "negative" classification were false negatives and only 8 of 93 (8.6%) reports classified as "positive" were false positives.

CONCLUSION: NLP can accurately identify IME from free-text reports of head CTs in approximately 80% of records, adequate to allow automatic calculation of MPM based on EHR data for many applications.

来源:Am J Emerg Med,查看原文。版权归原作者所有。

参考来源

  1. Gordon AJ, Banerjee I, Block J, et al. Natural language processing of head CT reports to identify intracranial mass effect: CTIME algorithm. Am J Emerg Med. 2021. PMID: 34839182. doi:10.1016/j.ajem.2021.11.001

供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。