文献速递主动脉疾病

验证自然语言处理在大型整合医疗系统中确定腹主动脉瘤存在及大小的准确性

Validation of natural language processing to determine the presence and size of abdominal aortic aneurysms in a large integrated health system.
PubMed 文献 · J Vasc Surg · 2021-02-04

本研究基于大型整合医疗系统2003至2017年230,660份影像报告,使用商业NLP算法识别腹主动脉瘤及最大直径,以18,000份盲法专家人工审阅为参考标准。NLP识别动脉瘤存在的kappa为0.84(95%CI 0.83-0.85),PPV 95%、敏感度88.5%;识别最大直径的kappa为0.88(95%CI 0.87-0.89),PPV 88.8%、敏感度88.2%。

要点

  • NLP可高精度识别影像报告中的腹主动脉瘤存在与最大直径。
  • NLP与专家审阅一致性高(kappa 0.79-0.88)。
  • 该工具可用于构建动脉瘤随访队列和筛查项目。

核心结论

  • NLP软件能准确分析大量放射报告数据以检测腹主动脉瘤,并构建基于主动脉直径的队列用于纵向分析。

研究设计与样本

  • 回顾性验证研究,纳入2003至2017年32,778例患者的前瞻性动脉瘤监测登记中230,660份影像报告,以18,000份随机抽取的盲法专家人工审阅为参考标准。

主要发现

  • 在18,000份专家审阅报告中,48.7%为腹主动脉瘤阳性。
  • NLP确认动脉瘤存在的kappa为0.84(95%CI 0.83-0.85),PPV 95%,敏感度88.5%。
  • NLP确认动脉瘤不存在的kappa为0.79(95%CI 0.799-0.80),PPV 77.7%,敏感度91.9%。
  • NLP正确识别最大主动脉直径的kappa为0.88(95%CI 0.87-0.89),PPV 88.8%,敏感度88.2%。

临床怎么用

  • 可用于从电子病历中识别术前和术后失访的腹主动脉瘤患者。
  • 可优化腹主动脉瘤持续监测中的人力资源配置。
  • 可促进腹主动脉瘤筛查项目的构建与实施。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVE: Previous studies of the natural history of abdominal aortic aneurysms (AAAs) have been limited by small cohort sizes or heterogeneous analyses of pooled data. By quickly and efficiently extracting imaging data from the health records, natural language processing (NLP) has the potential to substantially improve how we study and care for patients with AAAs. The aim of the present study was to test the ability of an NLP tool to accurately identify the presence or absence of AAAs and detect the maximal abdominal aortic diameter in a large dataset of imaging study reports. METHODS: Relevant imaging study reports (n = 230,660) from 2003 to 2017 were obtained for 32,778 patients followed up in a prospective aneurysm surveillance registry within a large, diverse, integrated healthcare system. A commercially available NLP algorithm was used to assess the presence of AAAs, confirm the absence of AAAs, and extract the maximal diameter of the abdominal aorta, if stated. A blinded expert manual review of 18,000 randomly selected imaging reports was used as the reference standard. The positive predictive value (PPV or precision), sensitivity (recall), and the kappa statistics were calculated. RESULTS: Of the randomly selected 18,000 studies that underwent expert manual review, 48.7% were positive for AAAs. In confirming the presence of an AAA, the interrater reliability of the NLP compared with the expert review showed a kappa value of 0.84 (95% confidence interval [CI], 0.83-0.85), with a PPV of 95% and sensitivity of 88.5%. The NLP algorithm showed similar results for confirming the absence of an AAA, with a kappa of 0.79 (95% CI, 0.799-0.80), PPV of 77.7%, and sensitivity of 91.9%. The kappa, PPV, and sensitivity of the NLP for correctly identifying the maximal aortic diameter was 0.88 (95% CI, 0.87-0.89), 88.8%, and 88.2% respectively. CONCLUSIONS: The use of NLP software can accurately analyze large volumes of radiology report data to detect AAA disease and assemble a contemporary aortic diameter-based cohort of patients for longitudinal analysis to guide surveillance, medical management, and operative decision making. It can also potentially be used to identify from the electronic medical records pre- and postoperative AAA patients "lost to follow-up," leverage human resources engaged in the ongoing surveillance of patients with AAAs, and facilitate the construction and implementation of AAA screening programs.

来源:J Vasc Surg,查看原文。版权归原作者所有。

引用

  1. McLenon M, Okuhn S, Lancaster EM, et al. Validation of natural language processing to determine the presence and size of abdominal aortic aneurysms in a large integrated health system. J Vasc Surg. 2021. PMID: 33548429. doi:10.1016/j.jvs.2020.12.090

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。