自然语言处理算法提取全髋关节置换术手术记录中常见数据元素的外部验证
该研究在OrthoCarolina的电子健康记录上外部验证并改进既往在Mayo Clinic开发的自然语言处理(NLP)算法,随机选取2018年至2021年的39份初次全髋关节置换术(THA)手术报告进行评估。手术入路、固定方式和承载面三种算法的准确率均达100%,而在开发中心的验证准确率分别为99.2%、90.7%和95.8%。结论认为NLP算法在外中心数据中展现出优异准确率,对经普通软件转换为可读文本的扫描PDF无功能性问题,为将扫描PDF作为大型注册登记数据来源提供了前景。
要点
- 在39份外中心THA手术报告上,手术入路、固定方式和承载面NLP算法的准确率均达100%。
- 开发中心验证准确率分别为手术入路99.2%、固定技术90.7%、承载面95.8%。
- 算法对经Adobe软件转换为可读文本的扫描PDF无功能性问题。
核心结论
- NLP算法应用于外中心数据时,在 delineating THA手术记录常见元素方面展现出优异准确率。
- 算法对经普通软件转换为可读文本的扫描PDF无功能性问题。
- 这些发现为NLP应用于扫描PDF作为开发大型注册登记的数据来源提供了前景,可从超大型非结构化数据集中快速且经济有效地可靠提取感兴趣数据。
研究设计与样本
- 本研究为外部验证与算法改进研究,在OrthoCarolina的电子健康记录上部署并改进既往在Mayo Clinic开发的NLP算法。
- 评估对象为随机选取的39份2018年至2021年的初次全髋关节置换术手术报告。
- 手术报告仅有PDF格式,需用Adobe软件转换为可读文本。
- 以人工病历审查为参照计算准确率统计量。
主要发现
- 手术入路算法在外中心数据上准确率为100%。
- 固定技术算法在外中心数据上准确率为100%。
- 承载面算法在外中心数据上准确率为100%。
- 在开发中心验证的性能为:手术入路准确率99.2%,固定技术准确率90.7%,承载面准确率95.8%。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: Natural language processing (NLP) systems are distinctive in their ability to extract critical information from raw text in electronic health records (EHR). We previously developed three algorithms for total hip arthroplasty (THA) operative notes with rules aimed at capturing (1) operative approach, (2) fixation method, and (3) bearing surface using inputs from a single institution. The purpose of this study was to externally validate and improve these algorithms as a prerequisite for broader adoption in automated registry data curation. METHODS: The previous NLP algorithms developed at Mayo Clinic were deployed and refined on EHRs from OrthoCarolina, evaluating 39 randomly selected primary THA operative reports from 2018 to 2021. Operative reports were available only in PDF format, requiring conversion to "readable" text with Adobe software. Accuracy statistics were calculated against manual chart review. RESULTS: The operative approach, fixation technique, and bearing surface algorithms all demonstrated perfect accuracy of 100%. By comparison, validated performance at the developing center yielded an accuracy of 99.2% for operative approach, 90.7% for fixation technique, and 95.8% for bearing surface. CONCLUSION: NLP algorithms applied to data from an external center demonstrated excellent accuracy in delineating common elements in THA operative notes. Notably, the algorithms had no functional problems evaluating scanned PDFs that were converted to "readable" text by common software. Taken together, these findings provide promise for NLP applied to scanned PDFs as a source to develop large registries by reliably extracting data of interest from very large unstructured data sets in an expeditious and cost-effective manner.
引用
- Wyles CC, Fu S, Odum SL, et al. External Validation of Natural Language Processing Algorithms to Extract Common Data Elements in THA Operative Notes. J Arthroplasty. 2022. PMID: 36280160. doi:10.1016/j.arth.2022.10.031
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。