文献速递关节外科

大语言模型与基于规则的NLP算法提取手术记录数据的比较

Comparison of Large Language Models with Rules-Based Natural Language Processing Algorithms for Extracting Data from Operative Notes.
PubMed 文献 · J Bone Joint Surg Am · 2026-03-24

本研究从机构登记库随机抽取1,000例初次全髋关节置换术(THA)手术记录,由两名人工标注者审核手术入路、摩擦界面和固定技术3个数据点,并划分为开发集(n=239)和测试集(n=719)。研究为每个数据点开发了结合迭代定制提示词与大语言模型(LLM)的自动化提取流程,并与现有基于规则的NLP算法比较,结果显示LLM在所有数据点上的准确率均更优。

要点

  • LLM提取手术入路准确率96%,优于NLP算法的94%。
  • LLM提取摩擦界面准确率89%,优于NLP算法的74%。
  • LLM对摩擦界面描述不清的记录中80%能准确推断。

核心结论

  • LLM流程可从THA手术记录中提取3个登记相关数据点,性能优于现有NLP算法。

研究设计与样本

  • 从机构登记库随机抽取1,000例初次THA手术记录,两名人工标注者手动审核手术入路、摩擦界面和固定技术3个数据点。
  • 标注记录分为开发集(n=239)和测试集(n=719),为每个数据点开发结合迭代定制提示词与LLM的提取流程,并与现有基于规则的NLP算法比较性能。

主要发现

  • LLM准确率在手术入路为96%(NLP为94%),摩擦界面为89%(NLP为74%),固定技术为96%(NLP为95%)。
  • LLM对摩擦界面描述不清的记录中80%能准确推断摩擦界面。

临床怎么用

  • LLM可用于从电子病历自由文本中提取登记数据,有望简化、改进并普及基于手术记录的登记数据库构建。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: We aimed to develop automated data extraction pipelines with large language models (LLMs) to extract registry data from total hip arthroplasty (THA) operative notes and compare the performance with that of existing natural language processing (NLP) algorithms. METHODS: We randomly sampled 1,000 primary THA cases from our institutional registry. Two human annotators manually reviewed each operative note for 3 data points: surgical approach, bearing surface, and fixation technique. All labeled THA notes were split into the development set (n = 239) and the testing set (n = 719). We developed a custom data extraction pipeline for each data point by combining an iteratively customized prompt with an LLM. The performance was compared with that of existing rules-based NLP algorithms. RESULTS: The accuracy of LLMs was superior to that of NLP algorithms for all data points: surgical approach (96% compared with 94%), bearing surface (89% compared with 74%), and fixation technique (96% compared with 95%). Furthermore, the LLM accurately inferred the bearing surface for 80% of the notes that were ambiguous about the bearing surface. CONCLUSIONS: We developed LLM pipelines for extracting 3 registry-relevant data points from THA operative notes, demonstrating superior performance to existing NLP algorithms. CLINICAL RELEVANCE: LLMs have the potential to impact clinical care, including the evaluation of electronic medical record free-text data. As registries serve as a cornerstone of orthopaedic evidence, this work demonstrates promise for LLMs to simplify, improve, and democratize the construction of registry databases from operative notes.

来源:J Bone Joint Surg Am,查看原文。版权归原作者所有。

引用

  1. Yang L, Mulford KL, Girod-Hoffman MM, et al. Comparison of Large Language Models with Rules-Based Natural Language Processing Algorithms for Extracting Data from Operative Notes. J Bone Joint Surg Am. 2026. PMID: 41875224. doi:10.2106/JBJS.25.01338

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。