利用大语言模型从放射肿瘤学非结构化病历中填充结构化临床病例报告表
本研究评估预训练大语言模型(Llama-3.1-8b)能否从非英文常规病历中自动提取并结构化放疗相关信息以填充电子病例报告表(eCRF)。纳入100例接受MR-Linac治疗的前列腺癌患者,90例用于提示开发、10例独立测试,模型平均每份病历处理16秒,开发集与测试集匹配准确率分别为83.6%和83.8%。结论认为大语言模型可高效结构化非英文临床病历,但需多中心验证。
要点
- Llama-3.1-8b在开发集与测试集匹配准确率分别为83.6%和83.8%。
- 模型平均每份病历处理时间16秒,可自动填充eCRF。
- 人工复核发现约7.5%常规采集测试数据与复核值不符,提示真实标签存在误差。
核心结论
- 预训练大语言模型可从非英文病历中高准确度提取并分类放疗相关信息以支持eCRF自动填充。
- 模型处理每份病历平均耗时16秒,可在极少人员参与下准确填充CRF。
- 研究结果需多中心验证后方可推广。
研究设计与样本
- 研究纳入100例接受MR-Linac治疗的前列腺癌患者,其真实标签数据来自MOMENTUM数据库。
- 90例用于提示开发,10例用于独立测试;病历经提取、匿名化并按时间点分类。
- 使用Llama-3.1-8b模型,提示采用思维链(CoT)逻辑并设五个上下文示例,输出经后处理并与真实标签比较。
主要发现
- 模型在开发集和测试集的匹配准确率分别为83.6%和83.8%。
- 模型在开发集8.1%的病例和测试集8.6%的病例中与特定值不一致。
- 模型评估前独立人工复核显示约7.5%的常规采集测试数据与复核值不符,提示常规获取的真实标签存在不准确。
关键数值与推荐
- 模型预测值平均生成时间为每份病历16秒。
- 开发集匹配准确率83.6%,测试集匹配准确率83.8%。
- 开发集8.1%病例、测试集8.6%病例出现特定值不一致。
- 约7.5%常规采集测试数据与人工复核值不匹配。
临床怎么用
- 可将大语言模型用于从非英文常规病历中自动提取放疗相关信息并填充eCRF,以减少人工数据管理负担。
- 应用时需注意常规采集的真实标签本身可能存在约7.5%的不准确,应结合人工复核。
- 当前证据来自单中心前列腺癌MR-Linac患者,推广前需多中心验证。
局限性
- 研究仅纳入前列腺癌接受MR-Linac治疗的患者,结论外推受限。
- 常规采集的真实标签存在约7.5%的不准确,可能影响准确性评估。
- 需多中心验证后方可推广。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND AND PURPOSE: Large language models (LLMs) have shown growing potential for clinical text processing, but their systematic application in radiation oncology-especially for non-English clinical documentation-remains underexplored. This study investigated whether pretrained LLMs can automatically extract, analyze, and structure radiotherapy-relevant information from routine unstructured medical notes, with the goal of supporting automated population of electronic case report forms (eCRFs). MATERIALS AND METHODS: This study examined prostate cancer patients treated with the MR-Linac, for whom ground truth data exist in the MOMENTUM database. A total of 100 patients were included, with 90 used for prompt development and 10 for independent testing. Medical notes were extracted, anonymized, and categorized by time points. The Llama-3.1-8b model was used, with prompts designed using chain-of-thought (CoT) logic with five in-context examples. The model output was post-processed, and extracted data was compared against ground truth. RESULTS: Medical notes were successfully processed, with predicted values generated in an average time of 16 s per note. The LLM achieved matching accuracies of 83.6% and 83.8% on the development and testing datasets. Analysis revealed that the model disagreed with specific values in 8.1% of development dataset cases and 8.6% of testing dataset cases. An independent manual review before model evaluation showed approximately 7.5% of routinely collected test data did not match reviewed values, indicating inaccuracies in the routinely acquired ground truth. CONCLUSION: This study demonstrated the effectiveness of LLMs in structuring clinical data from medical non-English notes, with high accuracy in extracting and categorizing information. While multi-institutional validation is needed, the results indicate a significant healthcare impact through efficient data management, processing notes in 16 s, and accurately populating CRFs with minimal staff involvement.
引用
- Nachbar M, Yi N, Büttner M, et al. Leveraging large language models to populate structured clinical case report forms from unstructured medical notes in radiation oncology. Clin Transl Radiat Oncol. 2026. PMID: 41859030. doi:10.1016/j.ctro.2026.101143
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。