深度学习模型在基于自然语言处理的非英语头颅CT报告分类中的比较
本研究回顾性收集2011至2018年希伯来语头颅CT报告,人工标注7784份(正常46.3%、异常53.7%,颅内出血7.1%),比较LSTM、LSTM-ATN及Word2Vec词嵌入与词袋模型。通用分类中LSTM-ATN-Word2Vec表现最佳(AUC=0.967±0.006,准确率90.8%±0.01);特定分类中所有方法准确率95.0%–95.9%,LSTM-ATN-Word2Vec与词袋模型AUC最高(0.970)。结论认为通用任务中词嵌入与注意力机制提升性能,特定任务中词袋与深度学习结果相似,模型需针对NLP任务探索与定制。
要点
- 通用正常vs异常分类中LSTM-ATN-Word2Vec最佳,AUC=0.967±0.006、准确率90.8%±0.01
- 颅内出血特定分类各方法准确率95.0%–95.9%,词袋与LSTM-ATN-Word2Vec的AUC均为0.970
- 使用大样本非英语报告预训练词嵌入并结合注意力机制可提升通用NLP任务性能
核心结论
- 通用分类任务中,利用大样本非英语头颅CT报告进行词嵌入并结合注意力机制可提升NLP性能。
- 更特定的颅内出血分类任务中,词袋模型与深度学习模型结果相似。
- 模型应针对具体NLP任务进行探索与定制。
研究设计与样本
- 回顾性收集2011–2018年希伯来语签署的头颅CT报告,共检索176,988份用于预训练嵌入层。
- 每年1–2月成人急诊报告进行人工标注,共7784份,正常占46.3%、异常占53.7%,颅内出血占7.1%。
- 比较LSTM与LSTM-ATN网络,并评估加入Word2Vec词嵌入的改进,同时与词袋模型对比。
诊断标准与关键数值
- 通用标注任务(正常vs异常):LSTM-ATN-Word2Vec的AUC=0.967±0.006、准确率90.8%±0.01。
- 特定标注任务(有无颅内出血):所有方法准确率在95.0%–95.9%之间。
- 特定标注任务中LSTM-ATN-Word2Vec与词袋模型的AUC均为0.970。
临床应用与阅片要点
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
PURPOSE: Natural language processing (NLP) can be used for automatic flagging of radiology reports. We assessed deep learning models for classifying non-English head CT reports.
METHODS: We retrospectively collected head CT reports (2011-2018). Reports were signed in Hebrew. Emergency department (ED) reports of adult patients from January to February for each year (2013-2018) were manually labeled. All other reports were used to pre-train an embedding layer. We explored two use cases: (1) general labeling use case, in which reports were labeled as normal vs. pathological; (2) specific labeling use case, in which reports were labeled as with and without intra-cranial hemorrhage. We tested long short-term memory (LSTM) and LSTM-attention (LSTM-ATN) networks for classifying reports. We also evaluated the improvement of adding Word2Vec word embedding. Deep learning models were compared with a bag-of-words (BOW) model.
RESULTS: We retrieved 176,988 head CT reports for pre-training. We manually labeled 7784 reports as normal (46.3%) or pathological (53.7%), and 7.1% with intra-cranial hemorrhage. For the general labeling, LSTM-ATN-Word2Vec showed the best results (AUC = 0.967 ± 0.006, accuracy 90.8% ± 0.01). For the specific labeling, all methods showed similar accuracies between 95.0 and 95.9%. Both LSTM-ATN-Word2Vec and BOW had the highest AUC (0.970).
CONCLUSION: For a general use case, word embedding using a large cohort of non-English head CT reports and ATN improves NLP performance. For a more specific task, BOW and deep learning showed similar results. Models should be explored and tailored to the NLP task.
引用
- Barash Y, Guralnik G, Tau N, et al. Comparison of deep learning models for natural language processing-based classification of non-English head CT reports. Neuroradiology. 2020. PMID: 32335686. doi:10.1007/s00234-020-02420-0
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。