文献速递中毒与环境急症

使用自然语言处理构建和验证用于分类过量死亡相关物质的机器学习模型

Development and Validation of Machine Models Using Natural Language Processing to Classify Substances Involved in Overdose Deaths.
PubMed 文献 · JAMA Netw Open · 2022-08-01
一句话结论NLP/ML算法在分类阿片类、芬太尼、海洛因等过量死亡相关物质上表现优异,但苯二氮䓬类分类欠佳,作者建议将其整合入工作流程以减少监测报告延迟。

该诊断研究比较了不同自然语言处理与机器学习算法,用于从法医和验尸官死亡记录中识别与过量死亡相关的物质,数据来自美国10个卫生辖区2020年1月1日至12月31日的35,433份记录。研究比较了三种特征表示方法(TF-IDF、GloVe、CUI嵌入),训练多种ML算法并以F分数选择最佳模型,在留出测试集上报告结果及95%CI。结果显示,CUI嵌入在除酒精外的所有物质上诊断指标相似或更优;ML分类器对任何阿片类、海洛因、芬太尼、处方阿片类、甲基苯丙胺、可卡因和酒精的分类达到完美或接近完美,而苯二氮䓬类在所有三种特征提取方法下分类均欠佳。作者建议将这些算法整合入工作流程以减少过量监测数据的报告延迟。

要点

  • 在35,433份死亡记录中,与死亡相关的最常见物质包括任何阿片类5739例(16%)、芬太尼4758例(13%)、酒精2866例(8%)。
  • CUI嵌入在除酒精外的所有物质上,诊断指标与词嵌入和TF-IDF相似或更优。
  • ML分类器对任何阿片类、海洛因、芬太尼、处方阿片类、甲基苯丙胺、可卡因和酒精的分类达到完美或接近完美。
  • 苯二氮䓬类在所有三种特征提取方法下分类均欠佳。
  • 作者建议将这些算法整合入工作流程,以减少过量监测数据的报告延迟。

适用人群与场景

  • 美国10个卫生辖区2020年1月1日至12月31日的法医和验尸官死亡记录,共35,433份;死者中位年龄58岁(IQR,41-72岁),男性24,449例(69%)。

研究设计与关键结果

  • 诊断研究,比较不同自然语言处理与机器学习算法,使用35,433份法医和验尸官死亡记录的非结构化文本,来自美国10个卫生辖区,时间范围为2020年1月1日至2020年12月31日。
  • 最常见物质:任何阿片类5739例(16%)、芬太尼4758例(13%)、酒精2866例(8%)、可卡因2247例(6%)、甲基苯丙胺1876例(5%)、海洛因1613例(5%)、处方阿片类1197例(3%)、任何苯二氮䓬类1076例(3%)。
  • CUI嵌入在除酒精外的所有物质上,诊断指标与词嵌入和TF-IDF相似或更优。
  • ML分类器对任何阿片类、海洛因、芬太尼、处方阿片类、甲基苯丙胺、可卡因和酒精的分类达到完美或接近完美。
  • 苯二氮䓬类在所有三种特征提取方法下分类均欠佳。

证据类型、等级与局限

  • 证据类型:诊断研究;局限:原文未报告局限性。

对床旁实践的改变

  • 作者认为,这些算法应整合入工作流程,以减少过量监测数据的报告延迟。

提炼说明

  • 本条依据论文摘要整理;摘要未涉及「诊断标准与关键阈值」「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开

IMPORTANCE: Overdose is one of the leading causes of death in the US; however, surveillance data lag considerably from medical examiner determination of the death to reporting in national surveillance reports.

OBJECTIVE: To automate the classification of deaths related to substances in medical examiner data using natural language processing (NLP) and machine learning (ML).

DESIGN, SETTING, AND PARTICIPANTS: Diagnostic study comparing different natural language processing and machine learning algorithms to identify substances related to overdose in 10 health jurisdictions in the US from January 1, 2020, to December 31, 2020. Unstructured text from 35 433 medical examiner and coroners' death records was examined.

EXPOSURES: Text from each case was manually classified to a substance that was related to the death. Three feature representation methods were used and compared: text frequency-inverse document frequency (TF-IDF), global vectors for word representations (GloVe), and concept unique identifier (CUI) embeddings. Several ML algorithms were trained and best models were selected based on F-scores. The best models were tested on a hold-out test set and results were reported with 95% CIs.

MAIN OUTCOMES AND MEASURES: Text data from death certificates were classified as any opioid, fentanyl, alcohol, cocaine, methamphetamine, heroin, prescription opioid, and an aggregate of other substances. Diagnostic metrics and 95% CIs were calculated for each combination of feature extraction method and machine learning classifier.

RESULTS: Of 35 433 death records analyzed (decedent median age, 58 years [IQR, 41-72 years]; 24 449 [69%] were male), the most common substances related to deaths included any opioid (5739 [16%]), fentanyl (4758 [13%]), alcohol (2866 [8%]), cocaine (2247 [6%]), methamphetamine (1876 [5%]), heroin (1613 [5%]), prescription opioids (1197 [3%]), and any benzodiazepine (1076 [3%]). The CUI embeddings had similar or better diagnostic metrics compared with word embeddings and TF-IDF for all substances except alcohol. ML classifiers had perfect or near perfect performance in classifying deaths related to any opioids, heroin, fentanyl, prescription opioids, methamphetamine, cocaine, and alcohol. Classification of benzodiazepines was suboptimal using all 3 feature extraction methods.

CONCLUSIONS AND RELEVANCE: In this diagnostic study, NLP/ML algorithms demonstrated excellent diagnostic performance at classifying substances related to overdoses. These algorithms should be integrated into workflows to decrease the lag time in reporting overdose surveillance data.

来源:JAMA Netw Open,查看原文。版权归原作者所有。

参考来源

  1. Goodman-Meza D, Shover CL, Medina JA, et al. Development and Validation of Machine Models Using Natural Language Processing to Classify Substances Involved in Overdose Deaths. JAMA Netw Open. 2022. PMID: 35939303. doi:10.1001/jamanetworkopen.2022.25593

供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。