指南与共识咽喉与嗓音

机器学习用于耳鼻咽喉科住院医师申请推荐信的预测分析

Machine Learning for Predictive Analysis of Otolaryngology Residency Letters of Recommendation.
指南 / 共识 · Laryngoscope · 2024-04-11

本研究回顾性分析2022-2023申请季来自单一机构的1642封推荐信,使用自然语言处理和五种机器学习模型预测耳鼻咽喉科住院医师申请者是否获得面试邀请。337名申请者中67人获得面试、270人未获得。表现最佳的模型为TF-IDF向量化决策树和CountVectorizer向量化决策树,预测效果优于随机。

要点

  • 推荐信文本经三种向量化技术处理后,五种机器学习模型可用于预测面试邀请。
  • 表现最佳的模型为TF-IDF向量化决策树和CountVectorizer向量化决策树。
  • 机器学习模型对面试邀请的预测优于随机,可辅助培训项目预测申请者获面试可能性。

核心结论

  • 机器学习模型与向量化组合可对耳鼻咽喉科住院医师申请者的面试邀请提供优于随机的预测。
  • 表现最佳的模型能从推荐信中分类出有意义的信息以预测申请者面试邀请。

研究设计与样本

  • 回顾性研究,从单一机构获取2022-2023申请季的1642封推荐信。
  • 推荐信经预处理后使用CountVectorizer、TF-IDF和Word2Vec三种技术向量化。
  • 在逻辑回归、朴素贝叶斯、决策树、随机森林和支持向量机五种模型上训练和测试。

主要发现

  • 337名申请者中67人获得面试邀请,270人未获得。
  • 共分析1642封推荐信,其中26.7%对应获得面试邀请的申请者。
  • 预测面试邀请表现最佳的两个模型为TF-IDF向量化决策树和CountVectorizer向量化决策树。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

INTRODUCTION: Letters of recommendation (LORs) are a highly influential yet subjective and often enigmatic aspect of the residency application process. This study hypothesizes that LORs do contain valuable insights into applicants and can be used to predict outcomes. This pilot study utilizes natural language processing and machine learning (ML) models using LOR text to predict interview invitations for otolaryngology residency applicants. METHODS: A total of 1642 LORs from the 2022-2023 application cycle were retrospectively retrieved from a single institution. LORs were preprocessed and vectorized using three different techniques to represent the text in a way that an ML model can understand written prose: CountVectorizer (CV), Term Frequency-Inverse Document Frequency (TF-IDF), and Word2Vec (WV). Then, the LORs were trained and tested on five ML models: Logistic Regression (LR), Naive Bayes (NB), Decision Tree (DT), Random Forest (RF), and Support Vector Machine (SVM). RESULTS: Of the 337 applicants, 67 were interviewed and 270 were not interviewed. In total, 1642 LORs (26.7% interviewed) were analyzed. The two best-performing ML models in predicting interview invitations were the TF-IDF vectorized DT and CV vectorized DT models. CONCLUSION: This preliminary study revealed that ML models and vectorization combinations can provide better-than-chance predictions for interview invitations for otolaryngology residency applicants. The high-performing ML models were able to classify meaningful information from the LORs to predict applicant interview invitation. The potential of an automated process to help predict an applicant's likelihood of obtaining an interview invitation could be a valuable tool for training programs in the future. LEVEL OF EVIDENCE: N/A Laryngoscope, 134:4016-4022, 2024.

来源:Laryngoscope,查看原文。版权归原作者所有。

引用

  1. Vasan V, Cheng CP, Lerner DK, et al. Machine Learning for Predictive Analysis of Otolaryngology Residency Letters of Recommendation. Laryngoscope. 2024. PMID: 38602257. doi:10.1002/lary.31439

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。