指南与共识功能神经外科

2021-2022年神经外科申请过程中标准化推荐信使用趋势

Trends in utilization of standardized letters of recommendation in the 2021-2022 neurosurgery application process.
指南 / 共识 · J Neurosurg · 2022-09-09

该横断面研究分析单一神经外科住院医师培训项目在2021-2022申请周期收到的所有标准化推荐信(SLOR),记录7个胜任力领域和总体评级并按推荐人学术类别分层。结果显示90%的SLOR将申请人评为前25%,但前1%和前2%-5%评级的使用显著减少;所有胜任力领域的评分者间信度均差。结论认为格式改变和使用模式使等级膨胀略有减轻,但评分者间信度仍差。

要点

  • 90%的SLOR将申请人评为前25%,但前1%和前2%-5%评级的使用显著减少。
  • 所有胜任力领域的评分者间信度均差。
  • 仅完成1封SLOR的推荐人比完成多封者评分更高且评分范围更窄。

核心结论

  • SLOR格式改变及后续使用模式使等级膨胀略有减轻,但评分者间信度仍差。
  • 评分范围最宽的评估者提交的SLOR数量最多,提示未来评估和使用SLOR应强调推荐人特征和提交SLOR数量。
  • 总体而言,SLOR在使用的第二年已被良好且广泛接受,并有细微改善。

研究设计与样本

  • 横断面研究,分析2021-2022申请周期提交至单一神经外科住院医师培训项目的所有SLOR。
  • 记录7个胜任力领域和总体评级的数据,并按推荐人学术类别分层。
  • 使用Krippendorff's alpha评估评分者间信度,使用Kruskal-Wallis H检验评估推荐信撰写频率。

主要发现

  • 90%的SLOR将申请人评为前25%,但前1%和前2%-5%评级的使用显著减少。
  • 所有胜任力领域的评分者间信度均差。
  • 仅完成1封SLOR的推荐人比完成多封SLOR的推荐人评分更高且评分范围更窄。

临床怎么用

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVE: Standardized letters of recommendation (SLORs) were introduced during the 2020-2021 neurosurgery residency application cycle, but suffered from poor interrater reliability and grade inflation. Changes were made to the SLOR template and utilization patterns in response to these shortcomings. The authors examined the second year of SLOR utilization. They hypothesized that grade inflation and interrater reliability would be improved from the first iteration. They also hypothesized that increased numbers of letters by single writers would correlate with broader rating distributions. METHODS: This cross-sectional study analyzed all SLORs submitted to a single neurosurgery residency program over the 2021-2022 cycle. Data from 7 competency domains and the overall rating were recorded and stratified by academic category of the letter writer. Interrater reliability was evaluated using Krippendorff's alpha. The frequency of letters written was evaluated using the Kruskal-Wallis H test. RESULTS: Ninety percent of SLORs rated applicants among the top 25%, but there was a significant decrease in the usage of the top 1% and top 2%-5% ratings. Interrater reliability was poor across all competencies. Writers who completed 1 SLOR rated applicants higher and had a narrower range than those who completed multiple SLORs. CONCLUSIONS: Changes in the format and subsequent utilization patterns of SLORs have slightly decreased grade inflation; however, interrater reliability remains poor. The most wide-ranging evaluators submitted the highest number of SLORs, suggesting that future evaluation and usage of SLORs should emphasize letter-writer characteristics and numbers of SLORs written. Overall, SLORs have been well and broadly accepted with subtle improvements in the second year of utilization.

来源:J Neurosurg,查看原文。版权归原作者所有。

引用

  1. Feldman MJ, Ortiz AV, Roth SG, et al. Trends in utilization of standardized letters of recommendation in the 2021-2022 neurosurgery application process. J Neurosurg. 2022. PMID: 36087327. doi:10.3171/2022.7.JNS221186

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。