指南与共识消化、代谢与肾脏急症

测量与预测标准化评估信的教师共识排名

Measuring and Predicting Faculty Consensus Rankings of Standardized Letters of Evaluation.
指南 / 共识 · J Grad Med Educ · 2024-02-17
一句话结论教师对标准化评估信(SLOE)判断申请者竞争力的共识度高(宽松一致93%),两种预测模型与共识排名高度相关(r=0.97)。

该研究以2021-2022年Match周期急诊医学住院医师主任委员会(CORD)SLOE数据库为蓝本,创建50份虚构SLOE,由7名教师按申请者竞争力(建议排名位置)排序。采用预先设定的界值评估共识,并测试基于点的系统和线性回归模型预测共识排名的能力。结果显示教师对SLOE解读共识强,「接近」和「宽松」一致分别为83%和93%。两种预测模型与共识排名高度相关(基于点的系统r=0.97,线性回归r=0.97)。结论认为教师对SLOE竞争力共识强,支持SLOE用于筛选和指导,两种模型可高精度预测共识排名。

要点

  • 教师对SLOE判断申请者竞争力的共识强,「接近」一致83%,「宽松」一致93%。
  • 基于点的系统与共识排名相关r=0.97。
  • 线性回归模型与共识排名相关r=0.97。
  • 研究为SLOE用于住院医师筛选和指导提供了进一步支持。
  • 两种模型能高精度预测共识竞争力排名。

适用人群与场景

  • 使用2021-2022年Match周期急诊医学住院医师主任委员会(CORD)SLOE数据库数据为蓝本,创建50份虚构SLOE。
  • 由7名教师对这些SLOE按申请者竞争力(建议排名位置)进行排序评估。

诊断标准与关键阈值

  • 共识评估采用预先设定的界值,定义「接近」一致和「宽松」一致。
  • 申请者竞争力定义为建议的排名位置。

研究设计与关键结果

  • 研究类型为基于虚构SLOE的教师评分研究,样本为50份虚构SLOE,由7名教师评估。
  • 教师对SLOE解读共识强,「接近」一致83%,「宽松」一致93%。
  • 基于点的系统预测共识排名r=0.97。
  • 线性回归模型预测共识排名r=0.97。

证据类型、等级与局限

  • 证据类型:基于虚构SLOE的教师评分研究。
  • 局限:原文未报告局限性。

对床旁实践的改变

  • 作者认为,教师对SLOE的竞争力共识强,进一步支持SLOE用于筛选和指导。
  • 两种模型能以高精度预测共识竞争力排名。

提炼说明

  • 本条依据论文摘要整理;摘要未涉及「处置流程与药物剂量」「红旗征与不能漏的情况」「指南或共识出处」,故未列出这些栏目;全文请点「阅读原文」。
英文原文(摘要)· 点击展开

Background Standardized letters of evaluation (SLOE) are becoming more widely incorporated into the residency application process to make the letter of recommendation, an already critical component in a residency application packet, more objective. However, it is not currently known if the reviewers of these letters share consensus regarding the strength of an applicant determined by their SLOE. Objective We measured the level of faculty agreement regarding applicant competitiveness as determined by SLOEs and the ability of 2 algorithms to predict faculty consensus rankings. Methods Using data from the 2021-2022 Match cycle from the Council of Residency Directors in Emergency Medicine SLOE Database as a blueprint, authors created 50 fictional SLOEs representative of the national data. Seven faculty then rated these SLOEs in order of applicant competitiveness, defined as suggested rank position. Consensus was evaluated using cutoffs established a priori, and 2 prediction models, a point-based system and a linear regression model, were tested to determine their ability to predict consensus rankings. Results There was strong faculty consensus regarding the interpretation of SLOEs. Within narrow windows of agreement, faculty demonstrated similar ranking patterns with 83% and 93% agreement for "close" and "loose" agreement, respectively. Predictive models yielded a strong correlation with the consensus ranking (point-based system r=0.97, linear regression r=0.97). Conclusions Faculty displayed strong consensus regarding the competitiveness of applicants via SLOEs, adding further support to the use of SLOEs for selection and advising. Two models predicted consensus competitiveness rankings with a high degree of accuracy.

来源:J Grad Med Educ,查看原文。版权归原作者所有。

参考来源

  1. Sehdev M, Schnapp B, Dubosh NM, et al. Measuring and Predicting Faculty Consensus Rankings of Standardized Letters of Evaluation. J Grad Med Educ. 2024. PMID: 38304605. doi:10.4300/JGME-D-22-00901.1

供执业医师抢救与诊疗参考,不替代临床判断;剂量请结合体重、肝肾功能和本院药品规格复核,以指南原文为准。