文献速递技术与安全

通过报告比较与Jaccard相似度量化放射科住院医师报告书写表现的学习曲线

Quantifying Radiology Residents' Learning Curves in Report Writing Performance Through Report Comparison and Jaccard Similarity.
PubMed 文献 · Radiology · 2024-01-01

该回顾性研究纳入2017年9月至2022年11月某三级放射科246984份报告,用Jaccard相似系数比较住院医师草稿或初步报告与最终报告,发现初步报告相似度从第1年0.86升至第5年0.92(提高6%),草稿报告从0.68升至0.82(提高14%),提示报告书写表现随经验增加。

要点

  • Jaccard相似度可量化住院医师报告书写学习曲线,初步报告第1年至第5年提高6%,草稿报告提高14%。
  • 住院医师经验年数与报告相似度呈强相关(按年聚合rs=0.99,按季度聚合rs=0.90)。
  • 各影像亚专科和检查方式均显示相似趋势,如骨肌影像从0.77升至0.91。

核心结论

  • 住院医师报告书写表现随经验增加而提高,Jaccard指数可量化该趋势,初步报告从第1年至第5年提高6%,表明其可作为评估培训进展和指导住院医师教育的工具。

研究设计与样本

  • 回顾性研究,纳入2017年9月至2022年11月某三级放射科246984份报告,53名住院医师(平均报告数4660±3546,经验1-5年),比较住院医师草稿或初步报告与带教医师审核后最终报告的Jaccard相似系数(范围0-1),采用Welch方差分析及事后检验和Spearman相关分析。

诊断标准与关键数值

  • Jaccard相似系数为定量表达编辑量的指标,范围0-1;初步报告平均Jaccard相似度第1年0.86±0.22,第5年0.92±0.15,提高6%(P < .001);草稿报告第1年0.68±0.27,第5年0.82±0.23,提高14%(P < .001);按年聚合Spearman相关系数rs=0.99(95%CI:0.85,1.00;P < .001),按季度聚合rs=0.90(95%CI:0.73,0.96;P < .001);骨肌影像从0.77±0.31升至0.91±0.16(P < .001),rs=0.98(95%CI:0.72,1.00;P < .001)。

临床应用与阅片要点

  • 作者认为Jaccard指数可作为评估培训进展和指导住院医师教育的工具;原文未据此给出具体阅片或报告操作推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

Background Report writing skills are a core competency to be acquired during residency, yet objective tools for tracking performance are lacking. Purpose To investigate whether the Jaccard index, derived from report comparison, can objectively illustrate learning curves in report writing performance throughout radiology residency. Materials and Methods Retrospective data from 246 984 radiology reports written from September 2017 to November 2022 in a tertiary care radiology department were included. Reports were scored using the Jaccard similarity coefficient (ie, a quantitative expression of the amount of edits performed; range, 0-1) of residents' draft (unsupervised initial attempt at a complete report) or preliminary reports (following joint readout with attending physicians) and faculty-reviewed final reports. Weighted mean Jaccard similarity was compared between years of experience using Welch analysis of variance with post hoc testing overall, per imaging division, and per modality. Relationships with years and quarters of resident experience were assessed using Spearman correlation. Results This study included 53 residents (mean report count, 4660 ± 3546; 1-5 years of experience). Mean Jaccard similarity of preliminary reports increased by 6% from 1st-year to 5th-year residents (0.86 ± 0.22 to 0.92 ± 0.15; P < .001). Spearman correlation demonstrated a strong relationship between residents' experience and higher report similarity when aggregated for years (rs = 0.99 [95% CI: 0.85, 1.00]; P < .001) or quarters of experience (rs = 0.90 [95% CI: 0.73, 0.96]; P < .001). For residents' draft reports, Jaccard similarity increased by 14% over the course of the 5-year residency program (0.68 ± 0.27 to 0.82 ± 0.23; P < .001). Subgroup analysis confirmed similar trends for all imaging divisions and modalities (eg, in musculoskeletal imaging, from 0.77 ± 0.31 to 0.91 ± 0.16 [P < .001]; rs = 0.98 [95% CI: 0.72, 1.00] [P < .001]). Conclusion Residents' report writing performance increases with experience. Trends can be quantified with the Jaccard index, with a 6% improvement from 1st- to 5th-year residents, indicating its effectiveness as a tool for evaluating training progress and guiding education over the course of residency. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Bruno in this issue.

来源:Radiology,查看原文。版权归原作者所有。

引用

  1. Vosshenrich J, Guntli C, Cyriac J, et al. Quantifying Radiology Residents' Learning Curves in Report Writing Performance Through Report Comparison and Jaccard Similarity. Radiology. 2024. PMID: 39315901. doi:10.1148/radiol.233065

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。