文献速递消化内镜

结肠镜操作技能直接观察评估工具在培训中评估能力发展的有效性研究

Colonoscopy Direct Observation of Procedural Skills Assessment Tool for Evaluating Competency Development During Training.
PubMed 文献 · Am J Gastroenterol · 2020-01-01

本研究前瞻性分析英国18个月内279个培训单元提交的10,749份结肠镜DOPS评估,涉及1,199名受训者。结果显示DOPS信度阈值需3名评估者各完成2次评估,90%评估项目达标对整体能力基准的敏感度90.2%、特异度87.2%,并建立了各领域能力获取的里程碑。

要点

  • 3名评估者各完成2次DOPS即可达到可接受的信度阈值(G>0.70)。
  • 90%评估项目达标是判断整体DOPS能力的最佳阈值(敏感度90.2%,特异度87.2%)。
  • 主动问题解决和袢管理是最后发展的项目级能力,与整体评分相关性最强。

核心结论

  • 结肠镜DOPS评估工具具有支持其作为能力评估工具的有效性和信度证据。
  • 本研究建立了结肠镜培训中能力获取的里程碑。

研究设计与样本

  • 前瞻性全国性研究,分析英国培训电子档案中18个月内提交的结肠镜DOPS评估。
  • 共纳入279个培训单元、1,199名受训者的10,749份DOPS。
  • 采用概化分析评估内部结构效度和信度,受试者工作特征分析进行基准设定,并建立学习曲线和多变量分析。

主要发现

  • 可接受的信度阈值(G>0.70)需3名评估者各完成2次DOPS评估达到。
  • DOPS能力率与无辅助盲肠插管率相关(rho 0.404,P < 0.001)。
  • 90%评估项目达标对整体DOPS能力基准的敏感度为90.2%、特异度为87.2%。
  • 各领域能力达标顺序依次为:操作前(50-99例)、内镜非技术技能和操作后(150-199例)、管理(200-249例)、操作(250-299例)。
  • 项目层面「主动问题解决」(rho 0.787)和「袢管理」(rho 0.780)与整体DOPS评分相关性最强(P < 0.001),且发展最晚。

关键数值与推荐

  • 信度阈值:G>0.70,需3名评估者各完成2次DOPS。
  • 整体DOPS能力基准:90%评估项目达标,敏感度90.2%,特异度87.2%。
  • 操作前领域能力达标对应50-99例,内镜非技术技能和操作后对应150-199例,管理对应200-249例,操作对应250-299例。

临床怎么用

  • 可采用3名评估者各2次DOPS的评估方案以达到可接受信度。
  • 以90%评估项目达标作为判断整体DOPS能力的基准阈值。
  • 参考各领域能力达标对应的操作例数里程碑,分阶段评估受训者能力发展。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

INTRODUCTION: Formative colonoscopy direct observation of procedural skills (DOPS) assessments were updated in 2016 and incorporated into UK training but lack validity evidence. We aimed to appraise the validity of DOPS assessments, benchmark performance, and evaluate competency development during training in diagnostic colonoscopy. METHODS: This prospective national study identified colonoscopy DOPS submitted over an 18-month period to the UK training e-portfolio. Generalizability analyses were conducted to evaluate internal structure validity and reliability. Benchmarking was performed using receiver operator characteristic analyses. Learning curves for DOPS items and domains were studied, and multivariable analyses were performed to identify predictors of DOPS competency. RESULTS: Across 279 training units, 10,749 DOPS submitted for 1,199 trainees were analyzed. The acceptable reliability threshold (G > 0.70) was achieved with 3 assessors performing 2 DOPS each. DOPS competency rates correlated with the unassisted caecal intubation rate (rho 0.404, P < 0.001). Demonstrating competency in 90% of assessed items provided optimal sensitivity (90.2%) and specificity (87.2%) for benchmarking overall DOPS competence. This threshold was attained in the following order: "preprocedure" (50-99 procedures), "endoscopic nontechnical skills" and "postprocedure" (150-199), "management" (200-249), and "procedure" (250-299) domain. At item level, competency in "proactive problem solving" (rho 0.787) and "loop management" (rho 0.780) correlated strongest with the overall DOPS rating (P < 0.001) and was the last to develop. Lifetime procedure count, DOPS count, trainer specialty, easier case difficulty, and higher cecal intubation rate were significant multivariable predictors of DOPS competence. DISCUSSION: This study establishes milestones for competency acquisition during colonoscopy training and provides novel validity and reliability evidence to support colonoscopy DOPS as a competency assessment tool.

来源:Am J Gastroenterol,查看原文。版权归原作者所有。

引用

  1. Siau K, Crossley J, Dunckley P, et al. Colonoscopy Direct Observation of Procedural Skills Assessment Tool for Evaluating Competency Development During Training. Am J Gastroenterol. 2020. PMID: 31738285. doi:10.14309/ajg.0000000000000426

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。