手持超声设备中自动B线识别与定量算法与专家超声医师判读的观察者间一致性与相关性
本研究前瞻性收集急诊科呼吸困难成年患者的超声图像,前124例用于算法开发,另随机选取80例独立受试者的图像由5位专家盲法分别进行手动计数和复核调整计数。算法与手动计数的ICC为0.84(95%CI 0.75-0.90),复核调整计数与算法计数的ICC为0.94(95%CI 0.90-0.96),手动与复核调整计数的ICC为0.94(95%CI 0.90-0.96)。作者结论认为手持超声设备的自动算法与专家在B线识别和定量上高度相关。
要点
- 算法与专家手动B线计数的ICC为0.84(95%CI 0.75-0.90),达到预设ICC 0.75的合格阈值。
- 专家复核调整计数与算法计数的ICC为0.94(95%CI 0.90-0.96),相关性高于手动计数。
- 加权kappa在算法与手动计数间为0.72(95%CI 0.49-0.95),在复核调整与算法间为0.88(95%CI 0.74-1)。
核心结论
- 手持超声设备的自动B线识别与定量算法与专家判读高度相关,算法与手动计数ICC为0.84(95%CI 0.75-0.90),复核调整计数与算法ICC为0.94(95%CI 0.90-0.96)。
研究设计与样本
- 前瞻性收集急诊科呼吸困难成年患者的超声图像,前124例用于算法开发,另随机选取80例独立受试者的图像用于测试,按预设B线比例(0条、1-2条、3条及以上)分层,由5位专家盲法分别进行手动计数和复核调整计数。
诊断标准与关键数值
- 预设合格性能阈值为ICC(3,k)≥0.75且精度≤0.3;算法与手动计数ICC 0.84(95%CI 0.75-0.90)、精度0.15;复核调整计数与算法ICC 0.94(95%CI 0.90-0.96);手动与复核调整计数ICC 0.94(95%CI 0.90-0.96);加权kappa分别为0.72(95%CI 0.49-0.95)、0.88(95%CI 0.74-1)、0.85(95%CI 0.89-0.96)。
临床应用与阅片要点
- 作者结论认为手持超声设备的自动算法与专家在B线识别和定量上高度相关。
- 作者提出未来研究可将该手持超声设备纳入不同场景和不同经验水平使用者的临床研究中。
局限性
- 原文结论提出未来研究可能将该手持超声设备纳入多场景和不同经验水平使用者的临床研究,提示现有证据限于当前测试条件。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVES: B-lines are ultrasound artifacts that can be used to detect a variety of pathologic lung conditions. Computer-aided methods to detect and quantify B-lines may standardize quantification and improve diagnosis by novice users. We sought to test the performance of an automated algorithm for the detection and quantification of B-lines in a handheld ultrasound device (HHUD).
METHODS: Ultrasound images were prospectively collected on adult emergency department patients with dyspnea. Images from the first 124 patients were used for algorithm development. Clips from 80 unique subjects for testing were randomly selected in a predefined proportion of B-lines (0 B-lines, 1-2 B-lines, 3 or more B-lines) and blindly reviewed by five experts using both a manual and reviewer-adjusted process. Intraclass correlation coefficient (ICC) and weighted kappa were used to measure agreement, while an a priori threshold of an ICC (3,k) of 0.75 and precision of 0.3 were used to define adequate performance.
RESULTS: ICC between the algorithm and manual count was 0.84 (95% confidence interval [CI] 0.75-0.90), with a precision of 0.15. ICC between the reviewer-adjusted count and the algorithm count was 0.94 (95% CI 0.90-0.96), and the ICC between the manual and reviewer-adjusted counts was 0.94 (95% CI 0.90-0.96). Weighted kappa was 0.72 (95% CI 0.49-0.95), 0.88 (95% CI 0.74-1), and 0.85 (95% CI 0.89-0.96), respectively.
CONCLUSIONS: This study demonstrates a high correlation between point-of-care ultrasound experts and an automated algorithm to identify and quantify B-lines using an HHUD. Future research may incorporate this HHUD in clinical studies in multiple settings and users of varying experience levels.
引用
- Moore, Wang, Battisti, et al. Interobserver Agreement and Correlation of an Automated Algorithm for B-Line Identification and Quantification With Expert Sonologist Review in a Handheld Ultrasound Device. J Ultrasound Med. 2021. PMID: 34964489. doi:10.1002/jum.15935
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。