文献速递儿科影像骨肌DR

基于手部X线片的深度学习儿童骨龄预测模型的泛化性与偏倚

Generalizability and Bias in a Deep Learning Pediatric Bone Age Prediction Model Using Hand Radiographs.
PubMed 文献 · Radiology · 2022-09-27

深度学习骨龄模型对外部测试集泛化良好,但存在性别、年龄和性成熟度相关的临床显著偏倚。

要点

  • 骨龄深度学习模型在外部测试集与内部验证集平均绝对差无显著差异(6.9 vs 6.8个月,P=0.64)。
  • 外部测试集中16%(194/1202)的图像模型预测会导致临床显著误差。
  • 内部验证集女孩平均绝对差大于男孩(P=0.01),外部测试集年龄和Tanner分期亚组间差异有统计学意义(P均<0.001)。

核心结论

  • 深度学习骨龄模型对外部测试集泛化良好,但存在性别、年龄和性成熟度相关的临床显著偏倚。

研究设计与样本

  • 回顾性评估2017年RSNA儿童骨龄挑战赛获胜深度学习模型,训练集为来自两家美国医院的12611例儿童手部X线片。

诊断标准与关键数值

  • 内部验证集1425例(773例男孩),外部测试集1202例(平均年龄133个月±60标准差,614例男孩),以平均绝对差(MAD)和临床显著误差率(改变临床诊断的误差率)为指标。

临床应用与阅片要点

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

Background Although deep learning (DL) models have demonstrated expert-level ability for pediatric bone age prediction, they have shown poor generalizability and bias in other use cases. Purpose To quantify generalizability and bias in a bone age DL model measured by performance on external versus internal test sets and performance differences between different demographic groups, respectively. Materials and Methods The winning DL model of the 2017 RSNA Pediatric Bone Age Challenge was retrospectively evaluated and trained on 12 611 pediatric hand radiographs from two U.S. hospitals. The DL model was tested from September 2021 to December 2021 on an internal validation set and an external test set of pediatric hand radiographs with diverse demographic representation. Images reporting ground-truth bone age were included for study. Mean absolute difference (MAD) between ground-truth bone age and the model prediction bone age was calculated for each set. Generalizability was evaluated by comparing MAD between internal and external evaluation sets with use of t tests. Bias was evaluated by comparing MAD and clinically significant error rate (rate of errors changing the clinical diagnosis) between demographic groups with use of t tests or analysis of variance and χ2 tests, respectively (statistically significant difference defined as P < .05). Results The internal validation set had images from 1425 individuals (773 boys), and the external test set had images from 1202 individuals (mean age, 133 months ± 60 [SD]; 614 boys). The bone age model generalized well to the external test set, with no difference in MAD (6.8 months in the validation set vs 6.9 months in the external set; P = .64). Model predictions would have led to clinically significant errors in 194 of 1202 images (16%) in the external test set. The MAD was greater for girls than boys in the internal validation set (P = .01) and in the subcategories of age and Tanner stage in the external test set (P < .001 for both). Conclusion A deep learning (DL) bone age model generalized well to an external test set, although clinically significant sex-, age-, and sexual maturity-based biases in DL bone age were identified. © RSNA, 2022 Online supplemental material is available for this article See also the editorial by Larson in this issue.

来源:Radiology,查看原文。版权归原作者所有。

引用

  1. Beheshtian, Putman, Santomartino, et al. Generalizability and Bias in a Deep Learning Pediatric Bone Age Prediction Model Using Hand Radiographs. Radiology. 2022. PMID: 36165796. doi:10.1148/radiol.220505

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。