胚胎学家评估囊胚种植概率的一致性:数据驱动预测能否解决胚胎评估的主观性问题?
本研究回顾性收集2018至2019年两个临床中心的136枚已知种植结局的囊胚时间-lapse视频,由来自13个国家的36名胚胎学家盲法评估种植概率(IPG 1–5级),并与深度学习神经网络(DNN)比较。胚胎学家平均种植预测准确率为51.9%,总体观察者间一致性为中等(κ=0.56),DNN总体准确率为62.5%,AUC为0.70,高于胚胎学家的0.61。
要点
- 36名胚胎学家对136枚囊胚的平均种植预测准确率为51.9%,总体观察者间一致性为中等(κ=0.56)。
- DNN总体准确率为62.5%,AUC为0.70,高于胚胎学家的AUC 0.61。
- 优质和劣质胚胎的胚胎学家平均准确率分别为57.5%和57.4%,一般质量胚胎为44.6%。
核心结论
- 大型胚胎学家面板的总体观察者间一致性为中等,预测准确率有限,而专用人工智能模型通常表现出更高的性能指标。
研究设计与样本
- 回顾性收集2018至2019年两个临床中心的136枚已知种植结局的囊胚时间-lapse视频,用于36名胚胎学家与深度神经网络(DNN)的性能评估和比较。
- 从13个不同国家招募39名胚胎学家,所有参与者对临床结局盲法;36名胚胎学家提供1至5级种植概率评分(IPG),3名经验超过5年的胚胎学家提供Gardner评分。
- 采用Fleiss kappa系数评估胚胎学家性能和一致性;开发10折交叉验证DNN为时间-lapse视频文件提供IPG,并与胚胎学家比较。
主要发现
- 胚胎学家对所有胚胎(N=136)的平均种植预测准确率为51.9%。
- 胚胎学家评估优质和劣质胚胎(按Gardner评分分类)的平均准确率分别为57.5%和57.4%,一般质量胚胎为44.6%。
- 总体观察者间一致性为中等(κ=0.56,N=136);劣质+优质组的最高一致性为κ=0.65(N=77),一般质量组较低为κ=0.25(N=59)。
- DNN总体准确率为62.5%,劣质、一般和优质组的准确率分别为62.2%、61%和65.6%。
- DNN的AUC总体高于胚胎学家(DNN 0.70 vs 胚胎学家0.61),在各Gardner组中亦然(DNN vs 胚胎学家—劣质:0.69 vs 0.62;一般:0.67 vs 0.53;优质:0.77 vs 0.54)。
关键数值与推荐
- 种植概率评分(IPG)采用1至5级,1表示种植可能性极低,5表示种植可能性极高;胚胎学家预测转换为种植(IPG≥3)与未种植(IPG≤2)。
- 逻辑回归用于以下混杂变量:居住国、学历水平、胚胎评分系统、经验年数对数和使用时间-lapse的经验;在α=0.05水平均未发现对胚胎学家性能有统计学显著影响。
临床怎么用
- 作者认为,使用数据驱动算法作为辅助工具可能帮助IVF专业人员提高成功率并促进IVF诊所亟需的标准化。
- 作者认为,研究结果表明需要对该领域的技术进步开展进一步研究。
局限性
- 囊胚评估使用从时间-lapse培养箱获取的视频文件,每个视频仅包含单一焦平面的数据。
- 关于不孕根本原因和移植前子宫内膜厚度的临床数据不可用,但可能解释种植失败和IPG准确率较低。
- 种植定义为存在妊娠囊,而检测胎心是胚胎活力的更可靠标志。
- 原始数据匿名化到无法量化研究中纳入的独特患者和周期数量,可能掩盖有限患者池偏倚的影响。
- 缺乏人口统计学数据,难以就数据集对更广泛人群的代表性得出结论。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
STUDY QUESTION: What is the accuracy and agreement of embryologists when assessing the implantation probability of blastocysts using time-lapse imaging (TLI), and can it be improved with a data-driven algorithm? SUMMARY ANSWER: The overall interobserver agreement of a large panel of embryologists was moderate and prediction accuracy was modest, while the purpose-built artificial intelligence model generally resulted in higher performance metrics. WHAT IS KNOWN ALREADY: Previous studies have demonstrated significant interobserver variability amongst embryologists when assessing embryo quality. However, data concerning embryologists' ability to predict implantation probability using TLI is still lacking. Emerging technologies based on data-driven tools have shown great promise for improving embryo selection and predicting clinical outcomes. STUDY DESIGN, SIZE, DURATION: TLI video files of 136 embryos with known implantation data were retrospectively collected from two clinical sites between 2018 and 2019 for the performance assessment of 36 embryologists and comparison with a deep neural network (DNN). PARTICIPANTS/MATERIALS, SETTING, METHODS: We recruited 39 embryologists from 13 different countries. All participants were blinded to clinical outcomes. A total of 136 TLI videos of embryos that reached the blastocyst stage were used for this experiment. Each embryo's likelihood of successfully implanting was assessed by 36 embryologists, providing implantation probability grades (IPGs) from 1 to 5, where 1 indicates a very low likelihood of implantation and 5 indicates a very high likelihood. Subsequently, three embryologists with over 5 years of experience provided Gardner scores. All 136 blastocysts were categorized into three quality groups based on their Gardner scores. Embryologist predictions were then converted into predictions of implantation (IPG ≥ 3) and no implantation (IPG ≤ 2). Embryologists' performance and agreement were assessed using Fleiss kappa coefficient. A 10-fold cross-validation DNN was developed to provide IPGs for TLI video files. The model's performance was compared to that of the embryologists. MAIN RESULTS AND THE ROLE OF CHANCE: Logistic regression was employed for the following confounding variables: country of residence, academic level, embryo scoring system, log years of experience and experience using TLI. None were found to have a statistically significant impact on embryologist performance at α = 0.05. The average implantation prediction accuracy for the embryologists was 51.9% for all embryos (N = 136). The average accuracy of the embryologists when assessing top quality and poor quality embryos (according to the Gardner score categorizations) was 57.5% and 57.4%, respectively, and 44.6% for fair quality embryos. Overall interobserver agreement was moderate (κ = 0.56, N = 136). The best agreement was achieved in the poor + top quality group (κ = 0.65, N = 77), while the agreement in the fair quality group was lower (κ = 0.25, N = 59). The DNN showed an overall accuracy rate of 62.5%, with accuracies of 62.2%, 61% and 65.6% for the poor, fair and top quality groups, respectively. The AUC for the DNN was higher than that of the embryologists overall (0.70 DNN vs 0.61 embryologists) as well as in all of the Gardner groups (DNN vs embryologists-Poor: 0.69 vs 0.62; Fair: 0.67 vs 0.53; Top: 0.77 vs 0.54). LIMITATIONS, REASONS FOR CAUTION: Blastocyst assessment was performed using video files acquired from time-lapse incubators, where each video contained data from a single focal plane. Clinical data regarding the underlying cause of infertility and endometrial thickness before the transfer was not available, yet may explain implantation failure and lower accuracy of IPGs. Implantation was defined as the presence of a gestational sac, whereas the detection of fetal heartbeat is a more robust marker of embryo viability. The raw data were anonymized to the extent that it was not possible to quantify the number of unique patients and cycles included in the study, potentially masking the effect of bias from a limited patient pool. Furthermore, the lack of demographic data makes it difficult to draw conclusions on how representative the dataset was of the wider population. Finally, embryologists were required to assess the implantation potential, not embryo quality. Although this is not the traditional approach to embryo evaluation, morphology/morphokinetics as a means of assessing embryo quality is believed to be strongly correlated with viability and, for some methods, implantation potential. WIDER IMPLICATIONS OF THE FINDINGS: Embryo selection is a key element in IVF success and continues to be a challenge. Improving the predictive ability could assist in optimizing implantation success rates and other clinical outcomes and could minimize the financial and emotional burden on the patient. This study demonstrates moderate agreement rates between embryologists, likely due to the subjective nature of embryo assessment. In particular, we found that average embryologist accuracy and agreement were significantly lower for fair quality embryos when compared with that for top and poor quality embryos. Using data-driven algorithms as an assistive tool may help IVF professionals increase success rates and promote much needed standardization in the IVF clinic. Our results indicate a need for further research regarding technological advancement in this field. STUDY FUNDING/COMPETING INTEREST(S): Embryonics Ltd is an Israel-based company. Funding for the study was partially provided by the Israeli Innovation Authority, grant #74556. TRIAL REGISTRATION NUMBER: N/A.
引用
- Fordham DE, Rosentraub D, Polsky AL, et al. Embryologist agreement when assessing blastocyst implantation probability: is data-driven prediction the solution to embryo assessment subjectivity? Hum Reprod. 2022. PMID: 35944167. doi:10.1093/humrep/deac171
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。