基于智能手机的机器学习辅助系统在甲状腺相关眼病数字图像中评估临床活动度评分的诊断性能初步评价
本研究将先前基于数码单反相机图像的机器学习辅助系统应用于六款智能手机拍摄的甲状腺相关眼病患者面部图像,以检测活动性甲状腺相关眼病(临床活动度评分≥3)。在100例患者中,该系统在排名前三的智能手机上敏感度为74.5%、特异度为84.8%、F1分数为0.70,与眼整形专科医师表现相当,优于住院医师和普通眼科医师,但作者指出需要验证性研究才能用于临床。
要点
- 智能手机型号影响活动性甲状腺相关眼病检测性能,F1分数为0.59至0.72。
- 排名前三的智能手机上,系统敏感度74.5%、特异度84.8%、F1分数0.70。
- 全部六款智能手机上,系统平均敏感度71.4%、特异度81.6%、F1分数0.66。
核心结论
- 基于智能手机的机器学习辅助系统在检测活动性甲状腺相关眼病方面显示出合理的准确性,与眼整形专科医师表现相当,优于住院医师和普通眼科医师。
- 该系统可能实现可靠的疾病活动度自我监测,但临床应用需要验证性研究。
研究设计与样本
- 将先前基于数码单反相机图像的机器学习架构应用于六款智能手机(Galaxy S21 Ultra、iPhone 12 pro、iPhone 11、iPhone SE 2020、Galaxy M20和Galaxy A21S)拍摄的面部图像。
- 使用100例甲状腺相关眼病患者经智能手机拍摄的图像评估性能。
- 由三名眼科住院医师、三名临床经验少于5年的普通眼科医师和三名眼整形专科医师独立解读同一组在摄影棚环境下拍摄的图像,并与智能手机机器学习辅助系统的结果进行比较。
- 参考临床活动度评分由三名眼整形专科医师共识确定。
主要发现
- 在100例患者中,28例被确定为活动性甲状腺相关眼病(临床活动度评分≥3)。
- 拍摄面部图像所用的智能手机型号影响活动性甲状腺相关眼病检测性能,F1分数为0.59至0.72。
- 在排名前三的智能手机上,基于智能手机的系统敏感度为74.5%、特异度为84.8%、F1分数为0.70。
- 在全部六款智能手机的图像上,平均敏感度、特异度和F1分数分别为71.4%、81.6%和0.66。
- 眼科住院医师的敏感度、特异度和F1分数分别为69.1%、55.1%和0.46;普通眼科医师分别为61.9%、79.6%和0.55;眼整形专科医师分别为73.8%、90.7%和0.75。
临床怎么用
- 原文未据此给出推荐。
局限性
- 作者指出该系统可能实现可靠的疾病活动度自我监测,但临床应用需要验证性研究。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Background: We previously developed a machine learning (ML)-assisted system for predicting the clinical activity score (CAS) in thyroid-associated orbitopathy (TAO) using digital facial images taken by a digital single-lens reflex camera in a studio setting. In this study, we aimed to apply this system to smartphones and detect active TAO (CAS ≥3) using facial images captured by smartphone cameras. We evaluated the performance of our system on various smartphone models and compared it with the performance of ophthalmologists with varying clinical experience. Methods: We applied the preexisting ML architecture to classify photos taken with smartphones (Galaxy S21 Ultra, iPhone 12 pro, iPhone 11, iPhone SE 2020, Galaxy M20, and Galaxy A21S). The performance was evaluated with smartphone-captured images from 100 patients with TAO. Three ophthalmology residents, three general ophthalmologists with <5 years of clinical experience, and three oculoplastic specialists independently interpreted the same set of images taken under a studio environment and compared their results with those generated by the smartphone-based ML-assisted system. Reference CAS was determined by a consensus of three oculoplastic specialists. Results: Active TAO (CAS ≥3) was identified in 28 patients. Smartphone model used in capturing facial images influenced active TAO detection performance (F1 score 0.59-0.72). The smartphone-based system showed 74.5% sensitivity, 84.8% specificity, and F1 score 0.70 on top three smartphones. On images from all six smartphones, average sensitivity, specificity, and F1 score were 71.4%, 81.6%, and 0.66, respectively. Ophthalmology residents' values were 69.1%, 55.1%, and 0.46. General ophthalmologists' values were 61.9%, 79.6%, and 0.55. Oculoplastic specialists' values were 73.8%, 90.7%, and 0.75. This smartphone-based ML-assisted system predicted CAS within 1 point of reference CAS in 90.7% using facial images from smartphones. Conclusions: Our smartphone-based ML-assisted system shows reasonable accuracy in detecting active TAO, comparable with oculoplastic specialists and outperforming residents and general ophthalmologists. It may enable reliable self-monitoring for disease activity, but confirmatory research is needed for clinical application.
引用
- Shin K, Choung H, Lee MJ, et al. A Preliminary Evaluation of the Diagnostic Performance of a Smartphone-Based Machine Learning-Assisted System for Evaluation of Clinical Activity Score in Digital Images of Thyroid-Associated Orbitopathy. Thyroid. 2024. PMID: 38623815. doi:10.1089/thy.2023.0621
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。