文献速递咽喉与嗓音

多模态深度学习在频闪喉镜下声带病变分类中的应用

A Multimodal Approach for Deep-Learning Classification of Vocal Fold Pathologies in Stroboscopy.
PubMed 文献 · Laryngoscope · 2026-01-05

本研究回顾性纳入54例单侧声带麻痹、42例声带病变和41例健康声带患者,构建基于频闪喉镜图像、嗓音和临床人口学数据的多模态深度学习分类器。多模态模型在测试集上准确率为76.9%,优于图像模型(61.5%)和音频模型(65.4%);外部数据集准确率降至45%,但仍高于单一模态模型。

要点

  • 多模态融合(图像+嗓音+临床数据)优于单一模态分类声带病变。
  • 测试集准确率76.9%,外部验证降至45%,泛化能力有限。
  • 未来需在更大数据集上验证模型性能。

核心结论

  • 多模态分类器在测试集上准确率76.9%,高于图像模型(61.5%)和音频模型(65.4%)。
  • 外部数据集上多模态准确率45%,仍高于视频模型(42%)和音频模型(31%)。

研究设计与样本

  • 回顾性研究,纳入单侧声带麻痹54例、声带病变42例、健康声带41例,按80%训练、20%测试划分。

主要发现

  • 测试集准确率:多模态76.9%,图像61.5%,音频65.4%。
  • 外部数据集准确率:多模态45%,视频42%,音频31%。

临床怎么用

  • 多模态深度学习有潜力辅助频闪喉镜下声带病变分类,但外部泛化能力需提升。

局限性

  • 样本量较小,外部数据集准确率下降明显,需更大数据集验证。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVE: To develop and validate a multimodal deep-learning classifier trained on stroboscopic image, voice, and clinicodemographic data, differentiating between three different vocal fold (VF) states: healthy (HVF), unilateral paralysis (UVFP), and VF lesions, including benign and malignant pathologies. METHODS: Patients with UVFP (n = 54), VF lesions (n = 42), and HVF (n = 41) were retrospectively identified. Image frames and voice samples were extracted from stroboscopic videos. Clinicodemographic variables were collected from the electronic health record. Patient-level data was independently divided into training (80%) and testing (20%). Visual features were extracted using a transformer DINOv2 and acoustic features were extracted using Librosa. All three feature modalities were combined using a custom multilayer perceptron. Unimodality models using only image or only voice data were trained for comparison. Accuracy and F1 scores were used to validate the models. RESULTS: On a hold-out test set, the multimodal classifier demonstrated stronger performance (76.9% accuracy) compared to the image classifier (61.5% accuracy) and audio classifier (65.4% accuracy). On an external dataset, the multimodal classifier accuracy dropped to 45%, though still an improvement compared to accuracies of 42% and 31% for the video-only and audio-only modalities, respectively. CONCLUSIONS: In this proof-of-concept study, we successfully developed a multimodal dataset and classifier for VF pathology, demonstrating the potential of combining stroboscopic frames, voice and text data. The multimodal classifier achieved higher accuracy than the image-only model and audio-only models. Future models should validate these findings on larger datasets.

来源:Laryngoscope,查看原文。版权归原作者所有。

引用

  1. Surapaneni S, Kutler RB, Setzen SA, et al. A Multimodal Approach for Deep-Learning Classification of Vocal Fold Pathologies in Stroboscopy. Laryngoscope. 2026. PMID: 41489089. doi:10.1002/lary.70355

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。