自动语音识别系统对聋人和重听人语音性能的量化评估
研究测试了四个商业语音转文本接口对24名聋人和重听人及9名正常听力者共850个音频文件的词错误率,发现聋人和重听人组平均词错误率为52.6%,约为正常听力组5.0%的10倍,且言语可懂度低、语前听力损失及以手语为主要沟通方式者错误率更高。
要点
- 聋人和重听人组平均词错误率52.6%,正常听力组5.0%,前者约为后者的10倍。
- 低、中言语可懂度组词错误率分别为85.9%和46.6%,显著差于高可懂度组的9.5%。
- 语前听力损失组词错误率80.5%,显著差于语后听力损失组的37.1%。
核心结论
- 商业自动语音识别系统对聋人和重听人表现不佳,尤其在言语可懂度低至中等、语前听力损失及以手语为主要沟通方式者中。
- 这与大型科技公司关于可及性的承诺形成对比,表明需要以异构聋人和重听人语音数据合乎伦理地训练自动语音识别系统。
研究设计与样本
- 使用孟菲斯大学言语感知评估实验室包含850个聋人和重听人及正常听力者音频文件的语料库,测试四个语音转文本应用程序接口:Amazon Web Services、Microsoft Azure、Google Chirp和OpenAI Whisper。
- 量化24名聋人和重听人及9名正常听力参与者转录的词错误率,并按言语可懂度分类、听力损失发病时间和主要沟通方式进行亚组分析。
主要发现
- 各接口平均词错误率在聋人和重听人组为52.6%,正常听力组为5.0%,前者约为后者的10倍。
- 低言语可懂度组词错误率85.9%,中言语可懂度组46.6%,均显著差于高言语可懂度组的9.5%(与正常听力组相当)。
- 语前听力损失组词错误率80.5%,显著差于语后听力损失组的37.1%。
- 以手语为主要沟通方式组词错误率70.2%,显著差于口语和手语兼用组的51.5%及仅口语组的19.7%。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVE: To evaluate the performance of commercial automatic speech recognition (ASR) systems on d/Deaf and hard-of-hearing (d/Dhh) speech. METHODS: A corpus containing 850 audio files of d/Dhh and normal hearing (NH) speech from the University of Memphis Speech Perception Assessment Laboratory was tested on four speech-to-text application program interfaces (APIs): Amazon Web Services, Microsoft Azure, Google Chirp, and OpenAI Whisper. We quantified the Word Error Rate (WER) of API transcriptions for 24 d/Dhh and nine NH participants and performed subgroup analysis by speech intelligibility classification (SIC), hearing loss (HL) onset, and primary communication mode. RESULTS: Mean WER averaged across APIs was 10 times higher for the d/Dhh group (52.6%) than the NH group (5.0%). APIs performed significantly worse for "low" and "medium" SIC (85.9% and 46.6% WER, respectively) as compared to "high" SIC group (9.5% WER, comparable to NH group). APIs performed significantly worse for speakers with prelingual HL relative to postlingual HL (80.5% and 37.1% WER, respectively). APIs performed significantly worse for speakers primarily communicating with sign language (70.2% WER) relative to speakers with both oral and sign language communication (51.5%) or oral communication only (19.7%). CONCLUSION: Commercial ASR systems underperform for d/Dhh individuals, especially those with "low" and "medium" SIC, prelingual onset of HL, and sign language as primary communication mode. This contrasts with Big Tech companies' promises of accessibility, indicating the need for ASR systems ethically trained on heterogeneous d/Dhh speech data. LEVEL OF EVIDENCE: 3 Laryngoscope, 135:191-197, 2025.
引用
- Zhao R, Choi ASG, Koenecke A, et al. Quantification of Automatic Speech Recognition System Performance on d/Deaf and Hard of Hearing Speech. Laryngoscope. 2024. PMID: 39157956. doi:10.1002/lary.31713
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。