文献速递耳科与耳神经

ChatGPT音频图解读的版本特异性表现:4.0与5.1对比

Version-Specific Performance of ChatGPT in Audiogram Interpretation: Comparison of ChatGPT-4.0 and ChatGPT-5.1.
PubMed 文献 · Otolaryngol Head Neck Surg · 2026-10-28

5.1在7项中6项显著优于4.0(P<.001),效应量0.53-0.94,但两版对含掩蔽错误的听力图均表现不佳。

要点

  • ChatGPT-5.1在听力图解读多数项目上显著优于4.0,最高提升在听力损失程度分级。
  • 两版模型对含掩蔽错误的听力图均表现不佳,提示需警惕输入数据质量。
  • 骨导阈值识别是唯一无显著版本差异的项目(P=.121)。

核心结论

  • ChatGPT-5.1在听力图解读上较4.0有实质性提升,尤其在高阶诊断推理任务。
  • 两版模型对临床有缺陷的输入数据均表现不佳。
  • 整合大语言模型到听力学实践需版本特异性基准测试与专家监督。

研究设计与样本

  • 听力图含20份故意设置掩蔽错误,通过标准化7步诊断提示提交给两版模型。
  • 两位具备听力学专长的委员会认证耳鼻咽喉科医生独立按5分Likert量表评分,每模型1400次评估。
  • 评分者间信度用二次加权Cohen kappa评估,模型比较用Wilcoxon符号秩检验。

主要发现

  • 总体评分者间信度可接受(κ=0.74-0.78),但ChatGPT-5.1在听力损失程度分类上一致性较低(κ=0.44)。
  • ChatGPT-5.1在7个问题中的6个显著高于ChatGPT-4.0(P<.001),效应量0.53-0.94。
  • 最大改善出现在听力损失程度分类(中位差异+1.0分)。
  • 骨导阈值识别是唯一无显著版本间差异的项目(P=.121)。
  • 两版模型对含掩蔽错误的听力图均表现不佳。

关键数值与推荐

  • 评分采用5分Likert量表,每模型共1400次评估。
  • 评分者间信度总体κ=0.74-0.78,ChatGPT-5.1程度分类κ=0.44。
  • 6/7问题版本间差异显著(P<.001),效应量0.53-0.94。
  • 骨导阈值识别版本间差异无统计学意义(P=.121)。

临床怎么用

  • 选用较新版本ChatGPT可能提升听力图解读的高阶诊断推理表现。
  • 对含掩蔽错误的听力图,无论模型版本均需人工复核,不能单独依赖模型输出。
  • 临床整合大语言模型时应进行版本特异性验证并保留专家监督。

局限性

  • 研究使用模拟软件生成的听力图,非真实临床数据。
  • 仅比较两个特定版本,结果不能外推至其他版本或模型。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVE: Large language models (LLMs) are increasingly explored for clinical decision support, yet how successive versions of the same model compare on diagnostic tasks remains unclear. This study aimed to quantify version-specific changes in diagnostic performance by directly comparing ChatGPT-4.0 and ChatGPT-5.1 on standardized pure-tone audiogram interpretation. STUDY DESIGN: Comparative diagnostic performance study. SETTING: Controlled experimental study using audiograms generated with simulation software. METHODS: One hundred audiograms, including 20 with intentional masking errors, were generated using AudSim Flex® software and submitted to both model versions through a standardized 7-step diagnostic prompt based on ASHA criteria. Two board-certified otolaryngologists with audiological expertise independently scored each response on a 5-point Likert scale, yielding 1400 evaluations per model. Inter-rater agreement was assessed using quadratic weighted Cohen's kappa, and model comparisons were performed with the Wilcoxon signed-rank test. RESULTS: Inter-rater reliability was acceptable overall (κ = 0.74-0.78), though agreement was lower for degree-of-loss classification with ChatGPT-5.1 (κ = 0.44). ChatGPT-5.1 scored significantly higher than ChatGPT-4.0 on 6 of 7 questions (P < .001), with effect sizes ranging from 0.53 to 0.94. The largest improvement appeared in degree-of-hearing-loss classification (median difference, +1.0 point). Bone conduction threshold identification was the only item without a significant between-version difference (P = .121). Both models performed poorly on audiograms containing masking errors. CONCLUSION: ChatGPT-5.1 demonstrated meaningful improvements over ChatGPT-4.0 in audiogram interpretation, particularly in higher-order diagnostic reasoning tasks. However, both versions struggled with clinically flawed input data. These findings underscore the need for version-specific benchmarking and expert oversight when integrating LLMs into audiological practice.

来源:Otolaryngol Head Neck Surg,查看原文。版权归原作者所有。

引用

  1. Tahir, Karaçaylı, Dünder, et al. Version-Specific Performance of ChatGPT in Audiogram Interpretation: Comparison of ChatGPT-4.0 and ChatGPT-5.1. Otolaryngol Head Neck Surg. 2026. PMID: 42949252. doi:10.1002/ohn.70500

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。