多模型人工智能在突发性感音神经性聋评估中的比较研究
本研究前瞻性比较了ChatGPT-4.0、Claude 3.7 Sonet和OpenAI Mini 3三种大语言模型在突发性感音神经性聋管理中的诊断准确性、语言清晰度和用户满意度。ChatGPT-4.0在临床准确性和满意度上得分最高,Claude 3.7在清晰度上更优,OpenAI Mini 3词汇多样性最高但总体得分较低。
要点
- ChatGPT-4.0在临床准确性和完整性上表现最佳
- Claude 3.7在语言清晰度和句子复杂度上更优
- 客观质量与主观满意度显著相关,支持混合模型应用
核心结论
- ChatGPT-4.0提供最临床对齐和满意的回答,Claude 3.7语言更精炼。
- 研究支持在耳鼻咽喉科中按情境混合使用大语言模型。
研究设计与样本
- 前瞻性、多维度比较分析,采用盲法专家评估。
- 10名认证耳鼻咽喉科医生使用QAMAI、AIPI和AISPE-Q三种工具评估。
主要发现
- ChatGPT-4.0的QAMAI临床准确性得分为4.57,完整性得分为4.53。
- ChatGPT-4.0的AISPE-Q满意度为94%。
- Claude 3.7在清晰度和句子复杂度上表现更优。
- OpenAI Mini 3词汇多样性最高,但总体得分较低。
- 评估者间信度强(ICC > 0.85),客观质量与主观满意度显著相关(r > 0.76)。
关键数值与推荐
- QAMAI评估临床准确性,ChatGPT-4.0得分为4.57。
- AISPE-Q评估满意度,ChatGPT-4.0为94%。
- ICC > 0.85表示评估者间信度强。
- r > 0.76表示客观质量与主观满意度显著相关。
临床怎么用
- 可根据临床场景选择模型:需高准确性和满意度时用ChatGPT-4.0,需语言清晰时用Claude 3.7。
- 可考虑混合大语言模型方法用于患者教育、诊断和AI分诊。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVE: To compare the diagnostic accuracy, linguistic clarity, and user satisfaction of three large language models (ChatGPT-4.0, Claude 3.7 Sonet, and OpenAI Mini 3) in managing sudden sensorineural hearing loss. STUDY DESIGN: Prospective, multi-domain comparative analysis using blinded expert evaluation. SETTING: Online artificial intelligence (AI) platforms accessed under standardized conditions. METHODS: Twenty-seven sudden sensorineural hearing loss-related questions-covering general knowledge, audiometric interpretation, and clinical case scenarios-were submitted to the three AI models. Responses were evaluated by 10 board-certified otolaryngologists using three validated tools: Quality Assessment of Medical Artificial Intelligence (QAMAI), Artificial Intelligence Performance Instrument (AIPI), and Artificial Intelligence Satisfaction and Performance Evaluation Questionnaire (AISPE-Q). Linguistic complexity was assessed using metrics such as word count, sentence length, lexical diversity, and clinical verb use. RESULTS: ChatGPT-4.0 demonstrated the highest scores in clinical accuracy (QAMAI: 4.57), completeness (4.53), and evaluator satisfaction (AISPE-Q: 94%). Claude 3.7 outperformed in clarity and sentence complexity, while OpenAI Mini 3 exhibited the highest lexical diversity and directive tone but scored lower overall. Inter-rater reliability was strong (intraclass correlation coefficient [ICC] > 0.85). Correlation analysis revealed a significant relationship between objective quality and subjective satisfaction (r > 0.76). CONCLUSION: ChatGPT-4.0 delivered the most clinically aligned and satisfactory responses, whereas Claude 3.7 provided linguistically refined outputs. Our findings support the context-specific application of hybrid large language model approaches in otolaryngology, particularly for patient education, diagnosis, and AI-driven triage. LEVEL OF EVIDENCE: 2-prospective comparative diagnostic accuracy study.
引用
- Aliyeva A, Muradova A, Hashimli R, et al. Multi-model Artificial Intelligence Evaluation in Sudden Sensorineural Hearing Loss. Otolaryngol Head Neck Surg. 2026. PMID: 41603577. doi:10.1002/ohn.70143
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。