人工智能驱动的喉镜检查:探索Google Gemini的未来应用
研究评估了Google Gemini 1.5 Pro多模态基础模型对来自Google Images和YouTube的喉镜帧及视频的解读能力。Gemini在87/88帧(98.9%)和15/15段视频喉镜检查(100%)中识别出操作为喉镜检查,在55/88帧(62.5%)和3/15段视频(20.0%)中准确诊断出病变,在58/88帧(65.9%)和6/15段视频(40%)中识别出病变侧别,并在未经微调的情况下叙述了两段手术视频喉镜检查。
要点
- Gemini识别喉镜检查操作:帧87/88(98.9%),视频15/15(100%)。
- Gemini诊断病变:帧55/88(62.5%),视频3/15(20.0%)。
- Gemini识别病变侧别:帧58/88(65.9%),视频6/15(40%)。
核心结论
- Gemini 1.5 Pro在分析喉镜检查方面显示出显著潜力,表明基础模型有潜力成为耳鼻咽喉科复杂专家任务中的临床决策支持工具。
研究设计与样本
- 评估Google Gemini 1.5 Pro对来自Google Images和YouTube的喉镜帧及视频的解读能力。
主要发现
- Gemini在87/88帧(98.9%)和15/15段视频喉镜检查(100%)中识别出操作为喉镜检查。
- Gemini在55/88帧(62.5%)和3/15段视频(20.0%)中准确诊断出病变。
- Gemini在58/88帧(65.9%)和6/15段视频(40%)中识别出病变侧别。
- Gemini在未经微调的情况下叙述了两段手术视频喉镜检查。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
Foundation models (FMs) are general-purpose artificial intelligence (AI) neural networks trained on massive datasets, including code, text, audio, images, and video, to handle myriad tasks from generating texts to analyzing images or composing music. We evaluated Google Gemini 1.5 Pro, currently the largest token context window multimodal FM and best-performing commercial model for video analysis, for interpreting laryngoscopy frames and videos from Google Images and YouTube. Gemini recognized the procedure as laryngoscopy in 87/88 frames (98.9%) and in 15/15 video-laryngoscopies (100%), accurately diagnosed a pathology in 55/88 frames (62.5%) and 3/15 videos (20.0%), identified lesion sides in 58/88 frames (65.9%) and 6/15 videos (40%) and narrated two operative video-laryngoscopies without fine-tuning. Findings suggest that Gemini 1.5 Pro shows significant potential for analyzing laryngoscopy, demonstrating the potential for FMs as clinical decision support tools in complex expert tasks in otolaryngology. LEVEL OF EVIDENCE: 3.
引用
- Setzen SA, Andreadis K, Elemento O, et al. AI-Powered Laryngoscopy: Exploring the Future With Google Gemini. Laryngoscope. 2025. PMID: 39976345. doi:10.1002/lary.32089
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。