人工智能聊天机器人对甲状腺射频消融患者教育回答的准确性、质量与可读性评估
该横断面分析使用20个标准化临床问题,由6名盲法评审医师对ChatGPT-4、Gemini、Copilot和Perplexity关于甲状腺射频消融的回答进行评分。Gemini在整体质量和准确性上得分最高,显著优于ChatGPT和Copilot(p<0.005)。聊天机器人对简单事实信息较可靠,但对判断性或情境依赖性评估不够可靠,应作为临床医生审核的患者教育材料的补充而非替代。
要点
- Gemini在整体质量和事实准确性上得分最高,显著优于ChatGPT和Copilot。
- ChatGPT回答显著更长且更易读,Gemini得分变异性最低。
- 聊天机器人对简单事实可靠,对判断性评估不可靠,应辅助而非替代医生教育。
核心结论
- AI聊天机器人在甲状腺射频消融患者查询中表现因平台而异,对简单事实信息较可靠,对判断或情境依赖性评估不够可靠。
- 这些工具应补充而非替代临床医生审核的患者教育和机构材料。
研究设计与样本
- 横断面分析,使用20个关于甲状腺射频消融的标准化临床问题,由6名盲法评审医师使用5点Likert量表评估回答的整体质量和事实准确性。
- 评估ChatGPT-4、Gemini、Copilot和Perplexity四个公开AI聊天机器人的回答,并分析可读性和回答长度,统计显著性定义为p<0.05。
主要发现
- Gemini整体质量平均得分4.08±0.87,准确性平均得分3.76±1.05,显著优于ChatGPT和Copilot(p<0.005)。
- ChatGPT回答显著更长且更易读,Gemini跨问题得分变异性最低。
- Copilot和Perplexity在多数领域排名最低。
- 问题级分析识别出能最佳区分平台的具体提示。
临床怎么用
- 临床医生应了解AI聊天机器人对甲状腺射频消融患者教育信息的质量差异,推荐使用时需谨慎。
- AI工具应作为临床医生审核的患者教育和机构材料的补充,而非替代。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: Artificial intelligence (AI) chatbots are increasingly being used by patients to obtain medical information. Comparison between platforms with specialty-specific physician assessment remains limited. This study compares the quality, factual accuracy, readability, and consistency of responses generated by four publicly available AI chatbots when answering patient-centered questions about thyroid radiofrequency ablation (RFA). METHODS: We conducted a cross-sectional analysis of chatbot-generated responses using 20 standardized clinical questions about thyroid RFA. Responses from ChatGPT-4, Gemini, Copilot, and Perplexity were evaluated by six blinded physician reviewers experienced in thyroid RFA using 5-point Likert scales for global quality and factual accuracy. Higher Likert scale scores indicated better performance. Readability and response length were analyzed with established metrics. Statistical significance was defined as p < 0.05. RESULTS: Gemini achieved the highest mean scores for global quality (4.08 ± 0.87) and accuracy (3.76 ± 1.05), with significantly better performance than ChatGPT and Copilot (p < 0.005). ChatGPT responses were significantly longer and more readable. Score variability across questions was lowest for Gemini. Copilot and Perplexity ranked lowest across most domains. Question-level analysis identified specific prompts that best discriminated between platforms. CONCLUSIONS: AI chatbot performance varied across platforms for thyroid RFA queries. Chatbots were generally reliable for straightforward factual information but were less dependable for judgment or context-dependent assessments. These AI tools should supplement, not replace, clinician-vetted patient education and institutional materials.
引用
- Beiriger J, Estephan LE, Campbell D, et al. Evaluating Artificial Intelligence Chatbots for Patient Education on Thyroid Radiofrequency Ablation: An Analysis of Accuracy, Quality, and Readability. Thyroid. 2026. PMID: 41791887. doi:10.1177/10507256251414974
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。