文献速递甲状腺疾病

人工智能聊天机器人对甲状腺射频消融患者教育回答的准确性、质量与可读性评估

Evaluating Artificial Intelligence Chatbots for Patient Education on Thyroid Radiofrequency Ablation: An Analysis of Accuracy, Quality, and Readability.
PubMed 文献 · Thyroid · 2026-01-13

该横断面分析使用20个标准化临床问题,由6名盲法评审医师对ChatGPT-4、Gemini、Copilot和Perplexity关于甲状腺射频消融的回答进行评分。Gemini在整体质量和准确性上得分最高,显著优于ChatGPT和Copilot(p<0.005)。聊天机器人对简单事实信息较可靠,但对判断性或情境依赖性评估不够可靠,应作为临床医生审核的患者教育材料的补充而非替代。

要点

  • Gemini在整体质量和事实准确性上得分最高,显著优于ChatGPT和Copilot。
  • ChatGPT回答显著更长且更易读,Gemini得分变异性最低。
  • 聊天机器人对简单事实可靠,对判断性评估不可靠,应辅助而非替代医生教育。

核心结论

  • AI聊天机器人在甲状腺射频消融患者查询中表现因平台而异,对简单事实信息较可靠,对判断或情境依赖性评估不够可靠。
  • 这些工具应补充而非替代临床医生审核的患者教育和机构材料。

研究设计与样本

  • 横断面分析,使用20个关于甲状腺射频消融的标准化临床问题,由6名盲法评审医师使用5点Likert量表评估回答的整体质量和事实准确性。
  • 评估ChatGPT-4、Gemini、Copilot和Perplexity四个公开AI聊天机器人的回答,并分析可读性和回答长度,统计显著性定义为p<0.05。

主要发现

  • Gemini整体质量平均得分4.08±0.87,准确性平均得分3.76±1.05,显著优于ChatGPT和Copilot(p<0.005)。
  • ChatGPT回答显著更长且更易读,Gemini跨问题得分变异性最低。
  • Copilot和Perplexity在多数领域排名最低。
  • 问题级分析识别出能最佳区分平台的具体提示。

临床怎么用

  • 临床医生应了解AI聊天机器人对甲状腺射频消融患者教育信息的质量差异,推荐使用时需谨慎。
  • AI工具应作为临床医生审核的患者教育和机构材料的补充,而非替代。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

BACKGROUND: Artificial intelligence (AI) chatbots are increasingly being used by patients to obtain medical information. Comparison between platforms with specialty-specific physician assessment remains limited. This study compares the quality, factual accuracy, readability, and consistency of responses generated by four publicly available AI chatbots when answering patient-centered questions about thyroid radiofrequency ablation (RFA). METHODS: We conducted a cross-sectional analysis of chatbot-generated responses using 20 standardized clinical questions about thyroid RFA. Responses from ChatGPT-4, Gemini, Copilot, and Perplexity were evaluated by six blinded physician reviewers experienced in thyroid RFA using 5-point Likert scales for global quality and factual accuracy. Higher Likert scale scores indicated better performance. Readability and response length were analyzed with established metrics. Statistical significance was defined as p < 0.05. RESULTS: Gemini achieved the highest mean scores for global quality (4.08 ± 0.87) and accuracy (3.76 ± 1.05), with significantly better performance than ChatGPT and Copilot (p < 0.005). ChatGPT responses were significantly longer and more readable. Score variability across questions was lowest for Gemini. Copilot and Perplexity ranked lowest across most domains. Question-level analysis identified specific prompts that best discriminated between platforms. CONCLUSIONS: AI chatbot performance varied across platforms for thyroid RFA queries. Chatbots were generally reliable for straightforward factual information but were less dependable for judgment or context-dependent assessments. These AI tools should supplement, not replace, clinician-vetted patient education and institutional materials.

来源:Thyroid,查看原文。版权归原作者所有。

引用

  1. Beiriger J, Estephan LE, Campbell D, et al. Evaluating Artificial Intelligence Chatbots for Patient Education on Thyroid Radiofrequency Ablation: An Analysis of Accuracy, Quality, and Readability. Thyroid. 2026. PMID: 41791887. doi:10.1177/10507256251414974

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。