文献速递耳科与耳神经

Google与人工智能(ChatGPT)上的良性阵发性位置性眩晕信息比较

BPPV Information on Google Versus AI (ChatGPT).
PubMed 文献 · Otolaryngol Head Neck Surg · 2023-08-25

本研究于2023年2月对比Google搜索前30条结果与ChatGPT对5个常见患者问题的回答,评估可读性、质量、可理解性和可操作性。ChatGPT回答的FKGL更高(13.9±2.5 vs 10.7±2.6,P<.001)、FRE更低(34.9±11.2 vs 46.5±14.3,P=.005),DISCERN第二部分得分更低(17.5±3.9 vs 25.4±7.5,P=.001),提示其信息更难读、质量更低、更难理解。

要点

  • ChatGPT回答的FKGL为13.9±2.5,高于Google网页的10.7±2.6(P<.001),可读性更低。
  • ChatGPT回答的FRE为34.9±11.2,低于Google网页的46.5±14.3(P=.005)。
  • Google网页DISCERN第二部分得分为25.4±7.5,高于单个ChatGPT回答的17.5±3.9(P=.001)。

核心结论

  • 本研究结果提示,与Google搜索信息相比,ChatGPT上的信息更难阅读、质量更低且更难理解。

研究设计与样本

  • 研究于2023年2月对比Google搜索「benign paroxysmal positional vertigo」的前30条结果与ChatGPT对5个常见患者问题的回答。
  • 由评审者使用Flesch-Kincaid Grade Level(FKGL)、Flesch Reading Ease(FRE)、DISCERN工具和印刷材料患者教育材料评估工具进行评分分析。

主要发现

  • Google网页的平均FKGL为10.7±2.6,ChatGPT回答为13.9±2.5(P<.001),对应可读性更低。
  • Google网页的平均FRE为46.5±14.3,ChatGPT回答为34.9±11.2(P=.005),对应可读性更低。
  • Google网页的DISCERN第二部分得分为25.4±7.5,单个ChatGPT回答为17.5±3.9(P=.001),合并ChatGPT回答为25.0±0.9(P=.928)。
  • 评审者对ChatGPT回答准确性的平均评分为4.19±0.82,时效性为4.31±0.67。

临床怎么用

  • 原文未据此给出推荐。

局限性

  • 原文未报告局限性。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

OBJECTIVE: To quantitatively compare online patient education materials found using traditional search engines (Google) versus conversational Artificial Intelligence (AI) models (ChatGPT) for benign paroxysmal positional vertigo (BPPV). STUDY DESIGN: The top 30 Google search results for "benign paroxysmal positional vertigo" were compared to the OpenAI conversational AI language model, ChatGPT, responses for 5 common patient questions posed about BPPV in February 2023. Metrics included readability, quality, understandability, and actionability. SETTING: Online information. METHODS: Validated online information metrics including Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease (FRE), DISCERN instrument score, and Patient Education Materials Assessment Tool for Printed Materials were analyzed and scored by reviewers. RESULTS: Mean readability scores, FKGL and FRE, for the Google webpages were 10.7 ± 2.6 and 46.5 ± 14.3, respectively. ChatGPT responses had a higher FKGL score of 13.9 ± 2.5 (P < .001) and a lower FRE score of 34.9 ± 11.2 (P = .005), both corresponding to lower readability. The Google webpages had a DISCERN part 2 score of 25.4 ± 7.5 compared to the individual ChatGPT responses with a score of 17.5 ± 3.9 (P = .001), and the combined ChatGPT responses with a score of 25.0 ± 0.9 (P = .928). The average scores of the reviewers for all ChatGPT responses for accuracy were 4.19 ± 0.82 and 4.31 ± 0.67 for currency. CONCLUSION: The results of this study suggest that the information on ChatGPT is more difficult to read, of lower quality, and more difficult to comprehend compared to information on Google searches.

来源:Otolaryngol Head Neck Surg,查看原文。版权归原作者所有。

引用

  1. Bellinger JR, De La Chapa JS, Kwak MW, et al. BPPV Information on Google Versus AI (ChatGPT). Otolaryngol Head Neck Surg. 2023. PMID: 37622581. doi:10.1002/ohn.506

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。