人工智能在喉科患者信息生成中的应用
该研究比较了ChatGPT-3.5与美国耳鼻咽喉头颈外科学会(AAO-HNS)生成的喉科患者信息,使用FKGL、Flesch阅读轻松度、DISCERN和PEMAT-P等工具评估可读性和质量。结果显示ChatGPT输出至少高两个阅读年级水平且阅读轻松度显著更低,但信息质量 largely 无差异。
要点
- ChatGPT-3.5生成的喉科患者信息阅读年级水平至少比AAO-HNS材料高两级(p < 0.001)。
- ChatGPT输出阅读轻松度平均分显著低于AAO-HNS(p < 0.001)。
- 声带麻痹主题中ChatGPT的PEMAT-P理解性评分低于AAO-HNS材料(p > 0.05,未达统计学显著)。
核心结论
- ENTHealth.org网站上部分喉科主题的患者信息平均年级水平更低、阅读轻松度更高,而ChatGPT生成的信息质量 largely 无差异。
研究设计与样本
- 研究查询ENTHealth.org和ChatGPT-3.5获取喉科主题患者信息,每个主题生成三份去标识文本文档:一份来自AAO-HNS,两份来自ChatGPT(ChatGPT输出1和输出2)。
- 使用单因素方差分析和Tukey事后检验比较三来源的年级水平和阅读轻松度;使用独立t检验比较AAO-HNS与ChatGPT输出1的DISCERN和PEMAT理解性及可操作性评分。
主要发现
- ChatGPT输出1和ChatGPT输出2的材料阅读年级水平至少比AAO-HNS材料高两级(p < 0.001)。
- ChatGPT输出1和ChatGPT输出2文档的阅读轻松度平均分显著低于AAO-HNS(p < 0.001)。
- ChatGPT输出1关于声带麻痹的材料PEMAT-P理解性低于AAO-HNS材料(p > 0.05)。
关键数值与推荐
- 阅读年级水平:ChatGPT输出1和输出2至少比AAO-HNS高两级(p < 0.001)。
- 阅读轻松度:ChatGPT输出1和输出2平均分显著低于AAO-HNS(p < 0.001)。
- PEMAT-P理解性:ChatGPT输出1声带麻痹材料低于AAO-HNS材料(p > 0.05)。
临床怎么用
- 原文未据此给出推荐。
局限性
- 原文未报告局限性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
OBJECTIVE: To evaluate and compare the readability and quality of patient information generated by Chat-Generative Pre-Trained Transformer-3.5 (ChatGPT) and the American Academy of Otolaryngology-Head and Neck Surgery (AAO-HNS) using validated instruments including Flesch-Kincaid Grade Level (FKGL), Flesch Reading Ease, DISCERN, and Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P). METHODS: ENTHealth.org and ChatGPT-3.5 were queried for patient information on laryngology topics. ChatGPT-3.5 was queried twice for a given topic to evaluate for reliability. This generated three de-identified text documents for each topic: one from AAO-HNS and two from ChatGPT (ChatGPT Output 1, ChatGPT Output 2). Grade level and reading ease were compared between the three sources using a one-way analysis of variance and Tukey's post hoc test. Independent t-tests were used to compare DISCERN and PEMAT understandability and actionability scores between AAO-HNS and ChatGPT Output 1. RESULTS: Material generated from ChatGPT Output 1 and ChatGPT Output 2 were at least two reading grade levels higher than that of material from AAO-HNS (p < 0.001). Regarding reading ease, ChatGPT Output 1 and ChatGPT Output 2 documents had significantly lower mean scores compared to AAO-HNS (p < 0.001). Moreover, ChatGPT Output 1 material on vocal cord paralysis had a lower PEMAT-P understandability compared to that of AAO-HNS material (p > 0.05). CONCLUSION: Patient information on the ENTHealth.org website for select laryngology topics was, on average, of a lower grade level and higher reading ease compared to that produced by ChatGPT, but interestingly with largely no difference in the quality of information provided. LEVEL OF EVIDENCE: NA Laryngoscope, 135:1295-1300, 2025.
引用
- Tran QL, Huynh PP, Le B, et al. Utilization of Artificial Intelligence in the Creation of Patient Information on Laryngology Topics. Laryngoscope. 2024. PMID: 39503508. doi:10.1002/lary.31891
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。