ChatGPT生成文章与人工撰写文章的比较
本研究由两名骨骼肌肉放射科医生对ChatGPT生成的5篇放射学文章与已发表或撰写中的文章进行独立评分(1-5分),发现4/5的ChatGPT文章明显不准确且参考文献虚构,1篇结构良好但参考文献全部虚构。结论认为ChatGPT能生成看似连贯的研究文章,但内容事实不准确且参考文献虚构。
要点
- ChatGPT生成的放射学文章常含虚构参考文献,事实准确性差。
- 未经训练的读者可能误认为ChatGPT生成文章是真实的。
- 使用AI生成学术内容需严格核查事实与参考文献。
核心结论
- ChatGPT生成的文章虽然连贯且初看可能类似真实学术文章,但所有评估文章均事实不准确且参考文献虚构。
研究设计与样本
- 比较研究,由两名骨骼肌肉放射科医生对ChatGPT生成的5篇放射学文章与已发表或撰写中的文章进行独立评分,评分范围1-5分(1为差且不准确,5为优秀且准确)。
临床应用与阅片要点
- 在学术写作或阅片报告中引用AI生成内容时,需人工核实所有事实和参考文献,避免使用虚构引用。
局限性
- 样本量小(仅5篇ChatGPT生成文章),仅由两名放射科医生评估,未报告评分者间一致性。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
英文原文(摘要)· 点击展开
OBJECTIVE: ChatGPT (Generative Pre-trained Transformer) is an artificial intelligence language tool developed by OpenAI that utilises machine learning algorithms to generate text that closely mimics human language. It has recently taken the internet by storm. There have been several concerns regarding the accuracy of documents it generates. This study compares the accuracy and quality of several ChatGPT-generated academic articles with those written by human authors.
MATERIAL AND METHODS: We performed a study to assess the accuracy of ChatGPT-generated radiology articles by comparing them with the published or written, and under review articles. These were independently analysed by two fellowship-trained musculoskeletal radiologists and graded from 1 to 5 (1 being bad and inaccurate to 5 being excellent and accurate).
RESULTS: In total, 4 of the 5 articles written by ChatGPT were significantly inaccurate with fictitious references. One of the papers was well written, with a good introduction and discussion; however, all references were fictitious.
CONCLUSION: ChatGPT is able to generate coherent research articles, which on initial review may closely resemble authentic articles published by academic researchers. However, all of the articles we assessed were factually inaccurate and had fictitious references. It is worth noting, however, that the articles generated may appear authentic to an untrained reader.
引用
- Ariyaratne, Iyengar, Nischal, et al. A comparison of ChatGPT-generated articles with human-written articles. Skeletal Radiol. 2023. PMID: 37059827. doi:10.1007/s00256-023-04340-5
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。