文献速递技术与安全

人工智能生成的放射学社论:专家编辑能否识别

Artificial Intelligence-Generated Editorials in Radiology: Can Expert Editors Detect Them?
PubMed 文献 · AJNR Am J Neuroradiol · 2025-03-04

研究将16篇人类撰写的社论摘要输入GPT-4生成人工智能社论,由6名经验丰富的编辑以非配对和配对方式评估内容并判断作者身份,同时进行人工智能检测和查重分析。人类撰写文章的中位人工智能概率评分为2.0%,人工智能撰写文章为58%;非配对正确分类率为58%,配对时升至70%。GPT-4能写出高质量、查重工具未标记为剽窃的社论,可能不被编辑察觉,检测工具有限识别。

要点

  • 人类撰写社论中位人工智能概率评分2.0%,人工智能撰写为58%。
  • 非配对作者身份正确分类率58%,配对评估时升至70%。
  • 编辑对感知为人类撰写的文章有正面偏好,82%强烈偏好发表。

核心结论

  • GPT-4能撰写高质量、iThenticate未标记为剽窃的社论,可能不被编辑察觉,检测工具仅有限识别;编辑对人类撰写文章存在正面偏好。

研究设计与样本

  • 研究纳入来自8种期刊的16篇社论;将16篇人类撰写社论的摘要输入GPT-4生成人工智能社论;6名经验丰富编辑先以非配对方式用1-5 Likert量表评估各文章内容并判断作者身份,再以配对方式判断哪篇由人工智能生成及哪篇应发表;最后用人工智能检测器和查重分析文章。

诊断标准与关键数值

  • 人类撰写文章中位人工智能概率评分为2.0%,人工智能撰写文章为58%。
  • 人工智能撰写文章间中位相似度评分为3%。
  • 非配对文章作者身份正确分类率为58%,配对设置下评分准确率提高至70%。

临床应用与阅片要点

  • 编辑在非配对设置下仅58%正确分类作者身份,配对设置下准确率70%。
  • 编辑对感知为人类撰写的文章存在正面偏好,配对设置下82%强烈偏好发表感知为人类撰写的文章。
  • iThenticate未将GPT-4撰写文章标记为剽窃。

局限性

  • 原文未报告局限性

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

BACKGROUND AND PURPOSE: Artificial intelligence is capable of generating complex texts that may be indistinguishable from those written by humans. We aimed to evaluate the ability of GPT-4 to write radiology editorials and to compare these with human-written counterparts, thereby determining their real-world applicability for scientific writing.

MATERIALS AND METHODS: Sixteen editorials from 8 journals were included. To generate the artificial intelligence (AI)-written editorials, the summary of 16 human-written editorials was fed into GPT-4. Six experienced editors reviewed the articles. First, an unpaired approach was used. The raters were asked to evaluate the content of each article by using a 1-5 Likert scale across specified metrics. Then, they determined whether the editorials were written by humans or AI. The articles were then evaluated in pairs to determine which article was generated by AI and which should be published. Finally, the articles were analyzed with an AI detector and for plagiarism.

RESULTS: The human-written articles had a median AI probability score of 2.0%, whereas the AI-written articles had 58%. The median similarity score among AI-written articles was 3%. Fifty-eight percent of unpaired articles were correctly classified regarding authorship. Rating accuracy was increased to 70% in the paired setting. AI-written articles received slightly higher scores in most metrics. When stratified by perception, human-written perceived articles were rated higher in most categories. In the paired setting, raters strongly preferred publishing the article they perceived as human-written (82%).

CONCLUSIONS: GPT-4 can write high-quality articles that iThenticate does not flag as plagiarized, which may go undetected by editors, and that detection tools can detect to a limited extent. Editors showed a positive bias toward human-written articles.

来源:AJNR Am J Neuroradiol,查看原文。版权归原作者所有。

引用

  1. Ozkara BB, Boutet A, Comstock BA, et al. Artificial Intelligence-Generated Editorials in Radiology: Can Expert Editors Detect Them? AJNR Am J Neuroradiol. 2025. PMID: 39288967. doi:10.3174/ajnr.A8505

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。