文献速递技术与安全

大语言模型可助力放射学研究所需的生物统计学与编程

Large Language Models can Help with Biostatistics and Coding Needed in Radiology Research.
PubMed 文献 · Acad Radiol · 2024-10-15

该研究通过两个实验评估ChatGPT-4o、ChatGPT-3.5和Google Gemini在放射学研究中的生物统计学建议与代码生成能力;ChatGPT-4o在8个统计问题中正确建议7个,且其生成的R代码运行时错误较少;两种模型均能为深度学习任务生成初始Python代码,但需用户完善和验证代码,并注意潜在错误、幻觉和数据隐私法规。

要点

  • ChatGPT-4o在8个统计问题中正确建议7个,ChatGPT-3.5正确6个,Gemini正确5个。
  • ChatGPT-4o生成的R代码运行时错误为6/7,ChatGPT-3.5为5/7,Gemini为5/7。
  • LLM可提供数据可视化、统计检验和深度学习模型的初始代码,但需用户完善和验证。

核心结论

  • LLM可协助放射学研究中的编程任务,为数据可视化、统计检验和深度学习模型提供初始代码,帮助具备基础生物统计学知识的研究者;LLM可提供有用起点,但需用户完善和验证代码,并需谨慎对待潜在错误、幻觉风险和数据隐私法规。

研究设计与样本

  • 研究使用ChatGPT-4o、ChatGPT-3.5和Google Gemini的网页版聊天界面进行两个实验:实验1使用Cancer Imaging Archive数据集评估生物统计学建议和R代码生成,实验2使用RSNA-STR Pneumonia Detection Challenge数据集评估生成基于Transformer的图像分类模型(Vision Transformer ViT-B/16)Python代码的能力。

诊断标准与关键数值

  • 在8个统计问题中,正确统计答案的建议数:ChatGPT-4o为7个,ChatGPT-3.5为6个,Gemini为5个。
  • R代码运行时错误数:ChatGPT-4o为6/7,ChatGPT-3.5为5/7,Gemini为5/7。

局限性

  • LLM需要用户完善和验证代码。
  • 存在潜在错误的风险。
  • 存在幻觉的风险。
  • 需注意数据隐私法规。

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

英文原文(摘要)· 点击展开

INTRODUCTION: Original research in radiology often involves handling large datasets, data manipulation, statistical tests, and coding. Recent studies show that large language models (LLMs) can solve bioinformatics tasks, suggesting their potential in radiology research. This study evaluates an LLM's ability to provide statistical and deep learning solutions and code for radiology research.

MATERIALS AND METHODS: We used web-based chat interfaces available for ChatGPT-4o, ChatGPT-3.5, and Google Gemini. EXPERIMENT 1: BIOSTATISTICS AND DATA VISUALIZATION: We assessed each LLMs' ability to suggest biostatistical tests and generate R code for the same using a Cancer Imaging Archive dataset. Prompts were based on statistical analyses from a peer-reviewed manuscript. The generated code was tested in R Studio for correctness, runtime errors and the ability to generate the requested visualization. EXPERIMENT 2: DEEP LEARNING: We used the RSNA-STR Pneumonia Detection Challenge dataset to evaluate ChatGPT-4o and Gemini's ability to generate Python code for transformer-based image classification models (Vision Transformer ViT-B/16). The generated code was tested in a Jupiter Notebook for functionality and run time errors.

RESULTS: Out of the 8 statistical questions posed, correct statistical answers were suggested for 7 (ChatGPT-4o), 6 (ChatGPT-3.5), and 5 (Gemini) scenarios. The R code output by ChatGPT-4o had fewer runtime errors (6 out of the 7 total codes provided) compared to ChatGPT-3.5 (5/7) and Gemini (5/7). Both ChatGPT4o and Gemini were able to generate visualization requested with a few run time errors. Iteratively copying runtime errors from the code generated by ChatGPT4o into the chat helped resolve them. Gemini initially hallucinated during code generation but was able to provide accurate code on restarting the experiment. ChatGPT4-o and Gemini successfully generated initial Python code for deep learning tasks. Errors encountered during implementation were resolved through iterations using the chat interface, demonstrating LLM utility in providing baseline code for further code refinement and resolving run time errors.

CONCLUSION: LLMs can assist in coding tasks for radiology research, providing initial code for data visualization, statistical tests, and deep learning models helping researchers with foundational biostatistical knowledge. While LLM can offer a useful starting point, they require users to refine and validate the code and caution is necessary due to potential errors, the risk of hallucinations and data privacy regulations.

SUMMARY STATEMENT: LLMs can help with coding and statistical problems in radiology research. This can help primary authors trouble shoot coding needed in radiology research.

来源:Acad Radiol,查看原文。版权归原作者所有。

引用

  1. Ghosh A, Li H, Trout AT, et al. Large Language Models can Help with Biostatistics and Coding Needed in Radiology Research. Acad Radiol. 2024. PMID: 39406582. doi:10.1016/j.acra.2024.09.042

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。