大型语言模型与人类对齐:ChatGPT药学能力综合调查
本研究构建了包含药物信息检索、先导化合物结构优化和研究趋势总结分析三项任务的药理学测试集,比较通用大语言模型GPT-3.5和GPT-4的表现。结果显示两者在基础药理学任务上展现显著潜力,但在药物识别查询、药物相互作用信息检索和药物结构模拟优化等任务上存在明显局限,建议探索检索增强生成或整合专有知识库与知识图谱。
要点
- GPT-3.5和GPT-4在药物药理学性质、药代动力学、作用机制和毒性预测等基础任务上表现良好。
- ChatGPT难以提供个体或特定药物的准确相互作用信息,且无法优化特定药物结构。
- 整合检索增强生成或专有知识库与知识图谱可优化大语言模型在药理学中的应用潜力。
核心结论
- GPT-3.5和GPT-4在基础药理学任务中展现显著潜力,但在药物识别查询、药物相互作用信息检索和药物结构模拟优化等任务上存在明显局限。
研究设计与样本
- 构建了包含药物信息检索、先导化合物结构优化和研究趋势总结分析三项任务的药理学测试集,并比较通用大语言模型GPT-3.5和GPT-4在该测试集上的表现。
主要发现
- GPT-3.5和GPT-4能更好地理解药理学中的信息检索、方案优化和趋势总结指令,在药物药理学性质、药代动力学、作用机制和毒性预测等领域展现显著潜力。
- 这些通用大语言模型能有效总结该领域当前挑战和未来趋势,是跨学科药学研究者的宝贵资源。
- ChatGPT在处理药物识别查询、药物相互作用信息检索和药物结构模拟优化等任务时局限性明显,难以提供个体或特定药物的准确相互作用信息,且无法优化特定药物。
临床怎么用
- 在基础药理学信息检索、药理学性质与药代动力学查询、毒性预测及研究趋势总结中可借助GPT-3.5和GPT-4。
- 涉及药物识别、药物相互作用信息检索和药物结构优化时,需谨慎对待ChatGPT输出,不宜单独依赖。
局限性
- ChatGPT在知识整合与分析深度方面不足,限制了其在科学研究和临床探索中的应用。
以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。
原文
BACKGROUND: Due to the lack of a comprehensive pharmacology test set, evaluating the potential and value of large language models (LLMs) in pharmacology is complex and challenging. AIMS: This study aims to provide a test set reference for assessing the application potential of both general-purpose and specialized LLMs in pharmacology. METHODS: We constructed a pharmacology test set consisting of three tasks: drug information retrieval, lead compound structure optimization, and research trend summarization and analysis. Subsequently, we compared the performance of general-purpose LLMs GPT-3.5 and GPT-4 on this test set. RESULTS: The results indicate that GPT-3.5 and GPT-4 can better understand instructions for information retrieval, scheme optimization, and trend summarization in pharmacology, showing significant potential in basic pharmacology tasks, especially in areas such as drug pharmacological properties, pharmacokinetics, mode of action, and toxicity prediction. These general LLMs also effectively summarize the current challenges and future trends in this field, proving their valuable resource for interdisciplinary pharmacology researchers. However, the limitations of ChatGPT become evident when handling tasks such as drug identification queries, drug interaction information retrieval, and drug structure simulation optimization. It struggles to provide accurate interaction information for individual or specific drugs and cannot optimize specific drugs. This lack of depth in knowledge integration and analysis limits its application in scientific research and clinical exploration. CONCLUSION: Therefore, exploring retrieval-augmented generation (RAG) or integrating proprietary knowledge bases and knowledge graphs into pharmacology-oriented ChatGPT systems would yield favorable results. This integration will further optimize the potential of LLMs in pharmacology.
引用
- Zhang Y, Ren S, Wang J, et al. Aligning Large Language Models with Humans: A Comprehensive Survey of ChatGPT's Aptitude in Pharmacology. Drugs. 2024. PMID: 39702867. doi:10.1007/s40265-024-02124-2
供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。