文献速递医学人工智能与信息学医学教育与科研方法

基于生成式人工智能的健康类应用的健康风险

The health risks of generative AI-based wellness apps.
PubMed 文献 · Nat Med · 2024-04-29

人工智能聊天机器人正被越来越多地用于帮助人们管理心理健康,尤其是「健康类」应用目前处于监管灰色地带,多数生成式人工智能驱动的健康类应用不会经过卫生监管机构审查。本文讨论此类应用的监管格局与潜在健康风险,聚焦美国并指出全球监管者面临类似挑战,提出该领域的待解优先事项。

要点

  • 多数生成式人工智能驱动的健康类应用不会经过卫生监管机构审查。
  • 用户有时用这些应用分享心理健康问题甚至在危机时寻求支持。
  • 应用有时以增加用户伤害风险的方式回应,现行美国监管结构难以应对。

核心结论

  • 生成式人工智能健康类应用存在潜在健康风险,现行美国监管结构难以应对,全球监管者面临类似挑战。

研究设计与样本

  • 本文为观点文章(Perspective),讨论人工智能健康类应用的监管格局与潜在健康风险。

主要发现

  • 多数生成式人工智能驱动的健康类应用不会经过卫生监管机构审查,目前处于监管灰色地带。
  • 近期研究提示,用户有时用这些应用分享心理健康问题甚至在危机时寻求支持,应用有时以增加用户伤害风险的方式回应。

局限性

  • 原文未报告局限性

以上解读依据论文摘要(全文未开放获取或许可不允许翻译转载)。

原文

Artificial intelligence (AI)-enabled chatbots are increasingly being used to help people manage their mental health. Chatbots for mental health and particularly 'wellness' applications currently exist in a regulatory 'gray area'. Indeed, most generative AI-powered wellness apps will not be reviewed by health regulators. However, recent findings suggest that users of these apps sometimes use them to share mental health problems and even to seek support during crises, and that the apps sometimes respond in a manner that increases the risk of harm to the user, a challenge that the current US regulatory structure is not well equipped to address. In this Perspective, we discuss the regulatory landscape and potential health risks of AI-enabled wellness apps. Although we focus on the United States, there are similar challenges for regulators across the globe. We discuss the problems that arise when AI-based wellness apps cross into medical territory and the implications for app developers and regulatory bodies, and we outline outstanding priorities for the field.

来源:Nat Med,查看原文。版权归原作者所有。

引用

  1. De Freitas J, Cohen IG. The health risks of generative AI-based wellness apps. Nat Med. 2024. PMID: 38684859. doi:10.1038/s41591-024-02943-6

供执业医师工作参考,不替代临床判断;数值与分级以指南原文为准。