FDA to review AI mental health chatbots over safety concerns, unpredictability

The Food and Drug Administration will convene an expert advisory committee on November 6 to address regulatory challenges for AI-powered mental health devices, as concerns mount over unpredictable chatbot outputs from large language models. The move signals the agency may soon implement stricter oversight of digital mental health tools that use generative artificial intelligence.

Why this matters: The FDA’s focus on AI mental health devices comes as more companies release chatbots powered by large language models, whose unpredictable responses could pose safety risks to vulnerable patients seeking mental health support.

What you should know: The Digital Health Advisory Committee (DHAC) meeting will specifically examine “Generative Artificial Intelligence-Enabled Digital Mental Health Medical Devices.”

• The FDA published the meeting notice in the Federal Register on Thursday, highlighting that newly released mental health products using AI pose “novel risks.”
• According to the agency, “as mental health devices continue to evolve in complexity, regulatory approaches ideally will also evolve to accommodate these novel challenges.”

The big picture: This regulatory attention reflects growing industry concerns about the reliability and safety of AI-powered mental health interventions, particularly as chatbots become more sophisticated but remain fundamentally unpredictable in their responses.

What’s next: The November meeting could lead to new regulatory frameworks specifically designed to address the unique challenges posed by generative AI in mental health applications, potentially affecting how companies develop and deploy these tools.