The FDA’s AI Chatbot for Drug Approval Isn’t Providing Assistance
The Food and Drug Administration’s new AI tool, referred to as Elsa, has been introduced to assist FDA employees with daily tasks such as meeting notes and emails, while also aiming to enhance the efficiency of drug and device approvals by analyzing vital application data. Though initially hailed as a revolutionary advancement by Secretary of Health and Human Services Robert F. Kennedy, Jr., early insights reveal that Elsa is experiencing more challenges than anticipated. According to FDA insiders who spoke to CNN under anonymity, the chatbot tends to produce inaccuracies, often fabricating medical studies or misinterpreting key data.
Although intended to support staff, it appears that some employees have set the tool aside due to its limitations. Sources indicate that Elsa is not equipped to be utilized in review processes and lacks access to essential internal documents promised to the team.
“It hallucinates confidently,” one FDA employee shared with CNN. The tool frequently provides incorrect responses on the FDA’s research areas and drug labels, and it struggles to reference third-party citations from external medical journals.
Despite earlier claims of its integration into clinical review protocols, FDA Commissioner Marty Makary clarified that the tool is currently being used solely for organizational tasks and is not mandatory for employees. The head of AI at the FDA acknowledged the chatbot’s tendency to produce inaccuracies, highlighting that further testing and training will be essential.
It’s important to note that not all language models have the same mission of approving life-saving treatments. The agency first announced this tool back in June. Vinay Prasad, director of the FDA’s Center for Biologics Evaluation and Research, along with Makary, expressed in a Journal of the American Medical Association article that fostering AI innovation is a significant priority for the agency. The tool was proposed as a potential solution to the lengthy and often scrutinized drug approval processes, following the launch of an AI-assisted scientific review pilot.
The administration has been promoting an accelerated “America-first” AI agenda, which includes recent federal guidance to create FDA-supported AI Centers of Excellence aimed at testing and implementing new AI technologies. Many individuals express concern that this push, along with deregulation efforts, may overlook essential oversight of these emerging technologies. The AI Action Plan notes that “Many of America’s most critical sectors, such as healthcare, are especially slow to adopt due to a variety of factors, including distrust or lack of understanding of the technology, a complex regulatory landscape, and a lack of clear governance and risk mitigation standards.” It emphasizes that a coordinated federal effort would be valuable in creating a dynamic, ‘try-first’ culture for AI across American industry.
