Photo: Cavan Images/Raffi Maghdessian/Getty Images
The Food and Drug Administration is requesting public comment to get feedback on current, practical approaches to measuring and evaluating the performance of AI-enabled medical devices in the real world.
This includes strategies for identifying and managing performance drift, such as detecting changes in input and output. Comments are due by Dec. 1.
The FDA said its request does not represent draft or final guidance. It is not intended to propose or implement policy changes regarding the evaluation of devices that integrate artificial intelligence, including generative AI-enabled technology.
It is also not intended to communicate the FDA's proposed regulatory expectations but is instead meant to seek early feedback from groups and individuals to advance a broader discussion among the AI healthcare ecosystem on this topic.
WHY THIS MATTERS
AI, including genAI, presents opportunities to improve patient outcomes, advance public health and accelerate medical innovation, the FDA said.
At the same time, these technologies introduce concerns regarding safety and effectiveness across the total product life cycle. Considerations include assessing these devices for their performance, safety and reliability after deployment in real-world settings.
The FDA is seeking information on best practices, methodologies and approaches for measuring and evaluating real-world performance of AI-enabled medical devices. This includes approaches to detect, assess and mitigate performance changes over time to help assure these medical devices remain safe and effective throughout their life cycle.
AI system performance can be influenced by changes in clinical practice, patient demographics, data inputs and healthcare infrastructure, among other factors. This data drift may lead to performance degradation, bias or reduced reliability, the FDA said.
User behavior, workflow integration and changes to clinical guidelines are also factors.
THE LARGER TREND
Many AI-enabled medical devices are evaluated primarily through retrospective testing or static benchmarks.
The request for public comment follows discussions during the November 2024 meeting of the FDA Digital Health Advisory Committee.
As of July, more than 1,000 AI-enabled medical devices had been authorized by the FDA.
Email the writer: SMorse@himss.org