Preparing for FDA's New Guidance for AI/ML Medical Devices

Predetermined change control plans will support iterative improvements in medical device software, but meeting their requirements means thinking ahead

Faster, smarter, more dynamic: artificial intelligence/machine learning (AI/ML) technology is driving the pace of innovation — as well as regulatory action — for life science firms.

With the ability to process vast amounts of data at speeds unrivaled by humans, AI/ML-enabled medical devices have the potential to add significant value for patients and providers alike by incorporating new data into continuous software updates in real time. In the short term, advances in AI applications could help increase patient access to more advanced technologies and precise diagnoses. In the long term, AI/ML could help accelerate a transition toward a more personalized, data-driven healthcare model that could improve patient outcomes and lower the cost of effective treatments. 

Already, remote patient monitoring that leverages adaptive AI/ML technology is facilitating efficiency gains by reducing in-person visits and identifying opportunities for early intervention. In diagnostics, for example, AI is automating time-consuming, high-volume repetitive tasks, particularly in medical imaging applications, where it meets or exceeds the performance of human experts in image-based diagnoses in several medical specialties. Customized therapies that use AI/ML medical devices to deliver treatments based on individual needs are also on the horizon.

FDA's pending finalized guidance on PCCPs promises to increase the number of AI/ML-enabled medical devices on the market.

As with any complex adaptive technology, patients need protection from pitfalls, such as biased algorithms and data breaches. To promote innovative advances while ensuring the safety and effectiveness of AI/ML devices as they evolve and change throughout the total product lifecycle, the Food and Drug Administration released the 2023 draft guidance "Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions." The agency has stated it will finalize the guidance in 2024.

The draft guidance outlines how device manufacturers can work collaboratively with FDA to specify — and seek marketing authorization for — future intended AI/ML-driven modifications to a medical device. These specified future modifications will be documented in a predetermined change control plan (PCCP) that describes how modifications will be made and assessed. By including a PCCP, premarket submissions for medical devices featuring AI/ML capabilities can present a framework to support safe and effective product updates that would otherwise require additional submissions (e.g., PMA supplement, De Novo submission, or 510(k)). Thus, once deployed, the device will undergo certain established revisions without a new marketing application. 

PCCPs are anticipated to support iterative improvements in medical device software and simplify the regulatory processes for manufacturers. However, designing and aligning various elements of the PCCP present challenges, including those related to strategy, documentation, verification/validation, and continuous monitoring to assure performance. 

Developing your predetermined change control plan

According to FDA, PCCPs can help with:

• Synchronizing regulatory processes with change management in artificial intelligence/machine learning enabled medical devices (MLMDs) 
• Mitigating risks by monitoring or improving device performance 
• Securing compliance with regulatory standards to ensure safety

For example, according to FDA, one appropriate modification to include in a PCCP would be re-training a machine-learning model to incorporate subpopulation data that was previously unavailable, thereby expanding the dataset and improving the device's use and performance. Critically, FDA notes that a PCCP must stay within the device's intended use.

To reap the benefits of a PCCP, fulfilling each of its three components — a description of modifications, a modification protocol, and an impact assessment — demands skillful execution.  

In FDA's draft guidance document, the PCCP description of modifications identifies specific modifications that, conditional on FDA approval of the PCCP, can be implemented without a new marketing submission. This requires that manufacturers develop a comprehensive modification protocol in order to carefully verify, validate, and document software updates in accordance with the manufacturer's quality system. To comply with FDA's PCCP guidance, stakeholders must ensure: 

• Verification: Extensive analyses of device design, including testing, inspections, complete documentation of results, and predetermined acceptance criteria.
• Validation: Robust validation protocols outlining validation activities and predetermined acceptance criteria, including usability studies and postmarket surveillance, to document that the device fulfills intended uses and functionality in the real world.
• Documentation: Comprehensive documentation that meets up-to-date required standards, such as FDA guidance or the EU Medical Device Regulation (MDR), and considers how general software validation, cybersecurity, and interoperability apply to the applications of an AI/ML-enabled medical device. 

Lastly, impact assessment requires a benefit/risk analysis for each specified modification and any necessary risk mitigation measures. This requires in-depth analysis comparing the current PCCP version to the proposed change and evaluating how all the proposed modifications could interact with each other.

Reading between the lines

What's not specified in the draft guidance document can be just as important as what is. The guidance states that "FDA review division will determine whether the scope of the modifications is appropriate for inclusion in a PCCP and what evidence and information are required to support proposed modifications in a marketing submission." Developers need to be mindful of these unique challenges to assure their strategy, testing plans, and documentation will be acceptable to FDA. An appropriate strategy from the outset can limit costly and time-consuming exchanges with the agency to determine what evidence and information is ultimately required.

Noting this is an evolving area and that there are limits to its experience, FDA proposes "to consider PCCPs for ML-DSFs [machine learning-enabled device software functions] where modifications are implemented automatically to the extent the Agency can properly review them for substantial equivalence to the predicate or a reasonable assurance of safety and effectiveness." This will require device developers to be able to essentially prove "substantial equivalence" (which will vary for each AI/ML algorithm) for each automatically implemented modification. 

While the PCCP process allows medical device manufacturers to update their AI/ML-enabled medical devices after their original FDA clearance or approval, the burden lies with manufacturers to not only document the current generation of their device but to plan for changes in future generations. Substantial future changes may not be covered by a PCCP and may require separate regulatory submissions. Furthermore, a PCCP provides for changes in device parameters, not changes in the PCCP itself. Changes in the PCCP are expected to require a new marketing submission.

The AI/ML-enabled medical device list

FDA's pending finalized guidance on PCCPs promises to increase the number of AI/ML-enabled medical devices on the market. To help medical device companies interested in incorporating AI/ML functionality into their products, FDA created a first-of-its-kind public resource, the FDA's AI/ML-enabled Medical Device List. This list gives medical device developers a broad perspective on the current market for AI/ML-enabled devices and helps them understand where these devices are and are not being used.

As of October 2023, FDA's list of approved AI/ML-enabled medical devices included 692 entries. Final decisions on these devices date as far back as 1995, but more than half (371) occurred just since the start of 2021, and so far, none use generative AI or artificial general intelligence (AGI) or are powered by large language models. As these technologies continue to improve, and as more manufacturers of AI/ML medical devices submit successful PCCPs, the number of AI/ML devices is expected to grow quickly.