The U.S. Food and Drug Administration (FDA) is continuing its effort to provide industries with updated guidance for digital health technologies. On April 3, 2023, the FDA released draft guidance titled “Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions,” requesting comments from stakeholders within 90 days of publication. This draft guidance follows a final guidance issued March 30, 2023, addressing the Section 3305 of the Consolidated Appropriations Act, 2023 amendment to the Food, Drug and Cosmetic Act (FD&C Act): “Cybersecurity in Medical Devices: Refuse to Accept Policy for Cyber Devices Under Section 524B of the FD&C Act.”
“Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions”
ML-enabled technologies have the power to transform patient care through the derivation of new insights from vast amounts of healthcare data. The FDA seeks to strike a balance between the use of powerful ML technologies and patient safety. Specifically, the FDA hopes this guidance will provide the least burdensome approach to support iterative improvement through modifications to machine learning-enabled device software functions (ML-DSF). The guidance applies to automatic or manual modifications of ML-DSFs that would normally require a premarket approval supplement, a de novo submission, or a new premarket notification (510[k]).
The FDA proposes the use of a Predetermined Change of Control Plan (PCCP) to be used in marketing submissions for ML-DSFs to streamline the implementation of modifications after the device has been authorized for commercialization. The FDA first raised the idea of a PCCP in this process in its 2019 discussion paper: “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD).” Section 3308 of the Food and Drug Omnibus Reform Act of 2023 grants the FDA express authority to approve PCCPs for devices requiring premarket approval or notification. Manufacturers can avoid the cumbersome process of providing additional submissions for each modification by using a PCCP at the initial marketing application phase.
A PCCP should be a standalone section within the marketing submission that includes three components: (1) a “Description of Modifications;” (2) a “Modification Protocol;” and (3) an “Impact Assessment.” We describe each below.
The Description of Modifications (DOM)
The DOM should include the list of individual proposed device modifications and the rationale for each planned change to the ML-DSF. Because the FDA will use the DOM to define the parameters of FDA-authorized specifications, the description should have detail sufficient for the FDA to perform this assessment. The FDA suggests describing how the modifications will be implemented (e.g.,automatically or manually); if the modifications will be implemented uniformly across all devices on the market; and how the modifications are intended to maintain or improve device effectiveness.
The Modification Protocol (MP)
The MP should serve as a step-by-step description of how the modifications in the DOM will be implemented, while also ensuring that the device will remain safe and effective for its intended use. Each modification requires a corresponding MP that includes the verification and validation activities that will be used for the modification. The MP also should explain how risks identified in the Impact Assessment will be mitigated through the MP. The FDA recommends that each planned modification accompany an MP with the following four subsections:
1) Data management practices
This section should describe how new data will be collected, curated, stored, retained, controlled, and used by the manufacturer for each modification. It should also clarify the relationship between the MP data and the data used to train and test initial and subsequent versions of the ML-DSF. Manufacturers should describe the process used to determine the reference standard, the quality assurance process, data sequestration strategies, and the process used to prevent access to data intended for performance testing during training/tuning.
2) Re-training practices
This section should describe the processing steps that are subject to change and how the change will be implemented. Manufacturers should define the objective of the re-training process, provide a description of the ML mode, specify the device components that may be modified, outline practices that will be followed, and identify any triggers for re-training.
3) Performance evaluation protocols
This section should include plans for the verification and validation of each individual modification, as well as verification and validation of the aggregate. Manufacturers should describe the following: how performance evaluation will be triggered, how sequestered test data will be applied for testing, which performance metrics will be computed, and which statistical analysis plans will be employed to test performance objectives.
4) Update procedures
This section should include descriptions of implementation and user transparency for each modification. Specifically, manufacturers should include confirmation that the verification and validation plans for the modified version are those that were performed for the version of the device prior to the implementation of the modification. Manufacturers should also describe how software updates will be implemented, how legacy users will be impacted by software updates, and how modifications will be communicated to users.
The Impact Assessment (IA)
The FDA will use the IA to determine if the proposed modifications are likely to introduce unmitigated risks. Manufacturers should include the following in the IA: a comparison of the devices before and after modification; a discussion of the benefits and risks of each modification; an explanation of how the proposed activities will continue to reasonably ensure that the device remains safe and effective; and a discussion of the impact of modifications on one another along with the collective impact of all the modifications.
Once a PCCP has been reviewed and established through a marketing submission, the PCCP is considered part of the marketing authorization.
Next Steps for Manufacturers
On April 13, 2023, the FDA will host a webinar for industry stakeholders to learn more about the draft guidance. Comments on the draft guidance are due by July 3, 2023.
“Cybersecurity in Medical Devices: Refuse to Accept Policy for Cyber Devices Under Section 524B of the FD&C Act”
On March 30, 2023, FDA issued a guidance reflecting the Agency’s implementation of section 524B of the Federal Food, Drug, and Cosmetic Act (FD&C Act) “Ensuring Cybersecurity of Medical Devices.” The new section, added in the December 29, 2022, Omnibus budget, highlights the need for mitigation of cybersecurity risks in medical devices.
The guidance applies to devices that qualify as a “cyber device,” which is defined under the guidance as a device that:
- includes software validated, installed, or authorized by the sponsor as a device or in a device;
- has the ability to connect to the internet; and
- contains any technological characteristics validated, installed, or authorized by the sponsor that could be vulnerable to cybersecurity threats.
The FDA recommends that any application or submission for a device that falls within this definition include:
- a plan to monitor, identify, and address, as appropriate, in a reasonable time, postmarket cybersecurity vulnerabilities and exploits, including coordinated vulnerability disclosure and related procedures; and
- a software bill of materials and a plan to monitor, identify, and address postmarket cybersecurity vulnerabilities and exploits.
The FDA also recommends that sponsors have in place processes and procedures to provide assurance that the device and any related systems are secure including cyclical updates and patches for unacceptable vulnerabilities and prompt updates and patches for critical vulnerabilities.
The FDA signaled that for premarket submissions submitted for cyber devices before October 1, 2023, the agency will not issue “refuse to accept” (RTA) decisions based solely on information required by section 524B of the FD&C Act, but will “work collaboratively” with sponsors during the review process. After October 1, 2023, the FDA expects that sponsors of cyber devices will prepare premarket submissions that contain the information required by section 524B of the FD&C Act, and the FDA may RTA premarket submissions that do not.
Next Steps for Manufacturers
Manufacturers of cyber devices will need to demonstrate that they prioritize cybersecurity in their product development as part of their submission. Manufacturers will need to provide the FDA with clear processes and procedures to provide assurance that the cyber device and any related systems are monitored and have the ability to be updated and patched in order to keep the cyber device secure.
