April Fools! 🔗
If you saw my LinkedIn post, it was an April Fools Joke! I hope it made you smile and is a fun start to the week.
FDA did not publish a draft guidance on the topic of generative AI applied to the review of premarket submissions!
We’ve helped several MDMs create and clear AI/ML-enabled devices. None have included generative AI. In fact, as of October 19, 2023, FDA has not cleared ANY devices with generative AI in them.
While the post was a joke, we are excited about the use of generative AI in the creation of medical devices. Of course, GitHub Co-Pilot is fantastic and many of our software engineers use it.
We also believe there are great use cases with regards to quality management, including:
- Acting as a quick reviewer of regulatory documents
- Critiquing the format of requirements (e.g., enforcing the design-input best practices described here)
- Summarizing large sets of reports (e.g., MAUDE reports within your product code)
- Identifying inconsistencies in regulatory documents.
- Identifying possible other documents that may be affected by updates to a particular document.
Of course, LLMs aren’t reliable enough for many more complex tasks.
We’ve published a few other resources about generative AI which you may be interested in. I discuss these below.
What does FDA think about LLMs? 🔗
We had a pre-sub with FDA where we asked them a few key questions:
- Is synthetic data permissible?
- Is fine-tuning permissible?
- Does the algorithm need to be run offline?
- What if the outputs are not fully deterministic?
- What if the provenance of training data is not known?
Sign up for a five-part email series on this topic.
Future of Generative AI in Software Medical Devices 🔗
Checkout this 50-minute video here, which discusses the potential avenues for getting generative AI applications cleared through FDA.
Original Post 🔗
New FDA Draft Guidance discussing the use of LLMs and ChatGPT!
I'm very excited about this guidance and continue to be impressed with FDA's engagement within the evolving AI/ML space. The full title of the draft guidance is "Recommendations and Risk Assessment of Generative (AI/ML)-Enabled Quality Management and Premarket Submission Review"
This guidance outlines the FDA's approach to leveraging the capabilities of large language models (LLMs) like ChatGPT, Llama, Claude, and PaLM to streamline quality management and premarket submission reviews.
Here are my key takeaways from the draft guidance:
1️⃣ Scope: The scope is limited to Premarket Notification (510(k)), Product Development Protocols (PDPs), Investigational Device Exemption (IDE) submissions, Humanitarian Device Exemption (HDE) submissions. It specifically excludes De Novo requests and Premarket Approval Applications (PMAs).
2️⃣ Adoption of Generative AI: Recognizing the widespread use of generative AI tools (e.g., ChatGPT, Llama, Claude, and PaLM), this guidance emphasizes how these technologies can enhance review efficiencies, balancing the need to ensure safe and effective medical innovation with the least-burdensome approach.
3️⃣ Risk-Based Application: The guidance recommends a risk-based approach to the application of generative AI, advocating for broader use in reviewing submissions for commonly used devices.
4️⃣ Generative AI for Quality Systems Improvement: The draft guidance also highlights the potential for medical device manufacturers and quality professionals to employ generative AI in enhancing their quality management systems. It suggests a risk-based approach to validating the use of generative AI which is consistent with the Software Validation Guidance.
5️⃣ Least Burdensome Approach: The guidance identifies generative AI as the least burdensome approach for achieving numerous regulatory and quality management objectives, thereby accelerating the delivery of innovative healthcare solutions to the market while balancing it with the need to ensure devices are safety and effective.
