Introduction 🔗
I talk with a lot of companies who want to turn clinical-research or general-wellness software into a medical device. I also talk with a lot of startups who’ve spun out of university incubators. They often have existing software or AI/ML models that they want to use in a medical device product.
They want to know: Do we have to throw away our existing code?
It depends.
In this article I’ll briefly review the regulatory context before discussing some of the major factors that affect when and how to use existing code.
FDA Cares How the Sausage is Made 🌭🤔 🔗
The FDA cares about the software development process, not just the final product. Said another way: They don’t just care about the sausage, they want to know how the sausage was made.
In the software validation guidance, FDA says: “Because of its complexity, the development process for software should be even more tightly controlled than for hardware, in order to prevent problems that cannot be easily detected later in the development process.”
That said, FDA also recognizes that a controlled development process isn’t appropriate when you’re in the research or concept phase. The following diagram is adapted from an FDA training slide on Design Controls. It shows how the transition from the research to the development phase is blurry and can be gradual.
Off-The-Shelf Software 🛍️ 🔗
Another angle to think about prototype software is to compare it to Off-The-Shelf (OTS) Software. OTS Software is a generally available software component that you didn’t develop. FDA allows developers to use Off-The-Shelf (OTS) Software, but you must validate it for its intended use, analyze the risk with using it, and so on.
The use of OTS Software in a medical device allows the manufacturer to concentrate on the application software needed to run device-specific functions. However, OTS Software intended for general-purpose computing may not be appropriate for a given specific use in a medical device. The medical device manufacturer using OTS Software generally gives up software life cycle control, but still bears the responsibility for the continued safe and effective performance of the medical device.
Like OTS Software, it won’t be appropriate in all cases based on the safety risks involved with using software that was developed without full life cycle controls.
Multi-Device Functions 🔗
Another factor to consider is Multi-Device Functions. The FDA is not allowed to regulate software functionality that isn’t for the medical device itself. For example, if you may consider your billing and payments module to be an “other functions”. There are some restrictions. First, you need to ensure this software is architecturally separated from the medical-device functions. Second, you still need to consider cybersecurity for these other functions.
When deciding if you’ll throw away your existing code, consider whether any useful functions can be split into separate modules that don’t need to be developed under full life-cycle controls.
In many cases we suggest splitting these modules into separate Git repositories to help ensure the interface between them and the core medical-device code is clearly defined and enforced.
The following diagram shows an example System and Software Architecture Diagram with several “other function” and OTS Software components. With SaMD, regulatory strategy and software architecture overlap when deciding how to define the boundaries of the medical device.
Factors Affecting a Rewrite 🔗
Now with the regulatory background out of the way, we can ask again:
Do we have to throw away our existing code?
It depends on the following factors:
- Safety Risk
- We wouldn’t use most OTS software in a Class III device. If we did, we would perform very thorough needs analysis, risk management, and validation. You should treat existing prototype code similarly.
- For most Class II SaMD, it is worth considering using the existing code.
- Code Quality and Maintainability
- There is a huge range of software quality, and the quality of the code may not be apparent from a pretty user interface. (You can read more about code quality in my other article here.)
- Did engineers on your team develop the code, or was it developed by a third party who’s no longer available?
- How well tested is the code (both unit tests and end-to-end tests)?
- Do the engineers on your team feel they can support it?
- How long has the code been used? (Code quality can improve with age as bugs are found and fixed—Things You Should Never Do is a classic on this topic.)
- How much code is there? Is it 10,000 LOC or 100,000 LOC?
- Requirements Changes
- How similar are the use cases that the existing code solves as compared to the final medical device?
- We often see that prototype code is focused on gathering data and trying many different algorithms for feasibility. Similarly, the UI that is used for this can be quite different from the final, clinical UI.
- If a lot of your old functionality is unnecessary, it may be easier to re-write the parts that are useful than to pull out the bits that aren’t.
- Architectural Decisions
- Some architectural design choices are very difficult to change without a full re-write.
- If you’re migrating platforms or programming languages anyway, it may be better to throw away your existing code. Note all languages are the same.
- A lot of prototype code is written in MATLAB or LabView. While it is possible to create and FDA clear medical device software with these platforms, they’re generally less appropriate for medical devices.
- AI/ML Considerations
- It is critical to keep training datasets separate from test datasets.
- It is also important not to use the test dataset too many times (or you risk over fitting). To avoid these issues, it is necessary to have robust data controls. Ideally, the developers won’t even have access to the test dataset and each time it is used you’ll log the use in a standardized format.
It is often feasible to extract a core algorithm or functionality from the prototype code, transfer it to the new platform, and move on using more robust design controls.
Code Rewrite Stories 🏰 🔗
Here are a few concrete re-write stories I’ve seen:
One client had outsourced development of some prototype software code. It was written in C# using Xamarin, and it included a significant amount of functionality that wasn’t required in the final device. When we started the project, the client felt they wanted to continue using the existing code—partly because they’d invested a significant amount of money in the prototype. After working on the project for a month, struggling to re-purpose the existing code, we convinced the client to do a full rewrite. We were able to rewrite the software using React Native and Typescript in a fraction of the time it had taken to develop the original code. The new code was architected to support automated tests from the outset and was much better suited for its purpose.
Another client had a sophisticated image-processing algorithm built using MATLAB. They wanted their engineers to continue supporting development of the model. Once we understood their requirements, we decided the best path forward was to use their existing code. We worked with them to remove all of the unnecessary functionality and to add automated tests. We then built a user interface around the existing functionality.
On another project, we had a very complex, high-performance signal-processing algorithm that had been written in obtuse C by an academic developer who wasn’t on our team. The existing team didn’t feel they could safely maintain the software, and so we successfully re-wrote the software using Rust.
Conclusion 🔗
The decision as to whether to use your code in your upcoming medical-device product is multi-faceted. It depends on the risk-level of your device, design changes, and more.
When making decisions like this, it is useful to have regulatory consultants with deep software expertise. We can help you find the middle way and save time and money. Our engineering team can also help re-write your code or re-purpose and document it for safe use in a medical device.
