Off-The-Shelf Software: Best Practices, FAQs, and Examples

Questions 🔗

Basics 🔗

What is OTS Software? 🔗

The FDA defines OTS Software as

a generally available software component, used by a medical device manufacturer for which the manufacturer cannot claim complete software life cycle control

in the 2019 FDA Guidance titled “Off-The-Shelf Software Use in Medical Devices”.

What are some examples of OTS Software? 🔗

Common examples of OTS Software include:

Third-party open-source packages (e.g., pytorch or React)
The operating system (e.g., Windows, macOS, or Linux)
Web browsers
Hardware drivers
Cloud services (e.g., AWS S3)
Other applications required for your device (e.g., a virus checker)

Is it okay if we include OTS Software in our medical device? 🔗

Usually. As is generally the case for most FDA regulations, the higher the risk of using it, the more verification and documentation you need. Thus, if a patient could be killed if the OTS Software failed, then you will need a lot of documentation for the OTS Software. If that documentation and control isn’t possible, it may simply not be appropriate to use it with the device. (See here for some more details on this point.)

That may sounds scary, but if your software doesn’t directly touch patients, chances are you’ll be okay using most OTS Software in your device.

Is it reasonable to require our software developers not to use open-source software in our device? 🔗

In most cases, no. Many open-source tools are widely used, thoroughly tested, and are higher quality than anything your engineers will be able to create within your budget. Requiring your engineers to not use open-source software would, in most cases, make your device very expensive to develop. This is especially true for many Software as a Medical Device (SaMD) products.

If your device is high risk, it may be reasonable to severely restrict the use of OTS Software in some cases.

Why is open-source software packages OTS Software? 🔗

They are “generally available software components” and you generally “cannot claim complete software life cycle control” over how the OTS software is developed.

Why are cloud services (e.g., AWS S3) considered OTS Software? 🔗

These services contain software and are “generally available” and you “cannot claim complete software life cycle control” over them.

(The FDA specifically discusses LAN hardware/software in the guidance appendix, which is a hardware/software component they consider OTS Software.)

What is the relationship between an SBOM and OTS Software? 🔗

An SBOM, or Software Bill of Materials, is a comprehensive list of components, libraries, and dependencies used in a software system. SBOMs are usually stored in a machine-readable file format and are generated using software tools.

SBOMs are meant primarily for keeping up with cybersecurity vulnerabilities and licensing issues.

Thus, your SBOM should contain a list of all the information about the OTS software in your device and certain metadata about them. The OTS Software documentation required by the FDA goes well beyond what is in an SBOM.

If you want to know more about SBOMs, see our in-depth article, “SBOMs: Best Practices, FAQs, and Examples”.

What is the relationship between SOUP and OTS Software? 🔗

SOUP and OTS Software can often be used interchangeably, but not always.

Software that was developed by a third-party for the purpose of being used in a medical device, may be OTS Software while not being SOUP.

Software that was developed by your company, for which you don’t have adequate records of its creation, may be SOUP while not being OTS Software.

The term SOUP comes from the IEC 62304 “Medical device software – Software life cycle processes” standard. Here is how it defines the term:

software of unknown provenance (SOUP)
SOFTWARE ITEM that is already developed and generally available and that has not been
developed for the purpose of being incorporated into the MEDICAL DEVICE (also known as “off-the-shelf software”) or software previously developed for which adequate records of the development PROCESSES are not available

You can contrast this with the definition of OTS Software above:

a generally available software component, used by a medical device manufacturer for which the manufacturer cannot claim complete software life cycle control

(As an aside, the FDA also used the term SOUP in their now-outdated 2005 software guidance, but they removed the reference in the 2023 guidance.)

Selecting OTS Documentation 🔗

How should we evaluate OTS Software for our medical device? 🔗

The higher-risk the software, the more careful and particular you need to be when selecting OTS Software for your device.

Open-source software can be very robust if it’s widely used and supported. We recommend evaluating the number of GitHub stars, downloads, commits, fixed issues, and so on when evaluating an open-source package. Consider the number of active contributors, as this also will determine how likely it is that bug-fixes will be made available. You don’t want to be one of the last companies using a piece of open-source software five years from now, especially if it is difficult to migrate to an alternative.

Is it okay to use beta software or libraries with 0.y.z versions? 🔗

In general, we recommend that you do not use beta versions of OTS Software, as they’re typically not appropriate for medical devices.

Similarly, OTS Software with versions starting with the number “0” is generally not appropriate for use in a medical device. Many open-source libraries follow the semver standard. According to this standard:

Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

If you use it, you should be able to justify why it is appropriate in your particular case. E.g., occasionally open-source libraries are widely used and even “industry standard” with a 0.y.z major version.

FDA Documentation 🔗

How do we document OTS Software for the FDA? 🔗

The basic process goes like this:

Identify all of the OTS software in your device
Analyze the risks associated with each OTS software component failing
Based on the level of risk, mitigate, verify, and document the OTS software

Here is an overview of what the FDA requires, increasing from low risk on the level to the highest risk on the right:

Minor Level of Concern before mitigations	Minor Level of Concern after mitigations	Moderate Level of Concern	Major Level of Concern after mitigations
Hazard Analysis	Hazard Analysis	Hazard Analysis	Hazard Analysis
Basic Documentation	Basic Documentation	Basic Documentation	Basic Documentation
	Hazard Mitigations	Hazard Mitigations	Hazard Mitigations
		Describe and Justify Residual Risk	Describe and Justify Residual Risk
			Special Documentation

How do you to an OTS Software Hazard Analysis? 🔗

We suggest the following your normal risk management process. Thus, the Hazard Analysis, Hazard Mitigations, and the Description and Justification of Residual Risks documentation would be documented alongside your normal software risks.

Note that when doing a hazard analysis, the FDA requires that the focus be on the severity of harm and not the likelihood. See the following comment from the guidance:

Because the risk estimates for hazards related to software cannot easily be estimated based on software failure rates, CDRH has concluded that engineering risk management for medical device software should focus on the severity of the harm that could result from the software failure.

We suggest noting the determined OTS level of concern for each OTS software component to keep a record that the analysis was in fact completed. This is because sometimes there are few to no risks originating in the OTS software.

What is involved with the “Basic Documentation”? 🔗

The basic documentation is required for all OTS software. It includes the following, taken straight from the guidance. There’s a lot here, but for a lot of OTS software many of these sections won’t apply. See the Examples section at the bottom of this article.

What is it?
- The title and manufacturer of the OTS software.
- Version, release date, patch number, etc. as appropriate.
- What OTS Software documentation will be provided to the end user?
- Why is this OTS Software appropriate for this medical device?
- What are the expected design limitations of the OTS Software?
What are the Computer System Specifications for the OTS Software? For what configuration will the OTS Software be validated?
- Hardware specifications: processor (manufacturer, speed, and features), RAM (memory size), hard disk size, other storage, communications, display, etc.
- Software specifications: operating system, drivers, utilities, etc. The software requirements specification (SRS) listing for each item should contain the name (e.g., Windows 10, Excel, Sun OS, etc.), specific version levels (e.g., 4.1, 5.0, etc.) and a complete list of any patches that have been provided by the OTS Software manufacturer.
How will you assure appropriate actions are taken by the End User?
- What aspects of the OTS Software and system can (and/or must) be installed/configured?
- What steps are permitted (or must be taken) to install and/or configure the product?
- How often will the configuration need to be changed?
- What education and training are suggested or required for the user of the OTS Software?
- What measures have been designed into the medical device to prevent the operation of any non-specified OTS Software, e.g., word processors, games? Operation of non-specified OTS Software may be prevented by system design, preventive measures, or labeling. Introduction may be prevented by disabling input (USB, CD, modems).
What is the OTS Software intended to do?
- What function does the OTS Software provide in this device?
- The sponsor’s design documentation should specify exactly which OTS components will be included in the design of the medical device and to what extent OTS Software is involved in error control and messaging in device error control.
- What are the links with other software including software outside the medical device (not reviewed as part of this or another application)? The links to outside software should be completely defined for each medical device/module. The design documentation should include a complete description of the linkage between the medical device software and any outside software (e.g., networks).
How do you know it works?
- Describe testing, verification, and validation of the OTS Software and ensure it is appropriate for the device hazards associated with the OTS Software. FDA recommends that software test, verification, and validation plans identify the exact OTS Software (title and version) that is to be used. When the software is tested it should be integrated and tested using the specific OTS Software that will be delivered to the user.
- Provide the results of the testing.
- Is there a current list of OTS Software problems (bugs) and access to updates?
How will you keep track of (control) the OTS Software?
- What measures have been designed into the medical device to prevent the introduction of incorrect versions? On startup, ideally, the medical device should check to verify that all software is the correct title, version level, and configuration. If the correct software is not loaded, the medical device should warn the operator and shut down to a safe state.
- How will you ensure proper maintenance and life cycle support for the OTS Software?
- How can OTS software failure result in patient harm, what is the estimated severity of this harm, etc.

What do we put for the manufacturer for open-source software? 🔗

We suggest writing “open source software” for the manufacturer.

What do we put for the “Patch Number” or “Upgrade Designation”? 🔗

Within the basic documentation, the FDA asks for the “Version Level, Release Date, Patch Number, and Upgrade Designation, as appropriate”.

We usually only include the version number, e.g. “Numpy 1.2.5”. The release date can be looked up from the version if needed, so we don’t think it is necessary to include it.

Here are some additional details if you’re curious:

We’re not really sure what specifically the FDA means by Version Level, Patch Number, and Upgrade Designation. As best we can tell, here’s what they mean.

A patch is a piece of software designed to fix or improve a computer program or its supporting data. This includes fixing bugs, replacing graphics, and improving performance or usability. Patch numbers typically increase with each patch that is released. For example, if you start with version 1.0, the first patch might be labeled as 1.01, the next as 1.02, and so on.

The “Upgrade Designation” usually refers to more significant changes or improvements to the software that are large enough to warrant a new version number. For example, an upgrade from version 1.0 to 2.0 would likely involve significant changes, new features, or a large number of bug fixes. The designation might include not just the new version number, but also a codename or title for the upgrade. For instance, Apple names its OS X upgrades after big cats (e.g., Lion, Mountain Lion, etc.), and macOS upgrades after California landmarks (e.g., Yosemite, El Capitan, etc.).

A lot of these questions don’t seem to apply. What do we do? 🔗

The OTS Guidance was originally written in 1999. Back then, software was simpler and often didn’t incorporate as many third-party dependencies. When they did, they were often not incorporated directly into the device software. This is why so many of the questions the FDA asks about don’t make sense any more, e.g., when considering third-party packages like numpy (a widely used numerical library used in most AI/ML or image-processing applications written with Python):

FDA: What OTS Software documentation will be provided to the end user?

Software Engineer: I don’t think we would give them the numpy docs… so I guess none.

FDA: What aspects of the OTS Software and system can be installed or configured?

Software Engineer: It’s incorporated into our Docker image. So none.

FDA: What measures have been designed into the medical device to prevent the operation of any non-specified OTS Software, e.g., word processors, games?

Software Engineer: Games?! Our software runs on a server in the cloud that only we control.

FDA: What measures have been designed into the medical device to prevent the introduction of incorrect versions?

Software Engineer: We pin the version in our requirements.txt file. Does that count?

Modern software packages, like numpy, are often low-level utilities that function well below the system requirements, software requirements, or even detailed software requirements. They tend to be widely used and battle tested. In many cases, their functionality will be effectively verified by the system-level verification.

So why does the FDA still ask for all of this information? To be fair, the FDA has a hard job. It’s difficult to write regulations for a fast-changing technology landscape. Furthermore, their questions still do make sense for some types of software running on machines the users control, or for packages that are critical for the device or fulfill the system-level requirements.

See the examples for some ways we’ve typically addressed these.

We use hundreds of third-party packages. What do we do? 🔗

Modern software is often developed with hundreds, or even thousands, of open-source packages. This is especially true for software that is built by engineers new to the medical device field—they don’t know better.

For lower risk devices, many of these packages provide low-level utilities. Here are a couple of strategies to save time on the documentation:

Only provide Basic Documentation for the most important OTS and note that the remaining OTS components are listed in your Software Bill of Materials (SBOM).
Lump your granular dependencies into “buckets” of related OTS Software and provide Basic Documentation for all of them together. For example, we may lump all of our image-processing Python dependencies into one bucket and answer all of the questions for them together. Many of the questions will have similar answers. You can then reference the SBOM for the exact versions and names of the sub-items. See the examples section below for one approach to doing this.
Another strategy that can work in some cases, is to isolate the dependencies used in the core software function from the dependencies used for “Other Functions” (See Multiple Device Functions Impact on FDA Submissions for more details about this.)

OTS Software Verification 🔗

What do we do if we support multiple versions of OTS Software? 🔗

For example, if you support multiple versions of Windows, or if you support macOS and Windows, then some verification testing should occur on each version that is supported. Often we suggest performing some regression testing against all the platforms and a full verification suite against the most commonly used version.

High-Risk OTS Software [WIP] 🔗

This section is a work in progress.

What goes into the Special Documentation? 🔗

What should we do if we can’t get access to the source code for commercial OTS software? 🔗

Can we use OTS Software that is no long receiving security updates? 🔗

Examples 🔗

Detailed Basic OTS Documentation for a Python Library on a Server 🔗

In this example we provide the full documentation for the Numpy library that runs on a server within the manufacturers control. Many of the questions the FDA asks about OTS software don’t apply in this situation.

Note that we take a conservative approach here and answer every detailed question from the FDA guidance. See the following section for a slightly reduced version of the same responses.

Documentation	Example Value	Commentary
Title	numpy
Manufacturer	Open-source software	If you are more conservative, you could list the numpy.org or the numfocus organization.
Version	1.25.0	We also could have said “See the SBOM”. Note that the FDA guidance asks
OTS Level of Concern	Minor	We suggest including this to record that the hazard analysis was performed
OTS Level of Concern Justification	The DEVICE represents no threat of direct harm to the patient. The risk of indirect harm from a misdiagnosis relating to medical device malfunction is small since the worst case is an incorrect measurement that is considered correct and in many cases any such errors would be evident from context. The OTS Software in this medical device thus represents a Minor Level of Concern and should satisfy Basic Documentation.	The language here is taken from the examples in the guidance. DEVICE should be replaced with your device name.
Any OTS Software documentation that will be provided to the end user.	N/A as the end-user doesn’t have access	This is often not applicable the case for many third-party components.
Why is this OTS Software appropriate for this medical device?	The OTS software is widely used, industry standard. Implementing the same functionality from scratch would be more error prone than using this OTS library.	We commonly use this argument for OTS software that is in fact widely used. We’ve even had SOPs for adding new OTS software that requires engineers to evaluate the number of stars a library has in GitHub or the number of downloads it has. Certainly less widely used software is higher risk.
What are the expected design limitations of the OTS Software?	None
What Hardware Specifications are there for the OTS Software to function properly?	N/A as it runs on the cloud environment that’s within our control and uses the same configuration used in validation	This is most commonly an issue for software installed by users on their own platform.
What Software Specifications are there for the OTS Software to function properly?	N/A as it runs on the cloud environment that’s within our control and uses the same configuration used in validation
What aspects of the OTS Software and system can (and/or must) be installed/configured?	None
What steps are permitted (or must be taken) to install and/or configure the product?	None
How often will the configuration need to be changed (by the end-user)?	Never
What education and training are suggested or required for the user of the OTS Software?	None
What measures have been designed into the medical device to prevent the operation of any non-specified OTS Software?	None, as the user doesn’t have control over this OTS software’s runtime environment
What is the OTS Software intended to do?	Provides a high-performance, multi-dimensional array utility to the backend code. Provides a fast-fourier transform.	If you are more conservative, you may include some software requirements for the OTS software in your design controls and then you can mention the requirements here. We usually don’t do this for Class II devices.
What are the links with other software including software outside the medical device?	N/A as this software is integrated directly into the source code of the device.
Describe testing, verification, and validation of the OTS Software	The OTS software was tested indirectly by the system verification. Note that it is not necessary to identify specific OTS versions as they are configured within the overall server build images. All verification was thus performed with the exact OTS versions that will be used in the final device.	If you are more conservative and written software requirements for the OTS software, you can list the verification protocol IDs that verify this OTS software. Alternatively, even if you haven’t written requirements of the OTS software, you can still reference specific relevant verification protocols here.
Provide the results of the testing	See the system-level verification records in DOC-00XY Software Verification Report.
Is there a current list of OTS Software problems (bugs) and access to updates?	Yes, the software is open source and has a public bug tracker and updates are provided by the community or could be provided by the manufacturer if needed. Also, cybersecurity vulnerabilities are identified per DOC-00XY Cybersecurity Management Plan.	If you are more conservative, you could also refer to the web page https://github.com/numpy/numpy/issues. Also note that IEC 62304 requires this information too.
What measures have been designed into the medical device to prevent the introduction of incorrect versions?	The software runs in a cloud environment that’s within our control and uses the same configuration used in validation.
How will you maintain the OTS Software configuration?	In the source code, the specific OTS software versions are pinned. In the runtime environment the python dependencies are bundled into the compiled system image.
Where and how will you store the OTS Software?	The python dependencies are bundled into the compiled system image.
How will you ensure proper installation of the OTS Software?	The python dependencies are bundled into the compiled system image, thus there are no specific installation instructions required to ensure proper installation of the OTS Software.
How will you ensure proper maintenance and life cycle support for the OTS Software?	Bug fixes and maintenance are provided by the community and version updates can be applied in a controlled manner following our software development process (DOC-00ZA), including design controls and configuration management. Cybersecurity vulnerabilities are monitored per DOC-00XY Cybersecurity Management Plan.

Summarized OTS Basic Documentation for a Python Library on a Server 🔗

This is a simplified version of the previous example, where several of the “basic information” questions from the guidance have been collapsed down into their “parent question.” We usually suggest using this approach.

Documentation	Example Value
Title	numpy
Manufacturer	Open-source software
Version	1.25.0
OTS Level of Concern	Minor
OTS Level of Concern Justification	The DEVICE represents no threat of direct harm to the patient. The risk of indirect harm from a misdiagnosis relating to medical device malfunction is small since the worst case is an incorrect measurement that is considered correct and in many cases any such errors would be evident from context. The OTS Software in this medical device thus represents a Minor Level of Concern and should satisfy Basic Documentation.
Any OTS Software documentation that will be provided to the end user.	N/A as the end-user doesn’t have access
Why is this OTS Software appropriate for this medical device?	The OTS software is widely used, industry standard. Implementing the same functionality from scratch would be more error prone than using this OTS library.
What are the expected design limitations of the OTS Software?	None
What are the Computer System Specifications for the OTS Software?	N/A as it runs on the cloud environment that’s within our control and uses the same configuration used in validation
How will you assure appropriate actions are taken by the End User?	N/A as end users don’t need to take any actions
What does the OTS Software do?	Provides a high-performance, multi-dimensional array utility to the backend code. Provides a fast-fourier transform.
How do you know it works?	The OTS software was tested indirectly by the system verification. Note that it is not necessary to identify specific OTS versions as they are configured within the overall server build images. All verification was thus performed with the exact OTS versions that will be used in the final device. See the system-level verification records in DOC-00XY Software Verification Report.
How will you keep track of (control) the OTS Software?	The software runs in a cloud environment that’s within our control and uses the same configuration used in validation. In the source code, the specific OTS software versions are pinned. In the runtime environment the python dependencies are bundled into the compiled system image.

Detailed Basic OTS Documentation for Windows OS on Device 🔗

In this example we provide the full documentation for the Windows OS that runs on hardware device within the manufacturers control. Many of the questions the FDA asks about OTS software don’t apply in this situation.