Off-The-Shelf Software: Best Practices, FAQs, and Examples

Questions 🔗

Basics 🔗

What is OTS Software? 🔗

The FDA defines OTS Software as

a generally available software component, used by a medical device manufacturer for which the manufacturer cannot claim complete software life cycle control

in the 2023 FDA Guidance titled “Off-The-Shelf Software Use in Medical Devices”.

What are examples of OTS Software? 🔗

Common examples of OTS Software include:

• Third-party open-source packages (e.g., Python or Redis)
• The operating system (e.g., Windows, macOS, or Linux)
• Web browsers
• Hardware drivers
• Cloud services (e.g., AWS S3)
• Other applications required for your device (e.g., a virus checker)

Can we use OTS Software in our medical device? 🔗

It depends on

• the risk-level of your device
• how widely used the OTS Software is
• how the OTS Software was developed
• how you’re using the OTS Software in your device.

If a patient could be killed if the OTS Software failed, then you will need to be very particular about which OTS Software you use and will need to provide a lot of documentation, including regarding how it was developed. If that documentation and control isn’t possible, it may simply not be appropriate to use it with the device. (See this section of the OTS Software Guidance for some more details on this point.)

That may sounds scary, but if your software doesn’t directly touch patients, chances are you’ll be okay using most OTS Software in your device.

Is OTS Software less safe than custom-built software? 🔗

It depends.

For high-risk software—software that can kill or seriously injure someone—you must ensure any OTS software was developed using an appropriate methodology. You want to make sure the engineers who built it followed a careful process, had design reviews, and understood the risks of their software failing. If it was developed for other purposes, or if you don’t have visibility into the code or the development process, you likely won’t be able to ensure it is safe enough for the device.

For lower-risk devices, including most SaMD, using OTS Software can make the device safer! How is this the case?

Software teams have limited resources (you can’t argue this to the FDA, but it’s true). If you have limited resources, would you want the engineers to spend them re-creating functionality in a widely-used library? Or would you want them to spend their time analyzing the risk and creating a good architecture for their core functionality? As a software engineer, I know I’d rather that they use OTS software.

(As an aside, this is why most cybersecurity standards and guidance suggest that you use OTS cryptography libraries—they’re safer than home-built ones!)

Many open-source tools are widely used, thoroughly tested, and are higher quality than anything your engineers will be able to create within your budget.

Thus, in many cases, if OTS software can be safer than software you develop yourself. You just need to make sure the OTS software is, in fact, widely used and has been around long enough for people to fix bugs.

We often make this argument when justifying why OTS software is appropriate for medical devices we’re working on.

Why are open-source software considered OTS? 🔗

They are “generally available software components” and you generally “cannot claim complete software life cycle control” over how the OTS software is developed.

See the questions below for strategies for documenting a large number of open-source OTS Software packages.

How is an SBOM related to OTS Software? 🔗

An SBOM, or Software Bill of Materials, is a comprehensive list of components, libraries, and dependencies used in a software system. SBOMs are usually stored in a machine-readable file format and are generated using software tools.

SBOMs are meant primarily for keeping up with cybersecurity vulnerabilities and licensing issues.

Thus, your SBOM should contain a list of all the information about the OTS software in your device and certain metadata about them. The OTS Software documentation required by the FDA goes well beyond what is in an SBOM.

If you want to know more about SBOMs, see our in-depth article, “SBOMs: Best Practices, FAQs, and Examples”.

How are SOUP and OTS Software related? 🔗

In practice, many people do not distinguish between the terms.

SOUP and OTS Software can often be used interchangeably, but not always.

Software that was developed by a third-party for the purpose of being used in a medical device, may be OTS Software while not being SOUP.

Software that was developed by your company, for which you don’t have adequate records of its creation, may be SOUP while not being OTS Software.

The term SOUP comes from the IEC 62304 “Medical device software – Software life cycle processes” standard. Here is how it defines the term:

software of unknown provenance (SOUP)
SOFTWARE ITEM that is already developed and generally available and that has not been developed for the purpose of being incorporated into the MEDICAL DEVICE (also known as “off-the-shelf software”) or software previously developed for which adequate records of the development PROCESSES are not available

You can contrast this with the definition of OTS Software above:

a generally available software component, used by a medical device manufacturer for which the manufacturer cannot claim complete software life cycle control

(As an aside, the FDA also used the term SOUP in their now-outdated 2005 software guidance, but they removed the reference in the 2023 guidance.)

Can we use SOUP in our medical device? 🔗

Yes. However, if you’re planning to declare conformance with the IEC 62304 standard, note that there are some process requirements surrounding SOUP which are not covered by the FDA’s documentation requirements.

Cloud Services 🔗

Why are cloud services (e.g., AWS S3) considered OTS Software? 🔗

These services contain software and are “generally available” and you “cannot claim complete software life cycle control” over them.

(The FDA specifically discusses LAN hardware/software in the guidance appendix, which is a hardware/software component they consider OTS Software.)

Can we use cloud services in our medical device? 🔗

It depends on your risk analysis. The main problem with cloud services is that you, as the manufacturer, can’t usually control changes that are made to the cloud platform. Therefore, you need to analyze risks associated with the cloud platform providers (e.g., AWS, Google, Azure) making frequent changes to the systems you use.

The FDA will expect to see risks, and mitigations, in your risk assessment. If you don’t include this in your risk assessment the FDA will ask you something like this (taken from a real submission):

It is not clear from your submission the potential risks related specifically to Amazon Web Service platform, such as changes to the platform specifically to Amazon Web Service platform [but it could be any other platform], such as changes to the platform that may affect the safety, security, and privacy of your device. Please clarify the detection and mitigation methods for risks introduced by the cloud service.

It is not clear if you anticipate the frequency of updates to the Amazon Web Service platform introducing additional risks for the DEVICE. For example, please explain if you have a monitoring process for promptly detecting an update and verifying whether your system was adversely impacted. Please provide a plan in case an update to this service adversely affects your software.

We suggest linking to the specific risks related to cloud services in your OTS Risk Assessment (see the examples).

After performing your risk assessment, you may realize that it is not acceptable to use cloud services (or particular cloud services) in your device.

How do you assess risks from uncontrolled changes to cloud services? 🔗

There are several pieces of information that may be considered:

• The guarantees in Service Level Agreements for the platform provider
• Backwards compatibility guarantees made by the platform provider for their services
• How new and/or widely used and supported a cloud service is (generally, widely used platforms have less risk of backwards-incompatible changes)
• The frequency and nature of changes to the service

The AAMI Consensus Report, “Appropriate Use of Public Cloud Computing for Quality Systems and Medical Devices AAMI/CR510:2021” has some useful discussion on this point.

How do you mitigate risks from uncontrolled changes to cloud services? 🔗

There are a number of mitigations you can use:

• Run a suite of smoke tests against a staging environment (that’s substantially similar to the production environment). This will allow you to quickly detect when cloud platform changes break your device.
• Have a process for reviewing cloud platform provider’s change logs or development timelines (if available)
• Deploy your software on multiple cloud platforms
• Have a plan for how changes in the platform will be monitored so you can quickly address and/or notify parties of issues that arise

The AAMI Consensus Report, “Appropriate Use of Public Cloud Computing for Quality Systems and Medical Devices AAMI/CR510:2021” has some useful discussion on this point.

Can I use LLMs (e.g., ChatGPT) in my medical device? 🔗

It depends. As of October 2023, FDA hadn’t cleared any devices that include generative AI. Of course there There are the many risks related to the use of generative AI broadly speaking. The first problem of course is they make stuff up. Beyond these risks, there is also the risks related to uncontrolled OTS software. For example, if you’re using OpenAI’s API to integrate with GPT4, you don’t fully control when OpenAI updates their models. E.g., see this blog post about model version support:

We previously communicated to developers that gpt-3.5-turbo-0301, gpt-4-0314 and gpt-4-32k-0314 models were scheduled for sunset on Sept 13, 2023.

While the majority of metrics have improved, there may be some tasks where the performance gets worse. This is why we allow API users to pin the model version. For example, you can use gpt-4-0314instead of the generic gpt-4, which points to the latest model version. Each individually pinned model is stable, meaning that we won’t make changes that impact the outputs…

We understand that model upgrades and behavior changes can be disruptive to your applications. We are working on ways to give developers more stability and visibility into how we release and deprecate models.

As discussed in the previous question, we’d suggest that you need to revalidate new versions of the API as you upgrade the version.

Selecting OTS Documentation 🔗

How do you evaluate using OTS Software in a medical device? 🔗

The higher-risk the software, the more careful and particular you need to be when selecting OTS Software for your device.

Open-source software can be very robust if it’s widely used and supported. We recommend evaluating the number of GitHub stars, downloads, commits, fixed issues, and so on when evaluating an open-source package. Consider the number of active contributors, as this also will determine how likely it is that bug-fixes will be made available. You don’t want to be one of the last companies using a piece of open-source software five years from now, especially if it is difficult to migrate to an alternative.

Can I use beta software or libraries with 0.y.z versions? 🔗

In general, we recommend that you do not use beta versions of OTS Software, as they’re typically not appropriate for medical devices.

Similarly, OTS Software with versions starting with the number “0” is generally not appropriate for use in a medical device. Many open-source libraries follow the semver standard. According to this standard:

Major version zero (0.y.z) is for initial development. Anything MAY change at any time. The public API SHOULD NOT be considered stable.

If you use it, you should be able to justify why it is appropriate in your particular case. E.g., occasionally open-source libraries are widely used, stable, and even “industry standard” with a 0.y.z major version.

FDA Documentation 🔗

How do we document OTS Software for the FDA? 🔗

The basic process goes like this:

1. Determine the Documentation Level of your device (see here)
2. Based on the level of risk, mitigate, verify, and document the OTS software

Here is an overview of what the FDA requires, increasing from low risk on the level to the highest risk on the right:

Software Documentation Elements	Basic Documentation Level (Lower Risk)	Enhanced Documentation Level (Higher Risk)
Description of OTS Software	An overview and description of the OTS software and actions taken for the continued safe and effective use of the medical device.	Same as Basic Documentation Level
Risk Assessment of OTS Software	Risk assessment demonstrating that risks related to the use of OTS software have been appropriately mitigated.	Same as Basic Documentation Level
Software Testing as part of Verification and Validation	Test plans and results for the OTS software, commensurate with the Documentation Level (i.e., Basic or Enhanced) for the device.	Same as Basic Documentation Level
Assurance of Development Methodologies and Continued Maintenance of OTS Software	FDA is not recommending this documentation as part of the premarket submission. Sponsors should document this information via the DHF for the device. During premarket review, FDA may request additional information, if needed, to evaluate the safety and effectiveness of the device.	Information to provide an assurance that the product development methodologies used by the OTS software developer are appropriate and sufficient, and mechanisms exist for assuring the continued performance, maintenance, and support of the OTS software.

What is the “Description of OTS Software”? 🔗

The description of OTS software is required for all OTS software. It includes the following, taken straight from the guidance. There’s a lot here, but for a lot of OTS software many of these sections won’t apply. See the Examples section at the bottom of this article.

• What is it?
  • The title and manufacturer of the OTS software.
  • Version, release date, patch number, etc. as appropriate.
  • What OTS software documentation will be provided to the end user?
  • Why is this OTS software appropriate for this medical device?
  • What are the expected design limitations of the OTS software?
• What are the Computer System Specifications for the OTS Software?
  • For what configuration will the OTS Software be validated?
  • Hardware specifications: processor (manufacturer, speed, and features), RAM (memory size), hard disk size, other storage, communications, display, etc.
  • Software specifications: operating system, drivers, utilities, etc. The software requirements specification (SRS) listing for each item should contain the name (e.g., Windows 10, Excel, Sun OS, etc.), specific version levels (e.g., 4.1, 5.0, etc.) and a complete list of any patches that have been provided by the OTS Software manufacturer.
• How will you assure appropriate actions are taken by the End User?
  • What aspects of the OTS Software and system can (and/or must) be installed/configured?
  • What steps are permitted (or must be taken) to install and/or configure the product?
  • How often will the configuration need to be changed?
  • What education and training are suggested or required for the user of the OTS Software?
  • What measures have been designed into the medical device to prevent the operation of any non-specified OTS Software, e.g., word processors, games? Operation of non-specified OTS Software may be prevented by system design, preventive measures, or labeling. Introduction may be prevented by disabling input (USB, CD, modems).
• What is the OTS Software intended to do?
  • What function does the OTS Software provide in this device?
  • The sponsor’s design documentation should specify exactly which OTS components will be included in the design of the medical device and to what extent OTS Software is involved in error control and messaging in device error control.
  • What are the links with other software including software outside the medical device (not reviewed as part of this or another application)? The links to outside software should be completely defined for each medical device/module. The design documentation should include a complete description of the linkage between the medical device software and any outside software (e.g., networks).
• How do you know it works?
  • Describe testing, verification, and validation of the OTS Software and ensure it is appropriate for the device hazards associated with the OTS Software. FDA recommends that software test, verification, and validation plans identify the exact OTS Software (title and version) that is to be used. When the software is tested it should be integrated and tested using the specific OTS Software that will be delivered to the user.
  • Provide the results of the testing.
  • Is there a current list of OTS Software problems (bugs) and access to updates?
• How will you keep track of (control) the OTS Software?
  • What measures have been designed into the medical device to prevent the introduction of incorrect versions? On startup, ideally, the medical device should check to verify that all software is the correct title, version level, and configuration. If the correct software is not loaded, the medical device should warn the operator and shut down to a safe state.
  • How will you ensure proper maintenance and life cycle support for the OTS Software?
  • How can OTS software failure result in patient harm, what is the estimated severity of this harm, etc.

Who is the “manufacturer” of open-source software? 🔗

We suggest writing “open source software” for the manufacturer. We haven’t yet had the FDA ask us to be more specific.

If you don’t want to take this risk, you could put the organization that manages it, if there is one. Here are a few examples:

Package	Organization
Python	Python Language Foundation
Node.js	OpenJS Foundation
TypeScript	Microsoft, although really TypeScript is a compiler and isn’t typically included in the final software
JavaScript	JavasScript (or ECMA Script) is a standard and there are multiple implementations.

What is the “Patch Number” or “Upgrade Designation”? 🔗

Within the basic documentation, the FDA asks for the “Version Level, Release Date, Patch Number, and Upgrade Designation, as appropriate”.

We usually only include the version number, e.g. “Numpy 1.2.5”. The release date can be looked up from the version if needed, so we don’t think it is necessary to include it.

Here are some additional details if you’re curious:

We’re not really sure what specifically the FDA means by Version Level, Patch Number, and Upgrade Designation. As best we can tell, here’s what they mean.

A patch is a piece of software designed to fix or improve a computer program or its supporting data. This includes fixing bugs, replacing graphics, and improving performance or usability. Patch numbers typically increase with each patch that is released. For example, if you start with version 1.0, the first patch might be labeled as 1.01, the next as 1.02, and so on.

The “Upgrade Designation” usually refers to more significant changes or improvements to the software that are large enough to warrant a new version number. For example, an upgrade from version 1.0 to 2.0 would likely involve significant changes, new features, or a large number of bug fixes. The designation might include not just the new version number, but also a codename or title for the upgrade. For instance, Apple names its OS X upgrades after big cats (e.g., Lion, Mountain Lion, etc.), and macOS upgrades after California landmarks (e.g., Yosemite, El Capitan, etc.).

What do you do if the questions don’t apply? 🔗

The OTS Guidance was originally written in 1999 and some outdated parts haven’t been revised in newer revisions of the guidance. Back then, software was simpler and often didn’t incorporate as many third-party dependencies. When they did, they were often not incorporated directly into the device software. This is why so many of the questions the FDA asks about don’t make sense any more, e.g., when considering third-party packages like numpy (a widely used numerical library used in most AI/ML or image-processing applications written with Python):

FDA: What OTS Software documentation will be provided to the end user?

Software Engineer: I don’t think we would give them the numpy docs… so I guess none.

FDA: What aspects of the OTS Software and system can be installed or configured?

Software Engineer: It’s incorporated into our Docker image. So none.

FDA: What measures have been designed into the medical device to prevent the operation of any non-specified OTS Software, e.g., word processors, games?

Software Engineer: Games?! Our software runs on a server in the cloud that only we control.

FDA: What measures have been designed into the medical device to prevent the introduction of incorrect versions?

Software Engineer: We pin the version in our requirements.txt file. Does that count?

Modern software packages, like numpy, are often low-level utilities that function well below the system requirements, software requirements, or even detailed software requirements. They tend to be widely used and battle tested. In many cases, their functionality will be effectively verified by the system-level verification.

So why does the FDA still ask for all of this information? To be fair, the FDA has a hard job. It’s difficult to write regulations for a fast-changing technology landscape. Furthermore, their questions still do make sense for some types of software running on machines the users control, or for packages that are critical for the device or fulfill the system-level requirements.

See the examples for some ways we’ve typically addressed these.

What do you do if you use hundreds of third-party packages? 🔗

Modern software is often developed with hundreds, or even thousands, of open-source packages. This is especially true for software that is built by engineers new to the medical device field—they don’t know better.

For lower risk devices, many of these packages provide low-level utilities. Here are a couple of strategies to save time on the documentation:

1. Only provide Documentation for your system-level OTS software. E.g., if you use Python and 30 different Python packages, then claim lifecycle control over the 30 Python packages but not Python. This will greatly reduce the number of OTS components you have. See the next question for more details about this approach.
2. Lump your granular dependencies into “buckets” of related OTS Software and provide Documentation for all of them together. For example, we may lump all of our image-processing Python dependencies into one bucket and answer all of the questions for them together. Many of the questions will have similar answers. You can then reference the SBOM for the exact versions and names of the sub-items. See the examples section below for one approach to doing this.
3. Another strategy that can work in some cases, is to isolate the dependencies used in the core software function from the dependencies used for “Other Functions” (See Multiple Device Functions Impact on FDA Submissions for more details about this.)

Can we claim “complete software life cycle control” for open-source software? 🔗

Well, what exactly does “complete software life cycle control” mean? If you have the full source code, one could argue that in some sense you fully control what goes in or out of the final product.

We have had more than one of our clients successfully use this strategy.

Whether you decide to use it really depends on your regulatory risk tolerance as well as the safety risk of your device.

I’d note that FDA seems increasingly likely to catch this now that SBOMs are used.

More importantly, FDA almost certainly won’t agree that controlling the source code means you have “complete software life-cycle control”. After all, you could in theory argue you fully control the source code to Python too (all million lines of it) it is open source after all!

Controlling the source code for the software does not mean you have “complete software life cycle control.” In particular, you didn’t have control over the various stages of software development, as applied to the library, including:

• Requirements Gathering
• Architectural Design
• Unit Testing
• Integration Testing
• System Testing
• Software Risk Analysis

and so on. Many of the regulatory controls that FDA places on software only make sense if you believe the software-development process itself is important for making good software. As they say in their 2002 “General Principles of Software Validation” guidance:

Because of its complexity, the development process for software should be even more tightly controlled than for hardware, in order to prevent problems that cannot be easily detected later in the development process.

In the case of a very small software library, where it would be possible to carefully review all of the source code yourself, I think one could argue you have “complete software life cycle control.” In most cases though, I think this is too much of a stretch to be warranted.

How do you do an OTS Software Risk Assessment? 🔗

Follow your normal risk management process.

Note that we’ve had FDA reviewers ask questions if there aren’t any risks associated with OTS Software. An obvious risk if you use cloud software is the risk of there being uncontrolled updates that crash your software.

OTS Software Verification 🔗

Can we support multiple versions of OTS Software? 🔗

Yes! If you support multiple versions of Windows, or if you support macOS and Windows, then some verification testing should occur on each version that is supported. Often we suggest performing some regression testing against all the platforms and a full verification suite against the most commonly used version.

Examples 🔗

Detailed Documentation for a Python Library on a Server 🔗

In this example we provide the full “Description of OTS Software” documentation for the Numpy library that runs on a server within the manufacturers control. Many of the questions the FDA asks about OTS software don’t apply in this situation.

Note that we take a conservative approach here and answer every detailed question from the FDA guidance. See the following section for a slightly reduced version of the same responses.

Documentation	Example Value	Commentary
Title	numpy
Manufacturer	Open-source software	If you are more conservative, you could list the numpy.org or the numfocus organization.
Version	1.25.0	We also could have said “See the SBOM”. Note that the FDA guidance asks
Risk Assessment of OTS Software	The DEVICE represents no threat of direct harm to the patient. The risk of indirect harm from a misdiagnosis relating to medical device malfunction is small since the worst case is an incorrect measurement that is considered correct and in many cases any such errors would be evident from context.	DEVICE should be replaced with your device name.
Any OTS Software documentation that will be provided to the end user.	N/A as the end-user doesn’t have access	This is often not applicable the case for many third-party components.
Why is this OTS Software appropriate for this medical device?	The OTS software is widely used, industry standard. Implementing the same functionality from scratch would be more error prone than using this OTS library.	We commonly use this argument for OTS software that is in fact widely used. We’ve even had SOPs for adding new OTS software that requires engineers to evaluate the number of stars a library has in GitHub or the number of downloads it has. Certainly less widely used software is higher risk.
What are the expected design limitations of the OTS Software?	None
What Hardware Specifications are there for the OTS Software to function properly?	There are no specific hardware specifications for this OTS software. It runs in a cloud environment whose hardware is not in our control, but where updates to the hardware can not foreseeably result in the OTS software not functioning properly.	This is most commonly an issue for software installed by users on their own platform. If your software does have hardware dependencies and it runs in the cloud, then concerns about uncontrolled changes to the cloud platform may come into play.
What Software Specifications are there for the OTS Software to function properly?	N/A as it runs in a cloud environment that let’s us control the configuration and version of the OTS software. Furthermore, the configuration and version are the same ones used in validation.
What aspects of the OTS Software and system can (and/or must) be installed/configured?	None
What steps are permitted (or must be taken) to install and/or configure the product?	None
How often will the configuration need to be changed (by the end-user)?	Never
What education and training are suggested or required for the user of the OTS Software?	None
What measures have been designed into the medical device to prevent the operation of any non-specified OTS Software?	None, as the user doesn’t have control over this OTS software’s runtime environment
What is the OTS Software intended to do?	Provides a high-performance, multi-dimensional array utility to the backend code. Provides a fast-fourier transform.	If you are more conservative, you may include some software requirements for the OTS software in your design controls and then you can mention the requirements here. We usually don’t do this for Class II devices.
What are the links with other software including software outside the medical device?	N/A as this software is integrated directly into the source code of the device.
Describe testing, verification, and validation of the OTS Software	The OTS software was tested indirectly by the system verification. Note that it is not necessary to identify specific OTS versions as they are configured within the overall server build images. All verification was thus performed with the exact OTS versions that will be used in the final device.	If you are more conservative and written software requirements for the OTS software, you can list the verification protocol IDs that verify this OTS software. Alternatively, even if you haven’t written requirements of the OTS software, you can still reference specific relevant verification protocols here.
Provide the results of the testing	See the system-level verification records in DOC-00XY Software Verification Report.
Is there a current list of OTS Software problems (bugs) and access to updates?	Yes, the software is open source and has a public bug tracker and updates are provided by the community or could be provided by the manufacturer if needed. Also, cybersecurity vulnerabilities are identified per DOC-00XY Cybersecurity Management Plan.	If you are more conservative, you could also refer to the web page https://github.com/numpy/numpy/issues. Also note that IEC 62304 requires this information too.
What measures have been designed into the medical device to prevent the introduction of incorrect versions?	N/A as it runs in a cloud environment that let’s us control the configuration and version of the OTS software. Furthermore, the configuration and version are the same ones used in validation.
How will you maintain the OTS Software configuration?	In the source code, the specific OTS software versions are pinned. In the runtime environment the python dependencies are bundled into the compiled system image.
Where and how will you store the OTS Software?	The python dependencies are bundled into the compiled system image.
How will you ensure proper installation of the OTS Software?	The python dependencies are bundled into the compiled system image, thus there are no specific installation instructions required to ensure proper installation of the OTS Software.
How will you ensure proper maintenance and life cycle support for the OTS Software?	Bug fixes and maintenance are provided by the community and version updates can be applied in a controlled manner following our software development process (DOC-00ZA), including design controls and configuration management. Cybersecurity vulnerabilities are monitored per DOC-00XY Cybersecurity Management Plan.

Summarized Documentation for a Python Library on a Server 🔗

This is a simplified version of the previous example, where several of the questions from the guidance have been collapsed down into their “parent question.” (See here.) We usually suggest starting with this approach and adding more detail if the FDA requests it.

Documentation	Example Value
Title	numpy
Manufacturer	Open-source software
Version	1.25.0
Risk Assessment of OTS Software	The DEVICE represents no threat of direct harm to the patient. The risk of indirect harm from a misdiagnosis relating to medical device malfunction is small since the worst case is an incorrect measurement that is considered correct and in many cases any such errors would be evident from context.
Any OTS Software documentation that will be provided to the end user.	N/A as the end-user doesn’t have access
Why is this OTS Software appropriate for this medical device?	The OTS software is widely used, industry standard. Implementing the same functionality from scratch would be more error prone than using this OTS library.
What are the expected design limitations of the OTS Software?	None
What are the Computer System Specifications for the OTS Software?	N/A as it runs in a cloud environment that let’s us control the configuration and version of the OTS software. Furthermore, the configuration and version are the same ones used in validation. It runs in a cloud environment whose hardware is not in our control, but uncontrolled updates to the hardware specifications can not foreseeably result in the OTS software not functioning properly.
How will you assure appropriate actions are taken by the End User?	N/A as end users don’t need to take any actions
What does the OTS Software do?	Provides a high-performance, multi-dimensional array utility to the backend code. Provides a fast-fourier transform.
How do you know it works?	The OTS software was tested indirectly by the system verification. Note that it is not necessary to identify specific OTS versions as they are configured within the overall server build images. All verification was thus performed with the exact OTS versions that will be used in the final device. See the system-level verification records in DOC-00XY Software Verification Report.
How will you keep track of (control) the OTS Software?	The software runs in a cloud environment that’s within our control and uses the same configuration used in validation. In the source code, the specific OTS software versions are pinned. In the runtime environment the python dependencies are bundled into the compiled system image.

Detailed Documentation for Windows OS on a Device 🔗

In this example we provide the full documentation for the Windows OS that runs on hardware device within the manufacturers control. Many of the questions the FDA asks about OTS software don’t apply in this situation.

Note that we take a conservative approach here and answer every detailed question from the FDA guidance. See the following section for a slightly reduced version of the same responses.

Documentation	Example Value	Commentary
Title	Windows 11 IoT Enterprise LTSC
Manufacturer	Microsoft
Version	Version 22H2
Risk Assessment of OTS Software	The OTS Software represents no threat of direct harm to the patient. There is a risk that a failure or latent flaw in this OTS could result in indirect harm due to a misdiagnosis. See RISK-YYY, RISK-XXX, and RISK-ZZZ for details.
Any OTS Software documentation that will be provided to the end user.	N/A as the end-user doesn’t have access to the OTS software.
Why is this OTS Software appropriate for this medical device?	The Windows OS is deemed appropriate for the medical device because it is a widely used operating system that provides a convenient abstraction over the underlying hardware.
What are the expected design limitations of the OTS Software?	None
What Hardware Specifications are there for the OTS Software to function properly?	N/A as it runs on controlled hardware and uses the same configuration used in validation
What Software Specifications are there for the OTS Software to function properly?	N/A as it runs on controlled hardware and uses the same configuration used in validation
What aspects of the OTS Software and system can (and/or must) be installed/configured?	None
What steps are permitted (or must be taken) to install and/or configure the product?	None
How often will the configuration need to be changed (by the end-user)?	Never
What education and training are suggested or required for the user of the OTS Software?	None
What measures have been designed into the medical device to prevent the operation of any non-specified OTS Software?	Windows Defender System Guard has been enabled, which (1) protects and maintains the integrity of the system as it starts up, and (2) validates that system integrity has been maintained through local and remote attestation. See REQ-XXX for details.
What is the OTS Software intended to do?	The Windows OS provides a standard operating system to run the software, including the following functionality: - File system access - Networking functionality - Graphics drivers - User input via keyboard and mouse	If you use very specific aspects of Windows, or if it were a higher-risk device, you’d need a lot more detail here. You’d also reference the requirement IDs that specify this functionality.
What are the links with other software including software outside the medical device?	The Windows OS provides networking functionality that is used to interact with external systems, as described in DOC-XXX System and Software Architecture Diagram.
Describe testing, verification, and validation of the OTS Software	The OTS software was tested by the system verification. Note that it is not necessary to identify specific OTS versions as they are configured within the overall server build images. All verification was thus performed with the exact OTS versions that will be used in the final device.	If you did call out specific requirements up above, you would want to refer to the verification of those requirements.
Provide the results of the testing	See the system-level verification records.
Is there a current list of OTS Software problems (bugs) and access to updates?	Yes, Windows provides updates to the software. [Link to windows bug list, if available. Perhaps also mention how Microsoft reports cybersecurity vulnerabilities.]
What measures have been designed into the medical device to prevent the introduction of incorrect versions?	Automatic software updates, except for Windows Defender, have been disabled.
How will you maintain the OTS Software configuration?	Users are not able to modify the Windows OS configuration.
Where and how will you store the OTS Software?	Full system images, including Windows OS, are archived according to DOC-XXX Build Protocol.
How will you ensure proper installation of the OTS Software?	The proper installation of Windows OS is handled by the manufacturer.
How will you ensure proper maintenance and life cycle support for the OTS Software?	Bug fixes and maintenance are provided by Microsoft for 10 years. Windows Updates can be applied in a controlled manner following our software development process (P-XXX Software Development Process). Cybersecurity vulnerabilities are monitored and addressed per DOC-XXX Cybersecurity Management Plan.

Summarized Documentation for a Bundle of Frontend JavaScript Dependencies 🔗

This example shows a basic method of documenting a bundle of related JavaScript dependencies used in a low-risk front-end component of a device. Often you’ll have a few “bundles” like this that are documented together. E.g., you may have a “backend python dependencies bundle” in addition to your frontend dependencies.

Documentation	Example Value	Commentary
Manufacturer	Open-source software
Version	See DOC-XXX Software Bill of Materials for exact versions	A more conservative approach would be to copy the versions from the SBOM over here to save the reviewer from having to do the lookup.
Risk Assessment of OTS Software	These OTS software represent no threat of direct harm to the patient. Furthermore, a failure or latent design flaw in the OTS software can not reasonably cause the hazardous situation where a the diagnostic result is invalid.
Any OTS Software documentation that will be provided to the end user.	N/A as the end-user doesn’t have access
Why is this OTS Software appropriate for this medical device?	These OTS software are widely used. Implementing the same functionality from scratch would be more error prone than using these OTS software.
What are the expected design limitations of the OTS Software?	None
What are the Computer System Specifications for the OTS Software?	N/A as it runs on the cloud environment that’s within our control and uses the same configuration used in validation
How will you assure appropriate actions are taken by the End User?	N/A as end users don’t need to take any actions
What does the OTS Software do?	- react, react-dom, react-hook-form, react-i18next, react-router-dom: Provides tools for building user interfaces in a more modular and efficient manner. - sentry/react: Provides tools for using the analytics service, Segment from apps. - @testing-library/cypress, cypress-real-events: Provides tools for end-to-end web app testing. - date-fns, date-fns-tz: Provides date and time utilities. - esbuild: Provides tools for bundling various software assets for web apps. - graphql, graphql-tag: Provides tools for communicating between apps and APIs in a fast, flexible, and efficient way. - i18next: Provides internationalization tools. - lodash: Provides various Javascript utilities. - polished: Provides UI tools for styling app components using Javascript. - @styled-icons/feather, @styled-system/theme-get, @tailwindcss/forms, postcss, postcss-import, tailwindcss, styled-components, styled-system: Provides UI tools for styling web components.
How do you know it works?	The OTS software was tested indirectly by the system verification.
How will you keep track of (control) the OTS Software?	The software runs in a cloud environment that’s within our control and uses the same configuration used in validation. In the cloud environment, the specific OTS software version is pinned.

Summarized Documentation for AWS 🔗

Documentation	Example Value	Commentary
Title	AWS Sagemaker
Manufacturer	Amazon Inc.
Version	The version of AWS Sagemaker is incremented automatically by AWS. The most recent version of the AWS Sagemaker python API, which is used to access the underlying Sagemaker functionality, can be found in their public changelog at https://github.com/aws/sagemaker-python-sdk/blob/master/CHANGELOG.md.	We don’t control the version of the AWS Sagemaker.
Risk Assessment of OTS Software	AWS Sagemaker is intended to run our docker container and facilitate the transfer of data and log files to and from the container. Given how the docker container is isolated and controlled separately from AWS Sagemaker, it’s not foreseeable that uncontrolled updates to AWS Sagemaker could lead to the algorithm producing incorrect outputs. It is foreseeable that updates to AWS Sagemaker could result in loss of availability. However, given the intended use of the device, loss of availability is not considered a hazardous situation.	This argument won’t always be true, depending on how your device is intended to be used. We usually reference specific risks that were identified for the uncontrolled cloud software, along with their mitigations.
Any OTS Software documentation that will be provided to the end user.	N/A as the end-user doesn’t have access or control over AWS Sagemaker.
Why is this OTS Software appropriate for this medical device?	AWS Sagemaker is widely used for deploying AI/ML pipelines. Implementing the same functionality from scratch would be more error prone than using this OTS library.
What are the expected design limitations of the OTS Software?	Lack of control or even fully visibility into when updates are made to AWS Sagemaker.
What are the Computer System Specifications for the OTS Software?	N/A as it runs in a cloud environment that’s managed by the manufacturer and uses the same hardware configuration as is used in validation.
How will you assure appropriate actions are taken by the End User?	N/A as end users don’t need to take any actions
What does the OTS Software do?	AWS Sagemaker exposes an API endpoint for triggering processing runs of our docker container. It ensures that each processing run is completed independently in its own isolated and reproducible environment.
How do you know it works?	The OTS software was tested indirectly by the system verification.
How will you keep track of (control) the OTS Software?	The OTS Software is an external cloud service whose configuration we do not control. The current version of the OTS Software is widely used, so the manufacturer is incentivized to prevent the introduction of incorrect versions and maintain backwards compatibility.

Revision History 🔗

Date	Summary of Change
18 July 2023	Initial Version
11 August 2023	Added the “Windows OS” example and the “JavaScript Frontend Dependencies” example. Also added a question about SOUP.
29 September 2023	Updated the questions to reflect the new 2023 FDA Guidance. Clarified responses to a few questions.
1 November 2023	Updated to fully remove references to the old OTS Level of Concern. Corrected a few of the examples based on FDA feedback from a recent submission. Added a section that discusses cloud OTS software.
23 January 2024	Added question about “complete software development life cycle control”. Added an example about AWS components.
29 April 2024	Added question about using LLMs or ChatGPT. Added details about a strategy for not considering Python or JavaScript dependencies OTS Software based on experience with recent FDA submissions. Shorten some of the questions a bit. Expand the question about a manufacturer a bit.