Medical Device Cybersecurity: Best Practices, FAQs, and Examples

Introduction 🔗

The FDA has become increasingly focused on medical device cybersecurity, as can been seen by the frequency and length of their guidance (2014, 2016, 2018, 2022, 2023, and 2024). This has only accelerated when, in 2023, congress gave FDA additional authority to regulate the cybersecurity of medical devices.

For groups who are less experienced with cybersecurity, reading the FDA guidance and the various standards can be overwhelming. Fortunately, when you boil down these abstract documents into the concrete actions you need to take, it’s actually much simpler. In this article, we collect a variety of best practices, questions, and examples related to cybersecurity. Our goal is to be as concrete and specific as we can.

We’ve also seen a large up-tick in the FDA’s requests for additional information in medical-device submissions.

If you like this article, you may also be interested in our other “FAQ, Best Practices, and Examples” style articles:

• Medical-Device Software Regulations: Best Practices, FAQs, and Examples
• Design Inputs: Best Practices, FAQs, and Examples
• SBOMs: Best Practices, FAQs, and Examples

If you have medical-device cybersecurity questions that aren’t answered here, feel free to reach out to us in the website chat and we’ll get back to you.

We are a software development and regulatory consulting firm that specializes in SaMD. We help manufacturers with cybersecurity, regulatory strategy and submissions, and provide end-to-end SaMD development.

FAQs 🔗

Please note that threat modeling is used in many industries. This article discusses threat modeling medical devices and focuses on the US regulations.

Basics 🔗

What is cybersecurity? 🔗

With regards to medical devices, the FDA defines cybersecurity as the process of preventing

• unauthorized access,
• modification,
• misuse or denial of use, or
• the unauthorized use of information that is stored, accessed, or transferred from a medical device to an external recipient.

Do all medical devices need to comply with the FDA’s cybersecurity guidance? 🔗

Per FDA’s eSTAR PDF template, you need to handle cybersecurity if your device

• Is hosted in the cloud
• Has a network connection (active or not)
• Has wireless communication of any form
• USB, serial ports, or removable media
• Support software upgrades (including patches).

If you meet any of these conditions, FDA will require you to provide cybersecurity documentation in your 510(k) submissions and have a plan for monitoring, assessing, fixing, and reporting vulnerabilities after your device is on the market.

Technically, here’s the regulatory definition of a “cyber device”:

A ‘cyber device’ means a device that—(1) includes software validated, installed, or authorized by the sponsor as a device or in a device; (2) has the ability to connect to the internet; and (3) contains any such technological characteristics validated, installed, or authorized by the sponsor that could be vulnerable to cybersecurity threats.

In our experience interacting with the FDA, they interpret “has the ability to connect to the internet” broadly and if you meet any of the conditions in the eSTAR template, you’ll have a hard time arguing you’re not a cyber device. For example, devices that only have a USB port are still considered cyber devices. So are docker images which do not connect to the internet but are expected to be run in an environment where they can connect. For example, here’s a response from the FDA when we had tried to argue a device with just a USB port is not a cyber device:

Though the USB interface may only be intended for transferring images, the capability for someone to use it outside of its intended purpose remains (i.e., loading malicious software, etc.). While medical device systems with network and/or cloud access would be expected to have more cybersecurity safety requirements than a medical device system that has only a USB interface, a device with a USB interface is still by definition a cyber device.

What cybersecurity activities must be done when creating a medical device? 🔗

The guidance suggests incorporating security into the full development life-cycle. The FDA guidance is written in a somewhat abstract way. Here, I’ll list the key activities as they fall within a typical software development lifecycle process:

• Planning
  • Identify the personnel responsible for security activities
  • Define how the likelihood, severity, and acceptability of security risks will be assessed
  • Identify the methods that will be used to identifying security risks, including at least threat modeling
  • Identify the threat-modeling methodology that will be used
  • Identify any secure coding standards or static code analysis tools that will be used during development
  • Plan for how the development environment will be kept secure (e.g., requiring two-factor authentication in GitHub)
  • Plan for how software updates will be delivered in a timely fashion to the device
• Architecture
  • Document key security views (these will be used in threat modeling)
  • Security risk assessment - given the initial architecture, analyze likely security risks
• Development
  • Implement the security controls that are identified through the security risk assessment
• Labeling
  • Generate an SBOM to be shared with customers (and used to monitor for vulnerabilities)
  • Cybersecurity documentation in labeling
• Design Verification
  • Penetration Testing
  • Vulnerability Testing
  • Verification of all security controls
• Design Validation
  • If applicable, validate that customer IT Staff have sufficient information in labeling to understand their security responsibilities
• Maintenance
  • Reporting security vulnerabilities that are identified
  • Monitoring your SBOM for vulnerabilities
  • On-going vulnerability testing
  • On-going penetration testing
  • Security updates

Security Risk Management 🔗

What is security risk management? 🔗

Security risk management is similar to safety risk management. Both attempt to identify “bad things” that could happen, assess how likely or severe they would be if they did happen, and then mitigate them from happening. They differ in a few key ways. First, the types of “bad things” are different. Thus, the severity level definitions and acceptance criteria are different. Second, in the case of cybersecurity, there may be a malicious actor attempting to make the bad things happen for some motive. Thus, probabilistic analysis of failure is less relevant.

How is security risk management related to safety risk management? 🔗

Safety risk management focuses on harm to patients and device users. Security risk management considers harm to assets due to cybersecurity breaches more generally, including business reputations, PHI, and other assets not accounted for in safety risk management. (As an aside, risk management is useful to model anytime you’re trying to anticipate harm more broadly, including harm to business profitability, regulatory submission risk, and so on.)

Security risks are sometimes, but not always, related to patient safety risks.

Safety risk management is clearly defined in the widely-adopted ISO-14971 standard.

Security risk management for medical devices is described in the AAMI TIR57 “Principles for medical device security—Risk Management” Technical Information Report, which the FDA has recognized as a consensus standard.

There is a very useful diagram, which I’ve copied from TIR57, that shows the relationship between the processes:

Is it okay to incorporate security risk management into safety risk management? 🔗

No. Previously, FDA had been more open to combining the two processes and we’d suggested (for simple devices) that our clients keep them combined. FDA has been clear in their 2023 guidance and in their newest eSTAR templates that the two processes need to be separate:

Please attach your security risk management report detailing a separate, parallel,
and interconnected security risk management process. This is different from your
safety risk management process.

How should security risks be assessed? 🔗

This is a complicated question. We’ve seen some companies use the Common Vulnerability Scoring System (CVSS). For most devices, we don’t suggest that CVSS be used out of the box as it is too complicated (e.g., see the CVSS Version 4.0 Calculator).

We assess security risks following a security risk-acceptability table with two axes along likelihood and severity. The tables below present the levels along each of these two access, which combine to present the overall acceptability. This is much like its done for ISO 14971.

The specifics of these tables should typically be modified for your particular device and use environment.

Here is an example set of security risk likelihood levels:

Label	Method of Access	Authentication Level	Nature of Vulnerabilities
Improbable (1)	Persistent physical access required	Multiple independent authentications required	Highly specialized conditions and equipment are required
Remote (2)	Temporary physical access required	Multi-factor authentication	Multiple conditions or equipment are required
Occasional (3)	Private network access required	Single factor	Specialized conditions required
Probable (4)	Local network or radio-frequency proximity required	Simple single factor (e.g., a short password or PIN)	Simple conditions required
Frequent (5)	Remote access required	None required	N/A

Here is an example set of security risk severity levels:

Label	Potential Impact to Device Operations	Potential Impact to Business Operations	Potential Impact to Data	Potential Impact to Other Organizations or the Environment
Negligible (1)	No negative effects	No negative effects	No negative effects	No negative effects
Minor (2)	Insignificant loss of non-essential functions; temporary loss of device operation resulting in inconvenience	Customer complaints or limited damage to reputation	Loss of confidentiality of non-confidential data¹	Device reveals information about itself beyond what is necessary
Moderate (3)	Significant loss of availability across a single or system communication can be tampered with	Loss of intellectual property; large damage to business reputation; minor legal issues	Loss of integrity of non-essential data²; loss of confidentiality of a small amount of confidential data¹	Device becomes a vector to gather information about other systems on the same network
Serious (4)	Significant loss of availability across multiple devices	Serious legal issues; inability to perform critical mission functions	Loss of integrity of essential data²; loss of confidentiality of a large amount of confidential data¹	Device becomes a vector to attack other systems

¹Confidential data is defined as data that is protected by regulatory bodies, such as HIPAA or the GDPR.

²The following sources are considered essential data:

• Device configuration
• Clinical reports
• … (Fill in with list of examples for the device in question)

What is multi-patient harm? 🔗

The concept of “multi-patient harm” is introduced in the 2023 FDA Cybersecurity Guidance.

A device can cause “multi-patient harm” if a cybersecurity incident could result in multiple patients being harmed “simultaneously or in rapid succession”.

What are some examples of multi-patient harm? 🔗

These examples are somewhat simplified, so they don’t provide all the details needed to make a firm conclusion either way. Still, I hope they’re helpful:

Cloud-Based Semi-Automated Diagnostic Radiology SaMD - This software is unlikely to cause multi-patient harm. The software does not directly interact with patients in anyway. It also isn’t on a network with other devices that directly interact with patients. Radiologists interact with the software to produce its outputs in a semi-automated way. This human-in-the loop slows down the effects of most exploits. Even if the software were fully automated or an exploit could modify the reports, there are still other clinicians downstream from the radiology report who introduce checks and delays. Thus, there wouldn’t be feasible for patients to be harmed simultaneous or in rapid succession.

Multiparameter Bedside Monitors - This example is taken from the FDA guidance. Multiparameter bedside monitors would be capable of causing multi-patient harm. They aren’t able to directly harm the patient, however, a cybersecurity attack could restart all of the monitors at once. This would leave all monitors connected to the same network no longer monitoring patient vitals and it is quite possible there wouldn’t be enough staff to monitor all patient vitals.

Threat Modeling 🔗

Do I need to create a threat model? 🔗

Yes, if your device is a cyber device, you will need to create a threat model for your FDA submission. See the threat modeling section of the 2023 cybersecurity guidance.

What is a threat? 🔗

The FDA defines a threat as follows:

“A threat is any circumstance or event with the potential to adversely impact the device, organizational operations (including mission, functions, image, or reputation), organizational assets, individuals, or other organizations through an information system via

• unauthorized access,
• destruction,
• disclosure,
• modification of information,
• and/or denial of service.”

What is threat modeling? 🔗

The FDA’s 2023 cybersecurity guidance defines threat modeling as follows:

“Threat modeling is a methodology for optimizing system, product, network, application, and connection security by identifying objectives and vulnerabilities, and then defining countermeasures to prevent, or mitigate the effects of, threats to the system.”

(Note that the 2016 post-market guidance has a slightly different definition.)

The output of the threat modeling process is a threat model.

What is typically included in a threat model? 🔗

There are many different ways of organizing and structuring threat models. Many of the approaches come from other industries. Here is the content we typically in include in our threat models which standardizes on the terminology found in the FDA guidance:

• Scope of the threat model
  • Rationale for the approach used
  • Who was involved in the threat modeling
  • Assumptions about the system and environment of use
• An inventory of security assets
• An inventory of connections
• An inventory of expected threat sources
• Architectural diagrams including
  • The key modules and layers of your device
  • All external systems
  • Their connections
  • The process by which software updates go from source code to deployment
• An analysis of the main threats (often enumerated following a few methodologies, such as STRIDE)
• A listing of the implemented mitigations

There’s not a standard format for threat models. We will provide examples of how we format our threat models below.

What should be the scope of the threat model? 🔗

The FDA emphasizes that threat modeling must account for the entire operating environment of the device. This raises a critical question: How extensively should we consider the surroundings of the medical device? It is not always an easy question:

• For devices sold to multiple institutions, the potential scenarios are only limited by your imagination.
• For devices used within a specific lab or institution, you may wonder how far into the surrounding IT infrastructure you need to go.

We haven’t found any heuristics to offer to answer this question, but in general we are guided by the likelihood and severity of the security risks.

What is an “asset” in this context? 🔗

FDA defines an “asset” as “anything that has value to an individual or an organization.” This is a very broad definition. Here are some common types of assets we consider:

• Reports
• Images
• PHI
• Databases
• User Account Information
• Log files
• Security Keys
• Backups
• Source Code
• Configuration Files
• Update Files
• Network Communication

Which threat modeling methodologies do you suggest using? 🔗

The FDA says the “rationale for the methodology(ies) selected should be provided with the threat modeling documentation.”

We suggest following the STRIDE or STRIDE and Attack Trees method for doing threat modeling. This is because the FDA-sponsored Threat Modeling Handbook mentions these methodologies, and thus, when justifying the threat-modeling methodology in your security management plan (which the FDA requires you to do), you can say that these were the methods suggested in the FDA's Threat Modeling Handbook.

How do you create a threat model? 🔗

Here is an example procedure to follow to create a threat model for an FDA submission using the STRIDE methodology.

1. Identify all major system items
   • Unless your system is very simple, you often need to make judgement calls regarding what level of detail to decompose your system into. Too much detail isn’t useful.
   • Threat models should encompass the full end-to-end system the device operates within, not just the device itself.
   • We recommend labeling each system item (e.g., SI-001, SI-002).
   • Examples: Backend Django App, AWS S3 Bucket, PACS, PostgreSQL Database, etc.
2. Create a system diagram, including connection
   • These diagrams should show all the connections to external systems or networks.
   • We recommend labeling each system item and connection within these diagrams.
   • Create the security architecture diagrams described in the FDA guidance:
     • Global System Views
     • Multi-Patient Harm View (when applicable)
     • Updatability and Patchability View
     • Security Use Case Views (often multiple)
3. Identify and assess cybersecurity assets
   • Common assets include PHI, data stores, log files, security keys, access to other systems, business reputation, etc.
   • Examples of assets include PHI, AWS credentials, configuration files, PostgreSQL Database, and more abstract things like business reputation.
   • We generally will indicate which system items the assets are stored within (sometimes the system items are themselves the assets).
   • Once the assets are identified, we analyze the impact of the loss of confidentiality, integrity, and availability of these assets.
4. List threats for each system item using STRIDE
   • Iterate through each system item and connection, reviewing the system diagrams as you go, and consider threat types according to STRIDE (Spoofing, Tampering, Repudiation, Info Disclosure, DDoS, Elevation of Privileges).
   • We follow STRIDE-per-element as described in Adam Shostack’s book
   • Write down these threats.
   • See and for more examples.
5. Eliminate or refine threats that are highly improbable or impossible to verify
   • If it is likely to impact patient safety, keep it.
   • If it is unlikely to impact patient safety and is difficult to verify, consider removing it.
   • If you’ve already mitigated it and it is easy to verify, keep it.
6. Trace threats to security risks and their mitigations
   • Example:
     • For the threat/mitigation pair
       • Threat: Malicious actor brute forces the clinician’s username and password.
       • Mitigation: Temporarily block login after 5 incorrect attempts.
     • You might have to create a new requirement to implement the threat mitigation
       • REQ-035: Throttle User Authentication
     • And add a new risk to cover the threat
       • SRSK-025: Brute Force Attack

Can you give examples of threats for each STRIDE category? 🔗

• Spoofing: Pretending to be something or someone you’re not
  • Example:
    • Threat: Malicious actor brute forces the clinician’s username and password.
    • Mitigation: Temporarily block login after 5 incorrect attempts.
• Tampering: Changing something in a malicious way (on the disk, db)
  • Example:
    • Threat: Malicious actor modifies PostgreSQL Database via SQL injection.
    • Mitigation: Implement input validation on frontend and backend.
  • Real Insulin Pump Example:
    • Background: This example is taken from the first FDA recall due to cybersecurity.
    • Threat: Due to a software vulnerability, threat actors could remotely tamper with the insulin pump’s configuration via wifi.
    • Mitigation: Ensure all connections to the software are properly authenticated.
• Repudiation: Doing something bad and claiming to not have done it
  • Example:
    • Threat: Malicious actor denies that a certain treatment has been prescribed.
    • Mitigation: Track all prescription actions in centralized logging system.
  • Drug Dispenser Example:
    • Threat: A IT-person steals an opioid from a drug dispenser and modifies the logs to remove a trail of them having done so.
    • Mitigation: Log important events and secure and back up the logs.
  • Malpractice Example:
    • Threat: A clinician makes a medical mistake; they’re worried about a law suite and try to cover up the mistake.
    • Mitigation: Log important events and secure and back up the logs.
• Information Disclosure: Having sensitive data leaked
  • Example:
    • Threat: Malicious actor performs man-in-the-middle attack and steals data.
    • Mitigation: Use encryption in transit (HTTPS).
• Denial of Service: Removing availability of the system
  • Example:
    • Threat: Malicious actor overwhelms backend with abnormal traffic.
    • Mitigation: Configure AWS Web Application Firewall for common exploits.
  • Malware Example:
    • Background: This is one of the most common and pervasive threats to the
• Elevation of Privileges: Gaining unauthorized access
  • Example:
    • Threat: Malicious user changes their role to admin via API.
    • Mitigation: Only allow admins to change user roles and enforce this.

Why do you need to trace the threats to security risks? 🔗

The main benefit of threat modeling is that it is a structured process. For example, if you’re following STRIDE, you’d analyze all six types of threats to each component of the system. By systematically analyzing each component, you can be sure you didn’t miss anything. Also, by recording this analysis, engineers working on future updates to the project will understand the thought process that was used.

The security risks, on the other hand, are typically higher level. You’ll have more entries in a STRIDE threat model table than you will in your security risks. (This is similar to how an FMEA will analyze each component of the system for failure modes, and will often have more detail than the corresponding safety risks.) That said, FDA says clearly in their guidance that the threat model must “identify medical device system risks and mitigations as well as inform the pre- and post-mitigation risks considered as part of the cybersecurity risk assessment.” We take this to mean the identified threats should trace to the higher level risks, their pre- and post-mitigation impacts and likelihood levels.

When should threat modeling be done? 🔗

I like to do an initial threat model once the system architecture is reasonably well defined. Within our software development process, this is at the end of Phase 1.

We then review the threat model multiple times during our second iterative design and development phase.

Finally, period reviews of the threat model should be performed in the post-market phase or as part of the development of new releases.

Who should create the threat model? 🔗

Software engineers who are familiar with the system should be able to perform the threat modeling.

Unlike with penetration testing, where we do suggest using an external firm, we don’t typically suggest having an outside firm independently put together a threat model. This is because the security risks are tightly integrated with design controls and safety risk management, and thus an external group will typically have a hard time incorporating their findings into the rest of the documentation unless they’re working closely with your team.

Having a security expert who knows the current threat landscape for your intended use environment is also helpful.

How is threat modeling related to security risk management? 🔗

In our opinion, the distinction between threat modeling and security risk management is vague. We suspect this is because threat modeling was developed outside the medical device industry and had its own terminology developed apart from of ISO 14971. In any event, it is straightforward to map the components of the Threat Modeling Manifesto to the AAMI TIR57 security risk management activities, as shown in the following diagram.

If we were forced to make a distinction, we would say that “threat modeling is a loose collection of methodologies used to systematically identify, mitigate, and document security risks.”

According to this distinction, threat modeling is to security risk management as FMEA is to safety risk management. FMEA is a structured way of identifying safety risks related to parts of your system breaking down. Unlike FMEA, which is rarely sufficient to identify all of the safety risks for a device because it misses process- and usability-related risks, threat modeling is in theory sufficient on its own. Thus, even this attempt at a distinction isn’t very useful.

What other resources exist for threat models? 🔗

• MITRE’s “Playbook for Threat Modeling Medical Devices”
• Promenade’s guide to their hybrid asset and vulnerability based threat modeling
• Adam Shostack’s Threat Modeling Book
• Threat Modeling Manifesto
• OWASP Threat Modeling Cheat Sheet
• The Ultimate List of STRIDE Threat Examples

Cybersecurity Testing Overview 🔗

What types of cybersecurity testing does FDA require? 🔗

FDA suggests considering the following four types of cybersecurity testing:

Testing Type	Description
Security Requirements Testing	This is normal design verification for all of your cybersecurity-related software requirements.
Threat Mitigation Testing	This is normal design verification for the risk control measures identified in your threat modeling and security risk assessment.
Vulnerability Testing	Vulnerability testing is a type of cybersecurity testing that focuses on identifying and characterizing security vulnerabilities.
Penetration Testing	A penetration test (pen test) is an authorized simulated attack performed on a medical-device system to evaluate its security.

Vulnerability Testing 🔗

What is Vulnerability Testing? 🔗

Vulnerability testing is a type of cybersecurity testing that focuses on identifying and characterizing security vulnerabilities. A vulnerability is a “a weakness in an information system, system security procedure(s), internal control(s), human behavior, or implementation that could be exploited.”

Do all cyber devices need vulnerability testing? 🔗

Yes! Unlike some of the other cybersecurity testing activities, the need for vulnerability testing is actually included in the legal statute itself, which says cyber devices must:

Submit a plan to monitor, identify, and address, as appropriate, in a reasonable time, postmarket cybersecurity vulnerabilities and exploits, including coordinated vulnerability disclosure and related procedures

What are some specific types of vulnerability testing we should do? 🔗

In the 2023 FDA guidance, the following types of testing are suggested for consideration:

Type	Example
Abuse or misuse cases, malformed and unexpected inputs	Entering special characters or extremely long strings into input fields
Robustness testing	Stress testing the system under high load or with limited resources
Fuzz testing	Automatically generating and submitting random data to API endpoints
Attack surface analysis	Identifying all possible entry points for attacks, such as open ports or APIs
Vulnerability chaining	Combining multiple low-severity vulnerabilities to create a high-severity exploit
Closed box testing of known vulnerability scanning	Using automated tools to scan for known vulnerabilities without prior knowledge of the system
Software composition analysis of binary executable files	Analyzing compiled code to identify potentially vulnerable third-party libraries

How should we document the results of our Vulnerability Testing? 🔗

Per the FDA guidance, for any testing tools or software used, the details provided may include, but may not be limited to, the name of the tool, version information as applicable, and any settings or configuration options for the tools used.

Fuzz Testing 🔗

What is fuzz testing? 🔗

Fuzz testing, also sometimes called fuzzing, is a particular type of vulnerability testing technique. Simply put, it involves feeding randomized inputs to the software and seeing if anything breaks.

Do we have to do fuzz testing? 🔗

FDA recommends that “fuzz testing” be considered for submission in their 2023 FDA Guidance. We’ve had mixed luck pushing back on the need for fuzz testing with FDA. In several submissions, FDA has accepted that fuzz testing was not suitable. On others, FDA has held their ground.

Traditionally, fuzz testing has focused on languages like C or C++ which are compiled and prone to memory errors, however, fuzzing can be applied to software built with interpreted languages such as JavaScript and Python too.

What attack surface should we use for our fuzz testing? 🔗

There are several different “attack surfaces” that could be fuzz-tested. At the lowest level, you can fuzz test your individual functions or classes. With regards to cybersecurity fuzz testing, the external interface of your device is most appropriate.

See Part 1 section 15 of UL-2900 Software Cybersecurity for Network-Connectable Products for some more structured details about “malformed input testing” (of which fuzz testing is a type). It says “The product shall continue to operate as intended when subject to invalid or unexpected inputs on its external interfaces.”

How much fuzz testing is enough? 🔗

Since fuzz testing involves feeding random inputs into an interface or API, there is always the question of “how much is enough”? There’s no clear answer here, but this is something your test reports that will be in your 510(k) submission would ideally address.

The UL-2900 standard says:

If generational malformed input testing is applied for a protocol on an interface, at least 1,000,000 unique and independent test cases or a minimum of 8 hours of test case execution shall be carried out, whichever comes first.

This may be a good starting place.

Can we rely on the fuzzing our pen tester did? 🔗

External pen testers may perform fuzz testing as part of their pen testing suite. If they do, you may be able to “take credit” for the fuzz testing they already did. If you plan ahead for this, it would be good to have information about the interfaces they tested, how long they fuzzed, and how many inputs they fed into the tests in the final report they produce.

Where can I learn more about Fuzzing? 🔗

Here are some good resources:

• Section 15 of the UL-2900-1 standard (costs money to purchase) has some requirements that may help build out a structure for a fuzz testing procedure.
• This Synopsys article has a great technical intro to fuzz testing.
• This OWASP page is semi-useful and has some links.

Penetration Testing 🔗

What is penetration testing? 🔗

A penetration test (pen test) is an authorized simulated attack performed on a medical-device system to evaluate its security. Penetration testers act as if they were attackers to find and demonstrate the business impacts of weaknesses in a device. Penetration tests usually simulate a variety of attacks that could threaten a device, using the same tools, techniques, and processes that real hackers use.

What is pentesting? 🔗

Pentesting is shorthand for penetration testing. We usually use the longer form because this is what the FDA uses in their guidance and in our documentation we generally try to follow the FDA’s lingo to make it easier on reviewers to follow our submissions.

How is penetration testing different from vulnerability testing? 🔗

Vulnerability testing focuses on identifying and characterizing vulnerabilities. Penetration testing involves trying to discover and exploit vulnerabilities and other weaknesses in the system. Thus, vulnerability testing will stop once the vulnerabilities are identified. Penetration testing will take the next step and will try to actually exploit the vulnerability.

Thus, penetration testing has fewer false positives than vulnerability testing, which often finds many vulnerabilities that can’t actually be exploited in practice.

Do all cyber devices need penetration testing? 🔗

No. Unlike, e.g., including an SBOM, FDA doesn’t strictly require penetration testing. They do suggest considering it.

In most cases we strongly suggest doing pen-testing.

We usually include pen testing in our submissions, but we have had a couple of devices with very limited threat surfaces get FDA-cleared without pen testing.

Can we do the penetration testing ourselves? 🔗

FDA suggests that the penetration testing have a level of independence from the team that developed the software. The best way to accomplish this is to use a separate firm.

We have had a few clients successfully use someone else on their team to do pen-testing, however it is important that:

1. The person was not involved in the development of the application
2. The person has credentials and certifications for doing pen-testing

FDA will almost certainly not allow an engineer who helped develop the device do the testing. They also won’t be content with someone who just picked up some pen-testing tools online and ran them against the software.

How do you scope penetration testing? 🔗

Here are some questions to ask yourself about this:

• What connections are being tested? E.g., will you be testing blue-tooth connections, USB ports, cloud infrastructure, your servers, etc.
• Will you pen-test the device’s update process?
• For SiMD, will you need to ship the device to the client or will the pen-testers need to fly on site?
• How much information about the device will you share with the pen-testers? Will it be black-box, gray-box, or white-box testing?
• Will you want the vendor to re-run the pen-tests after your team has had a chance to address issues?

How should we evaluate vendors to do our penetration testing? 🔗

Once you’ve identified the scope of your pen testing, you can share this with vendors to see if they’re a good fit and if they’ll provide you a quote.

Here are some questions you may want to ask:

• If you have SaMD, make sure they do application penetration testing (and not network IT infrastructure pen testing).
• If you have a physical device, make sure they do device penetration testing.
• How much lead time do you need before you can do pen testing?
  • This is important, since ideally you’ll do pen testing just before your submission and you don’t want a big delay to be a bottleneck.
• Can you provide us a sample report?
  • You will want to check that the testing report will include the following elements required by FDA:
    • Independence and technical expertise of testers;
    • Scope of testing;
    • Duration of testing;
    • Testing methods employed; and
    • Test results, findings, and observations.
• Do you have discounts for doing repeated tests?
  • Keep in mind you’ll likely be repeating the penetrating testing (e.g., annually), so it’s often helpful to have a single vendor you use.

Should I select pen-testers that focus on medical-devices? 🔗

If you need help with threat modeling or security risk management, it can be helpful to have a pen-tester who knows the FDA regulations. In general though, we don’t believe it is necessary to work with pen-testers who focus just on medical-devices. We’ve had several submissions with pen-testers who do not focus on the medical device sector.

Who are some good vendors to consider? 🔗

We’ve had clients use the following vendors:

Vendor	Comments
Cobalt	Quick Startup Time
Synopsys
MedSec	Focused on medical devices
Blue Goat	Focused on medical devices

How much does penetration testing cost? 🔗

It varies from vendor to vendor and based on the scope of the testing, but you can expect between $7k and $22k. We’ve even see as high as $200k in one instance.

When should we do our penetration testing for a premarket submission? 🔗

Ideally, your penetrating testing will be done on the final version of the software that you’re submitting to FDA. This is because FDA recognizes that the results of penetration testing degrade over time. It’s never happened to us, but it’s possible that FDA could reject results that were done too long ago.

How often should we do penetrating testing? 🔗

We suggest documenting in your Security Management Plan when and how often you plan to do penetration testing. In the FDA guidance, they suggest repeating it periodically:

After release, cybersecurity testing should be performed at regular intervals commensurate with the risk (e.g., annually) to ensure that potential vulnerabilities are identified and able to be addressed prior to their ability to be exploited.

We generally suggest that if you need penetrating testing that you repeat it annually. That said, this would most likely be a finding in an audit, and wouldn’t affect your pre-market submissions.

How to Learn More 🔗

What FDA guidance apply to medical-device cybersecurity? 🔗

There are two essential FDA guidance documents that apply to medical devices:

1. 2023 FDA Guidance - Cybersecurity in Medical Devices: Quality System Considerations and Content of Premarket Submissions This guidance is a good place to start.
2. 2016 FDA Guidance - Postmarket Management of Cybersecurity in Medical Devices This guidance goes into more detail about post-market surveillance, but there is overlap between the two.

There is also a short, older guidance about cybersecurity of OTS software.

What standards apply to medical-device cybersecurity? 🔗

The FDA has recognized several standards (search the FDA database using the “security” keyword and you can see most of them). The two we feel are the most relevant are:

1. ANSI/AAMI SW96:2023 Standard for medical device security—Security risk management for device manufacturers It describes how to create a security risk management process that’s similar to ISO 14971 and interacts with the safety risk management process. This builds on the AAMI TIR57 and AAMI TIR97, which are both technical reports.
2. IEC 81001-5-1:2021 Health software and health IT systems safety, effectiveness and security — Part 5-1: Security — Activities in the product life cycle It defines the activities you need to follow to build a secure device, similar to how IEC 62304 defines the activities you need to follow to build a safe device. The FDA suggests following a standard such as this in their 2023 guidance.

Examples 🔗

NOTE: I started adding this example, but I haven’t had a chance to finish writing it out. Still, I figured it was useful already in its current state.

Threat Model of a Common Radiology AI Deployment 🔗

Overview 🔗

This example is of a common architecture we see with our AI-enabled radiology software clients. It involves two primary software components. One sits within the hospitals network and is responsible for receiving medical images, anonymizing them, and forwarding them along to the cloud for processing. The cloud component processes the data, stores a copy online for the company to review and assess, and sends the results back to the local node. There, the local node generates a report

System Items 🔗

ID	Name	Type	Description
SI-1	DICOM Router	External Entity
SI-2	Local Node	Process
SI-3	Database	Data Store
SI-4

Asset Inventory 🔗

Asset	Location	Loss of Availability Impact	Loss of Confidentiality Impact	Loss of Integrity Impact

Connection Inventory 🔗

ID	Description	Authentication	Communication Protocol	Purpose

Global System View 🔗

Security Use Case View 🔗

Patchability and Updatability View 🔗

Stride Threats 🔗

Revision History 🔗

Date	Summary of Change
4 August 2023	Initial Version; It’s still rough around the edges, but useful enough to publish.
19 December 2023	Add more details about vulnerability testing and penetration testing, taking into account ideas from IEC 81001-5-1:2001
22 January 2024	Add a section about fuzz testing
30 June 2024	Added more details in the threat modeling document; fix various typos; add comments about multi-patient harm; add details about evaluating security risks
26 September 2024	Add more questions about cybersecurity testing
7 November 2024	Clarified the threat modeling questions; added an example threat model