2017 FDA Guidance - Software as a Medical Device (SAMD): Clinical Evaluation

About this Transcript 🔗

This document is a transcript of an official FDA (or IMDRF) guidance document. We transcribe the official PDFs into HTML so that we can share links to particular sections of the guidance when communicating internally and with our clients. We do our best to be accurate and have a thorough review process, but occasionally mistakes slip through. If you notice a typo, please email a screenshot of it to Miljana Antic at mantic@innolitics.com so we can fix it.

Preamble 🔗

Document issued on December 8, 2017.

The draft of this document was issued on October 14, 2016.

For questions about this document, contact the Office of the Center Director at 301-796-6900 or the Digital Health Program at digitalhealth@fda.hhs.gov.

Contains non-binding guidance.

IMDRF/SaMD WG/N41FINAL:2017.

1.0 Executive Summary 🔗

This document is the fourth issued by the International Medical Device Regulatory Forum (IMDRF) that provides a path for global regulators to converge on terminology, a risk-based framework, an understanding of quality management system principles, and in this document, an approach to making Software as a Medical Device (SaMD) clinically meaningful to users¹. This document focuses on the activities needed to clinically evaluate SaMD -- and relies on the reader to gain knowledge from the previous documents as a pre-requisite to this document.

This document, and previous documents, provides harmonized principles for individual jurisdictions to adopt based on their own regulatory framework. They are not regulations.

This document describes a converged approach for planning the process for clinical evaluation of a SaMD (software with a medical purpose as defined in *SaMD N10*[1]**², as illustrated in Figure 1, to establish that:

• There is a valid clinical association between the output of a SaMD and the targeted clinical condition (to include pathological process or state); and
• That the SaMD provides the expected technical and clinical data.

The knowledge gained from previous documents is critical to the understanding of information in this document. This document builds on previously introduced vocabulary, risk-based considerations, and SaMD lifecycle processes and activities to help emphasize the clinical considerations essential to the success and adoption of SaMD as illustrated in Figure 2.

2.0 Background 🔗

The IMDRF has acknowledged that software is an increasingly critical area of healthcare product development and has developed a series of documents concerning the definition, the categorization, and the application of quality systems principles of SaMD.

In 2013, IMDRF’s SaMD Working Group released SaMD N10[1] Software as a Medical Device (SaMD): Key Definitions to create a standard terminology for SaMD. The following year, IMDRF adopted SaMD N12[2] Software as a Medical Device: Possible Framework for Risk Categorization and Corresponding Considerations which proposes a method for categorizing SaMD based on the seriousness of the condition the SaMD is intended to address. In 2015, the SaMD Working Group published SaMD N23[3] Software as a Medical Device (SaMD): Application of Quality Management System, outlining how manufacturers should follow Quality Management System (QMS) Principles for medical devices as well as good software engineering practices.

Knowledge of the previous three IMDRF SaMD documents is a prerequisite for readers of this document.

This document, and previous documents, provides harmonized principles for individual jurisdictions to adopt based on their own regulatory framework. They are not regulations.

The goal, strategy, principles and concepts, and implementation pathway for a harmonized SaMD framework are illustrated below in Figure 3.

3.0 Introduction 🔗

The International Medical Device Regulators Forum (IMDRF) seeks to establish a common and converged understanding of clinical evaluation and principles for demonstrating the safety, effectiveness and performance of SaMD.

As illustrated in Figure 4, this document represents a global harmonization effort to articulate this process.

The document further explains that:

• Clinical evaluation should be an iterative and continuous process as part of the quality management system for medical devices (See SaMD N23[3]**³ for more information);
• Certain SaMD may require independent review of the results of the clinical evaluation, akin to peer review or design review, to ensure that the SaMD is clinically meaningful to users. The level of evaluation and independent review should be commensurate with the risk posed by the specific SaMD (See SaMD N12[2]for more information); and
• Software is unique in its level of connectivity, which may allow the continuous monitoring of the safety, effectiveness, and performance of SaMD. This document encourages manufacturers to use this feature to understand and modify software based on real-world performance. (See 9.0 Pathway for Continuous Learning Leveraging Real World Performance Data for more information).

Healthcare decisions increasingly rely on information provided by the output of SaMD where these decisions can impact clinical outcomes and patient care. As such, global regulators expect that performance metrics for a SaMD have a scientific level of rigor that is commensurate with the risk and impact of the SaMD to demonstrate assurance of safety, effectiveness, and performance.

4.0 Scope 🔗

This document focuses on the activities needed to clinically evaluate SaMD -- and relies on the reader to gain knowledge from the previous documents as a pre-requisite to this document. Specifically, this document:

• Expects readers to have knowledge of the vocabulary, approach, and common thinking of previous IMDRF SaMD documents;
• Expects readers to understand that the concepts are limited to SaMD, as defined in *SaMD* *N10*[1], which focuses on software with a medical purpose; and
• Refers to – and paraphrases as needed -- previous Global Harmonization Task Force (GHTF3) and IMDRF documents that provide a common understanding and application of terminology, concepts and principles for a clinical evaluation that demonstrates the performance metrics of a SaMD.

This document does NOT exhaustively address all regulatory requirements relevant to SaMD, which may vary by jurisdiction (e.g., informed consent, pre-market regulatory review). In addition, this document does not repeat the following concepts from previous SaMD documents:

• The definition of a SaMD (SaMD N10[1]);
• Examples of SaMD (SaMD N12[2]);
• Where a SaMD fits in the risk categorization and the descriptions of the risk categories (SaMD N12[2]); and
• Which Quality Management principles are appropriate for SaMD (SaMD N23[3]).

5.0 Definitions 🔗

5.1 Clinical Evaluation of a SaMD 🔗

For purposes of this document “Clinical evaluation of a SaMD” is defined as a set of ongoing activities conducted in the assessment and analysis of a SaMD’s clinical safety, effectiveness and performance as intended by the manufacturer in the SaMD’s definition statement.

This definition is consistent with prior SaMD documents and is adapted from GHTF SG5 N2R8:2007[8*\]*.

Clinical Evaluation
The assessment and analysis of clinical data pertaining to a medical device to verify the clinical safety, performance and effectiveness of the device when used as intended by the manufacturer.

Figure 5- Clinical Evaluation

5.2 Valid Clinical Association of a SaMD 🔗

For purposes of this document, valid clinical association, also known as scientific validity, is used to refer to the extent to which the SaMD’s output (concept, conclusion, measurements) is clinically accepted or well-founded (based on an established scientific framework or body of evidence⁴), and corresponds accurately in the real world to the healthcare situation and condition identified in the SaMD definition statement.

Valid Clinical Association
Is there a valid clinical association between your SaMD output and your SaMD’s targeted clinical condition?

Figure 6- Valid Clinical Association

A valid clinical association is an indicator of the level of clinical acceptance and how much meaning and confidence can be assigned to the clinical significance of the SaMD’s output in the intended healthcare situation and the clinical condition/physiological state.⁵

5.3 Analytical / Technical Validation of a SaMD 🔗

Analytical validation measures the ability of a SaMD to accurately, reliably and precisely generate the intended technical output from the input data. Said differently, analytical validation:

• Confirms and provides objective evidence that the software was correctly constructed – namely, correctly and reliably processes input data and generates output data with the appropriate level of accuracy, and repeatability and reproducibility (i.e., precision); and

Analytical Validation
Does your SaMD correctly process input data to generate accurate, reliable, and precise output data?

Figure 7-Analytical Validation

• Demonstrates that (a) the software meets its specifications and (b) the software specifications conform to user needs and intended uses.

The analytical validation is generally evaluated and determined by the manufacturer during the verification and validation phase of the software development lifecycle using a QMS.

Analytical validation is necessary for any SaMD.

5.4 Clinical Validation of a SaMD 🔗

Clinical validation measures the ability of a SaMD to yield a clinically meaningful output associated to the target use of SaMD output in the target health care situation or condition identified in the SaMD definition statement. Clinically meaningful means the positive impact of a SaMD on the health of an individual or population, to be specified as meaningful, measurable, patient-relevant clinical outcome(s), including outcome(s) related to the function of the SaMD (e.g., diagnosis, treatment, prediction of risk, prediction of treatment response), or a positive impact on individual or public health.

Clinical Validation
Does use of your SaMD’s accurate, reliable, and precise output data achieve your intended purpose in your target population in the context of clinical care?

Figure 8-Clinical Validation

Clinical validity is evaluated and determined by the manufacturer during the development of a SaMD before it is distributed for use (pre-market) and after distribution while the SaMD is in use (post-market).

Clinical validation of a SaMD can also be viewed as the relationship between the verification and validation results of the SaMD algorithm and the clinical conditions of interest. Clinical validation is a necessary component of clinical evaluation for all SaMD and can be demonstrated by either:

• Referencing existing data from studies conducted for the same intended use;
• Referencing existing data from studies conducted for a different intended use, where extrapolation of such data can be justified; or
• Generating new clinical data for a specific intended use.

Clinical validation is necessary for any SaMD.

6.0 General Principles and Context of SaMD Clinical Evaluation Process 🔗

A SaMD can best be described as software that utilizes an algorithm (logic, set of rules, or model) that operates on data input (digitized content) to produce an output that is intended for medical purposes as defined by the SaMD manufacturer (Figure 9). The risks and benefits posed by SaMD outputs are largely related to the risk of inaccurate or incorrect output of the SaMD, which may impact the clinical management of a patient.

6.1 SaMD Definition Statement and SaMD Category 🔗

The SaMD definition statement, as defined in SaMD N12**[2], is used by the SaMD manufacturer to identify the intended medical purpose of the SaMD (treat, diagnose, drive clinical management, inform clinical management), to state the healthcare situation or condition that the SaMD is intended for (critical, serious, non-serious), and to describe the core functionality of the SaMD.

The SaMD manufacturer will use the factors identified in the SaMD definition statement to determine the category of a SaMD in the SaMD categorization framework as illustrated in Figure 10.

State of Healthcare Situation or Condition	Significance of information provided by SaMD to the healthcare decision
	Treat or Diagnose	Drive Clinical Management	Inform Clinical Management
Critical	IV	III	II
Serious	III	II	I
Non-Serious	II	I	I

Figure 10 - SaMD N12[2] Framework

6.2 Clinical Evaluation Processes 🔗

A SaMD manufacturer is expected to implement on-going lifecycle processes to thoroughly evaluate the product’s performance in its intended market. As part of normal new product introduction processes, prior to product launch (pre-market) the manufacturer generates evidence of the product’s accuracy, specificity, sensitivity, reliability, limitations, and scope of use in the intended use environment with the intended user, and generates a SaMD definition statement. Once the product is on the market (post-market), as part of normal lifecycle management processes, the manufacturer continues to collect real world performance data (e.g., complaints, safety data), to further understand the customer’s needs to ensure the product is meeting those needs, and to monitor the product’s continued safety, effectiveness and performance in real-world use. This real world performance data allows the manufacturer to identify and correct any problems, support future expansions in functionality, meet anticipated user demands, or improve the effectiveness of the device.

Product lifecycle activities, which include clinical evaluation activities as illustrated in Figure 11, should follow appropriate planning processes as part of an organization’s lifecycle activities and processes, as described in *SaMD* *N23*[3].

Risk assessment done as part of the SaMD’s lifecycle activities and processes should also be considered when conducting clinical evaluation.

7.0 SaMD Clinical Evaluation Process Flow Chart 🔗

Clinical evaluation is a systematic and planned process to continuously generate, collect, analyze, and assess the clinical data⁶ pertaining to a SaMD in order to generate clinical evidence verifying the clinical association and the performance metrics of a SaMD when used as intended by the manufacturer. The quality and breadth of the clinical evaluation is determined by the role of the SaMD for the target clinical condition, and assures that the output of the SaMD is clinically valid and can be used reliably and predictably.

This section uses simple steps to help SaMD manufacturers through the approach to generating evidence for the clinical evaluation of a SaMD and provides links to techniques, definitions and sources that are available to help a SaMD manufacturer generate appropriate evidence.

Note: The examples used are not intended to be comprehensive. All data sources and statistical methods should be tailored to the specific SaMD and its intended use.

① Valid Clinical Association:

Is there a valid clinical association between your SaMD output, based on the inputs and algorithms selected, and your SaMD’s targeted clinical condition?

Step 1: Verify that the association between the SaMD output and the targeted clinical condition is supported by evidence.

Note: All SaMD should demonstrate a valid clinical association.

Question: How do I “generate evidence”?

You can verify by using existing evidence or you can verify by generating new evidence.

② Analytical Validation:

Does your SaMD meet technical requirements?

Step 1: Generate evidence that shows that the output of your SaMD is technically what you expected.

Note: All SaMD should demonstrate analytical validation.

Question: How do I “generate evidence”?

You can generate evidence during verification and validation activities as part of your quality management system or as part of your good software engineering practices, or by generating new evidence through use of curated databases or use of previously collected patient data.

③ Clinical Validation:

Does your SaMD generate clinically relevant outputs?

Step 1: Generate evidence that shows your:

• SaMD has been tested in your target population and for your intended use; and that
• Users can achieve clinically meaningful outcomes through predictable and reliable use.

Note: All SaMD should demonstrate clinical validation.

Question: How do I “generate evidence”?

You can generate evidence to validate clinical significance during verification and validation activities as part of your quality management system or as part of your good software engineering practices, referencing existing data sources from studies conducted for the same intended use. Where available data references studies conducted for a different intended use, extrapolation or generation of new clinical data may be required.

7.1 Considerations for Generating and Assessing Evidence 🔗

Being able to generate evidence to demonstrate the valid clinical association, analytical validation and clinical validation of a SaMD is essential to establishing the SaMD’s value for users. The degree of evidence generation needed for a given SaMD will depend on parameters including but not necessarily limited to the:

• Maturity of evidence underlying the clinical association; and
• Confidence in the evidence, as applied to a specific SaMD.

The purpose of the assessment of the evidence is to select information based on its merits and limitations to demonstrate that the clinical evaluation evidence is high-quality, relevant, and supportive of the SaMD intended use.

For example, an assessment of clinical association would classify a SaMD as having either a:

1. Well-established clinical association: These SaMD have outputs with well-documented association as identified in sources such as clinical guidelines, clinical studies in peer reviewed journals, consensus for the use of the SaMD, international reference materials or other similar well-established comparators in terms of previously marketed devices / SaMD; or a
2. Novel clinical association: These SaMD may involve new inputs, algorithms or outputs, new intended target population, or new intended u An example may include the combination of non-standard inputs such as mood or pollen count, with standard inputs such as gait, blood pressure or other physiological and environmental signals using novel algorithms to detect early onset of a deterioration of health or diagnosis of a disease.

What if I can’t generate evidence to demonstrate ①, ②, or ③?

• Perform ongoing data analysis
• Modify your intended use to one that can be supported by available evidence · Modify the target clinical association
• Make changes to the software

8.0 Importance of Independent Review of a SaMD’s Clinical Evaluation 🔗

SaMD categories are based on the levels of impact on the patient or public health where accurate information provided by the SaMD is important to treat or diagnose, drive clinical management or inform clinical management. For additional information on SaMD categorization, please see Section 7.0 in SaMD N12[2**]. As part of the risk-based approach, and subject to individual jurisdiction’s laws, independent review of clinical evidence of certain low-risk SaMD may be less important and the manufacturer may ‘self-declare’ the appropriateness of the evidence. Again, subject to individual jurisdiction’s laws, independent review of clinical evidence of more high-risk SaMD is more important in providing users the confidence in the SaMD’s performance metrics, including but not limited to, identification of design errors or limitation, broadening technical competence, testing the appropriateness of assumptions, and management of bias.

The recommendation for independent review highlights where the evidence generated from the clinical evaluation of the SaMD should be reviewed by someone who has not been significantly involved in the development of the SaMD, and who does not have anything to gain from the SaMD, and who can objectively assess the SaMD’s intended purpose and the conformity with the overall clinical evaluation evidence. The level of clinical evaluation and importance of independent review should be commensurate with the risk posed by the SaMD. This document recommends where independent review is more or less important.

Figure 13 illustrates where independent review is more or less important. In the figure, the red, vertical dividing line differentiates where independent review is less important and where independent review is more important for different SaMD categories. Independent review is more important for SaMD that ‘Treats/Diagnoses Serious and Critical’ health care situations and conditions and SaMD that ‘Drives Critical’ health care situations and conditions. Independent review in this document does not necessarily imply regulatory review but instead demonstrates the concept where independence in review of the results is important.

For purposes of this document ‘less important’ independent review is analogous to the concept of design review performed in the QMS system. Less important independent reviews can be conducted by individuals within the company or by utilizing outside experts.

For purposes of this document ‘more important’ independent review may be conducted by outside experts such as formal consultation with regulators, third parties on behalf of regulators, or the editorial board of a peer-reviewed journal, but may also be conducted by “non-conflicted” internal expert reviewers without significant involvement in the development of the SaMD.

9.0 Pathway for Continuous Learning Leveraging Real World Performance Data 🔗

SaMD may leverage connectivity between devices, and people to continuously monitor the safety, effectiveness and performance of the SaMD.

A SaMD manufacturer may have a hypothesis about future functionality and intended use of a SaMD that may be informed by continuously collecting and analyzing data on use of the SaMD in a post-market setting. Monitoring real world performance data can help the SaMD functionality and intended use evolve after initial introduction into the market. Such data may include post-market information such as safety data, results from performance studies, on-going clinical evidence generation for medical devices, new research publications / results that support or strengthen the clinical association of the SaMD output to a clinical condition, or direct end-user feedback, that can help the SaMD manufacturer understand the real world performance of the SaMD. This may lead to a change to the SaMD definition statement if supported by the clinical evidence generated through clinical evaluation leveraging real world performance data from the continuous monitoring as illustrated in Figure 14.

Learning may impact the original category of a SaMD in the following ways:

• Real world performance data may provide evidence that the analytical or clinical validity of a SaMD is superior to the performance measures initially evaluated by the SaMD manufacturer, or
• Real world performance data may provide evidence that analytical or clinical validity of a SaMD is inferior to the performance measures initially evaluated by the SaMD manufacturer.

As additional clinical evidence is gathered to confirm the hypothesis and create and support new intended use, the SaMD manufacturer will update the clinical evaluation and generate a new definition statement. Then the cycle repeats.

This document encourages SaMD manufacturers to leverage SaMD’s capability to capture real-world performance data to understand user interactions with the SaMD, and conduct ongoing monitoring of analytical and technical performance to support future intended uses.

9.1 Considerations for Continuous Learning Leveraging Real World Performance Data 🔗

• SaMD should facilitate post-market information gathering to allow for disablement of existing or enablement of new functionality within the SaMD.
• It is not necessary for the collection of real world performance data by the SaMD manufacturer to rely on the active involvement of the end user. The SaMD manufacturer should aim to impose the least burdensome approach possible in its data collection and leverage the capability of SaMD to collect clinical evidence.
• With ongoing clinical evaluation the risk categorisation may potentially change, necessitating a change in the SaMD definition statement.

• Real world performance data including post-market information may not be sufficient to generate complete clinical evidence necessary for a change to the SaMD definition statement; as such the SaMD manufacturer should appropriately take into account other clinical evaluation steps required to support the change in SaMD definition statement.
• During the continuous learning across the life cycle, SaMD manufacturers should consider the recommendations in the previous section on independent review when new information changes the category of the SaMD as illustrated in Figure 15.
• The “continuous learning” referred to here is not “machine learning software” (i.e., where software device keeps learning automatically after it has been released into the market); rather it refers to collecting post-market information.
• Manufacturers should appropriately review the post-market information collected to determine if there are any changes to the safety, effectiveness or performance, or possible impact on benefits and risks of the SaMD that would indicate a need for a design change or a labeling change regarding contraindications, warnings, precautions or instructions for use. The labeling should identify limitations of the SaMD relevant to its clinical performance and interpretation of its output in a way that is understood by end users. The assessment of post-market information may also lead to a change of intended use (e.g., expansion, modification, or restriction).

NOTE: A change to the SaMD definition statement may be subject to regulatory requirements in the individual jurisdiction and a SaMD manufacturer should consult with the regulatory authorities in their jurisdiction.

Appendix – Comparison of SaMD Clinical Evaluation Process to Process for Generating Clinical Evidence for IVD Medical Devices in GHTF/SG5/N7:2012[13] 🔗

References 🔗

IMDRF Documents: 🔗

1. SaMD N10: Software as a Medical Device (SaMD): Key Definitions http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-131209-samd-key-definitions-140901.pdf
2. SaMD N12: Software as a Medical Device (SaMD): Possible Framework for Risk Categorization and Corresponding Considerations *-*http://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-140918-samd-framework-risk-categorization-141013.pdf
3. SaMD N23: Software as a Medical Device (SaMD): Application of Quality Management System *-* *h*ttp://www.imdrf.org/docs/imdrf/final/technical/imdrf-tech-151002-samd-qms.pdf

GHTF Documents: 🔗

1. GHTF Pre-market: GHTF Study Group 1 documents - http://www.imdrf.org/documents/doc-ghtf-sg1.asp
2. GHTF Post-market: GHTF Study Group 2 documents-http://www.imdrf.org/documents/doc-ghtf-sg2.asp
3. GHTF SG3 N18:2010: Quality management system –Medical Devices – Guidance on corrective action and preventive action and related QMS processes *-*http://www.imdrf.org/docs/ghtf/final/sg3/technical-docs/ghtf-sg3-n18-2010-qms-guidance-on-corrective-preventative-action-101104.pdf
4. GHTF SG5 N1R8:2007: Clinical Evidence – Key Definitions and Concepts- http://www.imdrf.org/docs/ghtf/final/sg5/technical-docs/ghtf-sg5-n1r8-clinical-evaluation-key-definitions-070501.pdf
5. GHTF SG5 N2R8:2007: Clinical Evaluation -http://www.imdrf.org/docs/ghtf/final/sg5/technical-docs/ghtf-sg5-n2r8-2007-clinical-evaluation-070501.pdf
6. GHTF SG5 N3:2010: Clinical Investigations -http://www.imdrf.org/docs/ghtf/final/sg5/technical-docs/ghtf-sg5-n3-clinical-investigations-100212.pdf
7. GHTF SG5 N4: Post-Market Clinical Follow-up Studies -http://www.imdrf.org/docs/ghtf/final/sg5/technical-docs/ghtf-sg5-n4-post-market-clinical-studies-100218.pdf
8. GHTF SG5 N6:2012: Clinical Evidence for IVD medical devices – Key Definitions and Concepts -- http://www.imdrf.org/docs/ghtf/final/sg5/technical-docs/ghtf-sg5-n6-2012-clinical-evidence-ivd-medical-devices-121102.pdf
9. GHTF SG1 N68:2012: Essential Principles of Safety and Performance of Medical Devices http://www.imdrf.org/docs/ghtf/final/sg1/technical-docs/ghtf-sg1-n68-2012-safety-performance-medical-devices-121102.pdf
10. GHTF SG5 N7:2012: Clinical Evidence for IVD medical devices - Scientific Validity Determination and Performance Evaluation-http://www.imdrf.org/docs/ghtf/final/sg5/technical-docs/ghtf-sg5-n7-2012-scientific-validity-determination-evaluation-121102.pdf
11. GHTF SG1 N70:2011: Label and Instructions for Use for Medical Devices-http://www.imdrf.org/docs/ghtf/final/sg1/technical-docs/ghtf-sg1-n70-2011-label-instruction-use-medical-devices-110916.pdf
12. GHTF SG2 N79R11:2009: Medical Devices: Post Market Surveillance: National Competent Authority Report Exchange Criteria and Report Form *-*http://www.imdrf.org/docs/ghtf/final/sg2/technical-docs/ghtf-sg2-n79r11-medical-devices-post-market-surveillance-090217.pdf
13. GHTF SG5 N8:2012: Clinical Evidence for IVD Medical Devices - Clinical Performance Studies for In Vitro Diagnostic Medical Devices http://www.imdrf.org/docs/ghtf/final/sg5/technical-docs/ghtf-sg5-n8-2012-clinical-performance-studies-ivd-medical-devices-121102.pdf

International Standards: 🔗

1. ISO 14155:2011: Clinical investigation of medical devices for human subjects -- Good clinical practice *-*http://www.iso.org/iso/home/store/catalogue_ics/catalogue_detail_ics.htm?csnumber=45557
2. ISO 14971:2007: Application of risk management to medical devices *-*http://www.iso.org/iso/en/CatalogueDetailPage.CatalogueDetail?CSNUMBER=31550&ICS 1=11&ICS2=40&ICS3=1
3. ISO 62304/A1: 2015: Medical device software – Software life-cycle processes -https://www.iso.org/standard/64686.html
4. IEC 62366-1:2015: Medical Devices – Part 1: Application of usability engineering to medical devices - https://www.iso.org/standard/63179.html
5. IEC 80002-1:2009: Medical device software -- Part 1: Guidance on the application of ISO 14971 to medical device software -http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=54146
6. ISO 82304-1:2016: Health software – Part 1: General requirements for product safety -https://www.iso.org/standard/59543.html

Glossary 🔗

Algorithm -- a finite set of instructions (or rules) that defines a sequence of operations for solving a particular computational problem for all problem instances for a problem set.

Analytical Validation -- measures the ability of a SaMD to accurately and reliably generate the intended technical output, from the input data.

Basic Programming -- problem-solving process used to create a computer program.

Claim -- the objective intent of the manufacturer regarding the use of a product, process or service as reflected in the specifications, instructions and information provided by the manufacturer.

(Also see Intended Use / Purpose)

Additional resources: see Section 4.0 in GHTF SG1 N68:2012[12]

Clinical Association -- refers to the extent to which the SaMD’s output (concept, conclusion, measurements) is clinically accepted or well founded (existence of an established scientific framework or body of evidence) that corresponds accurately in the real world to the healthcare situation and condition identified in the SaMD definition statement.

(Also see Scientific Validity)

Clinical Considerations -- aspects that can raise or lower the potential to create hazardous situations to patients.

Additional resources: see Sections 4.0 and 9.1 in SaMD N12[2]

Clinical Data -- defined as safety and/or performance information that is generated from the clinical use of a medical device.

Additional resources: see GHTF SG5 N1R8:2007[7]

Clinical Evaluation -- the assessment and analysis of clinical data pertaining to a medical device to verify the clinical safety, performance and effectiveness of the device when used as intended by the manufacturer.

Additional resources: see GHTF N2R8:2007[8*\]*

Clinical Evidence -- an important component of the technical documentation of a medical device, which along with other design verification and validation documentation, device description, labelling, risk analysis and manufacturing information, is needed to allow a manufacturer to demonstrate conformity with the Essential Principles.

Additional resources: see Section 7.5 in SaMD N23[3] , and GHTF SG5 N8:2012[16*\]*, GHTF SG5 N6:2012[11], GHTF SG5 N1R8:2007[7]

Clinical Performance -- the ability of a device to yield results that are correlated with a particular clinical condition/physiological state in accordance with target population and intended user.

(**Also see Clinical Validation)

Additional resources: see Section 4.0 in GHTF SG1 N68:2012[12**]

Clinical Research -- use of clinical data generated through clinical use.

Additional resources: see Section 6.2 in GHTF G5 N2R8:2007[8*\]*

Clinical Trials -- A properly conducted clinical investigation, including compliance to the clinical investigation plan and local laws and regulations, ensures the protection of human subjects, the integrity of the data and that the data obtained is acceptable for the purpose of demonstrating conformity to the Essential Principles.

Additional resources: see Section 6 in GHTF SG5 N3:2010[9**]

Clinical Usability -- the means by which the user and a computer system interact, in particular the use of input devices and software and the evaluation of safety considerations for device users, use environments and user interfaces.

Additional resources see ISO/IEC 62366-1:2015[20], SaMD N12[2]Section 4.0, SaMD N23[3] Section 7.2 and 8.4

(**Also see Usability, User Interface)

Clinical Validation -- measures the ability of a SaMD to yield a clinically meaningful output associated to the target use of SaMD output in with the target health care situation or condition identified in the SaMD definition statement.

(**Also see Clinical Performance)

Continuous Monitoring -- collecting post-market information. Additional resources: see Section 7.5 in SaMD N23[3]

Confidence Interval -- An interval about a point estimate that quantifies the statistical uncertainty in the true value being estimated (e.g. an accuracy metric) due to variability in the subject/sample selection process. A 1 – α level confidence interval contains the true value in 100(1 – α) % of applications, but in any given application either contains it or does not.

Additional resources: see Section 7.4 in GHTF SG5 N8:2012[16*\]*

Critical (situation or condition) -- situations or conditions where accurate and/or timely diagnosis or treatment action is vital to avoid death, long-term disability or other serious deterioration of health of an individual patient or to mitigating impact to public health. Additional resources: see Section 5.2.1 in in SaMD N12[2]

Definition Statement -- clear and strong statement about intended use that explains how the SaMD meets one or more of the purposes described in the definition of a medical device and describes the SaMD's core functionality.

Additional resources: see Section 6.0 in in SaMD N12[2]

Diagnose (SaMD output to***)*** -- information provided by the SaMD will be used to take an immediate or near term action.

(**Also see Treat (SaMD output to))

Additional resources: see Section 5.1.1 in in SaMD N12[2]

Drive Clinical Management (SaMD output to) -- the information provided by the SaMD will be used to aid in treatment, aid in diagnoses, to triage or identify early signs of a disease or condition will be used to guide next diagnostics or next treatment interventions. Additional resources: see Section 5.1.2 in SaMD N12[2]

Effectiveness -- when it can be determined that a device, based upon valid scientific evidence, that in a significant portion of the target population, the use of the device for its intended uses and conditions of use, when accompanied by adequate directions for use and warnings against unsafe use, will provide clinically significant results.

(**Also see Safety, Performance)

Functionality -- identifies the critical features/functions of the SaMD that are essential to the intended significance of the information provided by the SaMD to the healthcare decision in the intended healthcare situation or condition.

Additional resources: see Sections 6.0, 7.3, 8.2, 9.1, and 10.1 in SaMD N12[2]

Global Harmonization Task Force -- was a voluntary group of representatives from national medical device regulatory authorities and industry representatives. GHTF was disbanded in 2012 and its mission has been taken over by the IMDRF.

Hypothesis -- a supposition or proposed explanation made as a starting point for further investigation. Evidence is not necessary to form a hypothesis.

Independent Review -- the process of subjecting a work, research, or ideas to the scrutiny of others who are experts in the same field.

Inform Clinical Management (SaMD output to) -- Informing clinical management infers that the information provided by the SaMD will not trigger an immediate or near term action. Additional resources: see Section 5.1.3 in in SaMD N12[2]

Input (SaMD) -- one or several defined numeric tables or models accepted by an algorithm. (Also see Basic Programming Model, Outputs)

Intended (Medical, Purpose, Use) -- the objective intent of the manufacturer regarding the use of a product, process or service as reflected in the specifications, instructions and information provided by the manufacturer.

(**Also see Claim)

International Medical Device Regulatory Forum -- a voluntary group of medical device regulators from around the world who have come together to build on the strong foundational work of the Global Harmonization Task Force on Medical Devices (GHTF), and to accelerate international medical device regulatory harmonization and convergence.

Labeling -- the label, instructions for use, and any other information that is related to identification, technical description, intended purpose and proper use of the medical device, but excluding shipping documents.

Additional resources: see Section 4.0 in GHTF SG1 N70:2011[14]

Least Burdensome -- addressing a premarket issue that involves the most appropriate investment of time, effort, and resources.

Likelihood Ratio Negative (LR-) -- (1 – sensitivity) / specificity = ratio of the probabilities of testing negative in patients with and without disease or clinical condition. It can be interpreted as the increase in the odds of disease given a test negative result relative to the pretest odds.

Additional resources: see Section 7.2 in GHTF SG5 N7:2012[13*\]*

Likelihood Ratio Positive (LR+) -- sensitivity / (1 – specificity) = ratio of the probabilities of testing positive in patients with and without disease or clinical condition. It can be interpreted as the increase in the odds of disease given a test positive result relative to the pretest odds.

Additional resources: see Section 7.2 in GHTF SG5 N7:2012[13*\]*

Literature Search -- use of published clinical data that establishes a valid clinical association. Additional resources: see Section 6.1 in GHTF SG5 N2R8:2007[8*\]*

Machine Learning Software (Incremental) -- software device for which input data is continuously used to automatically extend the existing device's knowledge i.e. to further train the device after it has been released into the market.

Negative Predictive Value (NPV) -- proportion of test negative patients who do not have the disease or clinical condition.

Additional resources: see Section 7.2 in GHTF SG5 N7:2012[13*\]*

Non-Serious (situation or condition) -- situations or conditions where an accurate diagnosis and treatment is important but not critical for interventions to mitigate long term irreversible consequences on an individual patient's health condition or public health.

Additional resources: see Section 5.2.3 in SaMD N12[2]

Number Needed To Harm (NNH) -- number of patients that need to be treated in order have an adverse effect on one patient.

Number Needed To Treat (NNT) -- average number of patients that need to be treated in order to have an impact on one person.

Odds Ratio (OR) -- ratio of the odds of disease or clinical condition given the SaMD test result is S to the odds of disease given the SaMD test result is not S. OR is equivalent to ratio of likelihood ratio positive to likelihood ratio negative.

Output -- results obtained from an algorithm.

Performance (Essential Principles) -- a product’s behavior must not cause harm to a person, place or thing if something goes wrong

(**Also see Effectiveness, Safety)

Performance (Metrics) -- measures behaviors, activities and performance.

Performance (Real World) -- information on real-world device use and performance from a wider patient population than a more controlled study or pertinent literature, and thus provide information that cannot be obtained through such studies.

(Also see Real World Performance)

Performance (Studies) -- establish or confirm aspects of device performance which cannot be determined by analytical validation, literature and/or previous experience gained by routine testing.

Additional resources: see Section 5.0 in GHTF SG5 N8:2012[16*\]*

Positive Predictive Value (PPV) -- proportion of test positive patients who have the disease or clinical condition.

Additional resources: see Section 7.2 in GHTF SG5 N7:2012[13*\]*

Post-market Surveillance -- the practice of monitoring the safety of a medical device after it has been released on the market.

Additional resources: see GHTF Study Group 2[5*\]* documents; GHTF SG2 N79R11:2009[15**]

Pre-market -- the period prior to a product being available for purchase. Additional resources: see GHTF Study Group 1[4*\]* documents

Professional Society Guidelines -- practices and documents recommended by leading authorities; use of existing, well-established standards and/or clinical data. Additional resources: see Section 9 in GHTF SG5 N2R8:2007[8*\]*

Real World (SaMD) Evidence -- evidence derived from aggregation and analysis of real world data elements.

Real World Data -- product information generated after a product has been released to the market that can provide insight into the performance of the product used in actual clinical settings, in routine medical practice, and by regular use by consumers.

Real World Performance -- information on real-world device use and performance from a wider patient population than a more controlled study or pertinent literature, and thus provide information that cannot be obtained through such studies.

(**Also see Performance (Real World))

Risk Categorization Framework (SaMD) -- a framework to determine the category of a SaMD based on the significance of the information provided to the healthcare decision and on the state of the healthcare situation or condition that the SaMD is intended for. Additional resources: see Section 7.0 in in SaMD N12[2]

Safety -- a product should be designed and manufactured in such a way that, when used under the conditions and for the purposes intended and, where applicable, by virtue of the technical knowledge, experience, education or training, and the medical and physical conditions of intended users, they will perform as intended.

(**Also see Effectiveness, Performance)

Safety Data -- adverse events and other problems with medical devices that impact the health and safety of patients; safety data may be collected in the same activity as performance data; absence of safety issues during clinical performance testing is an indicator of safety.

Scientific Validity -- refers to the extent to which the SaMD’s output (concept, conclusion, measurements) is clinically accepted or well founded (existence of an established scientific framework or body of evidence) that corresponds accurately in the real world to the healthcare situation and condition identified in the SaMD definition statement.

(**Also see Clinical Association)

Secondary Data Analysis -- use of analyzed data collected for another purpose.

Sensitivity -- effectiveness of a SaMD to correctly identifies patients with the intended clinical disease or condition.

Additional resources: see Section 4.0 in GHTF SG5 N7:2012[13*\]*

Serious (situation or condition) -- situations or conditions where accurate diagnosis or treatment is of vital importance to avoid unnecessary interventions (e.g., biopsy) or timely interventions are important to mitigate long term irreversible consequences on an individual patient’s health condition or public health.

Additional resources: see Section 5.2.2 in SaMD N12[2]

Specificity -- correctly identifies across a range of available measurements patients that do not have the intended disease or condition.

Additional resources: see Section 4.0 in GHTF SG5 N7:2012[13*\]*

Treat (SaMD output to) -- information provided by the SaMD will be used to take an immediate or near term action.

Additional resources: see Section 5.1.1 in in SaMD N12[2]

Usability -- the means by which the user and a computer system interact, in particular the use of input devices and software and the evaluation of safety considerations for device users, use environments and user interfaces.

Additional resources see ISO/IEC 62366-1:2015[20], SaMD N12[2] Section 4.0, SaMD N23[3] Section 7.2 and 8.4

(**Also see Clinical Usability, User Interface)

User Interface -- the means by which the user and a computer system interact, in particular the use of input devices and software and the evaluation of safety considerations for device users, use environments and user interfaces.

Additional resources see ISO/IEC 62366-1:2015[20], SaMD N12[2] Section 4.0, SaMD N23[3] Section 7.2 and 8.4

(**Also see Clinical Usability, Usability)

User(s) - includes patients, healthcare providers, specialized professionals, lay users, consumers.

Validation -- confirmation through provision of objective evidence that the requirements for a specific intended use or application have been fulfilled.

Additional resources: see Section 2.8 in GHTF SG3 N18:2010[6]

Verification -- confirmation through provision of objective evidence that specified requirements have been fulfilled.

Additional resources: see Section 2.7 in GHTF SG3 N18:2010[6]

Footnotes 🔗