2025 FDA Guidance - Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices

Innolitics introduction 🔗

Innolitics provides US FDA regulatory consulting to startups and established medical-device companies. We’re experts with medical-device software, cybersecurity, and AI/ML. See our services and solutions pages for more details.

We have practicing software engineers on our team, so unlike many regulatory firms, we speak both “software” and “regulatory”. We can guide your team through the process of writing software validation and cybersecurity documentation and we can even accelerate the process and write much of the documentation for you (see our Fast 510(k) Solution).

About this Transcript 🔗

This document is a transcript of an official FDA (or IMDRF) guidance document. We transcribe the official PDFs into HTML so that we can share links to particular sections of the guidance when communicating internally and with our clients. We do our best to be accurate and have a thorough review process, but occasionally mistakes slip through. If you notice a typo, please email a screenshot of it to Mihajlo at mgrcic@innolitics.com so we can fix it.

Preamble 🔗

Document issued on December 18, 2025.

Draft document issued on December 19, 2023.

This document supersedes “Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices,” issued on August 31, 2017.

For questions about this document regarding CDRH-regulated devices, contact the Office of Clinical Evidence and Analysis at CDRHClinicalEvidence@fda.hhs.gov. For questions about this document regarding CBER-regulated devices, contact the Office of Communication, Outreach, and Development (OCOD) at 800-835-4709 or 240-402-8010, or by email at industry.biologics@fda.hhs.gov.

Contains non-binding guidance.

I. Introduction 🔗

FDA is issuing this guidance to clarify how FDA evaluates real-world data to determine whether they are of sufficient quality for generating real-world evidence that can be used in FDA regulatory decision-making for medical devices. This guidance also provides expanded and updated recommendations to FDA staff and sponsors considering using real-world evidence to support a regulatory submission for medical devices.¹

Real-world data (RWD) are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources.

Examples of RWD sources may include electronic health records (EHRs),² medical claims data, data from product and disease registries, and data gathered from other sources (such as digital health technologies (DHTs)) that can inform on health status. Some RWD sources can also be used as data collection and analysis infrastructure to support many types of study designs including randomized and non-randomized controlled trials; single-arm studies with or without comparison to an objective performance criterion, performance goal, or external control; observational studies; and hybrid designs which combine elements of multiple study designs.

Real-world evidence (RWE) is the clinical evidence regarding the usage, and potential benefits or risks, of a medical product derived from analysis of RWD.

This guidance includes factors that FDA considers important to demonstrate whether the RWD are relevant and reliable for a particular regulatory decision relating to medical devices, as well as information on how FDA intends to assess these factors. The recommendations and considerations in this guidance will apply regardless of the RWD source and encompass processes for conducting studies to generate RWE. FDA recognizes that there may be other approaches to address the considerations identified in this document. We encourage sponsors to discuss their approach with FDA, especially if the approach diverges from the recommendations in this guidance.³ For example, sponsors may submit a Q-Submission when there are questions
on specific aspects of using RWD in a particular regulatory submission, such as questions regarding the study protocol (see Section VII.B) and/or the sponsor’s approach to their relevance and reliability assessment (see Section V).

FDA recognizes and anticipates that the Agency and industry may need up to 60 days to perform activities to operationalize the recommendations within this guidance. For regulatory submissions that are currently pending with FDA after publication of the guidance, as well as those submissions received within 60 days following publication of the guidance, FDA generally does not anticipate that sponsors will be ready to include the newly recommended information outlined in the guidance in their submission. FDA, however, intends to review any such information if submitted at any time.

In general, FDA’s guidance documents do not establish legally enforceable responsibilities. Instead, guidances describe the Agency’s current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidances means that something is suggested or recommended, but not required.

II. Background 🔗

To protect and promote public health, FDA needs to understand and evaluate the available evidence related to regulated products. For medical devices, common sources of available evidence include information from non-clinical and clinical studies⁴ that are provided to FDA by a device manufacturer or sponsor of a device in a premarket or postmarket submission. FDA recognizes that a wealth of clinical data in the form of RWD are collected during the treatment and management of patients. Although these data typically have different quality controls compared to data collected within a traditional clinical study, under certain circumstances RWD may be used to generate RWE to help inform or augment FDA’s understanding of the benefit-risk profile of devices at various points in their life cycle.

Many of the considerations and best practices for generating RWE are derived from the same principles that govern generation of clinical evidence from traditional clinical studies, which are generally referred to as good clinical practice (GCP). As with all clinical evidence, FDA’s assessment of RWE provided in support of a particular regulatory decision will be evaluated as part of the totality of information available to FDA. Per 21 CFR 860.7(c)(1), “[a]lthough [a] manufacturer may submit any form of evidence to the Food and Drug Administration in an attempt to substantiate the safety and effectiveness of a device, the agency relies upon only valid scientific evidence to determine whether there is reasonable assurance that the device is safe and effective.” RWE derived from relevant and reliable RWD may constitute valid scientific evidence,⁵ depending on the study question, study design, regulatory decision, data source(s), and design and analysis of the specific dataset derived from RWD source(s), and thus may be used to support regulatory decisions (see Section V for more information on the relevance and reliability assessment). The standard of valid scientific evidence will apply, as appropriate, regardless of the source of clinical evidence provided. Additionally, FDA’s recommendations for clinical evidence will generally apply to RWE, as appropriate, and sponsors should consider the recommendations in other FDA guidances regarding design considerations and clinical evidence generation.⁶

When appropriate, use of RWD may provide an efficient means of generating the necessary clinical evidence to support regulatory decisions. Some traditional clinical studies may be narrow in scope but allow for more control of sources of error and bias.⁷ In comparison, studies that leverage RWD may be able to evaluate broader clinical experience to support the research question. Clinical evidence can be generated from studies using RWD, alone or in combination with data from traditional clinical studies (see Section VI for additional information on study design and methodologies). Using appropriate design and methodologies, sponsors can leverage the strengths of these approaches while minimizing potential weaknesses, ensuring the right balance of external and internal validity for the study and regulatory purpose.

FDA recognizes that uncertainty regarding the benefits and risks of a device may remain after completing a study using RWD, just as with studies relying on other sources of clinical data. The recommendations in this guidance are intended to provide clarity on general considerations related to the use of RWD for regulatory decision-making, consistent with existing approaches to evaluating uncertainty when making benefit-risk determinations⁸ and the least burdensome principles.⁹ When weighing premarket regulatory decisions, FDA considers the totality of evidence regarding the probable benefits and probable risks of a device, including the extent of uncertainty in the benefit-risk information. The appropriate extent of uncertainty regarding a device’s benefits and risks depends on the type of regulatory decision and its context. Assessing the relevance and reliability of the RWD, as outlined in this guidance, can help to identify the degree of uncertainty of the benefits and risks of a device, and—in some instances—can resolve uncertainty (e.g., as introduced by other sources of evidence), which should be considered during the benefit-risk determinations for the device for a given regulatory purpose. Similarly, FDA believes that RWD may provide an efficient means to generate the necessary information to support a particular regulatory decision, consistent with FDA’s least burdensome principles.

RWD may be used for various purposes in a regulatory submission, including both pre- and post-market (see Section IV.A). Use of relevant and reliable RWD to generate RWE can benefit interested parties throughout the device ecosystem, including patients, health care providers, manufacturers, and FDA. Some potential benefits of using RWD may include:

• Obtaining information from broader clinical experiences than are usually represented in traditional clinical studies.
• Providing new insights into the performance of a device.
• Improving generalizability.
• Capturing data generated over a longer period of time than would be practical in a traditional clinical study and potentially allowing for data to be gathered on longer-term outcomes.
• Evaluating outcomes that may not be feasible in traditional clinical studies.
• Providing information with comparable or even superior characteristics to information collected and analyzed through a traditional clinical study. In some cases, a traditional clinical study may be impractical or excessively challenging to conduct (e.g., ethical issues regarding treatment assignment).
• Facilitating more timely completion of postmarket requirements and commitments.
• Decreasing time to market.

In 2017, FDA issued the guidance, “Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices,” in which we described the RWD relevance and reliability factors that FDA assesses to determine if RWD are sufficient for generating RWE. Subsequently, on December 29, 2022, the Food and Drug Omnibus Reform Act of 2022 (“FDORA”) was signed into law as part of the Consolidated Appropriations Act, 2023, Pub. L. No. 117-328. Section 3629 of FDORA “Facilitating the Use of Real World Evidence” directs FDA to issue or revise existing guidance on considerations for the use of RWD and RWE to support regulatory decision-making. FDA issued the 2023 draft guidance to propose revisions to the 2017 guidance, “Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices,” to satisfy the requirement under section 3629(a)(2). This guidance also fulfills a commitment in Section V.F of the Medical Device User Fee Amendments Performance Goals and Procedures, Fiscal Years 2023 Through 2027 (MDUFA V).¹⁰ This guidance is intended to provide expanded and updated recommendations to industry and FDA staff for conducting an assessment of the relevance and reliability of RWD to generate RWE for regulatory decision-making.

III. Scope 🔗

This guidance is applicable for the use of RWE to support regulatory submissions for medical devices.¹¹

The topics covered within this guidance are framed specifically for the use of RWD and RWE in regulatory submissions for medical devices (e.g., Investigational Device Exemption (IDE), premarket notification under section 510(k) of the Federal Food, Drug, and Cosmetic (FD&C) Act (510(k)), PMA, Humanitarian Device Exemption (HDE), De Novo classification request, submissions related to post-approval study requirements or postmarket surveillance (in response to an order issued under section 522 of the FD&C Act (522 order)), Clinical Laboratory Improvement Amendments (CLIA) Waiver by Applications (CW), Dual De Novo/510(k) and CW Submissions (Duals)). The considerations included in this guidance may be applicable to supporting uses of RWD across the medical device total product life cycle (TPLC).

This guidance does not address the use of non-clinical data, adverse event reports, secondary use of data originally generated from a traditional clinical study, or systematic literature reviews, nor does it address all possible study design/conduct or analytical methodologies. While it does describe the factors that FDA considers when evaluating relevance and reliability of RWD, it does not provide a specific set of criteria or other scoring tools for determining the suitability of any specific RWD source for generating RWE for a particular regulatory decision.

This guidance should not be construed to alter or change in any way the existing evidentiary standards applicable to FDA’s regulatory decision-making. Rather, this guidance describes the circumstances under which clinical evidence generated from RWD may be used to support a variety of FDA decisions based on the existing evidentiary standards. While FDA encourages the use of relevant and reliable data to generate clinical evidence, including RWE, this guidance neither mandates use of RWD and RWE nor restricts other means of providing evidence to support regulatory decision-making. This guidance does not affect any federal, state, or local laws or regulations, or foreign laws or regulations that may be applicable to the use or collection of RWD, or that provide protections for human subjects (including informed consent requirements) or patient privacy. This guidance should be used to complement, but not supersede, other device-specific and GCP guidance documents.

IV. Regulatory Context in Which Use of RWE May be Appropriate 🔗

A. General Considerations for the Use of RWE 🔗

In general, FDA considers the use of RWD to be appropriate to generate RWE when the RWD are relevant to and reliable for informing or supporting a particular regulatory decision. It is important to understand the strengths and limitations of the underlying RWD and how these qualities impact their relevance and reliability. Similarly, a central part of FDA’s evaluation of RWE is considering the specific regulatory decision in the context of FDA’s existing evidentiary standards.

FDA recognizes that RWE can be generated from a variety of RWD sources that are primarily intended for a purpose other than research. For example, administrative claims data¹² are typically collected for purposes of billing or payment for medical care. Disease-specific RWD sources may be useful for tracking progression or outcomes of diseases, such as specific rare or poorly understood diseases. Treatment-specific RWD sources may have several purposes, including for assessment and tracking of overall outcomes, providing assessment of hospital operations, informing performance improvement initiatives, or providing risk prediction and benchmarking data for specific therapies. The suitability of the RWD source may be determined by the factors outlined in Section V and the availability of sufficient data to address the study question of interest.

FDA does not endorse one type of RWD over another. Sponsors should select the appropriate RWD sources based on their suitability to address the specific study questions. Data sources that may be considered RWD sources, if they include data that are routinely collected, include the following:¹³

• Registries¹⁴ (e.g., medical device registry);¹⁵
• EHRs;
• Administrative claims data;
• Chargemaster and/or billing data;
• Patient-generated data¹⁶ (e.g., patient-reported outcomes or performance outcomes) that are created, reported, or gathered by patients, including in-home use, potentially using wearables and other DHTs;¹⁷
• Device-generated data (e.g., implantable devices, physiological monitoring devices);
• Public health surveillance data (e.g., COVID-19¹⁸ case surveillance);
• Clinically annotated biobanks; and
• Medical device data repositories (e.g., imaging, electrocardiography databases).

Some purposes for which use of RWD may potentially be applicable in a regulatory submission include the following:

• To generate hypotheses to be tested in a clinical study;
• As a historical control, an informative prior in a Bayesian analysis of a clinical trial,¹⁹ or as one source of data in a hierarchical model or a hybrid data synthesis;
• As a concurrent control group or as a mechanism for collecting data to support marketing authorization when a registry, EHR, claims data, or some other systematic data collection mechanism exists;
• As a mechanism for training and/or ongoing monitoring of an artificial intelligence (AI)-enabled medical device throughout the device’s TPLC;²⁰
• To generate evidence to identify, demonstrate, or support the clinical validity of a biomarker or clinical outcome assessment;
• To develop algorithms used in a medical device;
• To generate (primary) clinical evidence to support marketing authorization (e.g., 510(k), PMA, HDE, or De Novo request);
• To generate evidence directly by the subject device to provide new information on safety or effectiveness;
• To generate evidence to support a determination on whether the subject device meets the statutory criteria for a CLIA waiver²¹ (e.g., CW and Duals²²);
• To generate evidence to support the interpretability of the primary clinical evidence (e.g., to demonstrate that the study population for an investigation conducted outside the United States (OUS) is representative of the U.S. population,²³ or to provide context for an adverse event observed in the clinical study);
• To generate evidence to support a petition for reclassification of a medical device under section 513(e) or (f)(3) of the FD&C Act;
• To generate evidence for expanding the labeling of a device to include additional indications for use or to update the labeling to include new information on safety and effectiveness;²⁴
• To develop a performance goal (PG) or Objective Performance Criterion (OPC) using appropriate statistical methods, such as a patient-level meta-analysis from a sufficiently relevant and reliable RWD source. As device technology evolves over time, an OPC or PG could also be updated using RWD;²⁵
• To generate evidence for postmarket surveillance. Through ongoing surveillance, signals are at times identified that suggest there may be a safety issue with a medical device. RWE may be generated using RWD to refine these signals for purposes of informing appropriate corrective actions and communication;²⁶
• To conduct post-approval studies that are imposed as a condition of device approval, as well as to potentially preclude the need for or address postmarket surveillance requirements (i.e., 522 orders²⁷ and special controls²⁸); and
• To provide postmarket data in lieu of some premarket data, consistent with FDA’s policy on balancing premarket and postmarket data collection.²⁹

B. Application of Investigational Device Exemption (IDE) Requirements in 21 CFR 812 to Clinical Studies Using RWD 🔗

An approved IDE permits a device to be shipped lawfully for the purpose of conducting investigations of the device without complying with certain other requirements of the FD&C Act that would apply to devices in commercial distribution. The purpose of this investigational exemption, as set forth in 21 CFR 812.1, “is to encourage, to the extent consistent with the protection of public health and safety and with ethical standards, the discovery and development of useful devices intended for human use, and to that end to maintain optimum freedom for scientific investigators in their pursuit of this purpose.” The IDE regulations apply to all clinical investigations of devices to determine safety and effectiveness, with certain limited exceptions.³⁰ In many cases, an approved IDE is required before initiating an investigation.³¹

Whether the collection of RWD for a legally marketed device requires an IDE depends on the particular facts of the situation. Specifically, if the device is being used in the normal course of medical practice, an IDE would likely not be required. FDA recognizes that in clinical practice this could include use of a legally marketed device for uncleared or unapproved uses, where the device is being administered or prescribed under the authority of a health care practitioner within a legitimate practitioner-patient relationship. If data collection does not impact how the device is administered, and the administration and follow-up is within the normal course of medical care, an IDE would likely not be required. For example, analyses of extant RWD (i.e., RWD already collected) involving the use in medical care of a device that was not within the cleared or approved indications for use would generally not be subject to IDE regulations. However, similar to traditional clinical studies, if data are being gathered to determine the safety and effectiveness of the device, and the process for gathering the data would influence treatment decisions, such administration of the device would likely not be within the normal course of medical practice and an IDE may be required. It is important to note that certain follow-up activities performed for the purpose of research may also warrant submission of an IDE. The following represent hypothetical examples to illustrate when an IDE application³² to FDA may or may not be required for a study with RWD.

Example 1 - IDE application to FDA required

A sponsor plans to conduct a prospective clinical study for a new use of a legally marketed device that presents a potential for serious risk to the health, safety, or welfare of the patient. The hybrid study protocol prespecifies enrollment criteria, investigational device implantation, and follow-up. To collect safety and effectiveness data, the sponsor will use protocolized data collection for short-term outcomes, and leverage administrative claims data to monitor longer-term outcomes including re-admittance or additional treatments, and discharge of study participants. Administrative claims data are RWD that are routinely collected during the normal course of clinical care and not originally intended to answer a specific research question. This study would require an IDE application to be submitted to FDA, because 1) the device is being investigated for a new use, and enrollment, implantation of the device, and follow-up are being directed by a research protocol and are not within the normal course of clinical care; and 2) the study was determined to present a potential for serious risk to the health, safety, or welfare of the patient, and therefore the device would be considered a significant risk device³³ under the IDE regulations.³⁴

Example 2 - IDE application to FDA not required

The sponsor of a marketed implant plans to study the clinical use patterns of their device, including off-label uses, to better understand the associated safety and effectiveness profile. To collect the safety and effectiveness data, the sponsor will leverage extant data generated at least one year prior to the study initiation from a hospital’s EHR system to assess the preoperative patient characteristics, operative notes, and postoperative outcomes. The EHR data used for this study are a type of RWD that are routinely collected during the normal course of clinical care and were not originally intended to answer a specific research question. This study does not require an IDE application to FDA, because 1) the use of the device and follow-up for each patient was determined as part of routine clinical care; and 2) the data used in the study are routinely collected during clinical care and the process for collecting the data did not influence treatment or care decisions.

Example 3 - IDE application to FDA not required

The sponsor of a marketed non-implanted device plans to study the clinical use patterns of their device to better understand the associated safety and effectiveness profile. This is a prospective study with a research protocol that states that physicians are to treat patients per the normal course of medical care. The protocol does not specify how patients should be treated or followed up. Routinely collected data from use of the device are recorded in the hospital’s EHR system, and for this study those data generated as a result of use of the device in a procedure that is not within its labeled use are duplicated into a study-specific module. The module includes data on patient demographics, lab results, adverse events, hospitalizations post-procedure, and reinterventions. This study does not require an IDE application to FDA, because 1) the use of the device for each patient was determined as part of routine clinical care; and 2) the data used in the study are routinely collected during clinical care and the process of collecting the data did not influence treatment or care decisions.

If an IDE application is determined to be required, FDA intends to work with the IDE sponsor to develop the least burdensome approach to facilitate the efficient generation of RWE. Note that regardless of the applicability of 21 CFR Part 812, FDA regulations at 21 CFR Part 56 (Institutional Review Board (IRB) review), Part 50 (Protection of Human Subjects), and Part 54 (Financial Disclosure) may apply to RWE generation activities, as may other federal, state, and local laws regarding human subject protections. Should a sponsor or IRB be unclear regarding the applicability of any of these regulations for a particular RWD collection activity or use, the sponsor or IRB can contact FDA at CDRHClinicalEvidence@fda.hhs.gov.

C. Application of RWD from Devices Authorized for Emergency Use Under Section 564 of the FD&C Act 🔗

Section 564 of the FD&C Act provides that FDA may, after the HHS Secretary has made a declaration of emergency or threat justifying authorization of emergency use (an “EUA declaration”), authorize the emergency use of an unapproved product³⁵ or an unapproved use of an approved product for certain emergency circumstances.

The routine clinical use of a device authorized under an EUA, when used within the scope of its authorization, is not considered to be a clinical investigation (see section 564(k) of the FD&C Act and Section IV.B for more information on the application of IDE requirements in 21 CFR Part 812 to the collection of RWD). Clinical data routinely collected from the use of a device authorized under an EUA may be considered RWD and may be used to support regulatory decision-making, if determined to be relevant and reliable for the particular study question and regulatory purpose. Generally, the recommendations in this guidance may apply to RWD from devices authorized under an EUA. Additionally, Appendix B includes an example of RWD from a device authorized under an EUA that was used in a subsequent premarket submission.

Device use pursuant to an EUA may lead to additional sources and novel uses of RWD to support FDA decision-making. We encourage sponsors to consider the recommendations in this guidance for devices authorized under an EUA (e.g., devices authorized under an EUA during the COVID-19 pandemic).

V. Assessing Data Relevance and Reliability 🔗

To determine the potential suitability of RWD to generate RWE for regulatory decision-making, the sponsor should conduct a relevance and reliability assessment (see Section VII.C), and when a sponsor submits such an assessment, FDA intends to review and evaluate that assessment. The sponsor’s assessment should address relevance and reliability of the RWD source(s) (including data elements), as well as the study design and analytic components of the study. Sponsors should understand the strengths and limitations of generating evidence from RWD to address a specific regulatory purpose and discuss these limitations in their submission. A submission relying on RWD should describe the relevance and reliability factors listed in Section V.A and V.B, as applicable.

Whether RWD are sufficiently relevant and reliable for use will, in part, depend on the particular regulatory decision for which the data have been provided. As part of the relevance and reliability assessment of an RWD dataset, if multiple RWD sources are used, the submission should address how each RWD source contributes to the relevance and reliability of the final dataset.

The data should be accurate, as complete as possible, and of adequate data quality to credibly address the question at hand. Conducting a clinical investigation in accordance with GCP provides assurance that the data and results from the clinical investigation are credible and accurate and that the rights, safety, and well-being of subjects are protected.³⁶ Best practices are generally well-established for incorporating GCP into the design and execution of traditional clinical studies. RWD have certain characteristics, including that these data are typically collected outside of a research setting, such that additional precautions should be considered for RWD studies to ensure they are similarly credible and accurate, and that appropriate patient protections are in place. Sponsors should ensure that a submission containing RWD demonstrates that sufficient precautions were considered during study conduct and that appropriate patient protections are in place. Sponsors should demonstrate the RWD are credible and accurate through the relevance and reliability evaluation.

Additionally, sponsors should ensure that RWD were collected using good data management practices and are sufficiently robust. Sponsors should also consider data related to appropriate demographic characteristics and relevant covariates of the intended use population.³⁷

The submission should demonstrate that studies using RWD were designed to mitigate potential bias. FDA recommends that a study protocol and analysis plan be created prior to analyzing RWD, regardless of whether the RWD are extant or if they are to be collected in the future. An existing RWD source may have some inherent sources of bias that could limit the relevance or reliability for drawing causal inferences between medical device exposures³⁸ and outcomes.

To help ensure the relevance and reliability of the source data, FDA recommends sponsors consider the factors contained in this section. Appendix A includes an example of elements that FDA recommends sponsors document and have available for inspection, as well as recommended elements for sponsors to include in the appropriate regulatory submission for FDA review.

A. Relevance 🔗

Relevance includes consideration of availability, timeliness, and generalizability of the RWD. When needed information is not available in one data source, sponsors may choose to provide linkage to other data source(s). Important relevance factors that FDA intends to consider in determining whether RWD are suitable for generating RWE for regulatory use include the following:

(1) Data Availability 🔗

The RWD should contain sufficient detail to capture the information needed to evaluate the question being addressed in the target population. If the sponsor determines that the RWD source is insufficient on its own, the sponsor should consider whether supplemental data sources are available and sufficient to provide any missing information necessary to address the study question.

Relevant considerations should include whether the RWD contain information on the following:

• Use of the device or other exposure in the study population. If the RWD source captures the device identifier (DI)³⁹ portion of the unique device identifier (UDI),⁴⁰ this should be documented to support robustness of the device identification and minimize exposure misclassification. If the DI is not captured within the particular RWD source, alternative methodologies for device identification may be considered for device identification (e.g., other structured data or clinical notes within the same RWD source);
• Outcome(s) of interest in the study population;
• Covariates that may impact the exposure or outcomes of interest (e.g., RWD source contains signs, symptoms, treatments, procedures, diagnoses, patient and family history, pre-existing conditions, labs, demographics, and results which may be used to construct covariates that are relevant to the study question); and

  For example, the minimum set of data fields in a registry may be insufficient for a specific study question and additional data fields may be needed for the registry to be relevant. The registry should retain information documenting the start or stop of collection during the study time frame for data fields related to the specific study question.

• Longitudinality, including longevity (the length of time that data for an individual is captured within the RWD source) and continuity of care.

  Information across the continuum of care⁴¹ (i.e., data observability) may aid the assessment of the likelihood that all exposures and outcomes of interest will be captured for regulatory decision-making.

  For example, tertiary care hospitalization data may not have adequate data availability to study outcomes that are likely to be diagnosed in an emergency for all patients, because patients are likely to go to a nearby hospital in emergencies but may travel to another location for a specialty device procedure.

(2) Linkages 🔗

Sponsors should assess whether and how data from different sources can be obtained and integrated given the potential for heterogeneity in target population characteristics, clinical practices, and coding across data sources. A description of this assessment should be provided in the regulatory submission for FDA review.

Any linkages performed within and across RWD sources should use a predefined linkage methodology⁴² that is scientifically valid, protects the privacy of individuals (e.g., privacy preserving record linkage or PPRL) whose data will be used, supports interoperability, and accounts for differences in coding and reporting across sources. The following considerations should be assessed by the sponsor and addressed in the submission, as appropriate:

• Adequacy of line-level linkages (i.e., that the same individuals are being matched), including predefined rules to check for logical consistency and value ranges to confirm that data were retrieved accurately from a linked data source; and
• Application of strategies to correct for redundant data, to resolve any inconsistencies, and assess the potential for missing data.

  Because patients typically visit multiple health care sites, especially in geographically contiguous areas, the inclusion of de-identified data from many sites creates the possibility that there will be multiple records from different health care sites for a single individual (e.g., including when new technology such as patient-generated data start to be included in the health care record). This can result in overcounts of a particular data measure. Appropriate documentation, including justification for how data (e.g., covariates, outcomes) inconsistences were resolved, should be provided. Alternatively, if some site records are not available, this can result in a collection of histories that reflect only a fraction of the patient’s total health care history.

(3) Timeliness 🔗

As with traditional clinical studies, the time between data collection and release for research should be reasonable and the RWD considered for the study should reflect the current clinical environment (e.g., RWD from before a major change in clinical practice may not be timely). Sponsors should consider changes in clinical practice and guidelines over time (e.g., criteria for disease diagnosis, cancer staging), characteristics of a condition (e.g., prevalent strain of a pathogen), and health status of the population. If data are being collected within the RWD source during the study time frame, then the sponsors should specify within the protocol the timing for RWD updates and should define the reporting schedule in the regulatory submission for FDA review. Additionally, the sponsor should provide a rationale for the reporting schedule and whether it is appropriate for the regulatory purpose. When a sponsor provides such information to FDA, FDA intends to use that information to determine whether the reporting schedule is appropriate for the regulatory submission.

(4) Generalizability 🔗

Once a study question is defined, the study sample that is selected should be adequately representative of the patients in the RWD source that are reflective of the proposed intended use population. Identifying the study sample should take into consideration how the application of inclusion/exclusion criteria will affect the overall generalizability of the RWE to the U.S. intended use population (see Section VI). If upon quantitative assessment, the study sample is shown to not be representative of the target population, then analyses should be conducted to evaluate generalizability of the study findings.

B. Reliability 🔗

Reliability includes consideration of accrual, quality, and integrity of RWD. The sponsor should attempt to obtain all information and associated documentation related to processes, procedures, and methods around data accrual (see Section V.B.1) and data quality and integrity (see Section V.B.2), as described below. If the sponsor is not able to obtain this information and associated documentation, the sponsor should identify the entity(ies) which do have access/permission to the respective processes, procedures, and methods. The sponsor should then determine the level of access which could be shared with FDA, as well as the potential for third parties to provide the needed documentation directly to FDA. The availability of the respective processes, procedures, and methods should be described in the regulatory submission for FDA review. FDA recognizes that some data sources will not allow sponsors to obtain all information and associated documentation. Although FDA does not discourage use of these data sources, we note that uncertainty may arise if FDA does not have access to all of the necessary processes, procedures, and methods. FDA notes that the sponsor should conduct the reliability assessment, with all information that could be obtained, and provide a rationale for why the RWD is reliable for the regulatory purpose.

Important reliability factors that FDA considers in determining whether the RWD are suitable for generating RWE for regulatory use include the following:

(1) Data Accrual 🔗

To ensure the reliability of the RWD source, data should be collected and processed in a consistent and methodical manner. The manner of collection may differ for newly developed RWD sources which are actively collecting data (e.g., data dictionary to provide a common definitional framework in a registry), using nationally or internationally recognized coding systems (e.g., International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM), Logical Observation Identifiers, Names, and Codes (LOINC), UDI, Current Procedural Terminology (CPT) in EHR or Claims) or custom-designed structured data capture (e.g., data capture within a device), or using unstructured data capture (e.g., narrative portion of clinical notes). Any of these approaches may be able to demonstrate sufficient reliability to support regulatory decision-making. Factors FDA intends to consider in making this determination include:

• Adequacy of information and descriptors about data sources provided in the regulatory submission for FDA review, which should include information on:
  • Data types;
  • Health care settings/environment(s);
  • Purpose of data collection;
  • How data were obtained at point of data capture;
  • How data are accessed by study team and sponsor;
  • Any data transformations, including any modifications made for privacy protection;
  • Full data dictionary or common data capture form, if applicable;
  • Device information, including types of identifiers (e.g., DI) and indication for use;
  • Completeness of fields that would typically be completed for all individuals and needed for most study questions (e.g., age, sex, DI);
  • Time frame (including common temporal framework for collection of data) and latency of the data (including the timeliness of data entry, transmission, and availability);
  • Version control of the RWD source(s); and
  • Key technical and privacy-related information. Sponsors should document routine migration of data between various sources over time (e.g., indicate the date and time of data transfers, linkages).
• Adequacy of information about data accrual methods and procedures provided in the regulatory submission for FDA review, which should include information on:
  • Data collection procedures;
  • Use of common data capture forms;
  • Common definitional frameworks;
  • Data cleaning and cross-referencing procedures;
  • The sources and technical methods used for data element capture (e.g., chart abstraction, point of care entry, EHR integration, UDI capture, data records from the device, and linkage to administrative claims data); and
  • Methods for data retrieval and processes to minimize missing data extraction, implausible values, and data quality checks in data captured at the point of care (e.g., during clinical practice for manual or automated health care data collection processes) to ensure accuracy and completeness of core data fields.

(2) Data Quality and Integrity⁴³ 🔗

When considering RWD sources for regulatory purposes, sponsors should consider the methods and systems used to help ensure sufficient data quality, including any data quality assurance plans and procedures developed for the RWD source itself. Because evaluation of RWD sources may not always permit specific line-item source verification, important factors sponsors should consider and address in a submission include:

• Quality control processes;
  • Regardless of the original purpose for collecting the RWD, procedures for data collection and quality assurance should be put into place during the data source design and development stages to optimize the reliability, quality, and usefulness of the data, as appropriate. These procedures should be described in the regulatory submission for FDA review.
  • Where appropriate, processes should include site and data monitoring, data quality audit programs, and evaluation of ongoing training programs for data collection.
  • Records regarding the assessment of adherence to the RWD source’s established data quality assurance and quality control policies and procedures should be retained.
• Assessment of completeness, accuracy, and consistency across sites and over time;
  • Data should be captured in a manner designed to minimize missingness. Missingness and out of range values should be assessed for each data element. The amount of missingness per individual (across data elements) should also be assessed, if possible. The impact of missingness should be considered and thresholds for unacceptable levels of missingness should be pre-determined. Additionally, quantitative assessment of the potential bias associated with high missingness should be performed and included in the interpretation of the study.
  • Data should be reflective of the real-world patient experience (e.g., interactions with health care, disease trajectory, outcomes) with the condition of interest.
  • Consistency of data capture should be used across sites and over time.⁴⁴ If any changes are needed (e.g., where diagnostic criteria, definitions, or clinical practice change over several years), then sponsors should document those changes and assess their impact on the study question and provide summary information in the regulatory submission for FDA review.
  • Auditing rules, methods, and the mitigation strategies used to reduce errors should be documented.⁴⁵
  • Study sample size⁴⁶ should be justified to address the study question and regulatory purpose. It should be determined based on adequate statistical power or precision to detect a clinically meaningful difference.
    • If extant RWD are used, the number of individuals may not be adequate to meet the target sample size.
    • If the number of available RWD individuals is not large enough to meet the target sample size, sponsors should consider the use of multiple existing RWD sources.
    • If the sample size could be expected to increase in the near future (e.g., device is new to market), sponsors should consider accruing more data.
    • Sponsors should account for planned statistical analysis within the study size calculations (e.g., 1:1 matching of propensity scores in a study population where 10% of individuals receive the device would remove approximately 89% of individuals with comparator from the analysis).
• Establishment and adherence to data collection, recording, and source verification procedures;
  • As with all clinical evidence generation, data provenance and traceability are important. Sponsors should plan and document all aspects of data extraction, aggregation, curation, storage, and availability for research, as described below.
  • Sponsors should ensure any automated electronic transmission of data fields to a repository (e.g., registry or data warehouse) occurs in a secure, consistent, and reproducible fashion, including applicable processes.
  • Adherence to source verification procedures and data collection and recording procedures should be documented for completeness and consistency.
  • Data checks and procedures should be prespecified to help address identified errors (e.g., in coding or interpretation of the source documentation or transformation).
  • Sponsors should describe the mitigations used to address audit findings, including data corrections.
  • Sponsors should identify the source document(s) and first instance⁴⁷ of data available to sponsor. Sponsors should generate data quality documentation from the first instance through RWD dataset(s) used to address the study question.
  • If using a common data model, sponsors should ensure documentation of the transformation of data from the original source to the common data model is retained.
  • Data audit trail, including assessment of discrepancies, should be included, as applicable.

    For extant data sources, the sponsor may have access to the initial capture of data (e.g., direct access to EHR data as obtained via data entry) or may only have access to a partially curated data source (e.g., administrative claims data, aggregated EHR). We recommend sponsors maintain information on the data audit trail from the first instance of data available to the sponsor through all aspects of data analysis. Further, sponsors should obtain as much information as possible about the audit trail from the data holder if the first instance is not at the point of data entry.

  • As with all clinical evidence generation, FDA recommends that the sponsor have access to the RWD dataset from the first instance used for the analyses throughout the regulatory decision-making process. When the RWD source is not owned by the sponsor, where participant-level data is accessible, the sponsor should attempt to obtain participant-level data for each participant. If not available, the sponsor should define the entity(ies) which do have access/permission for data entry, quality assurance, storage, aggregation or other linkage, and assessment of traceability from data entry to dataset, as applicable. Sponsors should consider the level of access which could be shared with FDA and the potential for third parties to provide participant-level data directly to FDA. The availability of data should be described in the regulatory submission for FDA review. FDA recognizes that some data sources either provide only aggregate-level outputs or will not allow sponsors to access the participant-level data. FDA does not discourage use of these data sources, and a sponsor’s inability to obtain participant-level data does not preclude FDA’s evaluation. Sponsors should describe how lack of participant-level data does or does not affect the RWE.
• Adequate patient protections (e.g., methods to protect the privacy of individuals’ health data and adherence to applicable privacy and ethics standards) established in advance of executing the study protocol; and
• Prior demonstration of RWE generation from the data source.

  Sponsors should provide documentation (including the relevance and reliability assessment) of any previous use of the same RWD source for a similar target population and/or peer-reviewed literature of RWE generation from the data source, as applicable.

VI. Considerations for Methodologies for Collection and Analysis of RWD to Generate RWE 🔗

A study using relevant and reliable RWD in a well-designed and rigorously analyzed manner may be less burdensome than a traditional clinical study. Just as traditional clinical studies should be carefully designed, studies using RWD also should undergo careful assessment before embarking on and during development of the study to ensure that the data are relevant and reliable for the study question and regulatory purpose. FDA recognizes that some regulatory decisions may not be adequately supported using RWE, for various reasons, and we therefore recommend that sponsors consider methodologies to address factors that can impact interpretability of a study using RWD.

Scientifically sound clinical study planning in advance of statistically valid analyses is important regardless of whether a study uses a traditional clinical study approach, uses only RWD, or incorporates a hybrid design. Further, just like for traditional clinical studies and in addition to the study design and analysis considerations described in Sections VI.A and VI.B, FDA recommends that a sponsor document their decisions and the associated rationale for the following:

• Whether to include randomization, concurrent, or historical controls;
• The choice of performance goals and objective performance criteria;
• Type I and type II error control;
• Data gathering or dependence on extant data;
• Bias mitigation strategies;
• Precision of outcome measures and other data elements, as applicable; and
• All other known factors pertinent to interpretation of the study results (e.g., generalizability of the RWE findings to the intended use population).

Although many of the considerations in this section for data collection and analysis are not novel in the context of clinical evidence generation, there may be unique aspects of these considerations for studies using RWD. Additionally, the information presented in this section is intended to augment, not replace, information in other FDA guidances on the design of clinical studies for regulatory decision-making. The information in this section is intended to clarify implementation of these concepts and practices when using RWD. In particular, the information below is intended to complement the recommendations for clinical studies in the FDA guidance “Design Considerations for Pivotal Clinical Investigations for Medical Devices.”

A. Methods for Study Design Using RWD 🔗

Generally, FDA does not endorse a specific type of study design for clinical studies, regardless of whether it is a traditional clinical study or uses RWD. As with all clinical evidence generation, choosing the appropriate design for studies using RWD depends on the study question, device, outcome, key covariates, and the specific study objectives or hypotheses. Additionally, sponsors should consider the regulatory purpose of the generated clinical evidence. FDA recognizes that multiple types of study designs may also be useful to generate RWE. These study designs may include:

• Single-arm studies with comparisons to external controls, objective performance criteria, or performance goals, in whole or part;
• Non-interventional studies (observational studies) (e.g., comparative cohort studies, case-control studies, self-controlled studies, and descriptive studies); and
• Randomized controlled trials using RWD to supplement one or more study arms.

B. Defining Study Design Elements 🔗

For studies using RWD, as with all clinical studies, after determining the overarching study design, the study time frame and collection of data elements should be defined, followed by a system to capture specific data elements (e.g., in a case report form). Additional data capture requirements may necessitate justification or adjudication, especially for study endpoints. FDA recommends clearly defining the individual data elements derived from the RWD source to develop study-specific RWD. Similarly, FDA recommends that sponsors show that the data elements, as defined and applied within the study design, are relevant and reliable for the particular regulatory decision. For analysis of RWD and interpretation of RWE, sponsors should have a study design that describes the study time frame, the predefined set of data elements, and a systematic consideration that the proposed data elements are all necessary for inclusion and represent all the key data elements.

(1) Study Time, Relative to Index Date 🔗

In traditional clinical studies, a participant is often enrolled into the study, has a baseline visit, and then first uses the device or has a procedure on the “index date” (see Figure 1). After that, the participant is usually followed for a period of time until a final visit. Data elements are collected at each visit, although different information may be gathered at each visit, and additional data elements may be collected outside of clinical care (e.g., via patient-generated data such as a participant diary or wearable). The participant continues to be followed through a last study visit. If a visit or other data collection is missed, then the participant may be contacted or additional questions may be asked at the next visit to gather key information.

Figure 1. Traditional clinical studies - example of study time frame relative to index date

In clinical studies using RWD where data are collected after the study is designed, it is possible that a similar visit structure and data collection will be available (e.g., within a registry). However, follow-up visits may not occur on a set schedule or more patient-generated data may be collected (e.g., via a patient diary or wearable). For extant data such as EHR or administrative claims, baseline and follow-up data may not always be collected on a set schedule; rather data collection coincides with clinical care over a period of time (see Figure 2). Individuals may also enter or exit the source database as their life situation changes (e.g., move out of a geographic area or a change in health insurance). Thus, continuity of care is an important consideration for studies using RWD.

A visual depiction, such as that exemplified in Figure 2, may be helpful in identifying the timing of collection for each data element relative to the index date, which will help to identify potential bias. Data elements that may impact the initial device use should be collected before or at the time of initiation. Outcomes of device use occur after use of the device is initiated. Additionally, the index date for the use of the comparator to the device would occur at a similar point in the progression of disease. As with a traditional clinical study, discussion of these ideas with the study team or with FDA may be aided by the visual depiction of when each data element will be assessed.

Figure 2. Clinical study using RWD - example of study time frame relative to index date

Follow-up in a study using RWD typically extends from the index date of device use until either the end of the pre-planned follow-up time or the last time identified within the RWD source. FDA considers the study end date to be the last date that participant follow-up could occur. This date is set on a day when data checks/audits can ensure that the underlying data are of sufficient quality for use in research. Any data in the RWD source indicating that a participant had subsequent care is no longer included in the study (i.e., study participation is censored on this date). Thus, similar to the first site being ready for enrollment in traditional clinical studies, FDA anticipates that the study time frame will be defined to begin on the earliest date that the first data element could be collected and extend through the latest date that the last data element could be collected.

In addition to the study design elements discussed above, any change in the standard of care, availability of the device or other treatments, or other relevant factors (e.g., change in hospital care due to a PHE) should be included on the graphical depiction. This additional information may aid in systematically capturing these time-dependent data elements and provide support for their inclusion as covariates in analyses or consideration of sensitivity analysis (e.g., assessing whether a change in ICD-CM coding from a prior edition or a major change in clinical practice affects study results).

The amount of observed time for the individuals within the study population should be long enough to adequately measure all data elements in a study – from the beginning of the baseline period through the end of the follow-up needed to assess the outcome(s) of interest – in a sample large enough to provide adequate statistical power to detect the minimal clinically important difference in the primary outcome.

(2) Development of Conceptual and Operational Definitions for the Study Population, Device, Comparator, Outcome, and Covariates 🔗

As with any clinical study, all data elements should be defined before the start of a study using RWD and should address the specific study question when valid and appropriate analytical methods are applied (i.e., the data are amenable to sound clinical and statistical analysis). A “conceptual definition” that describes the construct or feature of each data element in general or quantitative terms should be generated using a shorthand name or notation. This conceptual definition should reflect the current medical and scientific thinking regarding the variable of interest, such as: (1) clinical criteria to define a condition for population selection or as an outcome of interest or a covariate; or (2) measurement of the device or procedure to define an exposure of interest. For example, a conceptual definition might be “acute myocardial infarction (AMI)” or “AMI evidenced by increased troponin.” For a traditional clinical study, the sponsor defines the collection and timing of each data element, whether at a visit or between visits, and usually has the ability to contact the participant to limit missing data or to solicit additional information if a visit was missed. In a study using RWD, the data elements may be collected in a similar fashion (e.g., registry) or need to be defined from clinical care visits (e.g., EHR or administrative claims data) or some other algorithm (e.g., combining unstructured EHR and patient-reported data).

An “operational definition” describing all of the components needed to identify complete and accurate data elements from the data source should also be generated. While an operational definition would typically be generated in a case report form in a traditional clinical study, operational definitions in a study using RWD frequently include combining structured codes or unstructured notes (e.g., clinical notes) in an algorithm to identify presence of the data element. FDA considers the operational definition to include three components, as applicable:

• Time frame over which assessment occurs;
• Specific codes/component(s) assessed (e.g., via code lists); and
• Algorithm for combining the components (leading to positive identification or lack of identification, including assumptions underlying how event and no event occurrence are defined). If machine learning is used to define criteria, sponsors should provide a full description of data management practices including the specifications of the model/algorithm (e.g., training, tuning, and testing), data collection and the data attributable to the proposed intended use population (e.g., with respect to race, ethnicity, disease severity, sex, age, socioeconomic characteristics). Sponsors should also provide information that demonstrates the machine learning approach is appropriate for defining the criteria, such as verification and validation of the machine learning approach.

The availability of different data types in studies using RWD may make it possible to establish operational definitions that are different from those typically used in traditional clinical studies. These definitions may or may not be appropriate in the context of the study question being addressed depending on the study question and regulatory purpose. It is important to consider whether the operational definition will capture the intended concept for each data element and FDA notes that small differences in the choice of operational definition in a specific data source (e.g., requiring two diagnoses rather than one diagnosis of AMI in the example above) may have a large impact on study results (e.g., considerably decrease the identification of the disease or condition under study). These data elements should be determined before conducting study analysis(es), with the exception of processes for developing operational definitions from conceptual definitions.

As with traditional clinical studies, in choosing existing operational definitions or developing new ones, sponsors should maximize identification of those who have the condition and to minimize incorrectly identifying those without the condition as having it (i.e., minimize misclassification). FDA considers minimizing misclassification to be a critical part of the process of defining an operational definition. FDA recommends reviewing previous studies using the RWD source, including published literature, and gathering expert opinion when developing operational definitions. For some data elements, a rationale for the operational definition based on previous studies or expert opinion may be sufficient. Some data elements may warrant more scrutiny to ensure that the interpretation of study results is not substantially impacted by their misclassification or missingness. Further exploration is recommended for data elements that are not aligned with expectation.

In some cases, it may be appropriate to conduct a validation study in which quantitative measurements of the operational definition are compared to a “ground truth” reference standard. This may result in updating the operational definition to ensure that these critical data elements are accurately identified. When conducting a validation study, a protocol should be developed before initiating the data collection and analysis specific to the validation. The protocol generally contains the plan to compare the operational definition in the RWD (e.g., administrative claims data) with the “ground truth” in the reference standard (e.g., validating that administrative billing diagnosis accurately represents a point-of-care diagnosis by comparing an operational definition in administrative claims against an EHR) and prespecification of the acceptance criterion for each validation measure (e.g., sensitivity, specificity, or positive and negative predictive values) that is of interest.

In some RWD sources, the data elements that would be preferred for traditional clinical studies may not be available to the sponsor. However, a proxy for this missing information could be developed based on the information that is collected in the RWD source. Proxies can be developed for a wide range of uses, including identifying study participants (i.e., applying inclusion/exclusion criteria) as well as certain study endpoints. While it may be possible to use the proxy, sponsors should determine the suitability of a proxy by considering whether the proxy is clinically relevant and may call for additional data gathering or conducting validation of the proposed operational definition for the data element, if using the proxy adds too much uncertainty to the study interpretation. FDA encourages use of measures that participants or practicing clinicians deem meaningful as potential data elements for studies using RWD. Development of endpoints or potential consideration of proxy outcomes may be warranted to address some study questions. Additionally, development of proxies for key covariates may also be appropriate to address some study questions.

The conceptual and operational definitions should be included in the study protocol (see Section VII.B).

(3) Appropriate Integration of Data Elements Within Study Design and Analysis 🔗

Once the device and outcome are determined per the study question, variables associated with the device or outcome (i.e., confounders, mediators, effect modifiers, and additional risk factors) should be addressed within the study design (as both conceptual and operational definitions) and analysis to minimize bias and uncertainty. Confounders (unobserved and observed) impact both the device and outcome, and may need to be conditioned upon to accurately measure the causal effect of the device on the outcome. Mediators, or intermediate variables, are impacted by the device and subsequently impact the outcome, and should be carefully considered, as conditioning on such variables may dilute the total effect of the device on the outcome or introduce bias. Effect modifiers or effect modification refers to heterogeneity of effect of the device on the outcome by strata of another variable (i.e., modifier), and reporting stratum-specific results may be appropriate. As with all clinical studies, prespecified subgroup analyses for sex, age, and other groups are recommended because there may be differential effects of the device on the outcome by subgroups.⁴⁸

Based on the prespecified research question(s) identified, the study protocol should concisely identify which data elements fall into each of the types of variables (e.g., exposure/device; outcome(s); covariates: confounders, mediators, effect modifiers, risk factors). An effective way to identify which data elements fall into each of these types of variables is to generate and analyze causal diagrams. Causal diagrams and their subsequent assessment may provide a rationale for the design and analysis choices, and therefore should also be provided in the study protocol. One example of a type of causal diagram is a directed acyclic graph (DAG), as depicted in Figure 3. Additionally, causal diagrams may provide a resource to aid discussions for selecting an appropriate study design amongst the study team or with FDA. Covariates affecting or affected by the exposure or outcome should be noted within the causal diagram, irrespective of availability within RWD, and assessed for potential relationships among other variables. Causal diagrams illustrate the relationships between covariates, including the direct causal path from the exposure to the outcome, indirect causal paths through mediators, and non-causal (backdoor) paths through confounders. Causal diagrams can also be used to identify a minimally sufficient set of confounders for which to control so that all non-causal paths between the exposure and outcome are blocked, without conditioning on variables that are influenced by both the exposure and outcome (which would introduce collider-stratification bias).⁴⁹ Thus, potentially only a subset of the covariates initially identified may be needed to control for confounding.⁵⁰

Figure 3. Example of directed acyclic graph to identify potential data elements and assess which are key for the study question

VII. Documentation for FDA Review 🔗

This section describes the documentation recommended to support the use of the RWD for generating RWE for regulatory purposes and applies to device regulatory submissions submitted to CDRH and CBER, including but not limited to, pre-submissions, 510(k)s, PMAs, Biologics License Applications (BLAs), HDEs, De Novos, IDEs, post-approval study PMA supplements, submissions in response to a 522 order, CLIA Waiver by Applications, and Duals.

A. Regulatory Submission Cover Letters 🔗

FDA recommends sponsors identify RWD and RWE as part of the regulatory submission cover letter to help facilitate review and internal tracking. Specifically, FDA recommends sponsors include the following in the cover letter for each submission that includes RWD:

• Purpose of using RWE to support the submission (see list of examples in Section IV.A);
• Study design (i.e., type of study) using RWD to generate RWE (see list of examples in Section VI.A);
• RWD source(s) used to generate RWE (see list of examples in Section IV.A); and
• Specific RWD source(s) and version, including the following information, if applicable:
  • Data source name;
  • Data source provider;
  • Version number; and
  • Date of extraction and date range of data extracted.

B. Protocol 🔗

As with traditional clinical studies, sponsors should include the protocol(s) for any RWD-based studies as part of the regulatory submission to FDA. FDA recommends that sponsors finalize the protocol and analysis plan prior to reviewing the outcome data of a study and before performing prespecified analyses. Sponsors should indicate in the regulatory submission whether or not the protocol and analysis plan were finalized prior to the analyses. While the protocol is being developed, sponsors should not have access to the outcome measure results. Beyond protocol development, it is also important to consider when individuals should not have access to outcomes in the dataset used for the study (e.g., generating summary scores, propensity score modeling, determining level of data missingness). Any revisions to the protocol should be dated and time-stamped, and the rationale for each change should be provided.

Sponsors should implement the relevance and reliability concepts in this guidance when developing their study protocol. Similar to protocols submitted with traditional clinical studies, sponsors should provide the following information, as appropriate, and avoiding redundancies, in the protocol for the study generating the RWE, when the RWD or RWE is included in the regulatory submission:

• Study synopsis;
• Background and study purpose, including:
  • Explanation of how the source data is or is not representative of the general disease/population with the condition, including sufficient previous research to interpret study results within the context of the target population, disease trajectory, and current clinical care;
  • Description of the device included in the study, including the DI portion of the UDI, if available. For devices excepted from the UDI requirements, sponsors should include the version(s) of the device; and
  • Reports of prior use of the device relevant to the study;
• Study aims and objectives;
• Study design, including study period (see Section VI.A);
• Study design diagrams are suggested to clarify (1) potential study entry dates within study period and (2) assessment of all other data elements in relation to cohort entry or index date;
• Data source(s), including a description of how the setting/environment(s) of data capture provided adequate continuity of care (see Section V);
  • Identification of any common data model structure used for housing the RWD source or for transformed study-specific RWD, if applicable; and
  • Identification of linkage methodology from original RWD sources and associated attributes that should be considered;
• Data elements (conceptual and operational definitions for all), including:
  • Determination of initiation, continuance, and discontinuation of device exposure, if applicable (see Section V.A);
  • Study population, including inclusion and exclusion criteria, and generalizability to the intended use population;
  • Device and comparator, if included in the study;
  • Outcome and endpoints; and
  • Covariates;
• Causal diagram;
• Statistical/data analysis plan;
• Data management and quality control plans (see Section V.B);
• Sample size and statistical power;
• Description of human subject protections, as appropriate, including informed consent, IRB determination, deidentification plan (e.g., to remove participant identifiers from patient-generated data or device-generated data), and data confidentiality plans;
• Plans for adverse event reporting;⁵¹
• Milestones and timeline;
• Auditing and monitoring plans, as applicable;
• If validation or adjudication of data element(s) were conducted (see Section VI.B), sponsors should include the study plan and results for validation or adjudication; and
• A copy of the data dictionary used, if one was used or developed.

C. Report Including the Relevance and Reliability Assessment 🔗

As with traditional clinical studies, sponsors should submit a study report for any RWD-based studies with the study results, discussion, and conclusion as part of the regulatory submission to FDA. If sponsors include RWE in regulatory submissions, they should include how the RWE supports the purpose of the submission and their relevance and reliability assessment of the RWD to generate RWE with the following:

• An assessment of the key relevance and reliability factors for the study using RWD (see Sections V and VI), which may include:
  • Data availability, linkages, timeliness, and timing of data availability, generalizability of data (see Section V.A);
  • Data accrual, quality, and integrity, date of data extraction (see Section V.B);
  • Completeness and accuracy of study sample reflecting the target population, study design and planning (see Section V.B.2);
  • Date of data extraction (see Section V.B); and
  • Study purpose, specific data elements, and assessment of confounding (see Section VI.B).

In addition to the relevance and reliability assessment of the RWD, we recommend that the sponsor provide the following contextual information, including:

• All elements from the protocol, updated to reflect how the study was conducted;
• Justification that any changes or modifications to the protocol did not affect the validity of the resulting RWE;
• Discussion of the occurrence and rationale for any protocol deviations;
• A description of how the RWE is being used within the context of all the clinical evidence submitted to FDA; and
• If unique considerations exist for the specific RWD source, sponsors should describe these considerations and how they impact the overall assessment of the data.

D. Additional Information 🔗

As with traditional clinical studies, sponsors should also provide the following information in regulatory submissions that include RWE:

• ClinicalTrials.gov National Clinical Trial (NCT) Number,⁵² if applicable;
• Informed consent and IRB documentation, as applicable;
  • Initial and continuing IRB review and approval; and
  • Initial and approved changes to informed consent;
• List of investigational sites, if any, including mailing address, contact information, and investigator name; and
• Case Report Form templates, as developed by the sponsor, if applicable (e.g., these templates may be helpful if EHR or claims data are not mapped to the dataset from the source).

Appendix A. Recommended Relevance and Reliability Elements for Documentation and FDA Review 🔗

The following is an example of recommended relevance and reliability elements, and is intended to assist sponsors and FDA in determining the relevance and reliability of RWD for a regulatory decision. The tables below summarize the recommended elements identified throughout the guidance for sponsors to document and provide in FDA submissions, and the recommended locations for where to include this information. The tables are not intended to serve as either a mandatory or exclusive checklist. Rather, the tables provide a simplified summary of the key elements that sponsors should use to assess relevance and reliability. Depending on the regulatory submission, additional considerations may be applicable as noted in Section V, even if not specified below.

Table 1- Recommended RWD Relevance Elements for Submission of RWE

Item (Linked to Section V)	Information for Sponsors to Document (e.g., to make available for inspection)	Information for Sponsors to Provide to FDA in Submission	Recommended Location in FDA Submission
Determine if the RWD contains sufficient detail to capture data elements and address the study question	x (detailed)	x (rationale)	Protocol (rationale for study question and data element definitions)
Assess longitudinality of data source		x	Protocol
Assess continuity of care in data source		x	Protocol and report
Ensure reasonable time between data collection and release for research		x	Protocol and report
Consider changes in clinical practice/guidelines over time		x	Protocol
Specify timing for RWD updates		x	Protocol and report
Assess whether and how data from different sources can be obtained and integrated, given the potential for heterogeneity in population characteristics, clinical practices, and coding across data sources		x	Protocol
If applicable, use of a predefined linkage methodology that is scientifically valid and accounts for differences in coding and reporting across sources	x (detailed)	x (high level)	Protocol
Assess adequacy of line-level linkages		x	Report
Apply strategies to correct for redundant data, to resolve any inconsistencies, and assess the potential for missing data	x
Demonstrate interoperability of the linked data systems	x
Demonstrate appropriate patient privacy protections are applied (e.g., PPRL) with linked data		x	Protocol and report
Justify how the study sample selected is adequately representative of the patients in the RWD source that are reflective of the proposed intended use population. Identifying the study sample should take into consideration how the application of inclusion/exclusion criteria will affect the overall generalizability of the RWE to the U.S. intended use population.		x	Protocol and report

Table 2- Recommended RWD Reliability Elements for Submission of RWE

Item (Linked to Section V)	Information for Sponsors to Document (e.g., to make available for inspection)	Information for Sponsors to Provide to FDA in Submission	Recommended Location in FDA Submission
Provide all information and associated documentation related to processes, procedures, and methods around data accrual, data quality, and integrity; if unable to do so, describe level of access obtained, and consider providing access for FDA	x (detailed)	x (high level)	Protocol
Provide information and descriptors about data source(s)		x	Protocol
If applicable, describe defined processes around data monitoring and programs for quality audit and on-going training of data collection	x
Document routine migration of data from various sources over time	x
Describe sources and technical methods used for data element capture	x
Describe methods for data retrieval and processes to minimize missing data extraction, implausible values, and data quality checks	x
If not data owner, describe level of access, attempt to gain participant-level data, if accessible, and consider providing access for FDA		x	Protocol
Describe the quality of the data captured	x (detailed)	x (high level)	Report
Plan and document the process of extraction, aggregation, curation, storage, and availability of data for research	x
Generate data flow and supporting data quality documentation from the first instance through RWD dataset(s) used to address the study question	x
Define and follow procedures for quality control, including data collection and quality assurance	x
Provide assessment of completeness, accuracy, and consistency across sites and over time		x	Report
Assess consistency of data capture across sites and over time; if any changes are needed (e.g., diagnostic criteria, definitions, or clinical practice change over several years), then document those changes and assess their impact on the study results	x (detailed)	x (high level)	Report
Assess missingness and out of range values for each data element		x	Protocol and report
Describe how data elements captured and included in the study are reflective of the real-world patient experience (e.g., interactions with health care, disease trajectory, outcomes) with the condition of interest		x	Protocol and report
Document the auditing rules and methods used and the mitigation strategies used to reduce errors	x
Justify how study sample size addresses the study question and regulatory purpose with adequate statistical power or precision for planned analyses		x	Protocol and report
Document adherence to source verification procedures and data collection and recording procedures for completeness and consistency	x
Prespecify data checks and procedures to help address identified errors	x
Describe mitigations to address audit findings, including data corrections	x
Provide documentation of any previous RWD source relevance and reliability assessment for a similar target population and/or peer-reviewed literature of RWE generation from data source, as applicable		x	Protocol
Ensure adequate patient protections (e.g., methods to protect the privacy of individuals’ health data and adherence to applicable privacy and ethics standards) established in advance of executing the study protocol		x	Protocol and report

Appendix B. Examples Where RWE is Used 🔗

Most of the following examples are generalized from actual uses of RWD in support of FDA regulatory decision-making. These examples do not represent a comprehensive list of all potential uses or sources of RWD but do describe some situations where RWE might be used to support regulatory decision-making. These examples also include high-level discussion of some considerations for relevance and reliability, rather than the more detailed information that should be provided in submissions, as recommended in Sections V-VII, and Appendix A. For additional examples of RWE used in regulatory decisions, see the following FDA document: Examples of Real-World Evidence (RWE) Used in Medical Device Regulatory Decisions. Should a sponsor wish to use RWD to generate RWE for similar regulatory purposes as described below, the acceptability of the use of RWD would depend on the specifics of the submission, such as the device, indications, and results of analyses.

Example 1: New approval of PMA

An implanted device, available OUS, used RWE as the primary clinical evidence to support the original PMA submitted to FDA. RWD from an OUS registry in one country, which included hundreds of patients with more than two years of follow-up, was compared against performance goals (PG) for device safety and effectiveness. In this example, an IDE application was not required for the clinical study because it was conducted OUS. An IDE application was also not required because the RWD comprised routinely collected extant data from an OUS registry. The PGs were derived from a prospectively defined meta-analysis of published literature and data from a U.S. registry of a comparator device legally marketed in the U.S. The study evaluated routinely collected functional outcome data and patient reported outcome measures. The safety assessment also included a comparison of the serious device-related adverse event rates for the subject device extracted from the OUS registry to rates in the same literature studies used to derive the PG. These analyses served as the primary basis supporting approval of the PMA.

Important considerations from FDA’s review, supporting the relevance and reliability of the RWD in this submission, included, in part: 1) devices of interest were clearly identified in the OUS registry, 2) data from the OUS registry were generalizable to U.S. patients, and 3) the OUS registry had adequate data capture methods to ensure completeness, accuracy, and consistency across sites of key variables, including covariates of interest and outcome measures.

Example 2: Expanded indications for use

An implanted device was authorized under a PMA and was subsequently available in the U.S. for several years for one indication for use. The sponsor of the original PMA submitted a PMA panel-track supplement seeking to expand the indication for this device, which included a systematic literature review as well as RWD. The systematic literature review served as the primary clinical evidence to support safety and effectiveness of the device for the new use. The RWD comprised information from the sponsor’s patient database linked with Medicare claims, and were provided as supplemental clinical evidence to support the safety of the device with the proposed indication. An IDE application was not required for the clinical study using RWD because the clinical data for the subject device were obtained from extant data that had been generated as part of routine clinical care. The RWE characterized the safety profile in patients with the specific condition (i.e., new indication) implanted with the device. A comprehensive review was conducted to identify validated diagnostic (i.e., ICD-CM) codes associated with safety outcomes, and used to assess these outcomes among patients receiving the implant and diagnosed with the specific condition compared to patients implanted with the device that did not have the condition. The sponsor’s patient database and Medicare claims were linked using probabilistic methods. Important considerations from FDA’s review, supporting the relevance and reliability of the RWD in this submission, included, in part: 1) the devices were clearly identified in the sponsor’s patient database, 2) Medicare claims captured the relevant outcomes of interest in the study population, 3) the linkage methods were considered adequate, and 4) the data accrual and quality of both RWD sources were considered acceptable. Although the safety information from the RWE indicated a higher adverse event rate for the subject device in patients with the specific condition compared to patients without the condition, this was expected for the respective patient population, and was similar to results from a separate systematic literature review provided by the sponsor. The RWE allowed for an assessment of device safety in a much larger patient population than the published randomized control trials in the device space and allowed for estimation of rare safety events. The RWE was part of the totality of clinical evidence to support that the probable benefits of the device outweigh the probable risks, to support approval of the submission.

Example 3: Submissions in response to a 522 order

The manufacturer of a class II device was issued a 522 order to address public health questions on the safety of the device. The device met the statutory criteria for issuance of a 522 order because its failure would be reasonably likely to have serious adverse health consequences, and because it was also expected to have significant use in pediatric populations. To meet the 522 postmarket surveillance study requirement, the manufacturer discussed plans with FDA, and designed a single-arm, prospective study using non-RWD to assess safety outcomes at 12 months and using RWD to support the continued assessment of the software for its intended use. The routinely collected device-generated data from patient use of the software were automatically uploaded via a mobile application into the manufacturer’s database, and were considered RWD. The device-generated data and method of collection was consistent with the device’s intended use. There was also monthly follow-up with participants via emailed survey as part of the study to gather adverse event information. This non-RWD from the follow-up were then linked to the routinely collected device-generated data to determine if the safety events were associated with device use. The outcomes were compared to a historical control. The device-generated data were considered sufficiently relevant and reliable to fulfill the postmarket requirement. Important considerations from FDA’s review, supporting the relevance and reliability of the RWD in this submission, included, in part, 1) outcomes of interest were adequately identified from the device-generated data, 2) the data from the RWD source were considered representative of the intended use population, including pediatric patients, and 3) adequate data capture methods were used to ensure completeness, accuracy, and consistency for the manufacturer’s database. At the end of the study, the 522 postmarket surveillance study met its endpoints and there were no unanticipated adverse device events. This study addressed the public health questions from the 522 order, using RWD from the device combined with information from separate study-defined follow-up information from surveys, to support overall generalizability to the intended use population, and to provide specific adverse event information beyond what was provided in the premarket clinical study.

Example 4: Control group

A sponsor submitted a PMA panel-track supplement seeking approval for an indication expansion, which relied on a comparison of participants from a single-arm traditional clinical study to an external control group from a U.S. registry. The devices from the control group were identified through the use of UDIs. The registry did not routinely collect one key covariate for the control group; therefore, the participating sites separately gathered and added this information to the registry from extant EHRs as this covariate was routinely collected and documented in the EHR. Both the single-arm study and the registry used a common data capture form with the same definitions for key variables (covariates and outcomes of interest). The sponsor proposed a proxy outcome from the registry, which was also a secondary endpoint in the traditional clinical study, because the primary endpoint typically used for traditional clinical studies in the device space could not be assessed in the registry. This proxy outcome from the registry was considered clinically acceptable for this particular study by the sponsor and FDA. To account for potential differences between the device and control groups, the sponsor performed a propensity-score adjusted analysis using pre-specified variables. The propensity score results were reviewed and considered acceptable by FDA before the sponsor performed the outcome analysis. Important considerations from FDA’s review, supporting the relevance and reliability of the RWD in this submission, included, in part: 1) use of the control device was definitively captured through UDIs in the registry, 2) the registry captured the outcomes and covariates of interest in the relevant patient population, and 3) acceptable data accrual methods were used, such as use of a common data capture form in the registry and adequate data abstraction from extant EHRs. Although there were missing data, sensitivity analyses demonstrated that the extent of missing data did not change the conclusions of the study. This RWE was part of the totality of clinical evidence to support that the probable benefits of the device outweigh the probable risks, and served as the primary basis for supporting approval of the PMA supplement.

Example 5: Granting of De Novo

A sponsor submitted data from two clinical studies to support a De Novo request for a device. One was a prospective, repeated measures, single-arm study, which served as the primary clinical evidence to support safety and effectiveness for the De Novo request. In addition, the sponsor provided a clinical study using RWD from EHRs from over 500 patients, as supplemental clinical evidence. The overall objective of this study using RWD was to confirm the performance of the subject device, and to further investigate trends in treatment outcomes for patients with different severities of the disease. The sponsor used this study to provide additional, powered, single-measure analyses evaluating the primary effectiveness endpoint in the cohort from the single-arm study, as well as in additional subgroup cohorts not studied in the prospective study. This evidence helped address uncertainty that remained related to the prospective, single-arm study. Important considerations from FDA’s review, supporting the relevance and reliability of the RWD in this submission, included, in part: 1) the appropriate length of follow-up for the endpoints of interest was available in the RWD source and 2) the data abstraction methods included prespecified, standardized definitions for key variables (including covariates, device and outcome measures) and were considered acceptable. The RWE generated from these data demonstrated consistent outcome measures with the prospective single arm study. Overall, the clinical evidence from the clinical studies and RWE supported granting of the De Novo request for this device.

Example 6: RWE obtained from use of EUA device

In response to the COVID-19 outbreak, FDA authorized the emergency use of certain devices under section 564 of the FD&C Act. A sponsor of a test initially authorized for emergency use utilized RWD from a clinically annotated biobank, where samples were routinely collected, to support traditional premarket clearance and the expansion of the indications for use under a Dual 510(k) and CW. The RWD from the clinically annotated biobank were provided in addition to data from a prospective clinical study. Important considerations from FDA’s review, supporting the relevance and reliability of the RWD in this submission, included, in part, 1) appropriate outcome data elements were captured and consistently recorded using established systems including LOINC or Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) for completeness and consistency; 2) the study population derived from the clinically annotated biobank was adequately representative of the intended use population and the samples from the biobank were appropriately selected. In addition, the results (e.g., usability and performance) could be reasonably generalized to support use of the device in this patient population. The sponsor and FDA also agreed that the candidate testing needed to be conducted in accordance with the authorized instructions for use. In addition, any samples that were excluded from the study were appropriately justified. The RWE in this submission allowed for an appropriate assessment of clinical claims made by the sponsor in a larger patient population than the prospective clinical study and complemented its findings.

Guidance History 🔗

Guidance History	Date	Description
Level 1 Final Guidance	December 2025	See Notice of Availability for more information. This guidance supersedes the final guidance titled “Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices” and published August 2017.
Reissued as Level 1 Draft Guidance	December 2023	See Notice of Availability for more information.

*This table was implemented, beginning December 2025, and previous guidance history may not be captured in totality.

**The Notice of Availability is accessible via the Search for FDA Guidance Documents webpage.

Footnotes 🔗