Clinical Stage 2 · Study design and pre-sub · Months 1–4

Write the protocol and statistical analysis plan

The clinical team and the biostatistician write the study protocol and the statistical analysis plan. Together they fix the primary and secondary endpoints, the acceptance criteria, the sample size and its power calculation, the inclusion and exclusion criteria, and the case report forms readers will fill in. The protocol also defines how the reference standard is built and how reader disagreements are resolved. The draft usually goes to FDA as Pre-Sub questions before any data is read.

On the map: Protocol and statistical analysis plan. Endpoints, sample size, case report forms. Adjudication design.

Open roles on this task: Principal engineer · Aware of

All 40 tasks

Rows are disciplines, columns are stages 1 to 8. Select a square to open that task.

Back to the full map
Four reference-standard pathways: two readers plus an adjudicator, three readers with automated consensus, and two forms of panel consensus.
Four reference-standard pathways: two readers plus an adjudicator, three readers with automated consensus, and two forms of panel consensus. Source: Definitive Guide to AI/ML SaMD Ground Truthing.

Who you work with

Build the sample size from engineering's pilot error, and run endpoints and acceptance criteria past regulatory before the pre-sub. FDA counts only acceptance criteria set in advance.

Decisions you make

Which endpoints and acceptance criteria?

Pick endpoints that match the output the label promises, such as agreement with the reader average for a measurement or sensitivity and specificity for a detection. Set limits from the predicate, the literature and pilot data, and keep secondary endpoints few, since every missed target has to be explained in the report.

How many cases?

Run a power calculation for each endpoint from an assumed error and spread, then take the largest number and add margin. Simulation is useful when the pass rule uses confidence intervals. The power number is a floor, since subgroup coverage often needs more cases than significance does.

How is the reference standard adjudicated?

Use an objective clinical anchor if one exists. Otherwise use three independent, blinded readers and combine them by a rule written in advance: median for measurements, majority for categories, STAPLE for masks. Escalate to an adjudicator only when a pre-set disagreement threshold is exceeded.

What to ask FDA in the Pre-Sub?

Ask about the parts that would be costly to redo: the reference standard, the separation of training and test data, and the sample size. Describe the algorithm clearly enough that FDA can judge how much proof the study needs.

Read more

Want to do this kind of work?

See the open positions and apply.

Apply Now
Apply now →