Engineering Stage 7 · FDA review · About 5 months

Answering performance deficiencies

When FDA sends an additional information request, engineering re-runs the frozen model on the same test set to produce the analyses the reviewer asked for, such as agreement plots, error distributions, extra metrics and subgroup breakdowns. Outliers are pulled and reviewed case by case to explain the failure mode. Where a claim or feature cannot be supported in the response window, the team removes it. Where the reviewer finds a gap in requirements or tests, engineering adds the requirement, writes or rewrites the test, and releases a new verified version under change control.

On the map: Re-run analyses. Investigate outliers, descope weak claims. Add a missing requirement and test.

All 40 tasks

Rows are disciplines, columns are stages 1 to 8. Select a square to open that task.

Back to the full map
Four Bland-Altman patterns: good agreement, systematic bias, wide limits, and very wide limits.
Four Bland-Altman patterns: good agreement, systematic bias, wide limits, and very wide limits. Source: Bland Altman Analysis: Best Practices, FAQs, and Examples.

Who you work with

Before changing code for a deficiency, ask regulatory whether narrowing the claim is cheaper, and ask quality which verification must re-run under change control.

Decisions you make

Can the request be answered from existing data, or does it need new data?

The team first checks whether the stored per-case outputs and reference annotations can answer the question. Agreement plots, absolute error, distance metrics and subgroup tables usually can. New cases or new reader annotations are a last resort because the response clock is fixed.

Defend a result or remove the claim?

A result that is close to a target and has a reasoned clinical argument, such as performance comparable to reader-to-reader variation, can be defended with added analysis. A feature that rests on an assumption the team cannot support with references, such as a default threshold, is removed and the labeling and version history are updated.

How is a code change made during review?

Every fix goes through the same change control as before submission: a requirement update, a new or revised verification protocol, a new version number, regression scope, and an updated version history. The response cites the new verification report so the reviewer sees evidence for the version that will ship.

Read more

Want to do this kind of work?

See the open positions and apply.

Apply Now
Apply now →