Engineers execute the approved verification protocols against one tagged build and record pass or fail, tester, date and the requirements each test covers. The locked model is run once on a sequestered test set that played no part in training or tuning, and the results are compared with the reference standard against acceptance criteria written before the data were unblinded. Every known defect or behavior that departs from expectation and is still open in the submitted version goes into the unresolved anomalies list with its safety and security impact and the reason it is left in.
On the map: Run verification protocols. Standalone performance on the locked test set. Unresolved anomalies.
Rows are disciplines, columns are stages 1 to 8. Select a square to open that task.
Back to the full map
Run verification on the exact build that cybersecurity pen-tests and clinical scores. One build tag across protocols, pen test and standalone study saves a re-run and a reconciliation question from FDA.
Verification runs on the release candidate that will be submitted. A failed test leads to a fix, a new build and a regression decision that names which protocols are re-run. The report records any changes made during verification so the reviewer can see that the evidence matches the shipped version.
The team checks that test cases come from patients and sites kept out of training and tuning, that the model weights were frozen before the reference standard was released to engineering, and that failed device runs count as failures and stay in the analysis set. Device outputs are compared with the reference standard using the endpoints and thresholds fixed in the protocol.
Engineering reviews the open bug tracker and user reports for the submitted version. Software defects and behavior that departs from user expectations go in. Feature requests, hardware root causes and residual risks that are working as designed stay out. Each entry gets a description, source, safety impact, security impact, workaround and the rationale for leaving it open. FDA may question an empty list on a complex device.
See the open positions and apply.