Most Machine Learning (ML) teams today can reliably release models using Machine Learning Operations or MLOps 1: reproducible training, automated deployments, and continuous monitoring for drift and performance regression. These capabilities are essential, but in high-consequence domains, they are not enough. A system that works is not automatically a trustworthy one.
Embodied AI systems – such as autonomous vessels, drones, and robots – do not live in datasets or benchmarks. They move through the world, interact with uncertain environments, and make decisions that affect people, assets, and operations in real time.
In safety-critical domains, the issue is rarely whether a system works in the average case. Failures tend to hide in the ‘tails’: unusual combinations of initial conditions, sensor noise, or environmental states that rarely arise under nominal operation, but carry severe consequences when they do.
When failures are rare, standard testing can easily spend most of its budget on uneventful scenarios and still say very little about how likely failure really is. What matters for assurance is not simply how many scenarios we test, but whether testing produces meaningful, defensible evidence about how likely failure is under realistic operating conditions.