Latency
The time between request and response — for AI services a decisive measure of user experience.
Latency is the delay between sending a request and the response arriving, measured in milliseconds. For interactive AI applications it decides whether a service feels fluid or sluggish.
On-premise operation has a structural advantage here: when inference happens on your own network, the detour across the public internet to a remote provider disappears. In Quinta’s benchmark latency was about 56 ms — even under load with 512 concurrent requests.
What matters is latency under load, not at idle: a system that answers quickly on its own but collapses under many parallel requests is of little value in production. This is exactly where the operating layer pays off.