Architecture
Throughput
The volume of requests a system successfully processes in a given time — the measure of scalability under load.
Throughput measures how many requests a system successfully handles per unit of time. Where latency describes the speed of a single response, throughput describes the capacity for many concurrent requests.
The decisive word is ‘successfully’: a server can accept many requests and still lose a large share of them under load. In Quinta’s benchmark the Quinta gateway processed an 18× higher success rate than plain vLLM across 31,200 requests and 512 concurrent connections.
High throughput at consistently low latency is the goal — and it comes not from raw power alone but from orderly admission control (bounded admission).