Same engine. 18× more success under load.
We put the same inference server under extreme load twice — once bare, once behind the Quinta gateway. The difference is not the model but the operating layer. Here are the numbers, the setup and the honest framing.
Relative view of the success rates (Quinta gateway = 18×, plain vLLM = 1×).
Why does the bare server buckle?
An inference server without admission control accepts every request immediately — until it blocks itself under too many parallel requests. The Quinta gateway meters the inflow (bounded admission): it admits only as much concurrent work as the hardware handles stably. The rest waits in order instead of tipping into timeouts.
Go deeper: Bounded admission: why an inference server buckles under load — or compact in the glossary: Bounded admission, Throughput.
How it was measured
Both runs used the same inference engine on the same machine — the only difference was the operating layer in front of it:
- Hardware
- NVIDIA DGX Spark, 128 GB
- Scope
- 31,200 requests, up to 512 concurrent
- Comparison
- Quinta gateway vs. plain vLLM — identical engine
- Constant
- Identical per-user token rate in both runs
- Metric
- Success rate: share of requests with a valid response
- Overhead
- ~56 ms added first-token latency from the platform layer
This is an internal measurement in a defined load scenario — not an independent certification. Results depend on hardware, model and load profile and do not transfer 1:1 to every environment. That is why we walk through the setup in detail in the demo — and, on request, re-run the measurement on your own hardware.
Measure it, don’t take our word for it.
In the demo we show the measurement setup — and gladly reproduce the test on your own hardware, with your load profile.