Learn · Comparison

AI gateway vs AI control plane.

Vidai, LiteLLM Proxy, Portkey and Kong AI Gateway side by side, on governance, drop-in compatibility and performance.

How the products compare

Cost control, policy and audit on every call.

Speed only matters if the boundary does the work your organisation needs: every pound charged to a team in real time, the runaway agent stopped before the invoice, policy enforced in the request path, and the audit trail your regulator asks for.

Stronger answers: inside the perimeter, built in, acts rather than alerts.

CapabilityVidaiLiteLLM ProxyPortkeyKong AI Gateway
CategorySovereign AI control planeSelf-hosted open-source gatewaySaaS gatewayAPI gateway plugin
DeploymentContainers with PostgreSQL, inside your networkSelf-hosted (FastAPI, Docker, Kubernetes)SaaS, hosted by PortkeyPlugin on the Kong gateway
Where requests travelInside your perimeter onlyInside your perimeterAcross Portkey's perimeter on every callInside (self-hosted); cloud (Konnect)
Where the audit trail livesYour storage, your retention, recorded in the request pathYour storagePortkey storageYour storage
Cost attributionPer team, key, app and model, in real time, on a dashboardPer virtual key, with a databasePer key, per workspacePer consumer
Spend control (acts, not just alerts)Hard budget breakers stop the next call when a cap is hit; automatic fallback to a cheaper modelPer-key budget limits with alertsSpend limits with alertsRate limits (calls, not £ or $)
Routing for costRoute by rule, and switch to a cheaper model when a budget is reached. What-if shows which models would have cost less on your real calls. You decide where calls go; nothing reroutes on its ownConfigurable fallback chainsProvider failover, not cost-ledPlugin chain
Where prices liveA rate card kept in step: our optional public rate-card server (best effort), your own rate-card server (Enterprise), or prices you enterPrices in config files; updates need a rebuildCatalogue managed by the vendorPer plugin
Rate history (priced correctly after a vendor price change)Built in (SCD2): every call priced at the rate in force at that momentNot in their documentationNot in their documentationNot in their documentation
Policy enforcement (PII redaction, region lock, model allow-list)Built into the request pathGuardrail integrationsAdd-on guardrails modulePlugin ecosystem
Guardrail modelsPattern checks plus VidaiGuard's own models; your own terms as patterns. We can help integrate your own modelsSet integrations (Lakera, custom hooks)The vendor's guardrail modulePer plugin
Regulatory mapping (EU AI Act, DORA, GDPR Art. 44, UK CTPR, FCA, ISO/IEC 42001)Mapped explicitly: the audit trail is the answer for the regulatorNot in their documentationNot in their documentationPer plugin
ProvidersOpenAI, Anthropic, Vertex, Bedrock, Azure OpenAI, plus self-hosted Llama and MistralThe same set plus the OpenAI-compatible long tailThe same setVia plugins
Install footprintA 25 MB engine; containers with PostgreSQL, inside your networkPython runtime, Postgres and Redis for full functionality (hundreds of MB once dependencies are resolved)SaaS by default; self-hosted is a Node.js app with Postgres and RedisKong gateway, Nginx, Postgres and the AI plugin

The cost question

Attribution in real time, hard budget breakers that act rather than alert, and prices that stay right after a vendor price change.

Cost governance →

The sovereignty question

Where requests travel, where the audit trail lives, and whether regulatory mapping comes with the product. With Vidai, all of these answers stay inside your perimeter.

Regulatory governance →

Performance · the head-to-head test

Governed traffic at proxy speed.

This is our head-to-head test, one of the three on the performance page. The other gateways ran in minimal pass-through mode: no auth, no rate limits, no policy. Vidai ran with production features on: auth, API key validation, rate limiting and routing. Doing that work, it still sustains higher throughput at lower latency.
17×the throughput of LiteLLM at sustained production load
3.9×the throughput of Portkey at sustained production load
12.8 msVidai p95 latency at sustained production load

Throughput, requests per second Higher is better

Load profileVidai with governanceLiteLLM pass-throughPortkey pass-throughvs LiteLLMvs Portkey
Low load1,71313149513×3.5×
Modest load1,95915155713×3.5×
Sustained production load2,52615265617×3.9×

p95 latency, milliseconds Lower is better

Load profileVidai with governanceLiteLLM pass-throughPortkey pass-throughLiteLLM, times slowerPortkey, times slower
Low load8.5 ms646 ms92 ms76×11×
Modest load7.8 ms303 ms68 ms39×9×
Sustained production load12.8 ms5,009 ms1,200 ms391×94×

Even doing more work, Vidai delivered 13 to 17 times the throughput of LiteLLM and 3.5 to 3.9 times that of Portkey across the three load profiles. Its own overhead, measured on one machine with no network in between, is 0.58 ms median; see the three tests.

What changes in your application

A base URL change, not an SDK rewrite.

Most products in this category reach a long list of providers through the OpenAI-compatible shape, and so does Vidai. The difference is the native SDKs: Vidai works with the native Anthropic SDK, the native Google GenAI SDK and the OpenAI SDK as they are.

Stronger answers: fewer code changes.

QuestionVidaiLiteLLM ProxyPortkeyKong AI Gateway
What changes in your applicationBase URL and API key. Nothing else.Base URL and key; non-OpenAI SDKs may need rewriting to the OpenAI shapeBase URL and key; mainly the OpenAI shapePer-plugin configuration
OpenAI SDK against any upstreamYesYesYesVia plugins
Anthropic SDK against any upstreamYes, including non-Anthropic upstreams (OpenAI, Vertex, Bedrock, Azure)LimitedLimitedLimited
Google GenAI SDK against any upstreamYes, including non-Google upstreamsLimitedLimitedLimited
Self-hosted open-weight models (Llama, Mistral, your fine-tunes)First-class, inside the VPCVia OpenAI-compatible adaptersRoutes to self-hosted endpoints; data still crosses PortkeyFirst-class

Which one is right for you

Pick the boundary that matches your constraint.

Each of these products is strong at something. Choose by the constraint that binds you first.

Vidai

You face data-residency or audit obligations, run agent-pace traffic, and want one boundary doing cost, policy and audit on every call.

LiteLLM Proxy

Strong for an early-stage, open-source-only setup with low throughput, where the Python runtime is not yet the limit.

Portkey

Strong when you have no perimeter constraints and want the fastest SaaS time to first request.

Kong AI Gateway

Strong for an existing Kong shop adding AI traffic to a gateway it already runs.

If your main constraint is cost attribution and enforced spend at agent pace, read cost governance. For the whole picture, see the platform and what a control plane is.

Frequently asked

Common questions on this comparison.

What is the difference between an AI gateway and an AI control plane?An AI gateway is an application-layer proxy: it routes a model call and emits a log line. An AI control plane is infrastructure: every request also passes through policy enforcement, real-time cost attribution and an audit trail recorded inside your own perimeter. The control plane is what regulated organisations reach for once gateway-shaped tools stop answering the audit, finance and sovereignty questions on their own.
Do I need an AI gateway or an AI control plane?If a single team uses one provider, throughput is low, and nobody is asking the compliance question, an AI gateway is the right choice. You need a control plane when more than one team is using the same infrastructure with different budgets, a regulator or auditor is asking what the AI traffic did, or agent-pace traffic has broken the cost dashboard you set up for chat.
Are there self-hosted alternatives to LiteLLM Proxy?Yes. LiteLLM Proxy itself is self-hosted (Python). Vidai is self-hosted (containers with PostgreSQL, inside your network) and adds in-path policy enforcement, per-team cost attribution and an audit trail mapped to regulatory frameworks. Kong AI Gateway is self-hosted via Kong's plugin model. Portkey is SaaS by default; the self-hosted option still crosses Portkey's perimeter when telemetry is collected.
What is the best AI gateway for enterprise?The honest answer is that the best AI gateway for enterprise is usually not a gateway; it is a control plane. Enterprises that start with a gateway typically rebuild it as something stricter once the audit, residency and multi-team cost questions arrive. The comparison on this page lays out the trade-offs.
Can I use Vidai with my existing Anthropic SDK or Google GenAI SDK?Yes. Vidai is drop-in with the native Anthropic SDK, the native Google GenAI SDK and the OpenAI SDK as-is. Base URL and API key change, nothing else. The full SDK × upstream support matrix is in the documentation at docs.vidai.uk/integrations/.

Competitor entries summarise each product's public documentation and announcements at the time of writing; check their current documentation before you decide. Product names are trademarks of their owners.

How we start together

Start with one team, on your own AI traffic.

After a 20-minute demo, we agree a two-week pilot with you: one app or team, your traffic, your infrastructure.
  1. 01

    Pick one app or team

    Point it at the Control room. One configuration change; nothing else changes.

  2. 02

    Watch for two weeks

    Every call priced, checked and recorded. Rules start in observe mode.

  3. 03

    See the evidence

    What it cost, what it caught and what it would have stopped, from your own traffic.