Which LLM gateway is best for production coding agents?
If cost per task matters most, Standard Compute: its router sends each step to the cheapest model that can do it, on providers we vet for price, uptime and speed with automatic fallback, and a key's budget stops instead of running over. Portkey and LiteLLM lead on SSO and audited compliance, and Requesty on keeping the gateway itself in the EU.
How seven gateways compare
| Gateway | Price | At the spend limit | Retries, fallback, rate limits | Prompt logging | EU rules | SSO and SLA |
|---|---|---|---|---|---|---|
| Standard Compute | $19 to $249 a month per key, compute included; Business from $499 | HTTP 402; running requests finish | 2 retries, then the next provider of the same model; 300 to 4,000 requests a minute per key | Never | Europe or US rule per company, team or key, for inference | No SSO; SLA only in custom Business contracts |
| OpenRouter | Provider price plus 5.5% (8% on Business) | HTTP 402 at a key's credit limit | Next provider by default, model fallback list; provider rate limits | Off unless you opt in | In-region routing on Business | SSO and contractual SLAs on Enterprise |
| Requesty | Provider price plus 5% | HTTP 412 at a key's cap; 402 on an empty balance | Fallback chains you write; no per-minute cap | Only on keys with logging on | Frankfurt gateway on every plan | SSO and SLAs on Enterprise |
| Portkey | $0 to $49 a month for logs, plus provider bills | Budgets on Enterprise and select Pro | Up to 5 retries and fallbacks you configure | On by default; metrics only on Enterprise | Your own VPC on Enterprise | SSO, SCIM and SLAs on Enterprise; SOC 2, ISO 27001, HIPAA |
| LiteLLM | Free to self-host; Enterprise license | Budget error; needs its database | Retries, fallbacks and RPM/TPM limits you configure | In your infrastructure | Wherever you host it | SSO free for 5 users; SLAs on Enterprise; SOC 2 Type II |
| Vercel AI Gateway | List price, no markup | HTTP 402, a soft cap | Provider by uptime and latency, model fallback list; no gateway limit on paid credits | Not published | EU or US inference per request | SSO on Vercel Enterprise; gateway SLA not published |
| Cloudflare AI Gateway | Free; 5% fee on billed credits | HTTP 429 or a cheaper fallback model, checked after each request | Up to 5 retries; 200 requests a minute per gateway on billed credits | On by default; can be switched off | Not covered by Regional Services | Dashboard SSO free; gateway SLA not published |
Sources: OpenRouter 1 2 3, Requesty 1 2 3 4 5 6 7, Portkey 1 2 3 4 5, LiteLLM 1 2 3 4 5, Vercel 1 2 3 4 5 6, Cloudflare 1 2 3 4 5 6. Standard Compute is our product.
What to test before agents depend on it
- Send one request past a test key's limit. The OpenAI SDKs retry a 429 but not a 402, so a gateway that answers 429 at a budget can put your agent in a retry loop.
- Ask when the gateway retries a stream. A retry after the first streamed token duplicates output; Standard Compute retries only before it.
- Find out who writes the fallback chain. On Requesty, Portkey, LiteLLM and Cloudflare you do; OpenRouter, Vercel and Standard Compute apply a provider order for you.
- Check what the gateway stores on day one, before anyone changes a setting.
When another gateway is the better pick
- Procurement needs SSO, SCIM or a SOC 2 report today: Portkey or LiteLLM Enterprise. We have none of these, and no published standard DPA.
- Prompts and keys must stay in your own infrastructure: self-hosted LiteLLM.
- The gateway itself must run in the EU: Requesty or OpenRouter's in-region routing. Our Europe rule sets where inference runs, while our gateway and OpenRouter can process the request outside the EU.
- You bring your own provider keys or credits: we don't support BYOK.
- You need a contractual SLA on a self-serve plan. Ours comes with a custom Business contract; the status page publishes a 99.9% monthly uptime target.
- Usage is light: list-price tokens or a free gateway can cost less than a monthly plan.
- You want a second network path: we reach providers through OpenRouter, so a broad OpenRouter outage reaches us too.
Where Standard Compute fits
One key per service with its own plan, a router that spends it on efficient models, and Europe or US rules your admins set once. We don't train on your prompts or store them.
FAQ
Which LLM gateway has the best governance features?
Portkey and LiteLLM Enterprise, if you need SSO, SCIM, guardrails and audited compliance. Standard Compute covers roles, an audit log, a budget per key and Europe or US rules inherited from company to team to key.
How do companies handle model switching and fallback across LLM vendors without rewriting agent code?
They put an OpenAI-compatible gateway between the agent and the providers, so only the base URL and key change. Then they either write the fallback chain themselves or let the gateway apply a provider order per model.
What happens when an LLM gateway budget runs out in the middle of an agent task?
The next request fails, and the agent stops unless it handles the error. Catch it, pause the queue and alert someone. On Standard Compute, requests already running finish.
Which AI gateways don't log prompts?
Standard Compute never logs prompt or response text, and OpenRouter stores none unless you opt in. Requesty stores it only for keys with logging on. Portkey and Cloudflare log it by default; Portkey's metrics-only mode is on Enterprise.
Is self-hosting LiteLLM cheaper than a hosted gateway in production?
It adds no fee on tokens, but you pay for the servers, its database and the time to run and patch it, and SSO beyond 5 users needs an Enterprise license. Standard Compute can also sit behind LiteLLM as an OpenAI-compatible provider.
Which LLM gateway keeps agent traffic in the EU?
Requesty's Frankfurt endpoint, with EU-hosted models, and OpenRouter's in-region routing on Business keep both the gateway and inference in the EU, as does LiteLLM hosted there. Vercel and Standard Compute pin where inference runs only.