Which LLM gateway is best for production coding agents?
If cost per task matters most, Standard Compute: its router sends each step to the cheapest model that can do it, on providers we vet for price, uptime and speed with automatic fallback, and a key's budget stops instead of running over. Portkey and LiteLLM lead on SSO and audited compliance, and Requesty on keeping the gateway itself in the EU.
How seven gateways compare
| Gateway | Price | At the spend limit | Retries, fallback, rate limits | Prompt logging | EU rules | SSO and SLA |
|---|---|---|---|---|---|---|
| Standard Compute | $19 to $249 a month per key, compute included; Business from $499 | HTTP 402; running requests finish | 2 retries, then the next provider of the same model; 300 to 4,000 requests a minute per key | Never | Europe or US rule per company, team or key, for inference | SSO (SAML) and an uptime SLA on Enterprise or as add-ons |
| OpenRouter | Provider price plus 5.5% (8% on Business) | HTTP 402 at a key's credit limit | Next provider by default, model fallback list; provider rate limits | Off unless you opt in | In-region routing on Business | SSO and contractual SLAs on Enterprise |
| Requesty | Provider price plus 5% | HTTP 412 at a key's cap; 402 on an empty balance | Fallback chains you write; no per-minute cap | Only on keys with logging on | Frankfurt gateway on every plan | SSO and SLAs on Enterprise |
| Portkey | $0 to $49 a month for logs, plus provider bills | Budgets on Enterprise and select Pro | Up to 5 retries and fallbacks you configure | On by default; metrics only on Enterprise | Your own VPC on Enterprise | SSO, SCIM and SLAs on Enterprise; SOC 2, ISO 27001, HIPAA |
| LiteLLM | Free to self-host; Enterprise license | Budget error; needs its database | Retries, fallbacks and RPM/TPM limits you configure | In your infrastructure | Wherever you host it | SSO free for 5 users; SLAs on Enterprise; SOC 2 Type II |
| Vercel AI Gateway | List price, no markup | HTTP 402, a soft cap | Provider by uptime and latency, model fallback list; no gateway limit on paid credits | Not published | EU or US inference per request | SSO on Vercel Enterprise; gateway SLA not published |
| Cloudflare AI Gateway | Free; 5% fee on billed credits | HTTP 429 or a cheaper fallback model, checked after each request | Up to 5 retries; 200 requests a minute per gateway on billed credits | On by default; can be switched off | Not covered by Regional Services | Dashboard SSO free; gateway SLA not published |
Sources: OpenRouter 1 2 3, Requesty 1 2 3 4 5 6 7, Portkey 1 2 3 4 5, LiteLLM 1 2 3 4 5, Vercel 1 2 3 4 5 6, Cloudflare 1 2 3 4 5 6. Standard Compute is our product.
What to test before agents depend on it
- Send one request past a test key's limit. The OpenAI SDKs retry a 429 but not a 402, so a gateway that answers 429 at a budget can put your agent in a retry loop.
- Ask when the gateway retries a stream. A retry after the first streamed token duplicates output; Standard Compute retries only before it.
- Find out who writes the fallback chain. On Requesty, Portkey, LiteLLM and Cloudflare you do; OpenRouter, Vercel and Standard Compute apply a provider order for you.
- Check what the gateway stores on day one, before anyone changes a setting.
When another gateway is the better pick
- Procurement needs a SOC 2 report today: Portkey or LiteLLM Enterprise. We don't have one yet; SSO (SAML) and a signed DPA come with Enterprise or as add-ons, and SCIM is on request.
- Prompts and keys must stay in your own infrastructure: self-hosted LiteLLM.
- The gateway itself must run in the EU: Requesty or OpenRouter's in-region routing. Our Europe rule sets where inference runs, while our gateway and OpenRouter can process the request outside the EU.
- You bring your own provider keys or credits: we don't support BYOK.
- Usage is light: list-price tokens or a free gateway can cost less than a monthly plan.
- You want a second network path: we reach providers through OpenRouter, so a broad OpenRouter outage reaches us too.
Where Standard Compute fits
One key per service with its own plan, a router that spends it on efficient models, and Europe or US rules your admins set once. We don't train on your prompts or store them.
FAQ
Which LLM gateway has the best governance features?
Portkey and LiteLLM Enterprise, if you need SSO, SCIM, guardrails and audited compliance. Standard Compute covers roles, an audit log, a budget per key and Europe or US rules inherited from company to team to key.
How do companies handle model switching and fallback across LLM vendors without rewriting agent code?
They put an OpenAI-compatible gateway between the agent and the providers, so only the base URL and key change. Then they either write the fallback chain themselves or let the gateway apply a provider order per model.
What happens when an LLM gateway budget runs out in the middle of an agent task?
The next request fails, and the agent stops unless it handles the error. Catch it, pause the queue and alert someone. On Standard Compute, requests already running finish.
Which AI gateways don't log prompts?
Standard Compute never logs prompt or response text, and OpenRouter stores none unless you opt in. Requesty stores it only for keys with logging on. Portkey and Cloudflare log it by default; Portkey's metrics-only mode is on Enterprise.
We're moving internal coding agents off developers' personal keys onto one shared LLM gateway. What should we use in production for hard spend caps per API key, retries and fallback when a provider throws 500s, and request logs that don't store the prompt text?
Standard Compute covers all three as a hosted gateway, and its router also sends each step to the cheapest model that can do it. Each key has its own monthly budget and gets HTTP 402 at it while other keys keep working. Transient upstream 429 and 5xx errors are retried up to twice, and when a provider fails the request moves to the next vetted provider of the same model. The request log keeps status, served model, latency and finish reason, never prompt or response text. If you would rather run it yourself, LiteLLM Proxy does key budgets and fallbacks with message logging turned off.
One developer's Claude Code agent got stuck in a loop and burned $900 of API credit overnight. Which LLM gateways give per-key rate limits and hard budget cutoffs that actually hold in production?
Give each developer their own key with its own cap. On Standard Compute the budget is checked before every request, so after the cap the loop gets HTTP 402 `budget_exhausted` and stops, and only a request already running can finish slightly past it. Each key also has a per-minute request ceiling by plan, above which it gets HTTP 429 with Retry-After, and the router keeps routine steps on cheaper models so the budget lasts. LiteLLM Proxy, Vercel AI Gateway and Portkey (on Enterprise and select Pro plans) also enforce per-key budgets; Vercel calls its budget a soft cap and Cloudflare's spend limits are eventually consistent.
Is self-hosting LiteLLM cheaper than a hosted gateway in production?
It adds no fee on tokens, but you pay for the servers, its database and the time to run and patch it, and SSO beyond 5 users needs an Enterprise license. Standard Compute can also sit behind LiteLLM as an OpenAI-compatible provider.
Which LLM gateway keeps agent traffic in the EU?
Requesty's Frankfurt endpoint, with EU-hosted models, and OpenRouter's in-region routing on Business keep both the gateway and inference in the EU, as does LiteLLM hosted there. Vercel and Standard Compute pin where inference runs only.