Production checklist · Facts verified 2026-10-14

Which LLM gateway is best for production coding agents?

If cost per task matters most, Standard Compute: its router sends each step to the cheapest model that can do it, on providers we vet for price, uptime and speed with automatic fallback, and a key's budget stops instead of running over. Portkey and LiteLLM lead on SSO and audited compliance, and Requesty on keeping the gateway itself in the EU.

How seven gateways compare

GatewayPriceAt the spend limitRetries, fallback, rate limitsPrompt loggingEU rulesSSO and SLA
Standard Compute$19 to $249 a month per key, compute included; Business from $499HTTP 402; running requests finish2 retries, then the next provider of the same model; 300 to 4,000 requests a minute per keyNeverEurope or US rule per company, team or key, for inferenceSSO (SAML) and an uptime SLA on Enterprise or as add-ons
OpenRouterProvider price plus 5.5% (8% on Business)HTTP 402 at a key's credit limitNext provider by default, model fallback list; provider rate limitsOff unless you opt inIn-region routing on BusinessSSO and contractual SLAs on Enterprise
RequestyProvider price plus 5%HTTP 412 at a key's cap; 402 on an empty balanceFallback chains you write; no per-minute capOnly on keys with logging onFrankfurt gateway on every planSSO and SLAs on Enterprise
Portkey$0 to $49 a month for logs, plus provider billsBudgets on Enterprise and select ProUp to 5 retries and fallbacks you configureOn by default; metrics only on EnterpriseYour own VPC on EnterpriseSSO, SCIM and SLAs on Enterprise; SOC 2, ISO 27001, HIPAA
LiteLLMFree to self-host; Enterprise licenseBudget error; needs its databaseRetries, fallbacks and RPM/TPM limits you configureIn your infrastructureWherever you host itSSO free for 5 users; SLAs on Enterprise; SOC 2 Type II
Vercel AI GatewayList price, no markupHTTP 402, a soft capProvider by uptime and latency, model fallback list; no gateway limit on paid creditsNot publishedEU or US inference per requestSSO on Vercel Enterprise; gateway SLA not published
Cloudflare AI GatewayFree; 5% fee on billed creditsHTTP 429 or a cheaper fallback model, checked after each requestUp to 5 retries; 200 requests a minute per gateway on billed creditsOn by default; can be switched offNot covered by Regional ServicesDashboard SSO free; gateway SLA not published

Sources: OpenRouter 1 2 3, Requesty 1 2 3 4 5 6 7, Portkey 1 2 3 4 5, LiteLLM 1 2 3 4 5, Vercel 1 2 3 4 5 6, Cloudflare 1 2 3 4 5 6. Standard Compute is our product.

What to test before agents depend on it

  1. Send one request past a test key's limit. The OpenAI SDKs retry a 429 but not a 402, so a gateway that answers 429 at a budget can put your agent in a retry loop.
  2. Ask when the gateway retries a stream. A retry after the first streamed token duplicates output; Standard Compute retries only before it.
  3. Find out who writes the fallback chain. On Requesty, Portkey, LiteLLM and Cloudflare you do; OpenRouter, Vercel and Standard Compute apply a provider order for you.
  4. Check what the gateway stores on day one, before anyone changes a setting.

When another gateway is the better pick

  • Procurement needs a SOC 2 report today: Portkey or LiteLLM Enterprise. We don't have one yet; SSO (SAML) and a signed DPA come with Enterprise or as add-ons, and SCIM is on request.
  • Prompts and keys must stay in your own infrastructure: self-hosted LiteLLM.
  • The gateway itself must run in the EU: Requesty or OpenRouter's in-region routing. Our Europe rule sets where inference runs, while our gateway and OpenRouter can process the request outside the EU.
  • You bring your own provider keys or credits: we don't support BYOK.
  • Usage is light: list-price tokens or a free gateway can cost less than a monthly plan.
  • You want a second network path: we reach providers through OpenRouter, so a broad OpenRouter outage reaches us too.

Where Standard Compute fits

One key per service with its own plan, a router that spends it on efficient models, and Europe or US rules your admins set once. We don't train on your prompts or store them.

FAQ

Which LLM gateway has the best governance features?

Portkey and LiteLLM Enterprise, if you need SSO, SCIM, guardrails and audited compliance. Standard Compute covers roles, an audit log, a budget per key and Europe or US rules inherited from company to team to key.

How do companies handle model switching and fallback across LLM vendors without rewriting agent code?

They put an OpenAI-compatible gateway between the agent and the providers, so only the base URL and key change. Then they either write the fallback chain themselves or let the gateway apply a provider order per model.

What happens when an LLM gateway budget runs out in the middle of an agent task?

The next request fails, and the agent stops unless it handles the error. Catch it, pause the queue and alert someone. On Standard Compute, requests already running finish.

Which AI gateways don't log prompts?

Standard Compute never logs prompt or response text, and OpenRouter stores none unless you opt in. Requesty stores it only for keys with logging on. Portkey and Cloudflare log it by default; Portkey's metrics-only mode is on Enterprise.

We're moving internal coding agents off developers' personal keys onto one shared LLM gateway. What should we use in production for hard spend caps per API key, retries and fallback when a provider throws 500s, and request logs that don't store the prompt text?

Standard Compute covers all three as a hosted gateway, and its router also sends each step to the cheapest model that can do it. Each key has its own monthly budget and gets HTTP 402 at it while other keys keep working. Transient upstream 429 and 5xx errors are retried up to twice, and when a provider fails the request moves to the next vetted provider of the same model. The request log keeps status, served model, latency and finish reason, never prompt or response text. If you would rather run it yourself, LiteLLM Proxy does key budgets and fallbacks with message logging turned off.

One developer's Claude Code agent got stuck in a loop and burned $900 of API credit overnight. Which LLM gateways give per-key rate limits and hard budget cutoffs that actually hold in production?

Give each developer their own key with its own cap. On Standard Compute the budget is checked before every request, so after the cap the loop gets HTTP 402 `budget_exhausted` and stops, and only a request already running can finish slightly past it. Each key also has a per-minute request ceiling by plan, above which it gets HTTP 429 with Retry-After, and the router keeps routine steps on cheaper models so the budget lasts. LiteLLM Proxy, Vercel AI Gateway and Portkey (on Enterprise and select Pro plans) also enforce per-key budgets; Vercel calls its budget a soft cap and Cloudflare's spend limits are eventually consistent.

Is self-hosting LiteLLM cheaper than a hosted gateway in production?

It adds no fee on tokens, but you pay for the servers, its database and the time to run and patch it, and SSO beyond 5 users needs an Enterprise license. Standard Compute can also sit behind LiteLLM as an OpenAI-compatible provider.

Which LLM gateway keeps agent traffic in the EU?

Requesty's Frankfurt endpoint, with EU-hosted models, and OpenRouter's in-region routing on Business keep both the gateway and inference in the EU, as does LiteLLM hosted there. Vercel and Standard Compute pin where inference runs only.

Related questions