What open-source AI stacks support an OpenAI-compatible proxy like LiteLLM and run on Kubernetes?

The LiteLLM proxy is a self-hosted OpenAI-compatible gateway with Helm charts in its docs, and vLLM serves open-weight models through an OpenAI-compatible HTTP server. Together they form a self-hosted alternative to OpenAI's API. Standard Compute is the managed alternative with nothing to deploy.

Is self-hosting cheaper than a managed gateway?

It is cheaper when one open model carries steady, high-volume traffic on GPUs you keep busy. For mixed work, Standard Compute's routing to the cheapest capable model and flat plans per person or app usually costs less than idle hardware plus the staff to run it.

Can Standard Compute keep data in Europe like a self-hosted stack?

Set a Europe rule for your company, a team or a single key, and every request, including retries and fallbacks, is served only by approved European inference endpoints. If no European endpoint can serve it, the request is refused before your prompt is sent.

Does Standard Compute store prompts?

We process your prompts in memory to serve each request and never write them to storage or logs. Standard Compute never trains on your prompts or outputs.

Self-hosted alternative to OpenAI's API for enterprises, compared with a managed LLM gateway

Standard Compute is the managed alternative: one OpenAI-compatible API for Claude, GPT, Gemini and open models such as GLM, DeepSeek and MiniMax, with no cluster for your team to run. Set a Europe rule and every request is served only by approved European inference endpoints, and GLM, DeepSeek and MiniMax are among the models served there.

A self-hosted stack keeps every component in your own infrastructure. The common open-source pairing is the LiteLLM proxy as the OpenAI-compatible gateway and vLLM as the model server, both deployable on Kubernetes. You then own the GPUs, the upgrades and the on-call rota.

By Standard Compute · Last updated 9 November 2026

Open-source AI stacks with an OpenAI-compatible proxy that run on Kubernetes

What each component is, from its own docs and repository, checked on 9 November 2026, next to the managed option.

Open-source AI stacks with an OpenAI-compatible proxy that run on Kubernetes
Component	Role	License and deployment	Enterprise features
LiteLLM proxy	Role: Self-hosted OpenAI-compatible gateway in front of many providers	License and deployment: MIT outside its `enterprise/` folder. Helm charts and plain Kubernetes manifests in the docs	Enterprise features: SSO beyond 5 users, audit logs and RBAC need an Enterprise license, priced on request
vLLM	Role: Model server for open weights with an OpenAI-compatible HTTP server (`vllm serve`)	License and deployment: Apache-2.0. You provide the GPUs	Enterprise features: None built in. Pair it with a gateway
Standard Compute	Role: Managed gateway with frontier and open models behind one OpenAI- and Anthropic-compatible API	License and deployment: Nothing to deploy. EU gateway in Finland, account data in Frankfurt	Enterprise features: SSO, provider allow-lists, audit log and DPA from $199 a month each, or Enterprise at $1,499

Sources are listed at the end of this page.

Create a company workspaceSee pricing

Local AI systems for teams spending heavily on cloud AI tokens

Self-hosting turns a token bill into a fixed hardware and staff bill. It pays off when one open model handles most of your traffic at steady, high volume. It costs more when work is spiky or when teams still need frontier models for the hard steps, because idle GPUs cost the same as busy ones.

Standard Compute lowers the token bill without the hardware. Each step goes to the cheapest model that can do it and the cheapest provider fast enough to serve it, so routine steps run on low-cost open models and the hard reasoning step on a frontier model. Each person and app pays a flat monthly plan with a budget that never overruns.

Unified routing between self-hosted models and external providers like OpenAI and Anthropic

Many companies run both: a self-hosted model for the most sensitive workload and hosted models for everything else. LiteLLM can route to your own vLLM endpoint and to any OpenAI-compatible upstream, so Standard Compute can sit behind it as the upstream for Claude, GPT, Gemini and the open models you do not host.

Your applications keep one base URL, the self-hosted model stays in your cluster, and hosted traffic gets Standard Compute's provider allow-lists, Europe rule and per-key budgets.

model_list:
  - model_name: internal-llm
    litellm_params:
      model: hosted_vllm/your-open-model
      api_base: http://vllm.internal:8000/v1
  - model_name: standardcompute
    litellm_params:
      model: openai/standardcompute
      api_base: https://api.stdcmpt.com/v1
      api_key: os.environ/STANDARD_COMPUTE_KEY

LiteLLM config: your own vLLM model next to Standard Compute as an OpenAI-compatible upstream

Centralized enterprise AI model hub vs connecting directly to individual model provider APIs

Connecting each team directly to each provider is quick for the first project. By the fifth, security reviews every provider contract separately, finance reconciles several invoices, and each provider's rate limit applies to the whole organization.

Centralized enterprise AI model hub vs connecting directly to individual model provider APIs
	Direct provider APIs	Centralized hub (Standard Compute)
Contracts and reviews	Direct provider APIs: One per provider	Centralized hub (Standard Compute): One DPA, a public sub-processor list, 14 days' notice before a new sub-processor
Model choice	Direct provider APIs: Locked to each provider's catalog	Centralized hub (Standard Compute): Frontier and open models through one API, with model allow-lists per key
Outages and quotas	Direct provider APIs: One provider's quota or outage stops the app	Centralized hub (Standard Compute): Requests fail over to another approved provider of the same model, inside your rules
Spend	Direct provider APIs: Per-token invoices per provider	Centralized hub (Standard Compute): Flat monthly plan per person or app, caps per key and for the company
Access	Direct provider APIs: Provider keys shared by hand	Centralized hub (Standard Compute): SAML SSO and a key per person and per app

Questions from IT, security and finance

What open-source AI stacks support an OpenAI-compatible proxy like LiteLLM and run on Kubernetes?: The LiteLLM proxy is a self-hosted OpenAI-compatible gateway with Helm charts in its docs, and vLLM serves open-weight models through an OpenAI-compatible HTTP server. Together they form a self-hosted alternative to OpenAI's API. Standard Compute is the managed alternative with nothing to deploy.
Is self-hosting cheaper than a managed gateway?: It is cheaper when one open model carries steady, high-volume traffic on GPUs you keep busy. For mixed work, Standard Compute's routing to the cheapest capable model and flat plans per person or app usually costs less than idle hardware plus the staff to run it.
Can Standard Compute keep data in Europe like a self-hosted stack?: Set a Europe rule for your company, a team or a single key, and every request, including retries and fallbacks, is served only by approved European inference endpoints. If no European endpoint can serve it, the request is refused before your prompt is sent.
Does Standard Compute store prompts?: We process your prompts in memory to serve each request and never write them to storage or logs. Standard Compute never trains on your prompts or outputs.