Is It Safe to Run Your Company's Code Through an LLM Gateway Subscription?
The three fears are legitimate and they have concrete answers. Training: reputable gateways don't train on your traffic and can force no-training terms upstream. Standard Compute does not train on your prompts. Keys under a Compliance region rule only reach provider endpoints with zero data retention and no training; keys without a rule use standard routing, where each provider's own terms apply. Retention: ask specifically about prompt/response payloads, not 'data'. Standard Compute passes payloads through in memory and does not store or log them; account metadata (email, usage counters) is what's retained. Quiet downgrades: routing does choose models per request (that's the product, not a trick), but the honest test is whether you can pin a model when it matters and whether requests are protected against silent cross-model fallbacks (ours are). The uncomfortable truth for the whole category: a gateway adds a party to your data path, so the right posture is verifying each link, which is why we publish a 95-provider residency and retention directory instead of asking you to trust adjectives.
The three fears, mapped to what you should verify
| Fear | What to verify | Standard Compute's answer |
|---|---|---|
| 'They train on my code' | Gateway's own training policy AND whether it enforces no-training upstream | We don't train on your prompts/outputs. Keys with a Compliance rule force no-training and zero retention upstream; keys without one follow each provider's own terms |
| 'My prompts sit in logs forever' | Payload retention specifically, not account metadata | Prompt/response payloads pass through in memory and are discarded on delivery; not stored, not logged |
| 'Routing = quietly serving the cheapest model' | Can you pin? Are fallbacks cross-model? Is the served model visible? | Pin any model in the dashboard; requests are pinned against cross-model fallbacks; routing otherwise picks per request (openly; it's the product) |
Where your traffic physically goes
A gateway request has two hops: the gateway's routing layer, then an upstream inference provider. The second hop is where most privacy variance lives: providers differ enormously on retention, training defaults, and where their GPUs physically sit. We mapped that landscape across ~95 providers (residency, retention, training posture, verified from primary sources) and published it as the /providers directory; it exists precisely because 'trust the gateway' is not an argument.
For regulated work, region enforcement matters more than promises. Compliance rules are opt-in: they restrict inference to Europe, the United States or both, for personal keys, a whole company, a team or one key, using exact provider endpoint allowlists. Restricted keys also require zero data retention and no training at the provider, and requests that can't be served inside the rule fail before any content is sent, rather than silently routing elsewhere. Keys without a rule route across all providers as usual. Restricting regions can reduce model availability and consume budget faster; that trade-off is disclosed before you save the policy. Which models run in Europe, and what stays outside the restriction: EU LLM API for coding agents.
The honest caveats no gateway advertises
- Provider-level zero-data-retention can still permit temporary prompt caches within the provider's ZDR terms. 'Zero retention' has a technical definition; read it.
- Per-request protections are not the same as end-to-end verification: we disclose openly that some upstream account-level settings are still under independent review, and we keep that disclosure up until it's verified. Ask any gateway for the equivalent honesty.
- A gateway concentrates risk as well as convenience: one party sees all your traffic instead of three seeing slices. If your threat model can't accept that, direct enterprise contracts with audited providers are the right answer, genuinely.
The catch
There is no zero-trust option in hosted inference: someone's GPUs see your prompts. The choice is between vendors you verify and vendors you assume. Concretely: get the payload-retention answer in writing, confirm no-training flows upstream, test that model pinning works, and treat any gateway that answers with adjectives instead of mechanisms as a no.
Where Standard Compute fits
Standard Compute's posture: no payload storage (in-memory pass-through), no training on your prompts, opt-in Compliance rules that restrict inference to Europe or the United States with zero-data-retention and no-training flags upstream and fail-closed behavior, model pinning with no cross-model fallbacks, and the /providers directory documenting where 95 providers actually stand on residency and retention. Full details at /data-privacy and /security, written to be checked, not skimmed.
FAQ
Does tool calling work properly through an OpenAI-compatible gateway?
Yes. Function/tool calls are part of the standard API shape and pass through unchanged; agents like Cline that lean hard on tool calls run against gateways daily (client-side tools work; server-side stateful features are the thing to check per gateway). The practical test: run one tool-heavy task before committing a workflow.
How much latency does the gateway hop add?
Routing overhead is a fraction of a second per request, slightly more when the router classifies, less on pinned models. For agent workloads, model generation time dominates round-trips by an order of magnitude, so the hop rarely shows up in wall-clock time; still, measure end-to-end on your own tasks rather than trusting any vendor's number, ours included.
Can these subscriptions sustain parallel agents overnight?
Plan-dependent, and this is a sizing question, not a trust question: parallel agents multiply budget consumption, and entry tiers on any flat service are sized for one agent at a time. On Standard Compute, parallel-agent work starts at the Economy tier and up; the budget is visible live in the dashboard, not hidden behind concurrency caps.
Does Claude Code with a custom base URL still work with MCP servers?
Yes. MCP servers run client-side in Claude Code and are unaffected by where the model requests go; the base-URL swap changes the inference endpoint, not your local tool integrations. Server-side features that depend on Anthropic's own infrastructure are the exception to verify.
If I switch my agent to a gateway and it doesn't work out, how hard is switching back?
It is exactly the config change in reverse: base URL and key back to the original provider. No data migration, no lock-in mechanics; your code and history never lived at the gateway. This reversibility is worth using: run a two-week real-work trial before moving anything permanent.
Can I expense a flat-rate LLM subscription with proper invoices?
Billing runs on Stripe. In a company workspace, each key's plan is its own Stripe subscription with a monthly invoice in the company's name, including address and VAT number, so finance gets company invoices instead of five expense claims.
Looking at a flat-rate LLM subscription for Cline: what is the data retention story, and is there a zero-retention option?
On Standard Compute, request logs hold metadata (model, tokens, cost, latency, status), not the text of your prompts, diffs or responses. The zero-retention option is a Compliance rule: turn on Europe, the United States or both for one key, a team or the whole company, and requests go only to provider endpoints with zero data retention and no training, with response storage switched off. Without a rule, each provider's own retention terms apply. Two limits we state openly: a provider's zero-retention terms can still allow temporary prompt caches, and for smart-routed requests on keys without a rule, the routing classifier's short note (up to 120 characters, which can quote the prompt) is kept in routing logs.
I build software for clients under NDA. If I use a subscription LLM API with Aider on client code, what should I check?
Whether the NDA allows sending code to third-party processors, and which ones. On Standard Compute the path is our gateway on Google Cloud, then OpenRouter, then the provider serving the model; we do not train on prompts, and request logs hold metadata, not prompt text. For client code, put a Compliance rule on that client's key so requests only reach provider endpoints with zero data retention and no training, in Europe or the United States, and fail instead of going elsewhere. Pin the model the client approved, which also skips the routing classifier, and /add only the files the task needs. This is not legal advice: if the NDA names approved vendors, get the client's answer in writing.
Do any flat-rate LLM gateways have SOC 2 or a DPA that procurement will accept?
Standard Compute does not have a SOC 2 report or ISO 27001 certification yet, and it does not publish a standard DPA, so if SOC 2 is a hard requirement it does not pass today. What exists in writing: Terms under Norwegian law (an EEA country, so GDPR applies), a data and privacy page with the sub-processor list, request logs that hold metadata rather than prompt or response text, opt-in Europe or United States inference rules that require zero data retention and no training at the provider, and an audit log of every compliance change in a company workspace.
Which upstream providers do subscription LLM APIs actually send requests to? I want a subprocessor list before piping our backend through a router.
On Standard Compute every inference request goes from our gateway, which runs on Google Cloud, to OpenRouter, and from there to an endpoint of the model's provider, picked from a fixed provider order per model; with the routed model standardcompute, the latest messages also go to a small classifier model through OpenRouter. Account data is held by Supabase on AWS in Frankfurt, and Stripe handles payments. The /providers directory documents where about 95 inference providers run and what they retain, and a Compliance rule narrows every model to approved endpoints in Europe or the United States.