Secure LLM API Gateway

TORHAUS.AI helps companies control their AI traffic, reduce compliance risks and cut AI costs while improving reliability.

TORHAUS.AI: the gatehouse for LLM traffic

Every company running AI in production hits the same wall once more than one team shares an API key: vendor keys grant access, but they don't answer who spent what, who may call which model, what was sent, or which provider answered – and each of those questions lands on a different desk. TORHAUS.AI puts one gate in front of every prompt and hands each desk full visibility and control.

Finance · Controlling

AI costs explode?

One line on the vendor invoice, and no way to say which team or project spent it. With TORHAUS.AI every request is priced and booked to a cost centre, including budgets that stop spend at the cap.

→ Control costs

IT · Security

AI governance chaos?

Sensitive data, PII, internal codes — all flowing into model APIs with no way to block, redact or enforce policy. TORHAUS.AI gives you guardrails that stop leaks before they happen.

→ Enforce policy

DPO · Legal

AI compliance blind spot?

No record of which prompts left the company, or what was in them. TORHAUS.AI gives you one audit row per request, so you can see and prove what actually left.

→ Audit prompts

Infrastructure · DevOps

AI routing inefficient?

Every provider switch is a code change and a redeploy across every app. Applications ask for an alias; TORHAUS.AI lets you decide which provider answers, with automatic failover.

→ Route providers

Sovereignty

We provide the control point, you decide what runs behind it.

TORHAUS.AI runs as one binary on infrastructure you control, in a region you chose, talking only to the model endpoints you configured. It has no home to phone. Where your data lives follows from where you install it and which providers you point it at, which is the only kind of sovereignty a piece of software can honestly offer you.

→ How you run it

Hosted in the EU, or nowhere but yours

How it works: Your gateway is a single‑tenant instance, administered through a single bootstrap account whose password the console forces you to rotate before anything else opens.

Chargeback and budgets

An invoice your finance team can take apart.

Finance teams need to know who spent what, and on which project. TORHAUS.AI prices every request at the vendor’s rate, books it to a cost centre, and enforces budgets before spend runs away — giving you the clarity your invoice never had.

Priced per vendor, in the vendor's currency

Prices and billing currency are set per provider endpoint, so a vendor invoicing in EUR and one invoicing in USD are each accounted natively, with nothing converted at write time.

Costs are computed in integer nanos, never floats, so millions of fractional-cent requests still add up to the right number.
A display currency rolls multi-currency spend up at the reference rate for the day the spend happened, so last quarter's total reads the same next quarter.
Traffic with no price list is flagged unpriced rather than quietly counted as free.

Budgets at three levels

Spend caps on an organization, a team or a single virtual key, per day, week, month or lifetime.

An exhausted budget returns 402, not 429. A 429 makes SDK retry logic back off and try again against a condition that will never clear on its own.
Enforcement is continuous rather than end-of-month, so you find out at the cap instead of on the invoice.
Rate limits sit beside them: requests per minute and tokens per minute, per key.

Tenancy that maps to your org chart

Organizations, teams under them, and any number of virtual keys per team, each with a cost centre you can hand to accounting.

Per-key allow-lists for models and for operations, so a key issued for embeddings cannot quietly start generating images.
A key carries 178 bits of entropy, is stored only as a digest, and is shown exactly once. The gateway cannot recover it.
Revocation is immediate and permanent.

Guardrails

Put a gate in front of every prompt.

TORHAUS.AI is your security layer that enforces data security at the prompt level: Guardrails let you define what must not leave your company — PII, secrets, internal identifiers — before a prompt reaches any provider. Policies apply globally or per team, and move between staging and production as files, not as hours of clicking.

Evaluators that ship with it

regex keywords pii token ceiling cost ceiling model allow-list model deny-list content length tool-argument schema webhook

The PII evaluator detects email addresses, phone numbers, credit card numbers, IBANs, IP addresses, US social security numbers and API keys, checksum-verified with Luhn and mod‑97 so a sixteen-digit order number is not mistaken for a card.
Rules are opt-in: you choose the detectors, the stage and the scope. A fresh gateway enforces nothing until you bind your first rule.
Redaction rewrites the prompt text on its way to the provider, and again in whatever you store.
The webhook evaluator hands the decision to your own HTTPS endpoint, so a classifier you already run stays the one that judges content.
Set fail_closed and a check that errors, an unreachable webhook say, blocks the request instead of letting it through.

Three things a rule can do

Block

The request never reaches the provider. The client gets an OpenAI-shaped error it already knows how to handle.

Redact

The match is rewritten before the request leaves, and before anything is written down.

Warn

The request passes and the verdict lands on the audit row. This is how you measure a rule before you enforce it.

Guardrails read the prompt text your users write. Whatever your policy needs beyond that, a house pattern or an internal project code, is a regex rule you add yourself.

Audit trail

One row per request. Complete traceability.

Compliance is becoming mandatory: the EU AI Act phases in from 2025, and DORA and NIS2 already demand logging, access control and vendor oversight. TORHAUS.AI gives you one structured audit row per request — the evidence you need to prove how AI is used inside your company.

Payload capture is off by default. Turning it on for a team is a data protection decision, and the console says so in those words.
Redactions run before storage, and a stored payload that was rewritten is flagged as non-verbatim, so a reader is never misled about what they are looking at.
Two retention clocks: months for request metadata, a shorter window in days for captured payloads, enforced by dropping whole time partitions rather than by row-by-row deletes.
The usage report exports as CSV and your entire guardrail policy set exports as YAML or JSON, so neither is trapped in the interface.
Trusted proxy depth defaults to zero, so X-Forwarded-For is ignored entirely until you say how many reverse proxies really sit in front. Believing that header by default would let any caller write whatever address it liked into your audit log.

REQUEST LOG 01JD8P4W2Q7RX3K…

team: engineering · org acme-gmbh
key: tor_live_…f30c active
alias → target: gateway-balanced → eu-vllm-01
attempts: eu-vllm-00 breaker open, eu-vllm-01 ok
tokens: 1 284 in · 611 out
cost: 0.004137 EUR priced
guardrails: pii redact (2) · token-ceiling pass
latency: total 1 942 ms · overhead 3.1 ms
payload: stored, non-verbatim

Illustrative row. Field names and units are the ones the gateway actually writes.

Alias · gateway-balanced

PRIORITY 1 eu-vllm-01 w 70 eu-vllm-00 w 30 PRIORITY 2 azure-westeu w 100 PRIORITY 3 anthropic w 100

Lower numbers are tried first. Targets sharing a number split traffic by weight. A struck-through target is one whose circuit breaker is open.

Routing

Switch providers without touching your code.

When routing is baked into your apps, every provider hiccup becomes your problem. TORHAUS.AI lifts routing out of the code: clients call an alias, and the gateway handles failover, skipping degraded targets and recovering cleanly.

Under the hood, each alias maps to targets arranged in priority tiers and weighted within a tier. If a provider degrades, its circuit breaker opens, the balancer skips it, and traffic flows to the next tier — with the audit row showing exactly which targets were tried. Recovery happens with a single probe, not a stampede.

Clients speak the OpenAI API. Upstream, TORHAUS.AI talks to OpenAI and anything OpenAI-compatible (Azure OpenAI, vLLM, OpenRouter, Ollama) and to Anthropic, translating in both directions.
Both dialects are hand-rolled rather than taken from a vendor SDK, so a vendor's release schedule is not your release schedule.
A workload moves between vendors by changing what an alias points at. No client-side change, no redeploy.
Per-endpoint timeout, retry count, failure threshold and cooldown, all set in the console.

Drop-in

Instantly compatible with your existing stack.

Integrating TORHAUS.AI in your existing stack is as easy as it can get: Point any OpenAI SDK at the gateway, give it a virtual key, and ask for an alias instead of a model name. That is it.

Your workflows stay exactly as they are. TORHAUS.AI simply sits in the middle and takes care of the hard parts: Upstream, it routes to whatever providers you trust. Downstream, your governance stays in one place. Same client, same methods, same responses — just governed, logged, and routed through your gateway.

app.py openai >= 1.0

# The only change your developers make.
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.example.com/v1",
    api_key="tor_...",       # a virtual key
)

client.chat.completions.create(
    model="gateway-balanced",  # an alias, not a model
    messages=[{"role": "user", "content": "Hello"}],
)

# Charged to a team. Checked against a budget.
# Screened by your guardrails. Written to the log.
→ 200 OK  x-request-id: 01JD8P4W2Q…

Simple Plug-in, no friction. Your applications keep calling the OpenAI API. The gateway keeps everything else working — routing, guardrails, budgets, logging, and model choice.

Latency breakdown · one request

Authenticationlatency_auth_us

Rate limit and budgetlatency_ratelimit_us

Guardrails, requestlatency_guardrail_pre_us

Routinglatency_routing_us

Providerlatency_provider_us

Time to first tokenlatency_ttft_us

Guardrails, responselatency_guardrail_post_us

Gateway overheadlatency_overhead_us

Proportions illustrative. The figures are whatever your own traffic measures, per request.

Speed

The gateway's own cost is a number you can see.

Every stage of a request is timed separately and written to the audit row in microseconds, including the one that matters: total minus the upstream call, which is everything TORHAUS.AI itself spent. The request detail page draws it as a stacked bar, so an outlier stage is obvious rather than argued about.

Written in Go. The routing table, the price list and the guardrail set are immutable snapshots behind an atomic pointer, so the request path takes no lock.
Virtual keys resolve from a hot-path cache; costs are computed in integer arithmetic, not floating point.
Audit writes are asynchronous, so recording history never sits on the request path of a latency-sensitive proxy. Back-pressure drops are counted and alertable, and the writer can be told to block instead if you would rather trade latency for completeness.
Streaming is first class, and time to first token is one of the nine timings.

How you run it

Three hosting models. You choose where your data lives.

Hosting isn’t a technical detail — it’s what your DPO, your Betriebsrat and your platform team must be able to stand behind, and what regulation expects you to prove. TORHAUS.AI allows you to choose the hosting model that best fits your company's needs.

EU-hosted

Managed by us

We run TORHAUS.AI for you on infrastructure in the European Union, and you get the console on day one instead of a deployment project.

Optionally use models from every provider we support without opening a single vendor account, billed to you through Lime Labs on one invoice.
Your teams, keys, budgets and guardrails, configured with you during onboarding.
Prompts reach whichever model provider you choose, under that provider's own terms. We will tell you exactly which ones apply before you send anything real.

Schedule a demo

Self-hosted in your cloud

The binary, in your account, in your region, under your own Kubernetes or Compose, with your own Postgres and Valkey.

Nothing is licensed at runtime, so there is no service that can switch you off.
The operator guide covers what changes for production: TLS termination in front, the split admin port, database SSL.
Support and hardening review from the people who wrote it.

Talk it through

On-premise and air-gapped

Your own hardware, your own network, and a gateway that will happily never see the internet.

Turn the exchange-rate fetch off, or pin a static rate, and the only outbound connections left are the model endpoints on your own network.
Pair it with a local vLLM or Ollama fleet and no prompt has anywhere to go but your own racks.
Split the admin console onto its own port and keep it off the network your developers use.

Talk it through

Learn more

Control your AI traffic. Put a gate in front of it.

Thirty minutes, your questions, and a live gateway rather than slides. Bring the person who will ask what happens to the prompts. That is the part we like doing.

Schedule a demo Talk to Lime Labs

Built and supported from Rosenheim, Germany