Everything the gateway does, on one page.
This is the complete operating manual for the lean-ctx AI Gateway — the same software that serves the live demo. Every command is real, every config block is lifted from a running deployment, and the published demo credentials work in every chapter. Skim the sidebar, or read straight through.
Navigate with ← → between chapters · ⌘P prints clean
One engine, two scales: your laptop or your whole org.
Everything else in the lean-ctx docs runs on one machine. The gateway is the same engine deployed once for the whole team: every IDE, CLI agent and script points at it instead of talking to the model providers directly. Requests are compressed on the wire, attributed per person, metered to your Postgres, governed by org policy — and the savings are recorded in a form you can hand to finance.
It ships compiled into the default binary (the gateway-server feature,
since v3.9.0). The local-free invariant holds: until you run
lean-ctx gateway serve, nothing about solo use changes.
Local proxy vs. team gateway
| Local proxy | Team gateway | |
|---|---|---|
| Bind | 127.0.0.1 only | proxy_bind_host (e.g. 0.0.0.0) + Host-header allowlist via proxy_allowed_hosts |
| Who calls it | your own AI tools | every seat in the org |
| Auth | local token / provider-key fallback | mandatory per-person Bearer keys — the fallback is hard-disabled on any non-loopback bind |
| Upstream credentials | forwards your key verbatim | injects org keys from env (api_key_env) — provider keys never live on laptops |
| Metering | local ledger | Postgres usage_events, per person / team / project / model |
| Governance | — | signed org policy: model ceiling, budgets, rate limits |
| Rate limit | unlimited | 50 rps floor (burst 100) by default, tunable via proxy_max_rps |
From zero to a running gateway in one command.
gateway init scaffolds a ready-to-run deployment — docker compose file,
.env, per-person key file and a step-by-step README. gateway doctor
preflights config, secrets, database and ports before go-live.
# 1 · scaffold a deployment for your org lean-ctx gateway init acme-gateway --org="Acme" --seats=800 \ --reference-model=claude-opus-4.5 \ --person=alice@acme.com --person=bob@acme.com # 2 · bring it up and preflight it cd acme-gateway && docker compose up -d lean-ctx gateway doctor # config, secrets, DB, ports # 3 · the proxy listens on :8484, the admin console on :8485
Each developer changes two environment variables — no plugin, no IDE change:
export ANTHROPIC_BASE_URL=https://gateway.acme.internal export ANTHROPIC_API_KEY=<personal gateway key> # Claude Code, Cursor and scripts work unchanged — # the gateway speaks the providers' own wire protocols.
LEAN_CTX_GATEWAY_ADMIN_TOKEN, the metering DSN from DATABASE_URL —
neither ever lives in the config file. The admin console and /metrics
(Prometheus) bind to loopback by default; exposing them is an explicit decision
([gateway_server] admin_bind_host).
One container in the request path — and nothing else changes.
gateway serve starts the multi-provider reverse proxy — Anthropic, OpenAI,
Gemini, Ollama and any custom registry provider — plus a token-protected admin console on a
separate port. Custom upstreams (Azure AI Foundry, OpenRouter, self-hosted vLLM/Ollama)
join as [[proxy.providers]] entries served under
/providers/<id>/…, and wire-shape translation means an
Anthropic-speaking IDE can call an OpenAI-hosted model and vice versa.
- Metering — every request lands in Postgres
usage_events, attributed person × team × project × model × provider. - Compression — prompts shrink on the wire before they reach the provider; each delta is measured, not estimated.
- Governance — a signed org policy turns the same path into an enforcement point (chapter 05).
Hashed, rotatable, revocable — attribution that survives.
Every seat authenticates with an individual Bearer key. The gateway stores
SHA-256 hashes only — plaintext is shown exactly once at creation.
rotate replaces all of a person's keys in one atomic file swap, so there is
no window where the person has zero valid keys, and team/project attribution survives
the swap.
lean-ctx gateway keys add --person=alice@acme.com --team=platform --project=checkout
lean-ctx gateway keys list
lean-ctx gateway keys rotate --person=alice@acme.com # atomic, attribution kept
lean-ctx gateway keys revoke --person=alice@acme.com
api_key_env) — they never sit on laptops. A leaver is one
revoke; a suspected leak is one rotate.
Identity at org scale — SSO & SCIM (enterprise plane)
Keys answer "who made this request"; the licensed enterprise plane answers
"who is allowed to exist here". It connects the gateway to your identity provider
(Entra ID, Okta): SSO (OIDC) signs people into the console and /me with
their corporate login, and SCIM provisioning keeps the seat list in sync with HR —
a joiner gets a key on day one, a leaver loses model access the moment they are offboarded,
with no manual list to forget.
| Plan | Identity | Audit retention |
|---|---|---|
| Open source | per-person keys (this demo) | your instance, your rules |
| Business | + SSO (OIDC) | 365 days |
| Enterprise | + SCIM provisioning | 10 years |
Your org's model names, your routing decision.
[proxy.routing.aliases] defines the org's model namespace: stable names
like acme/fast that resolve to a concrete provider:model target.
Clients discover them via GET /v1/models — content-negotiated, so OpenAI-shape
and Anthropic-shape clients each get their native list format. A model swap for 800 seats
is one config line, not a laptop rollout.
[proxy.routing] enabled = true [proxy.routing.aliases] "acme/fast" = "foundry:gpt-4o-mini" "acme/standard" = "anthropic:claude-sonnet-4-5" "acme/premium" = "anthropic:claude-opus-4-6"
And this is the actual routing block of the live demo — small open-source models on an in-stack Ollama, booked at a transparent shadow rate:
[[proxy.providers]] id = "ollama" shape = "openai" base_url = "http://leanctx-demo-ollama:11434" local = true # booked at the shadow rate, never $0 [proxy.routing.aliases] "demo/fast" = "ollama:gemma4:e2b" "demo/standard" = "ollama:gemma4:e4b" [proxy.baseline] reference_model = "claude-opus-4.5" local_shadow_rate_per_mtok = 0.25
Routing is fail-open by construction: any lookup or classification miss routes
nothing and forwards the request unchanged. The resolved route is stamped into the ledger
as routed_from, so every substitution is auditable.
Governance that answers in the caller's own wire shape.
Under a signed, pinned, enforced = true org policy, the forward path
refuses three things — and nothing else:
| Gate | Trigger | Answer |
|---|---|---|
| Model ceiling | model outside [routing].allowed_models | 403 in the caller's wire shape |
| Budgets | spend above [budgets] caps — per person/UTC-day or project/UTC-month | 429 |
| Rate limit | beyond max_requests_per_minute_per_person | 429 with an honest Retry-After |
Refusals are counted on
leanctx_policy_blocked_total{reason="model_ceiling"|"budget"|"rate_limit"}
for your Prometheus. Tools degrade gracefully because errors arrive in the protocol they
already speak.
Org-wide visibility for you, privacy for each engineer.
The admin console shows live spend, verified savings and the avoided-cost baseline —
person × project × model × provider. Its data API is Bearer-guarded and, on the demo,
strictly read-only. The personal /me view is the other half of
the deal: each person signs in with their own gateway key and sees exactly their
spend, savings, trend, models and projects — never anyone else's. Dark/light, 24h–90d
windows, savings-share KPI. Cost awareness without surveillance.
Admin console
The /me page
Savings your auditors can verify without trusting us.
Metering records both what each request cost and what it would have cost
uncompressed against a contractually frozen [proxy.baseline] reference_model.
Local inference (Ollama/vLLM) is booked at a transparent shadow rate instead of
zero, so savings stay honest.
gateway report renders the org's numbers as HTML;
gateway evidence exports Ed25519-signed daily aggregates that anyone
can verify offline — the foundation for savings-based pricing.
lean-ctx gateway report --out=lean-ctx-report.html # last 30 days lean-ctx gateway evidence --out=leanctx-evidence.json # signed export lean-ctx gateway evidence verify --file=leanctx-evidence.json
The demo regenerates its public value report
from live usage_events every 15 minutes — the same
gateway report a customer runs.
Built to be boring: fail-open, bounded, erasable.
Fail-open by design
The gateway never trades availability for bookkeeping: if Postgres is briefly unavailable, traffic keeps flowing and metering degrades — a request is never refused because a database was down. Only enforced policy gates refuse requests.
Retention & GDPR
[gateway_server] usage_retention_days bounds how long usage_events
rows live (90 days on the demo); pseudonymize_persons replaces person
identifiers with a stable keyed pseudonym before they reach metering, budgets, dashboards
and logs. Person-scoped export and erasure work either way:
lean-ctx gateway gdpr export --person=alice@acme.com --out=alice.json lean-ctx gateway gdpr delete --person=alice@acme.com --yes
Observability
/metrics exposes Prometheus counters on the admin port (loopback by default) —
including policy refusals by reason, so you can alert on governance events like any other
SLO.
What exactly is running behind demo.leanctx.com.
The demo is a real gateway instance: real open-source-model inference (in-stack
Ollama), real wire compression, real usage_events metering, real Ed25519
evidence. A traffic generator drives genuine agentic coding requests over real open-source
code through the public product path — it never writes to the database.
| Surface | What it is | Access |
|---|---|---|
| demo.leanctx.com | sales overview + live KPIs + value report | public |
| console.demo… | admin console (org-wide numbers) | published admin token — API is GET-only |
| me.demo… | personal /me view of the guest identity | published guest key — only /me*, /api/me/usage, /v1/models, /health are routed |
| inference endpoints | /v1/chat/completions, /v1/messages, /providers/* | not routed publicly — published keys cannot drive the demo's LLM |
…@demo.leanctx.com) exist so the org views have people in them; every number
they generate is measured, not invented.
Every subcommand, straight from --help.
This block is the gateway's own help text — reproduced verbatim from the binary that serves this page's demo:
lean-ctx gateway — self-hosted org gateway (proxy + usage store + admin console)
Usage:
lean-ctx gateway serve [--port=8484] [--admin-port=8485]
lean-ctx gateway init [dir] [--org="Acme AG"] [--seats=800]
[--reference-model=claude-opus-4.5] [--person=a@x …]
lean-ctx gateway keys <add|list|rotate|revoke> [--person=..] [--team=..]
[--project=..] [--file=..]
lean-ctx gateway doctor [--dir=.] [--port=8484] [--admin-port=8485]
lean-ctx gateway report [--from=ISO] [--to=ISO] [--out=report.html]
lean-ctx gateway evidence [--from=ISO] [--to=ISO] [--out=evidence.json]
lean-ctx gateway evidence verify --file=evidence.json
lean-ctx gateway gdpr <export|delete> --person=<email> [--out=..] [--yes]
Environment:
DATABASE_URL Postgres for usage_events (off → metering disabled, fail-open)
LEAN_CTX_GATEWAY_ADMIN_TOKEN Bearer token of the admin console (off → admin disabled)
LEAN_CTX_PROXY_TOKEN proxy Bearer token (else session token)
Config ([gateway_server] in config.toml): seats, org_label, admin_url.
Bind host/allowlist/rate limit: proxy_bind_host, proxy_allowed_hosts, proxy_max_rps.
Full engine documentation lives at leanctx.com/docs — including the local proxy, the savings ledger and the configuration reference.