Self-hosted deployment

One /v1 endpoint for every provider. Point your OpenAI client at a new base URL and key, bring your own provider keys, and pay 0% markup on tokens.

You can run Autonomous Relay Agents inside your own network or cloud account instead of using the hosted service. Prompts, completions, provider keys and the usage ledger then stay on infrastructure you control, and your applications call your own base URL.

Self-hosting is an Enterprise option, available only under a signed Enterprise agreement. The deployment package is not open source and is not offered under an open-source licence. To discuss it, contact us through the Enterprise page.

What the package contains

  • The gateway image: the same container we run in production. It serves the API and also runs the database migrations, the monthly partition maintenance and the model catalogue import.
  • The dashboard image: the web interface as static files behind nginx, which forwards /v1, /health and the OpenAPI document to the gateway so everything is on one origin.
  • A Docker Compose file for a single machine, with PostgreSQL 16, health checks and named volumes.
  • A Helm chart for Kubernetes. Migrations and the catalogue import run as hooks before each install or upgrade, and partition maintenance runs as a monthly CronJob.
  • An operator guide listing every setting, which ones are required and what each one turns on.

What you provide

  • PostgreSQL 16. We recommend a managed service with automated backups and point-in-time recovery.
  • A TLS endpoint, such as your load balancer or ingress controller, and the public address users will type.
  • A key-management key to protect stored provider keys. Every provider key is encrypted with its own data key, and that data key is wrapped by your key-management key, so a database copy alone reveals nothing. Today that key is a Google Cloud KMS key, which the gateway can reach from any network using a service-account credential. Tell us if you need another key manager.
  • Redis, only if you run more than one gateway replica, so rate limits are counted across all of them.
  • Outbound HTTPS to the model providers you use. Email alerts and Google or GitHub sign-in are optional and need outbound access to those services only when you turn them on.

No telemetry

A self-hosted gateway does not send usage, prompts or analytics to us. The dashboard image is built with analytics turned off. The only outbound calls are the ones you configure: model providers, and optionally email, OAuth sign-in, and the destinations you add for request records.

Upgrades

Each release is a new image tag. Pull it, run the migrations, then roll the gateway. The Helm chart does this in that order on helm upgrade, and a failed migration stops the upgrade while the previous version keeps serving. With Docker Compose, docker compose up -d runs the same steps.

Backups

The database is the only state: organisations, keys, encrypted provider credentials and the ledger. Back it up with your managed service’s snapshots or with pg_dump. Keep the key-management key and the API-key salt as safe as the backups themselves: without them, restored provider keys cannot be decrypted and issued API keys stop working.