Chat completions

The main endpoint, and OpenAI-compatible. Request and response bodies match the OpenAI Chat Completions shape, so existing clients and SDKs work without a wrapper.

Parameters

• model — a catalog id, author/slug. Required unless models is given.
• messages — the conversation, as in OpenAI.
• stream — server-sent events. See Streaming.
• max_completion_tokens (or max_tokens) — an integer from 1 to 10,000,000; anything else is a 400 — temperature, top_p, top_k, stop, seed
• frequency_penalty, presence_penalty, logit_bias, logprobs, top_logprobs; min_p and repetition_penalty where an open-weight host supports them
• tools, tool_choice, parallel_tool_calls, response_format, reasoning
• user — a stable id for your end user, passed to the provider for abuse attribution.
• models — a fallback chain. See below.
• provider — routing preferences. See Routing.

Message content is a string or a list of parts — text, image_url, input_audio, file, video_url. Tool calls and tool results work on every provider, not only the OpenAI-compatible ones. If an endpoint cannot accept one of your parts — audio sent to Anthropic, say — the request moves on to the next endpoint that can, and fails with a 400 if none can.

Model fallback

Pass models instead of model to give an ordered chain. If the first cannot be served, the next is tried, up to three attempts.

{
  "models": [
    "anthropic/claude-sonnet-5",
    "openai/gpt-5.5",
    "google/gemini-3.1-pro"
  ],
  "messages": [{"role": "user", "content": "Hello"}]
}

The chain is walked to exhaustion rather than retried once, and every attempt is recorded so a failure is attributable afterwards.

Response

{
  "id": "gen-...",
  "model": "anthropic/claude-sonnet-5",
  "choices": [{
    "index": 0,
    "message": {"role": "assistant", "content": "Hello."},
    "finish_reason": "stop",
    "native_finish_reason": "end_turn"
  }],
  "usage": {"prompt_tokens": 9, "completion_tokens": 3, "total_tokens": 12}
}

native_finish_reason is the provider’s own value, preserved verbatim alongside the normalised one. Providers disagree about what “stop” means, and collapsing that distinction loses information you may need.

Usage is the provider’s own count, not our estimate. You are billed on what they reported.

App attribution

Send HTTP-Referer with your app’s URL and X-Title with its name, and each request is recorded against your app. X-App-Title and X-OpenRouter-Title are accepted as X-Title, so a client already set up for another router needs no change.

curl -X POST https://autonomousrelay.com/v1/chat/completions \
  -H "Authorization: Bearer $AUTONOMOUSRELAY_API_KEY" \
  -H 'HTTP-Referer: https://myapp.example' \
  -H 'X-Title: My App' \
  -H 'content-type: application/json' \
  -d '{"model": "anthropic/claude-sonnet-5", "messages": [{"role": "user", "content": "Hello"}]}'

• Only the scheme, host and path of the referer are kept. The query string and fragment are dropped before anything is stored.
• X-App-Visibility: hidden marks the app private, so it will never appear in a public app listing. It is read on the app’s first request only.
• X-App-Categories takes up to two comma-separated categories per request, and an app keeps up to ten.
• A localhost or IP-address referer is recorded on the request but does not create an app.

Anthropic’s Messages API

Set an Anthropic SDK’s base URL to this host and it works unchanged: the key in x-api-key, the request and the answer in Anthropic’s shape, and a stream as Anthropic’s events. The request is served by the same pipeline as a chat completion, so routing, budgets, guardrails, presets and server tools all apply, and any model can answer it. A Claude name such as claude-opus-4-8 means anthropic/claude-opus-4.8; fallbacks takes up to three {"model": ...} objects. Anthropic’s own server tools are refused with a 400: declare autonomousrelay:* tools instead.

OpenAI’s Responses API

The Responses API, streaming included, served by the same pipeline as a chat completion and on any model. It is stateless: send the whole conversation as input. store: true, previous_response_id, conversation and background need a stored response and are refused with a 400 that says so. Function tools and autonomousrelay:* server tools work; OpenAI’s built-in tools are refused.

Legacy text completions

For clients that still call it. The prompt is sent as one user message, routed like any chat completion, and the answer comes back as choices[0].text. One prompt per request; suffix, echo, best_of, n above 1 and logprobs need a true completion model and are refused with a 400.

Embeddings

OpenAI’s request and response. input is a string or a list of up to 2,048 strings; encoding_format is float or base64; dimensionsworks where the model can shorten its output. model, models and provider route exactly as they do here, through the same budgets, content filters and guardrails, and the request is billed on the provider’s prompt-token count. Only embedding models are candidates: naming a chat model is a 400 that says so.

Reranking

Send a query and up to 1,000 documents (strings, or {"text": ...}); each comes back with a relevance_score, most relevant first. top_n cuts the list, return_documents includes the text, and instruction steers the task on models that take one. Routed and billed like embeddings. The Qwen3 rerankers are served by DeepInfra.

Images

OpenAI’s request and response: a prompt, up to 4 images (n), and a size such as 1024x1024. Images come back as b64_json. Each is billed at the model’s published price, per image or per megapixel of the size, which the model’s endpoint listing shows with its unit. A model whose price cannot be computed from the request is not served yet.

Audio

OpenAI’s audio API. Speech takes an input text and returns the audio, billed per character; each model has its own voice names. Transcription takes a multipart file of up to 25 MB and returns its text (json, verbose_json or text), billed per second of audio. Because a speech answer is audio, the request id is in the x-autonomous-relay-generation-id header.

Batches

Up to 10,000 chat requests, each {"custom_id": ..., "body": {...}}, run by the provider’s own batch API at its batch price, within 24 hours, on your provider key. Poll GET /v1/batches/:id; when it has ended, GET /v1/batches/:id/results streams one line per request. The batch is billed once, at the end, on its requests’ usage. Any model with a :batch row in the catalogue, on six providers; Batch API has the details.

Inspecting a request afterwards

Returns the metadata for one request: which endpoint served it, token counts, latency, cost, the price version applied and the app it came from. Never the content: this endpoint does not return it under any setting. If your organisation enables request logging — off by default, and not switchable yet — stored content is read from its own endpoint instead.

Feedback

Attach a score to a request your app made, with the same inference key, and see on the Activity page which models your users rate well. value runs from -1 (thumbs down) to 1 (thumbs up); map a finer rating into that range. label is an optional slug such as helpful. Posting again from the same key replaces the score. There is no comment field: free text is content, and content is not stored.

curl -X POST https://autonomousrelay.com/v1/generation/feedback \
  -H "Authorization: Bearer $AUTONOMOUSRELAY_API_KEY" \
  -H 'content-type: application/json' \
  -d '{"id": "gen-...", "value": 1, "label": "helpful"}'

GET /v1/generation/feedback?id=gen-... reads the scores back; another organisation’s id is a 404.

Caching

An exact-match response cache, off by default and turned on per inference key, answers a repeated non-streaming request with x-cache: HIT at no cost. Response cache has what a hit is, what is never cached, and the organisation’s view of it.