Chat completions

POST /v1/chat/completions follows the OpenAI Chat Completions format, streamed or not.

Parameters

Parameter	Notes
`model`	Required. A BaseURL model name, for example `gateway-fast`.
`messages`	Required. A non-empty array of messages, each with a `role`.
`max_tokens` / `max_completion_tokens`	The most tokens the answer may use. Set it: it also sets how much balance is held while the request runs. Without it, each model has a default cap, lowered to what your balance covers if needed.
`stream`	`true` streams the answer as server-sent events. The last chunk carries usage.
`temperature`, `tools` and other OpenAI parameters	Passed to the model unchanged. If the model does not accept one, you get its `400` error and are not charged.
`n`	Must be 1. Several choices per request are not supported.

Streaming

stream = client.chat.completions.create(
    model="gateway-fast",
    messages=[{"role": "user", "content": "Write a haiku about Lagos traffic."}],
    max_tokens=200,
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Each event is an OpenAI-style chat.completion.chunk. The stream ends with a chunk holding usage and then data: [DONE]. If you disconnect early, you pay only for what was generated.

Response headers

Header	Meaning
`X-Request-Id`	The request id. Include it when you contact support.
`X-Xllm-Cost-Units`	What the request cost, in USD micro-units (1,000,000 = $1.00). Non-streamed responses only; for streams, see your usage page.
`X-Xllm-Max-Tokens-Applied`	Present when BaseURL set or lowered `max_tokens` for you, with the value used.

Safe retries with Idempotency-Key

Send Idempotency-Key: <unique id> to retry a request safely after a network error. Within 24 hours, the same key with the same body replays the stored answer of a completed non-streamed request without charging again.

Situation	Answer
Same key, same body, request still running	`409 idempotency_key_in_progress`, retry after `Retry-After`
Same key, same body, completed stream	`409 idempotency_key_replay`; streams are not replayed
Same key, different body	`422 idempotency_key_reused`
The first request was refused or charged nothing	The key is freed: retry normally

Questions? Email [email protected] with your X-Request-Id.