--- title: Cold starts and SIE Relay description: "How SIE Cloud answers requests while a model's dedicated GPU capacity starts, how to see who served a request, and how to turn SIE Relay off." canonical_url: https://superlinked.com/docs/cloud/sie-relay last_updated: 2026-10-16 --- SIE Cloud runs models on dedicated GPU capacity, and that capacity can scale to zero when a model has no traffic. Starting it again takes time. While it starts, SIE Relay can answer the request through a vetted inference partner, so your application gets an answer instead of waiting for the cold start. A relayed request is validated the same way and returns the same response format, although latency and output can differ slightly from dedicated capacity. It costs the same as a request served by dedicated capacity, and the response headers say which side served it. ## When SIE Relay serves a request The platform decides which requests to relay. When a model's dedicated capacity is not ready, because it has scaled to zero or is still starting, a request for that model can be relayed. Dedicated capacity starts when sustained demand warrants it, so a model with little traffic can be served by SIE Relay for longer. Once dedicated capacity is ready, it serves the model's requests again. A request is relayed only before dedicated capacity accepts it, so it never runs twice. Some models and profiles are served only through SIE Relay. The console labels them **Served by SIE Relay**. For a model with dedicated capacity, a request is never relayed when: - it is an embedding or reranking request. SIE Relay serves only generation and extraction - it carries the `X-SIE-Remote: forbid` header, which the SDK `remote` option sets - your organization turned SIE Relay off - it names a profile that runs on dedicated capacity, or asks for grammar-constrained structured output - no SIE Relay partner serves the model in the region of your API key Such a request starts dedicated capacity if it is not running, then waits for it, as described in [Retry while capacity starts](#retry-while-capacity-starts). Batch jobs are never relayed either. For a model with dedicated capacity, a cold start delays them. The batch and jobs APIs refuse a model or profile served only through SIE Relay with `400 INVALID_REQUEST`, so send those requests to the realtime endpoints. ## See who served a request ### Response headers | Header | Meaning | | --- | --- | | `X-SIE-Served-By` | `local` for dedicated capacity, `remote` for anything else | | `X-SIE-Upstream` | Who served a `remote` request, `sie-relay` for SIE Relay | | `X-SIE-Fallback-Reason` | Why the request was relayed, for example `provisioning` | | `X-SIE-Fallback-Error` | Error code of a failed relay attempt | `provisioning` means dedicated capacity was not ready, because it had scaled to zero or was still starting. A model or profile served only through SIE Relay sends no `X-SIE-Fallback-Reason`, because nothing else serves it. When a relay attempt fails, the response is the refusal from dedicated capacity, with `X-SIE-Served-By: local` and the code of the failed attempt in `X-SIE-Fallback-Error`. The headers appear on successful responses and on server errors (`5xx`). Client errors (`4xx`) carry none of them. A relayed answer carries these headers: ```http HTTP/1.1 200 OK X-SIE-Served-By: remote X-SIE-Upstream: sie-relay X-SIE-Fallback-Reason: provisioning ``` ### In the SDKs The Python and TypeScript SDKs read these headers into the request metadata of each result: `result["request"]` in Python and `result.request` in TypeScript. A field is absent when the response did not carry its header. These fields and the `remote` client option need SDK version 0.10.0 or later. | Header | Python | TypeScript | | --- | --- | --- | | `X-SIE-Served-By` | `served_by` | `servedBy` | | `X-SIE-Upstream` | `upstream` | `upstream` | | `X-SIE-Fallback-Reason` | `fallback_reason` | `fallbackReason` | | `X-SIE-Fallback-Error` | `fallback_error` | `fallbackError` | In Python, `RequestError` and `ServerError` carry the same metadata in `.request`, read from the error response. In current SDK releases, `ProvisioningError` carries no request metadata, so read `X-SIE-Fallback-Error` from the raw response to see why a relay attempt failed. Streamed responses send the headers too, but the SDK streaming methods return chunks without them. #### Python ```python import os from sie_sdk import SIEClient with SIEClient( "https://api.superlinked.com", api_key=os.environ["SIE_API_KEY"], ) as client: result = client.generate( "Qwen/Qwen3.5-4B", "Explain vector search in one sentence.", max_new_tokens=64, ) request = result.get("request", {}) print(request.get("served_by")) # "local" or "remote" print(request.get("upstream")) # "sie-relay" when SIE Relay served it print(request.get("fallback_reason")) # "provisioning" when capacity was not ready ``` #### TypeScript ```typescript import { SIEClient } from "@superlinked/sie-sdk"; const client = new SIEClient("https://api.superlinked.com", { apiKey: process.env.SIE_API_KEY, }); try { const result = await client.generate( "Qwen/Qwen3.5-4B", "Explain vector search in one sentence.", { maxNewTokens: 64 }, ); const { servedBy, upstream, fallbackReason } = result.request ?? {}; console.log(servedBy); // "local" or "remote" console.log(upstream); // "sie-relay" when SIE Relay served it console.log(fallbackReason); // "provisioning" when capacity was not ready } finally { await client.close(); } ``` ## Opt out of SIE Relay A header opts out a single request or client. The organization setting opts out every request your organization makes. ### Keep requests on dedicated capacity Send the `X-SIE-Remote: forbid` header. Dedicated capacity then answers the request, including the `503` refusal while capacity starts. The header takes exactly one value. Any value other than `forbid`, or a second `X-SIE-Remote` header, is refused with `400`. A model or profile served only through SIE Relay has no dedicated capacity, so a `forbid` request for it is refused with `400` as well. On `/v1/encode`, `/v1/score` and `/v1/extract` the error code is `INVALID_INPUT`. `/v1/generate` and the OpenAI-compatible endpoints answer with an OpenAI error object instead, whose `code` is `invalid_request`. In the SDKs, the `remote` client option sends the header with every request the client makes. There is no per-call option, so create a second client for the requests that must stay on dedicated capacity. Both SDKs reject any value other than `"forbid"` when the client is created. #### Python ```python import os from sie_sdk import SIEAsyncClient, SIEClient # Every request from these clients stays on dedicated capacity. client = SIEClient( "https://api.superlinked.com", api_key=os.environ["SIE_API_KEY"], remote="forbid", ) async_client = SIEAsyncClient( "https://api.superlinked.com", api_key=os.environ["SIE_API_KEY"], remote="forbid", ) ``` #### TypeScript ```typescript import { SIEClient } from "@superlinked/sie-sdk"; // Every request from this client stays on dedicated capacity. const client = new SIEClient("https://api.superlinked.com", { apiKey: process.env.SIE_API_KEY, remote: "forbid", }); ``` Over plain HTTP, send the header yourself. `-i` prints the response headers: ```bash curl -i \ https://api.superlinked.com/v1/chat/completions \ -H "Authorization: Bearer $SIE_API_KEY" \ -H "Content-Type: application/json" \ -H "X-SIE-Remote: forbid" \ -d '{"model":"Qwen/Qwen3.5-4B","messages":[{"role":"user","content":"hello"}],"max_completion_tokens":16}' ``` ### Turn SIE Relay off for your organization SIE Relay is on by default. Owners and admins can turn it off for the whole organization in the [console](https://console.superlinked.com), under **Settings → SIE Relay**. With SIE Relay off: - requests wait for dedicated capacity during a cold start, as described in [Retry while capacity starts](#retry-while-capacity-starts) - a model or profile served only through SIE Relay is refused with `403 THIRD_PARTY_SERVING_DISABLED`, although it still appears in the model list ## Retry while capacity starts When a request is not relayed and dedicated capacity is still starting, the gateway answers `503` with the error code `PROVISIONING` and a `Retry-After` header, in seconds. Wait that long and send the request again. Capacity keeps starting in the meantime. Keep retrying while the answer is `503 PROVISIONING`. On the native endpoints (`/v1/encode`, `/v1/score`, `/v1/extract` and `/v1/generate`), `Retry-After` estimates how long capacity still needs to start, from 5 to 300 seconds. It is 60 seconds when there is no estimate yet. On the OpenAI-compatible endpoints, such as `/v1/chat/completions` and `/v1/embeddings`, and on `/v1/rerank`, it is at most 60 seconds, so a long start can take several retries. This refusal reaches requests that are [never relayed](#when-sie-relay-serves-a-request), and requests whose relay attempt failed. After a failed attempt, the `503` also carries `X-SIE-Fallback-Error`. Both SDKs retry for you by default. With capacity waiting on (`wait_for_capacity=True` in Python, `waitForCapacity: true` in TypeScript), the client honors `Retry-After` and retries until the provisioning budget runs out: `provision_timeout_s` in Python, 900 seconds by default, or `provisionTimeout` in TypeScript, 900000 milliseconds by default. With waiting off, or once the budget runs out, the client raises `ProvisioningError`. Its `retry_after` in Python (seconds) or `retryAfter` in TypeScript (milliseconds) holds the server's hint. ## Error codes | Code | HTTP status | Meaning | | --- | --- | --- | | `PROVISIONING` | 503 | Dedicated capacity is starting and the request was not relayed. Retry after the `Retry-After` delay | | `THIRD_PARTY_SERVING_DISABLED` | 403 | Your organization turned SIE Relay off, and the model or profile is served only through SIE Relay | | `THIRD_PARTY_RUNTIME_UNLISTED` | 403 | The request needs a model or profile that SIE Relay does not offer | | `THIRD_PARTY_REGION_UNAVAILABLE` | 403 | The model is not available through SIE Relay in the region of your API key | | `INVALID_REQUEST` | 400 | A batch or job names a model or profile served only through SIE Relay | | `INVALID_INPUT` | 400 | On `/v1/encode`, `/v1/score` or `/v1/extract`, `X-SIE-Remote` holds a value other than `forbid` or appears twice, or a `forbid` request asked for a model or profile served only through SIE Relay | | `invalid_request` | 400 | The same `X-SIE-Remote` refusal from `/v1/generate` or an OpenAI-compatible endpoint, in an OpenAI error object | The SDKs raise the `403` and `400` refusals as `RequestError`, with the code in `code` (`invalid_request` for a `400` from `/v1/generate` or an OpenAI-compatible endpoint), and do not retry them. ## Price A request costs the same whether dedicated capacity or SIE Relay serves it. Both are billed at the model's listed price. The usage export marks each row `dedicated` or `relay` in its `served_by` column. ### Usage export A `relay` row covers requests that SIE Relay answered, the ones whose responses carried `X-SIE-Upstream: sie-relay`. Everything else is `dedicated`, including requests served by remote capacity that SIE Cloud operates itself. The CSV from **Usage & billing → Export CSV** in the [console](https://console.superlinked.com) includes the column. ## Data handling SIE Relay partners state that they keep no copy of the requests SIE Relay sends them. SIE Relay serves a model for an API key only in the regions where SIE Cloud offers that model through SIE Relay. Otherwise a request for a model with dedicated capacity waits for that capacity, and a request for a model served only through SIE Relay is refused with `403 THIRD_PARTY_REGION_UNAVAILABLE`. An API key without a pinned region is checked against the region of the SIE Cloud deployment that receives the request.