Pay 36% less than GPT-6.1 Sol and keep 95% of its quality

Use a cheap model for requests it can handle; save the strong model for harder ones. SIE, an open-source inference engine, serves Qwen3 Reranker 4B, an open model that scores how well each request fits your cheap model.

Get started Run in your cloud

SIE Qwen3 Reranker 4B

Sent to GPT-6 Luna

Request

Solve for x: x/2 + x/3 = 5

GPT-6 Luna's answer

x = 6

An LLM as your router cuts the bill by 1.4% at best

GPT-6 Luna as the router (no reasoning)

Bill went up 0.7%

Moved from GPT-⁠6.1 Sol to GPT-⁠6 Luna: 0 of the 728 requests it rated

Claude Haiku 5.5 as the router (thinking off)

Cut the bill by only 1.4%

Moved from Claude Sonnet 5.5 to Claude Haiku 5.5: 116 of the 728 requests it rated

TechCrunch Uber caps employee AI spending after blowing through budget in 4 months
AI is getting expensive, and some companies are cutting back on usage in an attempt to moderate costs.
2 June 2026
ICONIQ Survey: 36% of large companies spend $1 million or more a month on internal AI tokens
AI Economics is a Design Problem

Among companies with 1,000 or more employees, 36% spend $1 million or more, and 14% that spend more than $5 million.
13 August 2026

95% of GPT-⁠6.1 Sol's quality, 36% cheaper; routing 106x faster than GPT-⁠6 Luna

Run this eval

Quality kept

share of GPT-6.1 Sol's ↑

Always GPT-6.1 Sol

Always GPT-6.1 Sol 100%

GPT-6 Luna as the router

GPT-6 Luna as the router 100%

SIE Qwen3 Reranker 4B

SIE Qwen3 Reranker 4B 95.9%

Always GPT-6 Luna

Always GPT-6 Luna 80.9%

Time per routing decision

median ↓

SIE Qwen3 Reranker 4B

8.5 ms SIE Qwen3 Reranker 4B

GPT-6 Luna as the router

GPT-6 Luna as the router 904 ms

Cost per 1,000 requests

USD ↓

Always GPT-6 Luna

$0.41 Always GPT-6 Luna

SIE Qwen3 Reranker 4B

SIE Qwen3 Reranker 4B $3.31

Always GPT-6.1 Sol

Always GPT-6.1 Sol $5.19

GPT-6 Luna as the router

GPT-6 Luna as the router $5.23

Tested: 728 held-out requests: hard science questions, math, Python tasks and open chat
Scored: Graded answers; open chat judged by the other provider's strong model
Prices: List prices, routing call included
Timing: SIE timed on a GPU server with the client alongside; providers over their public APIs

One score call decides the route

from sie_sdk import SIEClient
client = SIEClient(api_key="API keysk-sie-…")
query = "queryA request that a small, cheap AI model can answer correctly."
document = "document 1Solve the following math problem. Put your final answer within \boxed{}.

Solve for $x$: $\frac{x}2 + \frac{x}3 = 5$"
score = client.score(
    "modelQwen/Qwen3-Reranker-4B",
    {"text": query},
    [{"id": "0", "text": document}],
    instruction="instructionJudge whether the Document, a request sent to an AI assistant, fits the description in the Query.",
)["scores"][0]["score"]
print(score)

import { SIEClient } from '@superlinked/sie-sdk';

const client = new SIEClient(undefined, { apiKey: 'sk-sie-…' });

const query = "A request that a small, cheap AI model can answer correctly.";
const document = "Solve the following math problem. Put your final answer within \\boxed{}.\n\nSolve for $x$: $\\frac{x}2 + \\frac{x}3 = 5$";
const ranked = await client.score(
  'Qwen/Qwen3-Reranker-4B',
  { text: query },
  [{ id: '0', text: document }],
  { instruction: "Judge whether the Document, a request sent to an AI assistant, fits the description in the Query." },
);
console.log(ranked.scores[0].score);

curl https://api.superlinked.com/v1/score/Qwen%2FQwen3-Reranker-4B \
  -H "Authorization: Bearer sk-sie-…" \
  -H "Content-Type: application/json" \
  -d "{\"query\":{\"text\":\"A request that a small, cheap AI model can answer correctly.\"},\"items\":[{\"id\":\"0\",\"text\":\"Solve the following math problem. Put your final answer within \\\\boxed{}.\\n\\nSolve for \$x\$: \$\\\\frac{x}2 + \\\\frac{x}3 = 5\$\"}],\"instruction\":\"Judge whether the Document, a request sent to an AI assistant, fits the description in the Query.\"}"

Build the "Score a request against a description" capability into my app using the Superlinked Inference Engine (SIE).

Context
- SIE is an OpenAI-style inference API. Python SDK: `from sie_sdk import SIEClient`; TypeScript: `@superlinked/sie-sdk`.
- Base URL: https://api.superlinked.com (or my regional endpoint). Auth: Bearer key from env `SIE_API_KEY` (never hard-code it).
- Model: Qwen/Qwen3-Reranker-4B (SIE primitive: /score). Keep the model id configurable.

Task
- Input: a one-line description (the query) plus one request to score against it.
- Behaviour: score the one request against the description and return that one score.
- Send one POST https://api.superlinked.com/v1/score/Qwen%2FQwen3-Reranker-4B request (SDK: score) with the description as the query, the request as the only item and the instruction below. Read the one score from scores[0].score. Do not batch requests or rank anything. Instruction, sent as given: "Judge whether the Document, a request sent to an AI assistant, fits the description in the Query."

Deliverables
- A typed client wrapper, an application-level function for this task, error handling for timeouts/empty input, and unit tests with a stubbed client.
- Wire it into my existing stack (ask me which framework if unclear) and add a short usage example.

Run

Output

scores[0].score

0.114

Pay 36% less than always using GPT-6.1 Sol

Serverless API

We host the models; you call one API.

No idle GPUs, pay for what you use
Fits your stack: SDK, API, CLI, MCP
Self-host the same open-source engine
SOC 2 Type 2, US or EU data residency
Create an account to try Serverless

Get started

Self-hosted (Kubernetes)

Your data stays in your cloud, backed by our engineers.

Terraform modules for your cloud
Apache-2.0, same engine as Serverless
GPU workers scale to zero between jobs
Separate worker pools for each tenant

Agent prompt Deploy SIE to our AWS account with the superlinked/sie/aws Terraform module. Docs: superlinked.com/docs/deployment.

Deploy SIE to our Google Cloud project with the superlinked/sie/google Terraform module. Docs: superlinked.com/docs/deployment.

Deploy SIE to our Azure subscription with the superlinked/sie/azure Terraform module. Docs: superlinked.com/docs/deployment.

Deploy SIE to our Alibaba Cloud account with the superlinked/sie/alicloud Terraform module. Docs: superlinked.com/docs/deployment.

Talk to us Deploy guide

Run locally

The open-source server on your own machine.

Runs on your NVIDIA GPU
One container, no cluster
Process your data fully offline
Same SDK and model IDs as Serverless

docker run --gpus all -p 8080:8080 ghcr.io/superlinked/sie-server:latest-cuda12-default

Quickstart

Switch to SIE in 5 minutes

import json
from openai import OpenAI

openai = OpenAI()

rating = openai.responses.create(
    model="gpt-6-luna",
    instructions=rating_prompt,
    input=request,
    reasoning={"effort": "none"},
    text={"format": rating_format},
)
score = json.loads(rating.output_text)["score"]

model = "gpt-6-luna" if score >= threshold else "gpt-6.1-sol"

import json
from anthropic import Anthropic

anthropic = Anthropic()

rating = anthropic.messages.create(
    model="claude-haiku-5-5",
    system=rating_prompt,
    messages=[{"role": "user", "content": request}],
    thinking={"type": "disabled"},
    output_config={"format": rating_format},
    max_tokens=64,
)
score = json.loads(rating.content[0].text)["score"]

model = "claude-haiku-5-5" if score >= threshold else "claude-sonnet-5-5"

SIE

from sie_sdk import SIEClient

client = SIEClient(api_key="sk-sie-…")

score = client.score(
    "Qwen/Qwen3-Reranker-4B",
    {"text": cheap_model_handles},
    [{"id": "request", "text": request}],
    instruction=routing_instruction,
)["scores"][0]["score"]

model = "gpt-6-luna" if score >= threshold else "gpt-6.1-sol"

from sie_sdk import SIEClient

client = SIEClient(api_key="sk-sie-…")

score = client.score(
    "Qwen/Qwen3-Reranker-4B",
    {"text": cheap_model_handles},
    [{"id": "request", "text": request}],
    instruction=routing_instruction,
)["scores"][0]["score"]

model = "claude-haiku-5-5" if score >= threshold else "claude-sonnet-5-5"