Adding Models

Add any HuggingFace model by creating a config file. No code changes required.

---

Directory Layout

Model configs are flat YAML files in the models directory, named {Org}__{name}.yaml: the org and model name from the model’s sie_id joined by a double underscore, with original casing preserved. An org-less sie_id (e.g. docling) yields a bare {name}.yaml.

models/
  BAAI__bge-m3.yaml
  my-org__my-custom-model.yaml

For Docker deployments, mount your custom models directory:

docker run --gpus all -p 8080:8080 \
  -v /path/to/custom-models:/app/models:ro \
  ghcr.io/superlinked/sie-server:latest-cuda12-default

---

Config File Structure

Each model needs a config YAML file. Here is a minimal example:

sie_id: my-org/my-model
hf_id: my-org/my-model
inputs:
  text: true
tasks:
  encode:
    dense:
      dim: 768
max_sequence_length: 512
profiles:
  default:
    max_batch_tokens: 16384
    adapter_path: sie_server.adapters.pytorch_embedding:PyTorchEmbeddingAdapter

---

Required Fields

Field	Type	Description
sie_id	string	Model ID used in API requests
tasks	object	Tasks the model serves (encode, score, extract, generate) with output dims, e.g. tasks.encode.dense.dim
profiles	object	Named profiles; at least one. The profile named default is the default

Each profile must set adapter_path and max_batch_tokens, either directly or by inheriting them with extends.

Weight Source

At least one of hf_id, weights_path, or package_backed: true is required:

Field	Description
hf_id	HuggingFace model ID (e.g., BAAI/bge-m3)
weights_path	Local path to weights (takes precedence over hf_id)
package_backed	Set true for models whose weights ship with the installed package; must not be combined with hf_id, weights_path, or hf_revision

---

Optional Fields

Top-level:

Field	Type	Default	Description
inputs	object	text: true	Input modality flags: text, image, audio, video, document
hf_revision	string	null	Pin weights to an immutable 40-char commit SHA
max_sequence_length	int	null	Maximum input tokens

Per profile:

Field	Type	Default	Description
adapter_path	string	-	Adapter path module:Class (required unless extends)
max_batch_tokens	int	-	Maximum tokens per batch (required unless extends)
extends	string	null	Inherit settings from another profile
compute_precision	string	null	Override precision: float16, bfloat16, float32
adapter_options	object	{}	Nested loadtime and runtime option maps

Adapter behavior such as pooling (cls, mean, last_token, splade, none) and normalize is configured under adapter_options.runtime.

---

Profiles

Profiles define named combinations of adapter and runtime options. The profile named default is the default; other profiles typically use extends: default and override adapter_options.

profiles:
  default:
    max_batch_tokens: 16384
    compute_precision: bfloat16
    adapter_path: sie_server.adapters.bge_m3_flash:BGEM3FlashAdapter
    adapter_options:
      runtime:
        pooling: cls
        normalize: true
  sparse:
    extends: default
    adapter_options:
      runtime:
        pooling: cls
        normalize: true
        output_types:
          - sparse
  banking:
    extends: default
    adapter_options:
      runtime:
        pooling: cls
        normalize: true
        lora_id: saivamshiatukuri/bge-m3-banking77-lora
        instruction: Classify banking intent

A child profile’s non-empty adapter_options.runtime block fully replaces the parent’s, so repeat inherited keys like pooling and normalize.

Adapter Options

Options split into loadtime (require reload) and runtime (per-request override), nested under adapter_options in each profile:

profiles:
  default:
    max_batch_tokens: 16384
    compute_precision: bfloat16
    adapter_path: sie_server.adapters.sglang.embedding:SGLangEmbeddingAdapter
    adapter_options:
      loadtime:
        mem_fraction_static: 0.85
      runtime:
        pooling: last_token
        normalize: true
        query_template: |-
          Instruct: {instruction}
          Query: {text}
        default_instruction: Given a query, retrieve relevant passages that answer the query

---

Available Adapters

Each adapter lives in its own module under packages/sie_server/src/sie_server/adapters/. The adapter: path is module:Class, for example sie_server.adapters.pytorch_embedding:PyTorchEmbeddingAdapter. Browse the directory for the adapter that matches your model’s architecture, then copy its module:Class into the config.

---

Complete Example

A full config with inputs, tasks, profiles, and runtime options:

sie_id: sentence-transformers/all-MiniLM-L6-v2
hf_id: sentence-transformers/all-MiniLM-L6-v2
inputs:
  text: true
  image: false
  audio: false
  video: false
tasks:
  encode:
    dense:
      dim: 384
    sparse: null
    multivector: null
  score: null
  extract: null
max_sequence_length: 256
profiles:
  default:
    max_batch_tokens: 16384
    compute_precision: null
    adapter_path: sie_server.adapters.sentence_transformer:SentenceTransformerDenseAdapter
    adapter_options:
      loadtime:
        trust_remote_code: false
      runtime:
        pooling: mean
        normalize: true

---

Testing Your Model

After creating the config, verify the model loads and produces correct outputs.

1. Start the server

docker run --gpus all -p 8080:8080 \
  -v /path/to/custom-models:/app/models:ro \
  ghcr.io/superlinked/sie-server:latest-cuda12-default

2. Check model is listed

curl http://localhost:8080/v1/models | jq '.models[].name'

3. Generate embeddings

Python

from sie_sdk import SIEClient
from sie_sdk.types import Item

client = SIEClient("http://localhost:8080")
result = client.encode("my-org/my-model", Item(text="test input"))
print(result["dense"].shape)  # Should match tasks.encode.dense.dim

TypeScript

import { SIEClient } from "@superlinked/sie-sdk";

const client = new SIEClient("http://localhost:8080");
const result = await client.encode("my-org/my-model", { text: "test input" });
console.log(result.dense?.length); // Should match tasks.encode.dense.dim

4. Run quality eval

Evaluate retrieval quality against MTEB tasks to confirm the config produces embeddings that match the reference implementation. See Benchmarking.

---

Hot Reload

The server monitors the models directory for changes. Add new configs without restarting:

1. Create a new models/{Org}__{name}.yaml file
2. The server detects the new config automatically
3. Model weights load on first request

For Docker, the mounted volume updates are detected. Changes to existing configs are hot reloaded too: the watcher drains in-flight requests to the affected model, unloads it, and reloads it with the new config.

For adding models to a running cluster without filesystem changes, use the Config API.

---

What’s Next

• Config API - add models at runtime via REST
• Model Catalog - browse 100+ supported models
• Benchmarking - evaluate model quality and performance