---
title: How Does Haystack Work with Embedding Models?
description: Haystack is an open-source framework for building NLP and RAG pipelines by connecting modular components (document stores, retrievers, readers, and generators) into composable directed acyclic graphs. It works with embedding models by providing retriever components that call the embedding model at query and index ...
canonical_url: https://superlinked.com/glossary/how-does-haystack-work-with-embedding-models
last_updated: 2026-06-11
---

# How Does Haystack Work with Embedding Models?

Haystack is an open-source framework for building NLP and RAG pipelines by connecting modular components (document stores, retrievers, readers, and generators) into composable directed acyclic graphs. It works with embedding models by providing retriever components that call the embedding model at query and index time. SIE integrates with Haystack as a self-hosted embedding backend, replacing managed API calls with GPU inference in your own cloud.

---

## Why Haystack?

Haystack is the right choice when:

- You want a **structured pipeline framework** rather than writing retrieval logic from scratch
- You need to **compose complex multi-hop or multi-stage pipelines** (retrieve → rerank → generate → verify)
- You're building **production RAG systems** and want pre-built components for evaluation, caching, and monitoring
- You want to **swap components** (different retrievers, LLMs, document stores) without rewriting pipeline logic
- You need **multi-modal pipelines** handling both text and images

Haystack's component abstraction means you can prototype with OpenAI embeddings and switch to SIE self-hosted inference for production without changing pipeline logic.

---

## Core Haystack concepts

**Components**: individual pipeline steps with typed inputs and outputs (EmbeddingRetriever, SentenceTransformersTextEmbedder, OpenAIGenerator, etc.)

**Pipeline**: a directed acyclic graph of components connected by their input/output types

**Document Store**: the storage layer (InMemoryDocumentStore, QdrantDocumentStore, WeaviateDocumentStore, etc.)

**Documents**: Haystack's data type representing a piece of text with metadata and an optional embedding vector

---

## How SIE integrates with Haystack

SIE acts as the embedding backend. Use a custom `TextEmbedder` component that calls SIE:

```python
from haystack import component, Document, Pipeline
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from sie_sdk import SIEClient
from sie_sdk.types import Item

# Custom SIE embedder component
@component
class SIETextEmbedder:
    def __init__(self, model: str = "BAAI/bge-m3"):
        self.client = SIEClient("http://localhost:8080")
        self.model = model

    @component.output_types(embedding=list[float])
    def run(self, text: str):
        result = self.client.encode(self.model, Item(text=text), is_query=True)
        return {"embedding": result["dense"].tolist()}

@component
class SIEDocumentEmbedder:
    def __init__(self, model: str = "BAAI/bge-m3"):
        self.client = SIEClient("http://localhost:8080")
        self.model = model

    @component.output_types(documents=list[Document])
    def run(self, documents: list[Document]):
        texts = [doc.content for doc in documents]
        encode_results = self.client.encode(
            self.model,
            [Item(text=t) for t in texts],
        )
        for doc, res in zip(documents, encode_results):
            doc.embedding = res["dense"].tolist()
        return {"documents": documents}
```

---

## Building an indexing pipeline with SIE + Haystack

```python
from haystack import Pipeline
from haystack.components.writers import DocumentWriter

document_store = InMemoryDocumentStore()

# Indexing pipeline
indexing_pipeline = Pipeline()
indexing_pipeline.add_component("embedder", SIEDocumentEmbedder())
indexing_pipeline.add_component("writer", DocumentWriter(document_store=document_store))
indexing_pipeline.connect("embedder.documents", "writer.documents")

# Index documents
raw_docs = [Document(content=chunk, meta={"source": src}) for chunk, src in zip(chunks, sources)]
indexing_pipeline.run({"embedder": {"documents": raw_docs}})
```

---

## Building a RAG query pipeline with SIE + Haystack

```python
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders import PromptBuilder

PROMPT = """
Answer the question using the provided context.
Context: {% for doc in documents %}{{ doc.content }}{% endfor %}
Question: {{ query }}
"""

rag_pipeline = Pipeline()
rag_pipeline.add_component("query_embedder", SIETextEmbedder())
rag_pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store, top_k=5))
rag_pipeline.add_component("prompt_builder", PromptBuilder(template=PROMPT))
rag_pipeline.add_component("llm", OpenAIGenerator(model="gpt-4o"))

rag_pipeline.connect("query_embedder.embedding", "retriever.query_embedding")
rag_pipeline.connect("retriever.documents", "prompt_builder.documents")
rag_pipeline.connect("prompt_builder.prompt", "llm.prompt")

result = rag_pipeline.run({
    "query_embedder": {"text": "What are the termination conditions?"},
    "prompt_builder": {"query": "What are the termination conditions?"}
})

print(result["llm"]["replies"][0])
```

---

## Using Haystack with Qdrant and SIE

For production, swap `InMemoryDocumentStore` for `QdrantDocumentStore`:

```python
from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
from haystack_integrations.components.retrievers.qdrant import QdrantEmbeddingRetriever

document_store = QdrantDocumentStore(
    url="http://localhost:6333",
    index="documents",
    embedding_dim=1024,  # BGE-M3 output dim
)

retriever = QdrantEmbeddingRetriever(document_store=document_store, top_k=20)
```

---

## Haystack vs LangChain vs LlamaIndex

| | Haystack | LangChain | LlamaIndex |
|---|---|---|---|
| Pipeline model | Typed DAG | Chain / Agent | Query engine |
| Component typing | Strict | Loose | Medium |
| RAG focus | ✓ | General | ✓ |
| Evaluation tooling | Strong | Growing | Good |
| Production maturity | High | High | High |
| Best for | Production RAG, evaluation | Agents, diverse tasks | Document QA |

All three integrate well with SIE; the choice comes down to team familiarity and pipeline complexity.

---

## Frequently asked questions

**Does Haystack have a built-in SIE integration?**
The SIE SDK is used via a custom component as shown above. A native Haystack SIE integration is available. See the [SIE + Haystack integration guide](/docs/integrations/haystack) for the current implementation.

**Can Haystack pipelines be serialised and deployed?**
Yes. Haystack pipelines serialise to YAML, enabling reproducible deployments and version-controlled pipeline definitions.

**Does Haystack support reranking with SIE?**
Yes. Add a custom reranker component that calls `client.score()` between the retriever and prompt builder steps.

---

## Related resources

- [SIE + Haystack full integration guide](/docs/integrations/haystack)
- [What is RAG?](/glossary/what-is-rag)
- [What is a reranking pipeline?](/glossary/what-is-a-reranking-pipeline)
- [How does Qdrant work with embedding models?](/glossary/how-does-qdrant-work-with-embedding-models)
- [Browse embedding models on SIE](/models)