BGE-M3: dense, sparse and multi-vector retrieval from one model
BGE-M3 is the model to know if you are building hybrid search. One 568M-parameter encoder, one forward pass, and you get all the vector types a hybrid pipeline needs: a dense embedding, a learned sparse representation that behaves like a trainable BM25, and ColBERT-style multi-vectors for late interaction. The three M’s in the name are the pitch: multi-functionality, multi-linguality, multi-granularity. Released by BAAI in January 2024 (paper: arXiv 2402.03216), MIT licensed, and still the default sparse generator in most serious hybrid stacks we see, including the ones in our own integration docs.
Spec sheet
| Spec | Value |
|---|---|
| Parameters | 568M |
| Base | XLM-RoBERTa, position embeddings extended to 8,192 |
| Max input | 8,192 tokens |
| Dense output | 1,024 dims |
| Sparse output | vocabulary-sized (250,002) weighted terms |
| Multi-vector output | 1,024 dims per token, ColBERT-style |
| Languages | 100+ |
| License | MIT |
| Prefixes required | none |
Three of these rows explain its staying power. The 8,192-token window covers whole documents, a sixteen-fold advantage over the E5 family’s 512. It needs no query or passage prefixes, which removes the most common embedding integration bug entirely. And the sparse output is the practical star: BM25-like lexical matching that is learned rather than statistical, works across languages, and slots into any vector database with sparse vector support.
The three outputs, and when each earns its place
Dense is the general-purpose semantic vector; use it as you would any embedding. On our benchmark suite it is respectable rather than spectacular for English (SciFact 0.646 NDCG@10, StackOverflowQA 0.807, both a step below stella_en_400M_v5); monolingual English retrieval is not where you pick BGE-M3.
Sparse is why you pick it. Exact-term matching catches what dense misses: product codes, function names, rare entities, the query tokens that must appear. In a hybrid setup the two run side by side and a fusion step (typically RRF) merges rankings. The upstream FlagEmbedding project scores hybrid as a weighted sum across output types; in a vector database you usually store dense and sparse separately and fuse at query time. Our Qdrant integration documents exactly this pattern, with SIE emitting both vector types from one encode call and Qdrant’s prefetch handling RRF.
Multi-vector is the specialist. Late interaction scores query tokens against document tokens individually, which lifts precision on long documents at a real storage cost: one vector per token instead of one per document. Use it as a reranking stage over a dense-retrieved candidate set rather than as the primary index, unless your corpus is small.
BGE-M3 also works as a reranker outright; we serve it behind /score, where it posts 0.666 NDCG@10 on AskUbuntuDupQuestions at $0.076 per million tokens. Decent numbers, though purpose-built rerankers beat it; see the catalog’s score models before choosing it for that role.
Serving numbers
From the catalog, measured on an L4:
| Endpoint | Latency p50 | Throughput | Cost $/1M tok |
|---|---|---|---|
| /encode | 93 ms | 33,200 tok/s | $0.0067 |
| /score | 56 ms | 2,900 tok/s | $0.076 |
The encode cost line deserves a second look: $0.0067 per million tokens for dense plus sparse plus multi-vector in the same pass. Running a separate dense model and a separate SPLADE model to get the same coverage costs two forward passes and two model deployments. One deployment that emits everything is the operational argument for BGE-M3, and in a cluster-batched setup like SIE it compounds, since one warm model pool serves all three output types at 33K tokens per second.
Alternatives, honestly
For English-only dense retrieval, stella_en_400M_v5 is better and cheaper; BGE-M3 loses that head-to-head and it is not close on FiQA. For sparse-only English, SPLADE v3 is the purpose-built option. For long-context multilingual dense with adjustable dimensions, Qwen3-Embedding-0.6B is the modern rival, with 32k context against 8k, though it emits dense only.
Nothing else in our catalog replaces the combination. Multilingual corpus, hybrid retrieval, one model to operate: BGE-M3 keeps that seat in 2026.