BAAI/bge-m3 (Encode)

For more details please refer to our github repo: https://github.com/FlagOpen/FlagEmbedding

Architecture

XLM-RoBERTa

Parameters

568M

Tasks

Encode

Outputs

Dense, Sparse, Multi-Vec

Dimensions

Dense: 1,024, Sparse: 250,002, Multi-Vec: 1,024

Max Sequence Length

8,192 tokens

License

mit

View on HuggingFace →

Benchmarks

CQADupstackPhysicsRetrieval

scientific retrieval en

Duplicate question retrieval from StackExchange Physics

Corpus: 38,314 Queries: 1,039

multivector

Performance L4 b1 c16

Corpus 28.0K tok/s

Corpus p50 70.7ms

Query 3.2K tok/s

Query p50 44.7ms

dense

Performance L4 b1 c16

Corpus 27.3K tok/s

Corpus p50 74.8ms

Query 3.1K tok/s

Query p50 45.2ms

sparse

Performance L4 b1 c16

Corpus 27.8K tok/s

Corpus p50 70.8ms

Query 3.3K tok/s

Query p50 44.9ms

Reference →

CosQA

technology retrieval en

Code search with natural language queries

Corpus: 6,267 Queries: 500

multivector

Performance L4 b1 c16

Corpus 15.3K tok/s

Corpus p50 51.4ms

Query 1.8K tok/s

Query p50 45.8ms

dense

Performance L4 b1 c16

Corpus 15.8K tok/s

Corpus p50 51.3ms

Query 1.7K tok/s

Query p50 47.9ms

sparse

Performance L4 b1 c16

Corpus 16.9K tok/s

Corpus p50 51.0ms

Query 1.9K tok/s

Query p50 43.7ms

Reference →

FiQA2018

finance retrieval en

Financial opinion mining and question answering

Corpus: 57,599 Queries: 648

multivector

Performance L4 b1 c16

Corpus 33.8K tok/s

Corpus p50 77.8ms

Query 3.6K tok/s

Query p50 44.8ms

dense

Performance L4 b1 c16

Corpus 28.6K tok/s

Corpus p50 87.3ms

Query 3.1K tok/s

Query p50 48.4ms

sparse

Performance L4 b1 c16

Corpus 30.2K tok/s

Corpus p50 81.3ms

Query 3.3K tok/s

Query p50 46.2ms

Reference →

LegalBenchConsumerContractsQA

legal retrieval en

Question answering on consumer contracts

Corpus: 153 Queries: 396

multivector

Performance L4 b1 c16

Corpus 39.1K tok/s

Corpus p50 216.2ms

Query 5.1K tok/s

Query p50 44.0ms

dense

Performance L4 b1 c16

Corpus 39.5K tok/s

Corpus p50 210.7ms

Query 4.2K tok/s

Query p50 51.7ms

sparse

Performance L4 b1 c16

Corpus 40.6K tok/s

Corpus p50 208.1ms

Query 5.1K tok/s

Query p50 45.0ms

Reference →

NFCorpus

medical retrieval en

Biomedical literature search from NutritionFacts.org

Corpus: 3,593 Queries: 323

default

Quality

ndcg at 10 0.3144

map at 10 0.1174

mrr at 10 0.5243

Performance A10G b1 c4

Corpus 127 tok/s

Corpus p50 9.4s

Query 39 tok/s

Query p50 537.6ms

Performance L4 b1 c16

Corpus 38.1K tok/s

Corpus p50 100.0ms

Query 1.5K tok/s

Query p50 41.0ms

multivector

Performance L4 b1 c16

Corpus 37.9K tok/s

Corpus p50 126.6ms

Query 1.5K tok/s

Query p50 46.1ms

dense

Performance L4 b1 c16

Corpus 34.2K tok/s

Corpus p50 134.0ms

Query 937 tok/s

Query p50 64.2ms

sparse

Performance L4 b1 c16

Corpus 40.0K tok/s

Corpus p50 124.4ms

Query 1.4K tok/s

Query p50 45.0ms

Reference →

NanoFiQA2018Retrieval

finance retrieval en

Smaller subset of the FiQA financial QA dataset

multivector

Performance L4 b1 c16

Corpus 30.6K tok/s

Corpus p50 86.6ms

Query 2.7K tok/s

Query p50 54.1ms

dense

Performance L4 b1 c16

Corpus 28.1K tok/s

Corpus p50 89.9ms

Query 2.5K tok/s

Query p50 59.2ms

sparse

Performance L4 b1 c16

Corpus 30.3K tok/s

Corpus p50 87.5ms

Query 2.7K tok/s

Query p50 53.3ms

default

Quality

ndcg at 10 0.5726

map at 10 0.4957

mrr at 10 0.6467

Performance L4 b1 c16

Corpus 31.2K tok/s

Corpus p50 68.9ms

Query 2.9K tok/s

Query p50 43.3ms

Reference →

SCIDOCS

scientific retrieval en

Citation prediction, document classification, and recommendation for scientific papers

Corpus: 25,656 Queries: 1,000

multivector

Performance L4 b1 c16

Corpus 34.4K tok/s

Corpus p50 90.7ms

Query 3.4K tok/s

Query p50 43.0ms

dense

Performance L4 b1 c16

Corpus 33.2K tok/s

Corpus p50 94.8ms

Query 2.5K tok/s

Query p50 54.2ms

sparse

Performance L4 b1 c16

Corpus 32.5K tok/s

Corpus p50 93.4ms

Query 3.4K tok/s

Query p50 44.7ms

Reference →

SciFact

scientific retrieval en

Scientific claim verification using research literature

Corpus: 5,183 Queries: 300

multivector

Performance L4 b1 c16

Corpus 38.2K tok/s

Corpus p50 116.5ms

Query 4.9K tok/s

Query p50 44.4ms

dense

Performance L4 b1 c16

Corpus 37.9K tok/s

Corpus p50 118.2ms

Query 4.2K tok/s

Query p50 48.5ms

sparse

Performance L4 b1 c16

Corpus 37.8K tok/s

Corpus p50 117.6ms

Query 4.2K tok/s

Query p50 47.7ms

Reference →

StackOverflowQA

technology retrieval en

Programming question answering from Stack Overflow

Corpus: 19,931 Queries: 1,994

multivector

Performance L4 b1 c16

Corpus 33.2K tok/s

Corpus p50 107.6ms

Query 33.8K tok/s

Query p50 137.6ms

dense

Performance L4 b1 c16

Corpus 33.4K tok/s

Corpus p50 104.7ms

Query 33.8K tok/s

Query p50 138.5ms

sparse

Performance L4 b1 c16

Corpus 32.7K tok/s

Corpus p50 108.4ms

Query 34.8K tok/s

Query p50 137.6ms

Reference →

Benchmarks

CQADupstackPhysicsRetrieval

CosQA

FiQA2018

LegalBenchConsumerContractsQA

NFCorpus

NanoFiQA2018Retrieval

SCIDOCS

SciFact

StackOverflowQA

Self-hosted inference for search & document processing