sentence-transformers/all-MiniLM-L6-v2
This is a sentence-transformers model: It maps sentences & paragraphs to a 384 dimensional dense vector space and can be used for tasks like clustering or semantic search.
Overview
Benchmarks
CQADupstackPhysicsRetrieval
Duplicate question retrieval from StackExchange Physics
Corpus: 38,314 Queries: 1,039
Quality
ndcg at 10 0.4698
map at 10 0.4074
mrr at 10 0.4632
Performance A10G b1 c16
Corpus 1.7K tok/s
Corpus p50 1.1s
Query 512 tok/s
Query p50 327.0ms
Performance L4 b1 c16
Corpus 37.0K tok/s
Corpus p50 53.3ms
Query 2.8K tok/s
Query p50 49.8ms
CosQA
Code search with natural language queries
Corpus: 6,267 Queries: 500
Quality
ndcg at 10 0.3361
map at 10 0.2667
mrr at 10 0.2843
Performance A10G b1 c16
Corpus 1.1K tok/s
Corpus p50 750.0ms
Query 299 tok/s
Query p50 296.9ms
Performance L4 b1 c16
Corpus 17.1K tok/s
Corpus p50 50.1ms
Query 1.8K tok/s
Query p50 45.5ms
FiQA2018
Financial opinion mining and question answering
Corpus: 57,599 Queries: 648
Quality
ndcg at 10 0.1503
map at 10 0.1084
mrr at 10 0.1771
Performance A10G b1 c16
Corpus 1.9K tok/s
Corpus p50 1.3s
Query 587 tok/s
Query p50 345.4ms
Performance L4 b1 c16
Corpus 46.3K tok/s
Corpus p50 52.4ms
Query 3.7K tok/s
Query p50 43.8ms
LegalBenchConsumerContractsQA
Question answering on consumer contracts
Corpus: 153 Queries: 396
Quality
ndcg at 10 0.6561
map at 10 0.5883
mrr at 10 0.5883
Performance L4 b1 c16
Corpus 128.8K tok/s
Corpus p50 58.9ms
Query 4.3K tok/s
Query p50 59.1ms
NFCorpus
Biomedical literature search from NutritionFacts.org
Corpus: 3,593 Queries: 323
Quality
ndcg at 10 0.2324
map at 10 0.0703
mrr at 10 0.3935
Performance L4 b1 c16
Corpus 130.0K tok/s
Corpus p50 20.0ms
Query 2.6K tok/s
Query p50 17.1ms
NanoFiQA2018Retrieval
Smaller subset of the FiQA financial QA dataset
Quality
ndcg at 10 0.4774
map at 10 0.3931
mrr at 10 0.5476
Performance L4 b1 c16
Corpus 44.2K tok/s
Corpus p50 56.1ms
Query 2.8K tok/s
Query p50 49.9ms
SCIDOCS
Citation prediction, document classification, and recommendation for scientific papers
Corpus: 25,656 Queries: 1,000
Quality
ndcg at 10 0.0648
map at 10 0.0363
mrr at 10 0.1075
Performance L4 b1 c16
Corpus 55.3K tok/s
Corpus p50 51.8ms
Query 4.1K tok/s
Query p50 43.1ms
SciFact
Scientific claim verification using research literature
Corpus: 5,183 Queries: 300
Quality
ndcg at 10 0.6112
map at 10 0.5565
mrr at 10 0.5620
Performance L4 b1 c16
Corpus 75.7K tok/s
Corpus p50 52.8ms
Query 5.8K tok/s
Query p50 44.5ms
StackOverflowQA
Programming question answering from Stack Overflow
Corpus: 19,931 Queries: 1,994
Quality
ndcg at 10 0.8396
map at 10 0.8117
mrr at 10 0.8117
Performance L4 b1 c16
Corpus 58.9K tok/s
Corpus p50 56.0ms
Query 65.7K tok/s
Query p50 61.5ms