rasyosef/splade-mini
This is a SPLADE sparse retrieval model based on BERT-Mini (11M) that was trained by distilling a Cross-Encoder on the MSMARCO dataset. The cross-encoder used was ms-marco-MiniLM-L6-v2.
Benchmarks
CQADupstackPhysicsRetrieval
Duplicate question retrieval from StackExchange Physics
Corpus: 38,314 Queries: 1,039
Performance L4 b1 c16
Corpus 39.6K tok/s
Corpus p50 49.1ms
Query 3.8K tok/s
Query p50 44.3ms
CosQA
Code search with natural language queries
Corpus: 6,267 Queries: 500
Performance L4 b1 c16
Corpus 17.3K tok/s
Corpus p50 45.7ms
Query 2.1K tok/s
Query p50 46.1ms
FiQA2018
Financial opinion mining and question answering
Corpus: 57,599 Queries: 648
Performance L4 b1 c16
Corpus 40.1K tok/s
Corpus p50 58.1ms
Query 3.9K tok/s
Query p50 47.0ms
LegalBenchConsumerContractsQA
Question answering on consumer contracts
Corpus: 153 Queries: 396
Performance L4 b1 c16
Corpus 116.3K tok/s
Corpus p50 63.3ms
Query 5.1K tok/s
Query p50 50.5ms
NFCorpus
Biomedical literature search from NutritionFacts.org
Corpus: 3,593 Queries: 323
Quality
ndcg at 10 0.3090
map at 10 0.1164
mrr at 10 0.5183
Performance L4 b1 c16
Corpus 69.4K tok/s
Corpus p50 59.1ms
Query 1.6K tok/s
Query p50 49.6ms
SCIDOCS
Citation prediction, document classification, and recommendation for scientific papers
Corpus: 25,656 Queries: 1,000
Performance L4 b1 c16
Corpus 53.3K tok/s
Corpus p50 51.6ms
Query 3.8K tok/s
Query p50 45.5ms
SciFact
Scientific claim verification using research literature
Corpus: 5,183 Queries: 300
Performance L4 b1 c16
Corpus 70.9K tok/s
Corpus p50 55.0ms
Query 6.2K tok/s
Query p50 42.5ms
StackOverflowQA
Programming question answering from Stack Overflow
Corpus: 19,931 Queries: 1,994
Performance L4 b1 c16
Corpus 59.4K tok/s
Corpus p50 57.0ms
Query 80.8K tok/s
Query p50 56.4ms