naver/splade-cocondenser-selfdistil
SPLADE model for passage retrieval. For additional details, please visit: * paper: https://arxiv.org/abs/2205.04733 * code: https://github.com/naver/splade
Benchmarks
CQADupstackPhysicsRetrieval
Duplicate question retrieval from StackExchange Physics
Corpus: 38,314 Queries: 1,039
Performance L4 b1 c16
Corpus 32.8K tok/s
Corpus p50 60.2ms
Query 3.0K tok/s
Query p50 56.3ms
CosQA
Code search with natural language queries
Corpus: 6,267 Queries: 500
Performance L4 b1 c16
Corpus 14.9K tok/s
Corpus p50 53.6ms
Query 1.6K tok/s
Query p50 55.0ms
FiQA2018
Financial opinion mining and question answering
Corpus: 57,599 Queries: 648
Performance L4 b1 c16
Corpus 34.6K tok/s
Corpus p50 69.2ms
Query 3.2K tok/s
Query p50 56.2ms
LegalBenchConsumerContractsQA
Question answering on consumer contracts
Corpus: 153 Queries: 396
Performance L4 b1 c16
Corpus 84.4K tok/s
Corpus p50 88.5ms
Query 5.2K tok/s
Query p50 49.9ms
NFCorpus
Biomedical literature search from NutritionFacts.org
Corpus: 3,593 Queries: 323
Quality
ndcg at 10 0.3403
map at 10 0.1282
mrr at 10 0.5447
Performance L4 b1 c16
Corpus 55.9K tok/s
Corpus p50 79.2ms
Query 1.5K tok/s
Query p50 52.1ms
SCIDOCS
Citation prediction, document classification, and recommendation for scientific papers
Corpus: 25,656 Queries: 1,000
Performance L4 b1 c16
Corpus 42.1K tok/s
Corpus p50 68.2ms
Query 3.1K tok/s
Query p50 55.1ms
SciFact
Scientific claim verification using research literature
Corpus: 5,183 Queries: 300
Performance L4 b1 c16
Corpus 53.5K tok/s
Corpus p50 75.5ms
Query 4.8K tok/s
Query p50 54.3ms
StackOverflowQA
Programming question answering from Stack Overflow
Corpus: 19,931 Queries: 1,994
Performance L4 b1 c16
Corpus 37.9K tok/s
Corpus p50 76.9ms
Query 60.7K tok/s
Query p50 76.7ms