lightonai/GTE-ModernColBERT-v1 (Score)
This is a PyLate model trained on the ms-marco-en-bge-gemma dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.
Benchmarks
CQADupstackPhysicsRetrieval
Duplicate question retrieval from StackExchange Physics
Corpus: 38,314 Queries: 1,039
Performance L4-SPOT b1 c16
Corpus 1.9K tok/s
Corpus p50 509.4ms
Query 131 tok/s
Query p50 573.4ms
Performance L4 b1 c16
Corpus 1.9K tok/s
Corpus p50 509.4ms
Query 131 tok/s
Query p50 573.4ms
CosQA
Code search with natural language queries
Corpus: 6,267 Queries: 500
Performance L4-SPOT b1 c16
Corpus 890 tok/s
Corpus p50 454.2ms
Query 75 tok/s
Query p50 566.6ms
Performance L4 b1 c16
Corpus 890 tok/s
Corpus p50 454.2ms
Query 75 tok/s
Query p50 566.6ms
FiQA2018
Financial opinion mining and question answering
Corpus: 57,599 Queries: 648
Performance L4-SPOT b1 c16
Corpus 2.6K tok/s
Corpus p50 469.6ms
Query 303 tok/s
Query p50 278.2ms
Performance L4 b1 c16
Corpus 2.6K tok/s
Corpus p50 469.6ms
Query 303 tok/s
Query p50 278.2ms
LegalBenchConsumerContractsQA
Question answering on consumer contracts
Corpus: 153 Queries: 396
Performance L4-SPOT b1 c16
Corpus 6.2K tok/s
Corpus p50 532.8ms
Query 278 tok/s
Query p50 327.3ms
Performance L4 b1 c16
Corpus 6.2K tok/s
Corpus p50 532.8ms
Query 278 tok/s
Query p50 327.3ms
NFCorpus
Biomedical literature search from NutritionFacts.org
Corpus: 3,593 Queries: 323
Performance L4-SPOT b1 c16
Corpus 4.4K tok/s
Corpus p50 463.3ms
Query 111 tok/s
Query p50 299.7ms
Performance L4 b1 c16
Corpus 4.4K tok/s
Corpus p50 463.3ms
Query 111 tok/s
Query p50 299.7ms
SCIDOCS
Citation prediction, document classification, and recommendation for scientific papers
Corpus: 25,656 Queries: 1,000
Performance L4-SPOT b1 c16
Corpus 4.4K tok/s
Corpus p50 257.6ms
Query 184 tok/s
Query p50 327.2ms
Performance L4 b1 c16
Corpus 4.4K tok/s
Corpus p50 257.6ms
Query 184 tok/s
Query p50 327.2ms
SciFact
Scientific claim verification using research literature
Corpus: 5,183 Queries: 300
Performance L4-SPOT b1 c16
Corpus 9.2K tok/s
Corpus p50 241.6ms
Query 396 tok/s
Query p50 265.9ms
Performance L4 b1 c16
Corpus 9.2K tok/s
Corpus p50 241.6ms
Query 396 tok/s
Query p50 265.9ms
StackOverflowQA
Programming question answering from Stack Overflow
Corpus: 19,931 Queries: 1,994
Performance L4-SPOT b1 c16
Corpus 3.8K tok/s
Corpus p50 458.1ms
Query 9.2K tok/s
Query p50 222.9ms
Performance L4 b1 c16
Corpus 3.8K tok/s
Corpus p50 458.1ms
Query 9.2K tok/s
Query p50 222.9ms