Skip to content

Issues: embeddings-benchmark/mteb

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

104 Open 392 Closed
104 Open 392 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Add descriptive_stats to all tasks help wanted Extra attention is needed
#1475 opened Nov 19, 2024 by Samoed
@imenelydiaker
8
add google/siglip-so400m-patch16-256-i18n
#1474 opened Nov 19, 2024 by SaitejaUtpala
1
Can't load some datasets because of datasets 3.0 bug Something isn't working
#1465 opened Nov 16, 2024 by Samoed
2
Add NanoBEIR new-dataset
#1464 opened Nov 16, 2024 by Muennighoff
1
Importing mteb is slow good first issue Good for newcomers
#1463 opened Nov 16, 2024 by Muennighoff
@Napuh
3
[mieb] add support for LLM2CLIP mieb The image extension of MTEB
#1458 opened Nov 14, 2024 by SaitejaUtpala
[mieb] royokong/e5-v on EDIST2ITRetrieval 'NoneType' object has no attribute 'dtype' mieb The image extension of MTEB
#1455 opened Nov 14, 2024 by Muennighoff
[mieb] add support for voyage-multimodal-3: mieb The image extension of MTEB
#1446 opened Nov 14, 2024 by Muennighoff
"KeyError: 'document' not found and no similar keys were found.
#1445 opened Nov 13, 2024 by LeMoussel
5
Leaderboard: Add radar chart leaderboard issues related to the leaderboard
#1443 opened Nov 13, 2024 by KennethEnevoldsen
Model metas: E5 instruct not listed as sbert compatible
#1442 opened Nov 12, 2024 by x-tabdeveloping
Leaderboard: Add query parameters to link leaderboard issues related to the leaderboard
#1440 opened Nov 12, 2024 by x-tabdeveloping
Leaderboard: Add an Overview leaderboard leaderboard issues related to the leaderboard
#1432 opened Nov 11, 2024 by KennethEnevoldsen
2
Add new benchmark MAIR
#1426 opened Nov 10, 2024 by sunnweiwei
1
Evaluating LLM2Vec fails
#1422 opened Nov 9, 2024 by Muennighoff
@sivareddyg @vaibhavad
1
Evaluating nomic on BornholmBitextMining fails
#1421 opened Nov 9, 2024 by Muennighoff
[mieb] Salesforce/blip-image-captioning-large buggy mieb The image extension of MTEB
#1415 opened Nov 8, 2024 by Muennighoff
1
Running models before leaderboard release leaderboard issues related to the leaderboard
#1411 opened Nov 8, 2024 by x-tabdeveloping
12 tasks
6
Flexible post-evaluation filtering
#1410 opened Nov 8, 2024 by gnatesan
2
Integrate ArabicMTEB new-dataset
#1408 opened Nov 7, 2024 by KennethEnevoldsen
Error when creating task (SICK-BR-STS, XStance)
#1407 opened Nov 7, 2024 by AlexeyVatolin
Overview issue: Leaderboard 2.0 release leaderboard issues related to the leaderboard
#1405 opened Nov 7, 2024 by KennethEnevoldsen
3 of 8 tasks
leaderboard2.0
1
Leaderboard: Add runtime x performance plot leaderboard issues related to the leaderboard
#1396 opened Nov 6, 2024 by KennethEnevoldsen
@x-tabdeveloping
5
[mieb] kakaobrain/align-base fails on ImageNet10Clustering mieb The image extension of MTEB
#1387 opened Nov 4, 2024 by Muennighoff
[mieb] InfoSeekIT2ITRetrieval & InfoSeekIT2TRetrieval fail with BAAI/bge-visualized mieb The image extension of MTEB
#1386 opened Nov 4, 2024 by Muennighoff
2
ProTip! What’s not been updated in a month: updated:<2024-10-20.