diff --git a/README.md b/README.md index c8312f13..96e04da5 100644 --- a/README.md +++ b/README.md @@ -212,41 +212,7 @@ and [tutorials](https://kaiko-ai.github.io/eva/dev/user-guide/advanced/replicate ## Leaderboards -In this section you will find model benchmarks which were generated with _`eva`_. - -### Table I: WSI and microscopy image tasks - -
- -
- -| Model | BACH | CRC | MHIST | PCam | Camelyon16 | PANDA | CoNSeP | MoNuSAC | -|---------|-------|-------|-------|--------|------------|-------|------------|-------| -| ViT-S/16 _(random)_ [1] | 0.411|0.613|0.5|0.752|0.551|0.347|0.489|0.394| -| ViT-S/16 _(ImageNet)_ [1] | 0.675|0.936|0.827|0.861|0.751|0.676|0.54|0.512| -| DINO(p=16) [2] | 0.77|0.936|0.751|0.905|0.869|0.737|0.625|0.549| -| Phikon [3] | 0.715|0.942|0.766|0.925|0.879|0.784|0.68|0.554| -| UNI [4] | 0.797|0.95|0.835|0.939|0.933|0.774|0.67|0.575| -| ViT-S/16 _(kaiko.ai)_ [5] | 0.8|0.949|0.831|0.902|0.897|0.77|0.622|0.573| -| ViT-S/8 _(kaiko.ai)_ [5] | 0.825|0.948|0.826|0.887|0.879|0.741|0.677|0.617| -| ViT-B/16 _(kaiko.ai)_ [5] | 0.846|0.959|0.839|0.906|0.891|0.753|0.647|0.572| -| ViT-B/8 _(kaiko.ai)_ [5] | 0.867|0.952|0.814|0.921|0.939|0.761|0.706|0.661| -| ViT-L/14 _(kaiko.ai)_ [5] | 0.862|0.935|0.822|0.907|0.941|0.769|0.686|0.599| - -_Table I: Linear probing evaluation of FMs on patch-level downstream datasets.
We report balanced accuracy - for classification tasks and generalized Dice score for semgetnation tasks, averaged over 5 runs. Results are - reported on the "test" split if available and otherwise on the "validation" split._ - -
- -
- -_References_: -1. _"Emerging properties in self-supervised vision transformers”_, [arXiv](https://arxiv.org/abs/2104.14294) -2. _"Benchmarking self-supervised learning on diverse pathology datasets”_, [arXiv](https://arxiv.org/abs/2212.04690) -3. _"Scaling self-supervised learning for histopathology with masked image modeling”_, [medRxiv](https://www.medrxiv.org/content/10.1101/2023.07.21.23292757v1) -4. _"A General-Purpose Self-Supervised Model for Computational Pathology”_, [arXiv](https://arxiv.org/abs/2308.15474) -5. _"Towards Training Large-Scale Pathology Foundation Models: from TCGA to Hospital Scale”_, [arXiv](https://arxiv.org/pdf/2404.15217) +For the latest FM leaderboards please refer to eva's [documentation](https://kaiko-ai.github.io/eva/main/leaderboards/). ## Contributing