Skip to content

Commit

Permalink
remove leaderboard table from readme
Browse files Browse the repository at this point in the history
  • Loading branch information
nkaenzig committed Oct 18, 2024
1 parent c6af930 commit 25d8a0f
Showing 1 changed file with 1 addition and 35 deletions.
36 changes: 1 addition & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,41 +212,7 @@ and [tutorials](https://kaiko-ai.github.io/eva/dev/user-guide/advanced/replicate

## Leaderboards

In this section you will find model benchmarks which were generated with _`eva`_.

### Table I: WSI and microscopy image tasks

<br />

<div align="center">

| Model | BACH | CRC | MHIST | PCam | Camelyon16 | PANDA | CoNSeP | MoNuSAC |
|---------|-------|-------|-------|--------|------------|-------|------------|-------|
| ViT-S/16 _(random)_ <sup>[1]</sup> | 0.411|0.613|0.5|0.752|0.551|0.347|0.489|0.394|
| ViT-S/16 _(ImageNet)_ <sup>[1]</sup> | 0.675|0.936|0.827|0.861|0.751|0.676|0.54|0.512|
| DINO<sub>(p=16)</sub> <sup>[2]</sup> | 0.77|0.936|0.751|0.905|0.869|0.737|0.625|0.549|
| Phikon <sup>[3]</sup> | 0.715|0.942|0.766|0.925|0.879|0.784|0.68|0.554|
| UNI <sup>[4]</sup> | 0.797|0.95|0.835|0.939|0.933|0.774|0.67|0.575|
| ViT-S/16 _(kaiko.ai)_ <sup>[5]</sup> | 0.8|0.949|0.831|0.902|0.897|0.77|0.622|0.573|
| ViT-S/8 _(kaiko.ai)_ <sup>[5]</sup> | 0.825|0.948|0.826|0.887|0.879|0.741|0.677|0.617|
| ViT-B/16 _(kaiko.ai)_ <sup>[5]</sup> | 0.846|0.959|0.839|0.906|0.891|0.753|0.647|0.572|
| ViT-B/8 _(kaiko.ai)_ <sup>[5]</sup> | 0.867|0.952|0.814|0.921|0.939|0.761|0.706|0.661|
| ViT-L/14 _(kaiko.ai)_ <sup>[5]</sup> | 0.862|0.935|0.822|0.907|0.941|0.769|0.686|0.599|

_Table I: Linear probing evaluation of FMs on patch-level downstream datasets.<br> We report balanced accuracy
for classification tasks and generalized Dice score for semgetnation tasks, averaged over 5 runs. Results are
reported on the "test" split if available and otherwise on the "validation" split._

</div>

<br />

_References_:
1. _"Emerging properties in self-supervised vision transformers”_, [arXiv](https://arxiv.org/abs/2104.14294)
2. _"Benchmarking self-supervised learning on diverse pathology datasets”_, [arXiv](https://arxiv.org/abs/2212.04690)
3. _"Scaling self-supervised learning for histopathology with masked image modeling”_, [medRxiv](https://www.medrxiv.org/content/10.1101/2023.07.21.23292757v1)
4. _"A General-Purpose Self-Supervised Model for Computational Pathology”_, [arXiv](https://arxiv.org/abs/2308.15474)
5. _"Towards Training Large-Scale Pathology Foundation Models: from TCGA to Hospital Scale”_, [arXiv](https://arxiv.org/pdf/2404.15217)
For the latest FM leaderboards please refer to eva's [documentation](https://kaiko-ai.github.io/eva/main/leaderboards/).

## Contributing

Expand Down

0 comments on commit 25d8a0f

Please sign in to comment.