[Feature request] Dimensionality of model

[sachinruk]

I am trying to figure out which model to use as an embedding model so that I can do nearest neighbour search on images. My current requirements are 1. Have >80% accuracy on imagenet 2. Have a small dimensionality when I pass a image through the base model.

For 1. I can look at this page with the results. However, for 2. I need to initialise a model (without pretrained weights of course) to figure this out. I was wondering if we can store this information in a similar csv file (or anywhere really) so that I don't have to do this exercise.

TIA 🙏 .
p.s. feel free to let me know if the embedding via this method is a bad idea.

[rwightman]

@sachinruk info is not cached anywhere though guess it could be useful to do so, however, on most machines that are fit for ML it doesn't take that long to instantiate each model (without loading pretrained weights) and do the check

for name in timm.list_pretrained():
  model = timm.create_model(name, pretrained=False)
  pre_logits_size = model.head_hidden_size  # this is the dim if you extract pooled pre_logits (ie create with num_classes=0 or call forward_head manuall)

If you do you own pooling or want unpooled features, the featre dim is model.num_features

It won't take that long to run this, should be done after a coffee... using the final pre-classifier features for this is not a bad idea at all, it's a very common use for these models. Some of the large image-text pretrained models (LAION, DFN, DataComp, OpenAI CLIP models) are particularly good.