Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Target layers of ViT models #513

Open
WangYZ1608 opened this issue Jul 5, 2024 · 2 comments
Open

Target layers of ViT models #513

WangYZ1608 opened this issue Jul 5, 2024 · 2 comments

Comments

@WangYZ1608
Copy link

Hi @jacobgil. It's a nice work! Thanks to you and all the contributors behind it.

I tried replacing norm1 with block[-1].attn or norm2 without any other changes, but the gradient seems to be 0, as shown below (I test with GradCAM and LayerCAM. And I could get satisfactory result when using norm1). What is the reason for this?

Also, I would like to ask, how to visualize the class token? And how to use CAM in an image containing multiple instances?

original image
Boat_Tailed_Grackle_0082_33488

result
layercam_cam

@zermatt-luo
Copy link

Hello, did you resolve this issue? I have a similar problem!

@sophmrtn
Copy link

I also have the same issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants