Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attempt to change all Numpy calls to Torch calls #357

Draft
wants to merge 72 commits into
base: master
Choose a base branch
from

Conversation

TRex22
Copy link

@TRex22 TRex22 commented Nov 5, 2022

What does this PR do?

  • Attempts to convert all Numpy calls to torch calls so that less device memcpy commands need to be made
  • Hope is that this will have a positive performance impact
  • And allow an end-user to use Grad-CAM heatmaps on device over on CPU

Extremely experimental and proof of concept. Many areas may need improvement like resizing images

@@ -36,7 +36,7 @@ def get_cam_weights(self,
target_layers: List[torch.nn.Module],
targets: List[torch.nn.Module],
activations: torch.Tensor,
grads: torch.Tensor) -> np.ndarray:
grads: torch.Tensor) -> torch.Tensor:
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now Im trying to do minimal changes to create a proof of concept I can run.
Ive left this as a draft PR and its still definitely a WIP

I just find the PR user interface is great to observe changes

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related to: #356

img = img - np.min(img)
img = img / (1e-7 + np.max(img))
img = img - torch.min(img)
img = img / (1e-7 + torch.max(img))
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cv2 resize will need work to be done via a torch tensor. Will investigate once I get the concept working

result.append(img)
result = np.float32(result)
result = torch.tensor(np.array(result)).to(torch.float32) # TODO: Optimise this to use pre-initialised torch tensor
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite crude. My aim is to get torch working first. Then to go back to these hacks and get them optimal with a torch tensor approach once I know the rest of the changes to torch tensors are working.

Also helps identify the problem areas that need more attention.

None of this work is release ready

@TRex22
Copy link
Author

TRex22 commented Nov 5, 2022

Sorry about the deluge of commits. If its an issue please let me know.

I have remote access to the system Im running the library on and tend to commit in-order to execute my code there. Its part of my workflow but I can change it - if I generate too much noise

img = cv2.resize(img, target_size)
result.append(img)
result = np.float32(result)
# Disabled the target_size scaling for now
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cv2.resize is a strange function. Looking online for a torch alternative brings up a few results. All have pros and cons are not exactly the same (including the dimension transpose).

This method is used in a few places in the code base. However for the GradCam class which Im using for timing tests its primarily used for the transpose (AFAIK), which isnt necessary here (or at least Im getting sane results my side).

Will need to be fixed if this change to a pure torch approach is used

@@ -8,7 +8,7 @@

setuptools.setup(
name='grad-cam',
version='1.4.6',
version='1.4.7',
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just did this so on my compute node I know which version Im working with. In terms of what the real version bump would be Im open to a more major potential bump - as the final version of these changes would be substantial and potentially breaking

cam = cam[:, 0, :, :]
cams.append(cam)
cams.append(cam) # TODO: Handle this for torch tensors
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically pre-initialise a tensor. Ive found that to be drastically faster that lists when dealing with cuda / non-cpu devices

@TRex22
Copy link
Author

TRex22 commented Mar 9, 2023

For now Im just modifying the code as I need to get the last of the results I need for my masters dissertation ... Ill have to cleanup later on

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant