This repository has been archived by the owner on Feb 12, 2022. It is now read-only.
Uses "--no-cuda" and implements device-agnostic code #81
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This utility method sets the `args.device` parameter, which indicates if the computation will be made on the GPU using CUDA or the CPU. To choose between the two, the method checks the host compatibility and the `args.no_cuda` parameters. With the "args.device", one can initialize tensors or models without knowing if it should use the CPU or GPU. For example, initializing a tensor: `torch.rand(10, device=args.device), or model: `model = model().to(args.device)`. If CUDA is used and `args.seed` exists, the CUDA seed is manually set.
Using "args.device" removes the need for an if block, making the code lighter.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request adds two contributions:
--cudato--no-cuda, with consistent behaviorIn summary, I added a new utility function
init_device()inutils.py, which adds toargsadeviceproperty. This property is atorch.device()object which indicates where tensors and models should reside. We can then write code without knowing if the computation is on the CPU or GPU. For example with a tensor:data = torch.rand(10, device=args.device)or a model:model = model.to(args.device). This completely removes the need to write conditional blocks.If using the "--no-cuda" (or there is no CUDA device available), the device is set to the CPU.
This version was checked against the Experiments with consistent results.
Related to #23 and #6