Skip to content

Commit

Permalink
Initialize default device lazily
Browse files Browse the repository at this point in the history
This way, we can import the library without requiring a GPU to be
present.
  • Loading branch information
janEbert committed Sep 18, 2024
1 parent cc7614e commit 3206285
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion megablocks/layers/arguments.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ class Arguments:
# Initialization arguments.
fp16: bool = True
bf16: bool = False
device: Union[int, torch.device] = torch.cuda.current_device()
device: Union[int, torch.device] = dataclasses.field(default_factory=torch.cuda.current_device)
init_method: InitFn = partial(torch.nn.init.normal_, mean=0.0, std=0.02)
output_layer_init_method: InitFn = init_method

Expand Down

0 comments on commit 3206285

Please sign in to comment.