Enable MPS support for LitGPT #1724
Conversation
| result = x.new_zeros(*x.shape[:-1], self.linear.out_features) # (64, 64, 384) | ||
| return result.index_copy_(dim=-1, index=self.lora_ind, source=x) # (64, 64, 384) | ||
| else: | ||
| if all(self.enable_lora): |
There was a problem hiding this comment.
That shouldn't be in the compatibility mode.
In fact, only index copy should be altered with slicing in the compatibility mode.
|
To my mind, the implementation should not wire a mps compatibility mode through the model but just check the tensor device in index copy. |
Agree.
M1 runner for public repos should be free of charge: actions/runner-images#9254 |
|
Yeah, we need to switch to M1 anyways because x86 macs don't get PyTorch support anymore (I think the last version was 2.2). |
Re-enables MPS support for LitGPT with an alternative
index_copy_implementation.