Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release Upper Limit on Torch, Transformers, Accelerate, and Others #17

Open
fabianlim opened this issue May 23, 2024 · 5 comments
Open
Assignees
Labels
dependency Issue arises because of a dependency

Comments

@fabianlim
Copy link
Contributor

fabianlim commented May 23, 2024

Currently the torch dependency in framework is upper bounded as "< 2.3", however in accelerate versions has problems supporting torch 2.2. The latest numpy versions (>=2.0) also has incompatibilities with the current torch version and is bounded here in #42. Hence, we should consider releasing the upper bound soon.

Also can consider releasing the upper limit on transformers and accelerate

@fabianlim fabianlim self-assigned this May 23, 2024
@fabianlim fabianlim changed the title Release Upper Limit on Torch Release Upper Limit on Torch, Transformers, Accelerate May 28, 2024
@fabianlim
Copy link
Contributor Author

fabianlim commented Jun 24, 2024

we found out there is really no need to upper bound this torch dependency, as for us we are getting stuck only beause of this commit pytorch/pytorch#121635.

  • so the workaround for us can be just to downgrade to nvidia-nccl-cu12==2.19.3 Update: This is due to a NCCL_BUFFSIZE wrong setting.

Also for transformers, we just have to be weary of the sliding window mask issue of SPDA, and keep track of it to see when it will be fixed.

fabianlim added a commit that referenced this issue Jun 30, 2024
Signed-off-by: Yu Chin Fabian Lim <[email protected]>
@fabianlim
Copy link
Contributor Author

FMS has fixed the TRL issue foundation-model-stack/fms-hf-tuning#213

fabianlim added a commit that referenced this issue Jul 1, 2024
* add build and publish and bump version 0.1.0

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

* released torch and trl upper bounds #17

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

* update

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

* undo version bump and minor edit

Signed-off-by: Yu Chin Fabian Lim <[email protected]>

---------

Signed-off-by: Yu Chin Fabian Lim <[email protected]>
@fabianlim
Copy link
Contributor Author

fabianlim commented Jul 2, 2024

I think we need a lower limit on the bitsandbytes version that supports quant_storage. I have encourted that 0.41 didnt work, but 0.43 is ok.

Update: we also upper bound to 0.43.1 now as we encounted seg problems with the newer verisions. It could be that newer versions have certain CUDA or torch dependencies

@achew010
Copy link
Contributor

GPTQModel has a lower bound on transformers here but framework is bounded at transformers<4.40. This discrepancy should be resolved.

@fabianlim fabianlim added the dependency Issue arises because of a dependency label Jul 30, 2024
@fabianlim
Copy link
Contributor Author

We have removed the upper bound on transformers in #58

@fabianlim fabianlim changed the title Release Upper Limit on Torch, Transformers, Accelerate Release Upper Limit on Torch, Transformers, Accelerate, and Others Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependency Issue arises because of a dependency
Projects
None yet
Development

No branches or pull requests

2 participants