Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI/CD - Upgrade dependency versions in pipeline #671

Merged
merged 10 commits into from
Nov 27, 2024
Merged

Conversation

abuccts
Copy link
Member

@abuccts abuccts commented Nov 23, 2024

Upgrade dependency versions in Azure pipeline:

  • Remove Python 3.6 and add Python 3.10 for cpu-unit-test
  • Upgrade CUDA from 11.1 to 12.4 for cuda-unit-test
  • Update labels accordingly

Upgrade dependency versions in Azure pipeline:
* Remove Python 3.6 and add Python 3.10 for cpu-unit-test
* Upgrade CUDA from 11.1 to 12.4 for cuda-unit-test
* Update labels accordingly
@abuccts abuccts added the CI/CD Continuous integration or deployment label Nov 23, 2024
@abuccts abuccts requested a review from a team as a code owner November 23, 2024 02:41
Copy link

codecov bot commented Nov 23, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 85.58%. Comparing base (7cef624) to head (7d1f04e).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #671      +/-   ##
==========================================
- Coverage   85.75%   85.58%   -0.18%     
==========================================
  Files          98       98              
  Lines        7043     7046       +3     
==========================================
- Hits         6040     6030      -10     
- Misses       1003     1016      +13     
Flag Coverage Δ
cpu-python3.10-unit-test 71.84% <100.00%> (?)
cpu-python3.7-unit-test 71.81% <100.00%> (+<0.01%) ⬆️
cpu-python3.8-unit-test 71.84% <100.00%> (+<0.01%) ⬆️
cuda-unit-test 83.39% <100.00%> (-0.49%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Fix sudo issue inside container.
Update.
Fix build.
Fix build.
Fix unit test.
Update.

Co-authored-by: Dilip Patlolla <[email protected]>
Fix unit test.
Copy link
Contributor

@polarG polarG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

dpower4 and others added 2 commits November 26, 2024 20:37
Fix cache tests issues with cuda 12.4 and pytorch 2

https://pytorch.org/docs/stable/notes/cuda.html#cuda-memory-management:
```
PyTorch uses a caching memory allocator to speed up memory allocations. This allows fast memory deallocation without
device synchronizations. However, the unused memory managed by the allocator will still show as if used in nvidia-smi. 
You can use memory_allocated() and max_memory_allocated() to monitor memory occupied by tensors, and use 
memory_reserved() and max_memory_reserved() to monitor the total amount of memory managed by the caching allocator. 

Calling empty_cache() releases all unused cached memory from PyTorch so that those can be used by other GPU 
applications. However, the occupied GPU memory by tensors will not be freed so it can not increase the amount of GPU 
memory available for PyTorch.
```
Revert changes for transformer.
@abuccts abuccts merged commit 96f5cce into main Nov 27, 2024
19 of 20 checks passed
@abuccts abuccts deleted the xiongyf/upgrade-pipeline branch November 27, 2024 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI/CD Continuous integration or deployment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants