Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add device tensor documentation for GPU execution providers #20837

Merged
merged 3 commits into from
Jun 12, 2024
Merged

Add device tensor documentation for GPU execution providers #20837

merged 3 commits into from
Jun 12, 2024

Conversation

gedoensmax
Copy link
Contributor

This documentation adds documentation on:

  • how to allocate CUDA device tensors from C++ and python
  • how to use DML device tensors from C++ and python
  • it also shows how to leverage existing GPU allocations in ORT
  • how to overlap PCI copies and GPU execution using CUDA streams
  • how to overlap PCI copies and GPU execution using D3D12 Command Lists and custom resources

@gedoensmax
Copy link
Contributor Author

@PatriceVignola can you help review this for DML ?
@tianleiwu can you help review this for CUDA ?

@PatriceVignola
Copy link
Contributor

DirectML part looks good to me.

@gedoensmax
Copy link
Contributor Author

When will this be merged ? Is there anything else needed from my side ?

@tianleiwu tianleiwu merged commit f20db76 into microsoft:gh-pages Jun 12, 2024
1 of 2 checks passed
@tianleiwu tianleiwu mentioned this pull request Jun 13, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@PatriceVignola PatriceVignola PatriceVignola left review comments

@tianleiwu tianleiwu tianleiwu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@gedoensmax @PatriceVignola @tianleiwu