Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: to/from TensorFlow Tensor #3292

Conversation

maxymnaumchyk
Copy link
Collaborator

No description provided.

@maxymnaumchyk
Copy link
Collaborator Author

Note: as discussed here when converting a TensorFlow Tensor into Numpy array with tensor.numpy() it creates a copy since Numpy is mutable and TensorFlow Tensor is not.

When converting Awkward Array to TensorFlow (while on CPU), this function: tf.convert_to_tensor(numpy_array) also makes a copy. I suppose for the same reason.

Copy link

codecov bot commented Nov 6, 2024

Codecov Report

Attention: Patch coverage is 30.15873% with 44 lines in your changes missing coverage. Please review.

Project coverage is 82.06%. Comparing base (b749e49) to head (fbcf571).
Report is 182 commits behind head on main.

Files with missing lines Patch % Lines
src/awkward/operations/ak_to_tensorflow.py 28.12% 23 Missing ⚠️
src/awkward/operations/ak_from_tensorflow.py 27.58% 21 Missing ⚠️
Additional details and impacted files
Files with missing lines Coverage Δ
src/awkward/operations/__init__.py 100.00% <100.00%> (ø)
src/awkward/operations/ak_from_tensorflow.py 27.58% <27.58%> (ø)
src/awkward/operations/ak_to_tensorflow.py 28.12% <28.12%> (ø)

... and 158 files with indirect coverage changes

@maxymnaumchyk maxymnaumchyk marked this pull request as ready for review November 7, 2024 11:51
Copy link
Member

@jpivarski jpivarski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks perfect! Thank you again! This looks complete, and if you agree, I'll merge it into main.

On copying data from TensorFlow to NumPy: it might not be_technically_ necessary, since NumPy arrays have flags and can be marked as not writable. Thus, a NumPy array could zero-copy view a TensorFlow tensor in RAM (not GPU memory), if TensorFlow has stable pointers. Some systems, such as Java, copy data to different locations to defragment their memory use, which causes simple pointers (static integers) to get out of date. If TensorFlow does that, then it has to copy when converting to NumPy, even with NumPy's "writable" flag.

But this is probably out of our hands, since we have to rely on what functions TensorFlow gives us. They might have stable pointers that could be viewed as a non-writable NumPy array and yet just don't give us a function to do that—all we could do is ask them to add such a thing to their library (which we won't, because it's not such a big deal).

@jpivarski
Copy link
Member

(If you agree and it's ready to be merged, write a comment, rather than an emoji, since I get emails for comments but not emojis. Thanks!)

@jpivarski jpivarski linked an issue Nov 7, 2024 that may be closed by this pull request
7 tasks
@jpivarski
Copy link
Member

It looks done to me. I'm going to merge it. Thanks!

@jpivarski jpivarski merged commit fa7be40 into scikit-hep:main Nov 7, 2024
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add interoperability between Awkward Array and ML libraries
2 participants