-
-
Notifications
You must be signed in to change notification settings - Fork 876
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problem caching instances of torch modules and datasets #2339
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Caching chunks that create an instance of torch module or of a torch dataset yields an
external pointer is not valid
error when the instance is used in another chunk.Example with torch module:
Example with torch dataset:
If there is no cache, the chunks are executed without problems. However, when a cache exists, an error is created when trying to access the cached instance of the module or of the dataset:
This might be due to the fact that R torch package relies on reference classes (R6 and/or R7) and could be related to issue #2176. In any case, caching would be useful to cache trained instance of a module or instances of datasets which involve a lot processing during initialization.
At the moment, the only alternative is to save the torch model in the cached chunk with
torch_save
and load it in the uncached chunk withtorch_load
(see comments in the chunk above). However, afaik, there is no method to save and load torch datasets.The text was updated successfully, but these errors were encountered: