-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Transfer files to/from memory #137
Comments
Interessting feature, especially for small datafiles in automated workflows. I would suggest to make an (not too high) upper bound on the filesize where this is possible, as nowadays datafiles in the TB-range are not uncommon. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When Scitacean is tightly integrated into a workflow, there may not be a need to save data to disk. Instead, it could be useful to load some data directly into memory from the fileserver or write an in-memory buffer to the fileserver.
This would require support from
File
which could store aBytesIO
instead of aPath
forlocal
. And the file transfers would read from / write to the buffer instead of a actual file.There are some downsides:
The text was updated successfully, but these errors were encountered: