-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
mount the dedicated storage for each function #1408
Conversation
Based on the current function design, when the Function is provided in the custom image, the code running in the ray nodes is the function code only. The user code doesn't run in the ray nodes so it can not access to the directory. |
charts/qiskit-serverless/charts/gateway/templates/rayclustertemplate.yaml
Show resolved
Hide resolved
- mountPath: /function_data | ||
name: user-storage | ||
subPath: {{`{{ function_data }}`}} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to mount this with any kind of restricted permissions? In theory, and correct me if I'm wrong here, whoever is running the function (user or provider) would be able to access this location via the container...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My understanding is that with the current custom image function design, the only the code in the custom image is executed in the Ray node. No user code is sent or executed in the Ray node so only the function code read/write to this directory.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Users could in theory find a way around that restriction (if we misimplement something, CVS / exploits, etc.) so it's a risk, we'd just need to determine if it's an acceptable one
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we just add the path when a function is from a provider? That could solve part of the problem. And correct me if I'm wrong here @akihikokuroda but all the providers will write here so they could potentially see files from other providers with this approach. What is the problem to use the name of provider as sub-path instead of function_data
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My idea was that in that way we could attach the specific provider path only when it's a function from the specific provider (but I don't know if there are limitations in this approach).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought function-data was the provider space and data was for users ? since users would already go to data on their own functions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the provider, it may be useful to have both while they develop the function because it is the same environment when the function is executed by the user.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought function-data was the provider space and data was for users ?
yep
since users would already go to data on their own functions
basically that's my comment, for functions that come from the user we are adding /function_data
too. It's true that /function_data
and /data
are going to point to user.username
path so not a bid deal but probably we can just avoid add /function_data
for user functions.
BTW, another answer that I would understand is: "well, we don't overcomplicate more the template this way" and I would agree. Yep.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Django template in helm template is very tricky :-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hahaha I agree, I agree. I'm just reviewing the rest of the code, Aki and I will update my review 👍
Good from my side. Just need to resolve David's question about mounting |
Sorry, I didn't read your answers yesterday. |
The Function can write something that it wants to share with the user. I'm planing providing the loggers to the function one for private logs and the other for the shared with the user. The later log is written in the file in |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logic LGTM Aki, thank you! BTW something I would like to improve in the near future are logs. We need to introduce more logs in general if not debug this in case of an error would be a nightmare 😅
Summary
Mount the dedicated storage for each function
Details and comments
The dedicated storage area is mounted at
/function_data
for the function provider. When the function doesn't have the associated provider, the storage mounted at/function_date
is the same storage as/data
.