-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Make it possible to target a Cluster with Multiple Virtual Nodes #1768
Comments
@aleoli One clarification, only one VK deployment will be created when multiple VirtualNode CRD is created right? All VirtualNode of one remote cluster is handled by one VK deployment? Thank you for amazing feature. |
Our one remote cluster has 300’s of nodes, if we start creating VK deployment for every VirtualNode CRD of same remote cluster there will be 300 pods in host cluster for one remote cluster. If we peer 10 remote cluster, there will be 3000 pods. Operation effort is too much, also need to tune API server & etcd. It will be useful to have one VK deployment for all VirtualNode CRD of same remote cluster. |
Let's go by steps. You have a point, but the VirtualKubelet assumes to be handling only one node, and changing this will be a significant change and refactoring in its code. Let's start allowing multiple nodes for every cluster and then think about consolidating all these "similar" deployments into only one. (this is my personal opinion, open to discussion) And second, 300 nodes x 10 clusters will stress the API server in any case. On the other hand, do you need to see all these nodes in the central cluster? Can some policy of aggregation of nodes, for instance, one virtual node for each node poll, be helpful? |
I would suggest to go by steps:
This is intended to minimize disruptions while bringing this feature to life asap (incorporating incoming comments in the design). |
Thank you @aleoli and @frisso. I totally agree with you by going by steps. I agree with @aleoli that creating one VK for node pool be helpful as well. Will I get control on how VirtualNode CRD's are created? Sorry for my dummy questions. |
In the first implementation, some flags should be available to select pre-defined behaviors (i.e., one virtual node, one node per for each physical node, etc.). Note that a resource plugin interface can be implemented to have the preferred custom behavior. We should check and eventually fix it to work with multiple offers for the same request. I'm adding this step to the main issue steps. In the future, we should define other plugins in the library to handle the most exciting scenarios and add them to the repository. This way, we should avoid adding many flags to the primary resource offer controller. |
I agree with you. Looking forward to try it out. |
This is much needed feature for use as we have multiple node types like CPU, GPU and memory intensive. |
Any update on this. Is it possible we can help to implement or any timeline? Thank you |
Hi @Sharathmk99, at the moment you can create other nodes after peering. You need to use the command At the moment is not possible to target a subset of remote nodes with a virtual node. It is a feature that we are going to implement after the release of the new network. |
Just to give you some more info, we're deeply improving the internals of Liqo, with a strong focus on modularity. |
Hi @Sharathmk99! We will include full support for this feature in Liqo 0.12; we need to refactor the authentication module to support it fully #2382 However, you can add additional virtual nodes to your peering with Liqo 0.10 by creating additional VirtualNode CR or by using the |
Hi @aleoli, Thank you for the clarification. With v0.10.0 version can I add virtual node pointing to existing foreign cluster? Thank you!! |
Describe the solution you'd like
It should be possible to have multiple virtual nodes targeting the same remote cluster.
It could be helpful for many reasons, for instance:
The current
ResourceOffer
CRD spec should have two new fields (mutually exclusive to each other) to enable the remote cluster to handle node names.The creation of multiple resource offers will trigger the creation of multiple
VritualNode
resources and then multiple additional nodes.Required Steps
ResourceRequest
operator to enable the creation of aResourceOffer
for each nodeVirtualNode
(see [Feature] Add a Declarative Way to Handle Virtual Nodes Lifecycle #1766) CRD instead of VK DeploymentsFuture Steps
The text was updated successfully, but these errors were encountered: