Calculate the parallelism based on required VUs

[JorTurFer]

Feature Description

Hello
We are integrating K6 operator for our load tests, and we have noticed that we have to define the parallelism as part of CR directly. In our use case, the SRE/Platform team has created all the required stuff for development teams, who only have to trigger a process.

This means that development teams just need to specify the test case + config file and the trigger starts. For us, development teams shouldn't have to know about the underlying infrastructure because it's something managed by SRE team, but currently they need to know it because they have to set the parallelism.

Our problem is that the same team can trigger multiple test with different VUs (depending on test case and config.json) but teams have to set both, the VUs for the tests and the parallelism. If SRE team (who knows about the infrastructure) could define the runner capabilities, keeping only the definition of required capabilities to dev teams, it'd be awesome.

It would be nice if we can transfer these responsibilities, abstracting this configuration somehow. I have some ideas for this:

• The runner definition could specify the VUs per instance and the operator can calculate the parallelism based on test case VUs. Thanks to this, SRE teams can generate templates for dev teams with the runner already defined. The SRE team defines the runners so they know the capabilities of the runner (in terms of VUs) and dev team defines the used VUs in tests.
• A new CRD is introduced to predefine runners. This new CRD would allow configuring anything inside the runner (including the VUs) and could be useful to reuse runner definition across different categories. For example, linux-windows runners, bigger or smaller resources, tooling, etc. The test CRD would link this already defined runner and the operator would calculate the parallelism based on test VUs and VUs per runner.

I think that the second option could be the more complex but maybe it's also the more flexible for the future. The only point where I have doubts is in how to get the test VUs because maybe it's hidden for the operator. I mean, maybe the initializer pod (who has access to the test and config even if it's provided by localfiles or pvc) could expose this info somehow, giving to the operator the required info for starting the test instances.

I can't promise it (because next weeks will be complicated), but I'd be willing to contribute with this if you think it's useful

[JorTurFer]

Any update about this? Do you have any feedback about the proposal @yorugac ?

[yorugac]

Hi @JorTurFer, sorry for the delay!

Thank you for opening the issue. The problem you described is understandable and easy to relate 👍

As for solutions, it doesn't seem like a good idea to put this as part of K6 / TestRun CRD. That CRD is getting overwhelmed as it is while a config option of VUs per instance is not actually tied to any one test run specifically. Lastly I suspect cases like you describe will not end with just one field. There might be other cases popping up, requiring yet additional fields. (kind of related #95, #191) Basically, here the issue is not so much about K6 / TestRun CRD but about having a higher level of abstraction.

The proposal to add a new CRD is actually a bit of a funny coincidence since we are now actively working on a new feature PLZv2. It contains a new PrivateLoadZone CRD in k6-operator which would have parallelism of TestRun defined on-the-fly by k6 Cloud. Configuration though is expected to be done via resources: during creation of PrivateLoadZone CR one can specify how much CPU and memory will be available on a node for the tests. Those values are used as a basis for calculation of parallelism.

IOW, in PLZ we will have basically the same mechanism you described executed under the hood. Is there a possibility that you could use PLZ to solve your use case? PLZ will be a k6 Cloud feature.

If not, a more practical and fast solution would be to have your own custom setup on top of k6-operator. AFAIK, that's what some folk do, sometimes even by building another operator, specific for their needs. It doesn't need to be an operator: it can be any kind of external solution. Would that be possible?

I must emphasize the "fast" part here: because adding another CRD to k6-operator right now is unlikely, IMHO. Firstly, because we're making a lot of large changes now in k6-operator and it will take some time to stabilize them. Including addition of Helm and deprecation of K6 CRD itself, #226

Last but not least, it'll take time and some good grokking to even design such a CRD as a generic solution that will fit many use cases. On that note, any additional feedback here would be very welcome. I'd like to understand better if many people in community encounter a similar issue.