[FEATURE] Add backward compatibility test framework

[martin-gaievski]

Is your feature request related to a problem?

Plugin is missing the feature of running backward compatibility tests for newly added features. This is important as many features are released as part of this plugin and we need to have more confidence that our changes are not breaking existent customer experience.

What solution would you like?

A framework that is ideally possible to run locally in dev setup and also can be run automatically as part of CI. It should test rolling and restart upgrades as both are possible real life scenarios.
Something similar to what knn repo has.

What alternatives have you considered?

Today we are doing this manually.

[navneet1v]

Resolving as this is duplicate