Branching Strategy Implementation

[prudhvigodithi]

Is your feature request related to a problem? Please describe

Coming from the initial [Proposal] Branching strategy for build and the solution proposed as part of this proposal would be the closest implementation for the build repo.

To summarize the solution:

• The main branch is used for active development and targets the upcoming release version.
• A <MAJOR_VERSION>.x branch is cut for every major release version, example as 2.x, 3.x. This is inclined with the Core branching and with planned release cycles. Considering the next major release is 3.0.0, the main would have active development, new features and everything compatible for 3.x and above. A 2.x branch should have everything related to 2.x series. Now when the next upcoming release is 4.0.0, main would shift everything compatible for 4.x and above and 3.x branch will be added to the repo.
• During distribution jobs execution, for 2.2.1 release, the pipelines/jobs would use an input BRANCH_NAME and use the 2.x and reference.

NOTE: The on going activity to separate out the Jenkins shared libraries would indirectly influence this branching strategy. The jenkinsfiles should be modified to fetch the shared libs from external repo, example https://github.com/prudhvigodithi/opensearch-build-pipelines/blob/main/opensearch/distribution-build.jenkinsfile#L1-L4. With this, the jenkins shared libraries and its associated tests need not be part of this branching implementation.

There is other Open issue related to this topic to server the entire build repo as framework with right versioning, which can be later fetch during runtime using the right version.

• [POC] Build repo versioning framework. #2490

Describe the solution you'd like

• The Jenkinsfiles cannot be branched, hence the parking place for all the jenkinsfiles would be the main branch. The public Jenkins jobs will use Jenkinsfiles fetching from main branch.
• Add an input BUILD_BRANCH to the Jenkinsfiles so that jobs can clone the build repo with right BUILD_BRANCH used for the release.
  Jenkinsfiles to have an input BUILD_BRANCH. #2558
• Use gitScm within the Jenkinsfiles to clone to build repo BUILD_BRANCH to run the distribution workflows, so that the code is available to running workspace. Once the repo is branched the distribution workflows also are part of each branch, so when jenkins clones the right BUILD_BRANCH the job workspace will have all the workflows part of that specific BUILD_BRANCH.
  Jenkinsfiles to have an input BUILD_BRANCH. #2558
  https://github.com/prudhvigodithi/opensearch-build-pipelines/blob/main/opensearch/distribution-build.jenkinsfile#L78
• Modify/Ensure Jenkins job configuration points to the main branch of the build repo, where the jenkinsfiles are hosted.
• Create new branch 2.x and 3.x from main , main will be having the active development and 2.x, 3.x will have frequent backports. https://github.com/prudhvigodithi/opensearch-build/tree/2.x.
  NOTE: At this point the <MAJOR_VERSION>.x branch is expected to cut manually from main until an automation is derived.
• Configure the backport workflows to the build repo.
• In check for build Jenkins job, add the respective BRANCH_NAME to make sure the distribution jobs uses the right branch in build repo. Modify the workflow that raises an auto PR [AUTO] Updated input manifests. #2544 to update the check for build Jenkins job with BRANCH_NAME as well.
• Manifest workflow (that adds the manifests for every version) will always raises a PR to main, need to curate and backport the right manifests to the right <MAJOR_VERSION>.x branch.
• Update the README file with the branching solution incorporated.

Describe alternatives you've considered

No response

Additional context

No response

[dblock]

This branching strategy seems to assume that major version numbers are in sync. This is not the case.

The build repo builds many products that have significantly diverging build numbers, starting with OpenSearch and OpenSearch Dashboards that are going to be increasingly diverging as independent products. We follow semver, so there's no need to increment a major version number of OpenSearch unless there's a breaking change, and we avoid making breaking changes. Thus, within a year I suspect we'll be releasing OpenSearch 3.0 and OpenSearch Dashboards 5.0 at the same time.

[prudhvigodithi]

Hey @dblock I would like to move forward with branching based on existing setup where we publish OpenSearch, OpenSearch Dashboards together for every release. I'm not sure if we have a target date to separate out OpenSearch and OpenSearch Dashboards into a separate release channels. However if that is the long term goal, we should be going with OpenSearch Build repo own way of versioning, example the branch still be as 1.x, but this can support the upcoming releases like may be OS 3.3 and OSD 5.1 and we should be add a compatibility matrix for this. IF there is any breaking change for the build repo, we can go with 2.x which can support may be next OS 4.0 and OSD 6.0, in this scenario main would be still for active development, wdyt @dblock @bbarani any suggestions?

[prudhvigodithi]

Hey @dblock thanks for bringing this up, just to add we can start to cut the branch <MAJOR_VERSION>.x following the core, once the OS and OSD are diverged, we can keep the track using a compatibility matrix.

[dblock]

@prudhvigodithi Branching the build repo in its own version and following semver makes sense to me. I would start with a versioning scheme that clearly distinguishes it from OpenSearch or OpenSearch Dashboards (e.g. 0.1.0).

[prudhvigodithi]

@dblock just a thought, this would end up having lot of semVer branches created to manage both OpenSearch and OpenSearch Dashboards, also still main would be having both OpenSearch and OpenSearch Dashboards development included that needs to be backported to OpenSearch and OpenSearch Dashboard release semVer branches.

[dblock]

Yes, this plan is complicated. That's why we never branched this repo in the first place. But you seem to want to do it at all costs.

[prudhvigodithi]

Keeping this implementation on hold until a better approach is taken to address the scenario (future implementation), where the build repo handles multiple products and each of its version is offset with other. With this a single branch as <MAJOR_VERSION>.x cannot be the source of truth for a product which has an ongoing release <MAJOR_VERSION>+2.x
Example:
Lets say the build repo handles 2 products OpenSearch and OpenSearch Dashboards, OpenSearch 2.3.4 is about to get released and at the same time OpenSearch Dashboards 4.1.1 is bing released, then working on 2.x or 4.x branch does not solve the branching requirement and it would end up creating lot of branches per product as os-2.x, osd-4.x, etc

This implementation is more friendly to the existing approach (as of today September 12th 2022) where the build repo builds OpenSearch and OpenSearch Dashboards with same release version and are shipped together (Sample 2.3.0 release).

Related PR's and issues that are on hold:
#2559
#2558

As mentioned above the build repo can be branched as <PRODUCT>-<MAJOR_VERSION>.x, but with this keeping main as source of truth for all products does not solve the proper branching strategy as the <PRODUCT>-<MAJOR_VERSION>.x is cut from main, which means again every <PRODUCT>-<MAJOR_VERSION>.x has all other product build code.

Discussing with @dblock he proposes to split the src code to a separate repos and import it during runtime, to separate the python tooling and pull the right tooling from manifest using the branch (or version)
Something like

---
schema-version: '1.0'
build:
  name: OpenSearch
  version: 1.3.4
  build_tools: 1.3

@bbarani @dblock

[dblock]

Yes. Needing branches such as product-2.x should be telling us that we’re thinking about it wrong. The goal of this repo was to map 1:1 to what’s in Jenkins for all products and versions. That’s why we built backwards compatible manifest support. If a tool needs to be different across versions, it doesn’t belong in this repo but in its own. If a script needs to be different across versions, either both must be checked into this repo under different folders (like multiple manifests are) or live in the product repo that is branched already.

The example above that declares which version of tooling to use is perfect. It’s not an argument to the job, it’s part of the manifest that describes what’s needed to build it.

[bbarani]

@prudhvigodithi Can we close this issue for now and revisit later when we have sorted out the gaps?

[prudhvigodithi]

Hey @bbarani yes I'm ok to close this and re-open a new one that address the points we discussed above. I'm going to close this as not planned.