Split the package distribution again into one for stable (i.e. right now: 4.11) and one for devel (i.e., right now 4.12)

[fingolfin]

CC @alex-konovalov @ChrisJefferson @FriedrichRober @wilfwilson

In the old package distribution, we had (at least) two versions of the package distribution:

1. one for the current stable release, used on the corresponding stable branch (so right now: used for CI tests on stable-4.11, and also for any potential further 4.11.x release);
2. and one for the current development version / the GAP master branch.

With the new package distribution, we decided to simplify things and have only one distribution. The rationale went like this: we don't plan to make another 4.11.x release; and before 4.12 is out, there shouldn't be updates for packages that require GAP 4.12, because, well, 4.12 is not yet out.

Alas:

• the matgrp package has a pending update (see PR [matgrp] Update to 0.70 #227) which requires GAP 4.12 already
• some other package authors may wish to prepare for the 4.12 release by making releases of their packages that require GAP 4.12 and which they wish to be included with 4.12.0, i.e., those updates should be merged before 4.12 is released. In principle, we could do it very shortly before, i.e., merge them on the same day as 4.12.0 is released, but then we also have minimal time to test things with those new package releases.
• right now, stable-4.11 and master both reference the same distribution files. Once we start merging packages requiring 4.12, we need to update the stable-4.11 branch to pull a different (old) version of the package distribution...

I am not fully sure what the right conclusion to draw from this is... But suppose we do want to offer multiple versions of the distro again. Then IMHO the most natural way to do this would be to use branches: besides the main branch here (which serves GAP's master branch), add a stable-4.11 branch. Then adjust some workflows:

• assemble-distro.yml:
  • also trigger on pushes to stable-* branches
  • for those branches, replace latest by e.g. latest-4.11 as tag name
• scan-for-updates.yml:
  • for new packages create multiple PRs, one for main and one for each active stable-* branch (we'd probably only support one at a time, though)
  • however, only create PRs for stable-* if the package indicates it is compatible with the corresponding GAP version
• pull-request.yml:
  • don't run tests for both GAP stable-4.X and GAP master; just run them for the "appropriate" GAP branch (remember, there will be two separate PRs for each distro)
• likewise the reports of course could be update independently

Challenges:

• how do we sync the workflows and tools between the two (or more branches)?
  • we could again split this into two repositories, as it used to be (urgh), although integrating this with the workflows may be tricky
  • alternatively: keep the workflows and tools on the main branch; but put the packages into dedicated branches: say package-4.11 and packages-master (or packages-devel to avoid the boo-boo name master). This would also solve issue scan-for-updates.yml: only update "automatic" PRs when the package changed? #238

Thoughts, anyone?

[ChrisJefferson]

This all sounds good to me.

I think tools in main branch, packages in dedicated branches sounds cleanest. This would also make it easier at a later date to track changes to the code vs changes to the packages (although it's not too hard to separate).

One minor suggestion (and feel free to reject this if it would be difficult) -- rather than having packages-devel, we could just call it packages-4.12 from the start, this would mean it would be obvious from the branch name which version it was targeting. This would mean editing GAP master after a new release to point at "packages-4.12.tar.gz" (or whatever), but that doesn't feel like too difficult a request.

[FriedrichRober]

I think the solution with branches like package-4.11 and packages-master would be the most natural approach.

[fingolfin]

@ChrisJefferson to me both approach seem equivalent; the main differences are:

1. how we picture the "tree of branches" (either with a permanent package-master branch, of which we branch of packages-4.11, packages-4.12, etc.; or a constantly renamed "primary branch", which branches of the previous version. If one draws them, one gets the same tree just with different labels).
2. how we must adjust the GAP repository: once we create a stable-4.12 branch, we'll also create a packages-4.12 branch in this repository (resp. create a packages-4.12 branch following your model). And then we need to update Makefile.rules to use the "correct" package distro; the only difference between our two approach is whether this update needs to happen on the master branch (with your model) or on the newly created stable-4.12 branch (with my model; note that this is also how we did it so far, with the old distro).

So it is not really more or less difficult, just... different?

[fingolfin]

Just looked a bit into doing this, and run into a snag: managing our GitHub workflows. E.g. the workflow which is triggered on PRs... which workflow is used depends on the branch being targeted. So that means each packages-* branch would have to have a copy of these workflows. And if we want to tweak a workflow, we'd have to tweak it across all those workflows separately. I think that would be really undesirable!

A way out might be to reuse workflows which can call other workflows also in other repositories and/or branches -- we already use this to "call" test-all.yml from other workflows. So in that scenario, in the packages-* branches we'd have only a very "thin" bunch of workflows, which mostly consist of trigger conditions plus a call to the "real workflow" living on the main branch. Though note that "Reusable workflows can't call other reusable workflows", so we'd have to refactor some things.