Tezos/Octez: Mark old packages as unavailable

[killian-delarue]

This pull request makes the old Octez pacakges unavailable

[samoht]

Hey, could you tell us what the reason for this PR is?

Usually, the policy for opam-repository is to keep the old package version available as much as possible so that we can still run automated tests on those (for instance, on new compiler versions - which might not be very relevant there).

[killian-delarue]

Hey, could you tell us what the reason for this PR is?

Usually, the policy for opam-repository is to keep the old package version available as much as possible so that we can still run automated tests on those (for instance, on new compiler versions - which might not be very relevant there).

These old packages are now useless as it can't be used on any octez networks. I suggest this PR in case it could help reduce the charge of these automated tests.

Do you think that they shouln't ?

[kit-ty-kate]

We’ve already discussed this before but agree as a general concept (packages unusable and unused)

Those packages should still pass the linter. You can check them all in one go using:

opam lint tezos-*/*/opam octez-*/*/opam | grep Errors

[avsm]

I'm not in agreement with making these packages uninstallable without good reason. If the issue is that automated tests shouldn't be run on these, then we should modify the test systems to have a better filter, and not apply a hammer of making entire swathes of packages uninstallable.

But once a package enters the opam repository, you don't know who the downstream users are, and it's often useful years down the road to rebuild older versions of packages. I'd like a broader discussion or justification before disabling hundreds of packages, please. Where was the previous discussion that @kit-ty-kate mentions?

[emillon]

I opened #23697 to discuss about what should be done for this situation.

[mseri]

I was one of the advocate against removal, but I do think there is value in this PR marking the packages unavailable. The state of things is that most of them are not usable to interact with tezos but, nevertheless, we are still testing them all each times. People can still use them by allow availability in their own opam overlays, pinning to a specific version of opam (which should be fine if they are firm on older versions of packages) or ignoring the constraint manually.

The current core release is re-testing lots of tezos for all the versions, leading effectively to a large waste of resources (time and energy for sure): https://toxis.caelum.ci.dev/github/ocaml/opam-repository/commit/883b11d2cbddc106afbb20b05cfe1c27505e97d8
Were this be merged, we would have a rather smaller revdeps test matrix.

In fact, I start to think it may be worth start doing pruning for all packages at some point.

[emillon]

What do you think about the proposal in #23697?

[mseri]

I am just leaving a comment there. I think figuring out a proper policy will require more discussion though

[avsm]

I agree that we need to test fewer revdeps, but the answer isn't necessarily to make perfectly installable packages uninstallable. I do think we should have hints in the opam metadata that is less strong than available to indicate that the package should be avoided in day-to-day CI (e.g. for revdeps), but included in bulk builds of the full repository. There is precedent for this in the use of avoid-version.

[tmcgilchrist]

@avsm the other place to consider is when building documentation for ocaml.org. Perhaps this isn't the right place to discuss but the metadata you're talking about would benefit there too. We spend a good amount of time building older Tezos packages that might not be used, not longer relevant or actively discouraged due to bugs.

[mseri]

The CI just broke again mostly due to the thousands of revdeps jobs from tezos running in parallel PRs, this PR would have avoided a large part of those to be run. I do think that for the time being there is reason in marking these as unavailable.

Imho we can revert the change in the future once we have a working solution.

Ping @avsm

[mtelvers]

With fewer cluster resources at the moment, it would be great if this PR could be merged.

[mseri]

I am going on with this