weekly release 2.350 failing due to missing command mb

[dduportal]

Service(s)

release.ci.jenkins.io, Update center

Summary

The weekly core release 2.350 fails at the end of the packaging phase due to a missing command mb as raised by @timja in IRC.

Reproduction steps

No response

[dduportal]

Phase 1: diagnostics

• The error is thrown by the "packaging" pipeline: https://release.ci.jenkins.io/blue/organizations/jenkins/core%2Fpackage/detail/master/196/pipeline/ on this particular shell step: https://github.com/jenkins-infra/release/blob/master/Jenkinsfile.d/core/package#L336
• In particular, this is the following shell instruction which triggers the error: https://github.com/jenkins-infra/release/blob/master/utils/release.bash#L532 .
• This shell instruction executes a shell script /srv/releases/sync.sh on the remote virtual machine pkg.origin.jenkins.io which hosts the update center and the packages service.
• Following (mirrors.jenkins.io/http://mirrors.jenkins-ci.org/) Sunset the legacy "mirrorbrain" service in favor of get.jenkins.io #2888 , this machine is under puppet management again (after 2 years without). The scripts located in /src/releases used to be (years ago...) tracked in https://github.com/jenkins-infra/jenkins-infra/tree/be1dddf6c44bd8929f1721ddb3bdd241f55e0e87/dist/profile/files/mirrorbrain and managed in the mirrorbrain profile: https://github.com/jenkins-infra/jenkins-infra/blob/be1dddf6c44bd8929f1721ddb3bdd241f55e0e87/dist/profile/manifests/mirrorbrain.pp#L141-L145 . But it's not the case since 2 years.
• Hopefully, the directory /srv/release on the machine, which is also the home directory of the user mirrorbrain, is a git repository pointing to https://github.com/jenkins-infra/mirror-scripts
• Annnnd we got the culprit: https://github.com/jenkins-infra/mirror-scripts/blob/bd156b4fd45b1e68265051ac4160f285da064c0a/sync.sh#L87

[dduportal]

Phase 2: trying to fix

• Added a commit (jenkins-infra/mirror-scripts@bd6dcfc) that removes the mirrorbrain step code (hotfix)
• Carefully checked that the code on the VM (in /srv/release) is the same as the master branch ✅
• Created a local git branch to backup, named current-prod-20220601-10h08Z + backed up the sync.sh script to `/root/ on the VM
• Retrieved the latest changes on the VM: cd /srv/release && git checkout master && git pull origin master
• Triggered a new release: https://release.ci.jenkins.io/blue/organizations/jenkins/core%2Fpackage/detail/master/197/pipeline

[dduportal]

@timja seems like it's fixed: build #197 of the release packaging ran successfully.

Bonus: it does not sleep 2 min waiting for OSUSL (no need to now that mirrorbrain is gone) so faster release :)

[dduportal]

Many thanks for the help @lemeurherve !