Rawhide Bodhi updates created from sidetags not auto-pushed to stable

[nforro]

Normally, rawhide updates have Stable by Time: 0 days and are pushed to stable automatically after a few seconds/minutes.

However, the latest update for packit, specfile and ogr has been stuck in pending for 5 days now.

It would seem that updates created from sidetags behave differently in this regard, but for instance this one for packit and specfile from 2 weeks ago was successfully auto-pushed to stable.

Looking more closely, starting with packit-0.100.2-1.fc42 no rawhide update has been auto-pushed, so it would seem that it has something to do with rawhide switching from f41 to f42.

[mfocko]

Looking more closely, starting with packit-0.100.2-1.fc42 no rawhide update has been auto-pushed, so it would seem that it has something to do with rawhide switching from f41 to f42.

If that's the case, doesn't it suggest that it's an issue on the Bodhi side?

[LecrisUT]

It does seem to be specific to packit only:

• only packit is (still) stuck: https://bodhi.fedoraproject.org/updates/?search=&status=pending&releases=F42
• some other updates by packit in rawhide were obsoluted: https://bodhi.fedoraproject.org/updates/?search=&status=obsolete&releases=F42&page=1. Gating status there seems to be all over the place
• there were weird cases in f41 rawhide as well: https://bodhi.fedoraproject.org/updates/?search=&status=obsolete&releases=F41&gating=ignored&user=packit

Edit: seems like most other updates were obsoleted because of fast updates overwriting them, maybe it's packit specific

[nforro]

If that's the case, doesn't it suggest that it's an issue on the Bodhi side?

That's what I thought, but then there are updates that work as expected:
https://bodhi.fedoraproject.org/updates/FEDORA-2024-8dbf4f0df5

[decathorpe]

Looking at the history of one of the packages involved, it seems like the packit-stg user untagged them from the side-tag before it could be processed by bodhi:

$ koji list-history --build packit-0.101.1-1.fc42 
Wed Sep 18 15:37:59 2024 packit-0.101.1-1.fc42 tagged into f42-build-side-96225 by packit-stg
Wed Sep 18 15:38:10 2024 packit-0.101.1-1.fc42 untagged from f42-build-side-96225 by packit-stg
Sat Sep 21 04:33:04 2024 packit-0.101.1-1.fc42 tagged into f42-signing-pending by bodhi
Sat Sep 21 04:34:07 2024 packit-0.101.1-1.fc42 untagged from f42-signing-pending by autosign/autosign02.iad2.fedoraproject.org
Sat Sep 21 04:34:07 2024 packit-0.101.1-1.fc42 tagged into f42-updates-testing-pending by autosign/autosign02.iad2.fedoraproject.org [still active]

[nforro]

Thanks @decathorpe, removing the sidetag is most likely the cause of the issues. We need to prioritize and implement #2497, that should fix it.

[nforro]

Fixed by #2549.