Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Detect CentOS properly instead of defaulting to epel #525

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Detect CentOS properly instead of defaulting to epel #525

wants to merge 2 commits into from

Conversation

jakub-vavra-cz
Copy link

@jakub-vavra-cz jakub-vavra-cz commented Mar 1, 2024 •
edited
Loading

= changelog =
msg: Fix detection of CentOS Stream
type: bugfix
resolves: #524

@praiskup
Copy link
Member

praiskup commented Mar 8, 2024

Thank you for the PR!

The authors wanted to use epel-X on the Enterprise Linux family of distributions, and I believe this is what users are used to having - this might well be a breaking change (at least the part of PR affecting RHEL).

For the other cases, the dnf copr enable PROJECT centos-stream-9-x86_64 command variant lets you select the chroot. There's no logic for fallbacks (like try epel, then rhel, then centos-stream, amazonlinux, ...). Something along those lines was done in dnf5 copr enable.

@praiskup
Copy link
Member

praiskup commented Mar 8, 2024

Just for the record, dnf4 copr enable downloads repofiles from copr frontend, something like:
https://copr.fedorainfracloud.org/coprs/@mock/mock/repo/fedora-39/dnf.repo?arch=x86_64
With this approach, dnf copr has to a decision in advance, before the request is done.

The dnf5 copr enable though works with (a relatively new) Copr API that provides better "project introspection":
curl https://copr.fedorainfracloud.org/api_3/rpmrepo/@mock/mock/rhel-8/ | jq
dnf5 copr then builds the repo file contents on the client side.
Using the same API in dnf4 copr would be interesting, though the question is whether we want to do such a rewrite.

praiskup
praiskup reviewed Mar 8, 2024
View reviewed changes
plugins/copr.py
@@ -471,6 +471,8 @@ def _guess_chroot(self):
chroot = ("fedora-rawhide-" + distarch)
else:
chroot = ("fedora-{0}-{1}".format(dist[1], distarch))
elif "CentOS Stream" in dist:
chroot = ("centos-stream-{}-{}".format(dist[1].split(".", 1)[0], distarch if distarch else "x86_64"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the update.

This is not something I'd love to make decisions on. I'd suggest you: ask the CentOS Stream community.

Just note that the CentOS Stream build chroots have a lower chance to even exist in the project:
image

And then, from my observation only, it is a relatively known fact that epel-* is the default decision DNF copr plugin on the Enterprise Linux-based distros, so if anyone has to provide some "stable and supported content" in Copr for EL, they provide epel- chroots. The centos-stream-* or centos-stream+epel-* chroots in Fedora Copr are typically used just for CI testing.

@jakub-vavra-cz
Copy link
Author

Hello @praiskup ,
I have dropped the RHEL part of change as it might be breaking.
I still think that each distro should default to its appropriate chroot/repo and only if You want something else You should select chroot explicitly, but this is not the hill I die on.

We are running the same automation on fedora, centos and rhel so we want to just run "dnf copr enable PROJECT" and we expect that it will work properly pulling the right one.

I wanted to avoid writing my own _guess_chroot in ci automation.

According to Your plot there is a significant number of centos-stream-9-x86_64 and centos-stream-8-x86_64.

@praiskup
Copy link
Member

praiskup commented Mar 8, 2024

Just two more interesting queries, number of EL-based chroots (or "projects") currently in Fedora Copr (excluding the temporary projects):

coprdb=# select mock_chroot.id, mock_chroot.os_release, mock_chroot.os_version, count(*) as c from copr_chroot join mock_chroot on mock_chroot.id = copr_chroot.mock_chroot_id join copr on copr.id = copr_chroot.copr_id where mock_chroot_id in (select id from mock_chroot where os_version in ('8', '9') and copr.delete_after is null and is_active = true and os_release != 'mageia' order by os_release) group by mock_chroot.id order by c desc;
┌─────┬─────────────────────────┬────────────┬──────┐
│ id  │       os_release        │ os_version │  c   │
├─────┼─────────────────────────┼────────────┼──────┤
│  82 │ epel                    │ 8          │ 5732 │
│ 143 │ epel                    │ 9          │ 2999 │
│  85 │ centos-stream           │ 8          │ 2276 │
│ 130 │ centos-stream           │ 9          │ 1895 │
│  84 │ epel                    │ 8          │ 1871 │
│ 144 │ epel                    │ 9          │ 1354 │
│  86 │ centos-stream           │ 8          │ 1070 │
│ 131 │ centos-stream           │ 9          │ 1002 │
│  83 │ epel                    │ 8          │  943 │
│ 148 │ epel                    │ 9          │  717 │
│ 146 │ epel                    │ 9          │  637 │
│ 171 │ rhel                    │ 9          │  636 │
│ 145 │ epel                    │ 8          │  489 │
│  87 │ centos-stream           │ 8          │  464 │
│ 170 │ rhel                    │ 8          │  427 │
│ 168 │ centos-stream+epel-next │ 9          │  426 │
│ 147 │ centos-stream           │ 9          │  412 │
│ 155 │ centos-stream           │ 9          │  345 │
│ 178 │ rhel                    │ 9          │  295 │
│ 165 │ centos-stream+epel-next │ 9          │  276 │
│ 162 │ centos-stream+epel-next │ 8          │  258 │
│ 163 │ centos-stream+epel-next │ 8          │  173 │
│ 166 │ centos-stream+epel-next │ 9          │  170 │
│ 167 │ centos-stream+epel-next │ 9          │  164 │
│ 164 │ centos-stream+epel-next │ 8          │  126 │
│ 189 │ rhel                    │ 8          │  118 │
│ 188 │ rhel                    │ 9          │   97 │
│ 187 │ rhel                    │ 8          │   68 │
└─────┴─────────────────────────┴────────────┴──────┘

More epel-enabled chroots.

Here including the temporary projects:

coprdb=# select mock_chroot.id, mock_chroot.os_release, mock_chroot.os_version, count(*) as c from copr_chroot join mock_chroot on mock_chroot.id = copr_chroot.mock_chroot_id join copr on copr.id = copr_chroot.copr_id where mock_chroot_id in (select id from mock_chroot where os_version in ('8', '9') and is_active = true and os_release != 'mageia' order by os_release) group by mock_chroot.id order by c desc;
┌─────┬─────────────────────────┬────────────┬───────┐
│ id  │       os_release        │ os_version │   c   │
├─────┼─────────────────────────┼────────────┼───────┤
│  85 │ centos-stream           │ 8          │ 17167 │
│  82 │ epel                    │ 8          │ 16118 │
│ 130 │ centos-stream           │ 9          │ 15736 │
│ 143 │ epel                    │ 9          │  8688 │
│ 131 │ centos-stream           │ 9          │  5825 │
│  84 │ epel                    │ 8          │  4869 │
│ 170 │ rhel                    │ 8          │  4318 │
│  86 │ centos-stream           │ 8          │  4178 │
│ 171 │ rhel                    │ 9          │  3486 │
│ 168 │ centos-stream+epel-next │ 9          │  3016 │
│ 165 │ centos-stream+epel-next │ 9          │  2849 │
│ 162 │ centos-stream+epel-next │ 8          │  2827 │
│ 163 │ centos-stream+epel-next │ 8          │  2738 │
│ 144 │ epel                    │ 9          │  2380 │
│  83 │ epel                    │ 8          │  1242 │
│ 148 │ epel                    │ 9          │  1058 │
│ 146 │ epel                    │ 9          │   960 │
│ 178 │ rhel                    │ 9          │   954 │
│ 145 │ epel                    │ 8          │   736 │
│ 147 │ centos-stream           │ 9          │   612 │
│ 189 │ rhel                    │ 8          │   571 │
│ 155 │ centos-stream           │ 9          │   544 │
│  87 │ centos-stream           │ 8          │   471 │
│ 166 │ centos-stream+epel-next │ 9          │   172 │
│ 167 │ centos-stream+epel-next │ 9          │   166 │
│ 188 │ rhel                    │ 9          │   156 │
│ 164 │ centos-stream+epel-next │ 8          │   127 │
│ 187 │ rhel                    │ 8          │    70 │
└─────┴─────────────────────────┴────────────┴───────┘
(28 rows)

The temporary projects are usually created for CI purposes (e.g. Packit && pull-requests).
Data here.

@jakub-vavra-cz jakub-vavra-cz changed the title Detect CentOS and RHEL properly instead of defaulting to epel Detect CentOS properly instead of defaulting to epel Mar 8, 2024
@praiskup praiskup mentioned this pull request Mar 13, 2024
Open
@carlwgeorge carlwgeorge mentioned this pull request Apr 27, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@praiskup praiskup praiskup left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

copr plugin: _guess_chroot does not handle centos stream properly
2 participants
@jakub-vavra-cz @praiskup