Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix wrong directory content copying resulting in repodata/repodata #1144

Merged
merged 12 commits into from
May 10, 2024

Conversation

felixsch
Copy link
Contributor

@felixsch felixsch commented May 6, 2024

Description

card: https://trello.com/c/DZm26xHm/3342-rmt-metadata-is-not-correctly-moved-to-repoistory-root-but-into-repodata-repodata-if-exists
issue: #1136

Due to changed mechanics after removing the .old_* backup mechanism from https://github.com/SUSE/rmt/blob/master/lib/rmt/mirror/base.rb#L134 in 5aae83e does move the directory into the destination directory if already exists, instead of merging them.

Additional fixes in this PR:

  • Saw that licenses are not correctly mirrored if available. This is due to we scan for repositories ending with product instead of products/. See second commit.

  • Added a cleanup method obsolete or broken metadata. This means .old_* directories getting automatically removed and repodata/repodata is deleted if present.

How to test this pull request:

#### set the stage
$ cd <rmt-src>
$ git checkout master
$ bin/rmt-cli product enable SLES/15.5/x86_64

$ bin/rmt-cli mirror repository 5664
# Do it twice to trigger the issue
$ bin/rmt-cli mirror repository 5664

$ ls -la public/repo/SUSE/Products/SLE-Product-SLES/15-SP5/x86_64/product/repodata/
# expect: Now you have `repodata` in `repodata`

# For comparison later
$ stat public/repo/SUSE/Products/SLE-Product-SLES/15-SP5/x86_64/product/repodata/repomd.xml

$ ls -la public/repo/SUSE/Products/SLE-Product-SLES/15-SP5/x86_64/product.license/
# expect: Does not exist

#### perform the review
$ git checkout fix-broken-move-metadata

$ bin/rmt-cli mirror repository 5664
# expect: licenses are now mirrored, check the log output

$ ls -la public/repo/SUSE/Products/SLE-Product-SLES/15-SP5/x86_64/product/repodata/
# expect:  has no repodata in repodata

$ stat public/repo/SUSE/Products/SLE-Product-SLES/15-SP5/x86_64/product/repodata/repomd.xml
# expect: Compared to the stat above this should now be updated

$ ls -la public/repo/SUSE/Products/SLE-Product-SLES/15-SP5/x86_64/product.license/
# expect: Exists and has licenses mirrored

Thanks for the review. I you have questions please do no hesitate to ask! 🚀

cheers!

* Remove stale `repodata/repodata` and `.old_*` directories which might
  still exist in customers RMT instances
We need to match for `product/` to detect license enabled repositories
instead of `product` since RMT adds trailing slashes to all URLs.
@felixsch felixsch added the WIP Work in progress, do not merge. label May 6, 2024
@felixsch felixsch changed the title Fix broken move metadata Fix wrong directory content copying resulting in repodata/repodata May 6, 2024
@felixsch felixsch force-pushed the fix-broken-move-metadata branch 3 times, most recently from e21e11a to 181de5c Compare May 7, 2024 06:43
lib/rmt/mirror/base.rb Outdated Show resolved Hide resolved
@felixsch felixsch removed the WIP Work in progress, do not merge. label May 8, 2024
package/obs/rmt-server.changes Outdated Show resolved Hide resolved
Co-authored-by: Natnael Getahun <[email protected]>
Copy link
Contributor

@ngetahun ngetahun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Cleaning up wrong repodata/repodata is not implemented. verified that the wrong directory gets cleaned up

@ngetahun ngetahun merged commit e568da5 into master May 10, 2024
3 checks passed
@ngetahun ngetahun deleted the fix-broken-move-metadata branch May 10, 2024 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants