Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Change rm to use find to avoid deleting cache directory during snapshot cleanup #10756

Merged
merged 2 commits into from
Oct 17, 2024
Merged

Change rm to use find to avoid deleting cache directory during snapshot cleanup #10756

merged 2 commits into from
Oct 17, 2024

Conversation

sameluch
Copy link
Contributor

@sameluch sameluch commented Oct 16, 2024 •
edited
Loading

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

  • The toolchain has been rebuilt successfully (or no changes were made to it)
  • The toolchain/worker package manifests are up-to-date
  • Any updated packages successfully build (or no packages were changed)
  • Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
  • Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
  • All package sources are available
  • cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
  • LICENSE-MAP files are up-to-date (./LICENSES-AND-NOTICES/SPECS/data/licenses.json, ./LICENSES-AND-NOTICES/SPECS/LICENSES-MAP.md, ./LICENSES-AND-NOTICES/SPECS/LICENSE-EXCEPTIONS.PHOTON)
  • All source files have up-to-date hashes in the *.signatures.json files
  • sudo make go-tidy-all and sudo make go-test-coverage pass
  • Documentation has been updated to match any changes to the build system
  • Ready to merge
Summary

What does the PR accomplish, why was it needed?
When using the precacher, the cache directory can be removed after creation causing a file not found, breaking the build. Rather than removing the folder, we will instead delete all of the rpms contained within.

Change Log
  • use find -exec rm to clean up cached rpms instead of rm -rf the cache folder
Does this affect the toolchain?

NO

Test Methodology
  • Pipeline build id: xxxx

@sameluch sameluch requested a review from a team as a code owner October 16, 2024 21:38
PawelWMS
PawelWMS reviewed Oct 16, 2024
View reviewed changes
toolkit/scripts/imggen.mk Outdated
@@ -79,7 +79,7 @@ clean-imagegen:
# the same, but the actual .rpm files may be fundamentally different.
$(STATUS_FLAGS_DIR)/imagegen_cleanup.flag: $(depend_REPO_SNAPSHOT_TIME)
@echo "REPO_SNAPSHOT_TIME has changed, sanitizing rpm cache"
rm -rf $(local_and_external_rpm_cache)
find $(local_and_external_rpm_cache) -type f -name '*.rpm' -exec rm -f {} +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI find already has a -delete action, so you should be able to shorten this a bit. Also, just to be safe, it's good to surround paths with quotes in case there are any spaces in the path:

Suggested change
find $(local_and_external_rpm_cache) -type f -name '*.rpm' -exec rm -f {} +
find "$(local_and_external_rpm_cache)" -type f -name '*.rpm' -delete

mfrw
mfrw reviewed Oct 17, 2024
View reviewed changes
toolkit/scripts/pkggen.mk
@@ -269,7 +269,7 @@ clean-compress-srpms:
# the same, but the actual .rpm files may be fundamentally different.
$(STATUS_FLAGS_DIR)/build_packages_cache_cleanup.flag: $(depend_REPO_SNAPSHOT_TIME)
@echo "REPO_SNAPSHOT_TIME has changed, sanitizing rpm cache"
rm -rf $(remote_rpms_cache_dir)
find "$(remote_rpms_cache_dir)" -type f -name '*.rpm' -delete
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering why are using the fasttrack/3.0 branch for this change instead of 3.0-dev ?
/cc @PawelWMS

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mfrw we are putting this to fasttrack since it is a bug which blocks the build (fasttrack is using the precacher which is what discovered this build break). As a result, we will put this to fasttrack to unblock then cherry pick to 3.0-dev to follow the normal fasttrack procedure.

@jslobodzian jslobodzian merged commit a65729d into fasttrack/3.0 Oct 17, 2024
23 checks passed
@jslobodzian jslobodzian deleted the sammeluch/snapshot-cache-clean branch October 17, 2024 18:02
CBL-Mariner-Bot pushed a commit that referenced this pull request Oct 17, 2024
@sameluch @CBL-Mariner-Bot
Change rm to use find to avoid deleting cache directory during snapsh…
80e3a67 
…ot cleanup (#10756)

(cherry picked from commit a65729d)
@CBL-Mariner-Bot CBL-Mariner-Bot mentioned this pull request Oct 17, 2024
Merged
@CBL-Mariner-Bot
Copy link
Collaborator

Auto cherry-pick results:

Auto cherry-pick pipeline run -> https://dev.azure.com/mariner-org/mariner/_build/results?buildId=659301&view=results

@PawelWMS PawelWMS mentioned this pull request Oct 21, 2024
Merged
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mfrw mfrw mfrw left review comments

@PawelWMS PawelWMS PawelWMS approved these changes

@jslobodzian jslobodzian jslobodzian approved these changes

Assignees
No one assigned
Labels
Tools
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@sameluch @CBL-Mariner-Bot @mfrw @jslobodzian @PawelWMS