Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Fix caching-related warnings in GHA build-test-publish CI #475

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jhlegarreta
Copy link

@jhlegarreta jhlegarreta commented Dec 20, 2024

Fix caching-related warnings in GHA build-test-publish CI:

  • Save the apt-get cache in a directory other than /var/lib/apt to avoid permission issues.
  • Make the AFNI cache key name be specific to the CI matrix configuration to avoid clashes across cache names. Use the root part as the restore key name so that any cache found can be restored, as the AFNI version being installed is the same across configurations.
  • Check if the AFNI cache exists before trying to install it in GitHub Actions build-test-publish CI workflow.

Fixes:

Failed to save: "/usr/bin/tar" failed with error: The process '/usr/bin/tar' failed with exit code 2

The full log showing

2024-12-19T13:35:45.2830631Z
 [command]/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/sdcflows/sdcflows --files-from manifest.txt --use-compress-program zstdmt
2024-12-19T13:35:46.5536670Z
 Failed to save: Unable to reserve cache with key afni-v1, another job may be creating this cache. More details: Cache already exists. Scope: refs/heads/master, Key: afni-v1, Version: d04022ae09f8f21b8c0f9f00e4a784b6e510fe6a47d30aa3b0853a42885b92cb
2024-12-19T13:35:46.5924639Z
 Post job cleanup.
2024-12-19T13:35:46.7348688Z
 [command]/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/sdcflows/sdcflows --files-from manifest.txt --use-compress-program zstdmt
2024-12-19T13:35:46.8326360Z
 /usr/bin/tar: ../../../../../var/lib/apt/lists/lock: Cannot open: Permission denied
2024-12-19T13:35:47.1807249Z
 /usr/bin/tar: ../../../../../var/lib/apt/lists/partial: Cannot open: Permission denied
2024-12-19T13:35:47.2842971Z
 /usr/bin/tar: Exiting with failure status due to previous errors
2024-12-19T13:35:47.2851756Z
 ##[warning]Failed to save: "/usr/bin/tar" failed with error: The process '/usr/bin/tar' failed with exit code 2

raised for exmaple in:
https://github.com/nipreps/sdcflows/actions/runs/12413644206

Copy link

codecov bot commented Dec 20, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.40%. Comparing base (a50506a) to head (04618e8).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #475      +/-   ##
==========================================
- Coverage   83.86%   81.40%   -2.46%     
==========================================
  Files          30       30              
  Lines        2807     2807              
  Branches      360      360              
==========================================
- Hits         2354     2285      -69     
- Misses        383      448      +65     
- Partials       70       74       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jhlegarreta jhlegarreta force-pushed the FixBuildTestPublishGHATarWarning branch 3 times, most recently from e3dfa48 to e6a245b Compare December 20, 2024 22:15
Fix caching-related warnings in GHA `build-test-publish` CI:
- Save the `apt-get` cache in a directory other than `/var/lib/apt` to
  avoid permission issues.
- Make the AFNI cache key name be specific to the CI matrix
  configuration to avoid clashes across cache names. Use the root part
  as the restore key name so that any cache found can be restored, as
  the AFNI version being installed is the same across configurations.
- Check if the AFNI cache exists before trying to install it in GitHub
  Actions `build-test-publish` CI workflow.

Fixes:
```
Failed to save: "/usr/bin/tar" failed with error: The process '/usr/bin/tar' failed with exit code 2
```

The full log showing
```
2024-12-19T13:35:45.2830631Z
 [command]/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/sdcflows/sdcflows --files-from manifest.txt --use-compress-program zstdmt
2024-12-19T13:35:46.5536670Z
 Failed to save: Unable to reserve cache with key afni-v1, another job may be creating this cache. More details: Cache already exists. Scope: refs/heads/master, Key: afni-v1, Version: d04022ae09f8f21b8c0f9f00e4a784b6e510fe6a47d30aa3b0853a42885b92cb
2024-12-19T13:35:46.5924639Z
 Post job cleanup.
2024-12-19T13:35:46.7348688Z
 [command]/usr/bin/tar --posix -cf cache.tzst --exclude cache.tzst -P -C /home/runner/work/sdcflows/sdcflows --files-from manifest.txt --use-compress-program zstdmt
2024-12-19T13:35:46.8326360Z
 /usr/bin/tar: ../../../../../var/lib/apt/lists/lock: Cannot open: Permission denied
2024-12-19T13:35:47.1807249Z
 /usr/bin/tar: ../../../../../var/lib/apt/lists/partial: Cannot open: Permission denied
2024-12-19T13:35:47.2842971Z
 /usr/bin/tar: Exiting with failure status due to previous errors
2024-12-19T13:35:47.2851756Z
 ##[warning]Failed to save: "/usr/bin/tar" failed with error: The process '/usr/bin/tar' failed with exit code 2
```

raised for exmaple in:
https://github.com/nipreps/sdcflows/actions/runs/12413644206
@jhlegarreta jhlegarreta force-pushed the FixBuildTestPublishGHATarWarning branch from e6a245b to 04618e8 Compare December 20, 2024 22:41
@jhlegarreta jhlegarreta changed the title ENH: Check if cache exists before trying to install AFNI in GHA CI ENH: Fix caching-related warnings in GHA build-test-publish CI Dec 20, 2024
@jhlegarreta
Copy link
Author

Warnings are gone:
https://github.com/nipreps/sdcflows/actions/runs/12439499846

@effigies The 3.9, latest, slow and 3.12, latest, veryslow seem to be failing often within the scope of this PR, e.g.
https://github.com/nipreps/sdcflows/actions/runs/12438935967

I tried the problem had been solved naming the AFNI cache keys with distinct names::
https://github.com/nipreps/sdcflows/actions/runs/12439231605

But it does not seem to be the case:
https://github.com/nipreps/sdcflows/actions/runs/12439499846/job/34733630867

Any clue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant