fix: don't hardlink the /package source directory in npm_package_store #1533
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1412.
I'm not entirely sure on the reason that this resolves #1412 but I have some loose theories listed below and I've tested it against a consistent and fast repro of the issue in https://github.com/bazelbuild/examples and found that the two consistent fixes are:
--modify_execution_info=CopyDirectory=+no-remote
on the build/package
source directory in thecopy_directory_bin_action
insidenpm_package_store
(this PR)Somehow a cache hit for an npm_package_store that has hardlinked files within a tree artifact to the
/package
directory of the external repository leads to the failure:Is bazel is doing something unusual when writing or restoring cached tree artifacts containing hardlinks to a folder in an external repository? Another possibility is some timing issue where the repo rule is being re-run so the
/package
directory indeed does not exist momentarily and somehow that interacts badly with the hardlinks in the tree artifact that is cached.NB: hardlinks to source file in external repositories mean that Bazel will inadvertently set permissions of those source files in the external repository to read-only after the after the copy action completes, which is long after the repository rule executes. This happens because Bazel will set the files in the output tree to read-only (as it does for all files in the output tree) but, because hardlinks share an underlying file ACL, doing so will also set the files in the external repository to read-only. The changes to the ACL to files contained within a source directory that is an input to a cached action may be related to the flake. NB: the flake in #1412 was reported as being more common in Bazel 7+.
In any case, the hardlinks here were a minor optimization for npm packages with large files so removing them to fix this flake is worth the minor performance penalty on the small subset of npm packages with large files.
Type of change
Test plan
The issue does not repro in rules_js. The https://github.com/bazelbuild/examples frontend example was used to confirm this fix in this DNL PR: bazelbuild/examples#429.