Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3.pkgs.waitress-django/cudaPackages.saxpy/doc/test-runner: Use lib.fileset #300428

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

dasJ
Copy link
Member

@dasJ dasJ commented Mar 31, 2024

Description of changes

This causes default.nix to be skipped when copying to the store to prevent rebuilds when reformatting the files (or changing them in any way that should not cause rebuilds).

Should get rid of the rebuild in #299578.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

This prevents unnecessary rebuilds when the default.nix is changed in
any way.
This prevents unnecessary rebuilds when the default.nix is changed in
any way.
@dasJ dasJ changed the title django-waitress/cudaPackages.saxpy: Use lib.fileset python3.pkgs.waitress-django/cudaPackages.saxpy: Use lib.fileset Mar 31, 2024
@github-actions github-actions bot added 6.topic: nixos 8.has: documentation 6.topic: testing Tooling for automated testing of packages and modules 6.topic: agda labels Mar 31, 2024
@dasJ dasJ changed the title python3.pkgs.waitress-django/cudaPackages.saxpy: Use lib.fileset python3.pkgs.waitress-django/cudaPackages.saxpy/doc/test-runner: Use lib.fileset Mar 31, 2024
@infinisil
Copy link
Member

Nice! Though I'd recommend explicitly selecting the files to be included using fileset = unions [ ... ]. This way, if you add new files, they won't change the result unless you include them too.

@tfc
Copy link
Contributor

tfc commented Mar 31, 2024

Please regard this as just a nitpick: Many folders here are under your control, so it's possible refactor the folder structure to make filtering unnecessary.

@ConnorBaker
Copy link
Contributor

I don't see anything unusual going on, looks good to me.

cudaPackages.saxpy don't work on Jetson, and CUDA stuff don't work on Darwin.


aarch64-darwin

nixpkgs-review pr 300428 --checkout commit --no-shell

Result of nixpkgs-review pr 300428 run on aarch64-darwin 1

25 packages marked as broken and skipped:
  • cudaPackages.saxpy
  • cudaPackagesGoogle.saxpy
  • cudaPackages_10.saxpy
  • cudaPackages_11.saxpy
  • tests.cuda.cudaPackages.saxpy
  • tests.cuda.cudaPackagesGoogle.saxpy
  • tests.cuda.cudaPackages_10.saxpy
  • tests.cuda.cudaPackages_10_0.saxpy
  • tests.cuda.cudaPackages_10_1.saxpy
  • tests.cuda.cudaPackages_10_2.saxpy
  • tests.cuda.cudaPackages_11.saxpy
  • tests.cuda.cudaPackages_11_0.saxpy
  • tests.cuda.cudaPackages_11_1.saxpy
  • tests.cuda.cudaPackages_11_2.saxpy
  • tests.cuda.cudaPackages_11_3.saxpy
  • tests.cuda.cudaPackages_11_4.saxpy
  • tests.cuda.cudaPackages_11_5.saxpy
  • tests.cuda.cudaPackages_11_6.saxpy
  • tests.cuda.cudaPackages_11_7.saxpy
  • tests.cuda.cudaPackages_11_8.saxpy
  • tests.cuda.cudaPackages_12.saxpy
  • tests.cuda.cudaPackages_12_0.saxpy
  • tests.cuda.cudaPackages_12_1.saxpy
  • tests.cuda.cudaPackages_12_2.saxpy
  • tests.cuda.cudaPackages_12_3.saxpy
4 packages built:
  • python311Packages.waitress-django
  • python311Packages.waitress-django.dist
  • python312Packages.waitress-django
  • python312Packages.waitress-django.dist

aarch64-linux with CUDA (non-Jetson)

nixpkgs-review pr 300428 --checkout commit --no-shell --extra-nixpkgs-config '{ allowUnfree = true; cudaSupport = true; cudaCapabilities = [ "7.5" ]; }'

Result of nixpkgs-review pr 300428 --extra-nixpkgs-config '{ allowUnfree = true; cudaSupport = true; cudaCapabilities = [ "7.5" ]; }' run on aarch64-linux 1

9 packages marked as broken and skipped:
  • cudaPackages_10.saxpy
  • tests.cuda.cudaPackages_10.saxpy
  • tests.cuda.cudaPackages_10_0.saxpy
  • tests.cuda.cudaPackages_10_1.saxpy
  • tests.cuda.cudaPackages_10_2.saxpy
  • tests.cuda.cudaPackages_11_0.saxpy
  • tests.cuda.cudaPackages_11_1.saxpy
  • tests.cuda.cudaPackages_11_2.saxpy
  • tests.cuda.cudaPackages_11_3.saxpy
1 package blacklisted:
  • nixos-install-tools
13 packages built:
  • cudaPackages.saxpy (tests.cuda.cudaPackages.saxpy ,tests.cuda.cudaPackages_12.saxpy ,tests.cuda.cudaPackages_12_2.saxpy)
  • cudaPackages_11.saxpy (cudaPackagesGoogle.saxpy ,tests.cuda.cudaPackagesGoogle.saxpy ,tests.cuda.cudaPackages_11.saxpy ,tests.cuda.cudaPackages_11_8.saxpy)
  • python311Packages.waitress-django
  • python311Packages.waitress-django.dist
  • python312Packages.waitress-django
  • python312Packages.waitress-django.dist
  • tests.cuda.cudaPackages_11_4.saxpy
  • tests.cuda.cudaPackages_11_5.saxpy
  • tests.cuda.cudaPackages_11_6.saxpy
  • tests.cuda.cudaPackages_11_7.saxpy
  • tests.cuda.cudaPackages_12_0.saxpy
  • tests.cuda.cudaPackages_12_1.saxpy
  • tests.cuda.cudaPackages_12_3.saxpy

aarch64-linux with CUDA (Jetson)

nixpkgs-review pr 300428 --checkout commit --no-shell --extra-nixpkgs-config '{ allowUnfree = true; cudaSupport = true; cudaCapabilities = [ "7.2" ]; }'

Result of nixpkgs-review pr 300428 --extra-nixpkgs-config '{ allowUnfree = true; cudaSupport = true; cudaCapabilities = [ "7.2" ]; }' run on aarch64-linux 1

25 packages marked as broken and skipped:
  • cudaPackages.saxpy
  • cudaPackagesGoogle.saxpy
  • cudaPackages_10.saxpy
  • cudaPackages_11.saxpy
  • tests.cuda.cudaPackages.saxpy
  • tests.cuda.cudaPackagesGoogle.saxpy
  • tests.cuda.cudaPackages_10.saxpy
  • tests.cuda.cudaPackages_10_0.saxpy
  • tests.cuda.cudaPackages_10_1.saxpy
  • tests.cuda.cudaPackages_10_2.saxpy
  • tests.cuda.cudaPackages_11.saxpy
  • tests.cuda.cudaPackages_11_0.saxpy
  • tests.cuda.cudaPackages_11_1.saxpy
  • tests.cuda.cudaPackages_11_2.saxpy
  • tests.cuda.cudaPackages_11_3.saxpy
  • tests.cuda.cudaPackages_11_4.saxpy
  • tests.cuda.cudaPackages_11_5.saxpy
  • tests.cuda.cudaPackages_11_6.saxpy
  • tests.cuda.cudaPackages_11_7.saxpy
  • tests.cuda.cudaPackages_11_8.saxpy
  • tests.cuda.cudaPackages_12.saxpy
  • tests.cuda.cudaPackages_12_0.saxpy
  • tests.cuda.cudaPackages_12_1.saxpy
  • tests.cuda.cudaPackages_12_2.saxpy
  • tests.cuda.cudaPackages_12_3.saxpy
1 package blacklisted:
  • nixos-install-tools
4 packages built:
  • python311Packages.waitress-django
  • python311Packages.waitress-django.dist
  • python312Packages.waitress-django
  • python312Packages.waitress-django.dist

aarch64-linux without CUDA

nixpkgs-review pr 300428 --checkout commit --no-shell

Result of nixpkgs-review pr 300428 run on aarch64-linux 1

9 packages marked as broken and skipped:
  • cudaPackages_10.saxpy
  • tests.cuda.cudaPackages_10.saxpy
  • tests.cuda.cudaPackages_10_0.saxpy
  • tests.cuda.cudaPackages_10_1.saxpy
  • tests.cuda.cudaPackages_10_2.saxpy
  • tests.cuda.cudaPackages_11_0.saxpy
  • tests.cuda.cudaPackages_11_1.saxpy
  • tests.cuda.cudaPackages_11_2.saxpy
  • tests.cuda.cudaPackages_11_3.saxpy
1 package blacklisted:
  • nixos-install-tools
13 packages built:
  • cudaPackages.saxpy (tests.cuda.cudaPackages.saxpy ,tests.cuda.cudaPackages_12.saxpy ,tests.cuda.cudaPackages_12_2.saxpy)
  • cudaPackages_11.saxpy (cudaPackagesGoogle.saxpy ,tests.cuda.cudaPackagesGoogle.saxpy ,tests.cuda.cudaPackages_11.saxpy ,tests.cuda.cudaPackages_11_8.saxpy)
  • python311Packages.waitress-django
  • python311Packages.waitress-django.dist
  • python312Packages.waitress-django
  • python312Packages.waitress-django.dist
  • tests.cuda.cudaPackages_11_4.saxpy
  • tests.cuda.cudaPackages_11_5.saxpy
  • tests.cuda.cudaPackages_11_6.saxpy
  • tests.cuda.cudaPackages_11_7.saxpy
  • tests.cuda.cudaPackages_12_0.saxpy
  • tests.cuda.cudaPackages_12_1.saxpy
  • tests.cuda.cudaPackages_12_3.saxpy

x86_64-darwin

nixpkgs-review pr 300428 --checkout commit --no-shell

Result of nixpkgs-review pr 300428 run on x86_64-darwin 1

25 packages marked as broken and skipped:
  • cudaPackages.saxpy
  • cudaPackagesGoogle.saxpy
  • cudaPackages_10.saxpy
  • cudaPackages_11.saxpy
  • tests.cuda.cudaPackages.saxpy
  • tests.cuda.cudaPackagesGoogle.saxpy
  • tests.cuda.cudaPackages_10.saxpy
  • tests.cuda.cudaPackages_10_0.saxpy
  • tests.cuda.cudaPackages_10_1.saxpy
  • tests.cuda.cudaPackages_10_2.saxpy
  • tests.cuda.cudaPackages_11.saxpy
  • tests.cuda.cudaPackages_11_0.saxpy
  • tests.cuda.cudaPackages_11_1.saxpy
  • tests.cuda.cudaPackages_11_2.saxpy
  • tests.cuda.cudaPackages_11_3.saxpy
  • tests.cuda.cudaPackages_11_4.saxpy
  • tests.cuda.cudaPackages_11_5.saxpy
  • tests.cuda.cudaPackages_11_6.saxpy
  • tests.cuda.cudaPackages_11_7.saxpy
  • tests.cuda.cudaPackages_11_8.saxpy
  • tests.cuda.cudaPackages_12.saxpy
  • tests.cuda.cudaPackages_12_0.saxpy
  • tests.cuda.cudaPackages_12_1.saxpy
  • tests.cuda.cudaPackages_12_2.saxpy
  • tests.cuda.cudaPackages_12_3.saxpy
4 packages built:
  • python311Packages.waitress-django
  • python311Packages.waitress-django.dist
  • python312Packages.waitress-django
  • python312Packages.waitress-django.dist

x86_64-linux with CUDA

nixpkgs-review pr 300428 --checkout commit --no-shell --extra-nixpkgs-config '{ allowUnfree = true; cudaSupport = true; cudaCapabilities = [ "7.5" ]; }'

Result of nixpkgs-review pr 300428 --extra-nixpkgs-config '{ allowUnfree = true; cudaSupport = true; cudaCapabilities = [ "7.5" ]; }' run on x86_64-linux 1

2 packages blacklisted:
  • nixos-install-tools
  • tests.nixos-functions.nixos-test
23 packages built:
  • cudaPackages.saxpy (tests.cuda.cudaPackages.saxpy ,tests.cuda.cudaPackages_12.saxpy ,tests.cuda.cudaPackages_12_2.saxpy)
  • cudaPackages_11.saxpy (cudaPackagesGoogle.saxpy ,tests.cuda.cudaPackagesGoogle.saxpy ,tests.cuda.cudaPackages_11.saxpy ,tests.cuda.cudaPackages_11_8.saxpy)
  • cudaPackages_10.saxpy (tests.cuda.cudaPackages_10.saxpy ,tests.cuda.cudaPackages_10_2.saxpy)
  • python311Packages.waitress-django
  • python311Packages.waitress-django.dist
  • python312Packages.waitress-django
  • python312Packages.waitress-django.dist
  • tests.cuda.cudaPackages_10_0.saxpy
  • tests.cuda.cudaPackages_10_1.saxpy
  • tests.cuda.cudaPackages_11_0.saxpy
  • tests.cuda.cudaPackages_11_1.saxpy
  • tests.cuda.cudaPackages_11_2.saxpy
  • tests.cuda.cudaPackages_11_3.saxpy
  • tests.cuda.cudaPackages_11_4.saxpy
  • tests.cuda.cudaPackages_11_5.saxpy
  • tests.cuda.cudaPackages_11_6.saxpy
  • tests.cuda.cudaPackages_11_7.saxpy
  • tests.cuda.cudaPackages_12_0.saxpy
  • tests.cuda.cudaPackages_12_1.saxpy
  • tests.cuda.cudaPackages_12_3.saxpy
  • tests.testers.nixosTest-example
  • tests.testers.runNixOSTest-example
  • tests.trivial-builders.references

x86_64-linux without CUDA

nixpkgs-review pr 300428 --checkout commit --no-shell

Result of nixpkgs-review pr 300428 run on x86_64-linux 1

2 packages blacklisted:
  • nixos-install-tools
  • tests.nixos-functions.nixos-test
23 packages built:
  • cudaPackages.saxpy (tests.cuda.cudaPackages.saxpy ,tests.cuda.cudaPackages_12.saxpy ,tests.cuda.cudaPackages_12_2.saxpy)
  • cudaPackages_11.saxpy (cudaPackagesGoogle.saxpy ,tests.cuda.cudaPackagesGoogle.saxpy ,tests.cuda.cudaPackages_11.saxpy ,tests.cuda.cudaPackages_11_8.saxpy)
  • cudaPackages_10.saxpy (tests.cuda.cudaPackages_10.saxpy ,tests.cuda.cudaPackages_10_2.saxpy)
  • python311Packages.waitress-django
  • python311Packages.waitress-django.dist
  • python312Packages.waitress-django
  • python312Packages.waitress-django.dist
  • tests.cuda.cudaPackages_10_0.saxpy
  • tests.cuda.cudaPackages_10_1.saxpy
  • tests.cuda.cudaPackages_11_0.saxpy
  • tests.cuda.cudaPackages_11_1.saxpy
  • tests.cuda.cudaPackages_11_2.saxpy
  • tests.cuda.cudaPackages_11_3.saxpy
  • tests.cuda.cudaPackages_11_4.saxpy
  • tests.cuda.cudaPackages_11_5.saxpy
  • tests.cuda.cudaPackages_11_6.saxpy
  • tests.cuda.cudaPackages_11_7.saxpy
  • tests.cuda.cudaPackages_12_0.saxpy
  • tests.cuda.cudaPackages_12_1.saxpy
  • tests.cuda.cudaPackages_12_3.saxpy
  • tests.testers.nixosTest-example
  • tests.testers.runNixOSTest-example
  • tests.trivial-builders.references

src = ./.;
src = pkgs.lib.fileset.toSource {
root = ./.;
fileset = pkgs.lib.fileset.difference ./. (pkgs.lib.fileset.unions [ ./common.nix ./default.nix ./shell.nix ]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side notes:

  • I frequently find myself repeating some kind of a fileFilter to discard the nix files. Makes me wonder if we want a shorthand for this...
  • Can unnest e.g. via lib.pipe

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned earlier, I'd really recommend explicitly selecting the files to include:

Suggested change
fileset = pkgs.lib.fileset.difference ./. (pkgs.lib.fileset.unions [ ./common.nix ./default.nix ./shell.nix ]);
fileset = pkgs.lib.fileset.unions [ ... ];

That's going to be much more resilient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants