Treewide Nix reformat pass 1 [skip treewide]

[infinisil]

Note

Enforcement of formatting has been split into this separate PR: #326407

Description of changes

This is the next step towards accepted NixOS/rfcs#166 after #322512, see also #322520 and NixOS/nixfmt#153 🎉.

The @NixOS/nix-formatting will approve/merge this PR.

This PR formats all "inactive" files using nixfmt-rfc-style, where "inactive" means that there are no open PRs with activity in the last ~2 months that touch those files.

Doing it this way makes ensures that we don't cause any conflicts for recently active PRs that would be ready to merge. Only once those PRs get updated, CI will kick in and require the files to be formatted.

A bunch later, we can do another pass to get the rest.

Furthermore, this makes sure that we can merge this PR without having to constantly rebase it!

Verification

It's impossible to review this, but that's not necessary because you can review the code that generated it instead, which can be found here. At the same time this can be used to verify the contents of this PR by:

1. Checking out this PR in your Nixpkgs clone (with the GitHub CLI here):

   cd ~/src/nixpkgs
   gh co 322537
   

2. Building the verification script:

   nix-build https://github.com/infinisil/treewide-nixpkgs-reformat-script/archive/5439e16eda66856aed747db3b45ed595835de7c6.tar.gz \
     --argstr baseRev 5180ff0970d05347aa87492f64ed84a6e1e24b33
   

3. Running it on your clone (no worries if your tree is dirty, it uses a separate worktree and cleans up behind itself):

   result/bin/apply-formatting .
   

4. Verifying that the diff only contains the revision in .git-blame-ignore-revs (which changes because the date of the commit it creates changes)

Things done

• Merge and propagate the nixfmt fixes in Use --verify in Nixpkgs diff CI nixfmt#215
  • Merge the fixes
  • Update nixfmt in Nixpkgs: nixfmt-rfc-style: unstable-2024-07-03 -> unstable-2024-07-12 #326622
  • Update the pinned Nixpkgs for CI/shell.nix: ci: update tooling #328186
• Confirm that others are getting the same result with the script
• Confirm that there are (almost) no rebuilds (See [Tracking issue] remove all src = ./. entries that unintentionally include their own .nix files #301014)
  • Can be done separately, non-blocking in Prevent some rebuilds for future Nix reformats #326430
• Confirm that nix-instantiate --parse doesn't change
  • Discovered some (non-blocking) inconsistencies with nix-instantiate --parse from this:
    • parser: Remove empty multiline string parts earlier nix#11120
    • Fix non-deterministic inherit printing nix#11123
• Consider fixing this problem first
  • Maintain expanded attrs/lists/attr params/inherits nixfmt#224
• Make a Discourse post
• Prepare the same PR for release and staging branches, so that they can be merged at the same time. This ensures that there aren't any problems when backporting PRs or merging staging
  • [backport release-24.05] Treewide Nix reformat pass 1 [skip treewide] #327796

---

This work is sponsored by Antithesis ✨

Add a 👍 reaction to pull requests you find important.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/formatting-team-meeting-2024-06-25/47661/1

[oxalica]

I'd say we should block on NixOS/nixfmt#153. There are also many issues about either awkward formatting result or implementation correctness. It's no good to race reformatting everything before that.

This was done with this script in ~40 seconds

This sounds too long to me. If the whole-repo checking takes too long, it blocks auto-checking in git commit hook.
fd -e nix --exec-batch nixfmt takes 1:49.27 (~110s) on my machine and nixfmt is single-threaded. nixfmt --check on formatted files takes almost the same time.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-57/47823/1

[infinisil]

@oxalica The reason I'm pushing for this is because there's a lot of churn from people reformatting individual parts on their own, @mweinelt can attest to that. Let's not let perfect be the enemy of good, the formatter works and is unlikely to change drastically. We should definitely block on correctness bugs like NixOS/nixfmt#197 though, this is being worked on :)

Regarding the time it takes for a full format, the solution I'd propose is to use a pre-commit hook that only formats files that were changed. This also nicely aligns with CI only requiring changed files to be formatted. I'd be okay with blocking on that for this PR, I'll add it to the TODO items.

In addition to that, if CI fails, it shows the nixfmt command needed to make CI pass, only listing the files that need reformatting, see here.

Alternative: treefmt

One could also imagine using treefmt because it caches results, but this will be slightly problematic when we change the formatter in the future and need to do the same multi-pass approach to avoid merge conflicts. There's going to be times when not the entire codebase is formatted in the same way, and treefmt would mess that up.

[oxalica]

use a pre-commit hook that only formats files that were changed

That's kinda non-trivial to me, and you need to retrieve file list from git log on an unknown number of commits. Would be good to have an example (maybe in README) for copy-paste.

Also I opened NixOS/nixfmt#211 for tracking this.

[infinisil]

That's kinda non-trivial to me, and you need to retrieve file list from git log on an unknown number of commits. Would be good to have an example (maybe in README) for copy-paste.

Since it's the pre-commit hook, we only have to worry about the single commit that is being made, so I don't think it will be bad, but I'll see it when I try 😆. And we can use a shellHook automatically pre-commit hooks upon entering the shell, e.g. https://github.com/cachix/git-hooks.nix also works in that way.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/brainstorming-for-rfc-assimilate-home-manager-into-nixpkgs-monorepo/48230/13

[Aleksanaa]

I'd say we should block on NixOS/nixfmt#153. There are also many issues about either awkward formatting result or implementation correctness. It's no good to race reformatting everything before that.

The good thing here is we can skip formatting if something weird really happens, because it doesn't make subsequent CI check fail.

[emilazy]

I’d like to recommend a pre-push hook instead of a pre-commit one. It would still check all the commits that are to be pushed. The advantage is that it is easier to slice and dice existing commits and make imperfect work‐in‐progress commits to clean up later without having the formatting step get in the way. It also prevents making committing any slower, whereas pushes are already slow. For me, the experience of using a pre-commit hook is like having to satisfy a pedantic spell‐checker while trying to get into a flow state writing a stream‐of‐consciousness draft.

Another advantage is that tools like git-revise cannot necessarily run hooks due to their in‐memory design, which wouldn’t affect a pre-push hook; it’s more reliable to do the checks at the actual publication bottleneck.

If you need any help writing an efficient and correct Git hook for this, please ping me.

[infinisil]

To make this PR easier to review, I split off the enforcement for new files into #326407. So this PR is only for the treewide format now! Also note:

• I added instructions to the PR description on how the entire contents of this PR can be reproduced, allowing anybody to easily check it
• Some nixfmt fixes should be merged and propagated before this is merged, see the PR description

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/enforcing-nix-formatting-in-nixpkgs/49506/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/enforcing-nix-formatting-in-nixpkgs/49506/8

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/formatting-team-meeting-2024-08-06/50222/1

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/formatting-team-meeting-2024-09-03/51584/1