Skip to content

Commit

Permalink
Migrate to hs-nix-infra for the Nix setup (#30)
Browse files Browse the repository at this point in the history
This PR does the following:

## Unify the GHC derivation used across Kadena projects

Instead of depending on `nixpkgs` and `haskellNix` directly, this flake now depends on our new `hs-nix-infra` flake and uses the `nixpkgs` and `haskellNix` revisions provided by it. The hash of the `nixpkgs` and `haskellNix` flakes used for defining the `haskell.nix` `project` determines the hash of the GHC package that gets used to compile the Haskell modules.  

When multiple projects depend on `nixpkgs` and `haskellNix` independently, it's very hard (and not really well supported by nix CLI) to make sure that they don't deviate from each others' `nixpkgs` and `haskellNix` pins arbitrarily. I.e. updating two projects' `flake.lock` files at slightly different times is likely to cause one of the pins to be on a different revision, even though the difference doesn't matter functionally.

These unnecessarily different GHC packages put a lot of pressure on our CI infrastructure, taking hours to build functionally equivalent GHC packages and bloating the cache (with binaries from all the architectures we build and cache for). That also bloats the `/nix/store` of any `chainweb-data` user that wants to `nix build` an uncached `chainweb-data` version.

### The new workflow for updating Haskell-Nix toolchain

After this PR, the new workflow for managing our Haskell dependencies used by Nix will involve the following steps:
* If an update to the toolchain is needed in order to fix the build, the first thing to try is to bump the `hs-nix-infra` dependency of this flake to the latest version. If that's not enough, we need to open a PR to `hs-nix-infra` to bump its proper input:
    * If we need to update our hackage pin so that we can build with newly released Haskell packages, we can just `nix flake lock --update-input hackage` and get a newer hackage snapshot without needing to introduce a new GHC derivation.
    * If we need to use a new GHC version provided by a newer `nixpkgs` version or if we need to bump our `haskellNix` pin for any reason we need to bump `nixpkgs` and `haskellNix`. The PR would preferably bump both of them to the latest version.

Hopefully, this new workflow will reduce the number of `nixpkgs` + `haskellNix` versions we depend on across our Haskell projects.

## Add a `recursive` alternative to the `default` package

As part of the CI automation for this repo, we're building and caching the Nix binaries for `chainweb-data`, which makes it convenient for any user to `nix build` chainweb-data from any commit/branch since all the dependencies will come from our binary cache. However, even without building anything locally, *evaluating* the `default` package of this flake takes a significant amount of time and involves downloading ~2 GB of Nix dependencies. This is due to the complexity of what `haskellNix` does for us at Nix evaluation time.

This PR introduces a `recursive` package to this flake's output, which uses `recursive-nix` to push the Nix evaluation of the `default` package into the build of a derivation. This means, any user that tries to `nix build .#recursive` will fetch the chainweb-data binary from our cache without having to perform any complex Nix evaluation locally or downloading the Nix dependencies of any such evaluation as long as the `recursive` derivation they're building is already in our binary cache. If not, the `recursive-nix` derivation will be built locally (in which case make sure your local Nix setup has `recursive-nix` enabled), which is essentially as much work as building `default` itself. This might still be worthwhile however, since subsequent builds of the same `recursive` derivation will complete immediately, without having to evaluate the `default` derivation again.


* Migrate to hs-nix-infra

* Add a recursive output

* Add the recursive output to nix.yml

* Avoid merging the output with haskellNix flake

Instead, expose its project object directly

* Add a recursive version of bundled as well

* Add bundled.recursive to nix.yml
  • Loading branch information
enobayram authored Nov 17, 2023
1 parent 7f2868b commit db5b3d4
Show file tree
Hide file tree
Showing 3 changed files with 179 additions and 392 deletions.
9 changes: 7 additions & 2 deletions .github/workflows/nix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,11 @@ jobs:
uses: actions/checkout@v3

- name: Set up Nix with caching
uses: kadena-io/setup-nix-with-cache/by-root@v3
uses: kadena-io/setup-nix-with-cache/by-root@v3.1
with:
cache_url: s3://nixcache.chainweb.com?region=us-east-1
signing_private_key: ${{ secrets.NIX_CACHE_PRIVATE_KEY }}
additional_experimental_features: recursive-nix

- name: Set up AWS credentials
uses: aws-actions/configure-aws-credentials@v2
Expand All @@ -34,9 +35,13 @@ jobs:
aws-region: us-east-1

- name: Give root user AWS credentials
uses: kadena-io/setup-nix-with-cache/copy-root-aws-credentials@v3
uses: kadena-io/setup-nix-with-cache/copy-root-aws-credentials@v3.1

- name: Build and cache artifacts
run: |
echo Building the project
nix build .#check --log-lines 500 --show-trace
echo Build the recursive outputs
nix build .#recursive.allDerivations --log-lines 500 --show-trace
nix build .#bundled.recursive.allDerivations --log-lines 500 --show-trace
Loading

0 comments on commit db5b3d4

Please sign in to comment.